CN106840112A - A kind of space geometry measuring method of utilization free space eye gaze point measurement - Google Patents
A kind of space geometry measuring method of utilization free space eye gaze point measurement Download PDFInfo
- Publication number
- CN106840112A CN106840112A CN201710013278.6A CN201710013278A CN106840112A CN 106840112 A CN106840112 A CN 106840112A CN 201710013278 A CN201710013278 A CN 201710013278A CN 106840112 A CN106840112 A CN 106840112A
- Authority
- CN
- China
- Prior art keywords
- space
- eye
- camera
- pupil
- point
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C11/00—Photogrammetry or videogrammetry, e.g. stereogrammetry; Photographic surveying
- G01C11/04—Interpretation of pictures
- G01C11/06—Interpretation of pictures by comparison of two or more pictures of the same area
- G01C11/08—Interpretation of pictures by comparison of two or more pictures of the same area the pictures not being supported in the same relative position as when they were taken
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
- G01C21/005—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 with correlation of navigation data from several sources, e.g. map or contour matching
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S19/00—Satellite radio beacon positioning systems; Determining position, velocity or attitude using signals transmitted by such systems
- G01S19/38—Determining a navigation solution using signals transmitted by a satellite radio beacon positioning system
- G01S19/39—Determining a navigation solution using signals transmitted by a satellite radio beacon positioning system the satellite radio beacon positioning system transmitting time-stamped messages, e.g. GPS [Global Positioning System], GLONASS [Global Orbiting Navigation Satellite System] or GALILEO
- G01S19/42—Determining position
- G01S19/45—Determining position by combining measurements of signals from the satellite radio beacon positioning system with a supplementary measurement
Abstract
The present invention relates to a kind of space geometry measuring method of utilization free space eye gaze point measurement：The image of testee is obtained using scene camera；For each group of eye tracking binocular camera, the space coordinates of pupil center is determined；The normal direction of plane where the pupil of pupil center was defined for direction of visual lines, for left eye and right eye, the direction of visual lines expression formula for determining respectively；Obtain the coordinate of fixation point；According to fixation point in the constant characteristic of the coordinate of detected space earth coordinates, changed by the real-time pose of pose measurement component, obtain the geodetic coordinates of constant fixation point；After carrying out repeatedly staring operation to testee, if obtaining the space coordinates done of body surface, the distance between each point is calculated respectively, and combine the image of scene camera, complete the space geometry measurement to the measured object.
Description
Technical field
The present invention relates to eye tracking and space geometry measurement field.
Background technology
Visual Trace Technology has important use value in fields such as man-machine interaction, virtual realities, can greatly expand
Open up the work capacity of user.Contactless Visual Trace Technology based on machine vision includes two parts content：(1) sight line is special
Parameter extraction is levied, i.e. the position of eyes in the picture, generally characterized with pupil or iris center；(2) direction of visual lines modeling, i.e., by
The eye feature parameter tracking measurement direction of visual lines for detecting.Visual line characteristics parameter extraction is by image procossing from eye image
Extract pupil, iris and corneal reflection spot etc. and characterize the dynamic parameter of eye.Wherein, pupil tracking is the weight of visual line characteristics parameter detecting
Want step.Direction of visual lines trace model is set up, is mainly had based on two kinds of forms of two and three dimensions, wherein three-dimensional regards including 3D again
The direct tracking of line and the sight tracing based on 3D models.
Current wear-type sight tracking device both domestic and external mainly uses pupil-corneal reflection vector method.By fixed eye
Video camera obtains eyeball image, using bright pupil hole and the principle of dark pupil, extracts the pupil in eyeball image, anti-using cornea
The relative position of method correction eye camera and eyeball is penetrated, using corneal reflection point data as eye camera and the relative position of eyeball
Basic point, pupil center location coordinate means that the position of sight line.Such device can obtain eye tracking essence higher
Degree, but human eye is constrained in fixed space, it is difficult to accomplish the eye tracking of free space.
The content of the invention
It is an object of the invention to provide one kind improve Visual Trace Technology, there is provided one kind can realize free space sight line with
The method of track, and the space geometry measuring method using the measurement of free space eye gaze point is given on this basis.This hair
The free space sight tracing of bright utilization binocular stereo vision, solves with two pairs of real-time acquisition binocular images of video camera and in real time
Calculate eyes direction of visual lines and sight line to cross point coordinates, with reference to attitude transducer and GPS device, i.e., obtain eyes sight line in real time big
Space equation in ground coordinate system, resolves the space coordinates of fixation point, the comprehensive field on the basis of the calculating of fixation point space coordinates
The space geometry measurement of object is stared in the image of scape camera, completion.Technical scheme is as follows：
A kind of space geometry measuring method of utilization free space eye gaze point measurement, the measuring system bag for being used
Include the spectacle framework with left eye framework and right eye framework, two semi-transparent semi-reflecting lens, binoculars being fixed on spectacle framework
Stereoscopic vision eye tracking camera, at least one scene camera and pose measurement component, wherein, binocular stereo vision sight line
Tracking camera includes two groups of eye tracking binocular cameras, and two semi-transparent semi-reflecting lens are separately fixed at left eye framework and right eye frame
In frame, the eye tracking binocular camera by each semi-transparent semi-reflecting lens described in each setting group is used to by reflective surface to eyes
Imaging；Described scene camera is fixed on spectacle framework, for scene imaging；Described pose measurement component includes
The pose measurement sensor and GPS module of three-axis gyroscope and inclinator composition, pose measurement sensor coordinate GPS module real-time
Measurement user's head space pose；Space geometry measuring method is：
1) image of testee is obtained using scene camera；
2) for each group of eye tracking binocular camera, according to camera parameter, binocular stereo vision measurement model is set up；
Pupil edge is extracted in the eye image for being collected, by extracting the characteristic point of pupil edge, sky where pupil is determined
Between plane equation coefficient, obtain plane equation where pupil, calculate pupil center's point imager coordinate；Calculate the space of pupil center
Coordinate；
3) normal direction of plane where the pupil of pupil center was defined for direction of visual lines, for left eye and right eye, point
The direction of visual lines expression formula not determined；
4) coordinate of fixation point is obtained；
5) according to fixation point in the constant characteristic of the coordinate of detected space earth coordinates, by the reality of pose measurement component
Shi Weizi is changed, and obtains the geodetic coordinates of constant fixation point；
6) after carrying out repeatedly staring operation to testee, if obtaining the space coordinates that body surface is done, calculate respectively
The distance between each point, and the image of scene camera is combined, complete the space geometry measurement to the measured object.
In sum, certain is stared body surface in obtaining space using eye tracking measurement and head pose measurement
The space coordinates of point, and the space coordinates of coordinate system where all such fixation points is united, this can be obtained and stared
The physical dimension of body surface, completes the work of space geometry measurement.The method can realize measured object space geometry amount
Non-cpntact measurement, and need not be complicated the ambient As such as light source, possess more preferable versatility and property easy to use, while will be by
Survey object image to be effectively combined with space geometry size, create a kind of new spatial geometric measurement mode.
Brief description of the drawings
Fig. 1 system structure diagrams
Fig. 2 spaces fixation point measuring method schematic diagram
Fig. 3 measuring system preferred forms schematic diagrames
Description of reference numerals is as follows：
1 spectacle framework；2 left eye frameworks；3 right eye frameworks；4 semi-transparent semi-reflecting lens；5 scene cameras；
6 eye tracking binocular cameras；The pose measurement sensor of 7GPS modules 8.
Specific embodiment
Fig. 1 is the measuring system structural representation that the present invention is used, and whole measuring system includes carrying left eye framework and the right side
The spectacle framework of eye framework, two semi-transparent semi-reflecting lens being fixed on spectacle framework, binocular stereo vision eye tracking are taken the photograph
Camera, at least one scene camera and pose measurement component, wherein, binocular stereo vision eye tracking camera includes two groups
Eye tracking binocular camera, two semi-transparent semi-reflecting lens are separately fixed in left eye framework and right eye framework, semi-transparent partly at each
Eye tracking binocular camera described in each setting group in anti-mirror side, is used to by reflective surface to eye imaging；Described scene is taken the photograph
Camera is fixed on spectacle framework, for scene imaging；Described pose measurement component includes three-axis gyroscope and inclination angle
The pose measurement sensor and GPS module of instrument composition, it is empty that pose measurement sensor coordinates GPS module to measure user's head in real time
Between pose.
Concrete methods of realizing is as follows：
1st, pupil edge is extracted in the eye image that binocular camera is collected, by the spy for extracting pupil edge
Levy point (Xli,Yli,Zli), i=0,1 ... n-1 determine space plane equation coefficient (A where pupil with these characteristic pointsl,Bl,
Cl,Dl), obtain plane equation where pupil.Pupil edge through being fitted can calculate round centre coordinate, and regard similar to Circles
It is pupil center point imager coordinate (ul0,vl0), the space coordinates of pupil center is calculated using binocular stereo vision measurement model
(Xl0,Yl0,Zl0), from space geometry, the normal vector of plane is where pupilThe pupil of Ze Guo pupil center
The normal direction expression formula of plane is as follows where hole：
Above formula is made up of regarding for the normal direction expression formula of the central point of disk, i.e. measuring system definition pupil edge
Line direction.The direction of visual lines expression formula that right eye can equally be obtained is：
By simultaneous or so an eye line equation (1) and (2), you can obtain the coordinate (X of fixation pointn,Yn,Zn)。
2nd, eye gaze space of points coordinate measuring method such as Fig. 2.Coordinate under mobile working coordinate system where sight line, firm
Property connection (by wear-type device ensure) in the case of, when position appearance changes right overhead, binocular eye tracking stereoscopic vision survey
Relative position between amount system and head pose measurement component is constant, fixation point and binocular eye tracking stereo-visiuon measurement system
The relative position of system changes, but fixation point is constant in the coordinate of detected space earth coordinates, so while solidifying
Viewpoint the binocular eye tracking Stereo Vision Measurement System frame of reference changes in coordinates, but by head pose measurement group
The real-time pose conversion of part, can obtain the geodetic coordinates of constant fixation point.
Wherein (XW,YW,ZW) represent the geodetic coordinates of the fixation point before the conversion of head pose, (XW',YW',ZW') represent head
The geodetic coordinates of the fixation point after the appearance conversion of position, Δ RW,ΔTWThe pose conversion of head is represented, by head pose measurement component
Obtain pose compensated information.
The space geometry measuring method measured using free space eye gaze point can be by system reality as shown in Figure 3
It is existing：Above semi-transparent semi-reflecting lens, using two pairs of miniature CMOS cameras, by reflective surface to eye imaging；Under semi-transparent semi-reflecting lens
Side places a miniature CMOS scenes camera per one side, by reflective surface to scene imaging；Human eye by semi-transparent semi-reflecting lens and
Head-shield window observes scene.Said system layout is capable of achieving both observation of the without prejudice to human eye to scene, and covering scene video camera
Coverage, scene camera is reflected light into, while binocular camera can collect clearly eye image.The layout
Eye sight line direction can be accurately caught, eyes can be simultaneously monitored when measuring space fixation point is formed sight line space and cross product
Raw fixation point.Pose measurement sensor is made up of three-axis gyroscope and inclinator, coordinates double frequency differential GPS to measure user in real time
Head space pose.Head pose measurement component and gaze tracking system rigid attachment, constitute complete eye tracking and space
Geometric measurement system.Two pairs of eye tracking cameras track direction of visual lines, calculate fixation point and are sat relative to the space of human eye
Mark, the head pose data in conjunction with GPS and attitude transducer obtain space coordinates of the fixation point in earth coordinates.To quilt
After surveying object and carrying out repeatedly staring operation, if obtaining the space coordinates done of body surface, calculate respectively between each point away from
From, and with reference to the image of scene camera, complete the space geometry measurement to the measured object.
Claims (1)
1. a kind of space geometry measuring method that utilization free space eye gaze point is measured, the measuring system for being used includes
Spectacle framework with left eye framework and right eye framework, two semi-transparent semi-reflecting lens being fixed on spectacle framework, binocular stand
Body vision eye tracking camera, at least one scene camera and pose measurement component, wherein, binocular stereo vision sight line with
Track video camera includes two groups of eye tracking binocular cameras, and two semi-transparent semi-reflecting lens are separately fixed at left eye framework and right eye framework
It is interior, the eye tracking binocular camera by each semi-transparent semi-reflecting lens described in each setting group, be used to by reflective surface to eyes into
Picture；Described scene camera is fixed on spectacle framework, for scene imaging；Described pose measurement component includes three
The pose measurement sensor and GPS module of axle gyroscope and inclinator composition, pose measurement sensor coordinate GPS module to survey in real time
Amount user's head space pose；Space geometry measuring method is：
1) image of testee is obtained using scene camera；
2) for each group of eye tracking binocular camera, according to camera parameter, binocular stereo vision measurement model is set up；By its
Pupil edge is extracted in the eye image for collecting, by extracting the characteristic point of pupil edge, determines that space is put down where pupil
Face equation coefficient, obtains plane equation where pupil, calculates pupil center's point imager coordinate；The space for calculating pupil center is sat
Mark；
3) normal direction of plane where the pupil of pupil center was defined for direction of visual lines, it is true respectively for left eye and right eye
Fixed direction of visual lines expression formula；
4) coordinate of fixation point is obtained；
5) according to fixation point in the constant characteristic of the coordinate of detected space earth coordinates, by the real-time position of pose measurement component
Appearance is changed, and obtains the geodetic coordinates of constant fixation point；
6) after carrying out repeatedly staring operation to testee, if obtaining the space coordinates that body surface is done, each is calculated respectively
The distance between point, and the image of scene camera is combined, complete the space geometry measurement to the measured object.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710013278.6A CN106840112B (en) | 2017-01-09 | 2017-01-09 | A kind of space geometry measuring method measured using free space eye gaze point |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710013278.6A CN106840112B (en) | 2017-01-09 | 2017-01-09 | A kind of space geometry measuring method measured using free space eye gaze point |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106840112A true CN106840112A (en) | 2017-06-13 |
| CN106840112B CN106840112B (en) | 2019-03-22 |
Family
ID=59118109
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710013278.6A Expired - Fee Related CN106840112B (en) | 2017-01-09 | 2017-01-09 | A kind of space geometry measuring method measured using free space eye gaze point |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106840112B (en) |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107255446A (en) * | 2017-08-01 | 2017-10-17 | 南京农业大学 | A kind of Cold region apple fruit tree canopy three-dimensional map constructing system and method |
| CN107909061A (en) * | 2017-12-07 | 2018-04-13 | 电子科技大学 | A kind of head pose tracks of device and method based on incomplete feature |
| CN108646915A (en) * | 2018-05-03 | 2018-10-12 | 东南大学 | The method and system of object is captured in conjunction with three-dimensional eye tracking and brain-computer interface control machinery arm |
| WO2019015567A1 (en) * | 2017-07-21 | 2019-01-24 | 北京集思明智科技有限公司 | Brain function testing system and device thereof |
| CN110134222A (en) * | 2018-02-02 | 2019-08-16 | 上海集鹰科技有限公司 | A kind of VR shows positioning sighting system and its positioning method of sight |
| CN112394382A (en) * | 2020-10-14 | 2021-02-23 | 中国科学院西安光学精密机械研究所 | Long-time shielding resistant low-slow small target tracking device and method |
| CN112754420A (en) * | 2021-01-08 | 2021-05-07 | 瑞尔明康(杭州)视光科技有限公司 | Wavefront aberration measurement system, associated visual perception learning training system and method |
| CN109974667B (en) * | 2017-12-27 | 2021-07-23 | 宁波方太厨具有限公司 | Indoor human body positioning method |
| CN115601824A (en) * | 2022-10-19 | 2023-01-13 | 华中科技大学（Cn） | Device, system and method for marking human eye gaze direction in two-dimensional image |
| TWI800030B (en) * | 2020-12-22 | 2023-04-21 | 財團法人工業技術研究院 | Method and device for calculating the distance between eyes and the eyes’ target |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101901485A (en) * | 2010-08-11 | 2010-12-01 | 华中科技大学 | 3D free head moving type gaze tracking system |
| US20140111632A1 (en) * | 2012-10-23 | 2014-04-24 | Pixart Imaging Inc. | Pupil detection device |
| CN104089606A (en) * | 2014-06-30 | 2014-10-08 | 天津大学 | Free space eye tracking measurement method |
-
2017
- 2017-01-09 CN CN201710013278.6A patent/CN106840112B/en not_active Expired - Fee Related
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101901485A (en) * | 2010-08-11 | 2010-12-01 | 华中科技大学 | 3D free head moving type gaze tracking system |
| US20140111632A1 (en) * | 2012-10-23 | 2014-04-24 | Pixart Imaging Inc. | Pupil detection device |
| CN104089606A (en) * | 2014-06-30 | 2014-10-08 | 天津大学 | Free space eye tracking measurement method |
Cited By (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2019015567A1 (en) * | 2017-07-21 | 2019-01-24 | 北京集思明智科技有限公司 | Brain function testing system and device thereof |
| CN107255446B (en) * | 2017-08-01 | 2020-01-07 | 南京农业大学 | Dwarfing close-planting fruit tree canopy three-dimensional map construction system and method |
| CN107255446A (en) * | 2017-08-01 | 2017-10-17 | 南京农业大学 | A kind of Cold region apple fruit tree canopy three-dimensional map constructing system and method |
| CN107909061B (en) * | 2017-12-07 | 2021-03-30 | 电子科技大学 | Head posture tracking device and method based on incomplete features |
| CN107909061A (en) * | 2017-12-07 | 2018-04-13 | 电子科技大学 | A kind of head pose tracks of device and method based on incomplete feature |
| CN109974667B (en) * | 2017-12-27 | 2021-07-23 | 宁波方太厨具有限公司 | Indoor human body positioning method |
| CN110134222A (en) * | 2018-02-02 | 2019-08-16 | 上海集鹰科技有限公司 | A kind of VR shows positioning sighting system and its positioning method of sight |
| CN108646915B (en) * | 2018-05-03 | 2020-12-15 | 东南大学 | Method and system for controlling mechanical arm to grab object by combining three-dimensional sight tracking and brain-computer interface |
| CN108646915A (en) * | 2018-05-03 | 2018-10-12 | 东南大学 | The method and system of object is captured in conjunction with three-dimensional eye tracking and brain-computer interface control machinery arm |
| CN112394382A (en) * | 2020-10-14 | 2021-02-23 | 中国科学院西安光学精密机械研究所 | Long-time shielding resistant low-slow small target tracking device and method |
| CN112394382B (en) * | 2020-10-14 | 2023-12-08 | 中国科学院西安光学精密机械研究所 | Low-speed small target tracking device and method capable of resisting long-term shielding |
| TWI800030B (en) * | 2020-12-22 | 2023-04-21 | 財團法人工業技術研究院 | Method and device for calculating the distance between eyes and the eyes’ target |
| CN112754420A (en) * | 2021-01-08 | 2021-05-07 | 瑞尔明康(杭州)视光科技有限公司 | Wavefront aberration measurement system, associated visual perception learning training system and method |
| CN115601824A (en) * | 2022-10-19 | 2023-01-13 | 华中科技大学（Cn） | Device, system and method for marking human eye gaze direction in two-dimensional image |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106840112B (en) | 2019-03-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106840112B (en) | A kind of space geometry measuring method measured using free space eye gaze point | |
| CN104699247B (en) | A kind of virtual reality interactive system and method based on machine vision | |
| CN106168853B (en) | A kind of free space wear-type gaze tracking system | |
| CN110032278A (en) | A kind of method for recognizing position and attitude, the apparatus and system of human eye attention object | |
| US9779512B2 (en) | Automatic generation of virtual materials from real-world materials | |
| CN106643699A (en) | Space positioning device and positioning method in VR (virtual reality) system | |
| CN104036488B (en) | Binocular vision-based human body posture and action research method | |
| US10269139B2 (en) | Computer program, head-mounted display device, and calibration method | |
| CN106843456A (en) | A kind of display methods, device and virtual reality device followed the trail of based on attitude | |
| CN108846867A (en) | A kind of SLAM system based on more mesh panorama inertial navigations | |
| CN106131530A (en) | A kind of bore hole 3D virtual reality display system and methods of exhibiting thereof | |
| CN103543830B (en) | Method for mapping human skeleton points to virtual three-dimensional space points in three-dimensional display | |
| SE527257C2 (en) | Device and method for presenting an external image | |
| CN107230224B (en) | Three-dimensional virtual garment model production method and device | |
| JP2004233334A (en) | Method for measuring position and orientation | |
| CN105395252A (en) | Wearable three-dimensional image navigation device for vascular intervention operation and realizing man-machine interaction | |
| CN106214118A (en) | A kind of ocular movement based on virtual reality monitoring system | |
| CN106066701A (en) | A kind of AR and VR data handling equipment and method | |
| CN108154533A (en) | A kind of position and attitude determines method, apparatus and electronic equipment | |
| CN108124150B (en) | The method that virtual reality wears display equipment and observes real scene by it | |
| CA3085733A1 (en) | System and method of obtaining fit and fabrication measurements for eyeglasses using simultaneous localization and mapping | |
| CN104732586B (en) | A kind of dynamic body of 3 D human body and three-dimensional motion light stream fast reconstructing method | |
| CN107864372A (en) | Solid picture-taking method, apparatus and terminal | |
| JP6770208B2 (en) | Information processing device | |
| KR101525411B1 (en) | Image generating method for analysis of user's golf swing, analyzing method for user's golf swing using the same and apparatus for analyzing golf swing |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20190322 Termination date: 20220109 |
|
| CF01 | Termination of patent right due to non-payment of annual fee |