WO2011030212A1 - Method and apparatus for using generic software applications by means of ocular control and suitable methods of interaction - Google Patents
Method and apparatus for using generic software applications by means of ocular control and suitable methods of interaction Download PDFInfo
- Publication number
- WO2011030212A1 WO2011030212A1 PCT/IB2010/002271 IB2010002271W WO2011030212A1 WO 2011030212 A1 WO2011030212 A1 WO 2011030212A1 IB 2010002271 W IB2010002271 W IB 2010002271W WO 2011030212 A1 WO2011030212 A1 WO 2011030212A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- user
- interactors
- module
- coordinates
- data
- Prior art date
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
- G06F3/04842—Selection of displayed objects or displayed text elements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
- G06F3/013—Eye tracking input arrangements
Definitions
- the present invention refers to the field of systems and methods for the control of generic software applications by means of eye-tracking devices, i.e. by using ocular movements.
- the tracking of the ocular movements potentially offers the user the possibility of controlling the software of a personal computer by simply looking at the display thereof.
- eye-tracking systems are generally limited to software made specifically for disabled users in which a low pointing precision is required. Indeed, the user should make his/her selection from a list of relatively large cells in which the effect of the error of accuracy is small.
- eye-tracking systems for controlling the application, e.g. by means of the movement of the cursor which is updated many times per second by following the direction of the gaze, the error of accuracy makes the user's task very difficult, to whom a significant effort of concentration is required, which very soon generates fatigue.
- Some solutions in use in the state of the art provide resolving the aforesaid problem by means of enlarging a part of the display of the personal computer, so as to improve the accuracy in the positioning of the cursor.
- the functions wanted may be activated by pointing the gaze on icons arranged on strips of thin plastic-coated cardboard which are positioned on the vertical perimeter of the monitor: thereby the eye should select the functions external thereto before activating them on the icon or on the function wanted and then move the cursor which will be steered by the gaze towards the icons or applications wanted.
- the directions of the mouse pointer are activated with the gaze on the screen view on the monitor of the eye-tracking device and once the direction is selected the effect is observed on the other PC; as soon as the gaze is removed from the PC wanted to be controlled, the pointer stops and the commands wanted may be selected. Thereby, the second PC connected may be steered with the eye-tracking system, by means of the emulation of the mouse.
- One of the main requirements remains that of minimizing the cognitive effort required to manipulate the software by ensuring that the interface accepts "natural” inputs and responds "naturally” and is easily comprehensible.
- the object of the present invention provides a new mapping of the "original" native interactors of the operating system of the personal computer (icons, etc.) in new interactors modified and made suitable to the selection mode by means of ocular control. Therefore the present invention establishes a sort of "direct channel", as the emulation of the cursor is overcome by the fact that the native interactors are replaced by those modified and adapted to the need to use the gaze as input system.
- a Virtual Machine which permits to manage and create a virtual environment to carry out a further operating system with related software applications and which the user may use simultaneously to the one started, provides further advantages.
- One of them is security: a virtual machine is completely isolated and independent and a sudden crash of the virtual machine does not involve the hosting operating system to crash; therefore restarting the computer is not required but only terminating the virtual machine and starting it again while avoiding damaging e.g. the file system.
- the method according to the present invention is integrated in a communication suite, by using the virtual machine it is possible to simply, quickly and safely move (by means of suspension of the virtual machine) from using the classic PC applications, by means of the control techniques described below, to the communication suite designed specifically for being used via ocular control, thus overcoming the problems of the systems in the state of the art which provide rather complicated procedures for moving from one mode to the other.
- the user may directly choose, by means of suitable shortcuts, to carry out some predefined applications inside the communication suite thus overcoming the problem of the possible direct execution from the operating system.
- Fig. 1 shows the block diagram of the architecture of the method in accordance with the present invention.
- Fig. 2 shows the flow diagram of the method according to the present invention.
- Fig. 3 shows the flow diagram of the module relating to filtering the rough data coming from the eye-tracking device.
- Fig. 4 shows the flow diagram relating to the Application Control Module.
- Fig. 5 shows the flow diagram relating to the Coordinate Mapping Module.
- Fig. 6 shows the flow diagram relating to the data recovery strategies relating to the interactors in the screen views.
- Fig. 7 - 10 show, by way of non-limiting example, a possible implementation of the graphic interface of the application object of the present invention.
- the object of the present invention consists of a method and an apparatus for using a generic operating system and generic software applications connected thereto, by means of ocular control.
- a further object of the present invention consists of suitable methods of interaction developed by means of interaction techniques and an intuitive and easy-to-use user interface as described in the claims which form an integral part of the present description.
- the method object of the present invention therefore depicts a possible implementation of an assistive technology, extremely innovative in terms of control of a generic operating system and of the applications connected thereto, based on the use of alternative and natural inputs, such as the gaze.
- the apparatus object of the present invention comprises electronic means of data and of information processing, means for memorizing said data and information and user interfacing means.
- Said electronic data and information processing means comprise a suitable control section, preferably based on at least a micro-processor, and may, e.g., be provided by a personal computer.
- Said memorizing means preferably comprise hard disks and storage devices of flash type.
- Said user interfacing means preferably comprise data visualising means, such as e.g. displays, monitors or analogous external output units and eye-tracking devices adapted to interpret the direction of the user's gaze.
- Said micro-processor is preferably equipped with an operating system, with a suitable virtual environment by means of the use of a virtual machine and by a suitable software program which implements a method whose architecture, described in Figure 1 , comprises the following modules, in turn comprising a series of instructions adapted to performing a specific task: a filtering module 10 in which the coordinates of the user's gaze are processed so as to make the rough data coming from the used eye-tracking device more stable; a module, namely an Operating System/Applications Control 11 , responsible for controlling the operating system and the applications associated therewith and for running the developing application graphic interfaces, interface which contains the information about the interactors in the screen view and carries out the native action associated with the interactor fixed by the user at that moment, said interactors being the activatable elements in the interface: the icons, the pop-down menus, the check boxes etc., adapted to make the user carry out actions on the applications program to be controlled.
- a filtering module 10 in which the coordinates of the user's
- Said Operating System/Applications Control Module 11 is formed by two component sub-modules: a Controller Module 12 and a Client Module 13.
- Said Controller Module 12 is in charge of managing the presentation of the interactors and of defining the native action associated with each of them and in turn comprises three further modules which interact with each other:
- a Coordinate Mapping Module 12A which is in charge of carrying out new mapping of the coordinates relating to the screen views and to the interactors therein (different between Client and Controller);
- an Interactor Managing Module 12B which is in charge of carrying out the comparison with the incoming gaze to define which interactors are fixed by the user and presents them, suitably and possibly modified, on the interface, e.g. on a side panel;
- a Native Action Definition Module 12C which is in charge of defining the native action associated with each interactor of the Operating System and of sending it to said Client Module 13, thus making it available for successive processing.
- Said Coordinate Mapping Module 12A in turn consists of two sub-modules which interact with each other: a Coordinates Translation Sub-Module 14 which carries out a translation of the coordinates relating to screen views and interactors and an Adaptive Calibration Sub-Module 15 which carries out a further re-adjustment of the coordinates by means of geometrical deformation of the plane obtained by comparing the information on the interactors which the user may select and the coordinates of the gaze coming from the eye-tracker, the results of the combined actions of these 2 modules is the one described above concerning the Coordinates Mapping Module 12A.
- Said Client Module 13 is adapted to defining the position and the function of the interactors in the screen view and in turn comprises two further modules which interact with each other: the Interface Managing Module 13A which is in charge of analysing the screen views and sending the information relating thereto and to the present interactors, detected by means of different search methods, to the Controller Module 12; the Native Action Managing Module 13B which is in charge of receiving the information relating to the associated native action and of carrying it out as action on said operating system.
- the Interface Managing Module 13A which is in charge of analysing the screen views and sending the information relating thereto and to the present interactors, detected by means of different search methods, to the Controller Module 12
- the Native Action Managing Module 13B which is in charge of receiving the information relating to the associated native action and of carrying it out as action on said operating system.
- a control user interface adapted to permit the user to control the operating system and the application programs associated therewith of said electronic processing means are visualised 20.
- step e) The action to be carried out determined by the previous step is performed 23 and, possibly, said control user interface is suitably modified following the action itself. f) The sequence is repeated starting from step b) until the user decides to leave the application program which he/she is using.
- a pre-filtering 30 of the rough data is performed and the non-valid samples are detected and eliminated using, e.g., statistical criteria.
- the Interface Managing Module 13A analyses the screen views and sends 40 the information relating to the screen views and to the interactors in the current user interface to the Coordinate Mapping Module 12A.
- the Coordinate Mapping Module 12A carries out new mapping of the coordinates relating to the screen views and to the interactors and carries out a comparison with the data relating to the gaze coming from the eye- tracker. Said new mapping allows the coordinates coming from the Client Module 13 relating to the position of the interactors in the screen views to be defined with respect to another system of axes having different origin than the one based on which the coordinates of the gazes coming from the eye-tracking device are defined. Once this operation has been carried out the comparison permits to understand which interactor is fixed by the user.
- the Interactor Managing Module 12B draws again 42 the interactors present in the surroundings of the gazes detected by showing them suitably (e.g.
- the Interactor Managing Module 12B then waits for the data relating to the gazes coming from the eye-tracker and for the selection of one of the interactors in the side panel.
- the Native Action Definition Module 12C defines 43 the native action of the Operating System associated with each interactor (i.e. events which are simulated by the Client Module 13 and correspond to insertions of controls from the keyboard, simple click, drag & drop, etc.) and sends it to the Native Action Managing Module 13B.
- the Native Action Managing Module 13B receives the information relating to the native action associated with the interactor selected and carries it out 44 as action on the Operating System (e.g. send character, mouse movement in specific position, etc.).
- the Coordinate Translation Module 14 carries out 50 a translation of the coordinates relating to screen views and incoming interactors and sends these data to the Adaptive Calibration Module 15.
- the Adaptive Calibration Module 15 carries out 51 a further re-adjustment of the coordinates, by means of geometrical deformation of the plane obtained by comparing the information on the interactors which the user may select and the coordinates of the gaze coming from the eye-tracker, and sends the information for updating the mapping to the Coordinate
- the Interface Managing Module carries out the search of the interactors in the view screens continuously during the entire process described above, by means of the use of the steps described below and illustrated in Figure 6:
- the Interface Managing Module 13A queries 60 the API (Application Programming Interfaces) of the accessibility functions to track the position and the functions of the different interactors in the page.
- the Interface Managing Module 13A implements 61 an algorithm of recursive crossing of the diagram of windows in the page while trying to extract further information (in order to overcome the obstacle represented by insufficient information obtained by means of the accessibility API).
- the Interface Managing Module 13A consults 62 a pre-formed database of supported applications from which it obtains the strategies for using various applications.
- the database contains general information relating to how the various applications and the various application programs are made and heuristically obtains indications therefrom on how others of which nothing is known may be made.
- the method described allows, e.g., a disabled user to use a personal computer equipped, e.g., with an operating system and with application programs such as Windows® and the Microsoft® Office® package.
- the user is arranged in front of the eye-tracker connected to a monitor in which is visualised the screen view of the operating system/application wanted to be controlled by means of ocular movements.
- the user fixes e.g., the Start icon on the Windows® application bar close to which is the icon for the Word® application and the Windows® bar: the gaze moves in specific surroundings for the intrinsic features of the eye-tracker. Due to this and to the possible and intrinsic error of accuracy of the eye-tracking device, what the user is fixing on and what his/her intention is may not be said with certainty.
- all the interactors in the surrounding of the gaze e.g. Start button, Word® program icon, Windows®) bar
- are shown in a side panel suitably for selection by means of ocular control (well spaced and of suitable sizes).
- Such interactors are detected by means of suitable data recovery strategies (type, position, etc.) relating to the interactors in the screen view (accessibility API query, recursive crossing of windows diagram, database of predefined applications) and are shown in a "weighted” manner according to the order obtained by means of heuristic techniques (from the most probable to the least probable).
- the user fixes the interactor of interest on the side panel and as the buttons are well spaced and of suitable sizes, there is no ambiguity with respect to the user's choice.
- the button is therefore selected and the consequent action is carried out.
- the user firstly, arranges him/herself in front of the eye-tracker connected to a monitor in which is visualised the screen view 70 of the operation system/application wanted to be controlled by means of ocular movements ( Figure 7).
- the user fixes e.g., the Start icon on the Windows® application bar close to which is the icon for the Word® application and the Windows® bar: the gaze 71 moves in specific surroundings for the intrinsic features of the eye-tracker. Due to this and to the possible and intrinsic error of accuracy of the eye-tracking device, what the user is fixing on and what his/her intention is may not be said with certainty (Figure 7).
- All the interactors in the surrounding of the gaze 71 are shown in a side panel 72, suitably for selection by means of ocular control (well spaced and of suitable sizes).
- Such interactors are detected by means of suitable data recovery strategies (type, position, etc.) relating to the interactors in the screen view (accessibility API query, recursive crossing of windows diagram, database of predefined applications) and are shown in a "weighted" manner according to the order obtained by means of heuristic techniques (from the most probable to the least probable) ( Figure 8).
- the user fixes the interactor of interest 73 on the side panel 72: as the buttons are well spaced and of suitable sizes, there is no ambiguity with respect to the choice of the user ( Figure 9).
- the button is selected and the consequent action is carried out.
Landscapes
- Engineering & Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- User Interface Of Digital Computer (AREA)
- Position Input By Displaying (AREA)
- Prostheses (AREA)
- Stored Programmes (AREA)
Abstract
Description
Claims
Priority Applications (8)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
ES10787877.9T ES2661554T3 (en) | 2009-09-11 | 2010-09-13 | Method and apparatus for using generic software applications by means of an eye control and appropriate interaction methods |
NO10787877A NO2476043T3 (en) | 2009-09-11 | 2010-09-13 | |
DK10787877.9T DK2476043T3 (en) | 2009-09-11 | 2010-09-13 | Method and Device for Using Generic Software Applications by Eye Management and Appropriate Methods of Interaction |
EP10787877.9A EP2476043B1 (en) | 2009-09-11 | 2010-09-13 | Method and apparatus for using generic software applications by means of ocular control and suitable methods of interaction |
CN2010800394075A CN102483650A (en) | 2009-09-11 | 2010-09-13 | Method and apparatus for using generic software applications by means of ocular control and suitable methods of interaction |
BR112012005342-6A BR112012005342B1 (en) | 2009-09-11 | 2010-09-13 | METHOD FOR CONTROLLING THE OPERATION OF AN OPERATING SYSTEM AND ASSOCIATED APPLICATIONS PROGRAM |
US13/395,014 US9372605B2 (en) | 2009-09-11 | 2010-09-13 | Method and apparatus for controlling the operation of an operating system and application programs by ocular control |
CA2773636A CA2773636C (en) | 2009-09-11 | 2010-09-13 | Method and apparatus for using generic software applications by ocular control and suitable methods of interaction |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
ITFI2009A000198 | 2009-09-11 | ||
ITFI2009A000198A IT1399456B1 (en) | 2009-09-11 | 2009-09-11 | METHOD AND APPARATUS FOR THE USE OF GENERIC SOFTWARE APPLICATIONS THROUGH EYE CONTROL AND INTERACTION METHODS IS APPROPRIATE. |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2011030212A1 true WO2011030212A1 (en) | 2011-03-17 |
Family
ID=42102373
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/IB2010/002271 WO2011030212A1 (en) | 2009-09-11 | 2010-09-13 | Method and apparatus for using generic software applications by means of ocular control and suitable methods of interaction |
Country Status (10)
Country | Link |
---|---|
US (1) | US9372605B2 (en) |
EP (1) | EP2476043B1 (en) |
CN (1) | CN102483650A (en) |
CA (1) | CA2773636C (en) |
DK (1) | DK2476043T3 (en) |
ES (1) | ES2661554T3 (en) |
IT (1) | IT1399456B1 (en) |
NO (1) | NO2476043T3 (en) |
PT (1) | PT2476043T (en) |
WO (1) | WO2011030212A1 (en) |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
ITFI20120165A1 (en) * | 2012-08-08 | 2014-02-09 | Sr Labs S R L | INTERACTIVE EYE CONTROL MULTIMEDIA SYSTEM FOR ACTIVE AND PASSIVE TRACKING |
WO2013173148A3 (en) * | 2012-05-18 | 2016-06-09 | Microsoft Corporation | Interaction and management of devices using gaze detection |
Families Citing this family (14)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2014015521A1 (en) * | 2012-07-27 | 2014-01-30 | Nokia Corporation | Multimodal interaction with near-to-eye display |
US20140049462A1 (en) * | 2012-08-20 | 2014-02-20 | Google Inc. | User interface element focus based on user's gaze |
CN103870097A (en) * | 2012-12-12 | 2014-06-18 | 联想(北京)有限公司 | Information processing method and electronic equipment |
US9342145B2 (en) * | 2013-01-22 | 2016-05-17 | Kabushiki Kaisha Toshiba | Cursor control |
CN104007909B (en) * | 2013-02-25 | 2019-03-01 | 腾讯科技(深圳)有限公司 | Page automatic adjusting method and device |
US9829975B2 (en) | 2013-08-27 | 2017-11-28 | Auckland Uniservices Limited | Gaze-controlled interface method and system |
JP6115418B2 (en) | 2013-09-11 | 2017-04-19 | 富士通株式会社 | Information processing apparatus, method, and program |
US9310886B2 (en) * | 2014-02-13 | 2016-04-12 | Lenovo (Singapore) Pte. Ltd. | Transient message display control |
US9804753B2 (en) * | 2014-03-20 | 2017-10-31 | Microsoft Technology Licensing, Llc | Selection using eye gaze evaluation over time |
WO2015164467A1 (en) * | 2014-04-23 | 2015-10-29 | Abbott Medical Optics Inc. | Medical device data filtering for real time display |
US11430414B2 (en) * | 2019-10-17 | 2022-08-30 | Microsoft Technology Licensing, Llc | Eye gaze control of magnification user interface |
DE102022110291B3 (en) | 2022-04-27 | 2023-11-02 | Universität Stuttgart, Körperschaft Des Öffentlichen Rechts | Computer-implemented method and system for hands-free selection of a control on a screen |
CN115122342B (en) * | 2022-09-02 | 2022-12-09 | 北京壹点灵动科技有限公司 | Software system for controlling robot and control method of robot |
CN117369649B (en) * | 2023-12-05 | 2024-03-26 | 山东大学 | Virtual reality interaction system and method based on proprioception |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6323884B1 (en) * | 1999-03-31 | 2001-11-27 | International Business Machines Corporation | Assisting user selection of graphical user interface elements |
US20050047629A1 (en) * | 2003-08-25 | 2005-03-03 | International Business Machines Corporation | System and method for selectively expanding or contracting a portion of a display using eye-gaze tracking |
WO2006045843A1 (en) * | 2004-10-29 | 2006-05-04 | Sr Labs S.R.L. | Method and system of visualisation, processing and integrated analysis of medical images |
WO2007017500A1 (en) * | 2005-08-10 | 2007-02-15 | Sr Labs S.R.L. | Method and apparatus for secure insertion of an access code using an eye-tracking device |
Family Cites Families (35)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US4950069A (en) * | 1988-11-04 | 1990-08-21 | University Of Virginia | Eye movement detector with improved calibration and speed |
US5471542A (en) * | 1993-09-27 | 1995-11-28 | Ragland; Richard R. | Point-of-gaze tracker |
GB9518477D0 (en) * | 1995-09-09 | 1995-11-08 | Strachan John S | Point of observation tracking system |
JPH09114543A (en) * | 1995-10-02 | 1997-05-02 | Xybernaut Corp | Handfree computer system |
US5912721A (en) * | 1996-03-13 | 1999-06-15 | Kabushiki Kaisha Toshiba | Gaze detection apparatus and its method as well as information display apparatus |
US6373961B1 (en) * | 1996-03-26 | 2002-04-16 | Eye Control Technologies, Inc. | Eye controllable screen pointer |
US5731805A (en) * | 1996-06-25 | 1998-03-24 | Sun Microsystems, Inc. | Method and apparatus for eyetrack-driven text enlargement |
US5898423A (en) * | 1996-06-25 | 1999-04-27 | Sun Microsystems, Inc. | Method and apparatus for eyetrack-driven captioning |
US5886683A (en) * | 1996-06-25 | 1999-03-23 | Sun Microsystems, Inc. | Method and apparatus for eyetrack-driven information retrieval |
US5638176A (en) * | 1996-06-25 | 1997-06-10 | International Business Machines Corporation | Inexpensive interferometric eye tracking system |
US5850211A (en) * | 1996-06-26 | 1998-12-15 | Sun Microsystems, Inc. | Eyetrack-driven scrolling |
US6152563A (en) * | 1998-02-20 | 2000-11-28 | Hutchinson; Thomas E. | Eye gaze direction tracker |
US6204828B1 (en) * | 1998-03-31 | 2001-03-20 | International Business Machines Corporation | Integrated gaze/manual cursor positioning system |
GB9823977D0 (en) * | 1998-11-02 | 1998-12-30 | Scient Generics Ltd | Eye tracking method and apparatus |
US6577329B1 (en) * | 1999-02-25 | 2003-06-10 | International Business Machines Corporation | Method and system for relevance feedback through gaze tracking and ticker interfaces |
US6724402B1 (en) * | 2000-01-27 | 2004-04-20 | David R. Baquero | Method of launching computer programs within a graphical user interface |
US6456262B1 (en) * | 2000-05-09 | 2002-09-24 | Intel Corporation | Microdisplay with eye gaze detection |
US6603491B2 (en) * | 2000-05-26 | 2003-08-05 | Jerome H. Lemelson | System and methods for controlling automatic scrolling of information on a display or screen |
US6601021B2 (en) * | 2000-12-08 | 2003-07-29 | Xerox Corporation | System and method for analyzing eyetracker data |
US6578962B1 (en) * | 2001-04-27 | 2003-06-17 | International Business Machines Corporation | Calibration-free eye gaze tracking |
US6886137B2 (en) * | 2001-05-29 | 2005-04-26 | International Business Machines Corporation | Eye gaze control of dynamic information presentation |
US7913183B2 (en) * | 2002-10-08 | 2011-03-22 | Microsoft Corporation | System and method for managing software applications in a graphical user interface |
US7281248B2 (en) * | 2002-11-19 | 2007-10-09 | Microsoft Corporation | Virtualized and realized user interface controls |
US7206022B2 (en) * | 2002-11-25 | 2007-04-17 | Eastman Kodak Company | Camera system with eye monitoring |
US6637883B1 (en) * | 2003-01-23 | 2003-10-28 | Vishwas V. Tengshe | Gaze tracking system and method |
US7401920B1 (en) * | 2003-05-20 | 2008-07-22 | Elbit Systems Ltd. | Head mounted eye tracking and display system |
US7365738B2 (en) * | 2003-12-02 | 2008-04-29 | International Business Machines Corporation | Guides and indicators for eye movement monitoring systems |
ES2535364T3 (en) * | 2004-06-18 | 2015-05-08 | Tobii Ab | Eye control of computer equipment |
EP1679577A1 (en) * | 2005-01-10 | 2006-07-12 | Tobii Technology AB | Adaptive display of eye controllable objects |
WO2007107949A1 (en) * | 2006-03-23 | 2007-09-27 | Koninklijke Philips Electronics N.V. | Hotspots for eye track control of image manipulation |
US8225229B2 (en) * | 2006-11-09 | 2012-07-17 | Sony Mobile Communications Ab | Adjusting display brightness and/or refresh rates based on eye tracking |
US20080270910A1 (en) * | 2007-04-30 | 2008-10-30 | Lukasik Derek J | User selection of a remote session |
US7556377B2 (en) * | 2007-09-28 | 2009-07-07 | International Business Machines Corporation | System and method of detecting eye fixations using adaptive thresholds |
ITFI20080049A1 (en) * | 2008-03-12 | 2009-09-13 | Sr Labs Srl | APPARATUS FOR THE CREATION, RESCUE AND FORMAT OF TEXTUAL DOCUMENTS THROUGH EYE CONTROL AND ASSOCIATED METHOD BASED ON THE OPTIMIZED POSITIONING OF THE CURSOR. |
US20100182232A1 (en) * | 2009-01-22 | 2010-07-22 | Alcatel-Lucent Usa Inc. | Electronic Data Input System |
-
2009
- 2009-09-11 IT ITFI2009A000198A patent/IT1399456B1/en active
-
2010
- 2010-09-13 CN CN2010800394075A patent/CN102483650A/en active Pending
- 2010-09-13 CA CA2773636A patent/CA2773636C/en active Active
- 2010-09-13 PT PT107878779T patent/PT2476043T/en unknown
- 2010-09-13 US US13/395,014 patent/US9372605B2/en active Active
- 2010-09-13 ES ES10787877.9T patent/ES2661554T3/en active Active
- 2010-09-13 NO NO10787877A patent/NO2476043T3/no unknown
- 2010-09-13 EP EP10787877.9A patent/EP2476043B1/en active Active
- 2010-09-13 DK DK10787877.9T patent/DK2476043T3/en active
- 2010-09-13 WO PCT/IB2010/002271 patent/WO2011030212A1/en active Application Filing
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6323884B1 (en) * | 1999-03-31 | 2001-11-27 | International Business Machines Corporation | Assisting user selection of graphical user interface elements |
US20050047629A1 (en) * | 2003-08-25 | 2005-03-03 | International Business Machines Corporation | System and method for selectively expanding or contracting a portion of a display using eye-gaze tracking |
WO2006045843A1 (en) * | 2004-10-29 | 2006-05-04 | Sr Labs S.R.L. | Method and system of visualisation, processing and integrated analysis of medical images |
WO2007017500A1 (en) * | 2005-08-10 | 2007-02-15 | Sr Labs S.R.L. | Method and apparatus for secure insertion of an access code using an eye-tracking device |
Non-Patent Citations (1)
Title |
---|
KIYOHIKO ABE ET AL: "A system for web browsing by eye-gaze input", ELECTRONICS AND COMMUNICATIONS IN JAPAN, SCRIPTA TECHNICA. NEW YORK, US LNKD- DOI:10.1002/ECJ.10110, vol. 91, no. 5, 1 May 2008 (2008-05-01), pages 11 - 18, XP001523149 * |
Cited By (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2013173148A3 (en) * | 2012-05-18 | 2016-06-09 | Microsoft Corporation | Interaction and management of devices using gaze detection |
US9823742B2 (en) | 2012-05-18 | 2017-11-21 | Microsoft Technology Licensing, Llc | Interaction and management of devices using gaze detection |
US10067563B2 (en) | 2012-05-18 | 2018-09-04 | Microsoft Technology Licensing, Llc | Interaction and management of devices using gaze detection |
ITFI20120165A1 (en) * | 2012-08-08 | 2014-02-09 | Sr Labs S R L | INTERACTIVE EYE CONTROL MULTIMEDIA SYSTEM FOR ACTIVE AND PASSIVE TRACKING |
WO2014024159A1 (en) * | 2012-08-08 | 2014-02-13 | Sr Labs S.R.L. | Interactive eye-control multimedia system for active and passive tracking |
Also Published As
Publication number | Publication date |
---|---|
CN102483650A (en) | 2012-05-30 |
DK2476043T3 (en) | 2018-03-05 |
EP2476043B1 (en) | 2017-12-06 |
ES2661554T3 (en) | 2018-04-02 |
US20120173999A1 (en) | 2012-07-05 |
BR112012005342A2 (en) | 2021-11-03 |
EP2476043A1 (en) | 2012-07-18 |
ITFI20090198A1 (en) | 2011-03-12 |
CA2773636A1 (en) | 2011-03-17 |
IT1399456B1 (en) | 2013-04-19 |
US9372605B2 (en) | 2016-06-21 |
PT2476043T (en) | 2018-03-16 |
NO2476043T3 (en) | 2018-05-05 |
CA2773636C (en) | 2022-08-02 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CA2773636C (en) | Method and apparatus for using generic software applications by ocular control and suitable methods of interaction | |
US9952672B2 (en) | Arrangement, method and computer program for controlling a computer apparatus based on eye-tracking | |
US6963937B1 (en) | Method and apparatus for providing configurability and customization of adaptive user-input filtration | |
EP2266014B1 (en) | Apparatus to create, save and format text documents using gaze control and associated method | |
US8082070B2 (en) | Methods and systems for displaying assistance messages to aircraft operators | |
US20140049462A1 (en) | User interface element focus based on user's gaze | |
US10831346B2 (en) | Ergonomic and sensor analysis based user experience design | |
CN111696546B (en) | Using a multimodal interface to facilitate the discovery of verbal commands | |
Bacher | Augmented User Interaction | |
Istance et al. | Eye-based control of standard GUI software | |
CN114924646A (en) | Multi-mode interaction method and system for head-mounted display equipment | |
Bos | Design of an Eye-Gaze Interface for Controlling an Assistive Robot Arm | |
CN114967918A (en) | Multi-mode interaction method and system for head-mounted display equipment | |
Breuninger | Suitability of Touch Gestures and Virtual Physics in Touch Screen User Interfaces for Critical Tasks | |
Bacher | Web2cHMI. A multi-modal native user interface implementation for accelerator operations and maintenance applications | |
BR112012005342B1 (en) | METHOD FOR CONTROLLING THE OPERATION OF AN OPERATING SYSTEM AND ASSOCIATED APPLICATIONS PROGRAM | |
Rozado et al. | RESNA/NCART 2016 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
WWE | Wipo information: entry into national phase |
Ref document number: 201080039407.5 Country of ref document: CN |
|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 10787877 Country of ref document: EP Kind code of ref document: A1 |
|
WWE | Wipo information: entry into national phase |
Ref document number: 13395014 Country of ref document: US Ref document number: 2773636 Country of ref document: CA |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
WWE | Wipo information: entry into national phase |
Ref document number: 2010787877 Country of ref document: EP |
|
WWE | Wipo information: entry into national phase |
Ref document number: 3210/CHENP/2012 Country of ref document: IN |
|
REG | Reference to national code |
Ref country code: BR Ref legal event code: B01A Ref document number: 112012005342 Country of ref document: BR |
|
ENP | Entry into the national phase |
Ref document number: 112012005342 Country of ref document: BR Kind code of ref document: A2 Effective date: 20120309 |
|
ENP | Entry into the national phase |
Ref document number: 112012005342 Country of ref document: BR Kind code of ref document: A2 Effective date: 20120309 |