IL279705B2 - A gaze-based interface for an augmented reality environment - Google Patents

A gaze-based interface for an augmented reality environment

Info

Publication number
IL279705B2
IL279705B2 IL279705A IL27970520A IL279705B2 IL 279705 B2 IL279705 B2 IL 279705B2 IL 279705 A IL279705 A IL 279705A IL 27970520 A IL27970520 A IL 27970520A IL 279705 B2 IL279705 B2 IL 279705B2
Authority
IL
Israel
Prior art keywords
augmented reality
user
hmd
display
rotation
Prior art date
Application number
IL279705A
Other languages
English (en)
Hebrew (he)
Other versions
IL279705B1 (en
IL279705A (en
Original Assignee
Sentiar Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sentiar Inc filed Critical Sentiar Inc
Publication of IL279705A publication Critical patent/IL279705A/en
Publication of IL279705B1 publication Critical patent/IL279705B1/en
Publication of IL279705B2 publication Critical patent/IL279705B2/en

Links

Classifications

    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G02B27/017—Head mounted
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/012—Head tracking input arrangements
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/013—Eye tracking input arrangements
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
    • G06F3/04815—Interaction with a metaphor-based environment or interaction object displayed as three-dimensional [3D], e.g. changing the user viewpoint with respect to the environment or object
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating three-dimensional [3D] models or images for computer graphics
    • G06T19/006—Mixed reality
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G02B27/0179—Display position adjusting means not related to the information to be displayed
    • G02B2027/0187—Display position adjusting means not related to the information to be displayed slaved to motion of at least a part of the body of the user, e.g. head, eye

Landscapes

  • Engineering & Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Software Systems (AREA)
  • Computer Graphics (AREA)
  • Computer Hardware Design (AREA)
  • Optics & Photonics (AREA)
  • User Interface Of Digital Computer (AREA)
  • Processing Or Creating Images (AREA)
  • Position Input By Displaying (AREA)
IL279705A 2018-06-27 2019-06-25 A gaze-based interface for an augmented reality environment IL279705B2 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US201862690834P 2018-06-27 2018-06-27
PCT/US2019/039076 WO2020006002A1 (en) 2018-06-27 2019-06-25 Gaze based interface for augmented reality environment

Publications (3)

Publication Number Publication Date
IL279705A IL279705A (en) 2021-03-01
IL279705B1 IL279705B1 (en) 2024-12-01
IL279705B2 true IL279705B2 (en) 2025-04-01

Family

ID=68985564

Family Applications (1)

Application Number Title Priority Date Filing Date
IL279705A IL279705B2 (en) 2018-06-27 2019-06-25 A gaze-based interface for an augmented reality environment

Country Status (8)

Country Link
US (2) US11199898B2 (cg-RX-API-DMAC7.html)
EP (1) EP3814829B1 (cg-RX-API-DMAC7.html)
JP (1) JP7213899B2 (cg-RX-API-DMAC7.html)
KR (1) KR102926290B1 (cg-RX-API-DMAC7.html)
AU (1) AU2019294617B2 (cg-RX-API-DMAC7.html)
CA (1) CA3105192A1 (cg-RX-API-DMAC7.html)
IL (1) IL279705B2 (cg-RX-API-DMAC7.html)
WO (1) WO2020006002A1 (cg-RX-API-DMAC7.html)

Families Citing this family (56)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR102567007B1 (ko) 2017-02-24 2023-08-16 마시모 코오퍼레이션 의료 모니터링 데이터 표시 시스템
US11024064B2 (en) * 2017-02-24 2021-06-01 Masimo Corporation Augmented reality system for displaying patient data
WO2018191970A1 (zh) * 2017-04-21 2018-10-25 深圳前海达闼云端智能科技有限公司 一种机器人控制方法、机器人装置及机器人设备
JP7159208B2 (ja) 2017-05-08 2022-10-24 マシモ・コーポレイション ドングルを使用することによって医療システムをネットワークコントローラとペアリングするためのシステム
DK180639B1 (en) 2018-06-01 2021-11-04 Apple Inc DISABILITY OF ATTENTION-ATTENTIVE VIRTUAL ASSISTANT
JP2021182174A (ja) * 2018-08-07 2021-11-25 ソニーグループ株式会社 情報処理装置、情報処理方法およびプログラム
US10891792B1 (en) * 2019-01-31 2021-01-12 Splunk Inc. Precise plane detection and placement of virtual objects in an augmented reality environment
US12141923B2 (en) 2019-11-25 2024-11-12 Boe Technology Group Co., Ltd. Health management system, and human body information display method and human body model generation method applied to same
US11816757B1 (en) * 2019-12-11 2023-11-14 Meta Platforms Technologies, Llc Device-side capture of data representative of an artificial reality environment
US10921882B1 (en) * 2019-12-26 2021-02-16 Jie Li Human-machine interaction method, system and apparatus for controlling an electronic device
EP4495750A3 (en) 2020-03-27 2025-03-26 Apple Inc. Devices, methods, and graphical user interfaces for gaze-based navigation
US11922581B2 (en) 2020-05-08 2024-03-05 Coviden Lp Systems and methods of controlling an operating room display using an augmented reality headset
KR102833872B1 (ko) 2020-09-25 2025-07-15 애플 인크. 환경에서 객체들을 조작하기 위한 방법들
KR102959016B1 (ko) 2020-09-25 2026-04-28 애플 인크. 사용자 인터페이스와 연관된 몰입을 조정 및/또는 제어하기 위한 방법
WO2022067342A2 (en) 2020-09-25 2022-03-31 Apple Inc. Methods for interacting with virtual controls and/or an affordance for moving virtual objects in virtual environments
KR102876731B1 (ko) * 2020-12-16 2025-10-27 삼성전자 주식회사 저지연으로 복수의 어플리케이션 데이터 전송 방법 및 장치
CN116670627A (zh) 2020-12-31 2023-08-29 苹果公司 对环境中的用户界面进行分组的方法
US11995230B2 (en) 2021-02-11 2024-05-28 Apple Inc. Methods for presenting and sharing content in an environment
EP4392853A1 (en) 2021-08-27 2024-07-03 Apple Inc. System and method of augmented representation of an electronic device
KR102903963B1 (ko) 2021-09-25 2025-12-26 애플 인크. 가상 환경들에서 가상 객체들을 제시하기 위한 디바이스들, 방법들, 및 그래픽 사용자 인터페이스들
US12456271B1 (en) 2021-11-19 2025-10-28 Apple Inc. System and method of three-dimensional object cleanup and text annotation
CN118871875A (zh) 2022-01-12 2024-10-29 苹果公司 用于在环境中显示、选择以及移动对象和容器的方法
EP4466593A1 (en) 2022-01-19 2024-11-27 Apple Inc. Methods for displaying and repositioning objects in an environment
EP4398593A4 (en) 2022-02-11 2025-02-26 Samsung Electronics Co., Ltd. METHOD AND DEVICE FOR OBTAINING AN IMAGE OF AN OBJECT
US12272005B2 (en) 2022-02-28 2025-04-08 Apple Inc. System and method of three-dimensional immersive applications in multi-user communication sessions
US12541280B2 (en) 2022-02-28 2026-02-03 Apple Inc. System and method of three-dimensional placement and refinement in multi-user communication sessions
JP7856460B2 (ja) * 2022-03-25 2026-05-11 富士フイルム株式会社 制御装置、制御方法、及び制御プログラム
WO2023196258A1 (en) 2022-04-04 2023-10-12 Apple Inc. Methods for quick message response and dictation in a three-dimensional environment
WO2023205190A1 (en) 2022-04-18 2023-10-26 Lyv Technologies Inc. Mixed-reality beacons
WO2023205362A1 (en) 2022-04-20 2023-10-26 Apple Inc. Obstructed objects in a three-dimensional environment
EP4511723A1 (en) 2022-04-21 2025-02-26 Apple Inc. Representations of messages in a three-dimensional environment
US12283020B2 (en) 2022-05-17 2025-04-22 Apple Inc. Systems, methods, and user interfaces for generating a three-dimensional virtual representation of an object
US12394167B1 (en) 2022-06-30 2025-08-19 Apple Inc. Window resizing and virtual object rearrangement in 3D environments
JP2025534239A (ja) 2022-09-14 2025-10-15 アップル インコーポレイテッド 三次元環境内の深度競合軽減のための方法
US12148078B2 (en) 2022-09-16 2024-11-19 Apple Inc. System and method of spatial groups in multi-user communication sessions
US12112011B2 (en) 2022-09-16 2024-10-08 Apple Inc. System and method of application-based three-dimensional refinement in multi-user communication sessions
US12169598B2 (en) * 2022-09-19 2024-12-17 Snap Inc. AR glasses as IoT remote control
US12099653B2 (en) 2022-09-22 2024-09-24 Apple Inc. User interface response based on gaze-holding event assessment
CN120469584A (zh) 2022-09-23 2025-08-12 苹果公司 用于操纵虚拟对象的方法
US12405704B1 (en) 2022-09-23 2025-09-02 Apple Inc. Interpreting user movement as direct touch user interface interactions
CN120266083A (zh) 2022-09-24 2025-07-04 苹果公司 用于基于注意力与用户界面交互的方法
US12535931B2 (en) 2022-09-24 2026-01-27 Apple Inc. Methods for controlling and interacting with a three-dimensional environment
JP2024052410A (ja) * 2022-09-30 2024-04-11 富士フイルム株式会社 プロセッサ、情報処理方法、及び情報処理プログラム
US12361661B1 (en) 2022-12-21 2025-07-15 Meta Platforms Technologies, Llc Artificial reality (XR) location-based displays and interactions
US12501099B2 (en) 2023-01-17 2025-12-16 Snap Inc. AR glasses as IoT device for enhanced screen experience
CN121165940A (zh) 2023-01-30 2025-12-19 苹果公司 用于响应于注视和/或手势输入来显示多组控件的设备、方法和图形用户界面
WO2024186838A2 (en) * 2023-03-05 2024-09-12 Walker Stanley A system for an independent recursive virtual world with a voice controlled interface
US12657958B2 (en) 2023-03-31 2026-06-16 Apple Inc. Pinch recognition and rejection
WO2024226681A1 (en) 2023-04-25 2024-10-31 Apple Inc. Methods for displaying and rearranging objects in an environment
US12443286B2 (en) 2023-06-02 2025-10-14 Apple Inc. Input recognition based on distinguishing direct and indirect user interactions
US12118200B1 (en) 2023-06-02 2024-10-15 Apple Inc. Fuzzy hit testing
US20250078420A1 (en) 2023-06-04 2025-03-06 Apple Inc. Methods for managing overlapping windows and applying visual effects
US12099695B1 (en) 2023-06-04 2024-09-24 Apple Inc. Systems and methods of managing spatial groups in multi-user communication sessions
US12399550B2 (en) 2024-01-25 2025-08-26 Depuy Ireland Unlimited Company Triggering an acquisition during mixed-reality headset-based navigation
US12585131B2 (en) * 2024-04-09 2026-03-24 DISTANCE TECHNOLOGIES Oy Hybrid motion stabilisation for augmented reality displays
US20250378645A1 (en) 2024-06-09 2025-12-11 Apple Inc. Methods of interacting with content in a virtual environment

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090256840A1 (en) * 2008-04-14 2009-10-15 Google Inc. Panning Using Virtual Surfaces
US20160004306A1 (en) * 2010-07-23 2016-01-07 Telepatheye Inc. Eye-wearable device user interface and augmented reality method
US20160140766A1 (en) * 2012-12-12 2016-05-19 Sulon Technologies Inc. Surface projection system and method for augmented reality
US20170277256A1 (en) * 2016-03-22 2017-09-28 Aaron Mackay Burns Virtual-reality navigation

Family Cites Families (23)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130154913A1 (en) * 2010-12-16 2013-06-20 Siemens Corporation Systems and methods for a gaze and gesture interface
US20120257035A1 (en) 2011-04-08 2012-10-11 Sony Computer Entertainment Inc. Systems and methods for providing feedback by tracking user gaze and gestures
US8611015B2 (en) * 2011-11-22 2013-12-17 Google Inc. User interface
US11278353B2 (en) * 2016-03-16 2022-03-22 Synaptive Medical Inc. Trajectory alignment system and methods
US8988344B2 (en) * 2013-06-25 2015-03-24 Microsoft Technology Licensing, Llc User interface navigation
US9094677B1 (en) * 2013-07-25 2015-07-28 Google Inc. Head mounted display device with automated positioning
US20150153826A1 (en) * 2013-12-01 2015-06-04 Apx Labs, Llc Systems and methods for providing a virtual menu
US20150169070A1 (en) * 2013-12-17 2015-06-18 Google Inc. Visual Display of Interactive, Gesture-Controlled, Three-Dimensional (3D) Models for Head-Mountable Displays (HMDs)
US9696798B2 (en) * 2014-04-09 2017-07-04 Lenovo Enterprise Solutions (Singapore) Pte. Ltd. Eye gaze direction indicator
US10416760B2 (en) * 2014-07-25 2019-09-17 Microsoft Technology Licensing, Llc Gaze-based object placement within a virtual reality environment
US10186086B2 (en) * 2015-09-02 2019-01-22 Microsoft Technology Licensing, Llc Augmented reality control of computing device
BR112018007473A2 (pt) * 2015-10-14 2018-10-23 Surgical Theater LLC navegação cirúrgica de realidade aumentada
US10667981B2 (en) * 2016-02-29 2020-06-02 Mentor Acquisition One, Llc Reading assistance system for visually impaired
US10176641B2 (en) * 2016-03-21 2019-01-08 Microsoft Technology Licensing, Llc Displaying three-dimensional virtual objects based on field of view
CN108463765A (zh) * 2016-04-08 2018-08-28 谷歌有限责任公司 基于姿势信息在头戴式显示装置处编码图像数据
US10429647B2 (en) * 2016-06-10 2019-10-01 Facebook Technologies, Llc Focus adjusting virtual reality headset
JP6238381B1 (ja) 2016-06-30 2017-11-29 株式会社コナミデジタルエンタテインメント 端末装置、及びプログラム
US20180033198A1 (en) * 2016-07-29 2018-02-01 Microsoft Technology Licensing, Llc Forward direction determination for augmented reality and virtual reality
JP6939801B2 (ja) 2016-09-14 2021-09-22 ソニーグループ株式会社 情報処理装置、情報処理方法およびプログラム
US10664049B2 (en) * 2016-12-09 2020-05-26 Nvidia Corporation Systems and methods for gaze tracking
US10692287B2 (en) * 2017-04-17 2020-06-23 Microsoft Technology Licensing, Llc Multi-step placement of virtual objects
WO2018217951A1 (en) * 2017-05-24 2018-11-29 Camplex, Inc. Surgical visualization systems and displays
US10861236B2 (en) * 2017-09-08 2020-12-08 Surgical Theater, Inc. Dual mode augmented reality surgical system and method

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090256840A1 (en) * 2008-04-14 2009-10-15 Google Inc. Panning Using Virtual Surfaces
US20160004306A1 (en) * 2010-07-23 2016-01-07 Telepatheye Inc. Eye-wearable device user interface and augmented reality method
US20160140766A1 (en) * 2012-12-12 2016-05-19 Sulon Technologies Inc. Surface projection system and method for augmented reality
US20170277256A1 (en) * 2016-03-22 2017-09-28 Aaron Mackay Burns Virtual-reality navigation

Also Published As

Publication number Publication date
IL279705B1 (en) 2024-12-01
EP3814829A1 (en) 2021-05-05
KR20210059697A (ko) 2021-05-25
US20200004328A1 (en) 2020-01-02
IL279705A (en) 2021-03-01
AU2019294617B2 (en) 2025-02-06
EP3814829B1 (en) 2026-05-27
CA3105192A1 (en) 2020-01-02
JP2021528786A (ja) 2021-10-21
US11199898B2 (en) 2021-12-14
KR102926290B1 (ko) 2026-02-11
WO2020006002A1 (en) 2020-01-02
AU2019294617A1 (en) 2021-02-18
JP7213899B2 (ja) 2023-01-27
US11829526B2 (en) 2023-11-28
EP3814829A4 (en) 2022-03-23
US20220075449A1 (en) 2022-03-10

Similar Documents

Publication Publication Date Title
US11829526B2 (en) Gaze based interface for augmented reality environment
US11507336B2 (en) Augmented reality display sharing
Kim et al. Touch and hand gesture-based interactions for directly manipulating 3D virtual objects in mobile augmented reality
US10712901B2 (en) Gesture-based content sharing in artificial reality environments
JP7267209B2 (ja) 滅菌野及び他の作業環境のためのユーザインターフェースシステム
WO2021173839A1 (en) Hand gesture input for wearable system
US20130050069A1 (en) Method and system for use in providing three dimensional user interface
Lu et al. Immersive manipulation of virtual objects through glove-based hand gesture interaction
US12295784B2 (en) System and method for augmented reality data interaction for ultrasound imaging
TW202105129A (zh) 具有用於閘控使用者介面元件的個人助理元件之人工實境系統
CN116648683A (zh) 用于选择对象的方法和系统
EP4581459A1 (en) Physical gesture interaction with objects based on intuitive design
US20160004315A1 (en) System and method of touch-free operation of a picture archiving and communication system
US12322048B2 (en) Connecting spatially distinct settings
CN117784926A (zh) 控制装置、控制方法和计算机可读存储介质
JPH11237867A (ja) 仮想空間表示装置
US20160004318A1 (en) System and method of touch-free operation of a picture archiving and communication system
Ma et al. [POSTER] Natural User Interface for Ambient Objects
Takaki et al. Using gaze for 3-D direct manipulation interface
WO2025151641A1 (en) Techniques for updating a shared extended reality anchor based on operator interactions with a computer-assisted device
EP3958095A1 (en) A mobile computer-tethered virtual reality/augmented reality system using the mobile computer as a man machine interface
CN119112150A (zh) Mr设备及手术导航系统
JP2025171613A (ja) 情報処理装置
WO2026006568A1 (en) Virtual medical session interactions with non-powered objects