WO2023049418A3 - Devices, methods, and graphical user interfaces for interacting with media and three-dimensional environments - Google Patents

Devices, methods, and graphical user interfaces for interacting with media and three-dimensional environments Download PDF

Info

Publication number
WO2023049418A3
WO2023049418A3 PCT/US2022/044637 US2022044637W WO2023049418A3 WO 2023049418 A3 WO2023049418 A3 WO 2023049418A3 US 2022044637 W US2022044637 W US 2022044637W WO 2023049418 A3 WO2023049418 A3 WO 2023049418A3
Authority
WO
WIPO (PCT)
Prior art keywords
user interfaces
interacting
methods
graphical user
user
Prior art date
Application number
PCT/US2022/044637
Other languages
French (fr)
Other versions
WO2023049418A2 (en
Inventor
Israel PASTRANA VICENTE
Gregory M. Apodaca
Benjamin H. Boesel
Shih-Sang CHIU
Graham R. Clarke
Wesley M. Holder
Chia Yang Lin
Behkish J. MANZARI
Earl M. OLSON
James J. OWEN
Jonathan Ravasz
Miquel RODRIGUEZ ESTANY
William A. Sorrentino III
Original Assignee
Apple Inc.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Apple Inc. filed Critical Apple Inc.
Priority to KR1020247009664A priority Critical patent/KR20240047458A/en
Priority to EP22790141.0A priority patent/EP4384889A2/en
Priority to AU2022352756A priority patent/AU2022352756A1/en
Priority to CN202280062327.4A priority patent/CN117940880A/en
Publication of WO2023049418A2 publication Critical patent/WO2023049418A2/en
Publication of WO2023049418A3 publication Critical patent/WO2023049418A3/en
Priority to US18/374,562 priority patent/US20240028177A1/en

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/013—Eye tracking input arrangements
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
    • G06F3/0482—Interaction with lists of selectable items, e.g. menus
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
    • G06F3/0483—Interaction with page-structured environments, e.g. book metaphor
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
    • G06F3/04842—Selection of displayed objects or displayed text elements
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
    • G06F3/04845—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range for image manipulation, e.g. dragging, rotation, expansion or change of colour
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
    • G06F3/0485—Scrolling or panning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2203/00—Indexing scheme relating to G06F3/00 - G06F3/048
    • G06F2203/048—Indexing scheme relating to G06F3/048
    • G06F2203/04806—Zoom, i.e. interaction techniques or interactors for controlling the zooming operation

Landscapes

  • Engineering & Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

The present disclosure generally relates to user interfaces for electronic devices, including user interfaces for viewing and interacting with media items. The present disclosure also relates to user interfaces for providing an extended reality (XR) experience to a user, in accordance with some embodiments. The systems, methods, and graphical user interfaces (GUIs) described herein are directed to user interface interactions with virtual/augmented reality environments in multiple ways. In some embodiments, one or more visual characteristics of displayed content are modified based on where the gaze of a user is directed.
PCT/US2022/044637 2021-09-24 2022-09-24 Devices, methods, and graphical user interfaces for interacting with media and three-dimensional environments WO2023049418A2 (en)

Priority Applications (5)

Application Number Priority Date Filing Date Title
KR1020247009664A KR20240047458A (en) 2021-09-24 2022-09-24 Devices, methods and graphical user interfaces for interaction with media and three-dimensional environments
EP22790141.0A EP4384889A2 (en) 2021-09-24 2022-09-24 Devices, methods, and graphical user interfaces for interacting with media and three-dimensional environments
AU2022352756A AU2022352756A1 (en) 2021-09-24 2022-09-24 Devices, methods, and graphical user interfaces for interacting with media and three-dimensional environments
CN202280062327.4A CN117940880A (en) 2021-09-24 2022-09-24 Device, method and graphical user interface for interacting with media and three-dimensional environments
US18/374,562 US20240028177A1 (en) 2021-09-24 2023-09-28 Devices, methods, and graphical user interfaces for interacting with media and three-dimensional environments

Applications Claiming Priority (6)

Application Number Priority Date Filing Date Title
US202163248222P 2021-09-24 2021-09-24
US63/248,222 2021-09-24
US202217952206A 2022-09-23 2022-09-23
US202263409695P 2022-09-23 2022-09-23
US17/952,206 2022-09-23
US63/409,695 2022-09-23

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US18/374,562 Continuation US20240028177A1 (en) 2021-09-24 2023-09-28 Devices, methods, and graphical user interfaces for interacting with media and three-dimensional environments

Publications (2)

Publication Number Publication Date
WO2023049418A2 WO2023049418A2 (en) 2023-03-30
WO2023049418A3 true WO2023049418A3 (en) 2023-05-04

Family

ID=83693149

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2022/044637 WO2023049418A2 (en) 2021-09-24 2022-09-24 Devices, methods, and graphical user interfaces for interacting with media and three-dimensional environments

Country Status (4)

Country Link
EP (1) EP4384889A2 (en)
KR (1) KR20240047458A (en)
AU (1) AU2022352756A1 (en)
WO (1) WO2023049418A2 (en)

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130036380A1 (en) * 2011-08-02 2013-02-07 Apple Inc. Graphical User Interface for Tracking and Displaying Views of an Application
US20140195918A1 (en) * 2013-01-07 2014-07-10 Steven Friedlander Eye tracking user interface
US20150370323A1 (en) * 2014-06-19 2015-12-24 Apple Inc. User detection by a computing device
US20190155382A1 (en) * 2016-07-19 2019-05-23 Fujifilm Corporation Image display system, and control apparatus for head-mounted display and operation method therefor
US20200225746A1 (en) * 2017-09-29 2020-07-16 Apple Inc. Gaze-based user interactions
US20200379560A1 (en) * 2016-01-21 2020-12-03 Microsoft Technology Licensing, Llc Implicitly adaptive eye-tracking user interface

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130036380A1 (en) * 2011-08-02 2013-02-07 Apple Inc. Graphical User Interface for Tracking and Displaying Views of an Application
US20140195918A1 (en) * 2013-01-07 2014-07-10 Steven Friedlander Eye tracking user interface
US20150370323A1 (en) * 2014-06-19 2015-12-24 Apple Inc. User detection by a computing device
US20200379560A1 (en) * 2016-01-21 2020-12-03 Microsoft Technology Licensing, Llc Implicitly adaptive eye-tracking user interface
US20190155382A1 (en) * 2016-07-19 2019-05-23 Fujifilm Corporation Image display system, and control apparatus for head-mounted display and operation method therefor
US20200225746A1 (en) * 2017-09-29 2020-07-16 Apple Inc. Gaze-based user interactions

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
DAVID FONO ET AL: "EyeWindows", CHI 2005. CONFERENCE PROCEEDINGS. CONFERENCE ON HUMAN FACTORS IN COMPUTING SYSTEMS; [CHI CONFERENCE PROCEEDINGS. HUMAN FACTORS IN COMPUTING SYSTEMS], ACM, NY, US, 2 April 2005 (2005-04-02), pages 151 - 160, XP058137796, ISBN: 978-1-58113-998-3, DOI: 10.1145/1054972.1054994 *

Also Published As

Publication number Publication date
EP4384889A2 (en) 2024-06-19
AU2022352756A1 (en) 2024-04-04
KR20240047458A (en) 2024-04-12
WO2023049418A2 (en) 2023-03-30

Similar Documents

Publication Publication Date Title
Hubenschmid et al. Stream: Exploring the combination of spatially-aware tablets with augmented reality head-mounted displays for immersive analytics
WO2022067083A3 (en) Devices, methods, and graphical user interfaces for interacting with three-dimensional environments
Profita et al. Don't mind me touching my wrist: a case study of interacting with on-body technology in public
JP7168578B2 (en) Intensive rendering
EP3311249B1 (en) Three-dimensional user input
Piumsomboon et al. Empathic mixed reality: Sharing what you feel and interacting with what you see
WO2011123840A3 (en) Interacting with remote applications displayed within a virtual desktop of a tablet computing device
US11954245B2 (en) Displaying physical input devices as virtual objects
Funk et al. HoloLens is more than air Tap: natural and intuitive interaction with holograms
EP4273682A3 (en) Avatar integration with multiple applications
Hand Other faces of virtual reality
Park et al. Comparative study on the interface and interaction for manipulating 3d virtual objects in a virtual reality environment
Knierim et al. The nomadic office: A location independent workspace through mixed reality
Kim et al. Telegate: Immersive multi-user collaboration for mixed reality 360 video
Dewez et al. Do you need another hand? investigating dual body representations during anisomorphic 3d manipulation
Biener et al. Extended reality for knowledge work in everyday environments
WO2023049418A3 (en) Devices, methods, and graphical user interfaces for interacting with media and three-dimensional environments
Van Dam User interfaces: disappearing, dissolving, and evolving
Lee et al. Interaction design for tangible augmented reality applications
Oakley et al. Communicating with feeling
WO2019147699A8 (en) Devices, methods, and graphical user interfaces for system-wide behavior for 3d models
Liang et al. Workshop: Mixing realities: Cross-reality visualization, interaction, and collaboration
Flick et al. Trade-offs in augmented reality user interfaces for controlling a smart environment
Gonzalez-Franco et al. Guidelines for Productivity in Virtual Reality
Stotts et al. Support for distributed pair programming in the transparent video facetop

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22790141

Country of ref document: EP

Kind code of ref document: A2

WWE Wipo information: entry into national phase

Ref document number: 2022790141

Country of ref document: EP

WWE Wipo information: entry into national phase

Ref document number: 202280062327.4

Country of ref document: CN

WWE Wipo information: entry into national phase

Ref document number: 2022352756

Country of ref document: AU

Ref document number: AU2022352756

Country of ref document: AU

ENP Entry into the national phase

Ref document number: 20247009664

Country of ref document: KR

Kind code of ref document: A

ENP Entry into the national phase

Ref document number: 2022790141

Country of ref document: EP

Effective date: 20240311

ENP Entry into the national phase

Ref document number: 2022352756

Country of ref document: AU

Date of ref document: 20220924

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE