JP7815219B2 - 環境内のオブジェクトと相互作用する方法 - Google Patents

環境内のオブジェクトと相互作用する方法

Info

Publication number
JP7815219B2
JP7815219B2 JP2023516621A JP2023516621A JP7815219B2 JP 7815219 B2 JP7815219 B2 JP 7815219B2 JP 2023516621 A JP2023516621 A JP 2023516621A JP 2023516621 A JP2023516621 A JP 2023516621A JP 7815219 B2 JP7815219 B2 JP 7815219B2
Authority
JP
Japan
Prior art keywords
user
interface element
user interface
hand
accordance
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
JP2023516621A
Other languages
English (en)
Japanese (ja)
Other versions
JP2023541275A (ja
JPWO2022055822A5 (https=
JP2023541275A5 (https=
Inventor
イーロン エム． バーンズ，
アレクシ エイチ． パロンジェ，
ナサン ギッター，
ニコライ ゲオルク，
ベンジャミン アール． ブラクニツキー，
アルン ラケシュ ヨガナンダン，
ベンジャミン ヒラック，
アダム ジー． ポウロス，
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Apple Inc
Original Assignee
Apple Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Apple Inc filed Critical Apple Inc
Publication of JP2023541275A publication Critical patent/JP2023541275A/ja
Publication of JPWO2022055822A5 publication Critical patent/JPWO2022055822A5/ja
Publication of JP2023541275A5 publication Critical patent/JP2023541275A5/ja
Application granted granted Critical
Publication of JP7815219B2 publication Critical patent/JP7815219B2/ja
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/013—Eye tracking input arrangements
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
    • G06F3/04815—Interaction with a metaphor-based environment or interaction object displayed as three-dimensional [3D], e.g. changing the user viewpoint with respect to the environment or object
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
    • G06F3/04842—Selection of displayed objects or displayed text elements
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2203/00—Indexing scheme relating to G06F3/00 - G06F3/048
    • G06F2203/038—Indexing scheme relating to G06F3/038
    • G06F2203/0381—Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer

Landscapes

  • Engineering & Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • User Interface Of Digital Computer (AREA)
  • Position Input By Displaying (AREA)
JP2023516621A 2020-09-11 2021-09-03 環境内のオブジェクトと相互作用する方法 Active JP7815219B2 (ja)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US202063077472P 2020-09-11 2020-09-11
US63/077,472 2020-09-11
PCT/US2021/049131 WO2022055822A1 (en) 2020-09-11 2021-09-03 Method of interacting with objects in an environment

Publications (4)

Publication Number Publication Date
JP2023541275A JP2023541275A (ja) 2023-09-29
JPWO2022055822A5 JPWO2022055822A5 (https=) 2024-09-09
JP2023541275A5 JP2023541275A5 (https=) 2024-09-09
JP7815219B2 true JP7815219B2 (ja) 2026-02-17

Family

ID=77951874

Family Applications (1)

Application Number Title Priority Date Filing Date
JP2023516621A Active JP7815219B2 (ja) 2020-09-11 2021-09-03 環境内のオブジェクトと相互作用する方法

Country Status (6)

Country Link
US (1) US20230325004A1 (https=)
EP (1) EP4211542A1 (https=)
JP (1) JP7815219B2 (https=)
KR (1) KR102950671B1 (https=)
CN (1) CN116583816A (https=)
WO (1) WO2022055822A1 (https=)

Families Citing this family (54)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
USD1066508S1 (en) * 2013-03-11 2025-03-11 King Show Games, Inc. Gaming apparatus having display screen with graphical user interface
US11307647B2 (en) 2019-09-11 2022-04-19 Facebook Technologies, Llc Artificial reality triggered by physical object
US11227445B1 (en) 2020-08-31 2022-01-18 Facebook Technologies, Llc Artificial reality augments and surfaces
US11176755B1 (en) 2020-08-31 2021-11-16 Facebook Technologies, Llc Artificial reality augments and surfaces
US11520456B2 (en) 2020-09-25 2022-12-06 Apple Inc. Methods for adjusting and/or controlling immersion associated with user interfaces
EP4718404A2 (en) 2020-09-25 2026-04-01 Apple Inc. Methods for manipulating objects in an environment
WO2022067342A2 (en) 2020-09-25 2022-03-31 Apple Inc. Methods for interacting with virtual controls and/or an affordance for moving virtual objects in virtual environments
US11113893B1 (en) * 2020-11-17 2021-09-07 Facebook Technologies, Llc Artificial reality environment with glints displayed by an extra reality device
US11409405B1 (en) 2020-12-22 2022-08-09 Facebook Technologies, Llc Augment orchestration in an artificial reality environment
CN116670627A (zh) 2020-12-31 2023-08-29 苹果公司 对环境中的用户界面进行分组的方法
US11995230B2 (en) 2021-02-11 2024-05-28 Apple Inc. Methods for presenting and sharing content in an environment
KR20230169331A (ko) 2021-04-13 2023-12-15 애플 인크. 환경에 몰입형 경험을 제공하기 위한 방법
US12222214B2 (en) * 2021-05-03 2025-02-11 Chian Chiu Li Systems and methods for presenting map and changing direction based on pointing direction
US11676348B2 (en) 2021-06-02 2023-06-13 Meta Platforms Technologies, Llc Dynamic mixed reality content in virtual reality
US11762952B2 (en) 2021-06-28 2023-09-19 Meta Platforms Technologies, Llc Artificial reality application lifecycle
US11521361B1 (en) 2021-07-01 2022-12-06 Meta Platforms Technologies, Llc Environment model with surfaces and per-surface volumes
US12008717B2 (en) 2021-07-07 2024-06-11 Meta Platforms Technologies, Llc Artificial reality environment control through an artificial reality environment schema
US12056268B2 (en) 2021-08-17 2024-08-06 Meta Platforms Technologies, Llc Platformization of mixed reality objects in virtual reality environments
KR20260006060A (ko) 2021-09-25 2026-01-12 애플 인크. 가상 환경들에서 가상 객체들을 제시하기 위한 디바이스들, 방법들, 및 그래픽 사용자 인터페이스들
US11748944B2 (en) 2021-10-27 2023-09-05 Meta Platforms Technologies, Llc Virtual object structures and interrelationships
US11798247B2 (en) 2021-10-27 2023-10-24 Meta Platforms Technologies, Llc Virtual object structures and interrelationships
US12456271B1 (en) 2021-11-19 2025-10-28 Apple Inc. System and method of three-dimensional object cleanup and text annotation
EP4463754A1 (en) 2022-01-12 2024-11-20 Apple Inc. Methods for displaying, selecting and moving objects and containers in an environment
US12093447B2 (en) 2022-01-13 2024-09-17 Meta Platforms Technologies, Llc Ephemeral artificial reality experiences
JP7843357B2 (ja) 2022-01-19 2026-04-09 アップル インコーポレイテッド 環境内にオブジェクトを表示及び再位置決めするための方法
US12067688B2 (en) 2022-02-14 2024-08-20 Meta Platforms Technologies, Llc Coordination of interactions of virtual objects
US12541280B2 (en) 2022-02-28 2026-02-03 Apple Inc. System and method of three-dimensional placement and refinement in multi-user communication sessions
US12272005B2 (en) 2022-02-28 2025-04-08 Apple Inc. System and method of three-dimensional immersive applications in multi-user communication sessions
US12321666B2 (en) 2022-04-04 2025-06-03 Apple Inc. Methods for quick message response and dictation in a three-dimensional environment
CN119404170A (zh) 2022-04-20 2025-02-07 苹果公司 三维环境中的被遮蔽对象
CN119987551A (zh) 2022-04-21 2025-05-13 苹果公司 消息在三维环境中的表示
US12026527B2 (en) 2022-05-10 2024-07-02 Meta Platforms Technologies, Llc World-controlled and application-controlled augments in an artificial-reality environment
US12394167B1 (en) 2022-06-30 2025-08-19 Apple Inc. Window resizing and virtual object rearrangement in 3D environments
CN117492555A (zh) * 2022-07-25 2024-02-02 北京字跳网络技术有限公司 对象移动控制方法、装置、设备
US12112011B2 (en) 2022-09-16 2024-10-08 Apple Inc. System and method of application-based three-dimensional refinement in multi-user communication sessions
US12148078B2 (en) 2022-09-16 2024-11-19 Apple Inc. System and method of spatial groups in multi-user communication sessions
US12099653B2 (en) 2022-09-22 2024-09-24 Apple Inc. User interface response based on gaze-holding event assessment
US12405704B1 (en) 2022-09-23 2025-09-02 Apple Inc. Interpreting user movement as direct touch user interface interactions
US20240233288A1 (en) 2022-09-24 2024-07-11 Apple Inc. Methods for controlling and interacting with a three-dimensional environment
US20240103704A1 (en) 2022-09-24 2024-03-28 Apple Inc. Methods for interacting with user interfaces based on attention
WO2024064950A1 (en) 2022-09-24 2024-03-28 Apple Inc. Methods for time of day adjustments for environments and environment presentation during communication sessions
US12444152B1 (en) 2022-10-21 2025-10-14 Meta Platforms Technologies, Llc Application multitasking in a three-dimensional environment
CN120051749A (zh) * 2022-10-24 2025-05-27 索尼集团公司 信息处理装置和信息处理方法
US12547237B1 (en) 2022-10-24 2026-02-10 Meta Platforms Technologies, Llc Artificial reality input for two-dimensional virtual objects
US11947862B1 (en) 2022-12-30 2024-04-02 Meta Platforms Technologies, Llc Streaming native application content to artificial reality devices
CN120813918A (zh) 2023-01-30 2025-10-17 苹果公司 用于响应于注视和/或手势输入来显示多组控件的设备、方法和图形用户界面
US12108012B2 (en) 2023-02-27 2024-10-01 Apple Inc. System and method of managing spatial states and display modes in multi-user communication sessions
US12548245B2 (en) 2023-03-31 2026-02-10 Meta Platforms Technologies, Llc Rendering an artificial reality environment based on a defined hierarchy of multiple states including multiple artificial reality experiences with augments
US12118200B1 (en) 2023-06-02 2024-10-15 Apple Inc. Fuzzy hit testing
US12443286B2 (en) 2023-06-02 2025-10-14 Apple Inc. Input recognition based on distinguishing direct and indirect user interactions
WO2024254096A1 (en) 2023-06-04 2024-12-12 Apple Inc. Methods for managing overlapping windows and applying visual effects
US12099695B1 (en) 2023-06-04 2024-09-24 Apple Inc. Systems and methods of managing spatial groups in multi-user communication sessions
WO2024254066A1 (en) * 2023-06-05 2024-12-12 Apple Inc. Devices, methods, and graphical user interfaces for detecting inputs
US20250377719A1 (en) 2024-06-09 2025-12-11 Apple Inc. Methods of interacting with content in a virtual environment

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2019139425A (ja) 2018-02-08 2019-08-22 株式会社バンダイナムコスタジオ シミュレーションシステム及びプログラム
US20200226814A1 (en) 2019-01-11 2020-07-16 Microsoft Technology Licensing, Llc Holographic palm raycasting for targeting virtual objects
US20200225830A1 (en) 2019-01-11 2020-07-16 Microsoft Technology Licensing, Llc Near interaction mode for far virtual object
CN111566604A (zh) 2018-02-13 2020-08-21 三星电子株式会社 电子装置及其操作方法
JP2020144577A (ja) 2019-03-06 2020-09-10 日産自動車株式会社 仮想オブジェクト出力装置及び仮想オブジェクト出力方法

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20160098094A1 (en) * 2014-10-02 2016-04-07 Geegui Corporation User interface enabled by 3d reversals
EP4398566A3 (en) * 2016-08-11 2024-10-09 Magic Leap, Inc. Automatic placement of a virtual object in a three-dimensional space
US11077360B2 (en) * 2017-04-28 2021-08-03 Sony Interactive Entertainment Inc. Information processing device, control method of information processing device, and program
EP3542252B1 (en) * 2017-08-10 2023-08-02 Google LLC Context-sensitive hand interaction
EP3503101A1 (en) * 2017-12-20 2019-06-26 Nokia Technologies Oy Object based user interface
US10831265B2 (en) * 2018-04-20 2020-11-10 Microsoft Technology Licensing, Llc Systems and methods for gaze-informed target manipulation
US10712901B2 (en) * 2018-06-27 2020-07-14 Facebook Technologies, Llc Gesture-based content sharing in artificial reality environments
US11237641B2 (en) * 2020-03-27 2022-02-01 Lenovo (Singapore) Pte. Ltd. Palm based object position adjustment

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2019139425A (ja) 2018-02-08 2019-08-22 株式会社バンダイナムコスタジオ シミュレーションシステム及びプログラム
CN111566604A (zh) 2018-02-13 2020-08-21 三星电子株式会社 电子装置及其操作方法
US20200226814A1 (en) 2019-01-11 2020-07-16 Microsoft Technology Licensing, Llc Holographic palm raycasting for targeting virtual objects
US20200225830A1 (en) 2019-01-11 2020-07-16 Microsoft Technology Licensing, Llc Near interaction mode for far virtual object
JP2020144577A (ja) 2019-03-06 2020-09-10 日産自動車株式会社 仮想オブジェクト出力装置及び仮想オブジェクト出力方法

Also Published As

Publication number Publication date
WO2022055822A1 (en) 2022-03-17
JP2023541275A (ja) 2023-09-29
CN116583816A (zh) 2023-08-11
KR20230118070A (ko) 2023-08-10
KR102950671B1 (ko) 2026-04-09
US20230325004A1 (en) 2023-10-12
EP4211542A1 (en) 2023-07-19

Similar Documents

Publication Publication Date Title
JP7815219B2 (ja) 環境内のオブジェクトと相互作用する方法
US11995301B2 (en) Method of displaying user interfaces in an environment and corresponding electronic device and computer readable storage medium
US12141914B2 (en) Techniques for manipulating computer graphical light sources
US12141423B2 (en) Techniques for manipulating computer graphical objects
US12566532B2 (en) Methods for interacting with virtual controls and/or an affordance for moving virtual objects in virtual environments
US12455670B2 (en) Devices, methods, and graphical user interfaces for interacting with three-dimensional environments
US12307580B2 (en) Methods for manipulating objects in an environment
US20230185426A1 (en) Devices, Methods, and Graphical User Interfaces for Providing Computer-Generated Experiences
US12236546B1 (en) Object manipulations with a pointing device
KR20230017746A (ko) 객체들의 3차원 미리보기를 위한 디바이스들, 방법들 및 그래픽 사용자 인터페이스들
CN116888571A (zh) 在环境中操纵用户界面的方法
US12456271B1 (en) System and method of three-dimensional object cleanup and text annotation
US12411554B2 (en) Method of displaying selectable options
US20240104841A1 (en) Devices and methods for generating virtual objects
KR20260053086A (ko) 환경에서 객체들과 상호작용하는 방법

Legal Events

Date Code Title Description
A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20230517

A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20240830

A621 Written request for application examination

Free format text: JAPANESE INTERMEDIATE CODE: A621

Effective date: 20240830

A131 Notification of reasons for refusal

Free format text: JAPANESE INTERMEDIATE CODE: A131

Effective date: 20250801

A977 Report on retrieval

Free format text: JAPANESE INTERMEDIATE CODE: A971007

Effective date: 20250731

TRDD Decision of grant or rejection written
A01 Written decision to grant a patent or to grant a registration (utility model)

Free format text: JAPANESE INTERMEDIATE CODE: A01

Effective date: 20260109

A61 First payment of annual fees (during grant procedure)

Free format text: JAPANESE INTERMEDIATE CODE: A61

Effective date: 20260204

R150 Certificate of patent or registration of utility model

Ref document number: 7815219

Country of ref document: JP

Free format text: JAPANESE INTERMEDIATE CODE: R150