KR102950671B1 - 환경에서 객체들과 상호작용하는 방법 - Google Patents

환경에서 객체들과 상호작용하는 방법

Info

Publication number
KR102950671B1
KR102950671B1 KR1020237012296A KR20237012296A KR102950671B1 KR 102950671 B1 KR102950671 B1 KR 102950671B1 KR 1020237012296 A KR1020237012296 A KR 1020237012296A KR 20237012296 A KR20237012296 A KR 20237012296A KR 102950671 B1 KR102950671 B1 KR 102950671B1
Authority
KR
South Korea
Prior art keywords
user
hand
interface element
user interface
electronic device
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
KR1020237012296A
Other languages
English (en)
Korean (ko)
Other versions
KR20230118070A (ko
Inventor
아론 엠. 번스
알렉시스 에이치. 팔랑지
네이슨 기터
니콜라이 조르그
벤자민 알. 블라히니츠키
아룬 라케시 요가난단
벤자민 하일락
아담 지. 폴로스
Original Assignee
애플 인크.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 애플 인크. filed Critical 애플 인크.
Priority to KR1020267010589A priority Critical patent/KR20260053086A/ko
Publication of KR20230118070A publication Critical patent/KR20230118070A/ko
Application granted granted Critical
Publication of KR102950671B1 publication Critical patent/KR102950671B1/ko
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/013—Eye tracking input arrangements
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
    • G06F3/04815—Interaction with a metaphor-based environment or interaction object displayed as three-dimensional [3D], e.g. changing the user viewpoint with respect to the environment or object
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
    • G06F3/04842—Selection of displayed objects or displayed text elements
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2203/00—Indexing scheme relating to G06F3/00 - G06F3/048
    • G06F2203/038—Indexing scheme relating to G06F3/038
    • G06F2203/0381—Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer

Landscapes

  • Engineering & Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • User Interface Of Digital Computer (AREA)
  • Position Input By Displaying (AREA)
KR1020237012296A 2020-09-11 2021-09-03 환경에서 객체들과 상호작용하는 방법 Active KR102950671B1 (ko)

Priority Applications (1)

Application Number Priority Date Filing Date Title
KR1020267010589A KR20260053086A (ko) 2020-09-11 2021-09-03 환경에서 객체들과 상호작용하는 방법

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US202063077472P 2020-09-11 2020-09-11
US63/077,472 2020-09-11
PCT/US2021/049131 WO2022055822A1 (en) 2020-09-11 2021-09-03 Method of interacting with objects in an environment

Related Child Applications (1)

Application Number Title Priority Date Filing Date
KR1020267010589A Division KR20260053086A (ko) 2020-09-11 2021-09-03 환경에서 객체들과 상호작용하는 방법

Publications (2)

Publication Number Publication Date
KR20230118070A KR20230118070A (ko) 2023-08-10
KR102950671B1 true KR102950671B1 (ko) 2026-04-09

Family

ID=77951874

Family Applications (2)

Application Number Title Priority Date Filing Date
KR1020237012296A Active KR102950671B1 (ko) 2020-09-11 2021-09-03 환경에서 객체들과 상호작용하는 방법
KR1020267010589A Pending KR20260053086A (ko) 2020-09-11 2021-09-03 환경에서 객체들과 상호작용하는 방법

Family Applications After (1)

Application Number Title Priority Date Filing Date
KR1020267010589A Pending KR20260053086A (ko) 2020-09-11 2021-09-03 환경에서 객체들과 상호작용하는 방법

Country Status (6)

Country Link
US (1) US20230325004A1 (enExample)
EP (1) EP4211542A1 (enExample)
JP (2) JP7815219B2 (enExample)
KR (2) KR102950671B1 (enExample)
CN (1) CN116583816A (enExample)
WO (1) WO2022055822A1 (enExample)

Families Citing this family (61)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
USD1066508S1 (en) * 2013-03-11 2025-03-11 King Show Games, Inc. Gaming apparatus having display screen with graphical user interface
DK180639B1 (en) 2018-06-01 2021-11-04 Apple Inc DISABILITY OF ATTENTION-ATTENTIVE VIRTUAL ASSISTANT
US11307647B2 (en) 2019-09-11 2022-04-19 Facebook Technologies, Llc Artificial reality triggered by physical object
EP4495750A3 (en) 2020-03-27 2025-03-26 Apple Inc. Devices, methods, and graphical user interfaces for gaze-based navigation
US11176755B1 (en) 2020-08-31 2021-11-16 Facebook Technologies, Llc Artificial reality augments and surfaces
US11227445B1 (en) 2020-08-31 2022-01-18 Facebook Technologies, Llc Artificial reality augments and surfaces
KR102833872B1 (ko) 2020-09-25 2025-07-15 애플 인크. 환경에서 객체들을 조작하기 위한 방법들
WO2022067342A2 (en) 2020-09-25 2022-03-31 Apple Inc. Methods for interacting with virtual controls and/or an affordance for moving virtual objects in virtual environments
KR102959016B1 (ko) 2020-09-25 2026-04-28 애플 인크. 사용자 인터페이스와 연관된 몰입을 조정 및/또는 제어하기 위한 방법
US11113893B1 (en) * 2020-11-17 2021-09-07 Facebook Technologies, Llc Artificial reality environment with glints displayed by an extra reality device
US11409405B1 (en) 2020-12-22 2022-08-09 Facebook Technologies, Llc Augment orchestration in an artificial reality environment
CN116670627A (zh) 2020-12-31 2023-08-29 苹果公司 对环境中的用户界面进行分组的方法
US11995230B2 (en) 2021-02-11 2024-05-28 Apple Inc. Methods for presenting and sharing content in an environment
KR20250127178A (ko) 2021-04-13 2025-08-26 애플 인크. 환경에 몰입형 경험을 제공하기 위한 방법
US12222214B2 (en) * 2021-05-03 2025-02-11 Chian Chiu Li Systems and methods for presenting map and changing direction based on pointing direction
US11676348B2 (en) 2021-06-02 2023-06-13 Meta Platforms Technologies, Llc Dynamic mixed reality content in virtual reality
US11762952B2 (en) 2021-06-28 2023-09-19 Meta Platforms Technologies, Llc Artificial reality application lifecycle
US11521361B1 (en) 2021-07-01 2022-12-06 Meta Platforms Technologies, Llc Environment model with surfaces and per-surface volumes
US12008717B2 (en) 2021-07-07 2024-06-11 Meta Platforms Technologies, Llc Artificial reality environment control through an artificial reality environment schema
US12056268B2 (en) 2021-08-17 2024-08-06 Meta Platforms Technologies, Llc Platformization of mixed reality objects in virtual reality environments
EP4392853A1 (en) 2021-08-27 2024-07-03 Apple Inc. System and method of augmented representation of an electronic device
KR102903963B1 (ko) 2021-09-25 2025-12-26 애플 인크. 가상 환경들에서 가상 객체들을 제시하기 위한 디바이스들, 방법들, 및 그래픽 사용자 인터페이스들
US11798247B2 (en) 2021-10-27 2023-10-24 Meta Platforms Technologies, Llc Virtual object structures and interrelationships
US11748944B2 (en) 2021-10-27 2023-09-05 Meta Platforms Technologies, Llc Virtual object structures and interrelationships
US12456271B1 (en) 2021-11-19 2025-10-28 Apple Inc. System and method of three-dimensional object cleanup and text annotation
CN118871875A (zh) 2022-01-12 2024-10-29 苹果公司 用于在环境中显示、选择以及移动对象和容器的方法
US12093447B2 (en) 2022-01-13 2024-09-17 Meta Platforms Technologies, Llc Ephemeral artificial reality experiences
EP4466593A1 (en) 2022-01-19 2024-11-27 Apple Inc. Methods for displaying and repositioning objects in an environment
US12067688B2 (en) 2022-02-14 2024-08-20 Meta Platforms Technologies, Llc Coordination of interactions of virtual objects
US12272005B2 (en) 2022-02-28 2025-04-08 Apple Inc. System and method of three-dimensional immersive applications in multi-user communication sessions
US12541280B2 (en) 2022-02-28 2026-02-03 Apple Inc. System and method of three-dimensional placement and refinement in multi-user communication sessions
WO2023196258A1 (en) 2022-04-04 2023-10-12 Apple Inc. Methods for quick message response and dictation in a three-dimensional environment
WO2023205362A1 (en) 2022-04-20 2023-10-26 Apple Inc. Obstructed objects in a three-dimensional environment
EP4511723A1 (en) 2022-04-21 2025-02-26 Apple Inc. Representations of messages in a three-dimensional environment
US12026527B2 (en) 2022-05-10 2024-07-02 Meta Platforms Technologies, Llc World-controlled and application-controlled augments in an artificial-reality environment
US12394167B1 (en) 2022-06-30 2025-08-19 Apple Inc. Window resizing and virtual object rearrangement in 3D environments
CN117492555A (zh) * 2022-07-25 2024-02-02 北京字跳网络技术有限公司 对象移动控制方法、装置、设备
JP2025534239A (ja) 2022-09-14 2025-10-15 アップル インコーポレイテッド 三次元環境内の深度競合軽減のための方法
US12112011B2 (en) 2022-09-16 2024-10-08 Apple Inc. System and method of application-based three-dimensional refinement in multi-user communication sessions
US12148078B2 (en) 2022-09-16 2024-11-19 Apple Inc. System and method of spatial groups in multi-user communication sessions
US12099653B2 (en) 2022-09-22 2024-09-24 Apple Inc. User interface response based on gaze-holding event assessment
CN120469584A (zh) 2022-09-23 2025-08-12 苹果公司 用于操纵虚拟对象的方法
US12405704B1 (en) 2022-09-23 2025-09-02 Apple Inc. Interpreting user movement as direct touch user interface interactions
US20240104836A1 (en) 2022-09-24 2024-03-28 Apple Inc. Methods for time of day adjustments for environments and environment presentation during communication sessions
US12535931B2 (en) 2022-09-24 2026-01-27 Apple Inc. Methods for controlling and interacting with a three-dimensional environment
CN120266083A (zh) 2022-09-24 2025-07-04 苹果公司 用于基于注意力与用户界面交互的方法
US12444152B1 (en) 2022-10-21 2025-10-14 Meta Platforms Technologies, Llc Application multitasking in a three-dimensional environment
US12547237B1 (en) 2022-10-24 2026-02-10 Meta Platforms Technologies, Llc Artificial reality input for two-dimensional virtual objects
KR20250095637A (ko) * 2022-10-24 2025-06-26 소니그룹주식회사 정보 처리 장치 및 정보 처리 방법
US11947862B1 (en) 2022-12-30 2024-04-02 Meta Platforms Technologies, Llc Streaming native application content to artificial reality devices
CN121165940A (zh) 2023-01-30 2025-12-19 苹果公司 用于响应于注视和/或手势输入来显示多组控件的设备、方法和图形用户界面
US12108012B2 (en) 2023-02-27 2024-10-01 Apple Inc. System and method of managing spatial states and display modes in multi-user communication sessions
US12548245B2 (en) 2023-03-31 2026-02-10 Meta Platforms Technologies, Llc Rendering an artificial reality environment based on a defined hierarchy of multiple states including multiple artificial reality experiences with augments
US12657958B2 (en) 2023-03-31 2026-06-16 Apple Inc. Pinch recognition and rejection
WO2024226681A1 (en) 2023-04-25 2024-10-31 Apple Inc. Methods for displaying and rearranging objects in an environment
US12118200B1 (en) 2023-06-02 2024-10-15 Apple Inc. Fuzzy hit testing
US12443286B2 (en) 2023-06-02 2025-10-14 Apple Inc. Input recognition based on distinguishing direct and indirect user interactions
US20250078420A1 (en) 2023-06-04 2025-03-06 Apple Inc. Methods for managing overlapping windows and applying visual effects
US12099695B1 (en) 2023-06-04 2024-09-24 Apple Inc. Systems and methods of managing spatial groups in multi-user communication sessions
WO2024254066A1 (en) * 2023-06-05 2024-12-12 Apple Inc. Devices, methods, and graphical user interfaces for detecting inputs
US20250378645A1 (en) 2024-06-09 2025-12-11 Apple Inc. Methods of interacting with content in a virtual environment

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190050062A1 (en) * 2017-08-10 2019-02-14 Google Llc Context-sensitive hand interaction
US20200226814A1 (en) * 2019-01-11 2020-07-16 Microsoft Technology Licensing, Llc Holographic palm raycasting for targeting virtual objects

Family Cites Families (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20160098094A1 (en) * 2014-10-02 2016-04-07 Geegui Corporation User interface enabled by 3d reversals
IL264690B (en) * 2016-08-11 2022-06-01 Magic Leap Inc Automatic positioning of a virtual object in 3D space
JP6744990B2 (ja) * 2017-04-28 2020-08-19 株式会社ソニー・インタラクティブエンタテインメント 情報処理装置、情報処理装置の制御方法、及びプログラム
EP3503101A1 (en) * 2017-12-20 2019-06-26 Nokia Technologies Oy Object based user interface
JP7041888B2 (ja) 2018-02-08 2022-03-25 株式会社バンダイナムコ研究所 シミュレーションシステム及びプログラム
KR102469754B1 (ko) 2018-02-13 2022-11-22 삼성전자주식회사 전자 장치 및 그 동작 방법
US10831265B2 (en) * 2018-04-20 2020-11-10 Microsoft Technology Licensing, Llc Systems and methods for gaze-informed target manipulation
US10712901B2 (en) * 2018-06-27 2020-07-14 Facebook Technologies, Llc Gesture-based content sharing in artificial reality environments
US11320957B2 (en) 2019-01-11 2022-05-03 Microsoft Technology Licensing, Llc Near interaction mode for far virtual object
JP7165604B2 (ja) 2019-03-06 2022-11-04 日産自動車株式会社 仮想オブジェクト出力装置及び仮想オブジェクト出力方法
US11237641B2 (en) * 2020-03-27 2022-02-01 Lenovo (Singapore) Pte. Ltd. Palm based object position adjustment

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190050062A1 (en) * 2017-08-10 2019-02-14 Google Llc Context-sensitive hand interaction
US20200226814A1 (en) * 2019-01-11 2020-07-16 Microsoft Technology Licensing, Llc Holographic palm raycasting for targeting virtual objects

Also Published As

Publication number Publication date
CN116583816A (zh) 2023-08-11
JP2026102534A (ja) 2026-06-23
EP4211542A1 (en) 2023-07-19
KR20230118070A (ko) 2023-08-10
JP7815219B2 (ja) 2026-02-17
US20230325004A1 (en) 2023-10-12
WO2022055822A1 (en) 2022-03-17
JP2023541275A (ja) 2023-09-29
KR20260053086A (ko) 2026-04-20

Similar Documents

Publication Publication Date Title
JP7815219B2 (ja) 環境内のオブジェクトと相互作用する方法
US12141914B2 (en) Techniques for manipulating computer graphical light sources
US12141423B2 (en) Techniques for manipulating computer graphical objects
US12236546B1 (en) Object manipulations with a pointing device
US11995301B2 (en) Method of displaying user interfaces in an environment and corresponding electronic device and computer readable storage medium
KR102782160B1 (ko) 객체들의 3차원 미리보기를 위한 디바이스들, 방법들 및 그래픽 사용자 인터페이스들
CN115700433B (zh) 用于交互式三维预览的系统和方法
US20250104335A1 (en) Systems and methods of layout and presentation for creative workflows
US20250104367A1 (en) Systems and methods of layout and presentation for creative workflows
US11023035B1 (en) Virtual pinboard interaction using a peripheral device in artificial reality environments
US10976804B1 (en) Pointer-based interaction with a virtual surface using a peripheral device in artificial reality environments
US11714540B2 (en) Remote touch detection enabled by peripheral device
KR20230117639A (ko) 사용자 인터페이스와 연관된 몰입을 조정 및/또는 제어하기위한 방법
CN117882034A (zh) 显示和操纵用户界面元素
US11023036B1 (en) Virtual drawing surface interaction using a peripheral device in artificial reality environments
CN112041788A (zh) 使用眼睛注视来选择文本输入字段
US12411554B2 (en) Method of displaying selectable options
US20230095282A1 (en) Method And Device For Faciliating Interactions With A Peripheral Device
CN119722916A (zh) 增强模拟视觉
US20260094352A1 (en) Systems and methods of rendering techniques for virtual stages and scenes
US20240045579A1 (en) Systems and methods for launching and replacing applications

Legal Events

Date Code Title Description
PA0105 International application

St.27 status event code: A-0-1-A10-A15-nap-PA0105

T11-X000 Administrative time limit extension requested

St.27 status event code: U-3-3-T10-T11-oth-X000

A201 Request for examination
E13-X000 Pre-grant limitation requested

St.27 status event code: A-2-3-E10-E13-lim-X000

P11-X000 Amendment of application requested

St.27 status event code: A-2-2-P10-P11-nap-X000

P13-X000 Application amended

St.27 status event code: A-2-2-P10-P13-nap-X000

PA0201 Request for examination

St.27 status event code: A-1-2-D10-D11-exm-PA0201

PG1501 Laying open of application

St.27 status event code: A-1-1-Q10-Q12-nap-PG1501

E902 Notification of reason for refusal
PE0902 Notice of grounds for rejection

St.27 status event code: A-1-2-D10-D21-exm-PE0902

D22 Grant of ip right intended

Free format text: ST27 STATUS EVENT CODE: A-1-2-D10-D22-EXM-PE0701 (AS PROVIDED BY THE NATIONAL OFFICE)

PE0701 Decision of registration

St.27 status event code: A-1-2-D10-D22-exm-PE0701

A16 Divisional, continuation or continuation in part application filed

Free format text: ST27 STATUS EVENT CODE: A-0-1-A10-A16-DIV-PA0104 (AS PROVIDED BY THE NATIONAL OFFICE)

F11 Ip right granted following substantive examination

Free format text: ST27 STATUS EVENT CODE: A-2-4-F10-F11-EXM-PR0701 (AS PROVIDED BY THE NATIONAL OFFICE)

PA0104 Divisional application for international application

St.27 status event code: A-0-1-A10-A16-div-PA0104

PR0701 Registration of establishment

St.27 status event code: A-2-4-F10-F11-exm-PR0701

PR1002 Payment of registration fee

St.27 status event code: A-2-2-U10-U12-oth-PR1002

Fee payment year number: 1

U12 Designation fee paid

Free format text: ST27 STATUS EVENT CODE: A-2-2-U10-U12-OTH-PR1002 (AS PROVIDED BY THE NATIONAL OFFICE)

Year of fee payment: 1

PG1601 Publication of registration

St.27 status event code: A-4-4-Q10-Q13-nap-PG1601

Q13 Ip right document published

Free format text: ST27 STATUS EVENT CODE: A-4-4-Q10-Q13-NAP-PG1601 (AS PROVIDED BY THE NATIONAL OFFICE)