CN118159935A - 用于内容应用程序的设备、方法和图形用户界面 - Google Patents

用于内容应用程序的设备、方法和图形用户界面 Download PDF

Info

Publication number
CN118159935A
CN118159935A CN202280071068.1A CN202280071068A CN118159935A CN 118159935 A CN118159935 A CN 118159935A CN 202280071068 A CN202280071068 A CN 202280071068A CN 118159935 A CN118159935 A CN 118159935A
Authority
CN
China
Prior art keywords
user interface
content item
user
dimensional environment
viewpoint
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202280071068.1A
Other languages
English (en)
Chinese (zh)
Inventor
M·斯陶贝尔
I·帕斯特拉纳文森特
J·R·达斯科拉
S·O·勒梅
A·拉博利
I·阿罗拉
E·克日沃卢奇科
A·S·金
G·T·斯科特
L·莫雷诺鲁福
F·R·维纳
B·K·白石
S·田中
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Apple Inc
Original Assignee
Apple Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Apple Inc filed Critical Apple Inc
Priority to CN202411673128.4A priority Critical patent/CN119625154A/zh
Publication of CN118159935A publication Critical patent/CN118159935A/zh
Pending legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T15/00Three-dimensional [3D] image rendering
    • G06T15/50Lighting effects
    • G06T15/506Illumination models
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/013Eye tracking input arrangements
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/017Gesture based interaction, e.g. based on a set of recognized hand gestures
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0481Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
    • G06F3/04815Interaction with a metaphor-based environment or interaction object displayed as three-dimensional [3D], e.g. changing the user viewpoint with respect to the environment or object
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0481Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
    • G06F3/0482Interaction with lists of selectable items, e.g. menus
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0484Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
    • G06F3/04845Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range for image manipulation, e.g. dragging, rotation, expansion or change of colour
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0484Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
    • G06F3/04847Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/16Sound input; Sound output
    • G06F3/165Management of the audio stream, e.g. setting of volume, audio stream path
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T15/00Three-dimensional [3D] image rendering
    • G06T15/50Lighting effects
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00Manipulating three-dimensional [3D] models or images for computer graphics
    • G06T19/006Mixed reality
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00Manipulating three-dimensional [3D] models or images for computer graphics
    • G06T19/20Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/431Generation of visual interfaces for content selection or interaction; Content or additional data rendering
    • H04N21/4312Generation of visual interfaces for content selection or interaction; Content or additional data rendering involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F2203/00Indexing scheme relating to G06F3/00 - G06F3/048
    • G06F2203/038Indexing scheme relating to G06F3/038
    • G06F2203/0381Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F2203/00Indexing scheme relating to G06F3/00 - G06F3/048
    • G06F2203/048Indexing scheme relating to G06F3/048
    • G06F2203/048023D-info-object: information is displayed on the internal or external surface of a three dimensional manipulable object, e.g. on the faces of a cube that can be rotated by the user
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2200/00Indexing scheme for image data processing or generation, in general
    • G06T2200/24Indexing scheme for image data processing or generation, in general involving graphical user interfaces [GUIs]
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2219/00Indexing scheme for manipulating 3D models or images for computer graphics
    • G06T2219/20Indexing scheme for editing of 3D models
    • G06T2219/2012Colour editing, changing, or manipulating; Use of colour codes

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Computer Graphics (AREA)
  • Multimedia (AREA)
  • Computer Hardware Design (AREA)
  • Software Systems (AREA)
  • Architecture (AREA)
  • Signal Processing (AREA)
  • General Health & Medical Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Health & Medical Sciences (AREA)
  • User Interface Of Digital Computer (AREA)
  • Processing Or Creating Images (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
CN202280071068.1A 2021-09-23 2022-09-16 用于内容应用程序的设备、方法和图形用户界面 Pending CN118159935A (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202411673128.4A CN119625154A (zh) 2021-09-23 2022-09-16 用于内容应用程序的设备、方法和图形用户界面

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US202163261564P 2021-09-23 2021-09-23
US63/261,564 2021-09-23
PCT/US2022/076608 WO2023049671A1 (en) 2021-09-23 2022-09-16 Devices, methods, and graphical user interfaces for content applications

Related Child Applications (1)

Application Number Title Priority Date Filing Date
CN202411673128.4A Division CN119625154A (zh) 2021-09-23 2022-09-16 用于内容应用程序的设备、方法和图形用户界面

Publications (1)

Publication Number Publication Date
CN118159935A true CN118159935A (zh) 2024-06-07

Family

ID=83689679

Family Applications (2)

Application Number Title Priority Date Filing Date
CN202280071068.1A Pending CN118159935A (zh) 2021-09-23 2022-09-16 用于内容应用程序的设备、方法和图形用户界面
CN202411673128.4A Pending CN119625154A (zh) 2021-09-23 2022-09-16 用于内容应用程序的设备、方法和图形用户界面

Family Applications After (1)

Application Number Title Priority Date Filing Date
CN202411673128.4A Pending CN119625154A (zh) 2021-09-23 2022-09-16 用于内容应用程序的设备、方法和图形用户界面

Country Status (7)

Country Link
US (2) US20230094522A1 (enExample)
EP (1) EP4405797A1 (enExample)
JP (1) JP7799819B2 (enExample)
KR (1) KR20240065152A (enExample)
CN (2) CN118159935A (enExample)
AU (2) AU2022348895B2 (enExample)
WO (1) WO2023049671A1 (enExample)

Families Citing this family (54)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8977255B2 (en) 2007-04-03 2015-03-10 Apple Inc. Method and system for operating a multi-function portable electronic device using voice-activation
US8676904B2 (en) 2008-10-02 2014-03-18 Apple Inc. Electronic devices with voice command and contextual data processing capabilities
US9338493B2 (en) 2014-06-30 2016-05-10 Apple Inc. Intelligent automated assistant for TV user interactions
US10460227B2 (en) 2015-05-15 2019-10-29 Apple Inc. Virtual assistant in a communication session
US10331312B2 (en) 2015-09-08 2019-06-25 Apple Inc. Intelligent automated assistant in a media environment
US10747498B2 (en) 2015-09-08 2020-08-18 Apple Inc. Zero latency digital assistant
US10983688B2 (en) * 2016-06-12 2021-04-20 Apple Inc. Content scrubber bar with real-world time indications
DK180639B1 (en) 2018-06-01 2021-11-04 Apple Inc DISABILITY OF ATTENTION-ATTENTIVE VIRTUAL ASSISTANT
US10976989B2 (en) * 2018-09-26 2021-04-13 Apple Inc. Spatial management of audio
US11100349B2 (en) 2018-09-28 2021-08-24 Apple Inc. Audio assisted enrollment
US11462215B2 (en) 2018-09-28 2022-10-04 Apple Inc. Multi-modal inputs for voice commands
US12301635B2 (en) 2020-05-11 2025-05-13 Apple Inc. Digital assistant hardware abstraction
WO2022055821A1 (en) 2020-09-11 2022-03-17 Sterling Labs Llc Method of displaying user interfaces in an environment and corresponding electronic device and computer readable storage medium
EP4697149A3 (en) 2020-09-25 2026-03-11 Apple Inc. Methods for adjusting and/or controlling immersion associated with user interfaces
JP7784422B2 (ja) 2020-09-25 2025-12-11 アップル インコーポレイテッド ユーザインタフェースをナビゲートする方法
AU2021349381B2 (en) 2020-09-25 2024-02-22 Apple Inc. Methods for interacting with virtual controls and/or an affordance for moving virtual objects in virtual environments
US11995230B2 (en) 2021-02-11 2024-05-28 Apple Inc. Methods for presenting and sharing content in an environment
CN119960596A (zh) 2021-09-23 2025-05-09 苹果公司 用于内容应用的设备、方法和图形用户界面
US12456271B1 (en) 2021-11-19 2025-10-28 Apple Inc. System and method of three-dimensional object cleanup and text annotation
WO2023122799A1 (en) 2021-12-23 2023-06-29 Apple Inc. Methods for sharing content and interacting with physical devices in a three-dimensional environment
CN119556830A (zh) 2022-01-12 2025-03-04 苹果公司 用于在环境中显示、选择以及移动对象和容器的方法
JP7843357B2 (ja) 2022-01-19 2026-04-09 アップル インコーポレイテッド 環境内にオブジェクトを表示及び再位置決めするための方法
US12541280B2 (en) 2022-02-28 2026-02-03 Apple Inc. System and method of three-dimensional placement and refinement in multi-user communication sessions
US12272005B2 (en) 2022-02-28 2025-04-08 Apple Inc. System and method of three-dimensional immersive applications in multi-user communication sessions
EP4511722A1 (en) 2022-04-20 2025-02-26 Apple Inc. Obstructed objects in a three-dimensional environment
EP4390631A4 (en) * 2022-05-20 2025-05-21 Honor Device Co., Ltd. Method and device for opening pending notifications
US12340631B2 (en) 2022-06-05 2025-06-24 Apple Inc. Providing personalized audio
US20230409179A1 (en) 2022-06-15 2023-12-21 Apple Inc. Home automation device control and designation
US12112011B2 (en) 2022-09-16 2024-10-08 Apple Inc. System and method of application-based three-dimensional refinement in multi-user communication sessions
US12602865B2 (en) 2022-09-23 2026-04-14 Apple Inc. Methods for depth conflict mitigation in a three-dimensional environment
US12524956B2 (en) 2022-09-24 2026-01-13 Apple Inc. Methods for time of day adjustments for environments and environment presentation during communication sessions
CN120266083A (zh) 2022-09-24 2025-07-04 苹果公司 用于基于注意力与用户界面交互的方法
KR20250075620A (ko) 2022-09-24 2025-05-28 애플 인크. 3차원 환경을 제어하고 그와 상호작용하기 위한 방법들
US12279000B2 (en) * 2022-10-14 2025-04-15 Adeia Guides Inc. Systems and methods for personalized spatial video/light field content delivery
US20240143257A1 (en) * 2022-10-27 2024-05-02 Hewlett-Packard Development Company, L.P. Hybrid structure displays
CN121165940A (zh) 2023-01-30 2025-12-19 苹果公司 用于响应于注视和/或手势输入来显示多组控件的设备、方法和图形用户界面
US12321515B2 (en) 2023-04-25 2025-06-03 Apple Inc. System and method of representations of user interfaces of an electronic device
EP4455853A1 (en) 2023-04-25 2024-10-30 Apple Inc. System and method of representations of user interfaces of an electronic device
USD1090586S1 (en) * 2023-05-18 2025-08-26 Panasonic Holdings Corporation Display screen or portion thereof with graphical user interface
WO2024253973A1 (en) * 2023-06-03 2024-12-12 Apple Inc. Devices, methods, and graphical user interfaces for content applications
EP4720831A1 (en) * 2023-06-04 2026-04-08 Apple Inc. Devices and methods for presenting system user interfaces in an extended reality environment
CN121187445A (zh) 2023-06-04 2025-12-23 苹果公司 用于管理重叠窗口和应用视觉效果的方法
US20240428539A1 (en) * 2023-06-04 2024-12-26 Apple Inc. Devices, Methods, and Graphical User Interfaces for Selectively Accessing System Functions and Adjusting Settings of Computer Systems While Interacting with Three-Dimensional Environments
WO2024253835A1 (en) * 2023-06-05 2024-12-12 Apple Inc. Techniques for managing computer-generated experiences
US12373027B2 (en) * 2023-06-30 2025-07-29 Amazon Technologies, Inc. Gaze initiated actions
US11948234B1 (en) * 2023-08-30 2024-04-02 Illuscio, Inc. Systems and methods for dynamic enhancement of point cloud animations
US12282784B1 (en) * 2023-10-13 2025-04-22 Roku, Inc. Media device user interface and content personalization using natural language prompts
US12321585B2 (en) * 2023-11-06 2025-06-03 Huawei Technologies Co., Ltd. Methods and devices for interactions using combination gaze and touch inputs
USD1098183S1 (en) * 2024-02-01 2025-10-14 Apple Inc. Display screen or portion thereof with graphical user interface
USD1099133S1 (en) * 2024-02-01 2025-10-21 Apple Inc. Display screen or portion thereof showing a graphical user interface
USD1100929S1 (en) * 2024-02-02 2025-11-04 Apple Inc. Display screen or portion thereof showing a graphical user interface
WO2025216730A1 (en) * 2024-04-09 2025-10-16 President And Fellows Of Harvard College Augmenting sports videos with gaze-moderated embedded visualizations
WO2025259409A1 (en) * 2024-06-09 2025-12-18 Apple Inc. Methods of interacting with content in a virtual environment
US20250378645A1 (en) * 2024-06-09 2025-12-11 Apple Inc. Methods of interacting with content in a virtual environment

Family Cites Families (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
AU2001233019A1 (en) * 2000-01-28 2001-08-07 Intersense, Inc. Self-referenced tracking
WO2014066558A2 (en) * 2012-10-23 2014-05-01 Roam Holdings, LLC Three-dimensional virtual environment
EP4206870A1 (en) * 2014-06-14 2023-07-05 Magic Leap, Inc. Method for updating a virtual world
US10593113B2 (en) * 2014-07-08 2020-03-17 Samsung Electronics Co., Ltd. Device and method to display object with visual effect
US20160227267A1 (en) * 2015-01-30 2016-08-04 The Directv Group, Inc. Method and system for viewing set top box content in a virtual reality device
US10191541B2 (en) * 2016-06-30 2019-01-29 Sony Interactive Entertainment Inc. Augmenting virtual reality content with real world content
US10602133B2 (en) * 2016-10-04 2020-03-24 Facebook, Inc. Controls and interfaces for user interactions in virtual spaces
US20180095635A1 (en) * 2016-10-04 2018-04-05 Facebook, Inc. Controls and Interfaces for User Interactions in Virtual Spaces
WO2019226691A1 (en) * 2018-05-22 2019-11-28 Magic Leap, Inc. Transmodal input fusion for a wearable system
JP7136931B2 (ja) * 2018-06-05 2022-09-13 マジック リープ, インコーポレイテッド 空間3d環境へのコンテンツのマッチング
US11030796B2 (en) * 2018-10-17 2021-06-08 Adobe Inc. Interfaces and techniques to retarget 2D screencast videos into 3D tutorials in virtual reality
US10852814B1 (en) * 2019-08-13 2020-12-01 Microsoft Technology Licensing, Llc Bounding virtual object
US12175010B2 (en) * 2019-09-28 2024-12-24 Apple Inc. Devices, methods, and graphical user interfaces for interacting with three-dimensional environments
WO2021076122A1 (en) * 2019-10-16 2021-04-22 Google Llc Lighting estimation for augmented reality
CN111580652B (zh) * 2020-05-06 2024-01-16 Oppo广东移动通信有限公司 视频播放的控制方法、装置、增强现实设备及存储介质
US11343420B1 (en) * 2021-03-30 2022-05-24 Tectus Corporation Systems and methods for eye-based external camera selection and control

Also Published As

Publication number Publication date
AU2026201090A1 (en) 2026-03-05
US20230094522A1 (en) 2023-03-30
KR20240065152A (ko) 2024-05-14
WO2023049671A1 (en) 2023-03-30
JP2024538556A (ja) 2024-10-23
US20250130707A1 (en) 2025-04-24
AU2022348895A1 (en) 2024-04-11
JP7799819B2 (ja) 2026-01-15
AU2022348895B2 (en) 2025-11-13
EP4405797A1 (en) 2024-07-31
CN119625154A (zh) 2025-03-14

Similar Documents

Publication Publication Date Title
AU2022348895B2 (en) Devices, methods, and graphical user interfaces for content applications
US12455670B2 (en) Devices, methods, and graphical user interfaces for interacting with three-dimensional environments
US12124673B2 (en) Devices, methods, and graphical user interfaces for content applications
US12506953B2 (en) Device, methods, and graphical user interfaces for capturing and displaying media
CN115167676A (zh) 用于在三维环境中显示应用程序的设备和方法
CN119396287A (zh) 用于在虚拟环境中呈现虚拟对象的设备、方法和图形用户界面
CN119200818A (zh) 用于与三维环境交互的设备、方法和图形用户界面
CN118963605A (zh) 用于捕获和显示媒体的设备、方法和图形用户界面
CN118567475A (zh) 用于与三维环境进行交互的设备、方法和图形用户界面
CN117957581A (zh) 用于与三维环境交互的设备、方法和图形用户界面

Legal Events

Date Code Title Description
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination