AU2017370555B2 - Virtual user input controls in a mixed reality environment - Google Patents

Virtual user input controls in a mixed reality environment Download PDF

Info

Publication number
AU2017370555B2
AU2017370555B2 AU2017370555A AU2017370555A AU2017370555B2 AU 2017370555 B2 AU2017370555 B2 AU 2017370555B2 AU 2017370555 A AU2017370555 A AU 2017370555A AU 2017370555 A AU2017370555 A AU 2017370555A AU 2017370555 B2 AU2017370555 B2 AU 2017370555B2
Authority
AU
Australia
Prior art keywords
virtual
user
remote
active surface
wearable system
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Fee Related
Application number
AU2017370555A
Other languages
English (en)
Other versions
AU2017370555A1 (en
Inventor
John Adam Croston
Adrian Kaehler
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Magic Leap Inc
Original Assignee
Magic Leap Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Family has litigation
First worldwide family litigation filed litigation Critical https://patents.darts-ip.com/?family=62243916&utm_source=google_patent&utm_medium=platform_link&utm_campaign=public_patent_search&patent=AU2017370555(B2) "Global patent litigation dataset” by Darts-ip is licensed under a Creative Commons Attribution 4.0 International License.
Application filed by Magic Leap Inc filed Critical Magic Leap Inc
Publication of AU2017370555A1 publication Critical patent/AU2017370555A1/en
Application granted granted Critical
Publication of AU2017370555B2 publication Critical patent/AU2017370555B2/en
Expired - Fee Related legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating three-dimensional [3D] models or images for computer graphics
    • G06T19/006—Mixed reality
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
    • G06F3/04815—Interaction with a metaphor-based environment or interaction object displayed as three-dimensional [3D], e.g. changing the user viewpoint with respect to the environment or object
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G02B27/017—Head mounted
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/013—Eye tracking input arrangements
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
    • G06V10/462—Salient features, e.g. scale invariant feature transforms [SIFT]
    • G06V10/464—Salient features, e.g. scale invariant feature transforms [SIFT] using a plurality of salient features, e.g. bag-of-words [BoW] representations
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/20—Scenes; Scene-specific elements in augmented reality scenes
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/107—Static hand or arm

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Software Systems (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Computing Systems (AREA)
  • Evolutionary Computation (AREA)
  • Data Mining & Analysis (AREA)
  • Artificial Intelligence (AREA)
  • Mathematical Physics (AREA)
  • Computer Graphics (AREA)
  • Computer Hardware Design (AREA)
  • Medical Informatics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Optics & Photonics (AREA)
  • User Interface Of Digital Computer (AREA)
  • Processing Or Creating Images (AREA)
AU2017370555A 2016-12-05 2017-12-01 Virtual user input controls in a mixed reality environment Expired - Fee Related AU2017370555B2 (en)

Applications Claiming Priority (5)

Application Number Priority Date Filing Date Title
US201662430308P 2016-12-05 2016-12-05
US201662430279P 2016-12-05 2016-12-05
US62/430,279 2016-12-05
US62/430,308 2016-12-05
PCT/US2017/064311 WO2018106542A1 (en) 2016-12-05 2017-12-01 Virtual user input controls in a mixed reality environment

Publications (2)

Publication Number Publication Date
AU2017370555A1 AU2017370555A1 (en) 2019-06-13
AU2017370555B2 true AU2017370555B2 (en) 2022-12-15

Family

ID=62243916

Family Applications (1)

Application Number Title Priority Date Filing Date
AU2017370555A Expired - Fee Related AU2017370555B2 (en) 2016-12-05 2017-12-01 Virtual user input controls in a mixed reality environment

Country Status (9)

Country Link
US (4) US11150777B2 (https=)
EP (2) EP3549109B1 (https=)
JP (3) JP7112399B2 (https=)
KR (3) KR102413561B1 (https=)
CN (2) CN110249368B (https=)
AU (1) AU2017370555B2 (https=)
CA (1) CA3045664A1 (https=)
IL (2) IL308490A (https=)
WO (1) WO2018106542A1 (https=)

Families Citing this family (143)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8977255B2 (en) 2007-04-03 2015-03-10 Apple Inc. Method and system for operating a multi-function portable electronic device using voice-activation
US8676904B2 (en) 2008-10-02 2014-03-18 Apple Inc. Electronic devices with voice command and contextual data processing capabilities
DE112014000709B4 (de) 2013-02-07 2021-12-30 Apple Inc. Verfahren und vorrichtung zum betrieb eines sprachtriggers für einen digitalen assistenten
US9715875B2 (en) 2014-05-30 2017-07-25 Apple Inc. Reducing the need for manual start/end-pointing and trigger phrases
US10170123B2 (en) 2014-05-30 2019-01-01 Apple Inc. Intelligent assistant for home automation
US9338493B2 (en) 2014-06-30 2016-05-10 Apple Inc. Intelligent automated assistant for TV user interactions
US9886953B2 (en) 2015-03-08 2018-02-06 Apple Inc. Virtual assistant activation
US10460227B2 (en) 2015-05-15 2019-10-29 Apple Inc. Virtual assistant in a communication session
US10331312B2 (en) 2015-09-08 2019-06-25 Apple Inc. Intelligent automated assistant in a media environment
US10747498B2 (en) 2015-09-08 2020-08-18 Apple Inc. Zero latency digital assistant
US10586535B2 (en) 2016-06-10 2020-03-10 Apple Inc. Intelligent digital assistant in a multi-tasking environment
US12197817B2 (en) 2016-06-11 2025-01-14 Apple Inc. Intelligent device arbitration and control
DK201670540A1 (en) 2016-06-11 2018-01-08 Apple Inc Application integration with a digital assistant
US11269480B2 (en) * 2016-08-23 2022-03-08 Reavire, Inc. Controlling objects using virtual rays
JP7112399B2 (ja) 2016-12-05 2022-08-03 マジック リープ， インコーポレイテッド 複合現実環境における仮想ユーザ入力制御
US11204787B2 (en) 2017-01-09 2021-12-21 Apple Inc. Application integration with a digital assistant
US10768425B2 (en) * 2017-02-14 2020-09-08 Securiport Llc Augmented reality monitoring of border control systems
US10627895B2 (en) * 2017-03-21 2020-04-21 Lenovo (Singapore) Pte Ltd Providing a virtual control
US10303715B2 (en) 2017-05-16 2019-05-28 Apple Inc. Intelligent automated assistant for media exploration
IL293424A (en) 2017-07-26 2022-07-01 Magic Leap Inc Neural network training with displays of user interface devices
US10996742B2 (en) * 2017-10-17 2021-05-04 Logitech Europe S.A. Input device for AR/VR applications
CN110192169B (zh) * 2017-11-20 2020-10-02 腾讯科技（深圳）有限公司 虚拟场景中菜单处理方法、装置及存储介质
US10739861B2 (en) * 2018-01-10 2020-08-11 Facebook Technologies, Llc Long distance interaction with artificial reality objects using a near eye display interface
US10818288B2 (en) 2018-03-26 2020-10-27 Apple Inc. Natural assistant interaction
GB2574372B (en) * 2018-05-21 2021-08-11 Imagination Tech Ltd Implementing Traditional Computer Vision Algorithms As Neural Networks
DK180639B1 (en) 2018-06-01 2021-11-04 Apple Inc DISABILITY OF ATTENTION-ATTENTIVE VIRTUAL ASSISTANT
US10725629B2 (en) 2018-06-25 2020-07-28 Google Llc Identifying and controlling smart devices
JP7172207B2 (ja) * 2018-07-10 2022-11-16 オムロン株式会社 入力装置
WO2020023419A1 (en) 2018-07-24 2020-01-30 Magic Leap, Inc. Application sharing
US11348316B2 (en) * 2018-09-11 2022-05-31 Apple Inc. Location-based virtual element modality in three-dimensional content
EP3866402B1 (en) * 2018-09-24 2022-11-02 Google LLC Controlling a device based on processing of image data that captures the device and/or an installation environment of the device
US11462215B2 (en) 2018-09-28 2022-10-04 Apple Inc. Multi-modal inputs for voice commands
CN120122819A (zh) * 2018-09-28 2025-06-10 美国斯耐普公司 用于可穿戴或移动设备上的姿势、穿戴、活动或携带检测的神经网络系统
US10824244B2 (en) * 2018-11-19 2020-11-03 Facebook Technologies, Llc Systems and methods for transitioning between modes of tracking real-world objects for artificial reality interfaces
US11487351B2 (en) * 2018-11-23 2022-11-01 International Business Machines Corporation Intelligent directing system in an internet of things (IoT) computing environment
US11188148B2 (en) * 2018-12-27 2021-11-30 Facebook Technologies, Llc User interaction in head-mounted display with eye tracking
US10831268B1 (en) * 2019-03-15 2020-11-10 Facebook Technologies, Llc Systems and methods for using eye tracking to improve user interactions with objects in artificial reality
US11348573B2 (en) 2019-03-18 2022-05-31 Apple Inc. Multimodality in digital assistant systems
CN109935206B (zh) * 2019-04-15 2021-08-20 京东方科技集团股份有限公司 基于神经网络的显示器件亮度补偿方法、装置及设备
US11307752B2 (en) 2019-05-06 2022-04-19 Apple Inc. User configurable task triggers
DK201970509A1 (en) 2019-05-06 2021-01-15 Apple Inc Spoken notifications
US12210436B2 (en) * 2019-05-28 2025-01-28 Trane International Inc. Ranking user interface elements based on physical conditions
US10890983B2 (en) 2019-06-07 2021-01-12 Facebook Technologies, Llc Artificial reality system having a sliding menu
US10942585B2 (en) * 2019-07-22 2021-03-09 Zspace, Inc. Trackability enhancement of a passive stylus
GB2586059B (en) * 2019-08-01 2023-06-07 Sony Interactive Entertainment Inc System and method for generating user inputs for a video game
US11074081B2 (en) * 2019-08-02 2021-07-27 GM Global Technology Operations LLC Architecture and method supporting multiple vision stream using shared server on embedded platform
CN114207671B (zh) * 2019-08-09 2025-10-17 索尼集团公司 图像处理装置、图像处理方法和程序
CN114450624A (zh) 2019-08-19 2022-05-06 光场实验室公司 用于消费者装置的光场显示器
US10699124B1 (en) 2019-09-09 2020-06-30 Ar, Llc Augmented reality content selection and display based on printed objects having security features
US12579805B2 (en) * 2019-09-09 2026-03-17 Techinvest Company Limited Augmented, virtual and mixed-reality content selection and display for travel
CN112954422A (zh) * 2019-12-11 2021-06-11 天翼智慧家庭科技有限公司 一种用于在vr设备上实现iptv遥控器的方法
CN111006720B (zh) * 2019-12-18 2021-06-11 刘启 一种测量仪器的仿真模拟方法
US11687778B2 (en) 2020-01-06 2023-06-27 The Research Foundation For The State University Of New York Fakecatcher: detection of synthetic portrait videos using biological signals
EP3851939A1 (en) * 2020-01-14 2021-07-21 Apple Inc. Positioning a user-controlled spatial selector based on extremity tracking information and eye tracking information
JP7702955B2 (ja) 2020-01-27 2025-07-04 マジック リープ， インコーポレイテッド ユーザ入力デバイスの機能性の注視タイマベースの拡張
WO2021162769A1 (en) 2020-02-10 2021-08-19 Magic Leap, Inc. Body-centric content positioning relative to three-dimensional container in a mixed reality environment
US11830291B2 (en) * 2020-02-10 2023-11-28 University Of Maryland, College Park System and method for multimodal emotion recognition
CN115398316B (zh) * 2020-02-14 2025-08-26 奇跃公司 3d对象注释
WO2021163624A1 (en) * 2020-02-14 2021-08-19 Magic Leap, Inc. Tool bridge
CN115698818B (zh) 2020-02-14 2024-01-23 奇跃公司 会话管理器
CN111459266A (zh) * 2020-03-02 2020-07-28 重庆爱奇艺智能科技有限公司 一种在虚拟现实的3d场景中操作2d应用的方法和装置
KR20210128269A (ko) * 2020-04-16 2021-10-26 삼성전자주식회사 증강 현실(ar) 디바이스 및 증강 현실 디바이스에서 포즈를 예측하는 방법
US11798429B1 (en) 2020-05-04 2023-10-24 Snap Inc. Virtual tutorials for musical instruments with finger tracking in augmented reality
US11567571B2 (en) * 2020-05-05 2023-01-31 Autodesk, Inc. Remote control of a device via a virtual interface
US12301635B2 (en) 2020-05-11 2025-05-13 Apple Inc. Digital assistant hardware abstraction
US20210358294A1 (en) * 2020-05-15 2021-11-18 Microsoft Technology Licensing, Llc Holographic device control
CN111538252B (zh) * 2020-05-25 2021-09-21 厦门大学 一种应用vr技术的智能家居演示系统
CN111672115B (zh) 2020-06-05 2022-09-23 腾讯科技（深圳）有限公司 虚拟对象控制方法、装置、计算机设备及存储介质
CN113970888A (zh) * 2020-07-07 2022-01-25 华为技术有限公司 家居设备控制方法、终端设备及计算机可读存储介质
US11438683B2 (en) 2020-07-21 2022-09-06 Apple Inc. User identification using headphones
KR102477006B1 (ko) * 2020-08-14 2022-12-14 주식회사 브이터치 객체 제어를 지원하는 방법, 시스템 및 비일시성의 컴퓨터 판독 가능 기록 매체
US20220050432A1 (en) * 2020-08-14 2022-02-17 VTouch Co., Ltd. Method, system and non-transitory computer-readable recording medium for supporting object control
US11474690B2 (en) * 2020-08-14 2022-10-18 VTouch Co., Ltd. Method, system and non-transitory computer-readable recording medium for non-contact control
US11467403B2 (en) * 2020-08-20 2022-10-11 Htc Corporation Operating method and electronic system
CN114200895B (zh) 2020-09-17 2024-12-10 北京小米移动软件有限公司 设备控制方法、装置及存储介质
CN112164216A (zh) * 2020-09-18 2021-01-01 深圳市欧瑞博科技股份有限公司 智能遥控器的配置方法、装置、智能网关设备及存储介质
KR102911932B1 (ko) * 2020-11-16 2026-01-13 구글 엘엘씨 웨어러블 장치를 이용한 제어가능한 장치의 위치 식별
US11914835B2 (en) * 2020-11-16 2024-02-27 Samsung Electronics Co., Ltd. Method for displaying user interface and electronic device therefor
US11423585B2 (en) * 2020-11-30 2022-08-23 Microsoft Technology Licensing, Llc Velocity-based controls
JP2023512122A (ja) * 2020-12-30 2023-03-24 広東視場科技有限公司 スマート制御グラス
US11656690B2 (en) * 2021-03-05 2023-05-23 MediVis, Inc. User input and virtual touch pad in augmented reality for use in surgical settings
US12271658B2 (en) 2021-02-23 2025-04-08 Apple Inc. Digital assistant interactions in copresence sessions
WO2022216784A1 (en) * 2021-04-08 2022-10-13 Snap Inc. Bimanual interactions between mapped hand regions for controlling virtual and graphical elements
US12387435B2 (en) 2021-05-14 2025-08-12 Gridraster, Inc. Digital twin sub-millimeter alignment using multimodal 3D deep learning fusion system and method
US11250637B1 (en) * 2021-05-14 2022-02-15 Gridraster, Inc. Multimodal 3D deep learning fusion system and method for reducing the need of 3D training dataset of 3D object tracking for enterprise digital twin mixed reality
US20220392170A1 (en) * 2021-06-07 2022-12-08 Citrix Systems, Inc. Interactive Display Devices in Extended Reality Environments
US11579752B1 (en) 2021-07-20 2023-02-14 Sony Interactive Entertainment Inc. Augmented reality placement for user feedback
US11635808B2 (en) * 2021-08-12 2023-04-25 International Business Machines Corporation Rendering information in a gaze tracking device on controllable devices in a field of view to remotely control
WO2023028570A1 (en) * 2021-08-27 2023-03-02 Chinook Labs Llc Displaying and manipulating user interface elements
WO2023034012A1 (en) 2021-09-01 2023-03-09 Snap Inc. Handcrafted augmented reality effort evidence
CN117897680A (zh) * 2021-09-01 2024-04-16 斯纳普公司 基于物理动作的增强现实通信交换
KR20240056558A (ko) 2021-09-01 2024-04-30 스냅 인코포레이티드 핸드크래프티드 증강 현실 경험들
US12284324B2 (en) 2021-09-02 2025-04-22 Snap Inc. Augmented reality prop interactions
US12422933B2 (en) 2021-09-03 2025-09-23 Apple Inc. Multi-device gesture control
WO2023055492A1 (en) * 2021-09-28 2023-04-06 Arris Enterprises Llc Device and method to switch television operation to primary content mode after displaying an important content on secondary input mode
US12056275B2 (en) * 2021-10-26 2024-08-06 Meta Platforms Technologies, Llc Method and a system for interacting with physical devices via an artificial-reality device
US11836828B2 (en) * 2021-10-29 2023-12-05 Meta Platforms Technologies, Llc Controlling interactions with virtual objects
TW202324172A (zh) 2021-11-10 2023-06-16 美商元平台技術有限公司 自動建立人工實境世界
WO2023086392A1 (en) * 2021-11-10 2023-05-19 Drnc Holdings, Inc. Context aware object recognition for iot control
CN118251643A (zh) * 2021-11-17 2024-06-25 三星电子株式会社 用于锚定增强现实对象的电子设备和方法
KR20230073869A (ko) * 2021-11-19 2023-05-26 삼성전자주식회사 인터랙션을 지원하는 사용자 인터페이스를 제공하는 방법 및 이를 수행하는 전자 장치
US20230168786A1 (en) * 2021-11-30 2023-06-01 Verizon Patent And Licensing Inc. Methods and Systems for Location-Based Accessing of Predesignated Data Payloads Using Extended Reality
KR102949385B1 (ko) * 2022-01-06 2026-04-08 삼성전자주식회사 가상환경에서 실 세계의 외부 전자 장치를 제어할 수 있는 시스템, 전자 장치 및 방법
WO2023130435A1 (zh) * 2022-01-10 2023-07-13 深圳市闪至科技有限公司 交互方法、头戴式显示设备、系统及存储介质
US12020382B2 (en) 2022-03-24 2024-06-25 Kyndryl, Inc. Physical and virtual environment synchronization
TWI804294B (zh) * 2022-04-28 2023-06-01 中華電信股份有限公司 用於擴增實境協作之遠端設備控制系統、方法及電腦可讀媒介
US12423917B2 (en) 2022-06-10 2025-09-23 Apple Inc. Extended reality based digital assistant interactions
CN115437589A (zh) * 2022-06-13 2022-12-06 北京罗克维尔斯科技有限公司 显示设备的控制器、方法、装置、系统、介质及设备
US20230409179A1 (en) * 2022-06-15 2023-12-21 Apple Inc. Home automation device control and designation
US12524088B2 (en) 2022-07-11 2026-01-13 Apple Inc. Computer systems with handheld controllers
KR20240011477A (ko) 2022-07-19 2024-01-26 주식회사 엘지에너지솔루션 전극 조립체, 배터리 및 이를 포함하는 배터리 팩 및 자동차
CN115357114A (zh) * 2022-08-04 2022-11-18 闻泰通讯股份有限公司 一种设备交互方法和交互装置
US12026325B2 (en) 2022-08-17 2024-07-02 Apple Inc. Handheld input devices with sleeves
US12259560B2 (en) 2022-08-23 2025-03-25 Apple Inc. Handheld controllers with charging and storage systems
KR20240028880A (ko) * 2022-08-25 2024-03-05 삼성전자주식회사 원격 제어 장치를 이용하여 외부 전자 장치를 제어하기 위한 방법 및 이를 지원하는 전자 장치
US12073011B2 (en) 2022-09-01 2024-08-27 Snap Inc. Virtual interfaces for controlling IoT devices
US12148448B2 (en) 2022-09-01 2024-11-19 Snap Inc. Authoring tools for creating interactive AR experiences
US12524124B2 (en) 2022-09-01 2026-01-13 Snap Inc. Recording following behaviors between virtual objects and user avatars in AR experiences
US12045383B2 (en) 2022-09-01 2024-07-23 Snap Inc. Virtual AR interfaces for controlling IoT devices using mobile device orientation sensors
US12169598B2 (en) 2022-09-19 2024-12-17 Snap Inc. AR glasses as IoT remote control
US12130972B2 (en) 2022-09-21 2024-10-29 Apple Inc. Tracking devices for handheld controllers
KR20240058665A (ko) * 2022-10-26 2024-05-03 삼성전자주식회사 센서에 의해 식별된 데이터를 이용하여 시각적 객체를 변경하기 위한 웨어러블 장치 및 그 방법
CN115756230A (zh) * 2022-11-14 2023-03-07 北京字跳网络技术有限公司 面板互动方法、装置、设备和存储介质
US12548208B2 (en) 2022-11-21 2026-02-10 Samsung Electronics Co., Ltd. Wearable device for displaying visual object for controlling virtual object and method thereof
KR20240084665A (ko) * 2022-12-06 2024-06-14 삼성전자주식회사 외부 전자 장치의 제어와 관련된 사용자 인터페이스를 표시하기 위한 웨어러블 장치 및 그 방법
CN118192793A (zh) 2022-12-14 2024-06-14 北京小米移动软件有限公司 Ar眼镜交互显示方法、装置、设备及存储介质
US11869136B1 (en) * 2022-12-16 2024-01-09 Google Llc User-context aware rendering dataset selection
WO2024136383A1 (ko) * 2022-12-20 2024-06-27 삼성전자주식회사 외부 전자 장치에 의해 제공되는 멀티미디어 콘텐트를 표시하기 위한 웨어러블 장치 및 그 방법
US12501099B2 (en) 2023-01-17 2025-12-16 Snap Inc. AR glasses as IoT device for enhanced screen experience
US12400414B2 (en) 2023-02-08 2025-08-26 Meta Platforms Technologies, Llc Facilitating system user interface (UI) interactions in an artificial reality (XR) environment
US12387449B1 (en) 2023-02-08 2025-08-12 Meta Platforms Technologies, Llc Facilitating system user interface (UI) interactions in an artificial reality (XR) environment
US20240281070A1 (en) * 2023-02-16 2024-08-22 Meta Platforms Technologies, Llc Simultaneous Controller and Touch Interactions
EP4684304A1 (en) * 2023-03-20 2026-01-28 Magic Leap, Inc. Method and system for dynamic depth-based reprojection
US12567415B2 (en) 2023-05-09 2026-03-03 Apple Inc. Providing and controlling immersive three-dimensional environments
US12481514B2 (en) * 2023-06-04 2025-11-25 Apple Inc. Volumetric user interface layers for electronic devices
US12079903B1 (en) * 2023-06-26 2024-09-03 Google Llc Real screens in extended reality
WO2025018514A1 (ko) * 2023-07-19 2025-01-23 삼성전자주식회사 가상 환경에서 연결 상태를 표시하기 위한 전자 장치 및 방법
US20250036371A1 (en) * 2023-07-25 2025-01-30 simpleAR, Inc. Serializing and deserializing mixed reality experiences or portions thereof
US12613609B2 (en) * 2023-09-18 2026-04-28 Apple Inc. Systems and methods of motion-based user interfaces
KR102854751B1 (ko) * 2024-02-26 2025-09-03 (주)비엔티 웹 서비스 기반 이미지를 생성하는 대용량 데이터 처리시스템
WO2025224616A1 (en) * 2024-04-24 2025-10-30 Raven Spades Inc. Hand posture and hand gesture determination system
CN118644706B (zh) * 2024-05-29 2024-11-15 孚链艺统科技有限公司 一种基于aid鉴证技术的可移动文物登录方法
US12468381B1 (en) * 2024-09-06 2025-11-11 Snap Inc. Dynamic interaction zone system for accurate 3D button selection in augmented reality environments

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20160018985A1 (en) * 2014-07-15 2016-01-21 Rotem Bennet Holographic keyboard display
US20160165170A1 (en) * 2014-12-03 2016-06-09 VIZIO Inc. Augmented reality remote control

Family Cites Families (115)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5291560A (en) 1991-07-15 1994-03-01 Iri Scan Incorporated Biometric personal identification system based on iris analysis
US6222525B1 (en) 1992-03-05 2001-04-24 Brad A. Armstrong Image controllers with sheet connected sensors
US5583795A (en) 1995-03-17 1996-12-10 The United States Of America As Represented By The Secretary Of The Army Apparatus for measuring eye gaze and fixation duration, and method therefor
US5670988A (en) 1995-09-05 1997-09-23 Interlink Electronics, Inc. Trigger operated electronic device
JPH10200971A (ja) * 1997-01-13 1998-07-31 Mitsubishi Electric Corp パーソナルコンピュータによるワイヤレスリモコンの信号発生装置
US7227526B2 (en) * 2000-07-24 2007-06-05 Gesturetek, Inc. Video-based image control system
DE60332328D1 (de) 2002-03-13 2010-06-10 Imax Corp Systeme und verfahren für ein digitales remastering oder anderweitiges modifizieren von beweglichen bildern oder anderen bildsequenzdaten
US8098901B2 (en) 2005-01-26 2012-01-17 Honeywell International Inc. Standoff iris recognition system
KR20050025927A (ko) 2003-09-08 2005-03-14 유웅덕 홍채인식을 위한 동공 검출 방법 및 형상기술자 추출방법과 그를 이용한 홍채 특징 추출 장치 및 그 방법과홍채인식 시스템 및 그 방법
USD514570S1 (en) 2004-06-24 2006-02-07 Microsoft Corporation Region of a fingerprint scanning device with an illuminated ring
US7248720B2 (en) 2004-10-21 2007-07-24 Retica Systems, Inc. Method and system for generating a combined retina/iris pattern biometric
JP2006146803A (ja) 2004-11-24 2006-06-08 Olympus Corp 操作装置及び遠隔操作システム
US20070081123A1 (en) 2005-10-07 2007-04-12 Lewis Scott W Digital eyewear
US8696113B2 (en) 2005-10-07 2014-04-15 Percept Technologies Inc. Enhanced optical and perceptual digital eyewear
US11428937B2 (en) 2005-10-07 2022-08-30 Percept Technologies Enhanced optical and perceptual digital eyewear
JP4824420B2 (ja) 2006-02-07 2011-11-30 アイテック株式会社 視線ベクトル検出方法及び同装置
US7970179B2 (en) 2006-09-25 2011-06-28 Identix Incorporated Iris data extraction
US8363783B2 (en) 2007-06-04 2013-01-29 Oraya Therapeutics, Inc. Method and device for ocular alignment and coupling of ocular structures
KR20090034089A (ko) * 2007-10-02 2009-04-07 삼성전자주식회사 원격 제어 장치 및 방법
US8098891B2 (en) 2007-11-29 2012-01-17 Nec Laboratories America, Inc. Efficient multi-hypothesis multi-human 3D tracking in crowded scenes
KR100927009B1 (ko) 2008-02-04 2009-11-16 광주과학기술원 증강 현실에서의 햅틱 상호 작용 방법 및 그 시스템
JP5478618B2 (ja) 2008-07-03 2014-04-23 エヌイーシー ラボラトリーズ アメリカ インク 上皮層検出器およびそれに関連する方法
US20100131947A1 (en) 2008-11-24 2010-05-27 Disney Enterprises, Inc. System and method for enabling a local user of a real-life simulation environment to interact with a remote user of a corresponding virtual environment
JP2010134629A (ja) * 2008-12-03 2010-06-17 Sony Corp 情報処理装置および情報処理方法
JP5262681B2 (ja) * 2008-12-22 2013-08-14 ブラザー工業株式会社 ヘッドマウントディスプレイ及びそのプログラム
US8768014B2 (en) 2009-01-14 2014-07-01 Indiana University Research And Technology Corp. System and method for identifying a person with reference to a sclera image
JP5763681B2 (ja) 2010-01-22 2015-08-12 オプティメディカ・コーポレイション 走査レーザによる嚢切開を自動配置する装置
US8345984B2 (en) 2010-01-28 2013-01-01 Nec Laboratories America, Inc. 3D convolutional neural networks for automatic human action recognition
JP5114795B2 (ja) * 2010-01-29 2013-01-09 島根県 画像認識装置および操作判定方法並びにプログラム
US20150309316A1 (en) 2011-04-06 2015-10-29 Microsoft Technology Licensing, Llc Ar glasses with predictive control of external device based on event input
JP5743416B2 (ja) 2010-03-29 2015-07-01 ソニー株式会社 情報処理装置、情報処理方法、およびプログラム
US9977496B2 (en) * 2010-07-23 2018-05-22 Telepatheye Inc. Eye-wearable device user interface and augmented reality method
US9820008B2 (en) * 2010-08-27 2017-11-14 Intel Corporation Capture and recall of home entertainment system session
JP5257437B2 (ja) 2010-10-20 2013-08-07 コニカミノルタビジネステクノロジーズ株式会社 携帯端末及び処理装置の操作方法
US8683086B2 (en) * 2010-11-17 2014-03-25 Flextronics Ap, Llc. Universal remote control with automated setup
US9304319B2 (en) 2010-11-18 2016-04-05 Microsoft Technology Licensing, Llc Automatic focus improvement for augmented reality displays
NZ706893A (en) 2010-12-24 2017-02-24 Magic Leap Inc An ergonomic head mounted display device and optical system
US10156722B2 (en) 2010-12-24 2018-12-18 Magic Leap, Inc. Methods and systems for displaying stereoscopy with a freeform optical system with addressable focus for virtual and augmented reality
CA3035118C (en) 2011-05-06 2022-01-04 Magic Leap, Inc. Massive simultaneous remote digital presence world
JP5765133B2 (ja) * 2011-08-16 2015-08-19 富士通株式会社 入力装置、入力制御方法及び入力制御プログラム
EP2760363A4 (en) 2011-09-29 2015-06-24 Magic Leap Inc TACTILE GLOVE FOR HUMAN COMPUTER INTERACTION
US20130104085A1 (en) 2011-10-21 2013-04-25 Digital Artforms, Inc. Systems and methods for human-computer interaction using a two handed interface
RU2017115669A (ru) 2011-10-28 2019-01-28 Мэджик Лип, Инк. Система и способ для дополненной и виртуальной реальности
CN104067316B (zh) 2011-11-23 2017-10-27 奇跃公司 三维虚拟和增强现实显示系统
US9497501B2 (en) * 2011-12-06 2016-11-15 Microsoft Technology Licensing, Llc Augmented reality virtual monitor
US20130147686A1 (en) * 2011-12-12 2013-06-13 John Clavin Connecting Head Mounted Displays To External Displays And Other Communication Networks
JP2013172432A (ja) 2012-02-23 2013-09-02 Panasonic Corp 機器制御装置、機器制御方法、機器制御プログラム、及び集積回路
BR112014024941A2 (pt) 2012-04-05 2017-09-19 Magic Leap Inc dispositivo de imagem de campo de visão amplo com capacidade de focalização ativa
AU2013274359B2 (en) 2012-06-11 2017-05-25 Magic Leap, Inc. Multiple depth plane three-dimensional display using a wave guide reflector array projector
US9671566B2 (en) 2012-06-11 2017-06-06 Magic Leap, Inc. Planar waveguide apparatus with diffraction element(s) and system employing same
US9141916B1 (en) 2012-06-29 2015-09-22 Google Inc. Using embedding functions with a deep network
CA2870751C (en) 2012-07-31 2015-08-18 Japan Science And Technology Agency Point-of-gaze detection device, point-of-gaze detecting method, personal parameter calculating device, personal parameter calculating method, program, and computer-readable storage medium
US8369595B1 (en) 2012-08-10 2013-02-05 EyeVerify LLC Texture features for biometric authentication
US9329678B2 (en) 2012-08-14 2016-05-03 Microsoft Technology Licensing, Llc Augmented reality overlay for control devices
JP2015534108A (ja) 2012-09-11 2015-11-26 マジック リープ， インコーポレイテッド 人間工学的な頭部搭載型ディスプレイデバイスおよび光学システム
JP2014072576A (ja) * 2012-09-27 2014-04-21 Kyocera Corp 表示装置および制御方法
US20140152558A1 (en) * 2012-11-30 2014-06-05 Tom Salter Direct hologram manipulation using imu
KR20150103723A (ko) 2013-01-03 2015-09-11 메타 컴퍼니 가상 또는 증강매개된 비전을 위한 엑스트라미시브 공간 이미징 디지털 아이 글래스
IL301489B2 (en) 2013-01-15 2024-08-01 Magic Leap Inc A system for scanning electromagnetic imaging radiation
JP2014137774A (ja) * 2013-01-18 2014-07-28 Denso Corp 表示操作装置
KR20230044041A (ko) 2013-03-11 2023-03-31 매직 립, 인코포레이티드 증강 및 가상 현실을 위한 시스템 및 방법
US9147154B2 (en) 2013-03-13 2015-09-29 Google Inc. Classifying resources using a deep network
CN105229719B (zh) 2013-03-15 2018-04-27 奇跃公司 显示系统和方法
JP6165485B2 (ja) 2013-03-28 2017-07-19 国立大学法人埼玉大学 携帯端末向けａｒジェスチャユーザインタフェースシステム
WO2014182769A1 (en) 2013-05-07 2014-11-13 The Johns Hopkins University Automated and non-mydriatic fundus-perimetry camera for irreversible eye diseases
US20140341441A1 (en) 2013-05-20 2014-11-20 Motorola Mobility Llc Wearable device user authentication
US9275308B2 (en) 2013-05-31 2016-03-01 Google Inc. Object detection using deep neural networks
US10262462B2 (en) 2014-04-18 2019-04-16 Magic Leap, Inc. Systems and methods for augmented and virtual reality
US9874749B2 (en) 2013-11-27 2018-01-23 Magic Leap, Inc. Virtual and augmented reality systems and methods
US20140380249A1 (en) 2013-06-25 2014-12-25 Apple Inc. Visual recognition of gestures
CN103431840B (zh) 2013-07-31 2016-01-20 北京智谷睿拓技术服务有限公司 眼睛光学参数检测系统及方法
EP3058418B1 (en) 2013-10-16 2023-10-04 Magic Leap, Inc. Virtual or augmented reality headsets having adjustable interpupillary distance
US9202144B2 (en) 2013-10-30 2015-12-01 Nec Laboratories America, Inc. Regionlets with shift invariant neural patterns for object detection
US10095917B2 (en) 2013-11-04 2018-10-09 Facebook, Inc. Systems and methods for facial representation
JP6236296B2 (ja) 2013-11-14 2017-11-22 株式会社デンソーアイティーラボラトリ 学習装置、学習プログラム、及び学習方法
US9857591B2 (en) 2014-05-30 2018-01-02 Magic Leap, Inc. Methods and system for creating focal planes in virtual and augmented reality
KR102651578B1 (ko) 2013-11-27 2024-03-25 매직 립, 인코포레이티드 가상 및 증강 현실 시스템들 및 방법들
KR20150073378A (ko) * 2013-12-23 2015-07-01 삼성전자주식회사 동작인식을 기반으로 하는 가상 입력장치의 사용자 인터페이스(ui)를 표시하는 장치 및 방법
CN104750234B (zh) 2013-12-27 2018-12-21 中芯国际集成电路制造(北京)有限公司 可穿戴智能设备及可穿戴智能设备的互动方法
US10318007B2 (en) 2014-01-27 2019-06-11 Lg Electronics Inc. Head mounted display device for multi-tasking and method for controlling same
US9430829B2 (en) 2014-01-30 2016-08-30 Case Western Reserve University Automatic detection of mitosis using handcrafted and convolutional neural network features
NZ722903A (en) 2014-01-31 2020-05-29 Magic Leap Inc Multi-focal display system and method
KR102177133B1 (ko) 2014-01-31 2020-11-10 매직 립, 인코포레이티드 멀티-포컬 디스플레이 시스템 및 방법
US10203762B2 (en) 2014-03-11 2019-02-12 Magic Leap, Inc. Methods and systems for creating virtual and augmented reality
IL231862A (en) 2014-04-01 2015-04-30 Superfish Ltd Image representation using a neural network
WO2015164807A1 (en) 2014-04-25 2015-10-29 Texas State University Detection of brain injury and subject state with eye movement biometrics
JP6500477B2 (ja) 2015-02-12 2019-04-17 セイコーエプソン株式会社 頭部装着型表示装置、制御システム、頭部装着型表示装置の制御方法、および、コンピュータープログラム
US9766715B2 (en) 2014-05-01 2017-09-19 Seiko Epson Corporation Head-mount type display device, control system, method of controlling head-mount type display device, and computer program
EP3140779A4 (en) 2014-05-09 2017-11-29 Google LLC Systems and methods for using eye signals with secure mobile communications
USD759657S1 (en) 2014-05-19 2016-06-21 Microsoft Corporation Connector with illumination region
NZ764905A (en) 2014-05-30 2022-05-27 Magic Leap Inc Methods and systems for generating virtual content display with a virtual or augmented reality apparatus
USD752529S1 (en) 2014-06-09 2016-03-29 Comcast Cable Communications, Llc Electronic housing with illuminated region
US9536293B2 (en) 2014-07-30 2017-01-03 Adobe Systems Incorporated Image assessment using deep convolutional neural networks
US20160034811A1 (en) 2014-07-31 2016-02-04 Apple Inc. Efficient generation of complementary acoustic models for performing automatic speech recognition system combination
US9659384B2 (en) 2014-10-03 2017-05-23 EyeEm Mobile GmbH. Systems, methods, and computer program products for searching and sorting images by aesthetic quality
WO2016054779A1 (en) 2014-10-09 2016-04-14 Microsoft Technology Licensing, Llc Spatial pyramid pooling networks for image processing
WO2016054802A1 (en) 2014-10-10 2016-04-14 Beijing Kuangshi Technology Co., Ltd. Hierarchical interlinked multi-scale convolutional network for image parsing
KR102276339B1 (ko) 2014-12-09 2021-07-12 삼성전자주식회사 Ｃｎｎ의 근사화를 위한 학습 장치 및 방법
EP3234871B1 (en) 2014-12-17 2020-11-25 Google LLC Generating numeric embeddings of images
US9678664B2 (en) 2015-04-10 2017-06-13 Google Inc. Neural network for keyboard input decoding
US10007413B2 (en) * 2015-04-27 2018-06-26 Microsoft Technology Licensing, Llc Mixed environment display of attached control elements
USD758367S1 (en) 2015-05-14 2016-06-07 Magic Leap, Inc. Virtual reality headset
CN108135469B (zh) 2015-08-21 2021-03-09 奇跃公司 使用眼睛姿态测量的眼睑形状估计
USD805734S1 (en) 2016-03-04 2017-12-26 Nike, Inc. Shirt
USD794288S1 (en) 2016-03-11 2017-08-15 Nike, Inc. Shoe with illuminable sole light sequence
KR101678880B1 (ko) * 2016-06-28 2016-12-06 재단법인 실감교류인체감응솔루션연구단 가상 키보드 출력 장치 및 제어 방법
US10296792B2 (en) 2016-07-14 2019-05-21 Magic Leap, Inc. Iris boundary estimation using cornea curvature
KR102648770B1 (ko) 2016-07-14 2024-03-15 매직 립, 인코포레이티드 홍채 식별을 위한 딥 뉴럴 네트워크
EP3500911B1 (en) 2016-08-22 2023-09-27 Magic Leap, Inc. Augmented reality display device with deep learning sensors
RU2016138608A (ru) 2016-09-29 2018-03-30 Мэджик Лип, Инк. Нейронная сеть для сегментации изображения глаза и оценки качества изображения
IL293688B2 (en) 2016-10-04 2024-02-01 Magic Leap Inc Efficient data layouts for convolutional neural networks
CN115097937B (zh) 2016-11-15 2025-04-29 奇跃公司 用于长方体检测的深度学习系统
JP7112399B2 (ja) 2016-12-05 2022-08-03 マジック リープ， インコーポレイテッド 複合現実環境における仮想ユーザ入力制御
US10803616B1 (en) 2017-04-13 2020-10-13 Facebook Technologies, Llc Hand calibration using single depth camera
IL293424A (en) 2017-07-26 2022-07-01 Magic Leap Inc Neural network training with displays of user interface devices

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20160018985A1 (en) * 2014-07-15 2016-01-21 Rotem Bennet Holographic keyboard display
US20160165170A1 (en) * 2014-12-03 2016-06-09 VIZIO Inc. Augmented reality remote control

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
VALENTIN HEUN ET AL, "Smarter objects", 2013-04-27; 1077952576 - 1077952576, (2013-04-27), doi:10.1145/2468356.2468528, ISBN 978-1-4503-1952-2, pages 961 - 966 *

Also Published As

Publication number Publication date
AU2017370555A1 (en) 2019-06-13
US11720223B2 (en) 2023-08-08
KR102830052B1 (ko) 2025-07-03
KR102413561B1 (ko) 2022-06-24
EP3549109B1 (en) 2023-04-19
IL308490A (en) 2024-01-01
US11150777B2 (en) 2021-10-19
JP2022097726A (ja) 2022-06-30
JP2020513618A (ja) 2020-05-14
US20220083198A1 (en) 2022-03-17
JP7112399B2 (ja) 2022-08-03
US12175054B2 (en) 2024-12-24
KR102531542B1 (ko) 2023-05-10
IL266888B2 (en) 2024-05-01
IL266888B1 (en) 2024-01-01
EP4220572A1 (en) 2023-08-02
KR102830052B9 (ko) 2025-12-19
EP3549109A4 (en) 2020-07-29
EP4220572B1 (en) 2025-05-14
IL266888A (en) 2019-07-31
US20180157398A1 (en) 2018-06-07
KR20220088962A (ko) 2022-06-28
US20250068292A1 (en) 2025-02-27
KR20230070318A (ko) 2023-05-22
JP7339392B2 (ja) 2023-09-05
CN116883628A (zh) 2023-10-13
JP7720364B2 (ja) 2025-08-07
CN110249368A (zh) 2019-09-17
CA3045664A1 (en) 2018-06-14
US20230315250A1 (en) 2023-10-05
CN110249368B (zh) 2023-07-07
JP2023153357A (ja) 2023-10-17
EP3549109A1 (en) 2019-10-09
WO2018106542A1 (en) 2018-06-14
KR102531542B9 (ko) 2026-01-13
KR20190094381A (ko) 2019-08-13

Similar Documents

Publication Publication Date Title
US12175054B2 (en) Virtual user input controls in a mixed reality environment
JP7411133B2 (ja) 仮想現実ディスプレイシステム、拡張現実ディスプレイシステム、および複合現実ディスプレイシステムのためのキーボード
US10838484B2 (en) Visual aura around field of view
US20220148274A1 (en) Object interactivity in virtual space
CA3018758A1 (en) Interactions with 3d virtual objects using poses and multiple-dof controllers

Legal Events

Date Code Title Description
MK25 Application lapsed reg. 22.2i(2) - failure to pay acceptance fee