KR102795041B1 - 사용자 인터페이스 메뉴의 콘텍추얼 인식 - Google Patents

사용자 인터페이스 메뉴의 콘텍추얼 인식 Download PDF

Info

Publication number
KR102795041B1
KR102795041B1 KR1020237025009A KR20237025009A KR102795041B1 KR 102795041 B1 KR102795041 B1 KR 102795041B1 KR 1020237025009 A KR1020237025009 A KR 1020237025009A KR 20237025009 A KR20237025009 A KR 20237025009A KR 102795041 B1 KR102795041 B1 KR 102795041B1
Authority
KR
South Korea
Prior art keywords
user
environment
virtual
pose
space
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
KR1020237025009A
Other languages
English (en)
Korean (ko)
Other versions
KR20230113663A (ko
Inventor
제임스 엠. 파우덜리
알리샤 나플레스
폴 아미스테드 후버
터커 스포퍼드
Original Assignee
매직 립, 인코포레이티드
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 매직 립, 인코포레이티드 filed Critical 매직 립, 인코포레이티드
Publication of KR20230113663A publication Critical patent/KR20230113663A/ko
Application granted granted Critical
Publication of KR102795041B1 publication Critical patent/KR102795041B1/ko
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/01—Measuring temperature of body parts ; Diagnostic temperature sensing, e.g. for malignant or inflamed tissue
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/02—Detecting, measuring or recording for evaluating the cardiovascular system, e.g. pulse, heart rate, blood pressure or blood flow
    • A61B5/024—Measuring pulse rate or heart rate
    • A61B5/02405—Determining heart rate variability
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/03—Measuring fluid pressure within the body other than blood pressure, e.g. cerebral pressure ; Measuring pressure in body tissues or organs
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/05—Detecting, measuring or recording for diagnosis by means of electric currents or magnetic fields; Measuring using microwaves or radio waves
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G02B27/017—Head mounted
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/012—Head tracking input arrangements
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/013—Eye tracking input arrangements
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/015—Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/018—Input/output arrangements for oriental characters
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
    • G06F3/04815—Interaction with a metaphor-based environment or interaction object displayed as three-dimensional [3D], e.g. changing the user viewpoint with respect to the environment or object
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
    • G06F3/0482—Interaction with lists of selectable items, e.g. menus
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06K—GRAPHICAL DATA READING; PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
    • G06K19/00—Record carriers for use with machines and with at least a part designed to carry digital markings
    • G06K19/06—Record carriers for use with machines and with at least a part designed to carry digital markings characterised by the kind of the digital marking, e.g. shape, nature, code
    • G06K19/06009—Record carriers for use with machines and with at least a part designed to carry digital markings characterised by the kind of the digital marking, e.g. shape, nature, code with optically detectable marking
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06K—GRAPHICAL DATA READING; PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
    • G06K7/00—Methods or arrangements for sensing record carriers, e.g. for reading patterns
    • G06K7/10—Methods or arrangements for sensing record carriers, e.g. for reading patterns by electromagnetic radiation, e.g. optical sensing; by corpuscular radiation
    • G06K7/14—Methods or arrangements for sensing record carriers, e.g. for reading patterns by electromagnetic radiation, e.g. optical sensing; by corpuscular radiation using light without selection of wavelength, e.g. sensing reflected white light
    • G06K7/1404—Methods for optical code recognition
    • G06K7/1408—Methods for optical code recognition the method being specifically adapted for the type of code
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating three-dimensional [3D] models or images for computer graphics
    • G06T19/006—Mixed reality
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/107—Static hand or arm
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/20—Movements or behaviour, e.g. gesture recognition
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2203/00—Indexing scheme relating to G06F3/00 - G06F3/048
    • G06F2203/01—Indexing scheme relating to G06F3/01
    • G06F2203/011—Emotion or mood input determined on the basis of sensed human body parameters such as pulse, heart rate or beat, temperature of skin, facial expressions, iris, voice pitch, brain activity patterns
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Optics & Photonics (AREA)
  • Artificial Intelligence (AREA)
  • Toxicology (AREA)
  • Electromagnetism (AREA)
  • Multimedia (AREA)
  • Public Health (AREA)
  • Biophysics (AREA)
  • Surgery (AREA)
  • Molecular Biology (AREA)
  • Medical Informatics (AREA)
  • Veterinary Medicine (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Animal Behavior & Ethology (AREA)
  • Pathology (AREA)
  • Cardiology (AREA)
  • Neurology (AREA)
  • Dermatology (AREA)
  • Neurosurgery (AREA)
  • Computer Graphics (AREA)
  • Software Systems (AREA)
  • Computer Hardware Design (AREA)
  • Physiology (AREA)
  • Social Psychology (AREA)
  • Psychiatry (AREA)
  • Hematology (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Radiology & Medical Imaging (AREA)
  • User Interface Of Digital Computer (AREA)
  • Processing Or Creating Images (AREA)
KR1020237025009A 2016-05-20 2017-05-18 사용자 인터페이스 메뉴의 콘텍추얼 인식 Active KR102795041B1 (ko)

Applications Claiming Priority (6)

Application Number Priority Date Filing Date Title
US201662339572P 2016-05-20 2016-05-20
US62/339,572 2016-05-20
US201662380869P 2016-08-29 2016-08-29
US62/380,869 2016-08-29
PCT/US2017/033399 WO2017201329A1 (en) 2016-05-20 2017-05-18 Contextual awareness of user interface menus
KR1020227018789A KR102560558B1 (ko) 2016-05-20 2017-05-18 사용자 인터페이스 메뉴의 콘텍추얼 인식

Related Parent Applications (1)

Application Number Title Priority Date Filing Date
KR1020227018789A Division KR102560558B1 (ko) 2016-05-20 2017-05-18 사용자 인터페이스 메뉴의 콘텍추얼 인식

Publications (2)

Publication Number Publication Date
KR20230113663A KR20230113663A (ko) 2023-07-31
KR102795041B1 true KR102795041B1 (ko) 2025-04-11

Family

ID=60326188

Family Applications (3)

Application Number Title Priority Date Filing Date
KR1020237025009A Active KR102795041B1 (ko) 2016-05-20 2017-05-18 사용자 인터페이스 메뉴의 콘텍추얼 인식
KR1020227018789A Active KR102560558B1 (ko) 2016-05-20 2017-05-18 사용자 인터페이스 메뉴의 콘텍추얼 인식
KR1020187036979A Active KR102407111B1 (ko) 2016-05-20 2017-05-18 사용자 인터페이스 메뉴의 콘텍추얼 인식

Family Applications After (2)

Application Number Title Priority Date Filing Date
KR1020227018789A Active KR102560558B1 (ko) 2016-05-20 2017-05-18 사용자 인터페이스 메뉴의 콘텍추얼 인식
KR1020187036979A Active KR102407111B1 (ko) 2016-05-20 2017-05-18 사용자 인터페이스 메뉴의 콘텍추얼 인식

Country Status (9)

Country Link
US (3) US11328484B2 (https=)
EP (2) EP4060462B1 (https=)
JP (4) JP6880075B2 (https=)
KR (3) KR102795041B1 (https=)
CN (2) CN115185366A (https=)
AU (1) AU2017266933B2 (https=)
CA (1) CA3024689A1 (https=)
IL (2) IL290933B2 (https=)
WO (1) WO2017201329A1 (https=)

Families Citing this family (101)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105094345B (zh) * 2015-09-29 2018-07-27 腾讯科技（深圳）有限公司 一种信息处理方法、终端及计算机存储介质
KR102701209B1 (ko) 2015-10-20 2024-08-29 매직 립, 인코포레이티드 3차원 공간에서 가상 객체들 선택
WO2017172982A1 (en) 2016-03-31 2017-10-05 Magic Leap, Inc. Interactions with 3d virtual objects using poses and multiple-dof controllers
WO2017200279A1 (en) * 2016-05-17 2017-11-23 Samsung Electronics Co., Ltd. Method and apparatus for facilitating interaction with virtual reality equipment
WO2017201329A1 (en) 2016-05-20 2017-11-23 Magic Leap, Inc. Contextual awareness of user interface menus
US10335045B2 (en) 2016-06-24 2019-07-02 Universita Degli Studi Di Trento Self-adaptive matrix completion for heart rate estimation from face videos under realistic conditions
US9983684B2 (en) * 2016-11-02 2018-05-29 Microsoft Technology Licensing, Llc Virtual affordance display at virtual target
US10474336B2 (en) * 2016-12-20 2019-11-12 Adobe Inc. Providing a user experience with virtual reality content and user-selected, real world objects
US10290152B2 (en) * 2017-04-03 2019-05-14 Microsoft Technology Licensing, Llc Virtual object user interface display
US10768693B2 (en) 2017-04-19 2020-09-08 Magic Leap, Inc. Multimodal task execution and text editing for a wearable system
US10325409B2 (en) * 2017-06-16 2019-06-18 Microsoft Technology Licensing, Llc Object holographic augmentation
US10191566B1 (en) * 2017-07-05 2019-01-29 Sony Interactive Entertainment Inc. Interactive input controls in a simulated three-dimensional (3D) environment
CN107803028B (zh) * 2017-09-30 2019-03-08 网易（杭州）网络有限公司 信息处理方法、装置、电子设备及存储介质
US10732826B2 (en) * 2017-11-22 2020-08-04 Microsoft Technology Licensing, Llc Dynamic device interaction adaptation based on user engagement
EP3508972A1 (en) * 2018-01-04 2019-07-10 Harman International Industries, Incorporated Biometric personalized audio processing system
US20190212828A1 (en) * 2018-01-10 2019-07-11 Facebook Technologies, Llc Object enhancement in artificial reality via a near eye display interface
EP3756169B1 (en) * 2018-02-22 2023-04-19 Magic Leap, Inc. Browser for mixed reality systems
KR102111531B1 (ko) * 2018-03-06 2020-05-15 서울대학교병원 Vr 또는 ar을 기반으로 하는 병원에서의 진료 체험 제공 시스템
US11816886B1 (en) * 2018-06-28 2023-11-14 Meta Platforms Technologies, Llc Apparatus, system, and method for machine perception
US10679418B2 (en) * 2018-09-13 2020-06-09 International Business Machines Corporation Augmentation of item dimensions based on derived storage locations for online and physical shopping
US11068145B2 (en) * 2018-11-15 2021-07-20 Disney Enterprises, Inc. Techniques for creative review of 3D content in a production environment
CN111291746B (zh) * 2018-12-06 2024-01-26 宏达国际电子股份有限公司 影像处理系统及影像处理方法
EP3671410B1 (en) * 2018-12-19 2022-08-24 Siemens Healthcare GmbH Method and device to control a virtual reality display unit
WO2020210407A1 (en) * 2019-04-08 2020-10-15 Dubnov Tammuz System and layering method for fast input-driven composition and live-generation of mixed digital content
US11804014B1 (en) 2019-04-22 2023-10-31 Apple Inc. Context-based application placement
CN110109547A (zh) * 2019-05-05 2019-08-09 芋头科技（杭州）有限公司 基于手势识别的命令激活方法和系统
US10890983B2 (en) 2019-06-07 2021-01-12 Facebook Technologies, Llc Artificial reality system having a sliding menu
US11334212B2 (en) 2019-06-07 2022-05-17 Facebook Technologies, Llc Detecting input in artificial reality systems based on a pinch and pull gesture
EP3956748B1 (en) * 2019-06-14 2024-06-05 Hewlett-Packard Development Company, L.P. Headset signals to determine emotional states
JP2020202971A (ja) * 2019-06-17 2020-12-24 Ｉｍｐｕｔｅ株式会社 ストレス抑制システム
US11611608B1 (en) 2019-07-19 2023-03-21 Snap Inc. On-demand camera sharing over a network
US10976818B2 (en) * 2019-08-21 2021-04-13 Universal City Studios Llc Interactive attraction system and method for object and user association
US11462000B2 (en) * 2019-08-26 2022-10-04 Apple Inc. Image-based detection of surfaces that provide specular reflections and reflection modification
EP4029014A4 (en) * 2019-09-13 2023-03-29 Light Field Lab, Inc. LIGHT FIELD DISPLAY SYSTEM FOR ADULT APPLICATIONS
US10991163B2 (en) * 2019-09-20 2021-04-27 Facebook Technologies, Llc Projection casting in virtual environments
US11189099B2 (en) 2019-09-20 2021-11-30 Facebook Technologies, Llc Global and local mode virtual object interactions
US11762457B1 (en) * 2019-09-27 2023-09-19 Apple Inc. User comfort monitoring and notification
JP7436505B2 (ja) 2019-09-27 2024-02-21 アップル インコーポレイテッド 仮想オブジェクトの制御
CN110716646A (zh) * 2019-10-15 2020-01-21 北京市商汤科技开发有限公司 一种增强现实数据呈现方法、装置、设备及存储介质
CN110738737A (zh) * 2019-10-15 2020-01-31 北京市商汤科技开发有限公司 一种ar场景图像处理方法、装置、电子设备及存储介质
JP7150894B2 (ja) 2019-10-15 2022-10-11 ベイジン・センスタイム・テクノロジー・デベロップメント・カンパニー・リミテッド Ａｒシーン画像処理方法及び装置、電子機器並びに記憶媒体
US11557096B2 (en) * 2019-12-09 2023-01-17 At&T Intellectual Property I, L.P. Cognitive stimulation in vehicles
US11687778B2 (en) 2020-01-06 2023-06-27 The Research Foundation For The State University Of New York Fakecatcher: detection of synthetic portrait videos using biological signals
US11226678B2 (en) * 2020-01-27 2022-01-18 Magic Leap, Inc. Gaze timer based augmentation of functionality of a user input device
US11354026B1 (en) * 2020-01-28 2022-06-07 Apple Inc. Method and device for assigning an operation set
EP4104034A4 (en) 2020-02-10 2024-02-21 Magic Leap, Inc. Body-centric content positioning relative to three-dimensional container in a mixed reality environment
US10783800B1 (en) 2020-02-26 2020-09-22 University Of Central Florida Research Foundation, Inc. Sensor-based complexity modulation for therapeutic computer-simulations
SE546185C2 (en) * 2020-03-20 2024-06-25 Deep Forestry Ab A method, system and computer program product for generating labelled 3d data representations of real world objects
WO2021262476A1 (en) * 2020-06-22 2021-12-30 Limonox Projects Llc Event routing in 3d graphical environments
US11954249B1 (en) * 2020-06-26 2024-04-09 Apple Inc. Head-mounted systems with sensor for eye monitoring
CN115836271A (zh) * 2020-07-14 2023-03-21 苹果公司 生成用于工作空间的建议内容
US11418848B2 (en) * 2020-08-31 2022-08-16 TCL Research America Inc. Device and method for interactive video presentation
US12548263B1 (en) * 2020-09-02 2026-02-10 Apple Inc. Augmentation based on physical environment context
EP4211887B1 (en) * 2020-09-09 2026-04-22 Snap Inc. Augmented reality messenger system
US12284146B2 (en) 2020-09-16 2025-04-22 Snap Inc. Augmented reality auto reactions
EP4214901B1 (en) 2020-09-16 2025-11-19 Snap Inc. Context triggered augmented reality
US20220101002A1 (en) * 2020-09-30 2022-03-31 Kyndryl, Inc. Real-world object inclusion in a virtual reality experience
KR102393903B1 (ko) * 2020-11-09 2022-05-04 주식회사 나루스튜디오 Ar 전자 현미경 제공 단말, 방법 및 프로그램
US12524777B2 (en) 2020-11-30 2026-01-13 Snap Inc. Reward-based real-time communication session
KR102733854B1 (ko) * 2020-12-29 2024-11-26 삼성전자주식회사 증강 현실, 가상 현실, 혼합 현실 및 확장 현실 서비스를 제공하기 위한 장치 및 방법
US11610363B2 (en) * 2020-12-31 2023-03-21 Oberon Technologies, Inc. Systems and methods for virtual reality environments
KR102777056B1 (ko) * 2021-01-12 2025-03-10 한국전자통신연구원 개인 맞춤형 인터페이스 적용 장치 및 방법
WO2022212052A1 (en) * 2021-03-31 2022-10-06 Dathomir Laboratories Llc Stress detection
US11861315B2 (en) 2021-04-21 2024-01-02 Meta Platforms, Inc. Continuous learning for natural-language understanding models for assistant systems
US12118790B2 (en) 2021-04-21 2024-10-15 Meta Platforms, Inc. Auto-capture of interesting moments by assistant systems
US12125297B2 (en) 2021-04-21 2024-10-22 Meta Platforms, Inc. Task execution based on real-world text detection for assistant systems
US12131733B2 (en) 2021-04-21 2024-10-29 Meta Platforms, Inc. Active listening for assistant systems
US12134031B2 (en) 2021-09-28 2024-11-05 Sony Group Corporation Method for predefining in an XR space an entryway corresponding to a real entryway
US11596866B1 (en) 2021-09-28 2023-03-07 Sony Group Corporation Method to use recognition of nearby physical surfaces to generate NPC reactions to events
US11890534B2 (en) 2021-09-28 2024-02-06 Sony Group Corporation Method for predefining a virtual entryway at a virtual boundary of an XR space
US11612817B1 (en) 2021-09-28 2023-03-28 Sony Group Corporation Method for predefining activity zones in an extended reality (XR) environment
US11684848B2 (en) 2021-09-28 2023-06-27 Sony Group Corporation Method to improve user understanding of XR spaces based in part on mesh analysis of physical surfaces
US11944905B2 (en) 2021-09-28 2024-04-02 Sony Group Corporation Method to regulate jumps and falls by playable characters in XR spaces
US11617949B1 (en) 2021-09-28 2023-04-04 Sony Group Corporation Methods for predefining virtual staircases connecting platforms in extended reality (XR) environments
US11960652B2 (en) * 2021-10-12 2024-04-16 Qualcomm Incorporated User interactions with remote devices
CN119847337A (zh) * 2021-11-19 2025-04-18 苹果公司 用于电子设备应用程序的场景信息访问
US12443797B1 (en) 2021-12-06 2025-10-14 Meta Platforms, Inc. Low-resource task-oriented semantic parsing via intrinsic modeling for assistant systems
US11908356B2 (en) * 2021-12-15 2024-02-20 Motorola Mobility Llc Augmented reality display device having contextual adaptive brightness
US12197641B2 (en) * 2022-02-03 2025-01-14 Unity Technologies ApS Systems and methods for dynamic continuous input in mixed reality environments
CN116578365A (zh) * 2022-02-10 2023-08-11 苹果公司 用于表面化与电子消息相对应的虚拟对象的方法和设备
US11726578B1 (en) 2022-02-11 2023-08-15 Meta Platforms Technologies, Llc Scrolling and navigation in virtual reality
CN116737028A (zh) * 2022-03-02 2023-09-12 北京字跳网络技术有限公司 短视频的播放方法、装置及电子设备
JPWO2023203923A1 (https=) * 2022-04-22 2023-10-26
US12361455B2 (en) * 2022-06-15 2025-07-15 At&T Intellectual Property 1, L.P. Method and system for personalizing metaverse object recommendations or reviews
US12112441B2 (en) 2022-06-30 2024-10-08 Apple Inc. Content transformations based on reflective object recognition
WO2024034825A1 (ko) * 2022-08-08 2024-02-15 삼성전자주식회사 Ar 장치에게 표시하도록 명령할 컨텐츠를 결정하기 위한 방법 및 장치
US20240086059A1 (en) * 2022-09-12 2024-03-14 Luxsonic Technologies Inc. Gaze and Verbal/Gesture Command User Interface
US12393273B1 (en) 2022-09-23 2025-08-19 Apple Inc. Dynamic recording of an experience based on an emotional state and a scene understanding
US12548271B2 (en) 2022-12-22 2026-02-10 Apple Inc. Attention control in multi-user environments
US12579807B2 (en) * 2022-12-22 2026-03-17 Apple Inc. Contextualized visual search
US12387449B1 (en) 2023-02-08 2025-08-12 Meta Platforms Technologies, Llc Facilitating system user interface (UI) interactions in an artificial reality (XR) environment
US12400414B2 (en) 2023-02-08 2025-08-26 Meta Platforms Technologies, Llc Facilitating system user interface (UI) interactions in an artificial reality (XR) environment
US11991222B1 (en) 2023-05-02 2024-05-21 Meta Platforms Technologies, Llc Persistent call control user interface element in an artificial reality environment
WO2025034055A1 (ko) * 2023-08-09 2025-02-13 삼성전자 주식회사 가상 환경에서 객체 선택을 지원하는 방법 및 이를 지원하는 전자 장치
US20250061678A1 (en) * 2023-08-15 2025-02-20 Brent Hubbard Burgess System and method for enhanced physical interaction with virtual environments through pattern recognition and feedback mechanisms
CN117289791B (zh) * 2023-08-22 2024-10-01 空介(丽水)数字科技有限公司 元宇宙人工智能虚拟装备数据生成方法
US20250068297A1 (en) * 2023-08-23 2025-02-27 Meta Platforms Technologies, Llc Gesture-Engaged Virtual Menu for Controlling Actions on an Artificial Reality Device
TW202529046A (zh) * 2023-10-19 2025-07-16 美商尼安蒂克公司 擴增實境角色之基於大語言模型之動畫選擇
CN117369649B (zh) * 2023-12-05 2024-03-26 山东大学 基于本体感觉的虚拟现实交互系统及方法
WO2025135766A1 (ko) * 2023-12-18 2025-06-26 삼성전자 주식회사 몰입 경험을 제공하는 웨어러블 장치 및 그 제어 방법
WO2026049221A1 (ko) * 2024-08-28 2026-03-05 삼성전자 주식회사 사용자 조작을 기반으로 객체를 증강시키는 전자 장치, 방법 및 저장 매체

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2014134922A (ja) * 2013-01-09 2014-07-24 Sony Corp 情報処理装置、情報処理方法およびプログラム
JP2014174747A (ja) * 2013-03-08 2014-09-22 Sony Corp 情報処理装置、情報処理方法およびプログラム
JP2016031650A (ja) * 2014-07-29 2016-03-07 みこらった株式会社 電子機器、電子機器の表示画面の表示制御方法及びプログラム

Family Cites Families (99)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPS5985116U (ja) 1982-12-01 1984-06-08 株式会社資生堂 化粧用塗布具
JPH04372012A (ja) 1991-06-20 1992-12-25 Fuji Xerox Co Ltd 入力装置
US6222525B1 (en) 1992-03-05 2001-04-24 Brad A. Armstrong Image controllers with sheet connected sensors
JP3310086B2 (ja) 1993-03-31 2002-07-29 株式会社東芝 作業環境評価装置及び作業環境評価方法
US5590268A (en) 1993-03-31 1996-12-31 Kabushiki Kaisha Toshiba System and method for evaluating a workspace represented by a three-dimensional model
US5670988A (en) 1995-09-05 1997-09-23 Interlink Electronics, Inc. Trigger operated electronic device
JP3822357B2 (ja) 1998-02-09 2006-09-20 株式会社東芝 マルチモーダル入出力装置のインタフェース装置及びその方法
US6396509B1 (en) * 1998-02-21 2002-05-28 Koninklijke Philips Electronics N.V. Attention-based interaction in a virtual environment
US6771294B1 (en) * 1999-12-29 2004-08-03 Petri Pulli User interface
US7215322B2 (en) * 2001-05-31 2007-05-08 Siemens Corporate Research, Inc. Input devices for augmented reality applications
CN1547131A (zh) * 2003-12-03 2004-11-17 谭日辉 一种虚拟现实交互式控制系统
KR20050087377A (ko) * 2004-02-26 2005-08-31 삼성전자주식회사 이동 단말기의 인체 건강 지표 측정에 따른 홈 네트워크시스템에서의 주거 분위기 자동 조성 방법
US7561717B2 (en) 2004-07-09 2009-07-14 United Parcel Service Of America, Inc. System and method for displaying item information
MX2007002011A (es) * 2004-08-23 2007-08-07 Gamecaster Inc Aparato, metodos, y sistemas para observacion y manipulacion de un ambiente virtual.
JP4500632B2 (ja) * 2004-09-07 2010-07-14 キヤノン株式会社 仮想現実感提示装置および情報処理方法
JP4574323B2 (ja) 2004-10-26 2010-11-04 キヤノン株式会社 画像処理方法、画像処理装置
US7564476B1 (en) 2005-05-13 2009-07-21 Avaya Inc. Prevent video calls based on appearance
US11428937B2 (en) 2005-10-07 2022-08-30 Percept Technologies Enhanced optical and perceptual digital eyewear
US8696113B2 (en) 2005-10-07 2014-04-15 Percept Technologies Inc. Enhanced optical and perceptual digital eyewear
US20070081123A1 (en) 2005-10-07 2007-04-12 Lewis Scott W Digital eyewear
JP5305561B2 (ja) 2006-02-22 2013-10-02 任天堂株式会社 ゲームプログラムおよびゲーム装置
JP4989383B2 (ja) 2007-09-10 2012-08-01 キヤノン株式会社 情報処理装置、情報処理方法
JP5258399B2 (ja) 2008-06-06 2013-08-07 キヤノン株式会社 画像投影装置及びその制御方法
US8004391B2 (en) * 2008-11-19 2011-08-23 Immersion Corporation Method and apparatus for generating mood-based haptic feedback
US9741147B2 (en) * 2008-12-12 2017-08-22 International Business Machines Corporation System and method to modify avatar characteristics based on inferred conditions
US20100177117A1 (en) 2009-01-14 2010-07-15 International Business Machines Corporation Contextual templates for modifying objects in a virtual universe
EP2333651B1 (en) 2009-12-11 2016-07-20 Dassault Systèmes Method and system for duplicating an object using a touch-sensitive display
KR101114750B1 (ko) 2010-01-29 2012-03-05 주식회사 팬택 다차원 영상을 이용한 사용자 인터페이스 장치
JP5786854B2 (ja) * 2010-03-15 2015-09-30 日本電気株式会社 入力装置、入力方法及びプログラム
US20120005624A1 (en) * 2010-07-02 2012-01-05 Vesely Michael A User Interface Elements for Use within a Three Dimensional Scene
US9304319B2 (en) 2010-11-18 2016-04-05 Microsoft Technology Licensing, Llc Automatic focus improvement for augmented reality displays
US10156722B2 (en) 2010-12-24 2018-12-18 Magic Leap, Inc. Methods and systems for displaying stereoscopy with a freeform optical system with addressable focus for virtual and augmented reality
AU2011348122A1 (en) 2010-12-24 2013-07-11 Magic Leap Inc. An ergonomic head mounted display device and optical system
US9217867B2 (en) * 2011-03-24 2015-12-22 Seiko Epson Corporation Head-mounted display device and control method for the head-mounted display device
JP5784818B2 (ja) 2011-03-29 2015-09-24 クアルコム，インコーポレイテッド 拡張現実システムにおける実世界表面への仮想画像のアンカリング
CN102193631A (zh) * 2011-05-05 2011-09-21 上海大学 可穿戴式三维手势交互系统及其使用方法
JP6316186B2 (ja) 2011-05-06 2018-04-25 マジック リープ， インコーポレイテッドＭａｇｉｃ Ｌｅａｐ，Ｉｎｃ． 広範囲同時遠隔ディジタル提示世界
USD658094S1 (en) 2011-06-18 2012-04-24 Dunn Scott E Rounded square display device for a cord
US9417754B2 (en) 2011-08-05 2016-08-16 P4tents1, LLC User interface system, method, and computer program product
WO2013028813A1 (en) * 2011-08-23 2013-02-28 Microsoft Corporation Implicit sharing and privacy control through physical behaviors using sensor-rich devices
EP2760363A4 (en) 2011-09-29 2015-06-24 Magic Leap Inc TACTILE GLOVE FOR HUMAN COMPUTER INTERACTION
US20130104085A1 (en) 2011-10-21 2013-04-25 Digital Artforms, Inc. Systems and methods for human-computer interaction using a two handed interface
US9215293B2 (en) 2011-10-28 2015-12-15 Magic Leap, Inc. System and method for augmented and virtual reality
US20130125066A1 (en) 2011-11-14 2013-05-16 Microsoft Corporation Adaptive Area Cursor
US8950867B2 (en) 2011-11-23 2015-02-10 Magic Leap, Inc. Three dimensional virtual and augmented reality display system
JP2013152711A (ja) 2011-12-28 2013-08-08 Nikon Corp 投影装置及び表示装置
WO2013152205A1 (en) 2012-04-05 2013-10-10 Augmented Vision Inc. Wide-field of view (fov) imaging devices with active foveation capability
WO2013188464A1 (en) 2012-06-11 2013-12-19 Magic Leap, Inc. Multiple depth plane three-dimensional display using a wave guide reflector array projector
US9671566B2 (en) * 2012-06-11 2017-06-06 Magic Leap, Inc. Planar waveguide apparatus with diffraction element(s) and system employing same
US8754829B2 (en) 2012-08-04 2014-06-17 Paul Lapstun Scanning light field camera and display
US9530232B2 (en) * 2012-09-04 2016-12-27 Qualcomm Incorporated Augmented reality surface segmentation
KR20150054967A (ko) 2012-09-11 2015-05-20 매직 립, 인코포레이티드 인체공학적 헤드 마운티드 디스플레이 디바이스 및 광학 시스템
US10503359B2 (en) 2012-11-15 2019-12-10 Quantum Interface, Llc Selection attractive interfaces, systems and apparatuses including such interfaces, methods for making and using same
US20140160157A1 (en) 2012-12-11 2014-06-12 Adam G. Poulos People-triggered holographic reminders
US9317972B2 (en) 2012-12-18 2016-04-19 Qualcomm Incorporated User interface for augmented reality enabled devices
KR101800617B1 (ko) 2013-01-02 2017-12-20 삼성전자주식회사 디스플레이 장치 및 이의 화상 통화 방법
KR102141992B1 (ko) 2013-01-15 2020-08-06 매직 립, 인코포레이티드 초고해상도 스캐닝 섬유 디스플레이
GB2511096A (en) 2013-02-22 2014-08-27 Fox Murphy Ltd A Mobile Indoor Navigation System
US20140258942A1 (en) 2013-03-05 2014-09-11 Intel Corporation Interaction of multiple perceptual sensing inputs
IL308285B2 (en) 2013-03-11 2024-11-01 Magic Leap Inc System and method for augmentation and virtual reality
US9417452B2 (en) 2013-03-15 2016-08-16 Magic Leap, Inc. Display system and method
US9245388B2 (en) * 2013-05-13 2016-01-26 Microsoft Technology Licensing, Llc Interactions of virtual objects with surfaces
US9354702B2 (en) * 2013-06-03 2016-05-31 Daqri, Llc Manipulation of virtual object in augmented reality via thought
US9383819B2 (en) 2013-06-03 2016-07-05 Daqri, Llc Manipulation of virtual object in augmented reality via intent
US9874749B2 (en) 2013-11-27 2018-01-23 Magic Leap, Inc. Virtual and augmented reality systems and methods
US10262462B2 (en) 2014-04-18 2019-04-16 Magic Leap, Inc. Systems and methods for augmented and virtual reality
US9129430B2 (en) 2013-06-25 2015-09-08 Microsoft Technology Licensing, Llc Indicating out-of-view augmented reality images
KR20150037254A (ko) 2013-09-30 2015-04-08 엘지전자 주식회사 착용형 디스플레이 디바이스 및 레이어 제어 방법
WO2015051047A1 (en) 2013-10-01 2015-04-09 Quantum Interface, Llc. Selection attractive interfaces, systems and apparatuses including such interfaces, methods for making and using same
JP6479785B2 (ja) 2013-10-16 2019-03-06 マジック リープ， インコーポレイテッドＭａｇｉｃ Ｌｅａｐ，Ｉｎｃ． 調節可能な瞳孔間距離を有する仮想または拡張現実ヘッドセット
CN107272199B (zh) 2013-11-27 2023-04-07 奇跃公司 虚拟和增强现实系统与方法
US9857591B2 (en) 2014-05-30 2018-01-02 Magic Leap, Inc. Methods and system for creating focal planes in virtual and augmented reality
CN103713737B (zh) * 2013-12-12 2017-01-11 中国科学院深圳先进技术研究院 用于智能眼镜的虚拟键盘系统
US20150187357A1 (en) * 2013-12-30 2015-07-02 Samsung Electronics Co., Ltd. Natural input based virtual ui system for mobile devices
US20150371516A1 (en) * 2014-01-03 2015-12-24 Google Inc. Determining information flow using physiological parameters
US9244539B2 (en) 2014-01-07 2016-01-26 Microsoft Technology Licensing, Llc Target positioning with gaze tracking
US9836122B2 (en) * 2014-01-21 2017-12-05 Osterhout Group, Inc. Eye glint imaging in see-through computer display systems
CN106233189B (zh) 2014-01-31 2020-06-26 奇跃公司 多焦点显示系统和方法
KR102177133B1 (ko) 2014-01-31 2020-11-10 매직 립, 인코포레이티드 멀티-포컬 디스플레이 시스템 및 방법
US20150231509A1 (en) 2014-02-17 2015-08-20 Xaviant, LLC (a GA Limited Liability Company) System, Method, and Apparatus for Smart Targeting
US10203762B2 (en) 2014-03-11 2019-02-12 Magic Leap, Inc. Methods and systems for creating virtual and augmented reality
KR20150110257A (ko) * 2014-03-21 2015-10-02 삼성전자주식회사 웨어러블 디바이스에서 가상의 입력 인터페이스를 제공하는 방법 및 이를 위한 웨어러블 디바이스
WO2015161307A1 (en) 2014-04-18 2015-10-22 Magic Leap, Inc. Systems and methods for augmented and virtual reality
EP3140780B1 (en) 2014-05-09 2020-11-04 Google LLC Systems and methods for discerning eye signals and continuous biometric identification
NZ727350A (en) 2014-05-30 2020-08-28 Magic Leap Inc Methods and systems for generating virtual content display with a virtual or augmented reality apparatus
JP2017533487A (ja) 2014-08-15 2017-11-09 ザ・ユニバーシティ・オブ・ブリティッシュ・コロンビア 医療処置の実施と医療関連情報のアクセスおよび／または操作のための方法およびシステム
KR102029756B1 (ko) 2014-11-03 2019-10-08 삼성전자주식회사 웨어러블 디바이스 및 그 제어 방법
US9754416B2 (en) * 2014-12-23 2017-09-05 Intel Corporation Systems and methods for contextually augmented video creation and sharing
US9746921B2 (en) * 2014-12-31 2017-08-29 Sony Interactive Entertainment Inc. Signal generation and detector systems and methods for determining positions of fingers of a user
WO2016138178A1 (en) * 2015-02-25 2016-09-01 Brian Mullins Visual gestures for a head mounted device
US9576190B2 (en) 2015-03-18 2017-02-21 Snap Inc. Emotion recognition in video conferencing
USD758367S1 (en) 2015-05-14 2016-06-07 Magic Leap, Inc. Virtual reality headset
US10832479B2 (en) 2015-06-15 2020-11-10 Sony Corporation Information processing apparatus, information processing method, and program
US9824499B2 (en) * 2015-06-23 2017-11-21 Microsoft Technology Licensing, Llc Mixed-reality image capture
EP3118722B1 (en) 2015-07-14 2020-07-01 Nokia Technologies Oy Mediated reality
WO2017201329A1 (en) * 2016-05-20 2017-11-23 Magic Leap, Inc. Contextual awareness of user interface menus
CN111052046B (zh) 2017-09-29 2022-06-03 苹果公司 使用现实界面访问外部设备的功能
JP7007684B2 (ja) * 2018-12-26 2022-01-25 株式会社 東洋ブラザー工業 車及び扉の開閉装置
JP7124154B2 (ja) * 2020-09-29 2022-08-23 Ｌｉｎｅ株式会社 プログラム、情報処理方法、端末

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2014134922A (ja) * 2013-01-09 2014-07-24 Sony Corp 情報処理装置、情報処理方法およびプログラム
JP2014174747A (ja) * 2013-03-08 2014-09-22 Sony Corp 情報処理装置、情報処理方法およびプログラム
JP2016031650A (ja) * 2014-07-29 2016-03-07 みこらった株式会社 電子機器、電子機器の表示画面の表示制御方法及びプログラム

Also Published As

Publication number Publication date
IL262987B (en) 2022-04-01
CN109496331B (zh) 2022-06-21
KR20190008941A (ko) 2019-01-25
JP2021170384A (ja) 2021-10-28
JP2024009318A (ja) 2024-01-19
US20220230404A1 (en) 2022-07-21
CN115185366A (zh) 2022-10-14
JP2019517072A (ja) 2019-06-20
KR20220080030A (ko) 2022-06-14
JP2020161177A (ja) 2020-10-01
IL290933B2 (en) 2023-10-01
EP4060462A1 (en) 2022-09-21
CN109496331A (zh) 2019-03-19
KR102407111B1 (ko) 2022-06-08
EP3459071B1 (en) 2022-05-11
JP7682048B2 (ja) 2025-05-23
US20170337742A1 (en) 2017-11-23
JP7778127B2 (ja) 2025-12-01
JP6880075B2 (ja) 2021-06-02
EP3459071A4 (en) 2020-02-26
IL290933A (en) 2022-04-01
WO2017201329A1 (en) 2017-11-23
US12014464B2 (en) 2024-06-18
US11328484B2 (en) 2022-05-10
US20240282062A1 (en) 2024-08-22
CA3024689A1 (en) 2017-11-23
KR102560558B1 (ko) 2023-07-27
JP7168612B2 (ja) 2022-11-09
EP4060462B1 (en) 2025-07-30
AU2017266933A1 (en) 2018-12-06
KR20230113663A (ko) 2023-07-31
AU2017266933B2 (en) 2023-01-12
IL262987A (en) 2018-12-31
EP3459071A1 (en) 2019-03-27
IL290933B1 (en) 2023-06-01

Similar Documents

Publication Publication Date Title
JP7778127B2 (ja) ユーザインターフェースメニューのコンテキスト認識
US12293013B2 (en) Visual aura around field of view

Legal Events

Date Code Title Description
A107 Divisional application of patent
PA0104 Divisional application for international application

St.27 status event code: A-0-1-A10-A16-div-PA0104

St.27 status event code: A-0-1-A10-A18-div-PA0104

PA0201 Request for examination

St.27 status event code: A-1-2-D10-D11-exm-PA0201

P11-X000 Amendment of application requested

St.27 status event code: A-2-2-P10-P11-nap-X000

P13-X000 Application amended

St.27 status event code: A-2-2-P10-P13-nap-X000

PG1501 Laying open of application

St.27 status event code: A-1-1-Q10-Q12-nap-PG1501

E902 Notification of reason for refusal
PE0902 Notice of grounds for rejection

St.27 status event code: A-1-2-D10-D21-exm-PE0902

R17-X000 Change to representative recorded

St.27 status event code: A-3-3-R10-R17-oth-X000

T11-X000 Administrative time limit extension requested

St.27 status event code: U-3-3-T10-T11-oth-X000

P11-X000 Amendment of application requested

St.27 status event code: A-2-2-P10-P11-nap-X000

P13-X000 Application amended

St.27 status event code: A-2-2-P10-P13-nap-X000

PE0601 Decision on rejection of patent

St.27 status event code: N-2-6-B10-B15-exm-PE0601

P11-X000 Amendment of application requested

St.27 status event code: A-2-2-P10-P11-nap-X000

PX0701 Decision of registration after re-examination

St.27 status event code: A-3-4-F10-F13-rex-PX0701

X701 Decision to grant (after re-examination)
GRNT Written decision to grant
PR0701 Registration of establishment

St.27 status event code: A-2-4-F10-F11-exm-PR0701

PR1002 Payment of registration fee

Fee payment year number: 1

St.27 status event code: A-2-2-U10-U12-oth-PR1002

PG1601 Publication of registration

St.27 status event code: A-4-4-Q10-Q13-nap-PG1601