JP7464694B2 - 複合現実における空間命令およびガイド - Google Patents

複合現実における空間命令およびガイド Download PDF

Info

Publication number
JP7464694B2
JP7464694B2 JP2022507405A JP2022507405A JP7464694B2 JP 7464694 B2 JP7464694 B2 JP 7464694B2 JP 2022507405 A JP2022507405 A JP 2022507405A JP 2022507405 A JP2022507405 A JP 2022507405A JP 7464694 B2 JP7464694 B2 JP 7464694B2
Authority
JP
Japan
Prior art keywords
user
location
time
virtual content
virtual
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
JP2022507405A
Other languages
English (en)
Japanese (ja)
Other versions
JP2022544116A (ja
JP2022544116A5 (https=
JPWO2021026500A5 (https=
Inventor
トゥシャー アローラ,
スコット クラマリッチ,
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Magic Leap Inc
Original Assignee
Magic Leap Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Magic Leap Inc filed Critical Magic Leap Inc
Publication of JP2022544116A publication Critical patent/JP2022544116A/ja
Priority to JP2023109116A priority Critical patent/JP7627720B2/ja
Publication of JP2022544116A5 publication Critical patent/JP2022544116A5/ja
Publication of JPWO2021026500A5 publication Critical patent/JPWO2021026500A5/ja
Application granted granted Critical
Publication of JP7464694B2 publication Critical patent/JP7464694B2/ja
Priority to JP2025011261A priority patent/JP7778258B2/ja
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00Manipulating three-dimensional [3D] models or images for computer graphics
    • G06T19/006Mixed reality
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/012Head tracking input arrangements
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/033Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
    • G06F3/0346Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor with detection of the device orientation or free movement in a three-dimensional [3D] space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00Handling natural language data
    • G06F40/10Text processing
    • G06F40/166Editing, e.g. inserting or deleting
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00Handling natural language data
    • G06F40/40Processing or translation of natural language
    • G06F40/55Rule-based translation
    • G06F40/56Natural language generation
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/131Protocols for games, networked simulations or virtual reality
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04WWIRELESS COMMUNICATION NETWORKS
    • H04W4/00Services specially adapted for wireless communication networks; Facilities therefor
    • H04W4/02Services making use of location information
    • H04W4/024Guidance services
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04WWIRELESS COMMUNICATION NETWORKS
    • H04W4/00Services specially adapted for wireless communication networks; Facilities therefor
    • H04W4/02Services making use of location information
    • H04W4/025Services making use of location information using location based information parameters
    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01Head-up displays
    • G02B27/017Head mounted
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04RLOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; ELECTRIC HEARING AIDS; PUBLIC ADDRESS SYSTEMS
    • H04R2499/00Aspects covered by H04R or H04S not otherwise provided for in their subgroups
    • H04R2499/10General applications
    • H04R2499/15Transducers incorporated in visual displaying devices, e.g. televisions, computer displays, laptops
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04SSTEREOPHONIC SYSTEMS 
    • H04S7/00Indicating arrangements; Control arrangements, e.g. balance control
    • H04S7/30Control circuits for electronic adaptation of the sound field
    • H04S7/302Electronic adaptation of stereophonic sound system to listener position or orientation
    • H04S7/303Tracking of listener position or orientation

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Artificial Intelligence (AREA)
  • Health & Medical Sciences (AREA)
  • Computational Linguistics (AREA)
  • General Health & Medical Sciences (AREA)
  • Computer Graphics (AREA)
  • Computer Hardware Design (AREA)
  • Software Systems (AREA)
  • User Interface Of Digital Computer (AREA)
  • Processing Or Creating Images (AREA)
  • Stereophonic System (AREA)
  • Navigation (AREA)
JP2022507405A 2019-08-07 2020-08-07 複合現実における空間命令およびガイド Active JP7464694B2 (ja)

Priority Applications (2)

Application Number Priority Date Filing Date Title
JP2023109116A JP7627720B2 (ja) 2019-08-07 2023-07-03 複合現実における空間命令およびガイド
JP2025011261A JP7778258B2 (ja) 2019-08-07 2025-01-27 複合現実における空間命令およびガイド

Applications Claiming Priority (5)

Application Number Priority Date Filing Date Title
US201962884153P 2019-08-07 2019-08-07
US62/884,153 2019-08-07
US201962894448P 2019-08-30 2019-08-30
US62/894,448 2019-08-30
PCT/US2020/045496 WO2021026500A1 (en) 2019-08-07 2020-08-07 Spatial instructions and guides in mixed reality

Related Child Applications (1)

Application Number Title Priority Date Filing Date
JP2023109116A Division JP7627720B2 (ja) 2019-08-07 2023-07-03 複合現実における空間命令およびガイド

Publications (4)

Publication Number Publication Date
JP2022544116A JP2022544116A (ja) 2022-10-17
JP2022544116A5 JP2022544116A5 (https=) 2023-07-11
JPWO2021026500A5 JPWO2021026500A5 (https=) 2023-07-11
JP7464694B2 true JP7464694B2 (ja) 2024-04-09

Family

ID=74498940

Family Applications (3)

Application Number Title Priority Date Filing Date
JP2022507405A Active JP7464694B2 (ja) 2019-08-07 2020-08-07 複合現実における空間命令およびガイド
JP2023109116A Active JP7627720B2 (ja) 2019-08-07 2023-07-03 複合現実における空間命令およびガイド
JP2025011261A Active JP7778258B2 (ja) 2019-08-07 2025-01-27 複合現実における空間命令およびガイド

Family Applications After (2)

Application Number Title Priority Date Filing Date
JP2023109116A Active JP7627720B2 (ja) 2019-08-07 2023-07-03 複合現実における空間命令およびガイド
JP2025011261A Active JP7778258B2 (ja) 2019-08-07 2025-01-27 複合現実における空間命令およびガイド

Country Status (5)

Country Link
US (4) US11704874B2 (https=)
EP (1) EP4010660B1 (https=)
JP (3) JP7464694B2 (https=)
CN (1) CN114502921A (https=)
WO (1) WO2021026500A1 (https=)

Families Citing this family (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11195336B2 (en) * 2018-06-08 2021-12-07 Vulcan Inc. Framework for augmented reality applications
US12003585B2 (en) 2018-06-08 2024-06-04 Vale Group Llc Session-based information exchange
US11704874B2 (en) 2019-08-07 2023-07-18 Magic Leap, Inc. Spatial instructions and guides in mixed reality
US11302077B2 (en) 2020-05-29 2022-04-12 Snap Inc. Augmented reality guidance that generates guidance markers
US11880738B1 (en) * 2021-08-17 2024-01-23 Scandit Ag Visual odometry for optical pattern scanning in a real scene
US12488473B1 (en) 2020-07-27 2025-12-02 Scandit Ag Multi-barcode detection in a real scene using interface enhancement based on user input
WO2022046674A1 (en) * 2020-08-24 2022-03-03 Sterling Labs Llc Devices and methods for motion planning of computer characters
CN113630593A (zh) * 2021-08-17 2021-11-09 宁波未知数字信息技术有限公司 一种多模态高精度全空间混合定位系统
CN113727274B (zh) * 2021-08-20 2023-03-24 中国联合网络通信集团有限公司 一种用户移动性确定方法、装置、设备及存储介质
US11863596B2 (en) * 2021-12-07 2024-01-02 Snap Inc. Shared augmented reality session creation
US12284698B2 (en) * 2022-07-20 2025-04-22 Snap Inc. Secure peer-to-peer connections between mobile devices
US12472435B2 (en) 2022-08-12 2025-11-18 Snap Inc. External controller for an eyewear device
EP4350636A1 (fr) * 2022-10-04 2024-04-10 Atos France Passerelle temps réel entre un environnement virtuel et un environnement réel
US12243266B2 (en) 2022-12-29 2025-03-04 Snap Inc. Device pairing using machine-readable optical label
US12333658B2 (en) 2023-02-21 2025-06-17 Snap Inc. Generating user interfaces displaying augmented reality graphics
CN117170504B (zh) * 2023-11-01 2024-01-19 南京维赛客网络科技有限公司 在虚拟人物交互场景中带人观看的方法、系统及存储介质
US12353784B1 (en) * 2024-03-04 2025-07-08 Curio XR (VR EDU) Modified views for an extended reality environment

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2001312745A (ja) 2000-04-28 2001-11-09 Sony Corp 情報処理装置および方法、並びに格納媒体
JP2009271750A (ja) 2008-05-08 2009-11-19 Ntt Docomo Inc 仮想空間提供装置、プログラム及び仮想空間提供システム
JP2016533565A (ja) 2013-06-18 2016-10-27 マイクロソフト テクノロジー ライセンシング,エルエルシー 仮想オブジェクトの方向付け及び可視化
US20190098291A1 (en) 2016-10-04 2019-03-28 Facebook, Inc. Controls and Interfaces for User Interactions in Virtual Spaces
WO2020174554A1 (ja) 2019-02-26 2020-09-03 マクセル株式会社 映像表示装置及び映像表示方法

Family Cites Families (56)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4852988A (en) 1988-09-12 1989-08-01 Applied Science Laboratories Visor and camera providing a parallax-free field-of-view image for a head-mounted eye movement measurement system
US6847336B1 (en) 1996-10-02 2005-01-25 Jerome H. Lemelson Selectively controllable heads-up display system
US6433760B1 (en) 1999-01-14 2002-08-13 University Of Central Florida Head mounted display with eyetracking capability
US6491391B1 (en) 1999-07-02 2002-12-10 E-Vision Llc System, apparatus, and method for reducing birefringence
US6871107B1 (en) * 1999-07-01 2005-03-22 Ftr Pty, Ltd. Digital audio transcription system
CA2316473A1 (en) 1999-07-28 2001-01-28 Steve Mann Covert headworn information display or data display or viewfinder
CA2362895A1 (en) 2001-06-26 2002-12-26 Steve Mann Smart sunglasses or computer information display built into eyewear having ordinary appearance, possibly with sight license
DE10132872B4 (de) 2001-07-06 2018-10-11 Volkswagen Ag Kopfmontiertes optisches Durchsichtssystem
US20030030597A1 (en) 2001-08-13 2003-02-13 Geist Richard Edwin Virtual display apparatus for mobile activities
CA2388766A1 (en) 2002-06-17 2003-12-17 Steve Mann Eyeglass frames based computer display or eyeglasses with operationally, actually, or computationally, transparent frames
US6943754B2 (en) 2002-09-27 2005-09-13 The Boeing Company Gaze tracking system, eye-tracking assembly and an associated method of calibration
US7347551B2 (en) 2003-02-13 2008-03-25 Fergason Patent Properties, Llc Optical system for monitoring eye movement
US7500747B2 (en) 2003-10-09 2009-03-10 Ipventure, Inc. Eyeglasses with electrical components
MXPA06011168A (es) 2004-04-01 2007-04-16 William C Torch Biosensores, comunicadores, y controladores que verifican el movimiento del ojo y metodos para utilizarlos.
CN1731316A (zh) * 2005-08-19 2006-02-08 北京航空航天大学 虚拟猿戏的人机交互方法
US9658473B2 (en) 2005-10-07 2017-05-23 Percept Technologies Inc Enhanced optical and perceptual digital eyewear
US20070081123A1 (en) 2005-10-07 2007-04-12 Lewis Scott W Digital eyewear
US11428937B2 (en) 2005-10-07 2022-08-30 Percept Technologies Enhanced optical and perceptual digital eyewear
US8696113B2 (en) 2005-10-07 2014-04-15 Percept Technologies Inc. Enhanced optical and perceptual digital eyewear
US9021404B2 (en) * 2006-08-25 2015-04-28 Verizon Patent And Licensing Inc. Systems and methods for modifying content based on a positional relationship
US20100214111A1 (en) 2007-12-21 2010-08-26 Motorola, Inc. Mobile virtual and augmented reality system
US20110213664A1 (en) 2010-02-28 2011-09-01 Osterhout Group, Inc. Local advertising content on an interactive head-mounted eyepiece
US8890946B2 (en) 2010-03-01 2014-11-18 Eyefluence, Inc. Systems and methods for spatially controlled scene illumination
US8531355B2 (en) 2010-07-23 2013-09-10 Gregory A. Maltz Unitized, vision-controlled, wireless eyeglass transceiver
US9292973B2 (en) 2010-11-08 2016-03-22 Microsoft Technology Licensing, Llc Automatic variable virtual focus for augmented reality displays
US9323325B2 (en) 2011-08-30 2016-04-26 Microsoft Technology Licensing, Llc Enhancing an object of interest in a see-through, mixed reality display device
US20130077147A1 (en) 2011-09-22 2013-03-28 Los Alamos National Security, Llc Method for producing a partially coherent beam with fast pattern update rates
US8929589B2 (en) 2011-11-07 2015-01-06 Eyefluence, Inc. Systems and methods for high-resolution gaze tracking
US8611015B2 (en) 2011-11-22 2013-12-17 Google Inc. User interface
US8235529B1 (en) 2011-11-30 2012-08-07 Google Inc. Unlocking a screen using eye tracking information
US10013053B2 (en) 2012-01-04 2018-07-03 Tobii Ab System for gaze interaction
US8638498B2 (en) 2012-01-04 2014-01-28 David D. Bohn Eyebox adjustment for interpupillary distance
US9274338B2 (en) 2012-03-21 2016-03-01 Microsoft Technology Licensing, Llc Increasing field of view of reflective waveguide
JP6006536B2 (ja) 2012-06-01 2016-10-12 任天堂株式会社 情報処理プログラム、情報処理装置、情報処理システム、およびパノラマ動画表示方法
US8989535B2 (en) 2012-06-04 2015-03-24 Microsoft Technology Licensing, Llc Multiple waveguide imaging structure
CN104903818B (zh) 2012-12-06 2018-12-14 谷歌有限责任公司 眼睛跟踪佩戴式设备和使用方法
CA2896985A1 (en) 2013-01-03 2014-07-10 Meta Company Extramissive spatial imaging digital eye glass for virtual or augmediated vision
US20140195918A1 (en) 2013-01-07 2014-07-10 Steven Friedlander Eye tracking user interface
US9417452B2 (en) 2013-03-15 2016-08-16 Magic Leap, Inc. Display system and method
US9524588B2 (en) 2014-01-24 2016-12-20 Avaya Inc. Enhanced communication between remote participants using augmented and virtual reality
US9818225B2 (en) 2014-09-30 2017-11-14 Sony Interactive Entertainment Inc. Synchronizing multiple head-mounted displays to a unified space and correlating movement of objects in the unified space
US20170243403A1 (en) 2014-11-11 2017-08-24 Bent Image Lab, Llc Real-time shared augmented reality experience
EP3236336B1 (en) 2016-04-21 2019-03-27 Nokia Technologies Oy Virtual reality causal summary content
CN106227345A (zh) * 2016-07-21 2016-12-14 深圳市金立通信设备有限公司 一种虚拟现实眼镜及其控制方法
MX395185B (es) * 2016-12-05 2025-03-25 Univ Case Western Reserve Sistemas, métodos y medios para mostrar presentaciones interactivas de realidad aumentada.
US11249714B2 (en) 2017-09-13 2022-02-15 Magical Technologies, Llc Systems and methods of shareable virtual objects and virtual objects as message objects to facilitate communications sessions in an augmented reality environment
CN107967089A (zh) * 2017-12-20 2018-04-27 浙江煮艺文化科技有限公司 一种虚拟现实界面显示方法
US10467809B2 (en) * 2018-01-03 2019-11-05 Verizon Patent And Licensing Inc. Methods and systems for presenting a video stream within a persistent virtual reality world
GB201804383D0 (en) * 2018-03-19 2018-05-02 Microsoft Technology Licensing Llc Multi-endpoint mixed reality meetings
US10937249B2 (en) * 2018-05-07 2021-03-02 Google Llc Systems and methods for anchoring virtual objects to physical locations
US10665028B2 (en) * 2018-06-04 2020-05-26 Facebook, Inc. Mobile persistent augmented-reality experiences
CN108983968A (zh) * 2018-06-22 2018-12-11 湖南商学院 一种基于虚拟现实的图像大数据交互控制系统及方法
US10580218B1 (en) * 2018-07-12 2020-03-03 Facebook, Inc. Augmented-reality image processing
US10375009B1 (en) 2018-10-11 2019-08-06 Richard Fishman Augmented reality based social network with time limited posting
US11854148B2 (en) * 2018-11-16 2023-12-26 Microsoft Technology Licensing, Llc Virtual content display opportunity in mixed reality
US11704874B2 (en) 2019-08-07 2023-07-18 Magic Leap, Inc. Spatial instructions and guides in mixed reality

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2001312745A (ja) 2000-04-28 2001-11-09 Sony Corp 情報処理装置および方法、並びに格納媒体
JP2009271750A (ja) 2008-05-08 2009-11-19 Ntt Docomo Inc 仮想空間提供装置、プログラム及び仮想空間提供システム
JP2016533565A (ja) 2013-06-18 2016-10-27 マイクロソフト テクノロジー ライセンシング,エルエルシー 仮想オブジェクトの方向付け及び可視化
US20190098291A1 (en) 2016-10-04 2019-03-28 Facebook, Inc. Controls and Interfaces for User Interactions in Virtual Spaces
WO2020174554A1 (ja) 2019-02-26 2020-09-03 マクセル株式会社 映像表示装置及び映像表示方法

Also Published As

Publication number Publication date
JP2022544116A (ja) 2022-10-17
US12020391B2 (en) 2024-06-25
US20230326153A1 (en) 2023-10-12
US20210043005A1 (en) 2021-02-11
CN114502921A (zh) 2022-05-13
US20250391134A1 (en) 2025-12-25
US20240296639A1 (en) 2024-09-05
JP2023130426A (ja) 2023-09-20
EP4010660A1 (en) 2022-06-15
JP7778258B2 (ja) 2025-12-01
EP4010660B1 (en) 2025-09-24
US11704874B2 (en) 2023-07-18
JP2025069241A (ja) 2025-04-30
US12430860B2 (en) 2025-09-30
JP7627720B2 (ja) 2025-02-06
WO2021026500A1 (en) 2021-02-11
EP4010660A4 (en) 2022-10-05

Similar Documents

Publication Publication Date Title
JP7627720B2 (ja) 複合現実における空間命令およびガイド
JP7778260B2 (ja) 仮想コンテンツの動的併置
JP7629975B2 (ja) 反響フィンガプリント推定
JP7681609B2 (ja) 3dオブジェクト注釈
JP7539478B2 (ja) セッションマネージャ
JP7762780B2 (ja) 非均一ステレオレンダリング
JP7635249B2 (ja) 遅延オーディオ追従

Legal Events

Date Code Title Description
A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20230703

A621 Written request for application examination

Free format text: JAPANESE INTERMEDIATE CODE: A621

Effective date: 20230703

A871 Explanation of circumstances concerning accelerated examination

Free format text: JAPANESE INTERMEDIATE CODE: A871

Effective date: 20230703

A977 Report on retrieval

Free format text: JAPANESE INTERMEDIATE CODE: A971007

Effective date: 20231016

A131 Notification of reasons for refusal

Free format text: JAPANESE INTERMEDIATE CODE: A131

Effective date: 20231018

A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20240117

TRDD Decision of grant or rejection written
A01 Written decision to grant a patent or to grant a registration (utility model)

Free format text: JAPANESE INTERMEDIATE CODE: A01

Effective date: 20240301

A61 First payment of annual fees (during grant procedure)

Free format text: JAPANESE INTERMEDIATE CODE: A61

Effective date: 20240328

R150 Certificate of patent or registration of utility model

Ref document number: 7464694

Country of ref document: JP

Free format text: JAPANESE INTERMEDIATE CODE: R150