JP7777536B2 - 大規模環境のためのクロスリアリティシステム - Google Patents

大規模環境のためのクロスリアリティシステム

Info

Publication number
JP7777536B2
JP7777536B2 JP2022548662A JP2022548662A JP7777536B2 JP 7777536 B2 JP7777536 B2 JP 7777536B2 JP 2022548662 A JP2022548662 A JP 2022548662A JP 2022548662 A JP2022548662 A JP 2022548662A JP 7777536 B2 JP7777536 B2 JP 7777536B2
Authority
JP
Japan
Prior art keywords
map
features
user
maps
feature
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
JP2022548662A
Other languages
English (en)
Japanese (ja)
Other versions
JPWO2021163289A5 (https=
JP2023514206A (ja
JP2023514206A5 (https=
Inventor
エマニュエル ミュルタレア，
アディ ペリー，
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Magic Leap Inc
Original Assignee
Magic Leap Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Magic Leap Inc filed Critical Magic Leap Inc
Publication of JP2023514206A publication Critical patent/JP2023514206A/ja
Publication of JPWO2021163289A5 publication Critical patent/JPWO2021163289A5/ja
Publication of JP2023514206A5 publication Critical patent/JP2023514206A5/ja
Application granted granted Critical
Publication of JP7777536B2 publication Critical patent/JP7777536B2/ja
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/22—Matching criteria, e.g. proximity measures
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating three-dimensional [3D] models or images for computer graphics
    • G06T19/006—Mixed reality
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
    • G06V10/443—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components by matching or filtering
    • G06V10/449—Biologically inspired filters, e.g. difference of Gaussians [DoG] or Gabor filters
    • G06V10/451—Biologically inspired filters, e.g. difference of Gaussians [DoG] or Gabor filters with interaction between the filter responses, e.g. cortical complex cells
    • G06V10/454—Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
    • G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/98—Detection or correction of errors, e.g. by rescanning the pattern or by human intervention; Evaluation of the quality of the acquired patterns
    • G06V10/993—Evaluation of the quality of the acquired pattern
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/10—Terrestrial scenes
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/60—Type of objects
    • G06V20/64—Three-dimensional [3D] objects
    • G06V20/647—Three-dimensional [3D] objects by matching two-dimensional images to three-dimensional objects
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/012—Head tracking input arrangements
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04W—WIRELESS COMMUNICATION NETWORKS
    • H04W4/00—Services specially adapted for wireless communication networks; Facilities therefor
    • H04W4/02—Services making use of location information
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04W—WIRELESS COMMUNICATION NETWORKS
    • H04W4/00—Services specially adapted for wireless communication networks; Facilities therefor
    • H04W4/30—Services specially adapted for particular environments, situations or purposes
    • H04W4/38—Services specially adapted for particular environments, situations or purposes for collecting sensor information

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Quality & Reliability (AREA)
  • Data Mining & Analysis (AREA)
  • Medical Informatics (AREA)
  • Databases & Information Systems (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Computing Systems (AREA)
  • Computer Graphics (AREA)
  • Computer Hardware Design (AREA)
  • Molecular Biology (AREA)
  • Biodiversity & Conservation Biology (AREA)
  • Biomedical Technology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Human Computer Interaction (AREA)
  • User Interface Of Digital Computer (AREA)
  • Processing Or Creating Images (AREA)
  • Image Analysis (AREA)
JP2022548662A 2020-02-13 2021-02-11 大規模環境のためのクロスリアリティシステム Active JP7777536B2 (ja)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US202062976129P 2020-02-13 2020-02-13
US62/976,129 2020-02-13
PCT/US2021/017607 WO2021163289A1 (en) 2020-02-13 2021-02-11 Cross reality system for large scale environments

Publications (4)

Publication Number Publication Date
JP2023514206A JP2023514206A (ja) 2023-04-05
JPWO2021163289A5 JPWO2021163289A5 (https=) 2024-02-20
JP2023514206A5 JP2023514206A5 (https=) 2024-02-20
JP7777536B2 true JP7777536B2 (ja) 2025-11-28

Family

ID=77272775

Family Applications (1)

Application Number Title Priority Date Filing Date
JP2022548662A Active JP7777536B2 (ja) 2020-02-13 2021-02-11 大規模環境のためのクロスリアリティシステム

Country Status (5)

Country Link
US (2) US20210256766A1 (https=)
EP (1) EP4104144A4 (https=)
JP (1) JP7777536B2 (https=)
CN (1) CN115380264A (https=)
WO (1) WO2021163289A1 (https=)

Families Citing this family (29)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11227435B2 (en) 2018-08-13 2022-01-18 Magic Leap, Inc. Cross reality system
WO2020036898A1 (en) 2018-08-13 2020-02-20 Magic Leap, Inc. A cross reality system
EP3861387B1 (en) 2018-10-05 2025-05-21 Magic Leap, Inc. Rendering location specific virtual content in any location
US11257294B2 (en) 2019-10-15 2022-02-22 Magic Leap, Inc. Cross reality system supporting multiple device types
JP7713936B2 (ja) 2019-10-15 2025-07-28 マジック リープ， インコーポレイテッド 位置特定サービスを伴うクロスリアリティシステムのためのデバイス、方法、およびコンピュータ可読媒体
WO2021076748A1 (en) 2019-10-15 2021-04-22 Magic Leap, Inc. Cross reality system with wireless fingerprints
EP4052086A4 (en) 2019-10-31 2023-11-15 Magic Leap, Inc. EXTENDED REALITY SYSTEM PROVIDING QUALITY INFORMATION ABOUT PERSISTENT COORDINATE FRAMES
JP7525603B2 (ja) 2019-11-12 2024-07-30 マジック リープ， インコーポレイテッド 位置特定サービスおよび共有場所ベースのコンテンツを伴うクロスリアリティシステム
JP7748945B2 (ja) 2019-12-09 2025-10-03 マジック リープ， インコーポレイテッド 仮想コンテンツの簡略化されたプログラミングを伴うクロスリアリティシステム
US20210248826A1 (en) * 2020-02-07 2021-08-12 Krikey, Inc. Surface distinction for mobile rendered augmented reality
CN115398484A (zh) 2020-02-13 2022-11-25 奇跃公司 具有用于定位的地理定位信息优先级的交叉现实系统
WO2021163306A1 (en) 2020-02-13 2021-08-19 Magic Leap, Inc. Cross reality system with accurate shared maps
JP7768888B2 (ja) 2020-02-13 2025-11-12 マジック リープ， インコーポレイテッド マルチ分解能フレーム記述子を使用したマップ処理を伴うクロスリアリティシステム
US11551430B2 (en) 2020-02-26 2023-01-10 Magic Leap, Inc. Cross reality system with fast localization
US11107290B1 (en) * 2020-02-27 2021-08-31 Samsung Electronics Company, Ltd. Depth map re-projection on user electronic devices
WO2021222371A1 (en) * 2020-04-29 2021-11-04 Magic Leap, Inc. Cross reality system for large scale environments
US11755275B2 (en) * 2020-06-29 2023-09-12 Meta Platforms Technologies, Llc Generating augmented reality experiences utilizing physical objects to represent analogous virtual objects
KR20220030641A (ko) * 2020-09-03 2022-03-11 삼성전자주식회사 전자 장치 및 전자 장치의 동작 방법
DE102020215461A1 (de) * 2020-12-08 2022-06-09 Robert Bosch Gesellschaft mit beschränkter Haftung Vorrichtung und Verfahren zum Trainieren eines Bildgenerators
US11636570B2 (en) * 2021-04-01 2023-04-25 Adobe Inc. Generating digital images utilizing high-resolution sparse attention and semantic layout manipulation neural networks
CN113658278B (zh) * 2021-08-25 2025-06-06 优奈柯恩(北京)科技有限公司 用于空间定位的方法及装置
WO2023063937A1 (en) * 2021-10-13 2023-04-20 Innopeak Technology, Inc. Methods and systems for detecting planar regions using predicted depth
TW202324172A (zh) 2021-11-10 2023-06-16 美商元平台技術有限公司 自動建立人工實境世界
US12205327B2 (en) * 2021-12-29 2025-01-21 L3Harris Technologies, Inc. Adaptive alignment system
US20250147183A1 (en) * 2022-02-10 2025-05-08 Singulos Research Inc. System and method for object comprehension
CN115406452B (zh) * 2022-08-26 2025-11-28 优地机器人(无锡)股份有限公司 一种实时定位及建图方法、装置及终端设备
CN118334075A (zh) * 2023-01-12 2024-07-12 高通科技公司 位姿追踪方法及系统、可移动设备、电子设备及存储介质
CN116342826B (zh) * 2023-05-25 2023-10-10 上海维智卓新信息科技有限公司 一种ar地图的构建方法及装置
CN116834000A (zh) * 2023-05-31 2023-10-03 南京航空航天大学 一种用于工业机器人拧钉的钉孔位置定位方法

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2011008752A (ja) 2009-06-26 2011-01-13 Fuji Xerox Co Ltd ドキュメント操作システム、ドキュメント操作方法およびそのためのプログラム
US20170336511A1 (en) 2016-05-18 2017-11-23 Google Inc. System and method for concurrent odometry and mapping
US10504008B1 (en) 2016-07-18 2019-12-10 Occipital, Inc. System and method for relocalization and scene recognition

Family Cites Families (35)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP3631151B2 (ja) * 2000-11-30 2005-03-23 キヤノン株式会社 情報処理装置、複合現実感提示装置及びその方法並びに記憶媒体
JP4865557B2 (ja) * 2003-08-15 2012-02-01 スカーペ テクノロジーズ アクティーゼルスカブ 有界三次元対象物の分類と空間ローカライゼーションのためのコンピュータ視覚システム
JP5784818B2 (ja) * 2011-03-29 2015-09-24 クアルコム，インコーポレイテッド 拡張現実システムにおける実世界表面への仮想画像のアンカリング
CN102682091A (zh) * 2012-04-25 2012-09-19 腾讯科技（深圳）有限公司 基于云服务的视觉搜索方法和系统
US8798357B2 (en) * 2012-07-09 2014-08-05 Microsoft Corporation Image-based localization
US20150294189A1 (en) * 2012-07-23 2015-10-15 Selim BenHimane Method of providing image feature descriptors
GB201303707D0 (en) * 2013-03-01 2013-04-17 Tosas Bautista Martin System and method of interaction for mobile devices
US9349072B2 (en) * 2013-03-11 2016-05-24 Microsoft Technology Licensing, Llc Local feature based image compression
US10373366B2 (en) * 2015-05-14 2019-08-06 Qualcomm Incorporated Three-dimensional model generation
GB201517101D0 (en) * 2015-09-28 2015-11-11 Univ Essex Entpr Ltd Mixed-reality system
US20180122143A1 (en) * 2016-03-15 2018-05-03 Sutherland Cook Ellwood, JR. Hybrid photonic vr/ar systems
US20180005015A1 (en) * 2016-07-01 2018-01-04 Vangogh Imaging, Inc. Sparse simultaneous localization and matching with unified tracking
US10162362B2 (en) * 2016-08-29 2018-12-25 PerceptIn, Inc. Fault tolerance to provide robust tracking for autonomous positional awareness
US10380763B2 (en) * 2016-11-16 2019-08-13 Seiko Epson Corporation Hybrid corner and edge-based tracking
US10452133B2 (en) * 2016-12-12 2019-10-22 Microsoft Technology Licensing, Llc Interacting with an environment using a parent device and at least one companion device
US10163003B2 (en) * 2016-12-28 2018-12-25 Adobe Systems Incorporated Recognizing combinations of body shape, pose, and clothing in three-dimensional input images
US10659768B2 (en) * 2017-02-28 2020-05-19 Mitsubishi Electric Research Laboratories, Inc. System and method for virtually-augmented visual simultaneous localization and mapping
US10664993B1 (en) * 2017-03-13 2020-05-26 Occipital, Inc. System for determining a pose of an object
US10600252B2 (en) * 2017-03-30 2020-03-24 Microsoft Technology Licensing, Llc Coarse relocalization using signal fingerprints
US10768693B2 (en) * 2017-04-19 2020-09-08 Magic Leap, Inc. Multimodal task execution and text editing for a wearable system
US10705598B2 (en) * 2017-05-09 2020-07-07 Microsoft Technology Licensing, Llc Tracking wearable device and handheld object poses
CN111868738B (zh) * 2018-01-11 2023-09-26 云游公司 跨设备监控计算机视觉系统
WO2019164498A1 (en) * 2018-02-23 2019-08-29 Sony Mobile Communications Inc. Methods, devices and computer program products for global bundle adjustment of 3d images
US10997746B2 (en) * 2018-04-12 2021-05-04 Honda Motor Co., Ltd. Feature descriptor matching
US10679376B2 (en) * 2018-04-24 2020-06-09 Microsoft Technology Licensing, Llc Determining a pose of a handheld object
CN108665508B (zh) * 2018-04-26 2022-04-05 腾讯科技（深圳）有限公司 一种即时定位与地图构建方法、装置及存储介质
CN110515452B (zh) * 2018-05-22 2022-02-22 腾讯科技（深圳）有限公司 图像处理方法、装置、存储介质和计算机设备
US10964053B2 (en) * 2018-07-02 2021-03-30 Microsoft Technology Licensing, Llc Device pose estimation using 3D line clouds
US11182614B2 (en) * 2018-07-24 2021-11-23 Magic Leap, Inc. Methods and apparatuses for determining and/or evaluating localizing maps of image display devices
US11227435B2 (en) * 2018-08-13 2022-01-18 Magic Leap, Inc. Cross reality system
US10825246B2 (en) * 2018-09-27 2020-11-03 Adobe Inc. Generating immersive trip photograph visualizations
US10839556B2 (en) * 2018-10-23 2020-11-17 Microsoft Technology Licensing, Llc Camera pose estimation using obfuscated features
US10767997B1 (en) * 2019-02-25 2020-09-08 Qualcomm Incorporated Systems and methods for providing immersive extended reality experiences on moving platforms
US11074706B2 (en) * 2019-04-12 2021-07-27 Intel Corporation Accommodating depth noise in visual slam using map-point consensus
US20200401617A1 (en) * 2019-06-21 2020-12-24 White Raven Ltd Visual positioning system

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2011008752A (ja) 2009-06-26 2011-01-13 Fuji Xerox Co Ltd ドキュメント操作システム、ドキュメント操作方法およびそのためのプログラム
US20170336511A1 (en) 2016-05-18 2017-11-23 Google Inc. System and method for concurrent odometry and mapping
US10504008B1 (en) 2016-07-18 2019-12-10 Occipital, Inc. System and method for relocalization and scene recognition

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
Pei-Hsuan Chiu, et al.，Cloud computing based mobile augmented reality interactive system，2014 IEEE Wireless Communications and Networking Conference (WCNC)，2014年，https://ieeexplore.ieee.org/document/6953084

Also Published As

Publication number Publication date
CN115380264A (zh) 2022-11-22
EP4104144A4 (en) 2024-06-05
JP2023514206A (ja) 2023-04-05
WO2021163289A1 (en) 2021-08-19
US20210256766A1 (en) 2021-08-19
EP4104144A1 (en) 2022-12-21
US20250244825A1 (en) 2025-07-31

Similar Documents

Publication Publication Date Title
JP7777536B2 (ja) 大規模環境のためのクロスリアリティシステム
JP7825024B2 (ja) 位置特定サービスを伴うクロスリアリティシステム
JP7677985B2 (ja) 正確な共有マップを伴うクロスリアリティシステム
JP7784489B2 (ja) 位置特定サービスおよび共有場所ベースのコンテンツを伴うクロスリアリティシステム
JP7768888B2 (ja) マルチ分解能フレーム記述子を使用したマップ処理を伴うクロスリアリティシステム
JP7604475B2 (ja) 複数のデバイスタイプをサポートするクロスリアリティシステム
JP7671769B2 (ja) 高速位置特定を伴うクロスリアリティシステム
JP7653982B2 (ja) 無線フィンガプリントを用いたクロスリアリティシステム
JP7665839B2 (ja) Ｗｉｆｉ／ｇｐｓベースのマップマージを伴うクロスリアリティシステム
JP7772975B2 (ja) 大規模環境再構築のためのクロスリアリティシステム
JP7684321B2 (ja) 位置特定に関するジオロケーション情報の優先順位化を伴うクロスリアリティシステム
JP7604478B2 (ja) 持続座標フレームについての品質情報を伴うクロスリアリティシステム
JP7748945B2 (ja) 仮想コンテンツの簡略化されたプログラミングを伴うクロスリアリティシステム
JP7586918B2 (ja) 位置特定正確度のためのバッファを伴うクロスリアリティシステム
JP2023524446A (ja) 大規模環境のためのクロスリアリティシステム
JP7711263B2 (ja) クロスリアリティシステム

Legal Events

Date Code Title Description
A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20240209

A621 Written request for application examination

Free format text: JAPANESE INTERMEDIATE CODE: A621

Effective date: 20240209

A977 Report on retrieval

Free format text: JAPANESE INTERMEDIATE CODE: A971007

Effective date: 20241122

A131 Notification of reasons for refusal

Free format text: JAPANESE INTERMEDIATE CODE: A131

Effective date: 20241205

A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20250305

A131 Notification of reasons for refusal

Free format text: JAPANESE INTERMEDIATE CODE: A131

Effective date: 20250516

A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20250808

TRDD Decision of grant or rejection written
A01 Written decision to grant a patent or to grant a registration (utility model)

Free format text: JAPANESE INTERMEDIATE CODE: A01

Effective date: 20251029

A61 First payment of annual fees (during grant procedure)

Free format text: JAPANESE INTERMEDIATE CODE: A61

Effective date: 20251117

R150 Certificate of patent or registration of utility model

Ref document number: 7777536

Country of ref document: JP

Free format text: JAPANESE INTERMEDIATE CODE: R150