CN115380264A - 用于大规模环境的交叉现实系统 - Google Patents

用于大规模环境的交叉现实系统 Download PDF

Info

Publication number
CN115380264A
CN115380264A CN202180027900.3A CN202180027900A CN115380264A CN 115380264 A CN115380264 A CN 115380264A CN 202180027900 A CN202180027900 A CN 202180027900A CN 115380264 A CN115380264 A CN 115380264A
Authority
CN
China
Prior art keywords
map
features
user
maps
feature
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202180027900.3A
Other languages
English (en)
Chinese (zh)
Inventor
E·穆勒塔勒尔
A·佩里
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Magic Leap Inc
Original Assignee
Magic Leap Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Magic Leap Inc filed Critical Magic Leap Inc
Publication of CN115380264A publication Critical patent/CN115380264A/zh
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/012—Head tracking input arrangements
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/22—Matching criteria, e.g. proximity measures
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating three-dimensional [3D] models or images for computer graphics
    • G06T19/006—Mixed reality
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
    • G06V10/443—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components by matching or filtering
    • G06V10/449—Biologically inspired filters, e.g. difference of Gaussians [DoG] or Gabor filters
    • G06V10/451—Biologically inspired filters, e.g. difference of Gaussians [DoG] or Gabor filters with interaction between the filter responses, e.g. cortical complex cells
    • G06V10/454—Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
    • G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/98—Detection or correction of errors, e.g. by rescanning the pattern or by human intervention; Evaluation of the quality of the acquired patterns
    • G06V10/993—Evaluation of the quality of the acquired pattern
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/10—Terrestrial scenes
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/60—Type of objects
    • G06V20/64—Three-dimensional [3D] objects
    • G06V20/647—Three-dimensional [3D] objects by matching two-dimensional images to three-dimensional objects
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04W—WIRELESS COMMUNICATION NETWORKS
    • H04W4/00—Services specially adapted for wireless communication networks; Facilities therefor
    • H04W4/02—Services making use of location information
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04W—WIRELESS COMMUNICATION NETWORKS
    • H04W4/00—Services specially adapted for wireless communication networks; Facilities therefor
    • H04W4/30—Services specially adapted for particular environments, situations or purposes
    • H04W4/38—Services specially adapted for particular environments, situations or purposes for collecting sensor information

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Quality & Reliability (AREA)
  • Data Mining & Analysis (AREA)
  • Medical Informatics (AREA)
  • Databases & Information Systems (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Computing Systems (AREA)
  • Computer Graphics (AREA)
  • Computer Hardware Design (AREA)
  • Molecular Biology (AREA)
  • Biodiversity & Conservation Biology (AREA)
  • Biomedical Technology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Human Computer Interaction (AREA)
  • User Interface Of Digital Computer (AREA)
  • Processing Or Creating Images (AREA)
  • Image Analysis (AREA)
CN202180027900.3A 2020-02-13 2021-02-11 用于大规模环境的交叉现实系统 Pending CN115380264A (zh)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US202062976129P 2020-02-13 2020-02-13
US62/976,129 2020-02-13
PCT/US2021/017607 WO2021163289A1 (en) 2020-02-13 2021-02-11 Cross reality system for large scale environments

Publications (1)

Publication Number Publication Date
CN115380264A true CN115380264A (zh) 2022-11-22

Family

ID=77272775

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202180027900.3A Pending CN115380264A (zh) 2020-02-13 2021-02-11 用于大规模环境的交叉现实系统

Country Status (5)

Country Link
US (2) US20210256766A1 (https=)
EP (1) EP4104144A4 (https=)
JP (1) JP7777536B2 (https=)
CN (1) CN115380264A (https=)
WO (1) WO2021163289A1 (https=)

Families Citing this family (29)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11227435B2 (en) 2018-08-13 2022-01-18 Magic Leap, Inc. Cross reality system
WO2020036898A1 (en) 2018-08-13 2020-02-20 Magic Leap, Inc. A cross reality system
EP3861387B1 (en) 2018-10-05 2025-05-21 Magic Leap, Inc. Rendering location specific virtual content in any location
US11257294B2 (en) 2019-10-15 2022-02-22 Magic Leap, Inc. Cross reality system supporting multiple device types
JP7713936B2 (ja) 2019-10-15 2025-07-28 マジック リープ， インコーポレイテッド 位置特定サービスを伴うクロスリアリティシステムのためのデバイス、方法、およびコンピュータ可読媒体
WO2021076748A1 (en) 2019-10-15 2021-04-22 Magic Leap, Inc. Cross reality system with wireless fingerprints
EP4052086A4 (en) 2019-10-31 2023-11-15 Magic Leap, Inc. EXTENDED REALITY SYSTEM PROVIDING QUALITY INFORMATION ABOUT PERSISTENT COORDINATE FRAMES
JP7525603B2 (ja) 2019-11-12 2024-07-30 マジック リープ， インコーポレイテッド 位置特定サービスおよび共有場所ベースのコンテンツを伴うクロスリアリティシステム
JP7748945B2 (ja) 2019-12-09 2025-10-03 マジック リープ， インコーポレイテッド 仮想コンテンツの簡略化されたプログラミングを伴うクロスリアリティシステム
US20210248826A1 (en) * 2020-02-07 2021-08-12 Krikey, Inc. Surface distinction for mobile rendered augmented reality
CN115398484A (zh) 2020-02-13 2022-11-25 奇跃公司 具有用于定位的地理定位信息优先级的交叉现实系统
WO2021163306A1 (en) 2020-02-13 2021-08-19 Magic Leap, Inc. Cross reality system with accurate shared maps
JP7768888B2 (ja) 2020-02-13 2025-11-12 マジック リープ， インコーポレイテッド マルチ分解能フレーム記述子を使用したマップ処理を伴うクロスリアリティシステム
US11551430B2 (en) 2020-02-26 2023-01-10 Magic Leap, Inc. Cross reality system with fast localization
US11107290B1 (en) * 2020-02-27 2021-08-31 Samsung Electronics Company, Ltd. Depth map re-projection on user electronic devices
WO2021222371A1 (en) * 2020-04-29 2021-11-04 Magic Leap, Inc. Cross reality system for large scale environments
US11755275B2 (en) * 2020-06-29 2023-09-12 Meta Platforms Technologies, Llc Generating augmented reality experiences utilizing physical objects to represent analogous virtual objects
KR20220030641A (ko) * 2020-09-03 2022-03-11 삼성전자주식회사 전자 장치 및 전자 장치의 동작 방법
DE102020215461A1 (de) * 2020-12-08 2022-06-09 Robert Bosch Gesellschaft mit beschränkter Haftung Vorrichtung und Verfahren zum Trainieren eines Bildgenerators
US11636570B2 (en) * 2021-04-01 2023-04-25 Adobe Inc. Generating digital images utilizing high-resolution sparse attention and semantic layout manipulation neural networks
CN113658278B (zh) * 2021-08-25 2025-06-06 优奈柯恩(北京)科技有限公司 用于空间定位的方法及装置
WO2023063937A1 (en) * 2021-10-13 2023-04-20 Innopeak Technology, Inc. Methods and systems for detecting planar regions using predicted depth
TW202324172A (zh) 2021-11-10 2023-06-16 美商元平台技術有限公司 自動建立人工實境世界
US12205327B2 (en) * 2021-12-29 2025-01-21 L3Harris Technologies, Inc. Adaptive alignment system
US20250147183A1 (en) * 2022-02-10 2025-05-08 Singulos Research Inc. System and method for object comprehension
CN115406452B (zh) * 2022-08-26 2025-11-28 优地机器人(无锡)股份有限公司 一种实时定位及建图方法、装置及终端设备
CN118334075A (zh) * 2023-01-12 2024-07-12 高通科技公司 位姿追踪方法及系统、可移动设备、电子设备及存储介质
CN116342826B (zh) * 2023-05-25 2023-10-10 上海维智卓新信息科技有限公司 一种ar地图的构建方法及装置
CN116834000A (zh) * 2023-05-31 2023-10-03 南京航空航天大学 一种用于工业机器人拧钉的钉孔位置定位方法

Citations (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20020095265A1 (en) * 2000-11-30 2002-07-18 Kiyohide Satoh Information processing apparatus, mixed reality presentation apparatus, method thereof, and storage medium
US20070127816A1 (en) * 2003-08-15 2007-06-07 Ivar Balslev Computer-vision system for classification and spatial localization of bounded 3d-objects
US20100331041A1 (en) * 2009-06-26 2010-12-30 Fuji Xerox Co., Ltd. System and method for language-independent manipulations of digital copies of documents through a camera phone
US20140010407A1 (en) * 2012-07-09 2014-01-09 Microsoft Corporation Image-based localization
US20140254942A1 (en) * 2012-04-25 2014-09-11 Tencent Technology (Shenzhen) Co., Ltd. Systems and methods for obtaining information based on an image
US20140254936A1 (en) * 2013-03-11 2014-09-11 Microsoft Corporation Local feature based image compression
US20170336511A1 (en) * 2016-05-18 2017-11-23 Google Inc. System and method for concurrent odometry and mapping
US20180181802A1 (en) * 2016-12-28 2018-06-28 Adobe Systems Incorporated Recognizing combinations of body shape, pose, and clothing in three-dimensional input images
US20180284802A1 (en) * 2016-08-29 2018-10-04 PerceptIn, Inc. Fault Tolerance to Provide Robust Tracking for Autonomous Positional Awareness
US20190318502A1 (en) * 2018-04-12 2019-10-17 Honda Motor Co., Ltd. Feature descriptor matching
WO2019223463A1 (zh) * 2018-05-22 2019-11-28 腾讯科技（深圳）有限公司 图像处理方法、装置、存储介质和计算机设备
US10504008B1 (en) * 2016-07-18 2019-12-10 Occipital, Inc. System and method for relocalization and scene recognition
CN110582798A (zh) * 2017-02-28 2019-12-17 三菱电机株式会社 用于虚拟增强视觉同时定位和地图构建的系统和方法
US20200005486A1 (en) * 2018-07-02 2020-01-02 Microsoft Technology Licensing, Llc Device pose estimation using 3d line clouds
US20200020137A1 (en) * 2018-01-11 2020-01-16 Youar Inc. Cross-device supervisory computer vision system
US20200034624A1 (en) * 2018-07-24 2020-01-30 Magic Leap, Inc. Methods and apparatuses for determining and/or evaluating localizing maps of image display devices

Family Cites Families (22)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP5784818B2 (ja) * 2011-03-29 2015-09-24 クアルコム，インコーポレイテッド 拡張現実システムにおける実世界表面への仮想画像のアンカリング
US20150294189A1 (en) * 2012-07-23 2015-10-15 Selim BenHimane Method of providing image feature descriptors
GB201303707D0 (en) * 2013-03-01 2013-04-17 Tosas Bautista Martin System and method of interaction for mobile devices
US10373366B2 (en) * 2015-05-14 2019-08-06 Qualcomm Incorporated Three-dimensional model generation
GB201517101D0 (en) * 2015-09-28 2015-11-11 Univ Essex Entpr Ltd Mixed-reality system
US20180122143A1 (en) * 2016-03-15 2018-05-03 Sutherland Cook Ellwood, JR. Hybrid photonic vr/ar systems
US20180005015A1 (en) * 2016-07-01 2018-01-04 Vangogh Imaging, Inc. Sparse simultaneous localization and matching with unified tracking
US10380763B2 (en) * 2016-11-16 2019-08-13 Seiko Epson Corporation Hybrid corner and edge-based tracking
US10452133B2 (en) * 2016-12-12 2019-10-22 Microsoft Technology Licensing, Llc Interacting with an environment using a parent device and at least one companion device
US10664993B1 (en) * 2017-03-13 2020-05-26 Occipital, Inc. System for determining a pose of an object
US10600252B2 (en) * 2017-03-30 2020-03-24 Microsoft Technology Licensing, Llc Coarse relocalization using signal fingerprints
US10768693B2 (en) * 2017-04-19 2020-09-08 Magic Leap, Inc. Multimodal task execution and text editing for a wearable system
US10705598B2 (en) * 2017-05-09 2020-07-07 Microsoft Technology Licensing, Llc Tracking wearable device and handheld object poses
WO2019164498A1 (en) * 2018-02-23 2019-08-29 Sony Mobile Communications Inc. Methods, devices and computer program products for global bundle adjustment of 3d images
US10679376B2 (en) * 2018-04-24 2020-06-09 Microsoft Technology Licensing, Llc Determining a pose of a handheld object
CN108665508B (zh) * 2018-04-26 2022-04-05 腾讯科技（深圳）有限公司 一种即时定位与地图构建方法、装置及存储介质
US11227435B2 (en) * 2018-08-13 2022-01-18 Magic Leap, Inc. Cross reality system
US10825246B2 (en) * 2018-09-27 2020-11-03 Adobe Inc. Generating immersive trip photograph visualizations
US10839556B2 (en) * 2018-10-23 2020-11-17 Microsoft Technology Licensing, Llc Camera pose estimation using obfuscated features
US10767997B1 (en) * 2019-02-25 2020-09-08 Qualcomm Incorporated Systems and methods for providing immersive extended reality experiences on moving platforms
US11074706B2 (en) * 2019-04-12 2021-07-27 Intel Corporation Accommodating depth noise in visual slam using map-point consensus
US20200401617A1 (en) * 2019-06-21 2020-12-24 White Raven Ltd Visual positioning system

Patent Citations (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20020095265A1 (en) * 2000-11-30 2002-07-18 Kiyohide Satoh Information processing apparatus, mixed reality presentation apparatus, method thereof, and storage medium
US20070127816A1 (en) * 2003-08-15 2007-06-07 Ivar Balslev Computer-vision system for classification and spatial localization of bounded 3d-objects
US20100331041A1 (en) * 2009-06-26 2010-12-30 Fuji Xerox Co., Ltd. System and method for language-independent manipulations of digital copies of documents through a camera phone
US20140254942A1 (en) * 2012-04-25 2014-09-11 Tencent Technology (Shenzhen) Co., Ltd. Systems and methods for obtaining information based on an image
US20140010407A1 (en) * 2012-07-09 2014-01-09 Microsoft Corporation Image-based localization
US20140254936A1 (en) * 2013-03-11 2014-09-11 Microsoft Corporation Local feature based image compression
US20170336511A1 (en) * 2016-05-18 2017-11-23 Google Inc. System and method for concurrent odometry and mapping
US10504008B1 (en) * 2016-07-18 2019-12-10 Occipital, Inc. System and method for relocalization and scene recognition
US20180284802A1 (en) * 2016-08-29 2018-10-04 PerceptIn, Inc. Fault Tolerance to Provide Robust Tracking for Autonomous Positional Awareness
US20180181802A1 (en) * 2016-12-28 2018-06-28 Adobe Systems Incorporated Recognizing combinations of body shape, pose, and clothing in three-dimensional input images
CN110582798A (zh) * 2017-02-28 2019-12-17 三菱电机株式会社 用于虚拟增强视觉同时定位和地图构建的系统和方法
US20200020137A1 (en) * 2018-01-11 2020-01-16 Youar Inc. Cross-device supervisory computer vision system
US20190318502A1 (en) * 2018-04-12 2019-10-17 Honda Motor Co., Ltd. Feature descriptor matching
WO2019223463A1 (zh) * 2018-05-22 2019-11-28 腾讯科技（深圳）有限公司 图像处理方法、装置、存储介质和计算机设备
US20200005486A1 (en) * 2018-07-02 2020-01-02 Microsoft Technology Licensing, Llc Device pose estimation using 3d line clouds
US20200034624A1 (en) * 2018-07-24 2020-01-30 Magic Leap, Inc. Methods and apparatuses for determining and/or evaluating localizing maps of image display devices

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
PEI-HSUAN CHIU 等: "Cloud computing based mobile augmented reality interactive system", 2014 IEEE WIRELESS COMMUNICATIONS AND NETWORKING CONFERENCE (WCNC), 20 November 2014 (2014-11-20), pages 3320 - 3325 *

Also Published As

Publication number Publication date
EP4104144A4 (en) 2024-06-05
JP2023514206A (ja) 2023-04-05
WO2021163289A1 (en) 2021-08-19
US20210256766A1 (en) 2021-08-19
JP7777536B2 (ja) 2025-11-28
EP4104144A1 (en) 2022-12-21
US20250244825A1 (en) 2025-07-31

Similar Documents

Publication Publication Date Title
US12283011B2 (en) Cross reality system with accurate shared maps
JP7777536B2 (ja) 大規模環境のためのクロスリアリティシステム
JP7713936B2 (ja) 位置特定サービスを伴うクロスリアリティシステムのためのデバイス、方法、およびコンピュータ可読媒体
JP7604475B2 (ja) 複数のデバイスタイプをサポートするクロスリアリティシステム
CN115151947B (zh) 具有基于wifi/gps的地图合并的交叉现实系统
JP7772975B2 (ja) 大規模環境再構築のためのクロスリアリティシステム
JP7653982B2 (ja) 無線フィンガプリントを用いたクロスリアリティシステム
JP7671769B2 (ja) 高速位置特定を伴うクロスリアリティシステム
CN115398314B (zh) 使用多分辨率帧描述符进行地图处理的交叉现实系统
CN115176285B (zh) 利用缓冲的交叉现实系统用于定位精度
CN115398484A (zh) 具有用于定位的地理定位信息优先级的交叉现实系统
CN115803788A (zh) 用于大规模环境的交叉现实系统
JP7711263B2 (ja) クロスリアリティシステム

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination