WO2020072972A8 - A cross reality system - Google Patents

A cross reality system Download PDF

Info

Publication number
WO2020072972A8
WO2020072972A8 PCT/US2019/054819 US2019054819W WO2020072972A8 WO 2020072972 A8 WO2020072972 A8 WO 2020072972A8 US 2019054819 W US2019054819 W US 2019054819W WO 2020072972 A8 WO2020072972 A8 WO 2020072972A8
Authority
WO
WIPO (PCT)
Prior art keywords
persistent
reality system
spatial information
physical world
physical
Prior art date
Application number
PCT/US2019/054819
Other languages
French (fr)
Other versions
WO2020072972A1 (en
Inventor
Anush MOHAN
Robert Blake TAYLOR
Jeremy Dwayne MIRANDA
Rafael Domingos TORRES
Daniel OLSHANSKY
Ali Shahrokni
Eran Guendelman
Nick Kramer
Ken Tossell
Samuel A. Miller
Jehangir Tajik
Ashwin Swaminathan
Lomesh AGARWAL
Prateek SINGHAL
Joel David HOLDER
Xuan Zhao
Siddharth Choudhary
Helder Toshiro SUZUKI
Hiral Honar BAROT
Christian Ivan Robert Moore
Original Assignee
Magic Leap, Inc.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Magic Leap, Inc. filed Critical Magic Leap, Inc.
Priority to JP2021518528A priority Critical patent/JP2022509731A/en
Priority to CN201980080054.4A priority patent/CN113544748A/en
Priority to EP19868457.3A priority patent/EP3861533A4/en
Publication of WO2020072972A1 publication Critical patent/WO2020072972A1/en
Publication of WO2020072972A8 publication Critical patent/WO2020072972A8/en

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating 3D models or images for computer graphics
    • G06T19/006—Mixed reality
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/25—Fusion techniques
    • G06F18/253—Fusion techniques of extracted features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
    • G06F3/04815—Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
    • G06V10/462—Salient features, e.g. scale invariant feature transforms [SIFT]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/20—Scenes; Scene-specific elements in augmented reality scenes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G06V20/49—Segmenting video sequences, i.e. computational techniques such as parsing or cutting the sequence, low-level clustering or determining units such as shots or scenes

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Evolutionary Computation (AREA)
  • General Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Software Systems (AREA)
  • Computing Systems (AREA)
  • Artificial Intelligence (AREA)
  • Health & Medical Sciences (AREA)
  • Computer Hardware Design (AREA)
  • Computer Graphics (AREA)
  • Databases & Information Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Data Mining & Analysis (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Human Computer Interaction (AREA)
  • User Interface Of Digital Computer (AREA)
  • Processing Or Creating Images (AREA)

Abstract

A cross reality system that provides an immersive user experience by storing persistent spatial information about the physical world that one or multiple user devices can access to determine position within the physical world and that applications can access to specify the position of virtual objects within the physical world. Persistent spatial information enables users to have a shared virtual, as well as physical, experience when interacting with the cross reality system. Further, persistent spatial information may be used in maps of the physical world, enabling one or multiple devices to access and localize into previously stored maps, reducing the need to map a physical space before using the cross reality system in it. Persistent spatial information may be stored as persistent coordinate frames, which may include a transformation relative to a reference orientation and information derived from images in a location corresponding to the persistent coordinate frame.
PCT/US2019/054819 2018-10-05 2019-10-04 A cross reality system WO2020072972A1 (en)

Priority Applications (3)

Application Number Priority Date Filing Date Title
JP2021518528A JP2022509731A (en) 2018-10-05 2019-10-04 Cross reality system
CN201980080054.4A CN113544748A (en) 2018-10-05 2019-10-04 Cross reality system
EP19868457.3A EP3861533A4 (en) 2018-10-05 2019-10-04 A cross reality system

Applications Claiming Priority (12)

Application Number Priority Date Filing Date Title
US201862742237P 2018-10-05 2018-10-05
US62/742,237 2018-10-05
US201962812935P 2019-03-01 2019-03-01
US62/812,935 2019-03-01
US201962815955P 2019-03-08 2019-03-08
US62/815,955 2019-03-08
US201962868786P 2019-06-28 2019-06-28
US62/868,786 2019-06-28
US201962870954P 2019-07-05 2019-07-05
US62/870,954 2019-07-05
US201962884109P 2019-08-07 2019-08-07
US62/884,109 2019-08-07

Publications (2)

Publication Number Publication Date
WO2020072972A1 WO2020072972A1 (en) 2020-04-09
WO2020072972A8 true WO2020072972A8 (en) 2021-09-23

Family

ID=70055505

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2019/054819 WO2020072972A1 (en) 2018-10-05 2019-10-04 A cross reality system

Country Status (4)

Country Link
EP (1) EP3861533A4 (en)
JP (1) JP2022509731A (en)
CN (1) CN113544748A (en)
WO (1) WO2020072972A1 (en)

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
SG11202108241QA (en) * 2020-05-26 2021-12-30 Beijing Sensetime Technology Development Co Ltd Ar scene content generation method and presentation method, apparatuses, and storage medium
US11556961B2 (en) * 2020-07-23 2023-01-17 At&T Intellectual Property I, L.P. Techniques for real-time object creation in extended reality environments
CN112465890A (en) * 2020-11-24 2021-03-09 深圳市商汤科技有限公司 Depth detection method and device, electronic equipment and computer readable storage medium
US11200754B1 (en) * 2020-12-22 2021-12-14 Accenture Global Solutions Limited Extended reality environment generation
EP4384290A1 (en) * 2021-09-16 2024-06-19 Apple Inc. Aligning scanned environments for multi-user communication sessions

Family Cites Families (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7542034B2 (en) * 2004-09-23 2009-06-02 Conversion Works, Inc. System and method for processing video images
US20080090659A1 (en) * 2006-10-12 2008-04-17 Maximino Aguilar Virtual world event notification from a persistent world game server in a logically partitioned game console
JP4292426B2 (en) * 2007-05-15 2009-07-08 ソニー株式会社 Imaging apparatus and imaging data correction method
US10629003B2 (en) * 2013-03-11 2020-04-21 Magic Leap, Inc. System and method for augmented and virtual reality
GB2506338A (en) 2012-07-30 2014-04-02 Sony Comp Entertainment Europe A method of localisation and mapping
US10025486B2 (en) * 2013-03-15 2018-07-17 Elwha Llc Cross-reality select, drag, and drop for augmented reality systems
US10262462B2 (en) * 2014-04-18 2019-04-16 Magic Leap, Inc. Systems and methods for augmented and virtual reality
US9196022B2 (en) * 2014-03-10 2015-11-24 Omnivision Technologies, Inc. Image transformation and multi-view output systems and methods
US10007868B2 (en) * 2016-09-19 2018-06-26 Adobe Systems Incorporated Font replacement based on visual similarity

Also Published As

Publication number Publication date
EP3861533A1 (en) 2021-08-11
JP2022509731A (en) 2022-01-24
WO2020072972A1 (en) 2020-04-09
EP3861533A4 (en) 2022-12-21
CN113544748A (en) 2021-10-22

Similar Documents

Publication Publication Date Title
WO2020072972A8 (en) A cross reality system
IL275461B1 (en) Methods and system for managing and displaying virtual content in a mixed reality system
EP3629290B1 (en) Localization for mobile devices
WO2021076754A8 (en) Cross reality system with localization service
US20150317832A1 (en) World-locked display quality feedback
CN110533780B (en) Image processing method and device, equipment and storage medium thereof
WO2018026900A8 (en) Systems and methods for managing, tracking, and offering employment
WO2017054004A8 (en) Systems and methods for data visualization using three-dimensional displays
WO2015033253A3 (en) Generating a 3d interactive immersive experience from a 2d static image
JP2022524683A (en) Systems and methods for rendering real-world objects using depth information
WO2016077506A8 (en) Accurate positioning of augmented reality content
WO2017184251A3 (en) Selecting portions of vehicle-captured video to use for display
GB2581659A (en) Transmission of real-time visual data to a remote recipient
CN110633617B (en) Planar detection using semantic segmentation
CN105005970A (en) Augmented reality implementation method and apparatus
KR101851303B1 (en) Apparatus and method for reconstructing 3d space
CN111651051B (en) Virtual sand table display method and device
CN108876900A (en) A kind of virtual target projective techniques merged with reality scene and system
CN105095314A (en) Point of interest (POI) marking method, terminal, navigation server and navigation system
CN104501797B (en) A kind of air navigation aid based on augmented reality IP maps
CN111712777B (en) Determining content pose of virtual content segments
CN104501798A (en) Network object positioning and tracking method based on augmented reality IP map
CN109002162B (en) Scene switching method, device, terminal and computer storage medium
CN116486051B (en) Multi-user display cooperation method, device, equipment and storage medium
CN111918114A (en) Image display method, image display device, display equipment and computer readable storage medium

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19868457

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2021518528

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

WWE Wipo information: entry into national phase

Ref document number: 2019868457

Country of ref document: EP

ENP Entry into the national phase

Ref document number: 2019868457

Country of ref document: EP

Effective date: 20210506