CA3175003C - Determining traversable space from single images - Google Patents

Determining traversable space from single images Download PDF

Info

Publication number
CA3175003C
CA3175003C CA3175003A CA3175003A CA3175003C CA 3175003 C CA3175003 C CA 3175003C CA 3175003 A CA3175003 A CA 3175003A CA 3175003 A CA3175003 A CA 3175003A CA 3175003 C CA3175003 C CA 3175003C
Authority
CA
Canada
Prior art keywords
traversable
target
image
depth map
space
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CA3175003A
Other languages
English (en)
French (fr)
Other versions
CA3175003A1 (en
Inventor
James Watson
Michael David Firman
Aron Monszpart
Gabriel J. Brostow
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Niantic Inc
Original Assignee
Niantic Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Niantic Inc filed Critical Niantic Inc
Publication of CA3175003A1 publication Critical patent/CA3175003A1/en
Application granted granted Critical
Publication of CA3175003C publication Critical patent/CA3175003C/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating three-dimensional [3D] models or images for computer graphics
    • G06T19/006—Mixed reality
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/02—Control of position or course in two dimensions
    • G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
    • G05D1/0212—Control of position or course in two dimensions specially adapted to land vehicles with means for defining a desired trajectory
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/02—Control of position or course in two dimensions
    • G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
    • G05D1/0268—Control of position or course in two dimensions specially adapted to land vehicles using internal positioning means
    • G05D1/0274—Control of position or course in two dimensions specially adapted to land vehicles using internal positioning means using mapping information stored in a memory device
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three-dimensional [3D] modelling for computer graphics
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating three-dimensional [3D] models or images for computer graphics
    • G06T19/20—Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/11—Region-based segmentation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/50—Depth or shape recovery
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/10—Terrestrial scenes
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2219/00—Indexing scheme for manipulating 3D models or images for computer graphics
    • G06T2219/20—Indexing scheme for editing of 3D models
    • G06T2219/2004—Aligning objects, relative positioning of parts

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • Computer Graphics (AREA)
  • Computer Hardware Design (AREA)
  • Computing Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Medical Informatics (AREA)
  • Mathematical Physics (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Architecture (AREA)
  • Databases & Information Systems (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Aviation & Aerospace Engineering (AREA)
  • Remote Sensing (AREA)
  • Automation & Control Theory (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Molecular Biology (AREA)
  • Biomedical Technology (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Geometry (AREA)
  • Processing Or Creating Images (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)
  • Control And Safety Of Cranes (AREA)
  • Valve Device For Special Equipments (AREA)
  • Shafts, Cranks, Connecting Bars, And Related Bearings (AREA)
  • Traffic Control Systems (AREA)
CA3175003A 2020-03-10 2021-03-09 Determining traversable space from single images Active CA3175003C (en)

Applications Claiming Priority (5)

Application Number Priority Date Filing Date Title
US202062987849P 2020-03-10 2020-03-10
US62/987,849 2020-03-10
US17/193,878 US11741675B2 (en) 2020-03-10 2021-03-05 Determining traversable space from single images
US17/193,878 2021-03-05
PCT/IB2021/051947 WO2021181267A1 (en) 2020-03-10 2021-03-09 Determining traversable space from single images

Publications (2)

Publication Number Publication Date
CA3175003A1 CA3175003A1 (en) 2021-09-16
CA3175003C true CA3175003C (en) 2023-08-01

Family

ID=77665508

Family Applications (1)

Application Number Title Priority Date Filing Date
CA3175003A Active CA3175003C (en) 2020-03-10 2021-03-09 Determining traversable space from single images

Country Status (9)

Country Link
US (2) US11741675B2 (https=)
EP (1) EP4118618B1 (https=)
JP (1) JP7763183B2 (https=)
KR (1) KR102540594B1 (https=)
CN (1) CN115485718A (https=)
AU (1) AU2021235397B2 (https=)
CA (1) CA3175003C (https=)
TW (1) TWI797571B (https=)
WO (1) WO2021181267A1 (https=)

Families Citing this family (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR102647336B1 (ko) * 2019-04-01 2024-03-14 구글 엘엘씨 동적 깊이 이미지를 캡처 및 편집하는 기술
FR3106659B1 (fr) * 2020-01-23 2023-07-28 Second Bridge Inc Carte numérique pour la navigation
US11741675B2 (en) * 2020-03-10 2023-08-29 Niantic, Inc. Determining traversable space from single images
US20230260240A1 (en) * 2021-03-11 2023-08-17 Quintar, Inc. Alignment of 3d graphics extending beyond frame in augmented reality system with remote presentation
US12310283B2 (en) * 2021-05-25 2025-05-27 Scythe Robotics, Inc. Method and apparatus for modeling an environment proximate an autonomous system
US20240414427A1 (en) * 2021-09-21 2024-12-12 Sony Group Corporation Information processing device, video processing method, and program
US11636658B1 (en) * 2022-05-04 2023-04-25 Google Llc Dynamic augmented reality overlay display
US20240394839A1 (en) * 2023-05-23 2024-11-28 Disney Enterprises, Inc. Artistically controllable stereo conversion
TWI907823B (zh) * 2023-07-31 2025-12-11 威盛電子股份有限公司 應用於基於深度學習之單目深度預測的模型訓練方法與模型訓練系統
TWI854919B (zh) * 2023-12-13 2024-09-01 中華電信股份有限公司 虛擬或現實場景之人工智慧內容生成系統、方法及其電腦可讀媒介
CN118334592B (zh) * 2024-06-13 2024-09-10 中铁四局集团有限公司 图像检测方法、存储介质和电子设备

Family Cites Families (22)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8094928B2 (en) 2005-11-14 2012-01-10 Microsoft Corporation Stereo video for gaming
US20130063556A1 (en) * 2011-09-08 2013-03-14 Prism Skylabs, Inc. Extracting depth information from video from a single camera
KR101918057B1 (ko) 2011-11-30 2019-01-30 삼성전자주식회사 이미지의 깊이 정보를 복원하는 방법 및 장치
US9782668B1 (en) 2012-07-31 2017-10-10 Niantic, Inc. Placement of virtual elements in a virtual world associated with a location-based parallel reality game
US10977662B2 (en) * 2014-04-28 2021-04-13 RetailNext, Inc. Methods and systems for simulating agent behavior in a virtual environment
US9798943B2 (en) * 2014-06-09 2017-10-24 I.R.I.S. Optical character recognition method
US9304003B1 (en) 2015-03-18 2016-04-05 Microsoft Technology Licensing, Llc Augmented reality navigation
US10602126B2 (en) 2016-06-10 2020-03-24 Lucid VR, Inc. Digital camera device for 3D imaging
KR20190062390A (ko) 2016-10-11 2019-06-05 모빌아이 비젼 테크놀로지스 엘티디. 검출된 배리어에 기반한 차량의 항법
KR101982298B1 (ko) 2016-11-24 2019-05-24 고려대학교 산학협력단 차량에서의 돌발적으로 출현한 이동 객체를 탐지하는 방법 및 장치
WO2018123641A1 (ja) 2016-12-27 2018-07-05 日立オートモティブシステムズ株式会社 走行可能領域検出装置及び走行支援システム
US10007269B1 (en) 2017-06-23 2018-06-26 Uber Technologies, Inc. Collision-avoidance system for autonomous-capable vehicle
US10929987B2 (en) * 2017-08-16 2021-02-23 Nvidia Corporation Learning rigidity of dynamic scenes for three-dimensional scene flow estimation
US10678257B2 (en) * 2017-09-28 2020-06-09 Nec Corporation Generating occlusion-aware bird eye view representations of complex road scenes
US11448508B2 (en) * 2017-10-13 2022-09-20 Kohl's, Inc. Systems and methods for autonomous generation of maps
WO2019222467A1 (en) 2018-05-17 2019-11-21 Niantic, Inc. Self-supervised training of a depth estimation system
KR102192899B1 (ko) 2018-08-16 2020-12-18 주식회사 날비컴퍼니 이미지에 보케 효과를 적용하는 방법 및 기록매체
CN109215080B (zh) 2018-09-25 2020-08-11 清华大学 基于深度学习迭代匹配的6d姿态估计网络训练方法及装置
US11017586B2 (en) * 2019-04-18 2021-05-25 Adobe Inc. 3D motion effect from a 2D image
CN110400322B (zh) 2019-07-30 2021-03-16 江南大学 基于颜色和三维几何信息的水果点云分割方法
EP4058983A4 (en) * 2019-11-12 2024-08-07 Geomagical Labs, Inc. METHOD AND SYSTEM FOR MODIFYING SCENE IMAGES
US11741675B2 (en) * 2020-03-10 2023-08-29 Niantic, Inc. Determining traversable space from single images

Also Published As

Publication number Publication date
JP2023517661A (ja) 2023-04-26
TW202147179A (zh) 2021-12-16
EP4118618B1 (en) 2025-05-07
TWI797571B (zh) 2023-04-01
EP4118618A4 (en) 2024-04-10
CN115485718A (zh) 2022-12-16
US11741675B2 (en) 2023-08-29
US20210287385A1 (en) 2021-09-16
AU2021235397B2 (en) 2023-08-31
US12573153B2 (en) 2026-03-10
KR20220143957A (ko) 2022-10-25
CA3175003A1 (en) 2021-09-16
JP7763183B2 (ja) 2025-10-31
US20230360339A1 (en) 2023-11-09
KR102540594B1 (ko) 2023-06-08
WO2021181267A1 (en) 2021-09-16
AU2021235397A1 (en) 2022-11-03
EP4118618A1 (en) 2023-01-18

Similar Documents

Publication Publication Date Title
AU2021206838B2 (en) Self-supervised training of a depth estimation model using depth hints
CA3175003C (en) Determining traversable space from single images
US12159358B2 (en) High-speed real-time scene reconstruction from input image data
US11836965B2 (en) Determining visual overlap of images by using box embeddings

Legal Events

Date Code Title Description
EEER Examination request

Effective date: 20220908

MPN Maintenance fee for patent paid

Free format text: FEE DESCRIPTION TEXT: MF (PATENT, 4TH ANNIV.) - STANDARD

Year of fee payment: 4

U00 Fee paid

Free format text: ST27 STATUS EVENT CODE: A-4-4-U10-U00-U101 (AS PROVIDED BY THE NATIONAL OFFICE); EVENT TEXT: MAINTENANCE REQUEST RECEIVED

Effective date: 20241212

U11 Full renewal or maintenance fee paid

Free format text: ST27 STATUS EVENT CODE: A-4-4-U10-U11-U102 (AS PROVIDED BY THE NATIONAL OFFICE); EVENT TEXT: MAINTENANCE FEE PAYMENT PAID IN FULL

Effective date: 20241212

R11 Change to the name of applicant or owner or transfer of ownership requested

Free format text: ST27 STATUS EVENT CODE: A-4-4-R10-R11-R127 (AS PROVIDED BY THE NATIONAL OFFICE); EVENT TEXT: TRANSFER RECORDAL REQUEST OR RESPONSE

Effective date: 20250711

W00 Other event occurred

Free format text: ST27 STATUS EVENT CODE: A-4-4-W10-W00-W111 (AS PROVIDED BY THE NATIONAL OFFICE); EVENT TEXT: CORRESPONDENT DETERMINED COMPLIANT

Effective date: 20250711

R14 Transfer of ownership recorded

Free format text: ST27 STATUS EVENT CODE: A-4-4-R10-R14-R129 (AS PROVIDED BY THE NATIONAL OFFICE); EVENT TEXT: TRANSFER REQUIREMENTS DETERMINED COMPLIANT

Effective date: 20251101

MPN Maintenance fee for patent paid

Free format text: FEE DESCRIPTION TEXT: MF (PATENT, 5TH ANNIV.) - STANDARD

Year of fee payment: 5

U00 Fee paid

Free format text: ST27 STATUS EVENT CODE: A-4-4-U10-U00-U101 (AS PROVIDED BY THE NATIONAL OFFICE); EVENT TEXT: MAINTENANCE REQUEST RECEIVED

Effective date: 20251202

U11 Full renewal or maintenance fee paid

Free format text: ST27 STATUS EVENT CODE: A-4-4-U10-U11-U102 (AS PROVIDED BY THE NATIONAL OFFICE); EVENT TEXT: MAINTENANCE FEE PAYMENT PAID IN FULL

Effective date: 20251202