JP7763183B2 - 単一画像からの通行可能空間の決定方法 - Google Patents

単一画像からの通行可能空間の決定方法

Info

Publication number
JP7763183B2
JP7763183B2 JP2022554751A JP2022554751A JP7763183B2 JP 7763183 B2 JP7763183 B2 JP 7763183B2 JP 2022554751 A JP2022554751 A JP 2022554751A JP 2022554751 A JP2022554751 A JP 2022554751A JP 7763183 B2 JP7763183 B2 JP 7763183B2
Authority
JP
Japan
Prior art keywords
space
target
image
depth map
traversable
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
JP2022554751A
Other languages
English (en)
Japanese (ja)
Other versions
JP2023517661A (ja
JP2023517661A5 (https=
JPWO2021181267A5 (https=
Inventor
ワトソン ジェームズ
デイビッド ファーマン マイケル
モンスズパート アーロン
ジェイ．ブロストウ ガブリエル
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Niantic Inc
Original Assignee
Niantic Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Niantic Inc filed Critical Niantic Inc
Publication of JP2023517661A publication Critical patent/JP2023517661A/ja
Publication of JP2023517661A5 publication Critical patent/JP2023517661A5/ja
Publication of JPWO2021181267A5 publication Critical patent/JPWO2021181267A5/ja
Application granted granted Critical
Publication of JP7763183B2 publication Critical patent/JP7763183B2/ja
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/02—Control of position or course in two dimensions
    • G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
    • G05D1/0212—Control of position or course in two dimensions specially adapted to land vehicles with means for defining a desired trajectory
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/02—Control of position or course in two dimensions
    • G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
    • G05D1/0268—Control of position or course in two dimensions specially adapted to land vehicles using internal positioning means
    • G05D1/0274—Control of position or course in two dimensions specially adapted to land vehicles using internal positioning means using mapping information stored in a memory device
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three-dimensional [3D] modelling for computer graphics
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating three-dimensional [3D] models or images for computer graphics
    • G06T19/006—Mixed reality
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating three-dimensional [3D] models or images for computer graphics
    • G06T19/20—Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/11—Region-based segmentation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/50—Depth or shape recovery
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/10—Terrestrial scenes
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2219/00—Indexing scheme for manipulating 3D models or images for computer graphics
    • G06T2219/20—Indexing scheme for editing of 3D models
    • G06T2219/2004—Aligning objects, relative positioning of parts

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • Computer Graphics (AREA)
  • Computer Hardware Design (AREA)
  • Computing Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Medical Informatics (AREA)
  • Mathematical Physics (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Architecture (AREA)
  • Databases & Information Systems (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Aviation & Aerospace Engineering (AREA)
  • Remote Sensing (AREA)
  • Automation & Control Theory (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Molecular Biology (AREA)
  • Biomedical Technology (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Geometry (AREA)
  • Processing Or Creating Images (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)
  • Control And Safety Of Cranes (AREA)
  • Valve Device For Special Equipments (AREA)
  • Shafts, Cranks, Connecting Bars, And Related Bearings (AREA)
  • Traffic Control Systems (AREA)
JP2022554751A 2020-03-10 2021-03-09 単一画像からの通行可能空間の決定方法 Active JP7763183B2 (ja)

Applications Claiming Priority (5)

Application Number Priority Date Filing Date Title
US202062987849P 2020-03-10 2020-03-10
US62/987,849 2020-03-10
US17/193,878 US11741675B2 (en) 2020-03-10 2021-03-05 Determining traversable space from single images
US17/193,878 2021-03-05
PCT/IB2021/051947 WO2021181267A1 (en) 2020-03-10 2021-03-09 Determining traversable space from single images

Publications (4)

Publication Number Publication Date
JP2023517661A JP2023517661A (ja) 2023-04-26
JP2023517661A5 JP2023517661A5 (https=) 2024-03-12
JPWO2021181267A5 JPWO2021181267A5 (https=) 2024-03-12
JP7763183B2 true JP7763183B2 (ja) 2025-10-31

Family

ID=77665508

Family Applications (1)

Application Number Title Priority Date Filing Date
JP2022554751A Active JP7763183B2 (ja) 2020-03-10 2021-03-09 単一画像からの通行可能空間の決定方法

Country Status (9)

Country Link
US (2) US11741675B2 (https=)
EP (1) EP4118618B1 (https=)
JP (1) JP7763183B2 (https=)
KR (1) KR102540594B1 (https=)
CN (1) CN115485718A (https=)
AU (1) AU2021235397B2 (https=)
CA (1) CA3175003C (https=)
TW (1) TWI797571B (https=)
WO (1) WO2021181267A1 (https=)

Families Citing this family (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR102647336B1 (ko) * 2019-04-01 2024-03-14 구글 엘엘씨 동적 깊이 이미지를 캡처 및 편집하는 기술
FR3106659B1 (fr) * 2020-01-23 2023-07-28 Second Bridge Inc Carte numérique pour la navigation
US11741675B2 (en) * 2020-03-10 2023-08-29 Niantic, Inc. Determining traversable space from single images
US20230260240A1 (en) * 2021-03-11 2023-08-17 Quintar, Inc. Alignment of 3d graphics extending beyond frame in augmented reality system with remote presentation
US12310283B2 (en) * 2021-05-25 2025-05-27 Scythe Robotics, Inc. Method and apparatus for modeling an environment proximate an autonomous system
US20240414427A1 (en) * 2021-09-21 2024-12-12 Sony Group Corporation Information processing device, video processing method, and program
US11636658B1 (en) * 2022-05-04 2023-04-25 Google Llc Dynamic augmented reality overlay display
US20240394839A1 (en) * 2023-05-23 2024-11-28 Disney Enterprises, Inc. Artistically controllable stereo conversion
TWI907823B (zh) * 2023-07-31 2025-12-11 威盛電子股份有限公司 應用於基於深度學習之單目深度預測的模型訓練方法與模型訓練系統
TWI854919B (zh) * 2023-12-13 2024-09-01 中華電信股份有限公司 虛擬或現實場景之人工智慧內容生成系統、方法及其電腦可讀媒介
CN118334592B (zh) * 2024-06-13 2024-09-10 中铁四局集团有限公司 图像检测方法、存储介质和电子设备

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2019008796A (ja) 2017-06-23 2019-01-17 ウーバー テクノロジーズ，インコーポレイテッド 自律可能車両用衝突回避システム
CN110088801A (zh) 2016-12-27 2019-08-02 日立汽车系统株式会社 可行驶区域检测装置以及行驶辅助系统
KR20200020646A (ko) 2018-08-16 2020-02-26 주식회사 날비컴퍼니 이미지에 보케 효과를 적용하는 방법 및 기록매체

Family Cites Families (19)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8094928B2 (en) 2005-11-14 2012-01-10 Microsoft Corporation Stereo video for gaming
US20130063556A1 (en) * 2011-09-08 2013-03-14 Prism Skylabs, Inc. Extracting depth information from video from a single camera
KR101918057B1 (ko) 2011-11-30 2019-01-30 삼성전자주식회사 이미지의 깊이 정보를 복원하는 방법 및 장치
US9782668B1 (en) 2012-07-31 2017-10-10 Niantic, Inc. Placement of virtual elements in a virtual world associated with a location-based parallel reality game
US10977662B2 (en) * 2014-04-28 2021-04-13 RetailNext, Inc. Methods and systems for simulating agent behavior in a virtual environment
US9798943B2 (en) * 2014-06-09 2017-10-24 I.R.I.S. Optical character recognition method
US9304003B1 (en) 2015-03-18 2016-04-05 Microsoft Technology Licensing, Llc Augmented reality navigation
US10602126B2 (en) 2016-06-10 2020-03-24 Lucid VR, Inc. Digital camera device for 3D imaging
KR20190062390A (ko) 2016-10-11 2019-06-05 모빌아이 비젼 테크놀로지스 엘티디. 검출된 배리어에 기반한 차량의 항법
KR101982298B1 (ko) 2016-11-24 2019-05-24 고려대학교 산학협력단 차량에서의 돌발적으로 출현한 이동 객체를 탐지하는 방법 및 장치
US10929987B2 (en) * 2017-08-16 2021-02-23 Nvidia Corporation Learning rigidity of dynamic scenes for three-dimensional scene flow estimation
US10678257B2 (en) * 2017-09-28 2020-06-09 Nec Corporation Generating occlusion-aware bird eye view representations of complex road scenes
US11448508B2 (en) * 2017-10-13 2022-09-20 Kohl's, Inc. Systems and methods for autonomous generation of maps
WO2019222467A1 (en) 2018-05-17 2019-11-21 Niantic, Inc. Self-supervised training of a depth estimation system
CN109215080B (zh) 2018-09-25 2020-08-11 清华大学 基于深度学习迭代匹配的6d姿态估计网络训练方法及装置
US11017586B2 (en) * 2019-04-18 2021-05-25 Adobe Inc. 3D motion effect from a 2D image
CN110400322B (zh) 2019-07-30 2021-03-16 江南大学 基于颜色和三维几何信息的水果点云分割方法
EP4058983A4 (en) * 2019-11-12 2024-08-07 Geomagical Labs, Inc. METHOD AND SYSTEM FOR MODIFYING SCENE IMAGES
US11741675B2 (en) * 2020-03-10 2023-08-29 Niantic, Inc. Determining traversable space from single images

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110088801A (zh) 2016-12-27 2019-08-02 日立汽车系统株式会社 可行驶区域检测装置以及行驶辅助系统
JP2019008796A (ja) 2017-06-23 2019-01-17 ウーバー テクノロジーズ，インコーポレイテッド 自律可能車両用衝突回避システム
KR20200020646A (ko) 2018-08-16 2020-02-26 주식회사 날비컴퍼니 이미지에 보케 효과를 적용하는 방법 및 기록매체

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
Xiaochuan Wang et al.，"Viewpoint-Predicting-Based Remote Rendering on Mobile Devices Using Multiple Depth Images"，2015 International Conference on Virtual Reality and Visualization (ICVRV)[online]，IEEE，2015年，pp.216-223，[検索日 2024.12.25], インターネット:<URL:https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=7467239>，DOI: 10.1109/ICVRV.2015.23
竹村岩朗 外３名，自動運転ソフトウェアを活用した死角領域の半隠消表示，第２３回日本バーチャルリアリティ学会大会，日本，2018年09月19日

Also Published As

Publication number Publication date
JP2023517661A (ja) 2023-04-26
TW202147179A (zh) 2021-12-16
EP4118618B1 (en) 2025-05-07
TWI797571B (zh) 2023-04-01
EP4118618A4 (en) 2024-04-10
CN115485718A (zh) 2022-12-16
US11741675B2 (en) 2023-08-29
US20210287385A1 (en) 2021-09-16
AU2021235397B2 (en) 2023-08-31
US12573153B2 (en) 2026-03-10
KR20220143957A (ko) 2022-10-25
CA3175003C (en) 2023-08-01
CA3175003A1 (en) 2021-09-16
US20230360339A1 (en) 2023-11-09
KR102540594B1 (ko) 2023-06-08
WO2021181267A1 (en) 2021-09-16
AU2021235397A1 (en) 2022-11-03
EP4118618A1 (en) 2023-01-18

Similar Documents

Publication Publication Date Title
JP7763183B2 (ja) 単一画像からの通行可能空間の決定方法
AU2021206838B2 (en) Self-supervised training of a depth estimation model using depth hints
US12159358B2 (en) High-speed real-time scene reconstruction from input image data
US12073601B2 (en) Determining visual overlap of images by using box embeddings

Legal Events

Date Code Title Description
A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20240301

A621 Written request for application examination

Free format text: JAPANESE INTERMEDIATE CODE: A621

Effective date: 20240301

A977 Report on retrieval

Free format text: JAPANESE INTERMEDIATE CODE: A971007

Effective date: 20241224

A131 Notification of reasons for refusal

Free format text: JAPANESE INTERMEDIATE CODE: A131

Effective date: 20250107

A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20250403

A131 Notification of reasons for refusal

Free format text: JAPANESE INTERMEDIATE CODE: A131

Effective date: 20250603

A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20250902

TRDD Decision of grant or rejection written
A01 Written decision to grant a patent or to grant a registration (utility model)

Free format text: JAPANESE INTERMEDIATE CODE: A01

Effective date: 20250924

A61 First payment of annual fees (during grant procedure)

Free format text: JAPANESE INTERMEDIATE CODE: A61

Effective date: 20251021

R150 Certificate of patent or registration of utility model

Ref document number: 7763183

Country of ref document: JP

Free format text: JAPANESE INTERMEDIATE CODE: R150

S111 Request for change of ownership or part of ownership

Free format text: JAPANESE INTERMEDIATE CODE: R313113

R350 Written notification of registration of transfer

Free format text: JAPANESE INTERMEDIATE CODE: R350