KR102540594B1 - 단일 이미지로부터의 횡단 가능 공간 판정 - Google Patents

단일 이미지로부터의 횡단 가능 공간 판정 Download PDF

Info

Publication number
KR102540594B1
KR102540594B1 KR1020227035022A KR20227035022A KR102540594B1 KR 102540594 B1 KR102540594 B1 KR 102540594B1 KR 1020227035022 A KR1020227035022 A KR 1020227035022A KR 20227035022 A KR20227035022 A KR 20227035022A KR 102540594 B1 KR102540594 B1 KR 102540594B1
Authority
KR
South Korea
Prior art keywords
traversable
image
target
depth map
visible
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
KR1020227035022A
Other languages
English (en)
Korean (ko)
Other versions
KR20220143957A (ko
Inventor
제임스 왓슨
마이클 데이비드 퍼먼
아론 몬즈파트
가브리엘 제이. 브로스토
Original Assignee
나이앤틱, 인크.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 나이앤틱, 인크. filed Critical 나이앤틱, 인크.
Publication of KR20220143957A publication Critical patent/KR20220143957A/ko
Application granted granted Critical
Publication of KR102540594B1 publication Critical patent/KR102540594B1/ko
Assigned to 나이앤틱, 스파셜 인크. reassignment 나이앤틱, 스파셜 인크. 권리의 전부이전등록 Assignors: 나이앤틱, 인크.
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/50—Depth or shape recovery
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/02—Control of position or course in two dimensions
    • G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
    • G05D1/0212—Control of position or course in two dimensions specially adapted to land vehicles with means for defining a desired trajectory
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/02—Control of position or course in two dimensions
    • G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
    • G05D1/0268—Control of position or course in two dimensions specially adapted to land vehicles using internal positioning means
    • G05D1/0274—Control of position or course in two dimensions specially adapted to land vehicles using internal positioning means using mapping information stored in a memory device
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three-dimensional [3D] modelling for computer graphics
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating three-dimensional [3D] models or images for computer graphics
    • G06T19/006—Mixed reality
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating three-dimensional [3D] models or images for computer graphics
    • G06T19/20—Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/11—Region-based segmentation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/10—Terrestrial scenes
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2219/00—Indexing scheme for manipulating 3D models or images for computer graphics
    • G06T2219/20—Indexing scheme for editing of 3D models
    • G06T2219/2004—Aligning objects, relative positioning of parts

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • Computer Graphics (AREA)
  • Computer Hardware Design (AREA)
  • Computing Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Medical Informatics (AREA)
  • Mathematical Physics (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Architecture (AREA)
  • Databases & Information Systems (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Aviation & Aerospace Engineering (AREA)
  • Remote Sensing (AREA)
  • Automation & Control Theory (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Molecular Biology (AREA)
  • Biomedical Technology (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Geometry (AREA)
  • Processing Or Creating Images (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)
  • Control And Safety Of Cranes (AREA)
  • Valve Device For Special Equipments (AREA)
  • Shafts, Cranks, Connecting Bars, And Related Bearings (AREA)
  • Traffic Control Systems (AREA)
KR1020227035022A 2020-03-10 2021-03-09 단일 이미지로부터의 횡단 가능 공간 판정 Active KR102540594B1 (ko)

Applications Claiming Priority (5)

Application Number Priority Date Filing Date Title
US202062987849P 2020-03-10 2020-03-10
US62/987,849 2020-03-10
US17/193,878 US11741675B2 (en) 2020-03-10 2021-03-05 Determining traversable space from single images
US17/193,878 2021-03-05
PCT/IB2021/051947 WO2021181267A1 (en) 2020-03-10 2021-03-09 Determining traversable space from single images

Publications (2)

Publication Number Publication Date
KR20220143957A KR20220143957A (ko) 2022-10-25
KR102540594B1 true KR102540594B1 (ko) 2023-06-08

Family

ID=77665508

Family Applications (1)

Application Number Title Priority Date Filing Date
KR1020227035022A Active KR102540594B1 (ko) 2020-03-10 2021-03-09 단일 이미지로부터의 횡단 가능 공간 판정

Country Status (9)

Country Link
US (2) US11741675B2 (https=)
EP (1) EP4118618B1 (https=)
JP (1) JP7763183B2 (https=)
KR (1) KR102540594B1 (https=)
CN (1) CN115485718A (https=)
AU (1) AU2021235397B2 (https=)
CA (1) CA3175003C (https=)
TW (1) TWI797571B (https=)
WO (1) WO2021181267A1 (https=)

Families Citing this family (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR102647336B1 (ko) * 2019-04-01 2024-03-14 구글 엘엘씨 동적 깊이 이미지를 캡처 및 편집하는 기술
FR3106659B1 (fr) * 2020-01-23 2023-07-28 Second Bridge Inc Carte numérique pour la navigation
US11741675B2 (en) * 2020-03-10 2023-08-29 Niantic, Inc. Determining traversable space from single images
US20230260240A1 (en) * 2021-03-11 2023-08-17 Quintar, Inc. Alignment of 3d graphics extending beyond frame in augmented reality system with remote presentation
US12310283B2 (en) * 2021-05-25 2025-05-27 Scythe Robotics, Inc. Method and apparatus for modeling an environment proximate an autonomous system
US20240414427A1 (en) * 2021-09-21 2024-12-12 Sony Group Corporation Information processing device, video processing method, and program
US11636658B1 (en) * 2022-05-04 2023-04-25 Google Llc Dynamic augmented reality overlay display
US20240394839A1 (en) * 2023-05-23 2024-11-28 Disney Enterprises, Inc. Artistically controllable stereo conversion
TWI907823B (zh) * 2023-07-31 2025-12-11 威盛電子股份有限公司 應用於基於深度學習之單目深度預測的模型訓練方法與模型訓練系統
TWI854919B (zh) * 2023-12-13 2024-09-01 中華電信股份有限公司 虛擬或現實場景之人工智慧內容生成系統、方法及其電腦可讀媒介
CN118334592B (zh) * 2024-06-13 2024-09-10 中铁四局集团有限公司 图像检测方法、存储介质和电子设备

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130136299A1 (en) 2011-11-30 2013-05-30 Samsung Electronics Co., Ltd. Method and apparatus for recovering depth information of image
US20190057509A1 (en) 2017-08-16 2019-02-21 Nvidia Corporation Learning rigidity of dynamic scenes for three-dimensional scene flow estimation
JP2019537080A (ja) 2016-10-11 2019-12-19 モービルアイ ビジョン テクノロジーズ リミテッド 検出された障壁に基づく車両のナビゲーション

Family Cites Families (19)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8094928B2 (en) 2005-11-14 2012-01-10 Microsoft Corporation Stereo video for gaming
US20130063556A1 (en) * 2011-09-08 2013-03-14 Prism Skylabs, Inc. Extracting depth information from video from a single camera
US9782668B1 (en) 2012-07-31 2017-10-10 Niantic, Inc. Placement of virtual elements in a virtual world associated with a location-based parallel reality game
US10977662B2 (en) * 2014-04-28 2021-04-13 RetailNext, Inc. Methods and systems for simulating agent behavior in a virtual environment
US9798943B2 (en) * 2014-06-09 2017-10-24 I.R.I.S. Optical character recognition method
US9304003B1 (en) 2015-03-18 2016-04-05 Microsoft Technology Licensing, Llc Augmented reality navigation
US10602126B2 (en) 2016-06-10 2020-03-24 Lucid VR, Inc. Digital camera device for 3D imaging
KR101982298B1 (ko) 2016-11-24 2019-05-24 고려대학교 산학협력단 차량에서의 돌발적으로 출현한 이동 객체를 탐지하는 방법 및 장치
WO2018123641A1 (ja) 2016-12-27 2018-07-05 日立オートモティブシステムズ株式会社 走行可能領域検出装置及び走行支援システム
US10007269B1 (en) 2017-06-23 2018-06-26 Uber Technologies, Inc. Collision-avoidance system for autonomous-capable vehicle
US10678257B2 (en) * 2017-09-28 2020-06-09 Nec Corporation Generating occlusion-aware bird eye view representations of complex road scenes
US11448508B2 (en) * 2017-10-13 2022-09-20 Kohl's, Inc. Systems and methods for autonomous generation of maps
WO2019222467A1 (en) 2018-05-17 2019-11-21 Niantic, Inc. Self-supervised training of a depth estimation system
KR102192899B1 (ko) 2018-08-16 2020-12-18 주식회사 날비컴퍼니 이미지에 보케 효과를 적용하는 방법 및 기록매체
CN109215080B (zh) 2018-09-25 2020-08-11 清华大学 基于深度学习迭代匹配的6d姿态估计网络训练方法及装置
US11017586B2 (en) * 2019-04-18 2021-05-25 Adobe Inc. 3D motion effect from a 2D image
CN110400322B (zh) 2019-07-30 2021-03-16 江南大学 基于颜色和三维几何信息的水果点云分割方法
EP4058983A4 (en) * 2019-11-12 2024-08-07 Geomagical Labs, Inc. METHOD AND SYSTEM FOR MODIFYING SCENE IMAGES
US11741675B2 (en) * 2020-03-10 2023-08-29 Niantic, Inc. Determining traversable space from single images

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130136299A1 (en) 2011-11-30 2013-05-30 Samsung Electronics Co., Ltd. Method and apparatus for recovering depth information of image
JP2019537080A (ja) 2016-10-11 2019-12-19 モービルアイ ビジョン テクノロジーズ リミテッド 検出された障壁に基づく車両のナビゲーション
US20190057509A1 (en) 2017-08-16 2019-02-21 Nvidia Corporation Learning rigidity of dynamic scenes for three-dimensional scene flow estimation

Also Published As

Publication number Publication date
JP2023517661A (ja) 2023-04-26
TW202147179A (zh) 2021-12-16
EP4118618B1 (en) 2025-05-07
TWI797571B (zh) 2023-04-01
EP4118618A4 (en) 2024-04-10
CN115485718A (zh) 2022-12-16
US11741675B2 (en) 2023-08-29
US20210287385A1 (en) 2021-09-16
AU2021235397B2 (en) 2023-08-31
US12573153B2 (en) 2026-03-10
KR20220143957A (ko) 2022-10-25
CA3175003C (en) 2023-08-01
CA3175003A1 (en) 2021-09-16
JP7763183B2 (ja) 2025-10-31
US20230360339A1 (en) 2023-11-09
WO2021181267A1 (en) 2021-09-16
AU2021235397A1 (en) 2022-11-03
EP4118618A1 (en) 2023-01-18

Similar Documents

Publication Publication Date Title
KR102540594B1 (ko) 단일 이미지로부터의 횡단 가능 공간 판정
KR102493185B1 (ko) 깊이 힌트를 사용하는 깊이 추정 모델의 자가 감독 훈련
TW202238068A (zh) 自監督多圖框單眼深度估計模型
JP2025500929A (ja) 入力画像データからの高速リアルタイムシーン再構築
US11836965B2 (en) Determining visual overlap of images by using box embeddings

Legal Events

Date Code Title Description
A302 Request for accelerated examination
PA0105 International application

St.27 status event code: A-0-1-A10-A15-nap-PA0105

PA0201 Request for examination

St.27 status event code: A-1-2-D10-D11-exm-PA0201

PA0302 Request for accelerated examination

St.27 status event code: A-1-2-D10-D17-exm-PA0302

St.27 status event code: A-1-2-D10-D16-exm-PA0302

PG1501 Laying open of application

St.27 status event code: A-1-1-Q10-Q12-nap-PG1501

P22-X000 Classification modified

St.27 status event code: A-2-2-P10-P22-nap-X000

P22-X000 Classification modified

St.27 status event code: A-2-2-P10-P22-nap-X000

E902 Notification of reason for refusal
PE0902 Notice of grounds for rejection

St.27 status event code: A-1-2-D10-D21-exm-PE0902

P11-X000 Amendment of application requested

St.27 status event code: A-2-2-P10-P11-nap-X000

P13-X000 Application amended

St.27 status event code: A-2-2-P10-P13-nap-X000

E701 Decision to grant or registration of patent right
PE0701 Decision of registration

St.27 status event code: A-1-2-D10-D22-exm-PE0701

GRNT Written decision to grant
PR0701 Registration of establishment

St.27 status event code: A-2-4-F10-F11-exm-PR0701

PR1002 Payment of registration fee

St.27 status event code: A-2-2-U10-U12-oth-PR1002

Fee payment year number: 1

PG1601 Publication of registration

St.27 status event code: A-4-4-Q10-Q13-nap-PG1601

PN2301 Change of applicant

St.27 status event code: A-5-5-R10-R11-asn-PN2301

R11 Change to the name of applicant or owner or transfer of ownership requested

Free format text: ST27 STATUS EVENT CODE: A-5-5-R10-R11-ASN-PN2301 (AS PROVIDED BY THE NATIONAL OFFICE)

PN2301 Change of applicant

St.27 status event code: A-5-5-R10-R14-asn-PN2301

R14 Transfer of ownership recorded

Free format text: ST27 STATUS EVENT CODE: A-5-5-R10-R14-ASN-PN2301 (AS PROVIDED BY THE NATIONAL OFFICE)

PR1001 Payment of annual fee

St.27 status event code: A-4-4-U10-U11-oth-PR1001

Fee payment year number: 4

U11 Full renewal or maintenance fee paid

Free format text: ST27 STATUS EVENT CODE: A-4-4-U10-U11-OTH-PR1001 (AS PROVIDED BY THE NATIONAL OFFICE)

Year of fee payment: 4