CA3221973A1 - Depth segmentation in multi-view videos - Google Patents

Depth segmentation in multi-view videos Download PDF

Info

Publication number
CA3221973A1
CA3221973A1 CA3221973A CA3221973A CA3221973A1 CA 3221973 A1 CA3221973 A1 CA 3221973A1 CA 3221973 A CA3221973 A CA 3221973A CA 3221973 A CA3221973 A CA 3221973A CA 3221973 A1 CA3221973 A1 CA 3221973A1
Authority
CA
Canada
Prior art keywords
patch
depth
source view
patches
maps
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CA3221973A
Other languages
English (en)
French (fr)
Inventor
Christiaan Varekamp
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Koninklijke Philips NV
Original Assignee
Koninklijke Philips NV
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Koninklijke Philips NV filed Critical Koninklijke Philips NV
Publication of CA3221973A1 publication Critical patent/CA3221973A1/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T15/00—Three-dimensional [3D] image rendering
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/13—Edge detection
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/18—Image warping, e.g. rearranging pixels individually
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/136—Segmentation; Edge detection involving thresholding
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/194—Segmentation; Edge detection involving foreground-background segmentation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/50—Depth or shape recovery
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/50—Depth or shape recovery
    • G06T7/55—Depth or shape recovery from multiple images
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
    • G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
    • G06V10/751—Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
    • H04N13/106—Processing image signals
    • H04N13/139—Format conversion, e.g. of frame-rate or size
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
    • H04N13/106—Processing image signals
    • H04N13/161—Encoding, multiplexing or demultiplexing different image signal components
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10028—Range image; Depth image; 3D point clouds
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20212—Image combination
    • G06T2207/20221—Image fusion; Image merging
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20212—Image combination
    • G06T2207/20224—Image subtraction
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/20—Image signal generators
    • H04N13/275—Image signal generators from three-dimensional [3D] object models, e.g. computer-generated stereoscopic image signals
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N2213/00—Details of stereoscopic systems
    • H04N2213/003—Aspects relating to the "2D+depth" image format

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Databases & Information Systems (AREA)
  • Artificial Intelligence (AREA)
  • Computing Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Software Systems (AREA)
  • Computer Graphics (AREA)
  • Image Processing (AREA)
  • Processing Or Creating Images (AREA)
  • Testing, Inspecting, Measuring Of Stereoscopic Televisions And Televisions (AREA)
  • Image Analysis (AREA)
CA3221973A 2021-06-03 2022-05-25 Depth segmentation in multi-view videos Pending CA3221973A1 (en)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
EP21177608.3A EP4099270A1 (en) 2021-06-03 2021-06-03 Depth segmentation in multi-view videos
EP21177608.3 2021-06-03
PCT/EP2022/064243 WO2022253677A1 (en) 2021-06-03 2022-05-25 Depth segmentation in multi-view videos

Publications (1)

Publication Number Publication Date
CA3221973A1 true CA3221973A1 (en) 2022-12-08

Family

ID=76269657

Family Applications (1)

Application Number Title Priority Date Filing Date
CA3221973A Pending CA3221973A1 (en) 2021-06-03 2022-05-25 Depth segmentation in multi-view videos

Country Status (9)

Country Link
US (1) US12430836B2 (https=)
EP (2) EP4099270A1 (https=)
JP (1) JP2024522463A (https=)
KR (1) KR20240016401A (https=)
CN (1) CN117413294A (https=)
BR (1) BR112023025252A2 (https=)
CA (1) CA3221973A1 (https=)
TW (1) TW202305746A (https=)
WO (1) WO2022253677A1 (https=)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20250193443A1 (en) * 2023-12-07 2025-06-12 Electronics And Telecommunications Research Institute Method and apparatus for encoding multi plane image based volumetric video

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20240257369A1 (en) * 2023-01-31 2024-08-01 Sony Interactive Entertainment LLC Systems and methods for video data depth determination and video modification
US12567167B2 (en) * 2023-02-06 2026-03-03 Walmart Apollo, Llc Systems and methods for analyzing depth in images obtained in product storage facilities to detect outlier items
CN116168071B (zh) * 2023-02-08 2026-03-24 杭州海康机器人股份有限公司 深度数据获取方法、装置、电子设备及机器可读存储介质
US20240391695A1 (en) * 2023-05-24 2024-11-28 United States Postal Service Vehicle mounted sensors and methods of using the same
TWI900021B (zh) * 2024-05-22 2025-10-01 鴻海精密工業股份有限公司 深度估計方法、裝置、電子設備及儲存介質
CN120111200A (zh) * 2025-01-15 2025-06-06 北京天马辉电子技术有限责任公司 3d显示图像生成方法及相关设备

Family Cites Families (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP2225725A2 (en) * 2007-12-20 2010-09-08 Koninklijke Philips Electronics N.V. Segmentation of image data
WO2011036761A1 (ja) 2009-09-25 2011-03-31 株式会社東芝 多視点画像生成方法および装置
US20120236114A1 (en) 2011-03-18 2012-09-20 Te-Hao Chang Depth information generator for generating depth information output by only processing part of received images having different views, and related depth information generating method and depth adjusting apparatus thereof
CN104604232A (zh) 2012-04-30 2015-05-06 数码士控股有限公司 用于编码多视点图像的方法及装置,以及用于解码多视点图像的方法及装置
US10089740B2 (en) * 2014-03-07 2018-10-02 Fotonation Limited System and methods for depth regularization and semiautomatic interactive matting using RGB-D images
CN110291788B (zh) * 2017-02-20 2022-03-08 索尼公司 图像处理设备和图像处理方法
CN109147025B (zh) * 2018-07-11 2023-07-18 北京航空航天大学 一种面向rgbd三维重建的纹理生成方法
US10965932B2 (en) 2019-03-19 2021-03-30 Intel Corporation Multi-pass add-on tool for coherent and complete view synthesis
CN110148217A (zh) * 2019-05-24 2019-08-20 北京华捷艾米科技有限公司 一种实时三维重建方法、装置及设备
US11195283B2 (en) * 2019-07-15 2021-12-07 Google Llc Video background substraction using depth
US12230002B2 (en) * 2019-10-01 2025-02-18 Intel Corporation Object-based volumetric video coding
KR102696467B1 (ko) * 2022-09-16 2024-08-19 한국전자통신연구원 이동시점 동영상 생성 장치 및 방법

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20250193443A1 (en) * 2023-12-07 2025-06-12 Electronics And Telecommunications Research Institute Method and apparatus for encoding multi plane image based volumetric video

Also Published As

Publication number Publication date
JP2024522463A (ja) 2024-06-21
EP4348577A1 (en) 2024-04-10
CN117413294A (zh) 2024-01-16
US20240378789A1 (en) 2024-11-14
US12430836B2 (en) 2025-09-30
BR112023025252A2 (pt) 2024-02-20
WO2022253677A1 (en) 2022-12-08
EP4099270A1 (en) 2022-12-07
KR20240016401A (ko) 2024-02-06
TW202305746A (zh) 2023-02-01

Similar Documents

Publication Publication Date Title
US12430836B2 (en) Depth segmentation in multi-view videos
US11348267B2 (en) Method and apparatus for generating a three-dimensional model
US9406131B2 (en) Method and system for generating a 3D representation of a dynamically changing 3D scene
Zitnick et al. Stereo for image-based rendering using image over-segmentation
US10078913B2 (en) Capturing an environment with objects
GB2581957A (en) Image processing to determine object thickness
US20240062402A1 (en) Depth orders for multi-view frame storing and rendering
Khan et al. A homographic framework for the fusion of multi-view silhouettes
US20050035961A1 (en) Method and system for providing a volumetric representation of a three-dimensional object
Sankoh et al. Robust billboard-based, free-viewpoint video synthesis algorithm to overcome occlusions under challenging outdoor sport scenes
Yaguchi et al. Arbitrary viewpoint video synthesis from multiple uncalibrated cameras
Bastian et al. Interactive modelling for AR applications
Chari et al. Augmented reality using over-segmentation
Kim et al. Dynamic 3d scene reconstruction in outdoor environments
US12190444B2 (en) Image-based environment reconstruction with view-dependent colour
Leung et al. Embedded voxel colouring with adaptive threshold selection using globally minimal surfaces
Kim et al. Compensated visual hull for defective segmentation and occlusion
US12488531B2 (en) Multi object surface image (MOSI) format
Kim et al. Compensated visual hull with GPU-based optimization
Papadakis et al. Virtual camera synthesis for soccer game replays
Kilner Free-Viewpoint Video for Outdoor Sporting Events
Schumann et al. A matching shader technique for model-based tracking
Chen Background Estimation with GPU Speed Up

Legal Events

Date Code Title Description
MFA Maintenance fee for application paid

Free format text: FEE DESCRIPTION TEXT: MF (APPLICATION, 3RD ANNIV.) - STANDARD

Year of fee payment: 3

U00 Fee paid

Free format text: ST27 STATUS EVENT CODE: A-1-1-U10-U00-U101 (AS PROVIDED BY THE NATIONAL OFFICE); EVENT TEXT: MAINTENANCE REQUEST RECEIVED

Effective date: 20250513

U11 Full renewal or maintenance fee paid

Free format text: ST27 STATUS EVENT CODE: A-1-1-U10-U11-U102 (AS PROVIDED BY THE NATIONAL OFFICE); EVENT TEXT: MAINTENANCE FEE PAYMENT PAID IN FULL

Effective date: 20250513