EP4094186A4 - Procédés et systèmes d'augmentation de données de profondeur à partir d'un capteur de profondeur, par exemple avec des données provenant d'un système de caméra multivue - Google Patents

Procédés et systèmes d'augmentation de données de profondeur à partir d'un capteur de profondeur, par exemple avec des données provenant d'un système de caméra multivue

Info

Publication number
EP4094186A4
EP4094186A4 EP21744388.6A EP21744388A EP4094186A4 EP 4094186 A4 EP4094186 A4 EP 4094186A4 EP 21744388 A EP21744388 A EP 21744388A EP 4094186 A4 EP4094186 A4 EP 4094186A4
Authority
EP
European Patent Office
Prior art keywords
data
depth
augmenting
systems
methods
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
EP21744388.6A
Other languages
German (de)
English (en)
Other versions
EP4094186A1 (fr
Inventor
David Julio Colmenares
Thomas Ivan Nonn
James Andrew Youngquist
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Proprio Inc
Original Assignee
Proprio Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Proprio Inc filed Critical Proprio Inc
Publication of EP4094186A1 publication Critical patent/EP4094186A1/fr
Publication of EP4094186A4 publication Critical patent/EP4094186A4/fr
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/25—Fusion techniques
    • G06F18/253—Fusion techniques of extracted features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
    • G06T17/20—Finite element generation, e.g. wire-frame surface description, tesselation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/77—Retouching; Inpainting; Scratch removal
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/50—Depth or shape recovery
    • G06T7/55—Depth or shape recovery from multiple images
    • G06T7/557—Depth or shape recovery from multiple images from light fields, e.g. from plenoptic cameras
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
    • G06V10/806—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level of extracted features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/20—Scenes; Scene-specific elements in augmented reality scenes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/60—Type of objects
    • G06V20/64—Three-dimensional objects
    • G06V20/647—Three-dimensional objects by matching two-dimensional images to three-dimensional objects
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
    • H04N13/106—Processing image signals
    • H04N13/111—Transformation of image signals corresponding to virtual viewpoints, e.g. spatial image interpolation
    • H04N13/117—Transformation of image signals corresponding to virtual viewpoints, e.g. spatial image interpolation the virtual viewpoint locations being selected by the viewers or determined by viewer tracking
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/20—Image signal generators
    • H04N13/204—Image signal generators using stereoscopic image cameras
    • H04N13/243—Image signal generators using stereoscopic image cameras using three or more 2D image sensors
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/20—Image signal generators
    • H04N13/282—Image signal generators for generating image signals corresponding to three or more geometrical viewpoints, e.g. multi-view systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2200/00—Indexing scheme for image data processing or generation, in general
    • G06T2200/24—Indexing scheme for image data processing or generation, in general involving graphical user interfaces [GUIs]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10028—Range image; Depth image; 3D point clouds
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10052—Images from lightfield camera
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20092—Interactive image processing based on input by user
    • G06T2207/20104—Interactive definition of region of interest [ROI]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20212—Image combination
    • G06T2207/20221—Image fusion; Image merging
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30008—Bone
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30168—Image quality inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2210/00—Indexing scheme for image generation or computer graphics
    • G06T2210/41—Medical
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2210/00—Indexing scheme for image generation or computer graphics
    • G06T2210/56—Particle system, point based geometry or rendering
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N2013/0074—Stereoscopic image analysis
    • H04N2013/0081—Depth or disparity estimation from stereoscopic image signals

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Software Systems (AREA)
  • Signal Processing (AREA)
  • Computer Graphics (AREA)
  • Geometry (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • General Health & Medical Sciences (AREA)
  • Databases & Information Systems (AREA)
  • Computing Systems (AREA)
  • Medical Informatics (AREA)
  • Health & Medical Sciences (AREA)
  • Data Mining & Analysis (AREA)
  • General Engineering & Computer Science (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Image Processing (AREA)
  • Image Analysis (AREA)
  • Length Measuring Devices By Optical Means (AREA)
  • Measurement Of Optical Distance (AREA)
  • Apparatus For Radiation Diagnosis (AREA)
  • Processing Or Creating Images (AREA)
  • Testing, Inspecting, Measuring Of Stereoscopic Televisions And Televisions (AREA)
EP21744388.6A 2020-01-21 2021-01-21 Procédés et systèmes d'augmentation de données de profondeur à partir d'un capteur de profondeur, par exemple avec des données provenant d'un système de caméra multivue Pending EP4094186A4 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US202062963717P 2020-01-21 2020-01-21
PCT/US2021/014397 WO2021150741A1 (fr) 2020-01-21 2021-01-21 Procédés et systèmes d'augmentation de données de profondeur à partir d'un capteur de profondeur, par exemple avec des données provenant d'un système de caméra multivue

Publications (2)

Publication Number Publication Date
EP4094186A1 EP4094186A1 (fr) 2022-11-30
EP4094186A4 true EP4094186A4 (fr) 2024-01-24

Family

ID=76857295

Family Applications (1)

Application Number Title Priority Date Filing Date
EP21744388.6A Pending EP4094186A4 (fr) 2020-01-21 2021-01-21 Procédés et systèmes d'augmentation de données de profondeur à partir d'un capteur de profondeur, par exemple avec des données provenant d'un système de caméra multivue

Country Status (6)

Country Link
US (2) US11682165B2 (fr)
EP (1) EP4094186A4 (fr)
JP (1) JP2023511670A (fr)
AU (2) AU2021211677B2 (fr)
CA (1) CA3168799A1 (fr)
WO (1) WO2021150741A1 (fr)

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11836940B2 (en) * 2020-06-15 2023-12-05 Zebra Technologies Corporation Three-dimensional sensor acuity recovery assistance
CN117795952A (zh) * 2021-08-03 2024-03-29 镭亚股份有限公司 使用深度图的视图合成系统和方法
US20230355319A1 (en) * 2022-05-09 2023-11-09 Proprio, Inc. Methods and systems for calibrating instruments within an imaging system, such as a surgical imaging system
CN114782438B (zh) * 2022-06-20 2022-09-16 深圳市信润富联数字科技有限公司 物体点云修正方法、装置、电子设备和存储介质
US11935209B1 (en) * 2023-07-24 2024-03-19 Illuscio, Inc. Systems and methods for dynamic backfilling of a three-dimensional object

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1903303A2 (fr) * 2006-09-25 2008-03-26 Kabushiki Kaisha Topcon Procédé d'arpentage, système d'arpentage et programme de traitement des données d'arpentage
US20200005521A1 (en) * 2018-06-29 2020-01-02 Eloupes, Inc. Synthesizing an image from a virtual perspective using pixels from a physical imager array weighted based on depth error sensitivity

Family Cites Families (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7389041B2 (en) 2005-02-01 2008-06-17 Eastman Kodak Company Determining scene distance in digital camera images
DE102006055641B4 (de) * 2006-11-22 2013-01-31 Visumotion Gmbh Anordnung und Verfahren zur Aufnahme und Wiedergabe von Bildern einer Szene und/oder eines Objektes
US8780172B2 (en) * 2009-01-27 2014-07-15 Telefonaktiebolaget L M Ericsson (Publ) Depth and video co-processing
GB2490872B (en) * 2011-05-09 2015-07-29 Toshiba Res Europ Ltd Methods and systems for capturing 3d surface geometry
US9191646B2 (en) * 2011-08-29 2015-11-17 Nokia Technologies Oy Apparatus, a method and a computer program for video coding and decoding
EP2912405B1 (fr) 2012-10-29 2017-10-18 7D Surgical Inc. Système intégré d'éclairage et de détection de topologie de surface optique et procédés d'utilisation de ce dernier
CA2964512C (fr) * 2014-10-14 2018-04-24 Synaptive Medical (Barbados) Inc. Outil de reference patient
US10121064B2 (en) * 2015-04-16 2018-11-06 California Institute Of Technology Systems and methods for behavior detection using 3D tracking and machine learning
GB2556727B (en) 2015-06-22 2021-11-03 Synaptive Medical Inc System and method for mapping navigation space to patient space in a medical procedure
WO2017011892A1 (fr) * 2015-07-21 2017-01-26 Synaptive Medical (Barbados) Inc. Système et procédé de mise en correspondance d'un espace de navigation avec l'espace patient au cours d'un acte médical
US10357317B2 (en) * 2015-07-29 2019-07-23 Synaptive Medical (Barbados) Inc. Handheld scanner for rapid registration in a medical navigation system
WO2017023210A1 (fr) 2015-08-06 2017-02-09 Heptagon Micro Optics Pte. Ltd. Génération d'un nuage de points tridimensionnel et fusionné sur la base d'images capturées d'une scène
IL266041B (en) * 2016-10-18 2022-09-01 Photonic Sensors & Algorithms S L Device and method for obtaining distance information from a sight
US20200057778A1 (en) * 2017-04-11 2020-02-20 Siemens Mobility GmbH Depth image pose search with a bootstrapped-created database
US10395418B2 (en) * 2017-08-18 2019-08-27 Microsoft Technology Licensing, Llc Techniques for predictive prioritization of image portions in processing graphics
US10460509B2 (en) * 2017-11-07 2019-10-29 Dolby Laboratories Licensing Corporation Parameterizing 3D scenes for volumetric viewing
US10627901B2 (en) * 2018-09-14 2020-04-21 Facebook Technologies, Llc Vergence determination

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1903303A2 (fr) * 2006-09-25 2008-03-26 Kabushiki Kaisha Topcon Procédé d'arpentage, système d'arpentage et programme de traitement des données d'arpentage
US20200005521A1 (en) * 2018-06-29 2020-01-02 Eloupes, Inc. Synthesizing an image from a virtual perspective using pixels from a physical imager array weighted based on depth error sensitivity

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
HAHNE UWE ET AL: "Depth Imaging by Combining Time-of-Flight and On-Demand Stereo", 9 September 2009, SAT 2015 18TH INTERNATIONAL CONFERENCE, AUSTIN, TX, USA, SEPTEMBER 24-27, 2015; [LECTURE NOTES IN COMPUTER SCIENCE; LECT.NOTES COMPUTER], SPRINGER, BERLIN, HEIDELBERG, PAGE(S) 70 - 83, ISBN: 978-3-540-74549-5, XP047390412 *
See also references of WO2021150741A1 *
XIANGYIN MA ET AL: "Hybrid Scene Reconstruction by Integrating Scan Data and Stereo Image Pairs", 3-D DIGITAL IMAGING AND MODELING, 2007. 3DIM '07. SIXTH INTERNATIONAL CONFERENCE ON, IEEE, PISCATAWAY, NJ, USA, 1 August 2007 (2007-08-01), pages 393 - 399, XP031131021, ISBN: 978-0-7695-2939-4 *

Also Published As

Publication number Publication date
CA3168799A1 (fr) 2021-07-29
AU2024202120A1 (en) 2024-05-02
EP4094186A1 (fr) 2022-11-30
US20210225020A1 (en) 2021-07-22
WO2021150741A1 (fr) 2021-07-29
US11682165B2 (en) 2023-06-20
US20240005596A1 (en) 2024-01-04
AU2021211677B2 (en) 2024-01-04
AU2021211677A1 (en) 2022-08-25
JP2023511670A (ja) 2023-03-22

Similar Documents

Publication Publication Date Title
EP4094186A4 (fr) Procédés et systèmes d'augmentation de données de profondeur à partir d'un capteur de profondeur, par exemple avec des données provenant d'un système de caméra multivue
EP3652599A4 (fr) Systèmes de traitement de données distribuées pour traiter des données de capteur capturées à distance
EP3737997A4 (fr) Systèmes et procédés de rendu de données d'un environnement 3d
WO2018146558A3 (fr) Systèmes, procédés et appareils de vision stéréo et de suivi
GB2567845B (en) A method and system for combining sensor data
EP3537827A4 (fr) Procédé et système d'envoi et de réception de données
EP3682430A4 (fr) Système et procédé pour effectuer une transmission de données de détecteur de fumée à partir d'un détecteur de fumée
NZ747152A (en) Method and system for monitoring land deformation
EP3451674A4 (fr) Dispositif de traitement d'informations et procédé de traitement d'informations, et procédé d'émission de données d'images tridimensionnelles
GB201712579D0 (en) Memory control for electronic data processing system
AU2018299826A1 (en) System and method for integrating message content into a target data processing device
ZA201901278B (en) Data module management for data processing system
PT3595267T (pt) Processos, dispositivos e sistemas para a troca de dados entre um sistema de bancos de dados distribuído e dispositivos
EP3606084A4 (fr) Procédé de transmission de données concernant une image tridimensionnelle
EP3847548A4 (fr) Système et procédé de serveur de module hmi à la frontière
GB2553387B (en) Module or arrangement for, and method of, reading a target by image capture with an imaging reader having offset imaging and aiming systems
EP3573008A4 (fr) Procédé, dispositif et système de traitement d'informations d'objets de données
EP3474146A4 (fr) Procédé de traitement de données, système de mémorisation et dispositif d'échange
EP3637644A4 (fr) Procédé de saisie de données, dispositif associé et système.
SG11202112243SA (en) Method and device for aligning coordinate systems, electronic device and storage medium
GB202017199D0 (en) Data capture system and method for object dimensioning
GB2563596B (en) System and method for modeling a three dimensional space based on a two dimensional image
EP3697024A4 (fr) Procédé de traitement de données, dispositif et système de stockage distribué
EP3633890A4 (fr) Procédé, dispositif et système de transmission de données
EP3800889A4 (fr) Procédé, dispositif et système de traitement de données, dispositif de commande de système et dispositif de commande de module

Legal Events

Date Code Title Description
STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE

PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE

17P Request for examination filed

Effective date: 20220810

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

DAV Request for validation of the european patent (deleted)
DAX Request for extension of the european patent (deleted)
RAP3 Party data changed (applicant data changed or rights of an application transferred)

Owner name: PROPRIO, INC.

REG Reference to a national code

Ref country code: DE

Ref legal event code: R079

Free format text: PREVIOUS MAIN CLASS: G06K0009000000

Ipc: G06T0005000000

A4 Supplementary search report drawn up and despatched

Effective date: 20240102

RIC1 Information provided on ipc code assigned before grant

Ipc: G06T 5/00 20060101AFI20231219BHEP