EP2863336A3 - System and method for reconstructing 3d model - Google Patents

System and method for reconstructing 3d model Download PDF

Info

Publication number
EP2863336A3
EP2863336A3 EP14156980.6A EP14156980A EP2863336A3 EP 2863336 A3 EP2863336 A3 EP 2863336A3 EP 14156980 A EP14156980 A EP 14156980A EP 2863336 A3 EP2863336 A3 EP 2863336A3
Authority
EP
European Patent Office
Prior art keywords
model
depth map
depth
reconstructing
low resolution
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
EP14156980.6A
Other languages
German (de)
French (fr)
Other versions
EP2863336A2 (en
Inventor
Woo-Ram Son
Han Oh
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Samsung Electronics Co Ltd
Original Assignee
Samsung Electronics Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Samsung Electronics Co Ltd filed Critical Samsung Electronics Co Ltd
Publication of EP2863336A2 publication Critical patent/EP2863336A2/en
Publication of EP2863336A3 publication Critical patent/EP2863336A3/en
Ceased legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T15/00—3D [Three Dimensional] image rendering
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
    • G01B11/00—Measuring arrangements characterised by the use of optical techniques
    • G01B11/22—Measuring arrangements characterised by the use of optical techniques for measuring depth
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/50—Depth or shape recovery
    • G06T7/55—Depth or shape recovery from multiple images
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/60—Type of objects
    • G06V20/64—Three-dimensional objects
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/64—Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10028—Range image; Depth image; 3D point clouds
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20016—Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Multimedia (AREA)
  • Computer Graphics (AREA)
  • Signal Processing (AREA)
  • Geometry (AREA)
  • Software Systems (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Processing Or Creating Images (AREA)
  • Length Measuring Devices By Optical Means (AREA)

Abstract

A system and method for reconstructing a three-dimensional (3D) model are described. The 3D model reconstruction system includes: a low resolution reconstruction unit that converts a first depth map, acquired by scanning a scene with a depth camera, into a second depth map having a low resolution, processes the second depth map to extract pose change information about a pose change of the depth camera, and reconstructs a low resolution 3D model in realtime; and a high resolution reconstruction unit that processes the first depth map by using the pose change information of the depth camera that is extracted from the second depth map and reconstructs a high resolution 3D model.
EP14156980.6A 2013-10-17 2014-02-27 System and method for reconstructing 3d model Ceased EP2863336A3 (en)

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
KR1020130124152A KR102184766B1 (en) 2013-10-17 2013-10-17 System and method for 3D model reconstruction

Publications (2)

Publication Number Publication Date
EP2863336A2 EP2863336A2 (en) 2015-04-22
EP2863336A3 true EP2863336A3 (en) 2017-05-10

Family

ID=50241090

Family Applications (1)

Application Number Title Priority Date Filing Date
EP14156980.6A Ceased EP2863336A3 (en) 2013-10-17 2014-02-27 System and method for reconstructing 3d model

Country Status (3)

Country Link
US (1) US9574874B2 (en)
EP (1) EP2863336A3 (en)
KR (1) KR102184766B1 (en)

Families Citing this family (50)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9113043B1 (en) * 2011-10-24 2015-08-18 Disney Enterprises, Inc. Multi-perspective stereoscopy from light fields
CA2902430C (en) 2013-03-15 2020-09-01 Uber Technologies, Inc. Methods, systems, and apparatus for multi-sensory stereo vision for robotics
US9373190B2 (en) * 2014-07-09 2016-06-21 Google Inc. High-quality stereo reconstruction featuring depth map alignment and outlier identification
DE102014011821A1 (en) * 2014-08-08 2016-02-11 Cargometer Gmbh Device and method for determining the volume of an object moved by an industrial truck
US9378554B2 (en) * 2014-10-09 2016-06-28 Caterpillar Inc. Real-time range map generation
US9928412B2 (en) * 2014-10-17 2018-03-27 Ebay Inc. Method, medium, and system for fast 3D model fitting and anthropometrics
KR101671649B1 (en) * 2014-12-22 2016-11-01 장석준 Method and System for 3D manipulated image combined physical data and clothing data
US9924157B2 (en) * 2015-04-08 2018-03-20 Canon Kabushiki Kaisha Image processing device, image pickup apparatus, image processing method, and storage medium
CN107534764B (en) 2015-04-30 2020-03-17 深圳市大疆创新科技有限公司 System and method for enhancing image resolution
US9934590B1 (en) 2015-06-25 2018-04-03 The United States Of America As Represented By The Secretary Of The Air Force Tchebichef moment shape descriptor for partial point cloud characterization
JP6944441B2 (en) * 2015-09-25 2021-10-06 マジック リープ， インコーポレイテッドＭａｇｉｃ Ｌｅａｐ，Ｉｎｃ． Methods and systems for detecting and combining structural features in 3D reconstruction
US9892552B2 (en) 2015-12-15 2018-02-13 Samsung Electronics Co., Ltd. Method and apparatus for creating 3-dimensional model using volumetric closest point approach
KR102016413B1 (en) * 2016-01-05 2019-09-02 한국전자통신연구원 Apparatus and method for scanning item
WO2017132636A1 (en) 2016-01-29 2017-08-03 Pointivo, Inc. Systems and methods for extracting information about objects from scene information
US10077007B2 (en) * 2016-03-14 2018-09-18 Uber Technologies, Inc. Sidepod stereo camera system for an autonomous vehicle
US20170289515A1 (en) * 2016-04-01 2017-10-05 Intel Corporation High dynamic range depth generation for 3d imaging systems
US20170359561A1 (en) * 2016-06-08 2017-12-14 Uber Technologies, Inc. Disparity mapping for an autonomous vehicle
WO2018123801A1 (en) * 2016-12-28 2018-07-05 パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカ Three-dimensional model distribution method, three-dimensional model receiving method, three-dimensional model distribution device, and three-dimensional model receiving device
EP3577631A1 (en) * 2017-02-01 2019-12-11 PCMS Holdings, Inc. System and method for augmented reality content delivery in pre-captured environments
TWI672677B (en) * 2017-03-31 2019-09-21 鈺立微電子股份有限公司 Depth map generation device for merging multiple depth maps
CN114895785A (en) * 2017-06-15 2022-08-12 杜比国际公司 System comprising means for reproducing and storing media content and related device
US10600199B2 (en) * 2017-06-27 2020-03-24 Toyota Research Institute, Inc. Extending object detection and identification capability for an object sensor device
CN109559371B (en) * 2017-09-27 2023-12-26 虹软科技股份有限公司 Method and device for three-dimensional reconstruction
US10437065B2 (en) * 2017-10-03 2019-10-08 Microsoft Technology Licensing, Llc IPD correction and reprojection for accurate mixed reality object placement
KR102357355B1 (en) 2017-10-23 2022-01-28 삼성전자주식회사 Method and apparatus for generating virtual object
CN107909643B (en) * 2017-11-06 2020-04-24 清华大学 Mixed scene reconstruction method and device based on model segmentation
US10967862B2 (en) 2017-11-07 2021-04-06 Uatc, Llc Road anomaly detection for autonomous vehicle
US11393114B1 (en) * 2017-11-08 2022-07-19 AI Incorporated Method and system for collaborative construction of a map
EP3729381B1 (en) 2017-12-22 2023-01-18 Magic Leap, Inc. Viewpoint dependent brick selection for fast volumetric reconstruction
US10962355B2 (en) * 2017-12-25 2021-03-30 Htc Corporation 3D model reconstruction method, electronic device, and non-transitory computer readable storage medium thereof
WO2019167760A1 (en) * 2018-02-27 2019-09-06 シャープ株式会社 Image processing device, display device, image processing method, control program, and recording medium
CN108537878B (en) * 2018-03-26 2020-04-21 Oppo广东移动通信有限公司 Environment model generation method and device, storage medium and electronic equipment
US10902260B2 (en) * 2018-11-29 2021-01-26 International Business Machines Corporation Estimating a height of a cloud depicted in an image
KR102181859B1 (en) * 2018-12-12 2020-11-23 한국전자기술연구원 System and method for providing holoportation
US10823562B1 (en) 2019-01-10 2020-11-03 State Farm Mutual Automobile Insurance Company Systems and methods for enhanced base map generation
CN109934909B (en) * 2019-03-04 2023-05-23 深圳市迷你玩科技有限公司 Method and device for reconstructing 3D model, electronic equipment and storage medium
EP3973513A4 (en) 2019-05-21 2023-06-28 Magic Leap, Inc. Caching and updating of dense 3d reconstruction data
EP3991091A4 (en) 2019-06-26 2023-11-08 Magic Leap, Inc. Caching and updating of dense 3d reconstruction data
EP3991142A4 (en) 2019-06-28 2023-06-28 Magic Leap, Inc. Fast hand meshing for dynamic occlusion
CN112396686A (en) * 2019-07-31 2021-02-23 鸿富锦精密电子(天津)有限公司 Three-dimensional scene engineering simulation and live-action fusion system and method
CN112785682A (en) * 2019-11-08 2021-05-11 华为技术有限公司 Model generation method, model reconstruction method and device
CN111145339B (en) * 2019-12-25 2023-06-02 Oppo广东移动通信有限公司 Image processing method and device, equipment and storage medium
CN114519763A (en) * 2020-11-19 2022-05-20 中兴通讯股份有限公司 3D depth map construction method and device and AR glasses
CN112581598B (en) * 2020-12-04 2022-08-30 深圳市慧鲤科技有限公司 Three-dimensional model construction method, device, equipment and storage medium
CN112530014B (en) * 2020-12-18 2023-07-25 北京理工大学重庆创新中心 Three-dimensional reconstruction method and device for indoor scene of multiple unmanned aerial vehicles
CN113160287B (en) * 2021-03-17 2022-04-22 华中科技大学 Complex component point cloud splicing method and system based on feature fusion
US20230041814A1 (en) * 2021-08-06 2023-02-09 Lenovo (Singapore) Pte. Ltd. System and method for demonstrating objects at remote locations
US20230073587A1 (en) * 2021-09-09 2023-03-09 The Boeing Company Automated volumetric image capture of an object to support general visual inspection
US11967024B2 (en) * 2021-11-03 2024-04-23 Nvidia Corporation Extracting triangular 3-D models, materials, and lighting from images
CN114565689B (en) * 2022-02-28 2024-02-02 燕山大学 Axisymmetric three-dimensional model data compression reconstruction method

Family Cites Families (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR100951637B1 (en) * 2007-12-17 2010-04-09 한국전자통신연구원 Stereo vision system and its video processing method
US8442355B2 (en) * 2008-05-23 2013-05-14 Samsung Electronics Co., Ltd. System and method for generating a multi-dimensional image
KR101563729B1 (en) * 2008-10-10 2015-10-27 삼성전자주식회사 Apparatus and method for processing image to generate high resolution image
WO2011008066A2 (en) * 2009-07-17 2011-01-20 Samsung Electronics Co., Ltd. Apparatus and method for converting image in an image processing system
JP5858380B2 (en) * 2010-12-03 2016-02-10 国立大学法人名古屋大学 Virtual viewpoint image composition method and virtual viewpoint image composition system
US8401225B2 (en) * 2011-01-31 2013-03-19 Microsoft Corporation Moving object segmentation using depth images
US8401242B2 (en) 2011-01-31 2013-03-19 Microsoft Corporation Real-time camera tracking using depth maps
US9070216B2 (en) * 2011-12-14 2015-06-30 The Board Of Trustees Of The University Of Illinois Four-dimensional augmented reality models for interactive visualization and automated construction progress monitoring
US9715761B2 (en) * 2013-07-08 2017-07-25 Vangogh Imaging, Inc. Real-time 3D computer vision processing engine for object recognition, reconstruction, and analysis

Non-Patent Citations (4)

* Cited by examiner, † Cited by third party
Title
"3D Online Multimedia And Games: Processing, Transmission and Visualization", 1 January 2008, WORLD SCIENTIFIC, article BRUSCO ET AL.: "A Rate Distortion Theoretic Approach to Remote Visualization of 3D Models", pages: 197 - 200, XP055359827 *
FRANK STEINBRUCKER ET AL: "Real-time visual odometry from dense RGB-D images", COMPUTER VISION WORKSHOPS (ICCV WORKSHOPS), 2011 IEEE INTERNATIONAL CONFERENCE ON, IEEE, 6 November 2011 (2011-11-06), pages 719 - 722, XP032095318, ISBN: 978-1-4673-0062-9, DOI: 10.1109/ICCVW.2011.6130321 *
RICHARD A NEWCOMBE ET AL: "KinectFusion: Real-time dense surface mapping and tracking", MIXED AND AUGMENTED REALITY (ISMAR), 2011 10TH IEEE INTERNATIONAL SYMPOSIUM ON, IEEE, 26 October 2011 (2011-10-26), pages 127 - 136, XP032201443, ISBN: 978-1-4577-2183-0, DOI: 10.1109/ISMAR.2011.6092378 *
TOMMI TYKKALA ET AL: "Direct Iterative Closest Point for real-time visual odometry", COMPUTER VISION WORKSHOPS (ICCV WORKSHOPS), 2011 IEEE INTERNATIONAL CONFERENCE ON, IEEE, 6 November 2011 (2011-11-06), pages 2050 - 2056, XP032095497, ISBN: 978-1-4673-0062-9, DOI: 10.1109/ICCVW.2011.6130500 *

Also Published As

Publication number Publication date
US20150109415A1 (en) 2015-04-23
KR20150044730A (en) 2015-04-27
EP2863336A2 (en) 2015-04-22
US9574874B2 (en) 2017-02-21
KR102184766B1 (en) 2020-11-30

Similar Documents

Publication Publication Date Title
EP2863336A3 (en) System and method for reconstructing 3d model
EP2975578A3 (en) System and method of artifact correction in 3d imaging
EP3293723A3 (en) Method, storage medium, and electronic device for displaying images
EP3115971A3 (en) Method and apparatus for providing three-dimensional data of cloth
EP2790405A3 (en) Three-dimensional image conversion apparatus for converting two-dimensional image into three-dimensional image and method for controlling the conversion apparatus
WO2017029488A3 (en) Methods of generating personalized 3d head models or 3d body models
EP2442275A3 (en) Method and apparatus for three-dimensional image reconstruction
WO2008112806A3 (en) System and method for processing video images using point clouds
EP3108850A3 (en) Generating a design for a dental restorative product from dental images
EP2866202A3 (en) Processing of light fields by transforming to scale and depth space
EP2755187A3 (en) 3d-animation effect generation method and system
GB2507830A8 (en) System and Method of Image Augmentation
EP2899694A3 (en) Method for providing a threedimensional body model
EP3021283A3 (en) Image display methods
WO2015126205A3 (en) Tomography apparatus and method for reconstructing tomography image thereof
WO2014197104A3 (en) Image-based 3d panorama
EP3528276A3 (en) Microscopy imaging method
EP2437237A3 (en) Image processing apparatus, image processing method, and computer-readable storage medium
EP2662838A3 (en) Apparatus and method for augmented reality
EP2629267A3 (en) Real-time compositing of live recording-based and computer graphics-based media streams
EP2808842A3 (en) An apparatus and method for tracking and reconstructing three-dimensional objects
EP2874395A3 (en) Method, apparatus and computer program product for disparity estimation
EP2116977A3 (en) Method for editing 3D image segmentation maps
WO2014125502A3 (en) Segmenting objects in multimedia data
EP2618303A3 (en) Image processing method and apparatus for generating disparity value

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20140227

AK Designated contracting states

Kind code of ref document: A2

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

AX Request for extension of the european patent

Extension state: BA ME

PUAL Search report despatched

Free format text: ORIGINAL CODE: 0009013

AK Designated contracting states

Kind code of ref document: A3

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

AX Request for extension of the european patent

Extension state: BA ME

RIC1 Information provided on ipc code assigned before grant

Ipc: G06K 9/00 20060101AFI20170405BHEP

R17P Request for examination filed (corrected)

Effective date: 20171010

RBV Designated contracting states (corrected)

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

17Q First examination report despatched

Effective date: 20190916

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE APPLICATION HAS BEEN REFUSED

18R Application refused

Effective date: 20200915