CN107850782B - 用反射率图表示增强深度图表示 - Google Patents

用反射率图表示增强深度图表示 Download PDF

Info

Publication number
CN107850782B
CN107850782B CN201680041776.5A CN201680041776A CN107850782B CN 107850782 B CN107850782 B CN 107850782B CN 201680041776 A CN201680041776 A CN 201680041776A CN 107850782 B CN107850782 B CN 107850782B
Authority
CN
China
Prior art keywords
representation
map representation
pattern
pattern feature
captured image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201680041776.5A
Other languages
English (en)
Chinese (zh)
Other versions
CN107850782A (zh
Inventor
尼他耶·罗马诺
纳达夫·格罗辛格
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Meta Platforms Technologies LLC
Original Assignee
Facebook Technologies LLC
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Facebook Technologies LLC filed Critical Facebook Technologies LLC
Publication of CN107850782A publication Critical patent/CN107850782A/zh
Application granted granted Critical
Publication of CN107850782B publication Critical patent/CN107850782B/zh
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/50—Depth or shape recovery
    • G06T7/521—Depth or shape recovery from laser ranging, e.g. using interferometry; from the projection of structured light
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G02B27/0149—Head-up displays characterised by mechanical features
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B30/00—Optical systems or apparatus for producing three-dimensional [3D] effects, e.g. stereoscopic images
    • G02B30/20—Optical systems or apparatus for producing three-dimensional [3D] effects, e.g. stereoscopic images by providing first and second parallax images to an observer's left and right eyes
    • G02B30/22—Optical systems or apparatus for producing three-dimensional [3D] effects, e.g. stereoscopic images by providing first and second parallax images to an observer's left and right eyes of the stereoscopic type
    • G02B30/23—Optical systems or apparatus for producing three-dimensional [3D] effects, e.g. stereoscopic images by providing first and second parallax images to an observer's left and right eyes of the stereoscopic type using wavelength separation, e.g. using anaglyph techniques
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T15/00—3D [Three Dimensional] image rendering
    • G06T15/50—Lighting effects
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G02B27/0101—Head-up displays characterised by optical features
    • G02B2027/0138—Head-up displays characterised by optical features comprising image capture systems, e.g. camera
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G02B27/0149—Head-up displays characterised by mechanical features
    • G02B2027/0154—Head-up displays characterised by mechanical features with movable elements
    • G02B2027/0156—Head-up displays characterised by mechanical features with movable elements with optionally usable elements
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2200/00—Indexing scheme for image data processing or generation, in general
    • G06T2200/04—Indexing scheme for image data processing or generation, in general involving 3D image data
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2200/00—Indexing scheme for image data processing or generation, in general
    • G06T2200/08—Indexing scheme for image data processing or generation, in general involving all processing steps from image acquisition to 3D model generation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10024—Color image
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10141—Special mode during image acquisition
    • G06T2207/10152—Varying illumination
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30196—Human being; Person
    • G06T2207/30201—Face

Landscapes

  • Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Optics & Photonics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Graphics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Length Measuring Devices By Optical Means (AREA)
  • Image Processing (AREA)
  • Processing Or Creating Images (AREA)
CN201680041776.5A 2015-05-13 2016-05-13 用反射率图表示增强深度图表示 Active CN107850782B (zh)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US201562160957P 2015-05-13 2015-05-13
US62/160,957 2015-05-13
PCT/US2016/032230 WO2016183395A1 (en) 2015-05-13 2016-05-13 Augmenting a depth map representation with a reflectivity map representation

Publications (2)

Publication Number Publication Date
CN107850782A CN107850782A (zh) 2018-03-27
CN107850782B true CN107850782B (zh) 2019-10-25

Family

ID=57248653

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201680041776.5A Active CN107850782B (zh) 2015-05-13 2016-05-13 用反射率图表示增强深度图表示

Country Status (6)

Country Link
US (1) US9947098B2 (enExample)
EP (1) EP3295239B1 (enExample)
JP (1) JP6377863B2 (enExample)
KR (1) KR101892168B1 (enExample)
CN (1) CN107850782B (enExample)
WO (1) WO2016183395A1 (enExample)

Families Citing this family (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9507411B2 (en) * 2009-09-22 2016-11-29 Facebook, Inc. Hand tracker for device with display
CN107408204B (zh) * 2015-01-06 2021-03-09 脸谱科技有限责任公司 使用图案化光提供深度图的方法和系统
US10440355B2 (en) * 2015-11-06 2019-10-08 Facebook Technologies, Llc Depth mapping with a head mounted display using stereo cameras and structured light
US10276210B2 (en) * 2015-11-18 2019-04-30 International Business Machines Corporation Video enhancement
US9947099B2 (en) 2016-07-27 2018-04-17 Microsoft Technology Licensing, Llc Reflectivity map estimate from dot based structured light systems
WO2018145113A1 (en) 2017-02-06 2018-08-09 MODit3D, INC. System and method for 3d scanning
US10620316B2 (en) * 2017-05-05 2020-04-14 Qualcomm Incorporated Systems and methods for generating a structured light depth map with a non-uniform codeword pattern
KR102457462B1 (ko) * 2018-03-06 2022-10-21 삼성전자주식회사 전자장치의 이미지 처리 장치 및 방법
US10935376B2 (en) * 2018-03-30 2021-03-02 Koninklijke Philips N.V. System and method for 3D scanning
CN109087347B (zh) * 2018-08-15 2021-08-17 浙江光珀智能科技有限公司 一种图像处理方法及装置
US11100713B2 (en) 2018-08-17 2021-08-24 Disney Enterprises, Inc. System and method for aligning virtual objects on peripheral devices in low-cost augmented reality/virtual reality slip-in systems
US10909373B1 (en) * 2018-08-24 2021-02-02 Snap Inc. Augmented reality system using structured light
CN109375232B (zh) * 2018-10-17 2021-01-26 维沃移动通信（杭州）有限公司 一种距离测量方法及装置
KR102668245B1 (ko) * 2018-10-29 2024-05-23 삼성전자주식회사 3차원 깊이 측정 장치 및 방법
CN114402588A (zh) * 2019-03-13 2022-04-26 林戈阿尔公司 具有参考发光体的白平衡

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20040143786A1 (en) * 2001-05-14 2004-07-22 Stauder Juergen Device, server, system and method to generate mutual photometric effects
US20060120593A1 (en) * 2004-12-03 2006-06-08 Takahiro Oshino 3D image generation program, 3D image generation system, and 3D image generation apparatus
US20130342651A1 (en) * 2012-06-22 2013-12-26 Microsoft Corporation Encoding data in depth patterns
CN104539925A (zh) * 2014-12-15 2015-04-22 北京邮电大学 基于深度信息的三维场景增强现实的方法及系统

Family Cites Families (31)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5878152A (en) * 1997-05-21 1999-03-02 Cognex Corporation Depth from focal gradient analysis using object texture removal by albedo normalization
JP3714063B2 (ja) * 1999-10-19 2005-11-09 富士ゼロックス株式会社 ３次元形状計測装置
US6754370B1 (en) * 2000-08-14 2004-06-22 The Board Of Trustees Of The Leland Stanford Junior University Real-time structured light range scanning of moving scenes
US7385708B2 (en) * 2002-06-07 2008-06-10 The University Of North Carolina At Chapel Hill Methods and systems for laser based real-time structured light depth extraction
JP2005258622A (ja) * 2004-03-10 2005-09-22 Fuji Photo Film Co Ltd 三次元情報取得システムおよび三次元情報取得方法
US20080309662A1 (en) 2005-12-14 2008-12-18 Tal Hassner Example Based 3D Reconstruction
WO2007105215A2 (en) * 2006-03-14 2007-09-20 Prime Sense Ltd. Depth-varying light fields for three dimensional sensing
KR101331543B1 (ko) * 2006-03-14 2013-11-20 프라임센스 엘티디. 스페클 패턴을 이용한 3차원 센싱
US20110057930A1 (en) * 2006-07-26 2011-03-10 Inneroptic Technology Inc. System and method of using high-speed, high-resolution depth extraction to provide three-dimensional imagery for endoscopy
CN101512599B (zh) * 2006-09-21 2012-07-18 汤姆森特许公司 三维模型获取的方法和系统
US7769205B2 (en) * 2006-11-28 2010-08-03 Prefixa International Inc. Fast three dimensional recovery method and apparatus
US20090189858A1 (en) * 2008-01-30 2009-07-30 Jeff Lev Gesture Identification Using A Structured Light Pattern
US8717416B2 (en) * 2008-09-30 2014-05-06 Texas Instruments Incorporated 3D camera using flash with structured light
US8462207B2 (en) * 2009-02-12 2013-06-11 Primesense Ltd. Depth ranging with Moiré patterns
WO2011013079A1 (en) * 2009-07-30 2011-02-03 Primesense Ltd. Depth mapping based on pattern matching and stereoscopic information
US8908958B2 (en) * 2009-09-03 2014-12-09 Ron Kimmel Devices and methods of generating three dimensional (3D) colored models
US20120056988A1 (en) * 2010-09-07 2012-03-08 David Stanhill 3-d camera
US8724887B2 (en) * 2011-02-03 2014-05-13 Microsoft Corporation Environmental modifications to mitigate environmental factors
US10234941B2 (en) * 2012-10-04 2019-03-19 Microsoft Technology Licensing, Llc Wearable sensor for tracking articulated body-parts
RU2012145349A (ru) * 2012-10-24 2014-05-10 ЭлЭсАй Корпорейшн Способ и устройство обработки изображений для устранения артефактов глубины
US11509880B2 (en) * 2012-11-14 2022-11-22 Qualcomm Incorporated Dynamic adjustment of light source power in structured light active depth sensing systems
WO2014153022A1 (en) * 2013-03-14 2014-09-25 University Of Southern California Specular object scanner for measuring reflectance properties of objects
US10019843B2 (en) * 2013-08-08 2018-07-10 Facebook, Inc. Controlling a near eye display
US9299188B2 (en) * 2013-08-08 2016-03-29 Adobe Systems Incorporated Automatic geometry and lighting inference for realistic image editing
US9799117B2 (en) * 2013-09-30 2017-10-24 Lenovo (Beijing) Co., Ltd. Method for processing data and apparatus thereof
US20150302648A1 (en) * 2014-02-18 2015-10-22 Sulon Technologies Inc. Systems and methods for mapping an environment using structured light
US9507995B2 (en) * 2014-08-29 2016-11-29 X Development Llc Combination of stereo and structured-light processing
US9664507B2 (en) * 2014-09-17 2017-05-30 Canon Kabushiki Kaisha Depth value measurement using illumination by pixels
US9635231B2 (en) * 2014-12-22 2017-04-25 Google Inc. Time-of-flight camera system and method to improve measurement quality of weak field-of-view signal regions
US9948920B2 (en) * 2015-02-27 2018-04-17 Qualcomm Incorporated Systems and methods for error correction in structured light
US20160324580A1 (en) * 2015-03-23 2016-11-10 Justin Esterberg Systems and methods for assisted surgical navigation

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20040143786A1 (en) * 2001-05-14 2004-07-22 Stauder Juergen Device, server, system and method to generate mutual photometric effects
US20060120593A1 (en) * 2004-12-03 2006-06-08 Takahiro Oshino 3D image generation program, 3D image generation system, and 3D image generation apparatus
US20130342651A1 (en) * 2012-06-22 2013-12-26 Microsoft Corporation Encoding data in depth patterns
CN104539925A (zh) * 2014-12-15 2015-04-22 北京邮电大学 基于深度信息的三维场景增强现实的方法及系统

Also Published As

Publication number Publication date
EP3295239B1 (en) 2021-06-30
JP6377863B2 (ja) 2018-08-22
US9947098B2 (en) 2018-04-17
EP3295239A1 (en) 2018-03-21
US20160335773A1 (en) 2016-11-17
JP2018518750A (ja) 2018-07-12
KR20170141808A (ko) 2017-12-26
WO2016183395A1 (en) 2016-11-17
CN107850782A (zh) 2018-03-27
KR101892168B1 (ko) 2018-08-27
EP3295239A4 (en) 2018-12-26

Similar Documents

Publication Publication Date Title
CN107850782B (zh) 用反射率图表示增强深度图表示
US9392262B2 (en) System and method for 3D reconstruction using multiple multi-channel cameras
US12075020B2 (en) Active stereo matching for depth applications
EP3262439B1 (en) Using intensity variations in a light pattern for depth mapping of objects in a volume
US10762652B2 (en) Hybrid depth detection and movement detection
CN104335005B (zh) 3d扫描以及定位系统
US10565720B2 (en) External IR illuminator enabling improved head tracking and surface reconstruction for virtual reality
KR20200012043A (ko) 인사이드-아웃 위치, 사용자 신체 및 환경 추적을 갖는 가상 및 혼합 현실을 위한 머리 장착 디스플레이
US10728518B2 (en) Movement detection in low light environments
WO2017077277A1 (en) System and methods for imaging three-dimensional objects
TWI588441B (zh) Measuring method and apparatus for carrying out the measuring method
KR101296365B1 (ko) 카메라를 이용한 홀로그램 터치 인식방법
WO2014017095A1 (en) Low cost, non-intrusive, high accuracy head tracking apparatus and method

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
TA01 Transfer of patent application right
TA01 Transfer of patent application right

Effective date of registration: 20190326

Address after: California, USA

Applicant after: Facebook Technologies, LLC

Address before: American California

Applicant before: Facebook, Inc.

GR01 Patent grant
GR01 Patent grant
CP01 Change in the name or title of a patent holder
CP01 Change in the name or title of a patent holder

Address after: California, USA

Patentee after: Yuan Platform Technology Co.,Ltd.

Address before: California, USA

Patentee before: Facebook Technologies, LLC