KR101892168B1 - 반사도 맵 표현을 이용한 깊이 맵 표현의 증강 - Google Patents

반사도 맵 표현을 이용한 깊이 맵 표현의 증강 Download PDF

Info

Publication number
KR101892168B1
KR101892168B1 KR1020177035777A KR20177035777A KR101892168B1 KR 101892168 B1 KR101892168 B1 KR 101892168B1 KR 1020177035777 A KR1020177035777 A KR 1020177035777A KR 20177035777 A KR20177035777 A KR 20177035777A KR 101892168 B1 KR101892168 B1 KR 101892168B1
Authority
KR
South Korea
Prior art keywords
map representation
representation
reflectivity
pattern feature
pattern
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Fee Related
Application number
KR1020177035777A
Other languages
English (en)
Korean (ko)
Other versions
KR20170141808A (ko
Inventor
니타이 로마노
나다브 그로싱거
Original Assignee
페이스북, 인크.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 페이스북, 인크. filed Critical 페이스북, 인크.
Publication of KR20170141808A publication Critical patent/KR20170141808A/ko
Application granted granted Critical
Publication of KR101892168B1 publication Critical patent/KR101892168B1/ko
Expired - Fee Related legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/50—Depth or shape recovery
    • G06T7/521—Depth or shape recovery from laser ranging, e.g. using interferometry; from the projection of structured light
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G02B27/0149—Head-up displays characterised by mechanical features
    • G02B27/2207—
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B30/00—Optical systems or apparatus for producing three-dimensional [3D] effects, e.g. stereoscopic images
    • G02B30/20—Optical systems or apparatus for producing three-dimensional [3D] effects, e.g. stereoscopic images by providing first and second parallax images to an observer's left and right eyes
    • G02B30/22—Optical systems or apparatus for producing three-dimensional [3D] effects, e.g. stereoscopic images by providing first and second parallax images to an observer's left and right eyes of the stereoscopic type
    • G02B30/23—Optical systems or apparatus for producing three-dimensional [3D] effects, e.g. stereoscopic images by providing first and second parallax images to an observer's left and right eyes of the stereoscopic type using wavelength separation, e.g. using anaglyph techniques
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T15/00—3D [Three Dimensional] image rendering
    • G06T15/50—Lighting effects
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G02B27/0101—Head-up displays characterised by optical features
    • G02B2027/0138—Head-up displays characterised by optical features comprising image capture systems, e.g. camera
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G02B27/0149—Head-up displays characterised by mechanical features
    • G02B2027/0154—Head-up displays characterised by mechanical features with movable elements
    • G02B2027/0156—Head-up displays characterised by mechanical features with movable elements with optionally usable elements
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2200/00—Indexing scheme for image data processing or generation, in general
    • G06T2200/04—Indexing scheme for image data processing or generation, in general involving 3D image data
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2200/00—Indexing scheme for image data processing or generation, in general
    • G06T2200/08—Indexing scheme for image data processing or generation, in general involving all processing steps from image acquisition to 3D model generation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10024—Color image
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10141—Special mode during image acquisition
    • G06T2207/10152—Varying illumination
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30196—Human being; Person
    • G06T2207/30201—Face

Landscapes

  • Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Optics & Photonics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Graphics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Length Measuring Devices By Optical Means (AREA)
  • Image Processing (AREA)
  • Processing Or Creating Images (AREA)
KR1020177035777A 2015-05-13 2016-05-13 반사도 맵 표현을 이용한 깊이 맵 표현의 증강 Expired - Fee Related KR101892168B1 (ko)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US201562160957P 2015-05-13 2015-05-13
US62/160,957 2015-05-13
PCT/US2016/032230 WO2016183395A1 (en) 2015-05-13 2016-05-13 Augmenting a depth map representation with a reflectivity map representation

Publications (2)

Publication Number Publication Date
KR20170141808A KR20170141808A (ko) 2017-12-26
KR101892168B1 true KR101892168B1 (ko) 2018-08-27

Family

ID=57248653

Family Applications (1)

Application Number Title Priority Date Filing Date
KR1020177035777A Expired - Fee Related KR101892168B1 (ko) 2015-05-13 2016-05-13 반사도 맵 표현을 이용한 깊이 맵 표현의 증강

Country Status (6)

Country Link
US (1) US9947098B2 (enExample)
EP (1) EP3295239B1 (enExample)
JP (1) JP6377863B2 (enExample)
KR (1) KR101892168B1 (enExample)
CN (1) CN107850782B (enExample)
WO (1) WO2016183395A1 (enExample)

Families Citing this family (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP2480955B1 (en) * 2009-09-22 2018-05-16 Facebook Inc. Remote control of computer devices
JP6782239B2 (ja) * 2015-01-06 2020-11-11 フェイスブック・テクノロジーズ・リミテッド・ライアビリティ・カンパニーＦａｃｅｂｏｏｋ Ｔｅｃｈｎｏｌｏｇｉｅｓ， Ｌｌｃ パターン光を用いた深さマップを提供するための方法およびシステム
US10440355B2 (en) * 2015-11-06 2019-10-08 Facebook Technologies, Llc Depth mapping with a head mounted display using stereo cameras and structured light
US10276210B2 (en) 2015-11-18 2019-04-30 International Business Machines Corporation Video enhancement
US9947099B2 (en) 2016-07-27 2018-04-17 Microsoft Technology Licensing, Llc Reflectivity map estimate from dot based structured light systems
EP3577416B1 (en) 2017-02-06 2022-03-30 Riven, Inc. System and method for 3d scanning
US10620316B2 (en) * 2017-05-05 2020-04-14 Qualcomm Incorporated Systems and methods for generating a structured light depth map with a non-uniform codeword pattern
KR102457462B1 (ko) * 2018-03-06 2022-10-21 삼성전자주식회사 전자장치의 이미지 처리 장치 및 방법
US10935376B2 (en) * 2018-03-30 2021-03-02 Koninklijke Philips N.V. System and method for 3D scanning
CN109087347B (zh) * 2018-08-15 2021-08-17 浙江光珀智能科技有限公司 一种图像处理方法及装置
US11100713B2 (en) 2018-08-17 2021-08-24 Disney Enterprises, Inc. System and method for aligning virtual objects on peripheral devices in low-cost augmented reality/virtual reality slip-in systems
US10909373B1 (en) * 2018-08-24 2021-02-02 Snap Inc. Augmented reality system using structured light
CN109375232B (zh) * 2018-10-17 2021-01-26 维沃移动通信（杭州）有限公司 一种距离测量方法及装置
KR102668245B1 (ko) * 2018-10-29 2024-05-23 삼성전자주식회사 3차원 깊이 측정 장치 및 방법
WO2020186163A1 (en) * 2019-03-13 2020-09-17 Ringo Ai, Inc. White balance with reference illuminants

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6674893B1 (en) 1999-10-19 2004-01-06 Fuji Xerox Co., Ltd. Three-dimensional shape measuring apparatus
US20150043806A1 (en) 2013-08-08 2015-02-12 Adobe Systems Incorporated Automatic geometry and lighting inference for realistic image editing
US20150302648A1 (en) 2014-02-18 2015-10-22 Sulon Technologies Inc. Systems and methods for mapping an environment using structured light

Family Cites Families (32)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5878152A (en) * 1997-05-21 1999-03-02 Cognex Corporation Depth from focal gradient analysis using object texture removal by albedo normalization
US6754370B1 (en) * 2000-08-14 2004-06-22 The Board Of Trustees Of The Leland Stanford Junior University Real-time structured light range scanning of moving scenes
EP1258837A1 (en) * 2001-05-14 2002-11-20 Thomson Licensing S.A. Method to generate mutual photometric effects
US7385708B2 (en) * 2002-06-07 2008-06-10 The University Of North Carolina At Chapel Hill Methods and systems for laser based real-time structured light depth extraction
JP2005258622A (ja) * 2004-03-10 2005-09-22 Fuji Photo Film Co Ltd 三次元情報取得システムおよび三次元情報取得方法
JP2006163547A (ja) * 2004-12-03 2006-06-22 Canon Inc 立体画像生成プログラム、立体画像生成システム及び立体画像生成装置。
US8390821B2 (en) * 2005-10-11 2013-03-05 Primesense Ltd. Three-dimensional sensing using speckle patterns
WO2007069255A2 (en) 2005-12-14 2007-06-21 Yeda Research And Development Co. Ltd. Example based 3d reconstruction
WO2007105215A2 (en) * 2006-03-14 2007-09-20 Prime Sense Ltd. Depth-varying light fields for three dimensional sensing
US20110057930A1 (en) * 2006-07-26 2011-03-10 Inneroptic Technology Inc. System and method of using high-speed, high-resolution depth extraction to provide three-dimensional imagery for endoscopy
WO2008036092A1 (en) * 2006-09-21 2008-03-27 Thomson Licensing A method and system for three-dimensional model acquisition
US7769205B2 (en) * 2006-11-28 2010-08-03 Prefixa International Inc. Fast three dimensional recovery method and apparatus
US20090189858A1 (en) * 2008-01-30 2009-07-30 Jeff Lev Gesture Identification Using A Structured Light Pattern
US8717416B2 (en) * 2008-09-30 2014-05-06 Texas Instruments Incorporated 3D camera using flash with structured light
US8462207B2 (en) * 2009-02-12 2013-06-11 Primesense Ltd. Depth ranging with Moiré patterns
US9582889B2 (en) * 2009-07-30 2017-02-28 Apple Inc. Depth mapping based on pattern matching and stereoscopic information
US8908958B2 (en) * 2009-09-03 2014-12-09 Ron Kimmel Devices and methods of generating three dimensional (3D) colored models
US20120056988A1 (en) * 2010-09-07 2012-03-08 David Stanhill 3-d camera
US8724887B2 (en) * 2011-02-03 2014-05-13 Microsoft Corporation Environmental modifications to mitigate environmental factors
US9471864B2 (en) * 2012-06-22 2016-10-18 Microsoft Technology Licensing, Llc Encoding data in depth patterns
US10234941B2 (en) * 2012-10-04 2019-03-19 Microsoft Technology Licensing, Llc Wearable sensor for tracking articulated body-parts
RU2012145349A (ru) * 2012-10-24 2014-05-10 ЭлЭсАй Корпорейшн Способ и устройство обработки изображений для устранения артефактов глубины
US11509880B2 (en) 2012-11-14 2022-11-22 Qualcomm Incorporated Dynamic adjustment of light source power in structured light active depth sensing systems
WO2014153022A1 (en) * 2013-03-14 2014-09-25 University Of Southern California Specular object scanner for measuring reflectance properties of objects
US10019843B2 (en) * 2013-08-08 2018-07-10 Facebook, Inc. Controlling a near eye display
US9799117B2 (en) * 2013-09-30 2017-10-24 Lenovo (Beijing) Co., Ltd. Method for processing data and apparatus thereof
US9507995B2 (en) * 2014-08-29 2016-11-29 X Development Llc Combination of stereo and structured-light processing
US9664507B2 (en) * 2014-09-17 2017-05-30 Canon Kabushiki Kaisha Depth value measurement using illumination by pixels
CN104539925B (zh) * 2014-12-15 2016-10-05 北京邮电大学 基于深度信息的三维场景增强现实的方法及系统
US9635231B2 (en) * 2014-12-22 2017-04-25 Google Inc. Time-of-flight camera system and method to improve measurement quality of weak field-of-view signal regions
US9948920B2 (en) * 2015-02-27 2018-04-17 Qualcomm Incorporated Systems and methods for error correction in structured light
US20160324580A1 (en) * 2015-03-23 2016-11-10 Justin Esterberg Systems and methods for assisted surgical navigation

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6674893B1 (en) 1999-10-19 2004-01-06 Fuji Xerox Co., Ltd. Three-dimensional shape measuring apparatus
US20150043806A1 (en) 2013-08-08 2015-02-12 Adobe Systems Incorporated Automatic geometry and lighting inference for realistic image editing
US20150302648A1 (en) 2014-02-18 2015-10-22 Sulon Technologies Inc. Systems and methods for mapping an environment using structured light

Also Published As

Publication number Publication date
CN107850782B (zh) 2019-10-25
US20160335773A1 (en) 2016-11-17
JP2018518750A (ja) 2018-07-12
EP3295239B1 (en) 2021-06-30
KR20170141808A (ko) 2017-12-26
CN107850782A (zh) 2018-03-27
US9947098B2 (en) 2018-04-17
WO2016183395A1 (en) 2016-11-17
JP6377863B2 (ja) 2018-08-22
EP3295239A1 (en) 2018-03-21
EP3295239A4 (en) 2018-12-26

Similar Documents

Publication Publication Date Title
KR101892168B1 (ko) 반사도 맵 표현을 이용한 깊이 맵 표현의 증강
US11693242B2 (en) Head-mounted display for virtual and mixed reality with inside-out positional, user body and environment tracking
US12075020B2 (en) Active stereo matching for depth applications
US10762652B2 (en) Hybrid depth detection and movement detection
JP7254903B2 (ja) 構造化光源および飛行時間カメラ付き深度測定アセンブリ
JP7076368B2 (ja) レンジゲート式デプスカメラ部品
EP3262439B1 (en) Using intensity variations in a light pattern for depth mapping of objects in a volume
CN104335005B (zh) 3d扫描以及定位系统
CN108351523A (zh) 利用头戴式显示器使用立体照相机以及结构光的深度映射
JP6583734B2 (ja) 角膜反射位置推定システム、角膜反射位置推定方法、角膜反射位置推定プログラム、瞳孔検出システム、瞳孔検出方法、瞳孔検出プログラム、視線検出システム、視線検出方法、視線検出プログラム、顔姿勢検出システム、顔姿勢検出方法、および顔姿勢検出プログラム
US10728518B2 (en) Movement detection in low light environments
US20220398760A1 (en) Image processing device and three-dimensional measuring system
US20240386577A1 (en) Method and apparatus for data processing based on event camera
TW201329416A (zh) 測量方法及執行該測量方法之裝置

Legal Events

Date Code Title Description
PA0105 International application

St.27 status event code: A-0-1-A10-A15-nap-PA0105

PG1501 Laying open of application

St.27 status event code: A-1-1-Q10-Q12-nap-PG1501

P11-X000 Amendment of application requested

St.27 status event code: A-2-2-P10-P11-nap-X000

P13-X000 Application amended

St.27 status event code: A-2-2-P10-P13-nap-X000

PA0201 Request for examination

St.27 status event code: A-1-2-D10-D11-exm-PA0201

PA0302 Request for accelerated examination

St.27 status event code: A-1-2-D10-D17-exm-PA0302

St.27 status event code: A-1-2-D10-D16-exm-PA0302

E701 Decision to grant or registration of patent right
PE0701 Decision of registration

St.27 status event code: A-1-2-D10-D22-exm-PE0701

GRNT Written decision to grant
PR0701 Registration of establishment

St.27 status event code: A-2-4-F10-F11-exm-PR0701

PR1002 Payment of registration fee

St.27 status event code: A-2-2-U10-U12-oth-PR1002

Fee payment year number: 1

PG1601 Publication of registration

St.27 status event code: A-4-4-Q10-Q13-nap-PG1601

R17-X000 Change to representative recorded

St.27 status event code: A-5-5-R10-R17-oth-X000

P22-X000 Classification modified

St.27 status event code: A-4-4-P10-P22-nap-X000

P22-X000 Classification modified

St.27 status event code: A-4-4-P10-P22-nap-X000

PN2301 Change of applicant

St.27 status event code: A-5-5-R10-R11-asn-PN2301

PN2301 Change of applicant

St.27 status event code: A-5-5-R10-R14-asn-PN2301

PR1001 Payment of annual fee

St.27 status event code: A-4-4-U10-U11-oth-PR1001

Fee payment year number: 4

PN2301 Change of applicant

St.27 status event code: A-5-5-R10-R13-asn-PN2301

St.27 status event code: A-5-5-R10-R11-asn-PN2301

PR1001 Payment of annual fee

St.27 status event code: A-4-4-U10-U11-oth-PR1001

Fee payment year number: 5

PN2301 Change of applicant

St.27 status event code: A-5-5-R10-R13-asn-PN2301

St.27 status event code: A-5-5-R10-R11-asn-PN2301

PR1001 Payment of annual fee

St.27 status event code: A-4-4-U10-U11-oth-PR1001

Fee payment year number: 6

R18-X000 Changes to party contact information recorded

St.27 status event code: A-5-5-R10-R18-oth-X000

PC1903 Unpaid annual fee

St.27 status event code: A-4-4-U10-U13-oth-PC1903

Not in force date: 20240822

Payment event data comment text: Termination Category : DEFAULT_OF_REGISTRATION_FEE

PC1903 Unpaid annual fee

St.27 status event code: N-4-6-H10-H13-oth-PC1903

Ip right cessation event data comment text: Termination Category : DEFAULT_OF_REGISTRATION_FEE

Not in force date: 20240822