JP7431527B2 - 深度情報生成装置、撮像装置、深度情報生成方法、画像処理装置、画像処理方法、及びプログラム - Google Patents

深度情報生成装置、撮像装置、深度情報生成方法、画像処理装置、画像処理方法、及びプログラム Download PDF

Info

Publication number
JP7431527B2
JP7431527B2 JP2019145660A JP2019145660A JP7431527B2 JP 7431527 B2 JP7431527 B2 JP 7431527B2 JP 2019145660 A JP2019145660 A JP 2019145660A JP 2019145660 A JP2019145660 A JP 2019145660A JP 7431527 B2 JP7431527 B2 JP 7431527B2
Authority
JP
Japan
Prior art keywords
image
depth information
photographing
depth
generating
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
JP2019145660A
Other languages
English (en)
Japanese (ja)
Other versions
JP2021027523A5 (https=
JP2021027523A (ja
Inventor
洋平 堀川
武志 小川
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Canon Inc
Original Assignee
Canon Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Canon Inc filed Critical Canon Inc
Priority to JP2019145660A priority Critical patent/JP7431527B2/ja
Priority to US16/987,251 priority patent/US11468584B2/en
Publication of JP2021027523A publication Critical patent/JP2021027523A/ja
Publication of JP2021027523A5 publication Critical patent/JP2021027523A5/ja
Priority to US17/943,701 priority patent/US12243253B2/en
Application granted granted Critical
Publication of JP7431527B2 publication Critical patent/JP7431527B2/ja
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/50—Depth or shape recovery
    • G06T7/55—Depth or shape recovery from multiple images
    • G06T7/571—Depth or shape recovery from multiple images from focus
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
    • H04N13/106—Processing image signals
    • H04N13/128—Adjusting depth or disparity
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/50—Depth or shape recovery
    • G06T7/55—Depth or shape recovery from multiple images
    • G06T7/593—Depth or shape recovery from multiple images from stereo images
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/20—Image signal generators
    • H04N13/204—Image signal generators using stereoscopic image cameras
    • H04N13/207—Image signal generators using stereoscopic image cameras using a single two-dimensional [2D] image sensor
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/20—Image signal generators
    • H04N13/204—Image signal generators using stereoscopic image cameras
    • H04N13/243—Image signal generators using stereoscopic image cameras using three or more two-dimensional [2D] image sensors
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/63—Control of cameras or camera modules by using electronic viewfinders
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/667—Camera operation mode switching, e.g. between still and video, sport and normal or high- and low-resolution modes
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/67—Focus control based on electronic image sensor signals
    • H04N23/672—Focus control based on electronic image sensor signals based on the phase difference signals
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/95—Computational photography systems, e.g. light-field imaging systems
    • H04N23/958—Computational photography systems, e.g. light-field imaging systems for extended depth of field imaging
    • H04N23/959—Computational photography systems, e.g. light-field imaging systems for extended depth of field imaging by adjusting depth of field during image capture, e.g. maximising or setting range based on scene characteristics
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/70—SSIS architectures; Circuits associated therewith
    • H04N25/703—SSIS architectures incorporating pixels for producing signals other than image signals
    • H04N25/704—Pixels specially adapted for focusing, e.g. phase difference pixel sets
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/222—Studio circuitry; Studio devices; Studio equipment
    • H04N5/2224—Studio circuitry; Studio devices; Studio equipment related to virtual studio applications
    • H04N5/2226—Determination of depth image, e.g. for foreground/background separation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/222—Studio circuitry; Studio devices; Studio equipment
    • H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
    • H04N5/265—Mixing
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20212—Image combination
    • G06T2207/20224—Image subtraction
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/20—Image signal generators
    • H04N13/204—Image signal generators using stereoscopic image cameras
    • H04N13/207—Image signal generators using stereoscopic image cameras using a single two-dimensional [2D] image sensor
    • H04N13/229—Image signal generators using stereoscopic image cameras using a single two-dimensional [2D] image sensor using lenticular lenses, e.g. arrangements of cylindrical lenses
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/20—Image signal generators
    • H04N13/204—Image signal generators using stereoscopic image cameras
    • H04N13/207—Image signal generators using stereoscopic image cameras using a single two-dimensional [2D] image sensor
    • H04N13/232—Image signal generators using stereoscopic image cameras using a single two-dimensional [2D] image sensor using fly-eye lenses, e.g. arrangements of circular lenses
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N2013/0074—Stereoscopic image analysis
    • H04N2013/0081—Depth or disparity estimation from stereoscopic image signals

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computing Systems (AREA)
  • Studio Devices (AREA)
  • Automatic Focus Adjustment (AREA)
  • Transforming Light Signals Into Electric Signals (AREA)
  • Focusing (AREA)
JP2019145660A 2019-08-07 2019-08-07 深度情報生成装置、撮像装置、深度情報生成方法、画像処理装置、画像処理方法、及びプログラム Active JP7431527B2 (ja)

Priority Applications (3)

Application Number Priority Date Filing Date Title
JP2019145660A JP7431527B2 (ja) 2019-08-07 2019-08-07 深度情報生成装置、撮像装置、深度情報生成方法、画像処理装置、画像処理方法、及びプログラム
US16/987,251 US11468584B2 (en) 2019-08-07 2020-08-06 Depth information generating apparatus, image capturing apparatus, depth information generating method, image processing apparatus, and image processing method
US17/943,701 US12243253B2 (en) 2019-08-07 2022-09-13 Depth information generating apparatus, image capturing apparatus, depth information generating method, image processing apparatus, and image processing method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
JP2019145660A JP7431527B2 (ja) 2019-08-07 2019-08-07 深度情報生成装置、撮像装置、深度情報生成方法、画像処理装置、画像処理方法、及びプログラム

Publications (3)

Publication Number Publication Date
JP2021027523A JP2021027523A (ja) 2021-02-22
JP2021027523A5 JP2021027523A5 (https=) 2022-08-09
JP7431527B2 true JP7431527B2 (ja) 2024-02-15

Family

ID=74498945

Family Applications (1)

Application Number Title Priority Date Filing Date
JP2019145660A Active JP7431527B2 (ja) 2019-08-07 2019-08-07 深度情報生成装置、撮像装置、深度情報生成方法、画像処理装置、画像処理方法、及びプログラム

Country Status (2)

Country Link
US (2) US11468584B2 (https=)
JP (1) JP7431527B2 (https=)

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10429608B1 (en) * 2016-09-23 2019-10-01 Apple Inc. Primary-subordinate camera focus based on lens position sensing
WO2022178782A1 (en) * 2021-02-25 2022-09-01 Guangdong Oppo Mobile Telecommunications Corp., Ltd. Electric device, method of controlling electric device, and computer readable storage medium
US11893668B2 (en) 2021-03-31 2024-02-06 Leica Camera Ag Imaging system and method for generating a final digital image via applying a profile to image information
US12254644B2 (en) 2021-03-31 2025-03-18 Leica Camera Ag Imaging system and method
EP4134644A1 (en) * 2021-08-11 2023-02-15 Wooptix S.L. System and method for extracting information on the spatial distribution of wavefronts
WO2026009876A1 (ja) * 2024-07-01 2026-01-08 株式会社ニコン 撮像素子、撮像装置および撮像方法

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2017107204A (ja) 2015-12-08 2017-06-15 キヤノン株式会社 情報処理装置及び情報処理方法
JP2018007205A (ja) 2016-07-08 2018-01-11 キヤノン株式会社 画像処理装置、撮像装置、画像処理装置の制御方法及びプログラム
JP2018014658A (ja) 2016-07-22 2018-01-25 キヤノン株式会社 撮像装置、撮像装置の制御方法

Family Cites Families (50)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4410804A (en) 1981-07-13 1983-10-18 Honeywell Inc. Two dimensional image panel with range measurement capability
US5973700A (en) * 1992-09-16 1999-10-26 Eastman Kodak Company Method and apparatus for optimizing the resolution of images which have an apparent depth
JP3774597B2 (ja) 1999-09-13 2006-05-17 キヤノン株式会社 撮像装置
JP4809291B2 (ja) * 2007-06-01 2011-11-09 株式会社豊田中央研究所 計測装置及びプログラム
US20110025830A1 (en) * 2009-07-31 2011-02-03 3Dmedia Corporation Methods, systems, and computer-readable storage media for generating stereoscopic content via depth map creation
JP5346266B2 (ja) * 2009-09-30 2013-11-20 富士フイルム株式会社 画像処理装置、カメラ及び画像処理方法
KR101214536B1 (ko) * 2010-01-12 2013-01-10 삼성전자주식회사 뎁스 정보를 이용한 아웃 포커스 수행 방법 및 이를 적용한 카메라
US8547421B2 (en) * 2010-08-13 2013-10-01 Sharp Laboratories Of America, Inc. System for adaptive displays
KR20120028688A (ko) 2010-09-15 2012-03-23 삼성전기주식회사 혼합물 분리 응축 장치
US8784301B2 (en) * 2011-08-12 2014-07-22 Intuitive Surgical Operations, Inc. Image capture unit and method with an extended depth of field
JP5979849B2 (ja) 2011-11-21 2016-08-31 キヤノン株式会社 撮像素子及び撮像装置
JP5341286B1 (ja) * 2011-12-16 2013-11-13 オリンパスメディカルシステムズ株式会社 深度拡大装置
JP6071257B2 (ja) * 2012-06-07 2017-02-01 キヤノン株式会社 画像処理装置及びその制御方法、並びにプログラム
US9185387B2 (en) * 2012-07-03 2015-11-10 Gopro, Inc. Image blur based on 3D depth information
KR102111181B1 (ko) * 2012-08-21 2020-05-15 포토내이션 리미티드 어레이 카메라를 사용하여 포착된 영상에서의 시차 검출 및 보정을 위한 시스템 및 방법
US9769365B1 (en) * 2013-02-15 2017-09-19 Red.Com, Inc. Dense field imaging
JP2016524125A (ja) * 2013-03-15 2016-08-12 ペリカン イメージング コーポレイション カメラアレイを用いた立体撮像のためのシステムおよび方法
JP6214233B2 (ja) * 2013-06-21 2017-10-18 キヤノン株式会社 情報処理装置、情報処理システム、情報処理方法およびプログラム。
US9565416B1 (en) * 2013-09-30 2017-02-07 Google Inc. Depth-assisted focus in multi-camera systems
EP2871843B1 (en) * 2013-11-12 2019-05-29 LG Electronics Inc. -1- Digital device and method for processing three dimensional image thereof
JP6198590B2 (ja) 2013-11-28 2017-09-20 キヤノン株式会社 画像処理装置、撮像装置および画像処理方法
JP6260399B2 (ja) * 2014-03-28 2018-01-17 富士通株式会社 撮像装置、撮像方法および撮像プログラム
US9729857B2 (en) * 2014-04-08 2017-08-08 Semyon Nisenzon High resolution depth map computation using multiresolution camera clusters for 3D image generation
US9445417B2 (en) * 2014-04-08 2016-09-13 Qualcomm Incorporated Apparatus and method of sharing a transmit chain in wireless communications
JP6395429B2 (ja) * 2014-04-18 2018-09-26 キヤノン株式会社 画像処理装置、その制御方法及び記憶媒体
TWI538508B (zh) * 2014-08-15 2016-06-11 光寶科技股份有限公司 一種可獲得深度資訊的影像擷取系統與對焦方法
JP6642998B2 (ja) * 2014-09-17 2020-02-12 キヤノン株式会社 像ズレ量算出装置、撮像装置、および像ズレ量算出方法
KR102298652B1 (ko) * 2015-01-27 2021-09-06 삼성전자주식회사 시차 결정 방법 및 장치
US9369689B1 (en) * 2015-02-24 2016-06-14 HypeVR Lidar stereo fusion live action 3D model video reconstruction for six degrees of freedom 360° volumetric virtual reality video
US10356392B2 (en) * 2015-05-28 2019-07-16 University College Cork—National Univesity of Ireland, Cork Coded access optical sensor
KR101706627B1 (ko) * 2015-09-30 2017-02-17 주식회사 디아이랩 이배율 스테레오 영상을 이용한 거리 측정 장치 및 거리 측정 방법
CN108369639B (zh) * 2015-12-11 2022-06-21 虞晶怡 使用多相机和深度相机阵列的基于图像的图像渲染方法和系统
JP6611588B2 (ja) * 2015-12-17 2019-11-27 キヤノン株式会社 データ記録装置、撮像装置、データ記録方法およびプログラム
WO2017205294A1 (en) * 2016-05-26 2017-11-30 Dental Smartmirror, Inc. A dental mirror with an integrated camera, and applications thereof
US10109055B2 (en) * 2016-11-21 2018-10-23 Seiko Epson Corporation Multiple hypotheses segmentation-guided 3D object detection and pose estimation
US10554881B2 (en) * 2016-12-06 2020-02-04 Microsoft Technology Licensing, Llc Passive and active stereo vision 3D sensors with variable focal length lenses
US10397549B2 (en) * 2016-12-20 2019-08-27 Gopro, Inc. Compact array of imaging devices with supplemental imaging unit
US10277889B2 (en) * 2016-12-27 2019-04-30 Qualcomm Incorporated Method and system for depth estimation based upon object magnification
US11652978B2 (en) * 2017-01-05 2023-05-16 Eys3D Microelectronics, Co. Depth map generation device
DE102017100262A1 (de) * 2017-01-09 2018-07-12 Carl Zeiss Microscopy Gmbh Verfahren zur Erzeugung eines dreidimensionalen Modells einer Probe in einem digitalen Mikroskop und digitales Mikroskop
JP6779150B2 (ja) * 2017-02-03 2020-11-04 鹿島建設株式会社 情報取得装置
AU2017202910A1 (en) * 2017-05-02 2018-11-22 Canon Kabushiki Kaisha Image processing for turbulence compensation
US11095854B2 (en) * 2017-08-07 2021-08-17 Verizon Patent And Licensing Inc. Viewpoint-adaptive three-dimensional (3D) personas
JP6770500B2 (ja) * 2017-11-09 2020-10-14 株式会社モリタ製作所 口腔内観察装置、観察ユニット及び観察方法
JP7157303B2 (ja) * 2018-02-01 2022-10-20 ミツミ電機株式会社 認証装置
KR102472156B1 (ko) * 2018-04-19 2022-11-30 삼성전자주식회사 전자 장치 및 그 깊이 정보 생성 방법
JP2020010146A (ja) * 2018-07-05 2020-01-16 キヤノン株式会社 画像処理装置、画像処理方法、及びプログラム
US11044418B2 (en) * 2018-07-24 2021-06-22 Black Sesame International Holding Limited Three camera alignment in mobile devices
US11184561B2 (en) * 2018-12-31 2021-11-23 Aaron M. Benzel Method for improved acquisition of images for photogrammetry
JP7233261B2 (ja) * 2019-03-13 2023-03-06 キヤノン株式会社 ３次元測量装置、撮像装置、制御方法及びプログラム

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2017107204A (ja) 2015-12-08 2017-06-15 キヤノン株式会社 情報処理装置及び情報処理方法
JP2018007205A (ja) 2016-07-08 2018-01-11 キヤノン株式会社 画像処理装置、撮像装置、画像処理装置の制御方法及びプログラム
JP2018014658A (ja) 2016-07-22 2018-01-25 キヤノン株式会社 撮像装置、撮像装置の制御方法

Also Published As

Publication number Publication date
US20230186501A1 (en) 2023-06-15
US20210042949A1 (en) 2021-02-11
US12243253B2 (en) 2025-03-04
JP2021027523A (ja) 2021-02-22
US11468584B2 (en) 2022-10-11

Similar Documents

Publication Publication Date Title
JP7431527B2 (ja) 深度情報生成装置、撮像装置、深度情報生成方法、画像処理装置、画像処理方法、及びプログラム
EP2809063B1 (en) Image capturing apparatus and image processing method
JP6405243B2 (ja) 焦点検出装置及びその制御方法
JP5979961B2 (ja) 焦点検出装置、焦点検出方法及び撮像装置
EP2698658B1 (en) Image pickup apparatus, semiconductor integrated circuit and image pickup method
JP6053479B2 (ja) 焦点検出装置、撮像装置、撮像システム、および、焦点検出方法
CN107948519A (zh) 图像处理方法、装置及设备
US9008412B2 (en) Image processing device, image processing method and recording medium for combining image data using depth and color information
JP5002412B2 (ja) 撮像装置
CN103493484A (zh) 成像装置和成像方法
JP2019092119A (ja) 撮像装置、レンズ装置およびこれらの制御方法
EP3002940A1 (en) Imaging apparatus and imaging method
JP2017181797A (ja) 焦点検出装置およびその方法、撮像装置、プログラム、記憶媒体
CN109151265B (zh) 摄像设备、控制方法和存储介质
CN112866674B (zh) 深度图获取方法、装置、电子设备和计算机可读存储介质
US12477221B2 (en) Control apparatus, control method, and image capture apparatus
JP6000627B2 (ja) 焦点検出装置、その制御方法、および制御プログラム、並びに撮像装置
JP2016100868A (ja) 画像処理装置、画像処理方法およびプログラム、並びに撮像装置
JP6611559B2 (ja) 撮像装置、その制御方法、プログラム及び記憶媒体
JP2018007205A (ja) 画像処理装置、撮像装置、画像処理装置の制御方法及びプログラム
JP2005064749A (ja) カメラ
JP2015204579A (ja) 撮像装置
WO2024076363A1 (en) Field of view correction techniques for shutterless camera systems
HK1251749A1 (zh) 图像处理方法、装置及设备

Legal Events

Date Code Title Description
RD01 Notification of change of attorney

Free format text: JAPANESE INTERMEDIATE CODE: A7421

Effective date: 20210103

A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20210113

A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20220801

A621 Written request for application examination

Free format text: JAPANESE INTERMEDIATE CODE: A621

Effective date: 20220801

A977 Report on retrieval

Free format text: JAPANESE INTERMEDIATE CODE: A971007

Effective date: 20230731

A131 Notification of reasons for refusal

Free format text: JAPANESE INTERMEDIATE CODE: A131

Effective date: 20230901

A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20231027

TRDD Decision of grant or rejection written
A01 Written decision to grant a patent or to grant a registration (utility model)

Free format text: JAPANESE INTERMEDIATE CODE: A01

Effective date: 20240105

A61 First payment of annual fees (during grant procedure)

Free format text: JAPANESE INTERMEDIATE CODE: A61

Effective date: 20240202

R151 Written notification of patent or utility model registration

Ref document number: 7431527

Country of ref document: JP

Free format text: JAPANESE INTERMEDIATE CODE: R151