WO2021065065A1 - Dispositif d'imagerie - Google Patents

Dispositif d'imagerie Download PDF

Info

Publication number
WO2021065065A1
WO2021065065A1 PCT/JP2020/018918 JP2020018918W WO2021065065A1 WO 2021065065 A1 WO2021065065 A1 WO 2021065065A1 JP 2020018918 W JP2020018918 W JP 2020018918W WO 2021065065 A1 WO2021065065 A1 WO 2021065065A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
subject
optical system
image data
unit
Prior art date
Application number
PCT/JP2020/018918
Other languages
English (en)
Japanese (ja)
Inventor
宏道 小谷
真史 大野
岡田　茂之
Original Assignee
株式会社ザクティ
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 株式会社ザクティ filed Critical 株式会社ザクティ
Publication of WO2021065065A1 publication Critical patent/WO2021065065A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B7/00—Mountings, adjusting means, or light-tight connections, for optical elements
    • G02B7/28—Systems for automatic generation of focusing signals
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B7/00—Mountings, adjusting means, or light-tight connections, for optical elements
    • G02B7/28—Systems for automatic generation of focusing signals
    • G02B7/36—Systems for automatic generation of focusing signals using image sharpness techniques, e.g. image processing techniques for generating autofocus signals
    • G—PHYSICS
    • G03—PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
    • G03B—APPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR
    • G03B13/00—Viewfinders; Focusing aids for cameras; Means for focusing for cameras; Autofocus systems for cameras
    • G03B13/32—Means for focusing
    • G03B13/34—Power focusing
    • G03B13/36—Autofocus systems
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules

Definitions

  • the present invention relates to an imaging device.
  • DFD Depth From Defocus
  • the amount of blur that occurs in a captured image is uniquely determined for each imaging device according to the relationship between the focus position at the time of shooting and the subject distance. Utilizing this characteristic, in the DFD method, for example, two images with different amounts of blur are intentionally created by changing the focus position, and the difference in the amount of blur and the point spread function (PSF) are used. To specify the subject distance.
  • the image pickup apparatus can perform focus control based on the specified subject distance.
  • an object of the present invention is to reduce the time required for focus control and the H / W cost of a memory or the like in a DFD type image pickup apparatus.
  • the first aspect of the present disclosure is an optical system that forms a subject image of a subject, an imaging unit that captures a subject image formed through the optical system and generates image data, and adjustment conditions for the optical system.
  • a control unit that specifies a subject distance, which is a distance to the subject, and a plurality of first PSFs that are previously obtained for a plurality of different distances under the first adjustment condition of the optical system, based on two image data having different characteristics.
  • the control unit includes a storage unit that stores a plurality of second PSFs obtained in advance for the plurality of distances under the second adjustment condition of the optical system, and the control unit is generated under the first adjustment condition of the optical system.
  • a first blurred image generation unit that generates a plurality of first blurred images by convolving each of the plurality of second PSFs with respect to the first image data of the subject, and the second adjustment condition of the optical system are generated.
  • a second blurred image generation unit that generates a plurality of second blurred images by convolving each of the plurality of first PSFs with respect to the second image data of the subject, and each of the plurality of first blurred images.
  • An image pickup apparatus including a difference calculation unit that calculates a plurality of differences between the plurality of second blurred images and a distance specifying unit that specifies the subject distance based on the plurality of differences. Is.
  • the blurred image groups obtained by convolving (that is, crossing) the PSF group corresponding to the adjustment condition of the other image data into each of the two image data having different adjustment conditions of the optical system.
  • the subject distance is specified based on the difference. Therefore, the image processing algorithm can be simplified and the amount of data required for processing can be reduced. Therefore, the time required for focus control can be shortened, and the H / W cost of the memory or the like can be reduced.
  • the optical system has a focus position adjusting means, the first adjustment condition is a first focus position, and the second adjustment condition is a first. It is an imaging device that has two focus positions.
  • the subject distance can be specified based on two image data having different focus positions.
  • a third aspect of the present disclosure is, in the first aspect, the optical system includes aperture adjusting means, the first adjustment condition is a first aperture value, and the second adjustment condition is a second. It is an image pickup device that is an aperture value.
  • the subject distance can be specified based on two image data having different aperture values.
  • the optical system has a zoom adjusting means, the first adjusting condition is a first zoom state, and the second adjusting condition is a second. It is an image pickup device in a zoomed state.
  • the subject distance can be specified based on two image data having different zoom states.
  • a fifth aspect of the present disclosure is an imaging device that, in any one of the first to fourth aspects, the control unit performs focus control based on the subject distance.
  • focus control can be performed in a short time while suppressing the H / W cost.
  • a sixth aspect of the present disclosure is, in any one of the first to fifth aspects, the control unit divides the first image data and the second image data into a plurality of regions, and for each of the regions.
  • An image pickup device that specifies the subject distance.
  • the accuracy of the subject distance is improved.
  • FIG. 1 is a block diagram showing a configuration of an imaging device according to an embodiment.
  • FIG. 2 is a block diagram showing a configuration of a control unit of the image pickup apparatus shown in FIG.
  • FIG. 3 is a schematic diagram showing a DFD process by the control unit shown in FIG.
  • FIG. 4 is a diagram showing an example of a PSF group stored in the storage unit of the image pickup apparatus shown in FIG.
  • FIG. 5 is an example of an image shown by the first image data of the subject used in the control unit of the image pickup apparatus shown in FIG.
  • FIG. 6 is an example of an image shown by the second image data of the subject used in the control unit of the image pickup apparatus shown in FIG.
  • FIG. 7 is a map showing an example of the subject distance specified by the control unit of the image pickup apparatus shown in FIG. 1.
  • FIG. 1 is a block diagram showing the configuration of the image pickup apparatus 100 of the present embodiment.
  • the image pickup apparatus 100 mainly includes an optical system 10, an image pickup unit 20, a control unit 30, a storage unit 40, and a drive unit 50.
  • the optical system 10 forms a subject image on the image pickup unit 20 by the light from the subject.
  • the image capturing unit 20 captures a subject image formed by the optical system 10 and generates image data.
  • the control unit 30 processes the image data generated by the image pickup unit 20 (DFD or the like) and controls the entire image pickup apparatus 100.
  • the storage unit 40 stores image data, a control program for controlling the entire image pickup apparatus 100, and the like.
  • the drive unit 50 drives the optical system 10.
  • the optical system 10 includes a focus position adjusting means 11, a zoom adjusting means 12, and an aperture adjusting means 13.
  • the focus position adjusting means 11 is, for example, a focus lens
  • the focus of the subject image can be adjusted by moving the focus lens by the drive unit 50.
  • the focus lens may be moved along the optical axis 10a, for example.
  • the zoom adjusting means 12 is, for example, a zoom lens
  • the subject image can be enlarged or reduced by moving the zoom lens by the driving unit 50.
  • the zoom lens may be moved along the optical axis 10a, for example.
  • the aperture adjusting means 13 has, for example, an opening 13a located on the optical axis 10a, and the size of the opening 13a can be adjusted according to a user's setting or automatically, whereby the aperture adjusting means 13 can be adjusted. The amount of transmitted light is adjusted.
  • the configuration and operation of the focus position adjusting means 11, the zoom adjusting means 12, and the aperture adjusting means 13 are not limited to those described above.
  • the optical system 10 may further have a camera shake correction adjusting means such as a camera shake correction lens.
  • the imaging unit 20 is composed of, for example, a CMOS image sensor.
  • the imaging unit 20 performs various operations such as exposure, transfer, and electronic shutter.
  • the control unit 30 includes an image processing unit 31 that processes image data (DFD and the like) generated by the image pickup unit 20, and a CPU (Central Processing Unit) 32 that controls the entire image pickup device 100.
  • image processing unit 31 that processes image data (DFD and the like) generated by the image pickup unit 20, and a CPU (Central Processing Unit) 32 that controls the entire image pickup device 100.
  • CPU Central Processing Unit
  • the image processing unit 31 converts the analog image data generated by the imaging unit 20 into digital image data which is a digital signal, performs various processing on the image data, and generates data to be displayed to the user. Or, generate data to be stored in the storage unit 40. For example, the image processing unit 31 performs various processes such as gamma correction, white balance correction, and compression on the image data generated by the image pickup unit 20.
  • the image processing unit 31 is composed of a DSP (digital signal processor), a microcomputer, and the like.
  • the CPU 32 performs various calculations necessary for DFD processing described later.
  • the CPU 32 includes a processor that operates according to a program as a main hardware configuration.
  • the type of processor is not limited as long as the function can be realized by executing the program, and the processor may be composed of, for example, a semiconductor integrated circuit or one or a plurality of electronic circuits including an LSI.
  • the storage unit 40 stores image data processed by the control unit 30, a control program of the imaging device 100, data (PSF, etc.) required for image processing by the control unit 30, and the like.
  • the storage unit 40 is composed of a flash memory, a ferroelectric memory, and the like.
  • the image pickup apparatus 100 may further include a buffer that functions as a work memory for the image processing unit 31 and the CPU 32.
  • the buffer is composed of, for example, a DRAM or a ferroelectric memory.
  • the drive unit 50 includes an actuator for driving the focus position adjusting means 11, the zoom adjusting means 12, and the aperture adjusting means 13, and the actuator or the like is controlled by the control unit 30 through the driving unit 50.
  • the imaging device 100 may include a user interface that accepts operations from the user, such as a selection button and a cross key.
  • the image pickup apparatus 100 may further include a display monitor that presents information to the user.
  • the display monitor may have a screen capable of displaying an image indicated by the image data generated by the imaging unit 20 and an image indicated by the image data read from the storage unit 40.
  • the display monitor may display a menu screen or the like for making various settings of the image pickup apparatus 100.
  • a touch panel capable of accepting various touch operations by the user may be arranged on the screen of the display monitor. The instruction indicated by the touch operation on the touch panel is transmitted to the control unit 30 (CPU 32) to perform various processes.
  • control unit 30 specifies the subject distance, which is the distance to the subject, based on two image data having different adjustment conditions of the optical system 10. That is, the control unit 30 performs the DFD process.
  • FIG. 2 is a block diagram showing the configuration of the control unit 30,
  • FIG. 3 is a schematic diagram showing DFD processing by the control unit 30, and
  • FIG. 4 shows an example of a PSF group stored in the storage unit 40. It is a figure.
  • the image processing unit 31 of the control unit 30 has a first blurred image generation unit 33 and a second blurred image generation unit 34. Further, the CPU 32 of the control unit 30 has a difference calculation unit 35 and a distance specifying unit 36.
  • the first blurred image generation unit 33 is the first image data (g 1 (d)) of the subject generated under the first adjustment condition (specifically, the first focus position) of the optical system 10.
  • a plurality of first blurred images (first blurred image group) are generated by convolving the second PSF group (k 2 (d)).
  • the second blurred image generation unit 34 has a first PSF group with respect to the second image data (g 2 (d)) of the subject generated under the second adjustment condition (specifically, the second focus position) of the optical system 10.
  • a plurality of second blurred images are generated.
  • the difference calculation unit 35 has a plurality of differences (PSF) between the first blurred image group generated by the first blurred image generation unit 33 and the second blurred image group generated by the second blurred image generation unit 34.
  • PSF differences
  • the difference between "blurred images" with the same distance) is calculated.
  • the distance specifying unit 36 specifies the subject distance based on a plurality of differences calculated by the difference calculating unit 35.
  • the storage unit 40 stores a plurality of PSF groups obtained in advance for a plurality of different distances at each of the plurality of focus positions of the optical system 10.
  • the storage unit 40 has a plurality of first PSFs (first PSF group) obtained in advance for a plurality of different distances at the first focus position (first adjustment condition) of the optical system 10 for DFD processing by the control unit 30.
  • first PSF group first PSFs
  • second PSF groups obtained in advance for a plurality of distances at the second focus position (second adjustment condition) of the optical system 10.
  • the result of each calculation may be appropriately stored in the storage unit 40 or the buffer.
  • the DFD process by the control unit 30 will be described in detail below.
  • the control unit 30 intentionally generates two images (two image data) having different amounts of blur by changing the focus position of the optical system 10. Specifically, the control unit 30 controls the drive unit 50 to set the focus position of the focus position adjusting means 11 at time T1 to the “first focus position”. Similarly, the focus position of the focus position adjusting means 11 at time T2 is set to the "second focus position”.
  • the image pickup unit 20 takes an image of the subject at the "first focus position” at time T1 and generates "first image data”. Similarly, the imaging unit 20 images the subject at the “second focus position” at time T2 and generates “second image data”. Since the focus position of the focus position adjusting means 11 is different between the "first image data" and the "second image data”, they are image data obtained by capturing the same subject, but the amount of blurring is different from each other.
  • the PSF used for DFD processing is a function that indicates the response of the optical system to a point light source, and indicates a change in the amount of blur.
  • a blurred image can be generated by convolving the PSF into an image that corresponds to a set of point light sources.
  • a plurality of PSFs are stored in advance in the storage unit 40 for each of the plurality of focus positions corresponding to a plurality of different distances.
  • the control unit 30 (first blurred image generation unit 33 and second blurred image generation unit 34) stores the PSF group required for the above-mentioned convolution calculation together with the “first image data” and the “second image data” from the storage unit 40. Read selectively.
  • the "first image data (g 1 (d))" of the subject generated at the "first focus position” is the "first focus image f" of the subject.
  • the PSF corresponding to the subject distance in the first PSF group (k 1 (d)) of the "position” is convoluted.
  • the "second image data (g 2 (d))" of the subject generated at the "second focus position” is the second PSF group of the "second focus position” in the "omnifocal image f" of the subject.
  • the PSF corresponding to the subject distance is convoluted.
  • the blurred image corresponding to the subject distance is the theory. Matches.
  • the difference calculation unit 35 is used to calculate a plurality of differences between "blurred images” having the same PSF distance
  • the distance identification unit 36 is used to specify the "subject distance” based on the plurality of differences. ..
  • the distance specifying unit 36 may simply specify the distance of the PSF corresponding to the smallest of the plurality of differences as the “subject distance”.
  • the distance specifying unit 36 considers the change of the plurality of differences calculated by the difference calculating unit 35 with respect to the "distance", and, for example, sets the distance of the PSF corresponding to the smallest of the plurality of differences to " It may be specified as "subject distance”.
  • the difference from another difference whose "distance" is before or after is further taken to calculate the total, and the distance of the PSF corresponding to the one with the smallest total is specified as the "subject distance". You may.
  • control unit 30 divides the "first image data" and the "second image data” into a plurality of regions (for example, 16 ⁇ 16 or 32 ⁇ 32 pixel regions) for each region.
  • the subject distance may be specified. This improves the accuracy of the subject distance.
  • FIGS. 5 and 6 are examples of images shown by the “first image data” and “second image data” of the subject used by the control unit 30, respectively
  • FIG. 7 is based on the images shown in FIGS. 5 and 6, respectively. It is a figure which shows an example of the subject distance specified by the control unit 30 in the form of a map.
  • the subject distance shown in FIG. 7 is specified by dividing the "first image data” and the "second image data” into a plurality of regions and specifying each region.
  • control unit 30 may perform focus control through the focus position adjusting means 11 based on the subject distance specified by the distance specifying unit 36. As described above, when the subject distance is specified for each image area, focus control may be performed based on, for example, the closest subject distance or the subject distance in the center of the image.
  • the PSF group corresponding to the focus position of the other image data is convolved in each of the two image data having different focus positions of the optical system 10 (that is, a convolution).
  • the subject distance is specified based on the difference between the blurred image groups. Therefore, the image processing algorithm can be simplified and the amount of data required for processing can be reduced. Therefore, the time required for focus control can be shortened, and the H / W cost of the memory or the like can be reduced.
  • the subject distance is specified based on two image data having different focus positions of the optical system 10.
  • the subject distance may be specified based on two image data having different aperture values and zoom states of the optical system 10.
  • AF autofocus
  • contrast AF contrast AF
  • phase difference AF image plane phase difference AF
  • image plane phase difference AF image plane phase difference AF
  • the present disclosure is useful for imaging devices.
  • Imaging unit 10 Optical system 10a Optical axis 11 Focus position adjusting means 12 Zoom adjusting means 13 Aperture adjusting means 13a Aperture 20 Imaging unit 30 Control unit 31 Image processing unit 32 CPU 33 1st blurred image generation unit 34 2nd blurred image generation unit 35 Difference calculation unit 36 Distance identification unit 40 Storage unit 50 Drive unit 100 Imaging device

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Optics & Photonics (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Automatic Focus Adjustment (AREA)
  • Studio Devices (AREA)
  • Focusing (AREA)

Abstract

Des premières données d'image (g1(d)) d'un sujet est généré dans une première condition de réglage pour un système optique, et un second groupe PSF (k2(d)) de secondes données d'image (g2(d)) est convolutée sur les premières données d'image pour générer un premier groupe d'images de défocalisation. Des secondes données d'image (g2(d)) du sujet est généré dans une seconde condition de réglage pour le système optique, et un premier groupe PSF (k1(d)) de premières données d'image (g1(d)) est convolutée sur les secondes données d'image pour générer un second groupe d'images de défocalisation. Entre le premier groupe d'images de défocalisation et le second groupe d'images de défocalisation, une pluralité de différences entre des "images de défocalisation" ayant une distance PSF égale sont calculées, et une distance de sujet est déterminée sur la base de la pluralité de différences.
PCT/JP2020/018918 2019-10-03 2020-05-12 Dispositif d'imagerie WO2021065065A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2019182974A JP6687210B1 (ja) 2019-10-03 2019-10-03 撮像装置
JP2019-182974 2019-10-03

Publications (1)

Publication Number Publication Date
WO2021065065A1 true WO2021065065A1 (fr) 2021-04-08

Family

ID=70286808

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2020/018918 WO2021065065A1 (fr) 2019-10-03 2020-05-12 Dispositif d'imagerie

Country Status (2)

Country Link
JP (1) JP6687210B1 (fr)
WO (1) WO2021065065A1 (fr)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2010039448A (ja) * 2008-08-08 2010-02-18 Canon Inc 画像撮影装置およびその距離演算方法と合焦画像取得方法
JP2013044844A (ja) * 2011-08-23 2013-03-04 Panasonic Corp 画像処理装置および画像処理方法
JP2016057633A (ja) * 2013-11-19 2016-04-21 パナソニックＩｐマネジメント株式会社 撮像装置
JP2019016275A (ja) * 2017-07-10 2019-01-31 キヤノン株式会社 画像処理方法、画像処理プログラム、記憶媒体、画像処理装置、および撮像装置

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2010039448A (ja) * 2008-08-08 2010-02-18 Canon Inc 画像撮影装置およびその距離演算方法と合焦画像取得方法
JP2013044844A (ja) * 2011-08-23 2013-03-04 Panasonic Corp 画像処理装置および画像処理方法
JP2016057633A (ja) * 2013-11-19 2016-04-21 パナソニックＩｐマネジメント株式会社 撮像装置
JP2019016275A (ja) * 2017-07-10 2019-01-31 キヤノン株式会社 画像処理方法、画像処理プログラム、記憶媒体、画像処理装置、および撮像装置

Also Published As

Publication number Publication date
JP6687210B1 (ja) 2020-04-22
JP2021060450A (ja) 2021-04-15

Similar Documents

Publication Publication Date Title
JP5866493B2 (ja) 撮像装置
JP5895270B2 (ja) 撮像装置
US9066004B2 (en) Auto-focus apparatus, image pick-up apparatus, and auto-focus method for focusing using evaluation values
JP6405531B2 (ja) 撮像装置
JP6906947B2 (ja) 画像処理装置、撮像装置、画像処理方法およびコンピュータのプログラム
WO2007058100A1 (fr) Detecteur focal
JP7484866B2 (ja) ブレ補正装置、交換レンズ、撮像装置及び像ブレ補正方法
JPWO2006082967A1 (ja) 撮像装置
JP6357646B2 (ja) 撮像装置
JP6432038B2 (ja) 撮像装置
JP6326631B2 (ja) 撮像装置
JP6564284B2 (ja) 画像処理装置および画像処理方法
TWI693828B (zh) 顯示擷取裝置與其操作方法
JP2009284101A (ja) 撮像装置及びその制御方法
JP5866537B2 (ja) 撮像装置
WO2021065065A1 (fr) Dispositif d'imagerie
JP2020008785A (ja) 撮像装置
JP6089232B2 (ja) 撮像装置
JP2020036091A (ja) 撮像装置及びその制御方法、プログラム、記憶媒体
WO2015182021A1 (fr) Appareil et procédé de commande de capture d'image et appareil de capture d'image
US20190045123A1 (en) Imaging apparatus and control method thereof
JP2023055075A (ja) 撮像装置およびその制御方法
JP2019129492A (ja) 撮像装置および制御方法
JP2015201812A (ja) 画像処理方法、プログラムおよび撮像装置
JP2017138394A (ja) 画像表示制御装置、画像表示制御方法及び画像表示制御プログラム

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20871217

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20871217

Country of ref document: EP

Kind code of ref document: A1