WO2011070757A1 - 撮像装置および撮像方法 - Google Patents

撮像装置および撮像方法 Download PDF

Info

Publication number
WO2011070757A1
WO2011070757A1 PCT/JP2010/007061 JP2010007061W WO2011070757A1 WO 2011070757 A1 WO2011070757 A1 WO 2011070757A1 JP 2010007061 W JP2010007061 W JP 2010007061W WO 2011070757 A1 WO2011070757 A1 WO 2011070757A1
Authority
WO
WIPO (PCT)
Prior art keywords
focus position
initial
focus
exposure
imaging
Prior art date
Application number
PCT/JP2010/007061
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
河村　岳
Original Assignee
パナソニック株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by パナソニック株式会社 filed Critical パナソニック株式会社
Priority to EP10835683.3A priority Critical patent/EP2511747B1/en
Priority to CN201080006774.5A priority patent/CN102308242B/zh
Priority to JP2011545076A priority patent/JP5588461B2/ja
Priority to US13/147,886 priority patent/US8412036B2/en
Publication of WO2011070757A1 publication Critical patent/WO2011070757A1/ja

Links

Images

Classifications

    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B7/00—Mountings, adjusting means, or light-tight connections, for optical elements
    • G02B7/02—Mountings, adjusting means, or light-tight connections, for optical elements for lenses
    • G02B7/04—Mountings, adjusting means, or light-tight connections, for optical elements for lenses with mechanism for focusing or varying magnification
    • G02B7/08—Mountings, adjusting means, or light-tight connections, for optical elements for lenses with mechanism for focusing or varying magnification adapted to co-operate with a remote control mechanism
    • G—PHYSICS
    • G03—PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
    • G03B—APPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR
    • G03B13/00—Viewfinders; Focusing aids for cameras; Means for focusing for cameras; Autofocus systems for cameras
    • G03B13/32—Means for focusing
    • G03B13/34—Power focusing
    • G03B13/36—Autofocus systems
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/95—Computational photography systems, e.g. light-field imaging systems
    • H04N23/958—Computational photography systems, e.g. light-field imaging systems for extended depth of field imaging
    • H04N23/959—Computational photography systems, e.g. light-field imaging systems for extended depth of field imaging by adjusting depth of field during image capture, e.g. maximising or setting range based on scene characteristics
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B7/00—Mountings, adjusting means, or light-tight connections, for optical elements
    • G02B7/28—Systems for automatic generation of focusing signals

Definitions

  • the present invention relates to an image pickup apparatus and an image pickup method that realize a reduction in shutter time lag in an image pickup apparatus that employs a method for realizing extension of depth of field.
  • EDOF depth of field expansion
  • a blur in the depth direction is made uniform by inserting an optical element called a phase plate into the optical system.
  • an image restoration process is performed on the obtained image using a blur pattern measured in advance or a blur pattern calculated by simulation. Thereby, the method generates an EDOF image.
  • WFC Wavefront Coding
  • the second method measures the distance with high accuracy for each partial area of the image by devising the aperture shape.
  • image restoration processing is performed on each partial region using a blur pattern corresponding to each distance measured in advance. Thereby, the method generates an EDOF image.
  • This method is referred to as coded aperture (hereinafter referred to as CA) (see Non-Patent Document 2).
  • the third method convolves an image that is uniformly focused in the depth direction by moving the focus lens or image sensor during the exposure time (that is, synonymous with equalizing blur at each depth).
  • an image restoration process is performed on the obtained image using a blur pattern measured in advance or a blur pattern calculated by simulation. Thereby, the method generates an EDOF image.
  • This method is called “Flexible DOF” (hereinafter referred to as “F-DOF”) (see Non-Patent Document 3).
  • Non-Patent Document 4 depth estimation or image sharpness detection using axial chromatic aberration of the lens
  • a method for obtaining a sharp overall image by image processing see Non-Patent Document 4
  • a multifocal lens There is also a method (see Non-Patent Document 5) in which the blur in the depth direction is uniformed and image restoration processing is performed using a blur pattern measured in advance or a blur pattern calculated by simulation.
  • these methods have a drawback that the EDOF effect is small in principle compared with the above three methods.
  • Focal Stack has also existed for a long time.
  • a plurality of images with different in-focus positions (focus positions) are photographed, and regions that are considered in-focus are extracted from the respective images.
  • the said system produces
  • this method since a large number of images are required, there are a problem that it takes a relatively long time for imaging and a problem that a large amount of memory is consumed.
  • CPM Cubic Phase Mask
  • FPM Free-Form Phase Mask
  • Non-Patent Document 7 a common defect of WFCs is that the characteristics outside the optical axis of the lens are deteriorated by inserting a phase plate (see Non-Patent Document 7). Specifically, compared with incident light from the front, the same blur uniformity effect cannot be obtained for incident light from other than the front. For this reason, if restoration processing is performed with the blur pattern on the axis during image restoration, the image quality outside the optical axis after restoration is degraded.
  • the second CA in the three methods mentioned above increases the distance measurement accuracy by using an unusual aperture shape. Due to such characteristics of this method itself, a specific frequency component of a captured image and an image obtained after restoration processing is lost. That is, this method has a drawback that the image quality is deteriorated. In addition, this method is generally not suitable for shooting in a dark place because the amount of light is smaller than that of a normal shooting method regardless of the aperture shape.
  • the third F-DOF in the above-mentioned three systems is a system that can obtain the best image quality, and has a high EDOF effect.
  • the off-axis characteristics also depend on the lens characteristics themselves, so it is easy to improve performance.
  • it is necessary to use an image-side telecentric lens because the same subject needs to be folded on the same image position even if the focus position during exposure is moved.
  • EDOF technology examples include cameras mounted on mobile phones in recent years.
  • the camera can be downsized. That is, the EDOF effect makes it possible to obtain an omnifocal image (an image in which all subjects are in focus) without having an autofocus mechanism.
  • F-DOF itself is not adopted because it requires a mechanism to move the focus lens or image sensor, and a method using WFC or axial chromatic aberration is adopted. Yes.
  • a normal digital still camera can be considered.
  • the EDOF technique can be expected to have an effect of releasing from an all-in-focus image, that is, a focus error.
  • This application requires high image quality, the ability of the EDOF effect, the ability to arbitrarily change the EDOF range, and the ability to be realized by applying a normal autofocus mechanism (special optics F-DOF is the most excellent of the above methods because it is not necessary to prepare a system) and it is easy to switch between EDOF photographing and normal photographing.
  • a normal autofocus mechanism special optics F-DOF is the most excellent of the above methods because it is not necessary to prepare a system
  • German Patent No. 2301800 German Patent: Application 1973/1/15
  • Japanese Patent Publication No. 5-27084 Japanese Patent No. 3191928 US Patent Application Publication No. 2008/0013941 Japanese Patent No. 3084130
  • FIG. 13 is a block diagram showing a configuration of an imaging apparatus 500 that displaces the focus lens during the exposure time.
  • An imaging apparatus 500 shown in FIG. 13 includes an imaging element 1, a lens 2, a focus lens 20, a shutter 3, a focus lens displacement control unit 4, a shutter opening / closing instruction unit 5, a release receiving unit 6, and a focus.
  • the focus lens initial position detecting unit 7 detects the current position (initial position) of the focus lens 20. Then, the focus lens displacement control unit 4 moves the focus lens 20 so that the focus position moves to the end position of a predetermined focus range, for example, the closest end as shown in FIG.
  • a predetermined focus range for example, the closest end as shown in FIG.
  • the position closest to the imaging apparatus 500 with the imaging apparatus 500 as a reference is the nearest end
  • the position farthest from the imaging apparatus 500 is the farthest end.
  • the exposure / focus displacement synchronization management unit 10 instructs the shutter opening / closing instruction unit 5 to open / close the shutter 3, and simultaneously moves the focus lens 20 to the focus lens displacement control unit 4 from the nearest end to the farthest end. Instruct to move. Then, as shown in FIG. 15, the focus lens displacement control unit 4 moves the focus lens 20 so that the in-focus position reaches the farthest end simultaneously with the end of exposure.
  • FIG. 14 is a block diagram showing a configuration of an imaging apparatus 600 that displaces the imaging element 1 during the exposure time.
  • An imaging apparatus 600 shown in FIG. 14 includes a focus lens displacement control unit 4, a focus lens initial position detection unit 7, an exposure / focus displacement synchronization management unit 10, and a focus lens position of the imaging apparatus 500 that displaces the focus lens 20 described above.
  • an image sensor initial position detection unit 14 an exposure / image sensor displacement synchronization management unit 16, an image sensor displacement control unit 17, and an image sensor position initialization unit 19 are provided.
  • the imaging apparatus 600 shown in FIG. 14 is different from the imaging apparatus 500 shown in FIG. 13 that moves the focus lens 20 in that the imaging element 1 is moved to displace the in-focus position. Since other configurations and operations are the same as those of the imaging apparatus 500 shown in FIG. 13, the corresponding components are denoted by the same reference numerals, and detailed description thereof is omitted. Although not shown, the imaging apparatus 600 also includes the lens 2 including the focus lens 20.
  • the imaging apparatuses 500 and 600 that employ the F-DOF method need to initialize the position of the focus lens 20 or the imaging element from when the shutter button is pressed to when actual exposure is started. .
  • F-DOF is significantly disadvantageous compared to other EDOF systems.
  • the shutter time lag is the time required from when the user gives an instruction to start exposure (that is, when the shutter button is pressed) until the exposure is actually started.
  • the breakdown of the shutter time lag is the sum of the time required for focusing with a camera having an autofocus mechanism and the time required for various other processes (called release time lag).
  • the breakdown of the release time lag includes the time required for a camera such as a single-lens reflex camera to jump up, and the time required to squeeze the diaphragm blades according to a predetermined exposure.
  • the shutter time lag normally takes 100 milliseconds to several seconds, whereas the release time lag is about 10 to 130 milliseconds, which is overwhelmingly longer for autofocus.
  • the time required for focusing can be omitted because the autofocus mechanism itself is not required to obtain an omnifocal image. As a result, the shutter time lag can be greatly reduced.
  • the imaging apparatuses 500 and 600 that employ F-DOF as described above, it is possible to perform omnifocal image capturing, normal autofocus imaging, and manual focus imaging.
  • EDOF technology not only provides users with the value of being free from focusing errors by taking all-in-focus images, but also significantly reduces the shutter time lag, that is, immediately after the moment you want to take a picture. It is easily speculated that the provision of the advantage of cutting off can also be one of great value.
  • the conventional configuration has the problem that the latter advantage cannot be obtained because the processing is performed in the above-described procedure.
  • the present invention solves the above-described conventional problems, and an object of the present invention is to provide an imaging apparatus and an imaging method with a small shutter time lag even in F-DOF.
  • An imaging apparatus includes an imaging element, a lens that focuses light on the imaging element to form an image, and an in-focus position at the time when an instruction to start exposure is received from a user.
  • An initial focus position detection unit that detects a positional relationship between the imaging element and the lens for specifying a certain initial focus position, and an imaging in which the focus position during the exposure time is detected by the initial focus position detection unit From the initial in-focus position specified by the positional relationship between the element and the lens, pass through the nearest end and the farthest end of the predetermined in-focus range at least once and return to the initial in-focus position again.
  • the focus position is moved from the initial focus position at the start of exposure, and is again initialized at the end of exposure. Focus As to reach the location, and a displacement control unit for moving one of the imaging element and the lens.
  • the shutter time lag can be significantly reduced.
  • the order of passage of the nearest end and the farthest end may be either first.
  • the number of times of passage at the nearest end and the farthest end is not limited to one time, and may be a plurality of times. However, it is desirable that the number of passages at the nearest end and the farthest end coincide.
  • the imaging apparatus may include an exposure time determining unit that determines an exposure time according to an imaging scene.
  • the displacement pattern determination unit may increase the number of reciprocations indicating how many times the displacement pattern is executed during the exposure time as the exposure time determined by the exposure time determination unit is longer. Thereby, even if the exposure time changes, the displacement speed of the image sensor or the lens becomes substantially the same, so the burden on the driving device such as a motor is reduced.
  • the displacement pattern determination unit may return the focus position during the exposure time from the initial focus position to the initial focus position again through the closest end of the focus range and the farthest end of the focus range.
  • the displacement pattern of the focus position may be determined so as to return.
  • the displacement pattern determination unit determines that the in-focus position during the exposure time passes from the initial in-focus position to the farthest end of the in-focus range and the closest end of the in-focus range, and then again the initial in-focus. You may determine the displacement pattern of a focus position so that it may return to a position.
  • the displacement control unit may displace the in-focus position by moving the lens.
  • the displacement control unit may displace the in-focus position by moving the image sensor.
  • examples of specific methods for changing the in-focus position include a method for moving the lens and a method for moving the image sensor.
  • a mechanism for moving a lens can be shared with autofocus by adopting a method for moving the lens.
  • the imaging apparatus performs image restoration processing using the restoration PSF on the imaging data generated by the PSF storage unit that stores the restoration PSF (Point Spread Function) in advance and the imaging element.
  • An imaging method is a method in which an imaging device including an imaging device and a lens that focuses light on the imaging device to form an image picks up an image.
  • an initial focus position detection step for detecting a positional relationship between the imaging element and the lens for specifying an initial focus position that is a focus position at the time when an instruction to start exposure is received from a user;
  • the focus position during the exposure time is determined based on the initial focus position specified by the positional relationship between the imaging element and the lens detected in the initial focus position detection step.
  • a displacement pattern determining step for determining a displacement pattern at the in-focus position so as to return to the initial in-focus position again at least once through the end and the farthest end, and the displacement pattern determined in the displacement pattern determining step. Based on this, the imaging element and the lens are moved so that the in-focus position is moved from the initial in-focus position at the start of exposure, and is again reached at the end of exposure. And a displacement control step of moving one of the.
  • the present invention can be realized not only as an imaging apparatus and an imaging method, but also as a program for causing a computer to execute steps included in the imaging method, and a semiconductor integrated circuit that realizes part of the functions of the imaging apparatus It can be realized as (LSI).
  • the program can be distributed via a non-transitory computer-readable recording medium such as a CD-ROM and a transmission medium such as the Internet.
  • the shutter time lag in the F-DOF method can be significantly reduced by devising a displacement method during exposure of the image sensor or the focus lens.
  • FIG. 1 is a block diagram showing a schematic configuration of an imaging apparatus according to Embodiment 1 of the present invention.
  • FIG. 2 is a flowchart showing the operation of the imaging apparatus of FIG.
  • FIG. 3 is a block diagram showing a detailed configuration of the imaging apparatus according to Embodiment 1 of the present invention.
  • FIG. 4 is a block diagram showing a detailed configuration of the imaging apparatus according to Embodiment 1 of the present invention.
  • FIG. 5 is a diagram in which the positional relationship between the subject distance u and the image plane side distance v is defined.
  • FIG. 6 is a graph showing an example of the relationship between the subject distance u and the image plane side distance v.
  • FIG. 7 is a diagram illustrating an example of a displacement pattern of the focus lens or the image sensor.
  • FIG. 1 is a block diagram showing a schematic configuration of an imaging apparatus according to Embodiment 1 of the present invention.
  • FIG. 2 is a flowchart showing the operation of the imaging apparatus of FIG.
  • FIG. 8 is a diagram illustrating an example of a displacement pattern of the focus lens or the image sensor.
  • FIG. 9 is a diagram illustrating an example of a displacement pattern of the focus lens or the image sensor.
  • FIG. 10 is a diagram illustrating an example of a displacement pattern of the focus lens or the image sensor.
  • FIG. 11 is a block diagram showing a detailed configuration of the imaging apparatus according to Embodiment 2 of the present invention.
  • FIG. 12 is a block diagram showing a detailed configuration of the imaging apparatus according to Embodiment 2 of the present invention.
  • FIG. 13 is a block diagram of a conventional imaging device.
  • FIG. 14 is a block diagram of a conventional imaging device.
  • FIG. 15 is a diagram showing an example of a conventional focus lens or image sensor displacement pattern.
  • FIG. 16 illustrates the relationship between the shutter time lag and the release time lag.
  • Embodiment 1 An imaging apparatus according to Embodiment 1 of the present invention will be described with reference to FIGS.
  • FIG. 1 is a block diagram illustrating a schematic configuration of the imaging apparatus 1000.
  • FIG. 2 is a flowchart showing the operation of the imaging apparatus 1000.
  • the imaging apparatus 1000 includes an imaging device 110, a lens 120, an initial focus position detection unit 130, a displacement pattern determination unit 140, and a displacement control unit 150.
  • the image sensor 110 converts the formed image into an electrical signal and outputs it.
  • the specific example of the image pick-up element 110 is not specifically limited, For example, CCD (Charge Coupled Device Image Sensor) or CMOS (Complementary Metal Oxide Semiconductor Image Sensor) etc. are employable.
  • the lens 120 condenses light on the image sensor 110 to form an image.
  • the lens 120 includes a plurality of lenses including a focus lens.
  • the initial focus position detection unit 130 is an initial focus position (“focus initial position”) that is a focus position (also referred to as “focus position”, the same applies hereinafter) of the imaging apparatus 1000 when an instruction to start exposure is received from the user. The same shall apply hereinafter) (S11).
  • the in-focus position is determined by the positional relationship (more specifically, distance) between the image sensor 110 and the lens 120. That is, the initial focus position detection unit 130 actually detects the positional relationship between the imaging element 110 and the lens 120 for specifying the initial focus position.
  • the displacement pattern determination unit 140 is determined in advance from the initial focus position in which the focus position during the exposure time is specified by the positional relationship between the imaging element 110 and the lens 120 detected by the initial focus position detection unit 130.
  • a displacement pattern of the in-focus position is determined so as to return to the initial in-focus position again at least once through the nearest end and the farthest end of the in-focus range (S12).
  • the displacement pattern determined here may pass through either the nearest end or the farthest end of the focusing range first. That is, the in-focus position during the exposure time is a displacement pattern from the initial in-focus position to the initial focus position again through the closest end of the in-focus range and the farthest end of the in-focus range. Also good. Alternatively, the in-focus position during the exposure time is a displacement pattern from the initial in-focus position to the farthest end of the in-focus range and the closest end of the in-focus range, and then back to the initial in-focus position again. Also good.
  • the displacement pattern determination unit 140 determines the number of reciprocations indicating how many times the displacement pattern is executed during the exposure time. And the displacement pattern determination part 140 increases this reciprocation number, so that the exposure time determined in the exposure time determination part mentioned later is long. For example, if the number of reciprocations is 1 when the exposure time is 10 ms, the number of reciprocations is 2 when the exposure time is 20 ms. Thereby, even if the exposure time changes, the displacement speed of the image sensor 110 or the lens 120 becomes substantially the same, so the burden on a driving device such as a motor is reduced.
  • the displacement control unit 150 moves one of the image sensor 110 and the lens 120 in order to change the in-focus position based on the displacement pattern determined by the displacement pattern determination unit 140, so that the distance (relative (Position) is changed (S13). Specifically, the displacement control unit 150 moves the in-focus position from the initial in-focus position at the same time as the start of exposure so that the in-focus position reaches the initial in-focus position again at the same time as the exposure ends. One of the image sensor 110 and the lens 120 is moved.
  • the displacement control unit 150 may move the focus position by moving the image sensor 110 or may move the focus position by moving the lens 120.
  • FIG. 3 is a diagram showing a detailed configuration of the imaging apparatus 100 according to the first embodiment.
  • 3 includes an imaging device 1, a lens 2, a focus lens 20, a shutter 3, a focus lens displacement control unit 4, a shutter opening / closing instruction unit 5, a release receiving unit 6, and a focus.
  • the displacement control unit 150 corresponds to the pattern determination unit 9 and the focus lens displacement control unit 4, respectively.
  • FIG. 4 is a diagram illustrating a detailed configuration of the imaging apparatus 200 according to the first embodiment.
  • the configuration and operation are the same as those in FIG. 3 except that the image restoration processing unit 11 and the PSF storage unit 12 are omitted.
  • the imaging device 200 is characterized by directly recording an image obtained by exposure in the imaging data recording unit 13 as compared to the imaging device 100 shown in FIG. This is intended to realize the image restoration processing not in the imaging apparatus 200 but in another apparatus such as a personal computer, an image viewer, a network server, or the like thereafter. Since the other configuration is common to the imaging apparatus 100 of FIG. 3, the configuration and operation of the imaging apparatus 100 will be mainly described below.
  • the imaging apparatus 100 when the optical image of the subject is formed on the imaging device 1 by the lens 2 when the shutter 3 is in the open state, the formed optical image is converted into an electrical signal by the imaging device 1. .
  • the lens 2 is composed of a focus lens 20 and other lens groups in order to focus on a desired subject.
  • the focus lens 20 may be composed of a plurality of lenses.
  • the focus lens initial position detection unit 7 detects the current focus position (initial focus position) of the focus lens 20.
  • the exposure time determination unit 8 determines shooting parameters such as shutter speed (exposure time) and aperture value.
  • the exposure time may be determined by the exposure time determination unit 8 according to the surrounding brightness, for example, the exposure time is shortened when it is bright, and the exposure time is lengthened when it is dark. It may be determined by accepting a fast subject or a landscape.
  • the focus lens displacement pattern determination unit 9 uses the focus lens initial position information detected by the focus lens initial position detection unit 7 and the exposure time information determined by the exposure time determination unit 8, for example, as shown in FIGS. The correct focus displacement pattern.
  • the focus lens displacement pattern determination unit 9 determines a displacement pattern of the focus lens 20 according to the focus displacement pattern, and notifies the exposure / focus displacement synchronization management unit 10 of the displacement pattern.
  • the exposure / focus displacement synchronization management unit 10 performs synchronization management of exposure start and exposure end for the focus lens displacement control unit 4 and the shutter opening / closing instruction unit 5 based on the displacement pattern. That is, the exposure / focus displacement synchronization management unit 10 outputs an exposure start instruction to the shutter opening / closing instruction unit 5 and at the same time within the exposure time based on the displacement pattern determined by the focus lens displacement pattern determination unit 9. The focus lens displacement control unit 4 is instructed to displace the focus lens 20.
  • the shutter opening / closing instruction unit 5 performs control so that the shutter 3 opens as soon as an exposure start instruction is issued. After a predetermined exposure time, the exposure / focus displacement synchronization management unit 10 instructs the shutter opening / closing instruction unit 5 to end the exposure. The shutter opening / closing instruction unit 5 performs control so that the shutter 3 is closed as soon as an exposure end instruction is issued. Further, the focus lens displacement control unit 4 moves the focusing position from the initial focus position at the start of exposure based on the displacement pattern determined by the focus lens displacement pattern determination unit 9, and returns to the initial focus position at the end of exposure. The focus lens 20 is moved so as to reach it.
  • the exposure / focus displacement synchronization management unit 10 notifies the image restoration processing unit 11 that the exposure has been completed and that the focus displacement method has been captured using F-DOF.
  • the image restoration processing unit 11 After receiving the image signal, the image restoration processing unit 11 reads PSF data for restoration from the PSF storage unit 12 and performs image restoration processing. Specifically, a blur pattern due to focus displacement is obtained in advance by measurement or simulation, and this is stored in the PSF storage unit 12 as PSF data. Various methods such as Wiener Filter and Lucy-Richardson are known as image restoration methods, but any method may be used. Then, the image restoration processing unit 11 records the restored image signal in the imaging data recording unit 13 as imaging data.
  • FIG. 6 shows the relationship between the subject distance u and the image plane side distance v when f is 18 [mm].
  • the image plane side distance v which is the distance between the lens principal point and the image sensor, is displaced.
  • instructing the focus lens 20 to perform displacement control so that the focus position is displaced at a constant speed on the imaging device surface means that the changing speed of the image plane side distance v is constant.
  • the subject distance u which is the distance between the focal plane on the subject side and the lens principal point, is not displaced at a constant speed. Absent.
  • the vertical axis in FIGS. 7 to 10 is the image plane side distance v. For this reason, it should be noted that the relationship between the exposure time and the subject distance u is opposite in magnitude from the relationship between the exposure time and the image plane side distance v.
  • the closest end and the farthest end on the subject distance u side are switched on the image plane side to become the farthest end and the nearest end.
  • the focus position is displaced from the initial focus position to the nearest end on the subject side, and after reaching the nearest end, it is immediately turned back and displaced to the farthest end. After reaching the farthest end, it is immediately turned back again and displaced to the initial focus position.
  • the speed at the time of displacement is a constant speed, and this displacement speed is determined so as to reciprocate once in the exposure time determined by the exposure time determination unit 8.
  • the reason why the speed is constant in principle is that in order to perform the image restoration process with the uniform restoration PSF, it is necessary to make the blur uniform in the subject distance direction. In order to obtain uniform blur, it is necessary to make the displacement on the image plane side constant and to make the exposure amount from the nearest end to the farthest end constant.
  • FIG. 8 shows a case where the displacement pattern of FIG. 7 is reciprocated three times within the exposure time. In this way, the exposure amount from the nearest end to the farthest end can be made constant even if the reciprocation is made an integer number of times.
  • FIG. 9 shows an example of a displacement pattern that is displaced in the opposite direction to the displacement pattern of FIG.
  • the focus position is displaced from the initial focus position to the farthest end on the subject side, and after reaching the farthest end, it is immediately turned back and displaced to the nearest end. After reaching the closest end, it immediately turns back again and is displaced to the initial focus position.
  • the speed at the time of displacement is a constant speed, and this displacement speed is determined so as to reciprocate once in the exposure time determined by the exposure time determination unit 8.
  • FIG. 10 shows a case where the displacement pattern of FIG. 9 is reciprocated three times within the exposure time. As described above, even when the reverse displacement pattern is reciprocated an integer number of times, the exposure amount from the nearest end to the farthest end can be made constant.
  • FIG. 15 which is a conventional displacement pattern
  • FIGS. 7 to 10 which is a displacement pattern according to Embodiment 1 of the present application
  • the exposure amounts at the nearest and farthest ends are actually Strictly speaking, it is not constant compared with the exposure amount at the center.
  • the sharpest image is obtained at the in-focus position, but before and after that, the pattern has a pattern in which the sharpness gradually decreases as the distance increases. It is known.
  • the displacement is cut off, so that the defocus pattern from the side where the displacement disappears cannot contribute to the sharpness.
  • the sharpness at the nearest end and the farthest end is slightly lower than the sharpness at the center.
  • control may be performed such that the displacement speeds at the nearest end and the farthest end are reduced compared to the displacement speed at the center. In this case, it is necessary to take care that the time required for reciprocal displacement an integer number of times matches the exposure time.
  • the imaging apparatuses 100 and 200 having such a configuration perform the above-described control, so that the F-DOF system has high image quality, the magnitude of the EDOF effect, arbitrary setting of the EDOF range, EDOF imaging, and normal
  • the shutter time lag can be significantly reduced while maintaining the ease of switching between shootings.
  • Embodiment 2 An imaging apparatus according to Embodiment 2 of the present invention will be described with reference to FIGS.
  • FIG. 11 is a diagram illustrating a detailed configuration of the imaging apparatus 300 according to the second embodiment.
  • 11 includes an image sensor 1, a shutter 3, a shutter opening / closing instruction unit 5, a release receiving unit 6, an exposure time determining unit 8, an image restoration processing unit 11, and a PSF storage unit. 12, an imaging data recording unit 13, an imaging device initial position detection unit 14, an imaging device displacement pattern determination unit 15, an exposure / imaging device displacement synchronization management unit 16, and an imaging device displacement control unit 17.
  • the imaging apparatus 300 also includes the lens 2 including the focus lens 20.
  • the imaging device initial position detection unit 14 detects the current position (initial position) of the imaging device 1.
  • the exposure time determination unit 8 determines shooting parameters such as shutter speed and aperture value.
  • the exposure time may be determined by the exposure time determination unit 8 according to the surrounding brightness, for example, the exposure time is shortened when it is bright, and the exposure time is lengthened when it is dark. It may be determined by accepting a fast subject or a landscape.
  • the exposure / imaging element displacement synchronization management unit 16 performs an exposure start instruction on the imaging element displacement control unit 17 and the shutter opening / closing instruction unit 5 in synchronization with exposure start and exposure end. That is, the exposure / focus displacement synchronization management unit 10 outputs an exposure start instruction to the shutter opening / closing instruction unit 5 and at the same time within the exposure time based on the displacement pattern determined by the image sensor displacement pattern determination unit 15.
  • the image sensor displacement control unit 17 is instructed to displace the image sensor 1.
  • the shutter opening / closing instruction unit 5 performs control to open the shutter 3 as soon as an exposure start instruction is issued. Further, after a predetermined exposure time has elapsed, the exposure / imaging element displacement synchronization management unit 16 instructs the shutter opening / closing instruction unit 5 to end the exposure. The shutter opening / closing instruction unit 5 performs control so that the shutter 3 is closed as soon as an exposure end instruction is issued. Further, the image sensor displacement control unit 17 moves the focus position from the initial focus position at the start of exposure based on the displacement pattern determined by the image sensor displacement pattern determination unit 15, and then returns to the initial focus position at the end of exposure. The image sensor 1 is moved so as to reach it.
  • the formed optical image is converted into an electric signal (image signal) by the image sensor 1 and the image signal is transferred to the image restoration processing unit 11. .
  • the exposure / imaging element displacement synchronization management unit 16 notifies the image restoration processing unit 11 that the exposure has been completed and that imaging of the imaging element displacement method by F-DOF has been performed.
  • the other configuration conforms to the case where the focus lens 20 in FIG. 3 is displaced.
  • FIG. 12 is a diagram illustrating a detailed configuration of the imaging apparatus 400 according to the second embodiment.
  • the configuration and operation of the imaging apparatus 400 are the same as those in FIG. 11 except that the image restoration processing unit 11 and the PSF storage unit 12 are omitted.
  • the imaging apparatus 400 is characterized in that it directly records an image obtained by exposure in the imaging data recording unit 13. This is intended to realize the image restoration processing not in the imaging apparatus 400 but in another apparatus such as a personal computer, an image viewer, a network server, or the like thereafter.
  • the image sensor displacement pattern determination unit 15 uses the image sensor initial position information detected by the image sensor initial position detection unit 14 and the exposure time information determined by the exposure time determination unit 8, for example, as shown in FIGS. Such a focus displacement pattern is determined. Then, the image sensor displacement pattern determination unit 15 determines a displacement pattern of the image sensor 1 according to the focus displacement pattern and notifies the exposure / focus displacement synchronization management unit 10 of the displacement pattern.
  • the focus displacement pattern is the displacement pattern of the image pickup device 1 itself unless the position of the lens 2 is displaced.
  • the exposure / imaging element displacement synchronization management unit 16 performs synchronization management of exposure start and exposure end for the image sensor displacement control unit 17 and the shutter opening / closing instruction unit 5 based on the displacement pattern.
  • the displacement pattern of the image sensor 1 conforms to the patterns of FIGS. 7 to 10 shown in the first embodiment.
  • the configuration in which the image sensor 1 itself is displaced is the difference from the first embodiment, and the displacement pattern is common.
  • the imaging apparatuses 300 and 400 having such a configuration perform the above-described control, so that the F-DOF system has high image quality, the magnitude of the EDOF effect, arbitrary setting of the EDOF range, EDOF imaging, and normal
  • the shutter time lag can be significantly reduced while maintaining the ease of switching between shootings.
  • the imaging device according to the embodiment of the present invention has been described above, but the present invention is not limited to this embodiment.
  • At least a part of the plurality of processing units included in the imaging apparatus according to the above embodiment is realized as an LSI that is an integrated circuit. These may be individually made into one chip, or may be made into one chip so as to include a part or all of them.
  • circuits are not limited to LSI, and may be realized by a dedicated circuit or a general-purpose processor.
  • An FPGA Field Programmable Gate Array
  • reconfigurable processor that can reconfigure the connection and setting of circuit cells inside the LSI may be used.
  • a part of the functions of the imaging apparatus according to the embodiment of the present invention may be realized by a processor such as a CPU executing a program.
  • the present invention may be the above program or a non-transitory computer-readable recording medium on which the above program is recorded.
  • the program can be distributed via a transmission medium such as the Internet.
  • the image pickup apparatus and the image pickup method according to the present invention can realize a significant reduction in shutter time lag, which is a drawback of the F-DOF method, by devising how to move the image pickup element or the focus lens during exposure.

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Optics & Photonics (AREA)
  • Computing Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Studio Devices (AREA)
  • Automatic Focus Adjustment (AREA)
PCT/JP2010/007061 2009-12-07 2010-12-03 撮像装置および撮像方法 WO2011070757A1 (ja)

Priority Applications (4)

Application Number Priority Date Filing Date Title
EP10835683.3A EP2511747B1 (en) 2009-12-07 2010-12-03 Imaging device and imaging method
CN201080006774.5A CN102308242B (zh) 2009-12-07 2010-12-03 摄像装置以及拍摄方法
JP2011545076A JP5588461B2 (ja) 2009-12-07 2010-12-03 撮像装置および撮像方法
US13/147,886 US8412036B2 (en) 2009-12-07 2010-12-03 Imaging apparatus and method of controlling the apparatus

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2009277639 2009-12-07
JP2009-277639 2009-12-07

Publications (1)

Publication Number Publication Date
WO2011070757A1 true WO2011070757A1 (ja) 2011-06-16

Family

ID=44145319

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2010/007061 WO2011070757A1 (ja) 2009-12-07 2010-12-03 撮像装置および撮像方法

Country Status (5)

Country Link
US (1) US8412036B2 (zh)
EP (1) EP2511747B1 (zh)
JP (1) JP5588461B2 (zh)
CN (1) CN102308242B (zh)
WO (1) WO2011070757A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2013141192A (ja) * 2011-12-07 2013-07-18 Fujitsu Ltd 被写界深度拡張システム及び被写界深度拡張方法
US20130314586A1 (en) * 2011-03-02 2013-11-28 Panasonic Corporation Imaging device, semiconductor integrated circuit, and imaging method

Families Citing this family (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP5588461B2 (ja) * 2009-12-07 2014-09-10 パナソニック株式会社 撮像装置および撮像方法
WO2012124321A1 (ja) * 2011-03-14 2012-09-20 パナソニック株式会社 撮像装置、撮像方法、集積回路、コンピュータプログラム
CN102934003B (zh) * 2011-04-15 2016-06-08 松下电器产业株式会社 摄像装置、半导体集成电路以及摄像方法
US8890996B2 (en) * 2012-05-17 2014-11-18 Panasonic Corporation Imaging device, semiconductor integrated circuit and imaging method
US9191578B2 (en) * 2012-06-29 2015-11-17 Broadcom Corporation Enhanced image processing with lens motion
US9264630B2 (en) * 2013-01-04 2016-02-16 Nokia Technologies Oy Method and apparatus for creating exposure effects using an optical image stabilizing device
WO2014140413A1 (en) * 2013-03-15 2014-09-18 Nokia Corporation Apparatus, method and computer program for capturing images
US9485414B2 (en) 2014-06-20 2016-11-01 John Visosky Eye contact enabling device for video conferencing
JP5893713B1 (ja) * 2014-11-04 2016-03-23 オリンパス株式会社 撮像装置、撮像方法、処理プログラム
CN105491280A (zh) * 2015-11-23 2016-04-13 英华达（上海）科技有限公司 机器视觉中进行图像采集的方法及装置
CN114236745B (zh) * 2021-07-13 2024-04-05 广东弘景光电科技股份有限公司 基于三轴运动的镜头镜片调芯方法

Citations (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE2301800A1 (de) 1973-01-15 1974-10-10 Leitz Ernst Gmbh Verfahren zur erweiterung des schaerfentiefebereiches bei der optischen und elektronenmikroskopischen abbildung
JPH0527084B2 (zh) 1983-05-13 1993-04-20 Shimadzu Corp
JP3084130B2 (ja) 1992-05-12 2000-09-04 オリンパス光学工業株式会社 画像入力装置
JP3191928B2 (ja) 1988-02-23 2001-07-23 オリンパス光学工業株式会社 画像入出力装置
JP2003259194A (ja) * 2002-03-06 2003-09-12 Tamron Co Ltd 画像安定化装置
JP2007286118A (ja) * 2006-04-12 2007-11-01 Canon Inc 撮像装置及びその制御方法
US20080013941A1 (en) 2006-07-14 2008-01-17 Micron Technology, Inc. Method and apparatus for increasing depth of field for an imager
JP2008166947A (ja) * 2006-12-27 2008-07-17 Eastman Kodak Co 撮像装置
JP2009010521A (ja) * 2007-06-26 2009-01-15 Olympus Imaging Corp 撮像装置
JP2009098351A (ja) * 2007-10-16 2009-05-07 Fujifilm Corp 撮影方法及び装置
JP2009139867A (ja) * 2007-12-10 2009-06-25 Samsung Techwin Co Ltd 撮像装置および撮像方法
JP2009213032A (ja) * 2008-03-06 2009-09-17 Renesas Technology Corp ワイド・ダイナミック・レンジ画像生成処理装置

Family Cites Families (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE3905619C2 (de) 1988-02-23 2000-04-13 Olympus Optical Co Bildeingabe-/Ausgabevorrichtung
DE3931934C2 (de) 1988-10-03 1994-11-10 Olympus Optical Co Bild-Ein/Ausgabevorrichtung
US5237363A (en) * 1989-08-31 1993-08-17 Minolta Camera Kabushiki Kaisha Camera system with lens movable in vertical direction to optical axis of photo-taking lens
JPH0498230A (ja) * 1990-08-17 1992-03-30 Olympus Optical Co Ltd 撮影倍率設定モードを有するカメラ
JPH0527084A (ja) 1991-07-25 1993-02-05 Toshiba Eng & Constr Co Ltd 燃料集合体のチヤンネルボツクス載置確認装置
US5253007A (en) * 1991-08-30 1993-10-12 Canon Kabushiki Kaisha Camera having special photography function
JPH05249364A (ja) * 1992-03-06 1993-09-28 Canon Inc カメラ及びズ−ムレンズ
GB2271646B (en) * 1992-10-19 1996-12-04 Asahi Optical Co Ltd An optical camera and a method of conducting a focusing operation therein
JP2004153497A (ja) 2002-10-30 2004-05-27 Kyocera Corp ディジタルカメラの自動露出制御システム
US7493030B2 (en) * 2005-06-24 2009-02-17 Nokia Corporation Adaptive optical plane formation with rolling shutter
CN101191980B (zh) * 2006-11-27 2010-09-01 亚洲光学股份有限公司 自动对焦方法
JP2009031760A (ja) * 2007-07-04 2009-02-12 Sanyo Electric Co Ltd 撮像装置及びオートフォーカス制御方法
JP2009086017A (ja) * 2007-09-27 2009-04-23 Kyocera Corp 撮像装置および撮像方法
WO2009120718A1 (en) * 2008-03-24 2009-10-01 The Trustees Of Columbia University In The City Of New York Methods, systems, and media for controlling depth of field in images
JP5588461B2 (ja) * 2009-12-07 2014-09-10 パナソニック株式会社 撮像装置および撮像方法
JP5367094B2 (ja) * 2009-12-07 2013-12-11 パナソニック株式会社 撮像装置及びその制御方法
US20120026360A1 (en) * 2010-02-08 2012-02-02 Panasonic Corporation Imaging device

Patent Citations (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE2301800A1 (de) 1973-01-15 1974-10-10 Leitz Ernst Gmbh Verfahren zur erweiterung des schaerfentiefebereiches bei der optischen und elektronenmikroskopischen abbildung
JPH0527084B2 (zh) 1983-05-13 1993-04-20 Shimadzu Corp
JP3191928B2 (ja) 1988-02-23 2001-07-23 オリンパス光学工業株式会社 画像入出力装置
JP3084130B2 (ja) 1992-05-12 2000-09-04 オリンパス光学工業株式会社 画像入力装置
JP2003259194A (ja) * 2002-03-06 2003-09-12 Tamron Co Ltd 画像安定化装置
JP2007286118A (ja) * 2006-04-12 2007-11-01 Canon Inc 撮像装置及びその制御方法
US20080013941A1 (en) 2006-07-14 2008-01-17 Micron Technology, Inc. Method and apparatus for increasing depth of field for an imager
JP2008166947A (ja) * 2006-12-27 2008-07-17 Eastman Kodak Co 撮像装置
JP2009010521A (ja) * 2007-06-26 2009-01-15 Olympus Imaging Corp 撮像装置
JP2009098351A (ja) * 2007-10-16 2009-05-07 Fujifilm Corp 撮影方法及び装置
JP2009139867A (ja) * 2007-12-10 2009-06-25 Samsung Techwin Co Ltd 撮像装置および撮像方法
JP2009213032A (ja) * 2008-03-06 2009-09-17 Renesas Technology Corp ワイド・ダイナミック・レンジ画像生成処理装置

Non-Patent Citations (8)

* Cited by examiner, † Cited by third party
Title
A. LEVIN, R. FERGUS, F. DURAND, W. T. FREEMAN: "Image and Depth from a Conventional Camera with a Coded Aperture", ACM TRANSACTIONS ON GRAPHICS, vol. 26, no. 3, 2007, pages 70 - 1,70-9
C. TISSE, H. P. NGUYEN, R. TESIERES, M. PYANET, F. GUICHARD: "Extended Depth-of-field (EDOF) using sharpness transport across color channels", OPTICAL ENGINEERING + APPLICATIONS, PART OF SPIE OPTICS + PHOTONICS, SESSION 1 -IMAGING IN THE OPTICAL DESIGN PROCESS: DEPTH OF FIELD, 2008
E. R. DOWSKI, W. T. CATHEY: "Extended depth of field through wave-front coding", APPLIED OPTICS, vol. 34, no. 11, 1995, pages 1859 - 1866, XP002518082, DOI: doi:10.1364/AO.34.001859
H. NAGAHARA, S. KUTHIRUMMAL, C. ZHOU, S. NAYAR: "Flexible Depth of Field Photography", EUROPEAN CONFERENCE ON COMPUTER VISION (ECCV, 16 October 2008 (2008-10-16)
See also references of EP2511747A4
W. CHI, N. GEORGE: "Computational imaging with the logarithmic asphere: theory", OPTICAL SOCIETY OF AMERICA, vol. 20, no. 12, December 2003 (2003-12-01), XP055059195, DOI: doi:10.1364/JOSAA.20.002260
Y. TAKAHASHI, R. OBANA, S. KOMATSU: "Optimized phase mask for wave-front coding: Extended DOF in off axis field", OPTICS AND PHOTONICS JAPAN, 2007, pages 464 - 465
Y. TAKAHASHI, S. KOMATSU: "Optimized free-form phase mask for extension of depth of field in wavefront-coded imaging", OPTICAL SOCIETY OF AMERICA, OPTICS LETTERS, vol. 33, no. 13, 1 July 2003 (2003-07-01)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130314586A1 (en) * 2011-03-02 2013-11-28 Panasonic Corporation Imaging device, semiconductor integrated circuit, and imaging method
US9083880B2 (en) * 2011-03-02 2015-07-14 Panasonic Corporation Imaging device, semiconductor integrated circuit, and imaging method
JP2013141192A (ja) * 2011-12-07 2013-07-18 Fujitsu Ltd 被写界深度拡張システム及び被写界深度拡張方法

Also Published As

Publication number Publication date
JP5588461B2 (ja) 2014-09-10
CN102308242A (zh) 2012-01-04
EP2511747B1 (en) 2017-11-08
JPWO2011070757A1 (ja) 2013-04-22
CN102308242B (zh) 2014-08-20
EP2511747A4 (en) 2014-11-05
EP2511747A1 (en) 2012-10-17
US8412036B2 (en) 2013-04-02
US20110292364A1 (en) 2011-12-01

Similar Documents

Publication Publication Date Title
JP5588461B2 (ja) 撮像装置および撮像方法
JP5367094B2 (ja) 撮像装置及びその制御方法
JP5911531B2 (ja) 光学機器
JP4794963B2 (ja) 撮像装置および撮像プログラム
US8447179B2 (en) Method and apparatus for applying multi-autofocusing (AF) using contrast AF
JP2007096753A (ja) デジタルカメラ
JP5882898B2 (ja) 撮像装置、撮像方法、集積回路、コンピュータプログラム
JP2002258147A (ja) 撮像装置、焦点調整方法および記憶媒体
WO2012105222A1 (ja) 画像復元装置、撮像装置及び画像復元方法
JP2010026120A (ja) カメラシステム及びアダプタ
WO2017154366A1 (ja) ローパスフィルタ制御装置、およびローパスフィルタ制御方法、ならびに撮像装置
JP6218385B2 (ja) 交換レンズ及び撮像装置
JP2010145495A (ja) カメラシステム
JP2009092961A (ja) 合焦位置制御装置、及び合焦位置制御方法、合焦位置制御プログラム
JP2014142497A (ja) 撮像装置及びその制御方法
JP2017207702A (ja) 焦点検出装置及び焦点検出方法
JP5438496B2 (ja) 撮像装置及びその制御方法、プログラム
JP2007213047A (ja) 焦点調節装置、光学機器および焦点調節方法
JP5773659B2 (ja) 撮像装置および制御方法
WO2024058029A1 (ja) 撮像装置、情報処理装置及びプログラム
WO2014136703A1 (ja) 撮像装置及び画像表示方法
JP3841241B2 (ja) カメラのピント調整方法
JP6429485B2 (ja) 撮像装置、通信制御方法および撮像システム
JP2014179782A (ja) 撮像装置及び画像表示方法
JP2013178323A (ja) 撮像装置

Legal Events

Date Code Title Description
WWE Wipo information: entry into national phase

Ref document number: 201080006774.5

Country of ref document: CN

ENP Entry into the national phase

Ref document number: 2011545076

Country of ref document: JP

Kind code of ref document: A

REEP Request for entry into the european phase

Ref document number: 2010835683

Country of ref document: EP

WWE Wipo information: entry into national phase

Ref document number: 13147886

Country of ref document: US

Ref document number: 2010835683

Country of ref document: EP

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 10835683

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE