JP5407547B2 - Imaging device - Google Patents

Description

  The present invention relates to an imaging apparatus, and more particularly to an imaging apparatus that combines a plurality of images in order to expand a substantial dynamic range.

  2. Description of the Related Art Conventionally, there has been proposed an imaging apparatus that expands a substantial dynamic range by performing an imaging operation a plurality of times under different exposure conditions and synthesizing the obtained images.

  Japanese Patent Application Laid-Open No. 2004-228561 discloses an imaging device that extracts a compositable region based on an image shift between frames and synthesizes images in the region to expand a substantial dynamic range.

JP 05-7336 A

  However, since the image shift between frames is not eliminated, the compositable region is smaller than the image obtained by imaging.

  Accordingly, an object of the present invention is to provide an imaging apparatus that performs an imaging operation in a state where there is no image shift between frames.

  An image pickup apparatus according to the present invention includes an image pickup element that picks up an optical image incident through a lens and a movable lens included in the lens or an image pickup element perpendicular to the optical axis of the lens for an image pickup operation. A control unit that performs movement control to move relative to a lens other than the movable lens on a flat surface, and the imaging operation is performed a plurality of times, at least at the start of the exposure time of the plurality of imaging operations, In order to make the position of the subject's image sensor on the imaging surface constant, the position at the start of the exposure time of the second and subsequent imaging operations and the start of the exposure time of the first imaging operation among the multiple imaging operations Based on the difference from the position at the time, movement control at the start of the exposure time of the second and subsequent imaging operations is performed.

  Difference between the holding state of the imaging device, that is, the position of the imaging device at the start of the exposure time of the first imaging operation (holding state) and the position of the imaging device at the start of the exposure time of the second imaging operation (holding state) Accordingly, the imaging element (or movable lens) is moved, so that the imaging operation can be performed a plurality of times with no deviation between frames, that is, with the imaging plane position of the subject image kept constant. I can do it. For this reason, it is possible to perform the synthesis of the images obtained by the plurality of imaging operations performed to widen the substantial dynamic range without any deviation. For this reason, it is possible to obtain a composite image using the entire range of the image obtained by imaging.

  Preferably, even during the exposure time of a plurality of imaging operations, movement control is performed based on the position at the start of the exposure time of the first imaging operation in order to make the position of the subject on the imaging plane constant. Is called.

  Accordingly, it is possible to keep the imaging plane position of the subject constant in consideration of camera shake occurring during the exposure time.

  Preferably, the control unit generates a composite image by combining images obtained by a plurality of imaging operations.

  Preferably, the plurality of imaging operations are performed under different exposure conditions.

  As described above, according to the present invention, it is possible to provide an imaging apparatus that performs an imaging operation in a state where there is no image shift between frames.

It is a block diagram of the imaging device in this embodiment. It is a figure which shows the relationship between the 1st deviation angle and 1st difference in this embodiment. It is a flowchart which shows the procedure of the imaging operation of a multiple times, and a synthetic | combination process procedure. It is a figure which shows the relationship between the 1st deviation angle, a 1st difference, etc. in the imaging device of the still image which has a camera shake correction function. It is a figure which shows the relationship between the 1st shift angle in the imaging device of the moving image which has a camera shake correction function, and the shift angle after a camera shake correction process.

  Hereinafter, the present embodiment will be described with reference to the drawings. The imaging device 1 is a digital camera, and includes an operation unit 11, an AF unit 13, an AE unit 15, an aperture 17, a lens 19, a mirror 21, a shutter 23, an imaging unit 25 including an imaging element such as a CCD, a DSP, a CPU, and the like. A signal processing unit 27, an internal memory 29, a flash memory 31, an external memory 33, a display unit 35, a detection unit 37, and a correction unit 39 are provided.

  The operation unit 11 includes a release button, a use mode selection key, and a correction mode selection key.

  By pressing a release button (not shown) included in the operation unit 11 halfway, the photometry switch is turned on, and the AF unit 13 performs distance measurement and the AE unit 15 performs photometry. The distance measurement result is output from the AF unit 13 to the signal processing unit 27 and used for the focusing operation. The photometric result is output from the AE unit 15 to the signal processing unit 27 and used for calculation of exposure conditions (shutter speed and aperture).

  Further, when the release button is fully pressed, the release switch is turned on, and an imaging operation by the imaging unit 23 or the like is performed. In the imaging operation, the aperture 17 is opened / closed, the mirror 21 is opened / closed, the shutter 23 is opened / closed, and the imaging unit 25 is exposed at a predetermined timing.

  By operating a use mode selection key included in the operation unit 11, the first use mode and the second use mode are switched. When the imaging apparatus 1 is in the first use mode, a plurality of imaging operations are continuously performed with different exposure conditions, and a plurality of images obtained by the plurality of imaging operations are combined, An image having a substantially wide dynamic range is obtained. When the imaging device 1 is in the second usage mode, a normal imaging operation is performed.

  By operating a correction mode selection key included in the operation unit 11, on / off control of the shift correction mode in the first use mode is performed. When the imaging apparatus 1 is in the first use mode and the deviation correction mode, the deviation correction that makes the position of the subject on the imaging surface of the imaging unit 25 constant at the start of the exposure time of a plurality of imaging operations. Processing is performed, and the position of the imaging unit 25 is adjusted. When the imaging device 1 is in the second usage mode, and when the imaging device 1 is in the first usage mode but not in the shift correction mode, the imaging unit 25 is fixed at the center of the movement range.

  The signal processing unit 27 performs image processing on the image signal obtained by the imaging unit 25 in the imaging operation. In addition, the signal processing unit 27 records the image signal in the external memory 33 in a state where no image processing is performed or a state where image processing is performed. In addition, the signal processing unit 27 causes the display unit 35 to display an image obtained by performing image processing.

  In addition, when the imaging device 1 is in the first use mode, the signal processing unit 27 controls each unit so that the imaging operation is continuously performed a plurality of times under different exposure conditions, and the imaging is performed a plurality of times. Image processing is performed on the image signal obtained by the operation, a plurality of images obtained by performing the image processing are combined, and a single combined image is generated. In particular, when the imaging apparatus 1 is in the shift correction mode, the correction unit 39 is driven based on information from the detection unit 37 described later and the imaging unit 25 is moved during a plurality of imaging operations. Control (deviation correction processing) is performed.

  The internal memory 29 is used for temporary recording of data in image processing. In particular, in the first use mode, a plurality of images used for composition are temporarily recorded. The flash memory 31 records a program for performing each operation in the imaging apparatus 1 such as image processing. In particular, a deviation correction processing program and a composition processing program for images obtained by a plurality of imaging operations are installed.

  The detection unit 37 includes an angular velocity sensor, detects a first angular velocity based on yawing of the imaging apparatus 1 and a second angular velocity based on pitching, and outputs the first angular velocity to the signal processing unit 27 at regular time intervals (for example, 1 ms). The signal processing unit 27 integrates these angular velocities with respect to time, and calculates a movement amount of the integration time (a first shift angle based on yawing and a second shift angle based on pitching). Accordingly, the first and second deviation angles are updated at regular intervals. When the imaging apparatus 1 is in the first use mode and the deviation correction mode, the signal processing unit 27 starts the exposure time in the first imaging operation among a plurality of imaging operations performed in the deviation correction process. The first and second misalignment angles are temporarily recorded in the internal memory 29 as first and second initial values, respectively.

  The correction unit 39 is driven by the signal processing unit 27 and moves the imaging unit 25 on a plane perpendicular to the optical axis LX of the lens 19. The movement control of the imaging unit 25 by the correction unit 39 is performed by PID control using movement by electromagnetic force and position detection by a hall sensor, for example.

  When the imaging apparatus 1 is in the first use mode and the deviation correction mode, the correction unit 39 sets the imaging unit 25 at the start of the exposure time in the first imaging operation among a plurality of imaging operations. The image capturing unit 25 is moved to the center of the moving range, and the image capturing unit 25 is displaced from the first and second initial values at the start of the exposure time in the second and subsequent image capturing operations (the first and second misalignment angles and the first and second misalignments) The first and second differences from the initial value are moved to a position to be corrected (see FIG. 2). During the exposure time, the correction unit 39 maintains the position at the start time of the exposure time. Thereby, there is no shift in the imaging plane of the subject image caused by movement of the holding position of the imaging device 1 such as camera shake between frames, and the imaging plane position of the subject image is kept constant.

  When the imaging apparatus 1 is in the first use mode but not in the misalignment correction mode, the correction unit 39 captures an image at the start of the exposure time in the first imaging operation among a plurality of imaging operations. The unit 25 is moved to the center of the movement range, and the fixation to the center of the movement range is maintained in the second and subsequent imaging operations. During the exposure time, the correction unit 39 maintains the position at the start of the exposure time (the center of the movement range). In the second usage mode, the correction unit 39 fixes the imaging unit 25 at the center of the movement range.

  Next, a plurality of imaging operations in the first use mode and a composition processing procedure will be described with reference to the flowchart of FIG.

  The detection of the first and second angular velocities by the detection unit 37 and the calculation of the first and second deviation angles by the signal processing unit 27 are performed at regular intervals from the time when the power of the imaging device 1 is turned on. Has been done.

  When the release button is fully pressed in a state where the imaging device 1 is in the first use mode, in step S11, the signal processing unit 27 determines whether or not the imaging device 1 is in the deviation correction mode. If it is not in the deviation correction mode, the process proceeds to step S12. If it is in the deviation correction mode, the process proceeds to step S16.

  In step S12, the signal processing unit 27 drives the correction unit 39, moves the imaging unit 25 to the center of the movement range, and fixes the position to such a position until the start of the exposure time of the next imaging operation. In step S <b> 13, the signal processing unit 27 controls each unit of the imaging apparatus 1 to perform the imaging operation, and the imaging operation is performed. The image obtained by the imaging operation in step S13 is temporarily recorded in the internal memory 29. In step S14, the signal processing unit 27 determines whether or not a plurality of imaging operations set to obtain a composite image have been performed. In the present embodiment, an example in which the number of imaging operations set to obtain a composite image is two times will be described, but a mode in which three or more times are performed may be used. If so, the process proceeds to step S15. If not, the process returns to step S11, and the next imaging operation is performed while changing the exposure conditions.

  When a plurality of images are obtained, in step S15, the signal processing unit 27 synthesizes the plurality of images temporarily recorded in the internal memory 29 to generate a single image with a substantial dynamic range expanded. Then, recording in the external memory 33 and display on the display unit 35 are performed.

  In step S <b> 16, the signal processing unit 27 determines whether or not the imaging operation to be performed is the first imaging operation among a plurality of imaging operations set to obtain a composite image. If it is the first imaging operation, the process proceeds to step S17, and if it is the second or subsequent imaging operation, the process proceeds to step S18.

  In step S <b> 17, the signal processing unit 27 drives the correction unit 39 to move the imaging unit 25 to the center of the movement range, and fixes the position until the start of the exposure time of the next imaging operation. Further, the signal processing unit 27 temporarily records the first and second deviation angles at the start of the exposure time in the imaging operation in the internal memory 29 as the first and second initial values, respectively.

  In step S <b> 18, the signal processing unit 27 determines the first and second differences (first and second deviation angles at the start of the exposure time in the current imaging operation, and the first and second initial temporarily recorded in the internal memory 29. Difference). In step S <b> 19, the signal processing unit 27 determines whether the correction unit 39 can move the imaging unit 25 to a position for correcting the deviation based on the first and second differences. That is, it is determined whether or not the movement amount of the imaging unit 25 based on the first and second differences is within the movement range of the imaging unit 25. If it can be moved, the process proceeds to step S20; otherwise, the process proceeds to step S21.

  In step S20, the signal processing unit 27 drives the correction unit 39, moves the imaging unit 25 to a position for correcting the deviation based on the first and second differences, and starts the exposure time of the next imaging operation. Until it is fixed.

  In step S21, the signal processing unit 27 displays a warning on the display unit 35 that the deviation correction process cannot be continued, and the procedure is terminated.

  In the present embodiment, the holding state (first and second misalignment angles) of the imaging device 1, that is, the position (holding state) of the imaging device 1 at the start of the exposure time of the first imaging operation and the second imaging operation. Since the imaging unit 25 is moved according to the difference (first and second differences) from the position (holding state) of the imaging device 1 at the start of the exposure time, there is no deviation between frames, that is, the subject image The imaging operation can be performed a plurality of times while the imaging plane position is kept constant. For this reason, it is possible to perform the synthesis of the images obtained by the plurality of imaging operations performed to widen the substantial dynamic range without any deviation. For this reason, it is possible to obtain a composite image using the entire range of the image obtained by imaging.

  Further, even if the movement control of the imaging unit 25 is performed, the first position is such that the position of the subject on the imaging surface of the imaging element cannot be made constant at least at the start of the exposure time of the plurality of imaging operations. When the second difference is large, that is, when the shift cannot be eliminated by the movement of the imaging unit 25, it is determined that a composite image without a shift cannot be obtained, and the imaging operation and the image combining process are performed. Cancel.

  When a still image capturing device having a camera shake correction function continuously captures still images, the image capturing unit (or image blur correcting lens) moves to correct blur during the exposure period. Every time exposure is started in the imaging operation, the imaging unit is moved to the center of the movement range (see FIG. 4). For this reason, the imaging plane position of the subject image cannot be kept constant between frames.

  In addition, in a moving image capturing apparatus having a camera shake correction function, when capturing a moving image, the imaging unit is positioned at the center of the moving range at the start of the exposure time in the first imaging operation among a plurality of imaging operations. After that, the image pickup unit reduces the amount of deviation from the first and second initial values (first and second differences between the first and second deviation angles and the first and second initial values). Although it is adjusted to obtain a moving image with less blurring after taking into account large blurring that cannot be handled by camera shake correction such as panning, it is moved to a position where the deviation amount is completely zero. (See FIG. 5). For this reason, the imaging plane position of the subject image cannot be kept constant between frames.

  In addition, although the detection part 37 demonstrated the form which the 1st deviation angle based on yawing and the 2nd deviation angle based on pitching were detected in this embodiment, in addition to these, the 3rd deviation angle based on rolling was set. The form to detect may be sufficient. In this case, the image capturing unit 25 is controlled to move including rotation on a plane perpendicular to the optical axis LX of the lens 19 to perform a shift correction process based on the first to third shift angles.

  Specifically, the signal processing unit 27 temporarily records in the internal memory 29 the first to third deviation angles at the start of the exposure time in the imaging operation as first to third initial values, respectively. The signal processing unit 27 performs the first to third differences (difference between the first to third deviation angles at the start of the exposure time in the current imaging operation and the first to third initial values temporarily recorded in the internal memory 29. ) Is calculated. The signal processing unit 27 determines whether or not the correction unit 39 can move the imaging unit 25 to a position for correcting the deviation based on the first to third differences.

  If the signal can be moved, the signal processing unit 27 drives the correction unit 39 to move the imaging unit 25 including the rotation to a position for correcting the deviation based on the first to third differences. The position is fixed until the exposure time start point of the imaging operation is started. If it cannot be moved, the signal processing unit 27 displays a warning to the effect that the displacement correction process cannot be continued on the display unit 35, and stops the imaging operation and the image synthesis process.

  The third shift angle may be detected by an angular velocity sensor or an acceleration sensor that detects a gravitational acceleration component in a predetermined direction.

  Further, in the present embodiment, the mode in which the shift correction processing is performed by controlling the movement of the imaging unit 25 has been described. However, the movable lens provided in the lens 19 is moved and controlled on a plane perpendicular to the optical axis LX. The form which performs a deviation correction process may be sufficient. However, in this case, the deviation based on rolling is not considered.

  In the present embodiment, the position of the imaging unit 25 at the start of the second and subsequent exposure times is controlled based on the first and second differences, and the imaging plane position of the subject between frames is kept constant. As described above, the imaging unit 25 is moved so that the first and second differences are calculated at regular intervals during the exposure time and the shift (camera shake) is corrected based on the first and second differences. You may control. Accordingly, it is possible to keep the imaging plane position of the subject constant in consideration of camera shake occurring during the exposure time.

  In the present embodiment, the form in which the imaging operation is performed a plurality of times with different exposure conditions has been described. However, a form in which the imaging operation is performed a plurality of times with the same exposure condition may be employed.

DESCRIPTION OF SYMBOLS 1 Imaging device 11 Operation part 13 AF part 15 AE part 17 Aperture 19 Lens 21 Mirror 23 Shutter 25 Imaging part 27 Signal processing part 29 Internal memory 31 Flash memory 33 External memory 35 Display part 37 Detection part 39 Correction part LX Optical axis of lens

Claims (4)

An imaging device for imaging an optical image incident through a lens for imaging operation;
A control unit that performs movement control for moving either one of the movable lens included in the lens and the imaging element relative to a lens other than the movable lens on a plane perpendicular to the optical axis of the lens; With
The imaging operation is performed a plurality of times,
In order to make the position of the subject on the imaging surface of the imaging device constant at least at the start of the exposure time of the plurality of imaging operations, exposure of the second and subsequent imaging operations among the plurality of imaging operations is performed. Based on the difference between the position at the time start time and the position at the exposure time start time of the first imaging operation, the movement control is performed so as to reduce the difference at the exposure time start time of the second and subsequent imaging operations. An imaging apparatus characterized by being performed. The imaging apparatus according to claim 1 , wherein during the exposure time of the plurality of imaging operations , the movement by the movement control is stopped and the position is fixed .   The imaging apparatus according to claim 1, wherein the control unit generates a composite image by combining the images obtained by the plurality of imaging operations.   The imaging apparatus according to claim 1, wherein the plurality of imaging operations are performed under different exposure conditions.

Images