JP2006262220A - Imaging apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an imaging apparatus for properly suppressing camera-shake. <P>SOLUTION: The imaging apparatus for converting a photographed optical image into a signal and reading the signal by the start of exposure so as to generate a photographed image includes: a proper value setting means for setting proper exposure time prior to start of the exposure; a camera-shake detection means for detecting the camera-shake caused in the imaging apparatus during the exposure and detecting it as occurrence of the camera-shake when the magnitude of the camera-shake is a prescribed value or over; a moving amount calculation means for calculating a positional change in the picked-up image deviated due to the camera-shake as a shift amount; an image extraction means for dividing the proper exposure time into a plurality of sections upon the occurrence of the camera-shake and extracting an image signal for each section on the basis of the photographed optical image stored in one section; a correction means for correcting a positional deviation caused by the camera-shake between a plurality of extracted image signals by using the shift amount; and a composite output means for synthesizing a plurality of the corrected image signals and generating the photographed image on the basis of the composite image signal. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an imaging apparatus that generates a captured image by converting a captured optical image accumulated at the start of exposure into an image signal, and more particularly to correction of camera shake.

  2. Description of the Related Art Conventionally, there are various techniques for correcting camera shake during shooting in an imaging apparatus such as a digital still camera. For example, a camera shake detection mechanism using a gyro sensor is provided, and based on this detection value, a camera shake correction technique is performed by moving an optical system such as a lens, or an image sensor that stores a light image from a subject as a charge is moved. A technique for correcting camera shake is well known. With such a technique, it is possible to perform an appropriate camera shake correction and suppress the occurrence of camera shake, but a function of moving an optical system and a sensor is required. For this reason, the number of parts as a digital still camera increases, and the cost becomes high.

  For such camera shake correction, a technique for correcting camera shake by focusing on the exposure time at the time of shooting has been proposed. For example, in Patent Document 1 below, an appropriate exposure time and a shorter allowable exposure time are set based on the result of photometry, and the amount of blur exceeds the allowable value after the allowable exposure time has elapsed since the start of exposure. A technique is disclosed in which the charge during the allowable exposure time is used as an image signal. According to such a technique, an image based on a signal before occurrence of blur exceeding an allowable value is used as an output image, so that an image with less blur can be generated.

JP 2002-116477 A

  However, since an image based on the allowable exposure time, which is a short exposure time, is obtained by this method, there is a problem that noise increases. That is, an imaging signal having a short exposure time is amplified and used, but at that time, a noise component is similarly amplified. Therefore, using only the signal before blurring causes a feeling of roughness in the output image, which is insufficient for replacing the output image with an appropriate exposure time.

  An object of the present invention is to provide an imaging apparatus that appropriately suppresses camera shake without using a special camera shake correction mechanism, in view of the problem of insufficient exposure of an image obtained when camera shake occurs. .

  The imaging apparatus of the present invention has taken the following technique in view of the above problems. That is, an imaging apparatus that converts a photographic light image into a signal at the start of exposure, reads the signal, and generates a photographic image, and includes an appropriate value setting unit that sets an appropriate exposure time prior to the start of exposure, A blur detection unit that detects a blur occurring in the imaging apparatus during the period and detects the occurrence of a camera shake when the blur is equal to or greater than a predetermined value, and a change in the position of the imaging shifted due to the occurrence of the camera shake as a movement amount The appropriate amount of exposure time is divided into a plurality of sections triggered by the occurrence of camera shake and the movement amount calculating means to calculate, and an image signal based on the photographed light image accumulated in the one section is extracted for each section. An image extracting unit; a correcting unit that corrects a positional shift caused by the camera shake between the plurality of extracted image signals using the movement amount; and the corrected plurality of image signals are combined, Together It is summarized in that with a synthesizing output means for generating the captured image based on the image signal.

  The imaging method of the present invention is an imaging method of an imaging apparatus that generates a captured image by converting a captured optical image into a signal upon start of exposure, and sets an appropriate exposure time prior to the start of exposure. Then, a blur generated in the imaging apparatus during the exposure is detected, and when the blur is equal to or greater than a predetermined value, the occurrence of a camera shake is detected, and a change in the position of the imaging shifted due to the occurrence of the camera shake is a moving amount. Calculated as follows, and the appropriate exposure time is divided into a plurality of sections triggered by the occurrence of camera shake, and an image signal based on the photographed light image accumulated in the one section is extracted for each section, and the extracted The positional shift caused by the camera shake between a plurality of image signals is corrected using the movement amount, the corrected plurality of image signals are combined, and the captured image is combined based on the combined image signals. Need to generate It is set to.

  According to the imaging apparatus and imaging method of the present invention, the appropriate exposure time is divided into sections triggered by the occurrence of camera shake during exposure, the image signal is extracted for each section, the deviation is corrected, and these are combined. That is, camera shake is corrected in software. Therefore, it is not necessary to provide a configuration for correcting hardware camera shake, and camera shake can be appropriately suppressed. In addition, since a captured image is generated by combining a plurality of image signals, it is possible to obtain an image quality substantially equivalent to a captured image that is captured without causing camera shake.

  The appropriate value setting means of the image pickup apparatus having the above configuration is an amplification factor for extracting the image signal from the electrical signal as a photographic light image accumulated within the appropriate exposure time together with the appropriate exposure time. An appropriate gain amount is set, and the image extraction means sets an interval gain amount based on the appropriate gain amount for each interval, and extracts the image signal for each interval using the interval gain amount. good.

  According to such an imaging apparatus, the section gain amount is set for each section according to the appropriate exposure time. Therefore, the image signal can be easily extracted from the electrical signal in the section.

  The image extraction unit of the imaging apparatus having the above configuration may set a value obtained by multiplying the appropriate gain amount by a reciprocal of the ratio of the exposure time of the interval to the appropriate exposure time as the interval gain amount. good.

  According to such an imaging apparatus, the section gain amount is calculated by reflecting the ratio of the exposure time of the section with respect to the appropriate exposure time to the appropriate gain amount. Therefore, an appropriate gain amount can be set for each section.

  The shake detection unit of the imaging apparatus having the above-described configuration is a sensor that detects the camera shake as the movement amount, and may detect the occurrence of the camera shake based on the movement amount. By using such a sensor, the occurrence of camera shake can be easily detected along with the amount of movement.

  The shake detection unit of the imaging apparatus having the above-described configuration may detect the occurrence of the camera shake by comparing a preset allowable value with an accumulated value of the movement amount detected by the sensor.

  According to such an imaging apparatus, the occurrence of camera shake is detected using the accumulated value. For example, if the movement amount below a certain threshold value is continuously detected during exposure, the occurrence of camera shake is detected if the accumulated value exceeds the threshold value without exceeding the threshold value. Therefore, it is possible to accurately detect the occurrence of camera shake.

  The composite output unit of the imaging apparatus having the above-described configuration may combine the image signals by taking an addition average or a weighted average of the plurality of image signals.

  According to such an imaging apparatus, the image signal extracted using the section gain amount for each section is corrected and synthesized by addition averaging and weighted averaging. Since the addition average and the weighted average of the plurality of image signals are taken, the influence of noise included in the image signal can be reduced.

  Weighting in the weighted average by the composite output unit of the imaging apparatus having the above-described configuration may be performed by multiplying the image signal of the section by the ratio of the exposure time of the section to the appropriate exposure time.

  According to such an imaging apparatus, it is possible to reflect the influence of the exposure time for each section on a captured image generated by combining a plurality of image signals.

  The weighting in the weighted average by the composite output unit of the imaging apparatus having the above-described configuration is performed based on the movement amount for each section, and is high for the image signal in the section having the small movement amount among the plurality of image signals. It may be weighted.

  According to such an imaging apparatus, it is possible to reflect the influence of the movement amount based on camera shake for each section in a captured image generated by combining a plurality of image signals.

  The present invention can also be implemented as a program for an imaging apparatus and a medium recording the program. As the recording medium, various media that can be read by the imaging apparatus such as a flexible disk, a CD-ROM, a DVD-ROM / RAM, a magneto-optical disk, and a memory card can be used.

Hereinafter, embodiments of the present invention will be described in the following order based on examples.
A. Configuration of imaging device:
B. Image stabilization concept:
C. Image stabilization processing:
D. Variation:

A. Configuration of imaging device:
FIG. 1 is a block diagram showing a schematic configuration of a digital still camera which is an image pickup apparatus of the present invention. As shown in the figure, the digital still camera 10 of this embodiment includes a lens 15 that forms a photographic light image of a subject, a diaphragm 20 that adjusts the amount of light incident through the lens 15, and blocks incident light as necessary. Shutter 25, filter 30 including a low-pass filter, an infrared cut filter, an image sensor 35 that converts incident light into an electric signal, a CDS circuit 40 that performs correlated double sampling, and an AGC that amplifies the electric signal to generate an image signal An amplifier 45, an A / D conversion circuit 50 that converts an analog image signal into a digital image signal, and image processing such as white balance, gradation reproduction, color reproduction, edge enhancement, and compression for an AD-converted image signal And an image processing unit 60 for generating image data, a memory 70 for storing image data, a CPU, a ROM, a RAM, and the like. And it is configured from such as control unit 80.

  The imaging sensor 35, the CDS circuit 40, the AGC amplifier 45, the A / D conversion circuit 50, the image processing unit 60, and the memory 70 are electrically connected in this order, and the photographic light image incident through the lens 15 or the like is the final. It is stored in the memory 70 as image data which is a digital signal.

  The imaging sensor 35 uses a CCD, and light incident through the lens 15 or the like is temporarily accumulated in a light receiving element of the CCD as a charge by photoelectric conversion. The stored charges are sequentially transferred by the action of the charge transfer electrode in the CCD and output to the CDS circuit 40 as an electric signal. A CMOS (Complementary Metal-Oxide Semiconductor) sensor may be used instead of the CCD.

  The CDS circuit 40 reduces the noise of the input electric signal by the correlated double sampling process, and outputs this to the AGC amplifier 45. The AGC amplifier 45 amplifies the input electrical signal by giving a predetermined amplification factor (gain amount) to generate an image signal. That is, the AGC amplifier 45 adjusts sensitivity. The generated image signal is output to the A / D conversion circuit 50, and this image signal is an analog signal. The A / D conversion circuit 50 converts an analog signal into a digital signal and outputs the digital signal to the image processing unit 60.

  The image processing unit 60 performs the above various image processing as necessary. In particular, as compression processing of an image signal that is a digital signal, JPEG compression processing that is an irreversible method is performed, and image data with a reduced data amount is output. The compressed image data is stored in the memory 70. In this embodiment, CompactFlash (registered trademark) is used as the memory 70, but any medium can be used as long as it can be handled by a digital still camera (nonvolatile rewritable recording medium). May be used.

  The control unit 80 is electrically connected to each processing circuit, outputs an operation command signal to each processing circuit to control the entire digital still camera 10, and operates the lens 15, the diaphragm 20, and the shutter 25. Actuator, operation unit 90 having a plurality of buttons such as a shutter button 91 and a mode button 92, a subject to be imaged, a display unit 100 including a liquid crystal display for displaying a photographed image, and photometry for measuring reflected light from the subject The unit 140 is also electrically connected.

  The shutter button 91 of the operation unit 90 has a function of starting exposure in a completely depressed state and a function of performing photometry on a predetermined subject in a half-pressed state. When the shutter button 91 is pressed, the photometry unit 140 measures the brightness of the subject.

  The photometry unit 140 sets an appropriate aperture value and shutter speed (appropriate exposure time) set in advance based on the result of the photometry. The control unit 80 receives the appropriate values from the photometry unit 140 and controls the actuators of the diaphragm 20 and the shutter 25. That is, a so-called program AE (automatic exposure) function is provided.

  Further, the digital still camera 10 of this embodiment has a zoom function. For example, when the user performs a zoom operation via the operation unit 90, the control unit 80 issues a command to the actuator of the lens 15 in accordance with the user operation. The optical operation of the lens 15 is controlled. Furthermore, by operating the mode button or the like of the operation unit 90, not only the captured image but also the captured image stored in the memory 70 can be displayed on the display unit 100.

  In addition to these various devices, the digital still camera 10 of the present embodiment includes a shake detection unit 110 that detects camera shake during shooting and a shake correction unit 120 that corrects the detected shake. The shake detection unit 110 includes a gyro sensor that is a piezoelectric vibration type angular velocity sensor, and detects camera shake when the digital still camera 10 is shooting. The blur detection unit 110 is electrically connected to the control unit 80 and outputs an angular velocity, which is a detection value of the gyro sensor, to the control unit 80 as a change amount of the position of the digital still camera 10 itself. This amount of change in position corresponds to the amount of positional deviation between the imaging before the occurrence of camera shake and the imaging after the occurrence. Hereinafter, the amount of change in position caused by the occurrence of camera shake is referred to as a shake amount or a movement amount.

  The blur correction unit 120 is electrically connected to the control unit 80, the A / D conversion circuit 50, and the image processing unit 60, receives a command from the control unit 80, and executes a blur correction process based on the blur amount. This blur correction process is a process of acquiring the image signal that is the source of the two image data before and after the occurrence of the blur, correcting the position change, and synthesizing the two image signals. The blur correction process centered on the blur correction unit 120 will be described below.

B. Image stabilization concept:
FIG. 2 is an explanatory diagram conceptually showing the blur correction of the present embodiment. As shown in the figure, it is assumed that exposure is started from the pressing operation of the shutter button 91 (this is exposure time “0”), and the exposure is continued until the appropriate exposure time Tr. During this time, charges are gradually accumulated in the image sensor 35. It is assumed that camera shake exceeding an allowable value has occurred during this exposure, and the exposure time when this shake is detected is defined as shake detection time Tv.

  An image obtained by continuing the exposure until the appropriate exposure time Tr with the occurrence of camera shake is an image in which the image of the subject is blurred as shown in the upper part of FIG. In contrast, in the present embodiment, as indicated by the solid line “charge” in the figure, the first image (image signal) is obtained from the charge accumulated in the image sensor 35 from the exposure start 0 to the shake detection time Tv. A second image (image signal) is generated from the charge generated and accumulated in the imaging sensor 35 from the blur detection time Tv to the appropriate exposure time Tr, and an output image is generated by combining the two images.

  The combination of both is performed by correcting the second image based on the shake amount (movement amount) calculated by the gyro sensor of the shake detection unit 110 and taking the addition average of this and the first image. By performing such processing, an image with reduced blur can be generated as an output image.

C. Image stabilization processing:
FIG. 3 is a flowchart of the blur correction process of the present embodiment. This process is a process of correcting the camera shake and outputting an image, and is executed by the CPU of the control unit 80 when the user performs an operation of determining a predetermined composition and pressing the shutter button 91. The processing program is stored in the control unit 80 in the ROM, and the CPU reads it and develops it on the RAM to execute the processing.

  When the process is started, the control unit 80 acquires the appropriate exposure time Tr and the appropriate gain amount Gr based on the result of photometry performed by half-pressing the shutter button 91 (step S300). The control unit 80 acquires an appropriate aperture value and an appropriate exposure time Tr from the photometric result from the photometric unit 140, and acquires an appropriate gain amount Gr corresponding to the ISO sensitivity. Specifically, an appropriate input signal amplification factor (gain amount) is set according to the difference in ISO sensitivity such as ISO100, ISO200, and ISO400 from the photometric result. The control unit 80 controls the actuator of the diaphragm 20 to set an appropriate diaphragm value. The ISO sensitivity is set in advance.

  After the half-pressing operation of the shutter button 91, when the user completely presses the shutter button 91, the control unit 80 starts exposure using this as a trigger (step S310). The control unit 80 opens the shutter 25 and captures the optical image of the subject into the imaging sensor 35. After this step, charges are accumulated in the image sensor 35 in order.

  The control unit 80 detects camera shake at the start of exposure, and determines whether or not the amount of blur detected before the appropriate exposure time Tr exceeds a predetermined value (step S320). In this embodiment, an allowable value for the blur amount is set in advance, and the control unit 80 compares the detection value obtained by the gyro sensor with the allowable value. The control unit 80 determines that blurring has been detected when the detected value exceeds the allowable value.

  If it is determined in step S320 that the detected value is within the allowable value from the start of exposure to the appropriate exposure time Tr and no blur is detected (No), the charge accumulated in the image sensor 35 (CCD) is determined. The electrical signal is read out, converted into image data and written in the memory 70 (step S380), and the series of processing is terminated. That is, when no blur is detected between the start of exposure and the appropriate exposure time Tr and shooting without camera shake is completed, the image signal from the image sensor 35 is used as it is, converted into image data, and the memory 70. To remember.

  FIG. 4 is an explanatory diagram showing the concept of converting an input signal, which is an electrical signal, into an output signal (image signal) that is the source of image data. As indicated by the broken line in the figure, the electrical signal accumulated in the CCD of the image sensor 35 during the predetermined appropriate exposure time Tr is supplied to the AGC amplifier 45 via the CDS circuit 40. The AGC amplifier 45 amplifies the input signal using an appropriate gain amount Gr with respect to the set appropriate exposure time Tr, as shown by a solid line in the figure, based on a command from the control unit 80, and uses this as an output signal. The output signal thus amplified is compressed by the image processing unit 60 to become image data, which is stored in the memory 70. The image data stored in the memory 70 is an image with almost no blur.

  On the other hand, if it is determined in step S320 that the detected value exceeds the allowable value between the start of exposure and the appropriate exposure time Tr and shake is detected (Yes), the time when shake is detected (blur detection time Tv). ), An electric signal based on the electric charge accumulated in the imaging sensor 35 (CCD) is read, converted into an image signal, and temporarily written in the RAM (step S330). Here, the data temporarily written in the RAM is an image signal after A / D conversion, and is data before compression. By this step, the first image before occurrence of blurring is acquired.

  FIG. 5 is an explanatory diagram showing the concept of converting an input signal, which is an electrical signal, into an output signal (image signal) that is the source of image data. As in FIG. 4, the input signal is amplified by a predetermined gain amount and converted into an output signal. However, since the input signal by the CCD here is a signal before reaching the appropriate exposure time Tr, a preset appropriate gain is set. An appropriate image signal cannot be obtained by amplification using the amount Gr. Therefore, a new gain amount (first gain amount Gv1) based on the appropriate gain amount Gr is set.

  The first gain amount Gv1 is set based on the following equation based on the time ratio between the appropriate exposure time Tr and the shake detection time Tv.

  Gv1 = (Tr / Tv) × Gr (1)

  In other words, the amplification factor is increased more than the appropriate gain amount Gr in order to make the first image an electric signal with less charge that does not reach the appropriate exposure time Tr. The control unit 80 calculates the first gain amount Gv1 and instructs the AGC amplifier 45 to perform amplification using this value. The amplified electrical signal undergoes A / D conversion and is stored in the RAM in the control unit 80 as an image signal.

  Subsequently, the control unit 80 reads the charge accumulated in the image sensor 35 and clears the charge in the image sensor 35, and continues the exposure from the shake detection time Tv to the appropriate exposure time Tr (step S340). . This step corresponds to the start of exposure for acquiring the second image after occurrence of blurring.

  The control unit 80 calculates the amount of movement due to camera shake based on the detection value of the gyro sensor (step S350). The detection value from the shake detection unit 110 provided with the gyro sensor in the present embodiment is information on position change. The control unit 80 acquires this and uses it as a movement amount for correcting the shift of the position of the second image.

  The control unit 80 that has acquired the movement amount due to the shake acquires the second image from the charge accumulated in the imaging sensor 35 at the end of the appropriate exposure time Tr, and corrects the positional deviation of the second image while correcting the positional deviation of the second image. The first image is synthesized, converted into image data through a predetermined process, written in the memory 70 (step S360), and the series of processes is terminated.

  The second image acquired before the two images are combined is an image signal after A / D conversion. Here, similarly to the first image, a new gain amount (second gain amount Gv2) based on the appropriate gain amount Gr is set, and the amplified image signal is A / D converted using this.

  FIG. 6 is an explanatory diagram showing the concept of converting an input signal, which is an electrical signal, into an output signal (image signal) that is the source of image data. The exposure time from the shake detection time Tv to the appropriate exposure time Tr is also shorter than the appropriate exposure time Tr, and an appropriate image signal cannot be obtained by amplification using the preset appropriate gain amount Gr. Therefore, a second gain amount Gv2 based on the appropriate gain amount Gr is set.

  The second gain amount Gv2 is set based on the following equation based on the ratio between the appropriate exposure time Tr and the time from the shake detection time Tv to the proper exposure time Tr.

  Gv2 = (Tr / (Tr−Tv)) × Gr (2)

  Also in the acquisition of the second image, since an electric signal with a small amount of electric charge is used, the amplification factor is increased more than the appropriate gain amount Gr. The control unit 80 calculates the second gain amount Gv2, and instructs the AGC amplifier 45 to perform amplification using this value. The amplified electrical signal undergoes A / D conversion and becomes a second image (image signal) with a predetermined amount of positional deviation from the first image. The control unit 80 corrects the positional deviation of the second image acquired in this way, and combines the corrected second image and the first image stored in the ROM. For the synthesis, an addition average is used, and an image in which blurring is suppressed is written in the memory 70.

  According to the above-described camera shake correction process, when a camera shake occurs, two image signals are extracted before and after the camera shake, the camera shake is corrected, and the two image signals are combined to generate one image signal. That is, by correcting the positional deviation as the blur correction, the portion where the first image and the second image overlap is originally charged to the same charge as when the appropriate exposure time Tr is reached without camera shake. Based images can be obtained. Accordingly, it is possible to acquire appropriate image data with reduced effects of camera shake. Furthermore, since the noise included in the pre-combination image is averaged using the averaging method for synthesis, the noise of the output image is reduced compared to the case where the first image and the second image are output images alone. can do.

  In addition, since the blur (shift) of the light itself reaching the photodiode corresponding to one pixel of the CCD as the image sensor 35 is not adjusted, a mechanism for moving the lens 15 and the image sensor 35 according to the blur is provided. There is no need to prepare. Therefore, it is possible to construct a digital still camera with a reduced number of parts and a relatively inexpensive blur correction function.

  Further, the first image is temporarily stored in the RAM, and finally the image data based on the one image signal after synthesis is stored in the memory 70. Therefore, it is possible to perform a soft camera shake correction process without increasing the memory capacity.

  In the present embodiment, a predetermined shake tolerance value is set as a threshold value, and the occurrence of shake is determined based on whether or not the detected value exceeds this threshold value. Further, the gain amount for obtaining the first and second images is calculated based on the appropriate exposure time Tr and the appropriate gain amount Gr based on the photometric result. Therefore, the occurrence timing of camera shake and the gain amount for obtaining each image signal can be obtained relatively easily, and the shake correction process can be executed quickly.

  In addition, the correction of the image misalignment as the camera shake correction may be performed at any timing after the electrical signal is acquired from the CCD, but from the viewpoint of the accuracy of the misalignment correction of the two images, It is desirable to execute using the image signal after A / D change. Also, the synthesis of the two images may be performed at any timing after obtaining the electrical signal that is the source of the two images.

D. Variation:
In this embodiment, the detection (occurrence) of the blur is determined by comparing the detected value with a predetermined threshold value. However, the occurrence of the blur is determined using the accumulated value of the detected value by the gyro sensor. It is also good.

  FIG. 7 is an explanatory diagram conceptually showing blur detection based on cumulative values. As shown in the figure, the blur amount at a predetermined timing, which is a detection value by the gyro sensor, is accumulated at any time. When the cumulative value of blur amount (total blur amount) exceeds a predetermined threshold as in the hatched area, the exposure time may be set as the blur detection time Tv. For example, when a certain fixed value is used as a threshold value, the blurring threshold value is not reached, but no blur correction processing is performed on an image in a shooting state in which camera shake is always generated during the exposure time. . On the other hand, according to the blur detection method based on the cumulative value, it is possible to output an image with a slight blur reduction by applying the blur correction process even in such a shooting state.

  In this embodiment, the addition average is used for combining the two images. However, a weighted average with a predetermined weight may be used for the combination. FIG. 8 is an explanatory diagram of the weighted average based on the exposure time. As shown in the figure, when a blur is detected with the accumulated value exceeding a predetermined threshold, a first output signal Ov1 that is the basis of the first image from the start of exposure (exposure time 0) to the blur detection time Tv is acquired. . The first output signal Ov1 is a signal amplified by the first gain amount Gv1.

  Subsequently, a second output signal Ov2d that is a source of the second image from the shake detection time Tv to the appropriate exposure time Tr is acquired. The second output signal Ov2d is a signal amplified by the second gain amount Gv2. The second output signal Ov2d is converted into a corrected output signal Ov2 in which the blur amount is corrected.

  These two signals are combined with weights, and this is used as an output signal. Specifically, an output signal is obtained by the following equation.

  Output signal = (Tv / Tr) × Ov1 + ((Tr−Tv) / Tr) × Ov2 (3)

  That is, the weights for the first and second output signals are changed according to the ratio of the length of the exposure time from the start of exposure to the shake detection time Tv and from the shake detection time Tv to the appropriate exposure time Tr. For example, when the ratio between the start of exposure to the shake detection time Tv and the shake detection time Tv to the appropriate exposure time Tr is 1: 2, the first output signal Ov1 corresponding to the first image. Is added as 1/3 of the whole, and the second output signal Ov2 corresponding to the second image is added as 2/3 of the whole. That is, the influence on the entire first output signal Ov1 having a short exposure time is set low. By doing so, it is possible to output an image in which the influence of noise is further reduced.

  Such weighting may be set based on the blur amount. FIG. 9 is an explanatory diagram of the weighted average based on the maximum blur amount. Here, the blur detection time Tv is determined based on the cumulative blur value, and the peak value of the blur amount (hereinafter referred to as the maximum value) is detected.

  The first output signal Ov1 is acquired between the start of exposure and the blur detection time Tv, and the first maximum blur amount V1, which is the maximum blur amount during that time, is detected. Similarly, the second output signal Ov2 is acquired from the shake detection time Tv to the appropriate exposure time Tr, and the second maximum shake amount V2 that is the maximum value of the shake amount during that time is detected. Based on the maximum blur amounts V1 and V2 detected in this way, a weighted average according to the following equation is executed.

  Output signal = (V2 / (V1 + V2)) × Ov1 + (V1 / (V1 + V2)) × Ov2 (4)

  As shown in the figure, when V1 is larger than V2, for example, V1: V2 is 6: 1, the first output signal Ov1 with a large blur amount is 1/7 of the whole and the second output signal with a small blur amount. Ov2 is added as 6/7 as a whole. That is, it is determined that a signal (image) with a large amount of blur is likely to contain data inappropriate for outputting a high-quality image, and the amount of blur with respect to the entire output signal (image) is large. Reduce the influence of the signal (image). Therefore, a high-quality image can be output. Instead of the maximum value that is the peak value of the blur amount, an average value of the blur amount from the start of exposure to the blur detection time Tv and an average value of the blur amount from the blur detection time Tv to the appropriate exposure time Tr are obtained. These average values may be used for weighting.

  In this embodiment, once blurring is detected during the appropriate exposure time Tr, the exposure time is set as the blur detection time Tv, and an image is acquired before and after this time. The blur detection time Tv may be set to obtain one output image from a plurality of images.

  FIG. 10 is an explanatory diagram conceptually showing a technique for obtaining one output image from a plurality of images. As shown in the figure, it is assumed that the occurrence of blurring is detected twice before the appropriate exposure time Tr, and the detection times are referred to as blur detection times Tv1 and Tv2. Here, the occurrence of blur is detected using the cumulative value of the blur amount.

  For each of the three sections, the first section of exposure time 0 to shake detection time Tv1, the second section of shake detection time Tv1 to shake detection time Tv2, the third section of shake detection time Tv2 to the third section of appropriate exposure time Tr, An electric signal from the CCD is acquired, and a gain amount corresponding to the exposure time is calculated for each section. For the first interval, a gain amount Gv1 is given to the input signal based on the acquired electrical signal from the CCD and amplified to obtain an output signal Ov1. Similarly, an output signal Ov2 is obtained by giving a gain amount Gv2 to the second interval, and an output signal Ov3 is obtained by giving a gain amount Gv3 to the third interval. Note that the output signals Ov2 and Ov3 are data obtained by correcting the positional deviation by using the shake detection value by the gyro sensor. The three output signals (image data sources) thus obtained are each weighted and averaged according to the maximum blur amount to obtain a final output signal. That is, three images are acquired, blurring is corrected mutually, and these are synthesized by weighted average. By doing so, it is possible to generate a high-quality image in which image shift due to camera shake is further reduced.

  In this embodiment, the gyro sensor has been treated as having no directionality (a direction in which it can be easily detected). For example, when the gyro sensor has a direction, the digital still camera 10 is operated by pressing the shutter. It is desirable to arrange the gyro sensor so that the moving direction is sensitively detected. This is because there is a high possibility that camera shake will occur when the shutter is pressed. By doing so, camera shake detection can be performed with high accuracy, and shake correction processing can be appropriately executed.

  In this embodiment, the detection value from the gyro sensor is used to detect the blur amount to be corrected before the image is synthesized. However, the positional deviation between the images to be synthesized is performed by software from image processing. It is also good. For example, along with detection of camera shake of a predetermined value or more by a gyro sensor, the shutter is closed, the electric charge is read from the CCD, and converted into first image data before compression by a predetermined process. Subsequently, the shutter is opened to start exposure, and the charge up to the appropriate exposure time is read. The read charge is converted into second image data before compression by a predetermined process. With respect to the two pieces of image data acquired in this way, positional deviation is corrected using various matching methods such as dynamic pattern matching, and the corrected image data is synthesized. By using such image processing, for example, it is possible to correct a positional shift in units of pixels, and it is possible to perform accurate correction.

  The embodiments of the present invention have been described above. However, the present invention is not limited to these embodiments, and can of course be implemented in various forms without departing from the spirit of the present invention. is there. In this embodiment, a digital still camera as an image pickup apparatus has been described as an example, but a digital video camera may be used as a matter of course. By applying the processing of the present invention to a still image photographed with a digital video camera, it is possible to suppress output of a camera shake image. In this embodiment, the hardware-type camera shake correction mechanism is not provided. Of course, a hardware-type camera shake correction mechanism may be provided. In this case, the camera shake correction can be effectively performed by selectively performing the processing of the present invention according to the detection result of the camera shake.

1 is a block diagram illustrating a schematic configuration of a digital still camera that is an imaging apparatus of the present invention. FIG. It is explanatory drawing which showed notion the blurring correction of a present Example notionally. It is a flowchart of the blur correction process of a present Example. It is explanatory drawing which showed the concept which converts the input signal which is an electrical signal into the output signal used as the origin of image data. It is explanatory drawing which showed the concept which converts the input signal which is an electrical signal into the output signal used as the origin of image data. It is explanatory drawing which showed the concept which converts the input signal which is an electrical signal into the output signal used as the origin of image data. It is explanatory drawing which showed the blur detection by a cumulative value notionally. It is explanatory drawing of the weighted average based on exposure time. It is explanatory drawing of the weighted average based on the largest blur amount. It is explanatory drawing which showed notionally the method of obtaining one output image from several images.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 ... Digital still camera 15 ... Lens 20 ... Aperture 25 ... Shutter 30 ... Filter 35 ... Imaging sensor 40 ... CDS circuit 45 ... AGC amplifier 50 ... A / D conversion circuit 60 ... Image processing unit 70 ... Memory 80 ... Control unit 90 ... Operation unit 91 ... Shutter button 92 ... Mode button 100 ... Display unit 110 ... Shake detection unit 120 ... Shake correction unit 140 ... Metering unit Gr ... Proper gain amount Tr ... Proper exposure time Tv ... Shake detection time

Claims (11)

An imaging device that converts a photographic light image into a signal at the start of exposure and reads it to generate a photographic image,
Prior to the start of the exposure, appropriate value setting means for setting an appropriate exposure time,
A blur detection unit that detects a blur generated in the imaging apparatus during the exposure, and detects the occurrence of a camera shake when the blur is equal to or greater than a predetermined value;
A movement amount calculating means for calculating, as a movement amount, a change in the position of an image taken by the occurrence of camera shake;
An image extraction unit that divides the appropriate exposure time into a plurality of sections triggered by the occurrence of camera shake, and extracts an image signal based on the captured light image accumulated in the one section for each section;
Correction means for correcting a shift in position caused by the camera shake between the extracted plurality of image signals using the movement amount;
An image pickup apparatus comprising: a combined output unit configured to combine the plurality of corrected image signals and generate the captured image based on the combined image signals. The imaging apparatus according to claim 1,
The appropriate value setting means sets an appropriate gain amount that is an amplification factor for extracting the image signal from an electrical signal as a photographic light image accumulated within the appropriate exposure time together with the appropriate exposure time,
The image extracting unit sets a section gain amount based on the appropriate gain amount for each section, and extracts the image signal for each section using the section gain amount. The imaging apparatus according to claim 2,
The image extracting device sets the value obtained by multiplying the appropriate gain amount by a reciprocal of the ratio of the exposure time of the interval to the appropriate exposure time as the interval gain amount. The imaging apparatus according to any one of claims 1 to 3,
The camera shake detecting means is a sensor that detects the camera shake as the movement amount, and detects an occurrence of the camera shake based on the movement amount. The imaging apparatus according to claim 4,
The camera shake detecting unit detects the occurrence of camera shake by comparing a preset allowable value with a cumulative value of the movement amount detected by the sensor. The imaging apparatus according to any one of claims 1 to 5,
An image pickup apparatus that combines the image signals by taking the addition average or the weighted average of the plurality of image signals. The imaging apparatus according to claim 6,
The weighting average weighting is performed by multiplying the image signal of the section by the ratio of the exposure time of the section to the appropriate exposure time. The imaging apparatus according to claim 6,
The weighting average weighting is performed based on the movement amount for each section, and an image pickup apparatus that applies a high weight to an image signal of the section having a small movement amount among the plurality of image signals. An imaging method of an imaging apparatus that converts a photographic light image into a signal at the start of exposure and reads it to generate a photographic image,
Prior to the start of the exposure, set an appropriate exposure time,
Detecting a blur that occurs in the imaging apparatus during the exposure, and detecting the occurrence of a camera shake when the blur is equal to or greater than a predetermined value;
Calculate the position change of the imaging shifted due to the occurrence of camera shake as a movement amount,
The appropriate exposure time is divided into a plurality of sections triggered by the occurrence of camera shake, and an image signal based on the photographed light image accumulated in the one section is extracted for each section.
A positional shift caused by the camera shake between the extracted plurality of image signals is corrected using the movement amount,
An imaging method for combining the plurality of corrected image signals and generating the captured image based on the combined image signals. A program for converting an imaged light image into a signal at the start of exposure, reading the image, and causing the image capturing apparatus to generate a captured image,
A function for setting an appropriate exposure time prior to the start of the exposure;
A function of detecting a blur generated in the imaging apparatus during the exposure, and detecting the occurrence of a camera shake when the blur is equal to or greater than a predetermined value;
A function of calculating a change in the position of an image taken by the occurrence of camera shake as a movement amount;
A function of dividing the appropriate exposure time into a plurality of sections triggered by the occurrence of camera shake, and extracting an image signal based on the captured optical image accumulated in the one section for each section;
A function of correcting a shift in position caused by the camera shake between the plurality of extracted image signals using the movement amount;
A program causing the imaging apparatus to execute a function of combining the plurality of corrected image signals and generating the captured image based on the combined image signals.   A recording medium on which the program according to claim 10 is recorded so as to be readable by the imaging apparatus.

Images