JP2003032540A - Imaging apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an imaging apparatus that can obtain an image without any blurring by the absolute minimum arithmetic quantity with a simple configuration. SOLUTION: An electronic camera is provided with a CCD imaging device 12 that applies photoelectric conversion to an object image formed by an imaging lens to convert the image into an electric signal, a shake amount detection circuit 62 that detects a shake amount of a device main body from the start of exposure, an exposure control circuit 63 that repeats an exposure operation of reading an image signal from the imaging device 12 when a shift from the start of exposure by the shake detection circuit 62 reaches a prescribed amount until the exposure time from the start of initial exposure reaches a prescribed time, a correction circuit 66 that corrects mutual position deviations of a plurality of read image signals, an adder circuit 67 that sums image signals of pixels corresponding to the image signal corrected by the correction circuit 66, and a recording medium 32 that records the image summed by the adder circuit 67.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image pickup apparatus such as an electronic camera (digital still camera), and more particularly to an image pickup apparatus having a blur correction function.

[0002]

2. Description of the Related Art It is widely known that, when a still image is picked up by an image pickup device such as an electronic camera, the image is shaken due to camera shake or movement of a subject when a long time image pickup is performed. The “blurring” of this image is sometimes called “blurring” because it is a one-dimensional (including curved) image blur, but in this specification, it is expressed as “blurring”. Image blurring may be positively applied to radiography such as panning, but it is usually regarded as deterioration of image quality, and it is essential to prevent it. One of the typical methods is to fix the camera stably using a tripod and the other is to use short-time exposure (high-speed shutter), but both are applied if the situation does not allow it. No, it is not applicable to handheld low-light photography.

As a technique which can be used even in such a case, there is, for example, one described in Japanese Patent Laid-Open No. 10-336510. That is, an optical system that detects the relative movement of the subject image with respect to the image pickup surface in advance and shifts the formed image at the time of image pickup based on the information (the image pickup block if grasped as "optical system + image pickup element"). It is driven to make the subject image relatively stationary with respect to the imaging surface. In addition, as a known conventional technique, this publication also describes "camera shake correction control means" especially for moving images. As the correction means, an optical type (an optical system for shifting the formed image as described above is used). There are two types, one that is used) and an electronic type (one that moves an image frame extracted from the entire image-capable area by driving a memory or an image sensor). Further, as the “hand shake detection method”, there are mentioned a motion vector detection for detecting a moving amount and a direction of an object by image processing, and an angular velocity detection for directly detecting a shake of a camera body by an angular velocity sensor.

[0004]

However, without waiting for the indication in the above publication, the conventional image stabilization technology for moving images eliminates the image blur in one image taken for a long time in still image shooting. You cannot do it.
Regarding the technique disclosed in the above publication, the following 2
There are two problems. That is, (1) a special optical system (imaging block) that shifts the formed image is required, resulting in an increase in the shape, power consumption, and cost of the device, and further a decrease in imaging performance due to the adoption of the special optical system. Occurs. (2) Since correction is performed based on prior information prior to imaging,
If there is a change in the movement of the image, the correction will be erroneous, and the image quality will deteriorate.

The Japanese Patent Application No. 2000-21 filed by the present applicant
No. 3894 proposes a method of performing exposure by dividing the exposure time into a plurality of consecutive exposure times so that the exposure time becomes equal to or shorter than a predetermined value when an exposure time longer than the predetermined value is set. Then, assuming that a general photographer holds the camera by hand as the divided exposure time, the image blur of the recorded image caused by the camera shake caused by the photographing operation is a limit value within the allowable limit. Is set to. However, since the relationship between the amount of camera shake and time is not uniform, it is necessary to set the value of the exposure time to be divided to a considerably short value in order to provide an image without camera shake to all photographers. It was In this case, there are problems that the capacity of the memory that stores the image signal that is repeatedly exposed increases, and that it takes a long time to perform motion compensation calculations.

The present invention has been made in consideration of the above circumstances, and an object of the present invention is to provide an image pickup apparatus capable of obtaining an image without blur with a minimum necessary calculation amount by a simple configuration. To provide.

[0007]

(Structure) In order to solve the above problems, the present invention adopts the following structure.

That is, according to the present invention, in an image pickup device such as an electronic camera, an image pickup device for photoelectrically converting a subject image formed by a photographing lens to convert it into an electric signal, and detecting a shake amount of the device body from the start of exposure. And an exposure operation for reading an image signal from the image sensor when the blur amount from the exposure start by the blur amount detection unit reaches a predetermined amount, and an exposure time from the exposure start of the initial exposure operation. Of exposure control means for repeatedly performing until a predetermined time is reached, correction means for correcting mutual positional deviation of the plurality of read image signals, and corresponding pixels of the image signals corrected by the correction means. It is characterized in that it has an adding means for adding the image signals and a recording medium for recording the image added by the adding means.

Further, the present invention is necessary in an image pickup apparatus such as an electronic camera for obtaining an appropriate exposure of the image pickup element for photoelectrically converting a subject image formed by a photographing lens into an electric signal. Exposure time calculation means for calculating the exposure time, and when it is determined that the exposure time calculated by the exposure time calculation means is longer than a predetermined time,
A diaphragm control means for increasing the size of the diaphragm aperture of the photographing lens, a shake amount detecting means for detecting a shake amount of the apparatus main body from the start of exposure, and a shake amount from the start of exposure by the shake amount detecting means. Exposure control means for repeatedly performing an exposure operation of reading an image signal from the image sensor when a predetermined amount is reached, until an exposure time from the start of exposure of the initial exposure operation reaches a predetermined time; A correction unit that corrects the mutual positional deviation of the image signals, an addition unit that adds the image signals of the corresponding pixels of the image signals corrected by the correction unit, and a recording that records the images added by the addition unit. And a medium.

The following are preferred embodiments of the present invention. (1) The correction means should be based on the output signal of the blur amount detection means. (2) The correction means is to correct the positional deviation between the images by image processing. (3) The shake amount detecting means detects the shake amount of the apparatus body based on the output of the angle sensor provided in the apparatus body.

(Operation) According to the present invention, the blur amount detecting means for detecting the blur amount of the apparatus main body from the start of the exposure is provided, and the blur amount from the start of the exposure by the blur amount detecting means reaches the predetermined amount. The operation of detecting that the image signal is read out from the image sensor is repeated until the exposure time from the start of the initial exposure reaches a predetermined time, and the image signals read out repeatedly are added to obtain one image signal. The image signal is obtained. Here, each of the divided image signals has a positional deviation due to the blurring, but this can be corrected by the correcting means, and therefore the blurring can be kept within the allowable range in the image signals obtained by addition. it can.

That is, instead of obtaining one image signal with a large blur, a plurality of image signals with a little blur are obtained, and these are corrected and added to obtain one image signal with little blur. . In this case, there is no need for a special optical system, and it is possible to cope with the case where the movement of the image changes. Moreover, since the value of the exposure time to be divided based on the blur amount by the blur amount detecting means is set,
The value of the exposure time can be set to a relatively long value, and the increase in memory capacity and motion compensation calculation can be minimized. Therefore, it is possible to obtain a blur-free image with a minimum amount of calculation with a simple configuration.

Further, the exposure time becomes long for a low-luminance subject. In this case, if one exposure is completed at the time when the blur amount reaches the allowable limit, the amount of charge accumulated in the image sensor is reduced. Therefore, the image signals read later by a plurality of exposure operations are added and S is added. Although the / N can be improved, the S / N as a whole is lower than expected because of errors associated with the reading and the addition processing. Therefore, if it is predicted that the exposure time will be longer than the predetermined value, the size of the aperture of the diaphragm will be increased, and the charge accumulation amount of the image sensor during the exposure time when the shake amount reaches the permissible limit will be increased as much as possible. In addition, it is possible to obtain an image with good S / N.

[0014]

DETAILED DESCRIPTION OF THE INVENTION The details of the present invention will be described below with reference to the illustrated embodiments.

FIG. 1 is a block diagram showing a circuit configuration of an electronic camera according to an embodiment of the present invention.

Reference numeral 12 in the drawing denotes C for picking up a subject image.
CD image sensor, 14 is a correlated double sampling circuit (CDS), 16 is a gain control amplifier (AM
P), 18 is an A / D converter, 20 is a timing generator (TG), 22 is a signal generator (SG),
24 is a CPU, 26 is an information processing unit, 28 is a DRAM, 3
0 is a compression / expansion circuit, 32 is a recording medium, 34 is a liquid crystal display unit, 36 is an interface unit, 38 is a lens drive system,
40 is a lens, 42 is a diaphragm drive system, 44 is a diaphragm, 50,
Reference numeral 51 is an angle sensor, and 52 and 53 are A / D converters.

The CCD image sensor 12 is, for example, 10
This is an interline CCD image sensor having more than 0,000 pixels, and has a Bayer array color filter suitable for reading all pixels by line-sequential scanning. Then, it is driven according to the transfer pulse supplied from the TG 20. The CDS 14 is driven according to the sample hold pulse supplied from the TG 20. TG20 is SG
Drives in synchronization with each other in accordance with the synchronization signal generated in 22.

The information processing section 26 processes the pixel signals supplied from the A / D converter 18 to form an image. DRA
M28 temporarily stores the image data supplied from the information processing unit 26, the compression / expansion circuit 30 compresses the image data stored in the DRAM 28, and the recording medium 32 is compressed supplied from the compression / expansion circuit 30. The recorded image data is recorded. Further, the compression / expansion circuit 30 expands the compressed image data recorded in the recording medium 32, and the DRAM 28
Temporarily records the expanded image data supplied from the compression / expansion circuit 30.

The interface unit 36 is a terminal that enables data exchange with an external device such as a monitor or a personal computer, and enables the image data supplied from the information processing unit 26 or the DRAM 28 to be output to the external device. Alternatively, in some cases, the image data can be taken into the device from an external device.

The liquid crystal display unit 34 displays the image data supplied from the information processing unit 26 or the expanded image data supplied from the DRAM 28. CPU24 is TG
20, SG22, lens drive system 38, and diaphragm drive system 4
2 is controlled. Specifically, the drive mode of the CCD image sensor 12 is switched according to a command from the trigger 46 that instructs the capture of a still image, and the DRA
The lens 40 based on the image data supplied from M28
Is performed to control the autofocus control for driving, the control for changing the aperture of the diaphragm 44, and the control of the exposure amount of the CCD image sensor 12.

The angle sensor 50 is for detecting the angular velocity when the camera is rotated about the X-axis direction which is the left-right direction when the subject is viewed from the camera. The analog signal representing the angular velocity detected by the angle sensor 50 is converted into a digital signal by the A / D converter 52 at predetermined time intervals, and the converted digital signal is time-integrated by the CPU 24. The time-integrated digital signal corresponds to the amount of rotation about the X axis of the camera body. Further, the rotation direction is determined by the positive or negative of the analog output signal of the angular velocity sensor 50.

On the other hand, the angular velocity sensor 51 is for detecting the angular velocity when the camera is rotated about the Y-axis direction when the vertical direction of the camera is the Y-axis direction. The analog signal representing the angular velocity detected by the angular velocity sensor 51 is converted into a digital signal by the A / D converter 53 at predetermined time intervals, and the converted digital signal is time-integrated by the CPU 24. The time-integrated digital signal corresponds to the amount of rotation about the Y axis of the camera body. Further, the rotation direction is determined by the positive or negative of the analog output signal of the angular velocity sensor 51.

FIG. 2 is a functional block diagram showing a circuit configuration relating to a function for performing blur correction in this embodiment.

The subject image formed by the taking lens is photoelectrically converted by the image pickup device 12. The amount of blur of the camera body from the start of exposure is determined by the angle sensors 50, 5
1 and CPU 24 etc. shake amount detection circuit 62
Detected by. Then, based on the output of the blur amount detection circuit 62, the exposure amount control circuit 63 performs an exposure operation of reading an image signal from the image sensor 12 when the blur amount from the start of exposure reaches a predetermined amount, and the exposure operation of the initial exposure operation. The exposure is repeated from the start until a predetermined time is reached. The exposure amount control circuit 63 uses the TG2
0, SG 22, CPU 24 and the like.

The image signals sequentially read from the image pickup device 12 under the control of the exposure amount control circuit 63 are the CDS1.
4, processed by the signal processing circuit 64 including the AMP 16, the A / D 18, the information processing unit 26, and the like, and then stored in the memory 65 including a plurality of frame memories. In addition,
The memory 65 may be the DRAM 28, or may be specially provided for temporary storage of a plurality of images.

On the other hand, based on the output signal of the blur amount detection circuit 32, the correction circuit 66 corrects the mutual positional deviation of the image signals stored in the memory 65. In particular,
The address is shifted according to the shift amount for each frame, or the amount to shift the address is set for each frame. Then, the image signals of the pixels corresponding to the image signal corrected by the correction circuit 66 are added by the addition circuit 67 to obtain one image signal. And
The added image signal is recorded on the recording medium 32.

3 and 4 are flowcharts for explaining the flow of the entire operation in this embodiment.

The process starts at S100. This operation is started by turning on the power of the camera or operating an operation start switch (not shown). S10
In step 1, it is determined whether or not the first release switch (SW1 in FIG. 1) is closed by the release operation. SW1
If is not closed, the process branches to J100 and the same operation is repeated. Actually, during the operations of J100 and S101, an operation such as detection of a state of display or other key input (not shown) is performed, but since it is not directly related to the present invention, SW1 in FIG. The detection of the above condition shall be repeated.

If SW1 is closed in S101,
Next, in S102, photometry is performed. This photometry is CCD
By monitoring the level of the image signal repeatedly output from the image pickup device 12, the aperture and shutter speed for obtaining proper exposure are calculated. That is, C
The image signal read from the CD image pickup device 12 is converted into a digital value by the A / D conversion circuit 18, and is temporarily stored in the DRAM 28. Of the stored image signals, the image signal of a predetermined area near the center of the entire image is the CPU 2
4, the arithmetic mean value of the level is obtained, and the shutter speed and aperture value necessary for obtaining proper exposure are calculated based on the value.

Next, in S103, when it is determined that the shutter speed required to obtain the proper exposure determined as described above is longer than the predetermined time t _L ,
In S104, the aperture of the diaphragm 44 is enlarged to increase the intensity of light incident on the CCD image pickup device 12. this is,
As will be described later, when a plurality of exposures are performed at a predetermined shake limit exposure time t _B where hand shake does not occur, if the brightness of the subject is dark, a sufficient image signal cannot be obtained at the shake limit exposure time t _B and S / N is worse, but this is because the amount of light is increased to prevent this. Then, in S105, the exposure time t _E is calculated again based on the photometric value in S102 and the aperture value reset in S104. When the shutter speed is higher than t _L in S103, the process branches to J101, and the shutter speed itself obtained in S102 becomes the exposure time t _E.

Next, in S106, it is detected whether or not the second release switch SW2 is closed. If SW2 is closed, the initial value 0 is input to the memory n that stores the number of exposures performed in the blurring limit exposure time t _B in which blurring occurs in S107. Then, the exposure is started in S108. It should be noted that the exposure is started by discharging the charges in the charge storage section of the CCD image pickup device 12. This is a well-known technique and will not be described in detail here.

Next, the angular velocity sensors 50, 5 for detecting blurring
Based on the detection output of 1, the X-axis of the camera from the start of exposure,
From the amount of rotation around the Y axis, the amount of image blur in the horizontal and vertical directions of the image is calculated. This blurring amount is sequentially obtained at constant time intervals, but the blurring amount obtained as a result of the computation is slightly behind the actual one due to the delay of the computation, the response delay of the angular velocity sensors 50 and 51, and the like. Therefore, the response delay is compensated by performing a prediction calculation known in Japanese Patent Laid-Open No. 5-24012. Then, S109
At, the shake limit exposure time t _{B at} which the shake amount from the start of exposure reaches the allowable limit shake amount is obtained.

Next, in S110, in S109 described above.
Determined blurring limit exposure time t_BAnd obtained in S107 above
product nt with n_BAnd the exposure time t obtained in S105_E
And the difference (hereinafter, this difference is referred to as "non-exposure time").
Therefore, the unexposed time is the blur limit exposure time t_BLonger
Determine whether or not. Unexposed time blurring limit exposure time t
_BIf the time is longer than the
Exposure time from 108 blurring limit exposure time t_BHas reached
If the exposure is completed, the next
Then, in S112, the image signal is read out, and in S113, this reading is performed.
The output image signal is temporarily stored in the DRAM 28, and S
At 114, 1 is added to n and the process branches to J102.

After performing the exposure for the blurring limit exposure time t _B n times, when the unexposed time is a short time of the blurring limit exposure time t _B or less in S110, the exposure is completed in S115. Determine if When it is determined that the exposure is completed, the image signal is read in S116, and the read image signal is temporarily stored in the DRAM 28 in S117. Next, in step S118, the image signal that has been exposed by the camera shake limit exposure time n times and the image signal that was finally exposed and read with an exposure time equal to or less than the camera shake limit exposure time are aligned. This alignment is performed because the images exposed and read at the respective blurring limit exposure times are not blurred, but each image may be relatively blurred beyond the blurring limit, which will be described later. This is because it is necessary to correct the relative displacement between the images before adding the image signals. This alignment can be performed by known image processing, or can be performed by using the amount of image shift obtained from the output of the angular velocity sensor.

Next, in step S119, the plurality of aligned image signals are added, and in step S120, the added image signals are image-compressed and recorded on the recording medium 32, and the process is ended.

As described above, according to the present embodiment, the exposure time is divided according to the blur amount of the camera body, the positional deviation is corrected for the plurality of image signals obtained by the division, and the positional deviation is corrected. By adding each image signal, one image signal with less blur can be obtained. In this case, there is no need for a special optical system, and it is possible to cope with the case where the movement of the image changes. Moreover, since the value of the exposure time to be divided is determined based on the blur amount by the blur amount detection circuit 62, the value of the exposure time can be set to a relatively long value, and the memory capacity is increased and the motion compensation calculation is performed. Can be minimized. Further, in a low-brightness subject, by increasing the size of the aperture opening and increasing the charge storage amount of the image sensor during the exposure time when the blur amount reaches the allowable limit, there is no blur and the S / N is good. It is possible to obtain a clear image.

The present invention is not limited to the above-described embodiment, and various modifications can be carried out without departing from the scope of the invention.

[0038]

As described above in detail, according to the present invention, the charge of the image pickup device is repeatedly read until the exposure amount becomes appropriate at the time when the shake amount of the apparatus main body from the start of exposure exceeds the allowable limit, Since the positional deviation of the read image is corrected and then added, it is possible to obtain an image without blur with a minimum amount of calculation with a simple configuration.

[Brief description of drawings]

FIG. 1 is a block diagram showing a circuit configuration of an electronic camera according to an embodiment of the present invention.

FIG. 2 is a functional block diagram showing a circuit configuration for performing blur correction in the same embodiment.

FIG. 3 is a flowchart for explaining the flow of the overall operation in the same embodiment.

FIG. 4 is a flowchart for explaining the flow of the overall operation in the same embodiment.

[Explanation of symbols]

12 ... CCD image sensor 14 ... Correlated double sampling circuit (CDS) 16 ... Gain control amplifier (AMP) 18, 52, 53 ... A / D converter 20 ... Timing generator (TG) 22 ... Signal generator (SG) 24 ... CPU 26 ... Information processing unit 28 ... DRAM 30 ... Compression / decompression circuit 32 ... Recording medium 34 ... Liquid crystal display section 36 ... Interface part 38 ... Lens drive system 40 ... Lens 42 ... Aperture drive system 44 ... Aperture 50, 51 ... Angle sensor 62 ... Blurring amount detection circuit 63 ... Exposure amount control circuit 64 ... Signal processing circuit 65 ... Memory 66 ... Correction circuit 67 ... Adder circuit

─────────────────────────────────────────────────── ─── Continuation of front page (51) Int.Cl. ⁷ Identification code FI theme code (reference) // H04N 101: 00 H04N 101: 00

Claims (5)

[Claims] 1. An image pickup device for photoelectrically converting a subject image formed by a photographing lens into an electric signal, a blur amount detecting means for detecting a blur amount of an apparatus main body from the start of exposure, and the blur amount. Exposure control for repeatedly performing an exposure operation of reading an image signal from the image sensor when the amount of blurring from the start of exposure by the detection unit reaches a predetermined amount until the exposure time from the start of exposure of the initial exposure operation reaches a predetermined time Means, a correction means for correcting the positional deviation between the read-out plurality of image signals, an addition means for adding the image signals of corresponding pixels of the image signals corrected by the correction means, and the addition And a recording medium for recording the image added by the means. 2. An image pickup device for photoelectrically converting a subject image formed by a photographing lens into an electric signal, and an exposure time calculation means for calculating an exposure time required to obtain proper exposure of the image pickup device. When it is determined that the exposure time calculated by the exposure time calculation means is longer than a predetermined time, an aperture control means for increasing the size of the aperture opening of the photographing lens, and a shake of the apparatus main body from the start of exposure. A blur amount detecting unit for detecting the amount, and an exposure operation of reading an image signal from the image sensor when the blur amount from the start of exposure by the blur amount detecting unit reaches a predetermined amount. Exposure control means for repeatedly performing the exposure time from a predetermined time to Adding means for adding the image signals between the pixels corresponding to the image signals Tadashisa imaging apparatus characterized by having a recording medium for recording an image are added by the adding means. 3. The image pickup apparatus according to claim 1, wherein the correction means performs correction based on an output signal of the blur amount detection means. 4. The image pickup apparatus according to claim 1, wherein the correction means obtains and corrects a positional deviation between images by image processing. 5. The blur amount detecting means detects the blur amount of the apparatus main body based on an output of an angle sensor provided in the apparatus main body. Imaging device.