JP2008092059A - Imaging apparatus and method, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To obtain an excellent composite image by suppressing a noise due to image composition. <P>SOLUTION: A digital camera 100 includes a signal processing section 7 for detecting camera shake in image pickup of a subject by a CMOS sensor 2; and a CMOS controller 3 for stopping exposure of the CMOS sensor every time the camera shake is detected and resuming the exposure of the CMOS sensor every time camera shake is not detected. The signal processing section executes processing of generating a subject image on the basis of an image signal accumulated in the CMOS sensor every time the exposure of the CMOS sensor is stopped, and processing for compositing a plurality of generated subject images. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an imaging apparatus, an imaging method, and a program that synthesize a plurality of images.

In recent years, an imaging apparatus such as a digital camera can acquire a clear image by synthesizing images acquired by a plurality of times of imaging.
In addition, in order to prevent the images from shifting during image composition, the amount of camera shake is determined for images acquired by continuous shooting at high speed, and the read minute time image data is used as the main image according to the amount of camera shake detected. A technique is known that obtains a captured image free from camera shake by combining the remaining images without using them as data (see, for example, Patent Document 1).
JP 2005-277812 A

  However, in the case of the above-mentioned patent document 1 and the like, there is a problem that when many images are synthesized, noise corresponding to the number of image synthesis occurs.

  Therefore, an object of the present invention is to provide an imaging apparatus, an imaging method, and a program that can acquire a good synthesized image while suppressing noise related to image synthesis.

The imaging apparatus according to the first aspect of the invention (for example, the digital camera 100 of FIG. 1)
An imaging unit that has an imaging element (for example, the CMOS image sensor 2 of FIG. 1 and the like) and that images the subject by exposing the imaging element;
Camera shake detection means (for example, the signal processing unit 7 in FIG. 1) for detecting camera shake when the subject is imaged by the imaging means;
Each time the camera shake is detected by the camera shake detector, the exposure of the image sensor is stopped, and each time the camera shake is not detected by the camera shake detector, the exposure control means during camera shake is resumed. (For example, the CMOS control unit 3 in FIG. 1);
Image generation means (for example, the signal processing unit 7 in FIG. 1) that generates a subject image based on the image signal accumulated in the image sensor every time the exposure of the image sensor is stopped by the camera shake exposure control means. When,
It is characterized by comprising image synthesizing means (for example, the signal processing unit 7 in FIG. 1) for synthesizing the plurality of subject images generated by the image generating means.

The invention according to claim 2 is the imaging apparatus according to claim 1,
Reference imaging time storage means (for example, the storage unit 15 in FIG. 1) that stores a reference imaging time that is a reference of imaging time by the imaging means;
An exposure time measuring means (for example, the timing unit 14 in FIG. 1) for measuring an exposure time during which the image sensor is exposed when the subject is imaged by the imaging means;
Exposure time determination means for determining whether or not the exposure time measured by the exposure time measurement means has reached the reference imaging time stored in the reference imaging time storage means (for example, the CPU 16 in FIG. 1). When,
Exposure termination means (for example, the CMOS control unit 3 in FIG. 1) that terminates exposure of the image sensor when the exposure time determination means determines that the exposure time has reached the reference imaging time. It is characterized by that.

The invention according to claim 3 is the imaging apparatus according to claim 1,
Reference imaging time storage means (for example, the storage unit 15 in FIG. 1) that stores a reference imaging time that is a reference of imaging time by the imaging means;
An exposure time measuring means (for example, the timing unit 14 in FIG. 1) for measuring an exposure time during which the image sensor is exposed when the subject is imaged by the imaging means;
An exposure time for determining whether or not the exposure time measured by the exposure time measuring means has reached the reference imaging time stored in the reference imaging time storage means after the imaging of the subject by the imaging means is completed. Determination means (for example, the CPU 16 in FIG. 1);
When the exposure time determining means determines that the exposure time has not reached the reference imaging time, a brightness adjusting means for adjusting the brightness of the synthesized image synthesized by the image synthesizing means (for example, the signal in FIG. And a processing unit 7).

According to a fourth aspect of the present invention, in the imaging apparatus according to the first aspect,
Reference imaging time storage means (for example, the storage unit 15 in FIG. 1) that stores a reference imaging time that is a reference of imaging time by the imaging means;
When the subject is imaged by the imaging means, an exposure stop time measuring means for measuring an exposure stop time during which the exposure of the image sensor is stopped by the camera shake exposure control means (for example, the timing unit 14 in FIG. 1). )When,
An imaging time extension means (for example, an extension of the imaging time by the imaging means) by adding the exposure stop time measured by the exposure stop time measuring means to the reference imaging time stored in the reference imaging time storage means , And the like.

Invention of Claim 5 is an imaging device as described in any one of Claims 1-4,
The image sensor is configured to be capable of nondestructive readout of the image signal accumulated in the image sensor,
The camera shake detection means includes image signal analysis means (for example, the signal processing unit 7 in FIG. 1) that detects the camera shake by analyzing the image signal accumulated in the imaging device.

The invention described in claim 6 is the imaging apparatus according to claim 5,
The image sensor is a CMOS image sensor (for example, the CMOS image sensor 2 in FIG. 1) having a plurality of photoelectric conversion units,
The image signal analysis means includes partial image signal analysis means (for example, the signal processing unit 7 in FIG. 1) that analyzes a part of the image signal stored in the CMOS type image sensor and detects camera shake. It is characterized by that.

The invention according to claim 7 is the imaging apparatus according to any one of claims 1 to 4,
The camera shake detection means includes a gyro sensor (for example, the gyro sensor 217 in FIG. 6) that detects camera shake.

The invention according to claim 8 is the imaging apparatus according to any one of claims 1 to 4,
The image sensor is a CCD image sensor (for example, the CCD image sensor 302 in FIG. 8) having a plurality of photoelectric conversion units,
The camera shake detecting means analyzes partial image signal analyzing means for detecting camera shake by analyzing image signals accumulated in a partial area along the image signal transfer direction of the CCD image sensor (for example, FIG. 1). The signal processing unit 7 or the like is provided.

The invention according to claim 10 is:
An image pickup apparatus (for example, the digital camera 100 in FIG. 1) having an image pickup element (for example, the CMOS image sensor 2 in FIG. 1) and provided with an image pickup unit that takes an image of a subject by exposing the image pickup element. An imaging method used,
A process of detecting camera shake when the subject is imaged by the imaging means;
A process of stopping exposure of the image sensor every time the camera shake is detected, and restarting exposure of the image sensor each time the camera shake is no longer detected;
A process of generating a subject image based on an image signal accumulated in the image sensor each time exposure of the image sensor is stopped;
A process of combining the plurality of generated subject images is performed.

The program of the invention described in claim 11 is
An image pickup apparatus (for example, the digital camera 100 in FIG. 1) having an image pickup element (for example, the CMOS image sensor 2 in FIG. 1) and including an image pickup unit that takes an image of a subject by exposing the image pickup element. ,
A function of detecting camera shake when the subject is imaged by the imaging means;
A function of stopping exposure of the image sensor every time the camera shake is detected and restarting exposure of the image sensor every time the camera shake is no longer detected;
A function of generating a subject image based on an image signal accumulated in the image sensor each time exposure of the image sensor is stopped;
And a function of synthesizing the plurality of generated subject images.

  According to the present invention, it is possible to acquire a good composite image while suppressing noise related to image composition.

  Hereinafter, specific embodiments of the present invention will be described with reference to the drawings. However, the scope of the invention is not limited to the illustrated examples.

[Embodiment 1]
The digital camera (an imaging device or an electronic device with an imaging unit) 100 according to the first embodiment stops the exposure of the CMOS image sensor 2 every time a camera shake is detected in the high-sensitivity imaging mode, and is stored in the sensor 2. A subject image is generated based on the obtained image signal, and an imaging process for synthesizing the plurality of generated subject images is performed.

FIG. 1 is a block diagram showing a schematic configuration of a digital camera 100 according to a first preferred embodiment of an imaging apparatus to which the present invention is applied.
As shown in FIG. 1, a digital camera 100 includes a lens unit 1, a CMOS (Complementary Metal-oxide Semiconductor) image sensor (hereinafter referred to as “CMOS sensor”) 2, a CMOS control unit 3, and an A / D converter 4. , Image frame buffer 5, displacement detection buffer 6, signal processing unit 7, liquid crystal display control unit 8, liquid crystal display screen 9, lens control unit 10, focus filter unit 11, shutter button 12, mode button 13, and timer unit 14, a storage unit 15, a CPU 16 and the like are provided.

  The lens unit 1 forms, for example, a subject image, and is disposed so as to be exposed from the front side of the housing 1A.

In the CMOS sensor 2, for example, a plurality of photoelectric conversion units that photoelectrically convert the light that has passed through the lens unit 1 according to the amount of light and store them are arranged in a two-dimensional matrix.
In addition, the CMOS sensor 2 is configured to be capable of nondestructive reading of charges accumulated in each of a plurality of photoelectric conversion units, for example. That is, only the charge of a specific photoelectric conversion unit among a plurality of photoelectric conversion units can be read, and the charge is held after the reading.
The plurality of photoelectric conversion units are regions other than the position shift detection unit 2a and the position shift detection unit 2a for acquiring an image for detecting a position shift due to camera shake or the like. A main image acquisition unit 2b for acquiring a main image for image generation is provided.
As the misregistration detection unit 2a, for example, as shown in FIG. 2A, it is provided at four corners of the CMOS sensor 2, or as shown in FIG. Examples include a substantially rectangular frame shape so as to surround, and as shown in FIG. 2C, the main image acquisition unit 2b is provided along both upper and lower edges (Y direction).
When non-destructive reading is performed, the positional deviation detection is not limited to the above position, and the reading may be performed from an appropriate position.

  Note that, for example, when the camera shake detection process is performed, the positional deviation detection unit 2a is preferably gain-adjusted to increase sensitivity.

For example, the CMOS controller 3 sequentially reads out the charges accumulated in the CMOS sensor 2 under the control of the CPU 16 and outputs them to the A / D converter 4.
In addition, the CMOS control unit 3 outputs (transfers) the charges accumulated in the misregistration detection unit 2a, for example, at a constant time interval that is sufficiently short so that camera shake does not occur.

  The lens unit 1, the CMOS sensor 2, the CMOS control unit 3, and the like constitute the imaging unit of this embodiment.

  The A / D converter 4 converts, for example, an input analog signal into digital image data, and outputs the digital image data to the image frame buffer 5, the positional deviation detection buffer 6, and the focus filter unit 11.

  The image frame buffer 5 temporarily stores input image data, for example, and reads out the image data based on a control signal output from the CPU 16 and input it to the signal processing unit 7. It has become.

  The misregistration detection buffer 6 temporarily stores input misregistration detection image data, for example, and reads out the image data based on a control signal output from the CPU 16 and inputs the signal data. It outputs to the part 7.

  For example, the signal processing unit 7 is driven under the control of the CPU 16 and performs predetermined image processing on the input image data, and outputs the image data after the image processing to the liquid crystal display control unit 8. It is like that.

  For example, the liquid crystal display control unit 8 controls the liquid crystal display screen 9 according to a display control signal output from the CPU 16 and input, and displays a captured image corresponding to the image data output from the signal processing unit 7 and input. 9 is displayed.

  The liquid crystal display screen 9 is disposed so as to be exposed from the back side of the housing 1A, for example.

  For example, the lens control unit 10 is driven under the control of the CPU 16 to control the movement of the focus lens and zoom lens (none of which are not shown) of the lens unit 1 in the optical axis direction.

  The focus filter unit 11 is, for example, for extracting image data of an area related to focus adjustment by the lens control unit 10 from the image data output and input from the A / D converter 4.

The shutter button 12 instructs, for example, an imaging operation, and outputs an operation signal to the CPU 16 in accordance with a predetermined operation by the user.
The mode button 13 is for switching an imaging mode such as a normal imaging mode or a high sensitivity imaging mode, and outputs an operation signal to the CPU 16 in accordance with a predetermined operation by the user.

The timer unit 14 includes, for example, a timer, a timer circuit, and the like. For example, under the control of the CPU 16, an exposure time measuring unit that measures the exposure time during which the CMOS sensor 2 is exposed when the subject is imaged. Is configured.
Here, the exposure time is the time for accumulating charges in the photoelectric conversion unit of the CMOS sensor 2, and the charge is output as it is without being accumulated, although it is photoelectrically converted by the photoelectric conversion unit as in a through image. Time is not included.

  For example, the CPU (Central Processing Unit) 16 reads various programs stored in the storage unit 15, develops them in a work area formed in the storage unit 15, and controls the respective units according to the program. is there.

The storage unit 15 includes, for example, a memory that stores various programs and data, a memory that constitutes a work area of the CPU 16, and the like.
Specifically, in the storage unit 15, for example, as shown in FIG. 3, a camera shake detection program 15a, a camera shake exposure control program 15b, an image generation program 15c, an exposure time determination program 15d, an exposure end program 15e, and an image composition Programs such as the program 15f, reference imaging time information d1, and the like are stored.

The camera shake detection program 15 a causes the signal processing unit 7 to function as a camera shake detection unit under the control of the CPU 16. That is, the camera shake detection program 15a is a program for causing the signal processing unit 7 to realize a function related to camera shake detection processing for detecting camera shake during the subject imaging processing.
Specifically, based on the execution of the camera shake detection program 15a by the CPU 16, the signal processing unit 7 continuously outputs the image signals accumulated in the positional deviation detection unit 2a of the CMOS sensor 2 at a predetermined timing as a camera shake detection unit. Acquiring and detecting the occurrence of camera shake according to whether or not the image signals are shifted by a predetermined distance (for example, two pixels) or more.
Here, the signal processing unit 7 constitutes partial image signal analysis means (image signal analysis means) for analyzing a part of the image signal stored in the CMOS sensor 2 and detecting camera shake. Here, a threshold value indicating whether or not the shift is greater than or equal to a predetermined distance is determined to determine camera shake. However, this need not be the case and a predetermined threshold value is not necessarily provided.

The camera shake exposure control program 15b causes the CMOS control unit 3 to function as camera shake exposure control means under the control of the CPU 16. That is, based on the execution of the camera shake exposure control program 15b by the CPU 16, the exposure of the CMOS sensor 2 (charge accumulation in the photoelectric conversion unit) is stopped and camera shake is detected every time camera shake is detected in the camera shake detection process. This is a program for causing the CMOS control unit 3 to realize a function related to the camera shake exposure control process for resuming the exposure of the CMOS sensor 2 (charge accumulation in the photoelectric conversion unit) each time it runs out.
Specifically, when detecting a camera shake in the camera shake detection process, the CPU 16 outputs a state change notification for changing the exposure state of the CMOS sensor 2 to the CMOS control unit 3, and the CMOS control unit 3 causes the CMOS sensor 2 to When the exposure is stopped and it is determined that the camera shake has stopped, a state change notification is output to the CMOS control unit 3, and the CMOS control unit 3 performs a process of restarting the exposure of the CMOS sensor 2. .

The image generation program 15 c causes the signal processing unit 7 to function as an image generation unit under the control of the CPU 16. That is, the image generation program 15c performs image generation processing for generating a subject image based on the image signal accumulated in the photoelectric conversion unit of the sensor every time the exposure of the CMOS sensor 2 is stopped in the camera shake exposure control processing. This is a program for causing the signal processing unit 7 to realize such a function.
Specifically, based on the execution of the image generation program 15c by the CPU 16, every time the exposure of the CMOS sensor 2 is stopped in the camera shake exposure control process, the main image acquisition unit 2b and the position deviation detection unit 2a of the CMOS sensor 2 are stopped. The image signal output from is transferred to the signal processing unit 7 via the A / D converter 4 and the image frame buffer 5, and the signal processing unit 7 generates a subject image based on the image signal. Yes.
Here, for example, the signal processing unit 7 normalizes and synthesizes the image signal output from the positional deviation detection unit 2a of the CMOS sensor 2 so as to be substantially equal to the exposure time of the main image acquisition unit 2b. May be.

  The exposure time determination program 15d causes the CPU 16 to function as exposure time determination means. That is, the exposure time determination program 15d determines the exposure time of the CMOS sensor 2 measured by the time measuring unit 14, specifically, the exposure time accumulated by stopping and restarting the exposure in the camera shake exposure control process. This is a program for causing the CPU 16 to realize a function related to an exposure time determination process for determining whether or not a predetermined exposure amount has been obtained depending on whether or not the reference imaging time stored in the storage unit 15 has been reached. .

  The exposure end program 15e causes the CMOS control unit 3 to function as an exposure end unit under the control of the CPU 16. That is, the exposure end program 15e gives the CMOS control unit 3 a function related to the exposure end process for ending the exposure of the CMOS sensor 2 when it is determined in the exposure time determination process that the exposure time has reached the reference imaging time. It is a program for realizing.

The image composition program 15f causes the signal processing unit 7 to function as an image composition unit under the control of the CPU 16. That is, the image composition program 15f is a program for realizing a function related to an image composition process for composing a plurality of subject images generated by the image generation process.
Specifically, based on the execution of the image composition program 15f by the CPU 16, the signal processing unit 7 acquires the shift amounts of the plurality of subject images generated by the image generation process, and based on the shift amounts. Images are synthesized so that a plurality of subject images overlap each other.

The reference imaging time information d1 is a reference for imaging time in the normal imaging mode and the high-sensitivity imaging mode by the digital camera 100.
Here, as a method for determining the imaging time, for example, the reflectance of the subject image is acquired by AE processing executed based on a half-pressing operation of the shutter button 12 by the user, and the reflectance and the reference imaging time information are obtained. There is a method of determining the imaging time so that the captured image has a predetermined exposure amount based on d1.
Here, the storage unit 15 constitutes a reference imaging time storage unit that stores the reference imaging time.

Next, imaging processing by the digital camera 100 will be described with reference to FIGS. 4 and 5.
Here, FIG. 4 is a flowchart showing an example of the operation related to the imaging process, and FIG. 5 is a timing chart showing an example of the operation related to the imaging process.

  First, after the imaging mode is set to the high-sensitivity imaging mode based on a predetermined operation of the mode button 13 by the user, when the shutter button 12 is pressed at a desired timing (time t1; see FIG. 5) (step) S1) Under the control of the CPU 16, the lens control unit 10 adjusts the position of the lens unit 1 in the optical axis direction, and the CMOS control unit 3 causes the CMOS sensor 2 to start exposure (step S2).

Then, under the control of the CPU 16, the CMOS control unit 3 reads and outputs the charge (image signal) accumulated in the misregistration detection unit 2a at a constant time interval that is short enough to prevent camera shake (step S3). ).
When the image signal output from the misregistration detection unit 2a is input to the signal processing unit 7 via the A / D converter 4 and the misregistration detection buffer 6, the CPU 16 executes the camera shake detection program 15a, The signal processing unit 7 is caused to execute a camera shake detection process for determining whether or not a camera shake has occurred according to the fluctuation amount of the image signal (step S4).
If it is determined that camera shake has occurred (step S4; YES), the CPU 16 executes the camera shake exposure control program 15b and outputs a state change notification to the CMOS control unit 3 (time t2). ; See FIG. 5), the CMOS controller 3 stops the exposure of the CMOS sensor 2 (time t3; see FIG. 5), and the charges accumulated in the main image acquisition unit 2b and the misregistration detection unit 2a of the CMOS sensor 2 Is output to the A / D converter 4.
Subsequently, the CPU 16 executes the image generation program 15c, and outputs the image signal output from the main image acquisition unit 2b and the misregistration detection unit 2a of the CMOS sensor 2 to the signal processing unit 7 via the image frame buffer 5. The signal is transferred, and the signal processing unit 7 executes image generation processing for generating a subject image based on the image signal (step S5).

Thereafter, under the control of the CPU 16, the CMOS control unit 3 reads and outputs the charge (image signal) accumulated in the misregistration detection unit 2a at a constant time interval that is short enough to prevent camera shake (step S6). ).
When the image signal output from the misregistration detection unit 2a is input to the signal processing unit 7, the CPU 16 executes the camera shake detection program 15a, and whether the camera shake is settled in the signal processing unit 7 according to the variation amount of the image signal. It is determined whether or not (step S7).
Here, when it is determined that the image signal of the misregistration detection unit 2a is not changed and the camera shake is settled (step S7; NO), the CPU 16 executes the camera shake exposure control program 15b and proceeds to step S2. Then, a state change notification is output to the CMOS control unit 3 (time t4; see FIG. 5), and the CMOS control unit 3 resumes exposure of the CMOS sensor 2 (time t5; see FIG. 5).

On the other hand, when it is determined in step S4 that camera shake has not occurred (step S4; NO), the CPU 16 executes the exposure time determination program 15d, and the exposure of the CMOS sensor 2 measured by the timer unit 14 is performed. It is determined whether or not a predetermined exposure amount has been obtained depending on whether or not the time has reached the reference imaging time (step S8).
Here, if it is determined that the exposure time of the CMOS sensor 2 has not reached the reference imaging time (step S8; NO), the CPU 16 proceeds to step S2 and controls the execution of the subsequent processing.

If it is determined in step S8 that the exposure time of the CMOS sensor 2 has reached the reference imaging time (step S8; YES), that is, as shown in FIG. When it is determined that the exposure time of the CMOS sensor 2 has reached the reference imaging time by performing a predetermined number of times of exposure, exposure stop, and exposure restart at predetermined times t2 to t9, the CPU 16 executes an exposure end process ( Time t10; see FIG.
That is, the CPU 16 executes the exposure end program 15e, controls the CMOS control unit 3 to end the exposure of the CMOS sensor 2, and causes the main image acquisition unit 2b and the misregistration detection unit 2a of the CMOS sensor 2 to The accumulated charge is read out and output to the A / D converter 4.
Subsequently, the CPU 16 executes the image generation program 15c to transfer the image signal output from the main image acquisition unit 2b and the misregistration detection unit 2a of the CMOS sensor 2 to the signal processing unit 7 so as to generate the image signal. Based on this, an image generation process for causing the signal processing unit 7 to generate a subject image is executed (step S9).

Next, the CPU 16 executes the image composition program 15f, and performs signal composition processing (frame image calculation) for compositing a plurality of subject images generated by a plurality of times (for example, three times) of image generation processing. The process is executed by the unit 7 (step S10). At this time, if the position of the image to be combined is shifted due to camera shake between exposures, the image may be combined after adjusting the positional shift.
Subsequently, the CPU 16 controls the signal processing unit 7 to perform predetermined image processing such as JPEG compression processing on the composite image, and controls the liquid crystal display control unit 8 to display the composite image on the liquid crystal display screen 9. At the same time, the composite image data is stored in a recording medium (not shown) (step S11).

  As described above, according to the digital camera 100 of the first embodiment, the exposure of the CMOS sensor 2 is stopped every time the camera shake is detected in the imaging process, and the exposure of the CMOS sensor 2 is restarted after the camera shake is stopped. Each time the exposure of the CMOS sensor 2 is stopped, a subject image can be generated based on the image signal stored in the CMOS sensor 2 and a plurality of generated subject images can be synthesized. It is possible to reduce the number of images to be combined. As a result, it is possible to reduce the time required for the image combining process and to acquire a combined image in which noise is suppressed.

  If it is determined that the exposure time for exposing the CMOS sensor 2 has reached the reference imaging time, the exposure of the CMOS sensor 2 is terminated, so that a sufficiently exposed composite image can be acquired. That is, even if the exposure of the CMOS sensor 2 is stopped every time a camera shake is detected, the time when the CMOS sensor 2 is actually exposed can be obtained by measuring the exposure time by the time measuring unit 14. It is possible to prevent the image from being underexposed.

  Further, by using the CMOS sensor 2 as an image sensor, non-destructive reading of the charge accumulated in the misalignment detection unit 2a among the plurality of photoelectric conversion units is possible, and the misalignment detection unit is used for detecting camera shake. Even if the electric charge accumulated in 2a is used, the image signal of the subject image can be appropriately acquired.

  In the first embodiment, the CMOS sensor 2 is exemplified as the image sensor. However, the present invention is not limited to this, and any image sensor can be used as long as the image signal stored in the photoelectric conversion unit can be read nondestructively. There may be.

[Embodiment 2]
The digital camera 200 according to the second embodiment will be described below with reference to FIGS.
Here, FIG. 6 is a block diagram showing a schematic configuration of a digital camera 200 according to a second preferred embodiment of the imaging apparatus to which the present invention is applied. FIG. 7 is a flowchart illustrating an example of an operation related to the imaging process.
The digital camera 200 according to the second embodiment is substantially the same as the first embodiment except for the configuration of the image sensor and the gyro sensor 217 related to camera shake detection. The description is omitted.

  For example, as shown in FIG. 6, the digital camera 200 according to the second embodiment includes a CCD (Charge Coupled Device) type image sensor 202 as an image sensor, and includes a gyro sensor 217 and a sensor control unit 218 for camera shake detection. I have.

  In the CCD image sensor (hereinafter referred to as “CCD sensor”) 202, for example, a plurality of photoelectric conversion units are arranged in a two-dimensional matrix and stored in the photoelectric conversion unit under the control of the CCD control unit 203. Charges are sequentially transferred and read in a predetermined direction (for example, the vertical direction (Y direction)).

The gyro sensor 217 detects, for example, an acceleration generated when the digital camera 200 is moved in an imaging process, and outputs an acceleration detection signal to the CPU 16.
When the acceleration detection signal output from the gyro sensor 217 is input as the camera shake detection unit, the CPU 16 executes the camera shake detection program 15a, calculates the movement amount from the detected acceleration, and performs the movement. A camera shake detection process is performed to determine the occurrence of camera shake depending on whether the amount is deviated by a predetermined value (for example, 0.1 mm) or more (steps S204 and S207; see FIG. 7). Note that hand shake may be detected directly from the detected acceleration. Further, here, a threshold value indicating whether or not the movement amount is equal to or greater than a predetermined value is provided to determine the occurrence of camera shake. However, this need not be the case, and the predetermined threshold value is not necessarily provided.
Note that the imaging process illustrated in FIG. 7 is different from the imaging process of the first embodiment in the camera shake detection process method (steps S204 and S207), and other processes are substantially the same. Shall be omitted.

As described above, according to the digital camera 200 of the second embodiment, even if the CCD sensor 202 that cannot perform non-destructive reading of the charges accumulated in the plurality of photoelectric conversion units is used as the image pickup device, the gyro sensor 217 is used. By mounting, camera shake can be detected based on the acceleration detection signal output from the gyro sensor 217.
Therefore, the CCD sensor 202 can appropriately acquire the image signal of the subject image.

The present invention is not limited to the above-described embodiment, and various improvements and design changes may be made without departing from the spirit of the present invention.
Hereinafter, modified examples of the digital camera will be described.

<Modification 1>
The digital camera according to the first modification is a modification according to the second embodiment. For example, as illustrated in FIG. Camera shake is detected by reading and analyzing the charge accumulated in both edge portions (partial region) 302a.
That is, under the control of the CPU 16, the CCD control unit 203 reads out the electric charges accumulated in the both edge portions 302a of the CCD sensor 302 at intervals that are sufficiently short so that camera shake does not occur, and the image signal is subjected to signal processing. When input to the unit 7, the CPU 16 causes the signal processing unit 7 to execute a camera shake detection process for determining whether or not a camera shake has occurred according to the variation amount of the image signal based on the execution of the camera shake detection program 15 a. It is like that.
Here, the signal processing unit 7 constitutes partial image signal analysis means for analyzing image signals accumulated in a partial region along the image signal transfer direction of the CCD sensor 302 and detecting camera shake. .

  In the case of the digital camera according to the first modification, the charges accumulated in the both edge portions 302a of the CCD sensor 302 are read out during camera shake detection. Therefore, a plurality of CCD sensors 302 are used for image generation processing and image synthesis processing. Among the photoelectric conversion units, charges accumulated in the photoelectric conversion units in the region 302b other than the both edge portions 302a are used.

Therefore, according to the digital camera of the first modification, even a camera equipped with the CCD sensor 302 can detect camera shake and acquire an image signal of a subject image without including the gyro sensor 217.
That is, by using the electric charges accumulated in both edge portions 302a of the CCD sensor 302 for detecting camera shake, the subject image capturing range is slightly narrowed, but the main subject image is obtained by the photoelectric conversion unit in the other region 302b. The image signal of at least can be acquired.

<Modification 2>
The digital camera of the second modification is a modification of the first and second embodiments. When the subject is imaged by the timer unit (exposure stop time measuring means) 14, the image sensor (CMOS sensor or The CPU 16 measures the exposure stop time when the exposure of the CCD sensor or the like is stopped, and the CPU 16 adds the exposure stop time to the reference imaging time and executes an imaging time extension process for extending the imaging time of the camera. It has become.
That is, the CPU 16 reads out and executes the imaging time extension program 415g (see FIG. 9) from the storage unit 415 every time the exposure of the imaging device stops as the imaging time extension means, and the exposure timed by the timing unit 14. The stop time is added to the reference imaging time to extend the imaging time of the camera.

  Therefore, according to the digital camera of the modified example 2, the exposure time when the exposure of the image sensor is stopped can be added to the reference imaging time to extend the imaging time. Can be acquired. That is, even if the exposure of the image sensor is stopped every time camera shake is detected, the time during which the image sensor is exposed can be extended by measuring the exposure stop time by the timer 14 and adding it to the reference imaging time. It is possible to prevent the composite image from being underexposed.

<Modification 3>
The digital camera of the third modification is a modification of the first and second embodiments, and it is determined whether or not the exposure time measured by the time measuring unit 14 has reached the reference imaging time, and the exposure time has reached the reference imaging time. If it is determined that it is not, the signal processing unit 7 executes a luminance adjustment process for adjusting the luminance of the composite image.

That is, the CPU 16 reads out the imaging time determination program 515h (see FIG. 10) from the storage unit 515 as an exposure time determination unit, and after the imaging of the subject is completed, that is, after the reference imaging time has elapsed in the imaging processing of the subject. An exposure time determination process for determining whether or not the exposure time measured by the timer unit 14 at the time of imaging the subject has reached the reference imaging time is executed.
Further, when the exposure time has not reached the reference imaging time, the signal processing unit 7 serves as the luminance adjustment means based on the execution of the luminance adjustment program 515i (see FIG. 10) by the CPU 16 so that the luminance becomes a predetermined value. The brightness of the composite image is adjusted.

  Therefore, according to the digital camera of the third modification, when the exposure time does not reach the reference imaging time, that is, when the composite image becomes underexposed by stopping the exposure of the imaging device every time camera shake is detected. However, a clear image can be obtained by adjusting the brightness of the composite image.

  In the third modification, when it is determined that the exposure time has not reached the reference imaging time, a predetermined time (for example, the exposure stop time measured during the camera shake exposure control process) (for example, (Half time) may be added to the reference imaging time to extend the imaging time of the camera. As a result, the exposure time of the composite image can be made longer, and a clear image whose luminance has been adjusted can be acquired.

  In addition, in the above-described embodiment, functions as a camera shake detection unit, an image signal analysis unit, a partial image signal analysis unit, an image generation unit, an image synthesis unit, a luminance adjustment unit, an exposure time determination unit, and an imaging time extension unit are provided. The CPU 16 is configured to be realized by executing a predetermined program or the like. However, the configuration is not limited to this. For example, the CPU 16 may be configured by a logic circuit or the like for realizing various functions.

It is a block diagram which shows schematic structure of the digital camera of suitable Embodiment 1 of the imaging device to which this invention is applied. It is a figure which shows typically the CMOS type image sensor with which the digital camera of FIG. 1 is equipped. It is a figure which shows the memory content of the memory | storage part with which the digital camera of FIG. 1 is equipped. 3 is a flowchart illustrating an example of an operation related to an imaging process by the digital camera in FIG. 1. 5 is a timing chart showing an example of an operation related to the imaging process of FIG. 4. It is a block diagram which shows schematic structure of the digital camera of suitable Embodiment 2 of the imaging device to which this invention is applied. It is a flowchart which shows an example of the operation | movement which concerns on the imaging process by the digital camera of FIG. FIG. 6 is a diagram schematically showing a CCD type image sensor provided in a digital camera of Modification 1; FIG. 11 is a diagram illustrating the storage contents of a storage unit provided in a digital camera of Modification Example 2. FIG. 10 is a diagram illustrating the storage contents of a storage unit provided in a digital camera of Modification 3;

Explanation of symbols

100, 200 Digital camera (imaging device)
2 CMOS image sensor (imaging device, imaging means)
2a Misalignment detection unit 2b Main image acquisition unit 3 CMOS control unit (imaging means, camera shake exposure control means, exposure end means)
7 Signal processing unit (camera shake detection means, image signal analysis means, partial image signal analysis means, image generation means, image synthesis means, brightness adjustment means)
14 Timekeeping unit (exposure time measuring means, exposure stop time measuring means, imaging time measuring means)
15 Storage unit (reference imaging time storage means)
16 CPU (exposure time determination means, imaging time extension means)
202 CCD image sensor (imaging device, imaging means)
203 CCD control section (imaging means, camera shake exposure control means, exposure end means)
217 Gyro sensor

Claims (10)

An image pickup unit having an image pickup device and picking up an image of a subject by exposing the image pickup device;
Camera shake detection means for detecting camera shake when the subject is imaged by the imaging means;
Each time the camera shake is detected by the camera shake detector, the exposure of the image sensor is stopped, and each time the camera shake is not detected by the camera shake detector, the exposure control means during camera shake is resumed. When,
Image generation means for generating a subject image based on an image signal accumulated in the image sensor each time exposure of the image sensor is stopped by the camera shake exposure control means;
An image pickup apparatus comprising: an image synthesis unit that synthesizes the plurality of subject images generated by the image generation unit. Reference imaging time storage means for storing a reference imaging time which is a reference of imaging time by the imaging means;
An exposure time measuring means for measuring an exposure time during which the image sensor is exposed when the subject is imaged by the imaging means;
Exposure time determination means for determining whether or not the exposure time measured by the exposure time measurement means has reached the reference imaging time stored in the reference imaging time storage means;
2. The imaging according to claim 1, further comprising an exposure end unit that ends the exposure of the image sensor when the exposure time determination unit determines that the exposure time has reached the reference imaging time. apparatus. Reference imaging time storage means for storing a reference imaging time which is a reference of imaging time by the imaging means;
An exposure time measuring means for measuring an exposure time during which the image sensor is exposed when the subject is imaged by the imaging means;
An exposure time for determining whether or not the exposure time measured by the exposure time measuring means has reached the reference imaging time stored in the reference imaging time storage means after the imaging of the subject by the imaging means is completed. A determination means;
A brightness adjusting unit that adjusts the brightness of the synthesized image synthesized by the image synthesizing unit when the exposure time judging unit judges that the exposure time has not reached the reference imaging time. The imaging device according to claim 1. Reference imaging time storage means for storing a reference imaging time which is a reference of imaging time by the imaging means;
An exposure stop time measuring means for measuring an exposure stop time during which exposure of the image sensor is stopped by the camera shake exposure control means when the subject is imaged by the imaging means;
An imaging time extending means for adding the exposure stop time measured by the exposure stop time measuring means to the reference imaging time stored in the reference imaging time storage means to extend the imaging time by the imaging means; The imaging apparatus according to claim 1, further comprising: The image sensor is configured to be capable of nondestructive readout of the image signal accumulated in the image sensor,
5. The imaging according to claim 1, wherein the camera shake detection unit includes an image signal analysis unit that analyzes the image signal accumulated in the image sensor and detects the camera shake. 6. apparatus. The image sensor is a CMOS image sensor having a plurality of photoelectric conversion units,
6. The imaging according to claim 5, wherein the image signal analysis means includes partial image signal analysis means for analyzing a part of the image signal accumulated in the CMOS image sensor and detecting camera shake. apparatus.   The imaging apparatus according to claim 1, wherein the camera shake detection unit includes a gyro sensor that detects camera shake. The image sensor is a CCD image sensor having a plurality of photoelectric conversion units,
The camera shake detection means includes partial image signal analysis means for detecting a camera shake by analyzing an image signal accumulated in a partial area along the transfer direction of the image signal of the CCD image sensor. The imaging device according to any one of claims 1 to 4. An imaging method using an imaging device having an imaging device and including an imaging means for imaging a subject by exposing the imaging device,
A process of detecting camera shake when the subject is imaged by the imaging means;
A process of stopping exposure of the image sensor every time the camera shake is detected, and restarting exposure of the image sensor each time the camera shake is no longer detected;
A process of generating a subject image based on an image signal accumulated in the image sensor each time exposure of the image sensor is stopped;
And a process of combining the plurality of generated subject images. An image pickup apparatus having an image pickup element and having an image pickup unit that picks up an image of a subject by exposing the image pickup element.
A function of detecting camera shake when the subject is imaged by the imaging means;
A function of stopping exposure of the image sensor every time the camera shake is detected and restarting exposure of the image sensor every time the camera shake is no longer detected;
A function of generating a subject image based on an image signal accumulated in the image sensor each time exposure of the image sensor is stopped;
A program for realizing a function of combining the plurality of generated subject images.

Images