JP4378602B2 - Imaging apparatus and imaging method - Google Patents

Description

  The present invention relates to an imaging apparatus. More specifically, the present invention relates to an imaging apparatus that prevents cumulative addition of image signals that degrade image quality (resolution) among image signals that have undergone camera shake correction.

  In an imaging apparatus that can capture and record images such as a digital still camera and a video camera in the prior art, an image captured due to vibration caused by a camera shake of a photographer may be blurred or the recorded image may be deteriorated (bleeded or shifted). In order to prevent this, various image stabilization techniques have been devised.

For example, an imaging device that detects the amount of blur at the time of shooting from an image (still image) captured during a predetermined period or a predetermined number of times and records an image (still image) with the smallest amount of blur has been devised (for example, Patent Document 1).
Japanese Patent Laid-Open No. 11-317904 (page 11-12, FIGS. 1 and 26)

  However, in a normal camera shake correction process, correction in units of pixels is rough, so correction between pixels (correction less than an integer pixel) must be performed, but the minimum unit for sampling an image signal is in units of one pixel. Therefore, interpolation processing between pixels is necessary.

  This interpolation processing includes methods such as linear interpolation processing and CUBIC interpolation processing. However, there is a problem that an image signal including a component having a frequency equal to or higher than the Nyquist frequency deteriorates (bleeds or shifts) during the interpolation processing. .

  Therefore, in an imaging device that cumulatively adds captured image signals and stores them in a memory, the image signals that have undergone interpolation processing and subjected to camera shake correction are cumulatively added. For example, as shown in FIG. If all the image signals that have undergone camera shake correction with correction vector data that takes a range of +1 are cumulatively added in the time axis direction and stored in the memory, the image quality (resolution) of the stored image signal is deteriorated. Yes.

  Therefore, in an imaging apparatus having means for accumulating and storing captured image signals in a memory, it is possible to prevent accumulation of image signals that degrade image quality (resolution) among image signals that have undergone camera shake correction. Having problems that must be solved.

In order to solve the above problems, an imaging apparatus and an imaging method according to the present invention are configured as follows.

(1) An imaging apparatus having means for accumulating and storing in a memory the image signals captured by the imaging unit, and detecting a vibration amount due to camera shake of the imaging unit, and according to the detected camera shake vibration amount A camera shake detection unit that outputs a camera shake signal, a camera shake correction amount calculation unit that calculates a camera shake correction amount for correcting the image signal based on the camera shake signal output from the camera shake detection unit, and the camera shake correction amount calculation unit Interpolation coefficient calculation means for calculating an interpolation coefficient value for interpolating between pixels of the image signal photographed by the imaging unit based on the camera shake correction quantity calculated in step 1, and the camera shake correction quantity calculated by the camera shake correction quantity calculation means Based on the interpolation coefficient value calculated by the interpolation coefficient calculation means, and a camera shake correction means for performing camera shake correction by performing an interpolation process on the image signal captured by the imaging unit, and the interpolation coefficient calculation Compared with a predetermined threshold value which is set in advance as the interpolation coefficient value calculated in step, cumulative addition or the interpolation coefficient value is a good image even when accumulating the image signal when a value of the predetermined range An image discriminating unit that performs an image signal discriminating process for accumulating and adding only an image that may be cumulatively added to the memory after discriminating that the image may not be an image, and the image stabilization unit according to the image discriminating unit And an accumulative addition control means for controlling the accumulative addition processing of the image signal corrected in step (b).

(2) An image pickup apparatus having means for accumulatively adding image signals taken by the image pickup unit and storing them in a memory, wherein the vibration amount due to camera shake of the image pickup unit is detected, and according to the detected amount of camera shake vibration A camera shake detection unit that outputs a camera shake signal, a camera shake correction amount calculation unit that calculates a camera shake correction amount for correcting the image signal based on the camera shake signal output from the camera shake detection unit, and the camera shake correction amount calculation unit Interpolation coefficient calculation means for calculating an interpolation coefficient value for interpolating between pixels of the image signal photographed by the imaging unit based on the camera shake correction quantity calculated in step 1, and the camera shake correction quantity calculated by the camera shake correction quantity calculation means Based on the interpolation coefficient value calculated by the interpolation coefficient calculation means, and a camera shake correction means for performing camera shake correction by performing an interpolation process on the image signal captured by the imaging unit, and the interpolation coefficient calculation Compared with a predetermined threshold value which is set in advance as the interpolation coefficient value calculated in step, cumulative addition or the interpolation coefficient value is a good image even when accumulating the image signal when a value of the predetermined range An image discriminating unit that performs an image signal discriminating process for accumulating and adding only an image that may be cumulatively added to the memory after discriminating that the image may not be an image, and the image stabilization unit according to the image discriminating unit The cumulative addition control means for controlling the cumulative addition processing of the image signal corrected in step (i), the coefficient value calculated according to the number of times of cumulative addition processing by the cumulative addition control means, and the image signal cumulatively added and stored in the memory And an image output means for outputting the result of multiplication.
(3) An imaging apparatus including means for accumulatively adding image signals photographed by the imaging unit and storing them in a memory, wherein an amount of vibration due to camera shake of the imaging unit is detected, and according to the detected amount of camera shake vibration A camera shake detection step for outputting a camera shake signal, a camera shake correction amount calculation step for calculating a camera shake correction amount for correcting the image signal based on the camera shake signal output in the camera shake detection step, and a camera shake correction amount calculation step. An interpolation coefficient calculating step for calculating an interpolation coefficient value for interpolating between pixels of the image signal captured by the imaging unit based on the image stabilization amount, and the image stabilization amount calculated in the image stabilization amount calculating step and the interpolation Based on the interpolation coefficient value calculated in the coefficient calculation step, a camera shake correction step for performing camera shake correction by performing interpolation processing of the image signal captured by the imaging unit. Flop, wherein comparing the interpolation coefficient calculation step predetermined threshold calculated interpolation coefficient value is set in advance in, the image signal when the interpolation coefficient value is a value of the predetermined range may be cumulatively added An image determination step for performing an image signal determination process in which only an image that may be cumulatively added is cumulatively added to the memory after determining whether the image is an image or an image that may be cumulatively added. And a cumulative addition control step for controlling the cumulative addition processing of the image signals corrected in the camera shake correction step.

  The image pickup apparatus having such a configuration controls the cumulative addition processing of the image signal corrected by the camera shake correction unit by comparing the interpolation coefficient value calculated by the interpolation coefficient calculation unit with a predetermined threshold value set in advance. Further, it is possible to prevent the cumulative addition of the image signals whose resolution is deteriorated by the interpolation process among the image signals subjected to the camera shake correction by the interpolation process.

  The interpolation coefficient value calculated for the interpolation process is compared with a predetermined threshold value, and only the image signal that has been subjected to the interpolation process with the interpolation coefficient value within the predetermined range and subjected to camera shake correction is accumulated and stored. Therefore, an image signal whose resolution is deteriorated during the interpolation process is not cumulatively added, and an excellent effect of improving the image quality of the image signal stored in the memory while maintaining the camera shake correction processing accuracy is obtained. be able to.

  In addition, since the image signal stored in the above-described memory is multiplied and output by a coefficient value corresponding to the number of cumulative additions, an image signal (still image) with a good image quality (S / N ratio) is output to a recording medium, It can be recorded on a recording device.

Next, embodiments of the imaging apparatus and imaging method of the present invention will be described with reference to the drawings. However, the drawings are only for explanation, and do not limit the technical scope of the present invention.

  FIG. 1 is a block diagram schematically illustrating main components in an imaging apparatus according to the present invention. The imaging unit 110, a correction circuit 120, a cumulative addition circuit 130, a multiplier 140, a recording unit 150, a microcomputer 160, It includes a camera shake detection unit 170 and the like.

  The imaging unit 110 includes an imaging element such as a lens and a CCD (Charge Coupled Device), a signal processing circuit, and the like, converts light from a subject input via the lens into an image signal by the imaging element, and performs signal processing. The circuit performs predetermined signal processing on the image signal and sends it to the correction circuit 120.

The correction circuit 120 corrects the image signal sent from the imaging unit 110 based on the correction vector data specified from the microcomputer 160 and sends the corrected image signal to the cumulative addition circuit 130 .

  The accumulative addition circuit 130 accumulatively adds the image signals sent from the correction circuit 120 in the time axis direction and stores them in the memory every time capture is performed under the control of the microcomputer 160.

  The multiplier 140 multiplies the image signal output from the cumulative addition circuit 130 based on the coefficient value designated by the microcomputer 160 and outputs the image signal to the recording unit 150 under the control of the microcomputer 160.

  The recording unit 150 records the image signal output from the multiplier 140 under the control of the microcomputer 160. For example, an image signal is recorded on a recording medium such as a hard disk device, a memory card, a magnetic tape, or an optical disk.

  The microcomputer 160 controls each unit and each circuit in the imaging apparatus based on a program or data stored in advance.

  For example, an interpolation for calculating a camera shake correction amount and correction vector data for correcting camera shake based on a camera shake signal detected by the camera shake detection unit 170, or performing an interpolation process for correcting camera shake between pixels. The coefficient value is calculated and sent to the correction circuit 120.

  Further, the degree of resolution degradation of the image signal is determined based on the calculated interpolation coefficient value to control the cumulative addition circuit 130, the coefficient value to the multiplier 140 is calculated and transmitted, and the image signal is recorded in the recording unit 150. It also controls the timing when recording data.

  The camera shake detection unit 170 detects the vibration amount due to the camera shake of the imaging unit 110 and sends the detected vibration amount to the microcomputer 160 as a camera shake signal. The camera shake signal is detected by using a gyro sensor or an image recognition technology, and a value obtained by integrating the vibration amount due to the camera shake is detected as a camera shake signal every time the image signal is captured (captured). Send it out.

  For example, in an integrator representing complete integration as shown in FIG. 2, when the input signal X (z) and the output signal Y (z) are used, the camera shake signal can be expressed by the following equation.

Y (z) = 1 / {(1-Z ⁻¹ ) · X (z)}

  Here, a method of obtaining a correction amount between pixels (correction amount less than an integer pixel) by linear interpolation when performing an interpolation process for correcting a camera shake between pixels in an image signal will be described.

  For example, as shown in FIG. 3, when correction is performed between the pixel 1 and the pixel 2 in the still image, the coordinates of the image 1 are “x”, the coordinates of the pixel 2 are “x + 1”, the pixels 1 and 2 are If the distance (vector) is 1 and the value after the decimal point of the correction amount (correction vector) is a and the signal level of image 1 at coordinate x is g (x), The correction amount (signal level) of the pixel A generated by the interpolation processing of the pixel 2 can be expressed by the following equation.

  g (x + a) = (1-a) * g (x) + a * g (x + 1); x is an integer

  In the above formula, x is an integer, and (1-a) is an interpolation coefficient, and the degree of resolution degradation is determined based on the value of this interpolation coefficient.

  Next, a method for determining the resolution of the image signal based on the interpolation coefficient will be described.

  FIG. 4 shows the relationship between the interpolation coefficient and the resolution, and is a graph schematically showing the relationship between the interpolation coefficient and the gain (pixel signal level) in the spatial frequency domain.

  As shown in the graph of FIG. 4, when the interpolation coefficient takes a value between 0 and 1, the gain (pixel signal level) decreases as the interpolation coefficient value approaches 0.5. Basically, the higher the spatial frequency of the image, the more the gain (pixel signal level) decreases. For this reason, a decrease in gain (pixel signal level) causes degradation in resolution. That is, the resolution deteriorates as the value of the interpolation coefficient approaches 0.5 (approaches an intermediate point between pixels).

  Generally, when the change in resolution becomes significant due to a change in the interpolation coefficient, it will cause unnaturalness in the image. Therefore, it is necessary to avoid an interpolation coefficient that degrades the resolution. In order to maintain, there is a method of correcting the interpolation coefficient to the nearest integer value (the signal level of the nearest pixel).

  In the imaging apparatus of the present invention, since the image signal is cumulatively added in the time axis direction, when the camera shake correction processing is performed by changing the value of the interpolation coefficient, the cumulatively added image looks double or the like. Since unnaturalness occurs, by setting a predetermined threshold A as shown in FIG. 4 and discriminating the interpolation coefficient, cumulative addition of image signals that have undergone camera shake correction by the interpolation coefficient that causes resolution degradation is prevented.

  An operation flow in the imaging apparatus having such a configuration will be described with reference to the flowchart of FIG.

  When shooting starts, first, the microcomputer 160 determines whether or not the value of a capture counter (hereinafter referred to as CC) that counts the number of captured image signals (hereinafter referred to as capture number) is “0” (ST110). ).

  When CC is “0”, the microcomputer 160 clears the correction amount for the correction circuit 120 to “0” (ST111).

  Subsequently, the cumulative addition circuit 130 captures the image signal photographed by the imaging unit 110 and cumulatively adds it to the memory 132 (ST112).

  As a result, the first reference image signal is first stored in the memory of the cumulative addition circuit 130. This is because the correction amount of the first image signal is cleared to “0” and does not exceed the maximum correction amount. In the image signal captured thereafter, the correction amount of camera shake always exceeds the maximum correction amount. In such a case, the image signal is prevented from being captured at all.

  Then, the microcomputer 160 increments the value of CC by one, further increments the value of the time counter (hereinafter referred to as TC) by one, and ends the processing (ST113, ST114 → end).

  On the other hand, when CC is not “0”, the captured image signal is cumulatively added to the cumulative addition circuit 130, and the microcomputer 160 then sets the CC value to the maximum number of captures set in advance. It is determined whether or not the value is equal to or greater than a value (hereinafter referred to as CCmax) (ST110 → ST120).

  Here, when the CC value is equal to or greater than the maximum value CCmax (CC ≧ CCmax), the process proceeds to processing for recording the image signal in the recording unit 150 (ST120 → ST130 →...).

  On the other hand, when the CC value is smaller than the maximum value CCmax (CC <CCmax), the microcomputer 160 determines whether or not the TC value is equal to or longer than a predetermined set time (hereinafter, TCmax) (ST120 → ST121).

  Here, when the value of TC is equal to or longer than the set time TCmax (TC ≧ TCmax), the process proceeds to processing for recording the image signal on the recording unit 150 (ST121 → ST130 →...).

  On the other hand, when the value of TC is smaller than the set time TCmax (TC <TCmax), the microcomputer 160 calculates a camera shake correction amount and correction vector data based on a camera shake signal from the camera shake detection unit 170, or between pixels. An interpolation coefficient value for performing an interpolation process for correcting camera shake is calculated and sent to the correction circuit 120, and the correction circuit 120 sends it from the imaging unit 110 based on the correction vector data and the interpolation coefficient value. Image signal interpolation processing and camera shake correction are performed (ST121 → ST122).

  Subsequently, the microcomputer 160 determines whether or not the interpolation coefficient value calculated for performing the inter-pixel interpolation processing is within a predetermined range (ST122 → ST123).

  Here, the determination of whether or not the correction coefficient value is within a predetermined range is, for example, “a” for the value (vector component) below the decimal point in the absolute value of the correction vector data, and “0” for the reference value of the correction coefficient. .5 ”and the predetermined inter-pixel distance is“ b ”, the calculated interpolation coefficient values are“ 0 ≦ a <(0.5−b) ”and“ (0.5 + b) <a <1 ”. This is done by determining whether or not the range satisfies the condition. At this time, the resolution deterioration is reduced as “b” increases.

  When the interpolation coefficient value is within a predetermined range, the cumulative addition circuit 130 cumulatively adds the image signals from the correction circuit 120 and stores them in the memory (ST123 → ST112).

  Next, the microcomputer 160 increments the value of CC by one, further increments the value of a time counter (hereinafter referred to as TC) by one, and ends the process (ST113, ST114 → end).

  On the other hand, when the interpolation coefficient value exceeds the predetermined range, the value of TC is incremented by 1 and the process is terminated (ST123 → ST114 → End).

  Further, when the CC value is equal to or greater than CCmax (CC ≧ CCmax) in step 120 described above, or when the TC value is equal to or greater than TCmax (TC ≧ TCmax) in step 121 described above, the value is stored in the cumulative addition circuit 130. Then, processing for recording the image signal in the recording unit 150 is started.

  First, the microcomputer 160 gives the inverse of the CC value to the multiplier 140 as a predetermined coefficient value (ST120 → ST130 or ST121 → ST130).

  Next, the multiplier 140 multiplies the predetermined coefficient value given from the microcomputer 160 and the image signal cumulatively added to the cumulative addition circuit 130 and outputs the result to the recording unit 150. The recording unit 150 is a multiplier. The image signal output from 140 is recorded on a predetermined recording medium (recording apparatus) (ST131).

  When the image signal is recorded in the recording unit 150, the microcomputer 160 clears the CC value, then clears the TC value, and ends the process (ST132 → ST133 → end).

  FIG. 6 is a graph schematically showing a relationship between correction vector data calculated based on a camera shake signal and an image signal to be cumulatively added in the imaging apparatus operating according to the operation flow of FIG. 5 described above.

  In the graph of FIG. 6, the image signal when the interpolation coefficient value is within a predetermined range corresponds to the solid line, and the other image signals correspond to the dotted line, and all the corrected image signals as shown in FIG. 7 are cumulatively added. Without image stabilization, only image signals whose interpolation coefficient values are within a predetermined range among image signals that have undergone camera shake correction, that is, image signals whose resolution has not deteriorated due to interpolation processing, are cumulatively added and stored in the memory. Become.

It is the figure which showed the block diagram which simplified and showed the main structural part in the imaging device which concerns on this invention. It is explanatory drawing for demonstrating operation | movement of the camera shake detection part in the imaging device shown in FIG. It is explanatory drawing for demonstrating a linear interpolation process. It is explanatory drawing for demonstrating the relationship between the interpolation factor in a linear interpolation process, and a gain (signal level). 3 is an operation flowchart of the imaging apparatus shown in FIG. 1. FIG. 2 is an explanatory diagram for explaining a relationship between correction vector data for an image signal and an image signal to be cumulatively added in the imaging apparatus shown in FIG. 1. In the conventional imaging device, it is explanatory drawing for demonstrating the relationship between the correction vector data with respect to an image signal, and the image signal which carries out cumulative addition.

Explanation of symbols

110; Imaging unit 120; Correction circuit 130; Cumulative addition circuit 131; Adder 132; Memory 133; Switch 140; Multiplier 150; Recording unit 160;

Claims (3)

An image pickup apparatus comprising means for accumulating and storing image signals taken by an image pickup unit in a memory,
Camera shake detection means for detecting a vibration amount due to camera shake of the imaging unit and outputting a camera shake signal corresponding to the detected camera shake vibration amount;
A camera shake correction amount calculating unit that calculates a camera shake correction amount for correcting the image signal based on the camera shake signal output from the camera shake detecting unit;
Interpolation coefficient calculating means for calculating an interpolation coefficient value for performing interpolation processing between pixels of the image signal captured by the imaging unit based on the camera shake correction amount calculated by the camera shake correction amount calculating means;
Based on the camera shake correction amount calculated by the camera shake correction amount calculation unit and the interpolation coefficient value calculated by the interpolation coefficient calculation unit, the camera shake correction unit that performs the camera shake correction by performing interpolation processing of the image signal captured by the imaging unit;
An image in which the interpolation coefficient value calculated by the interpolation coefficient calculation means is compared with a predetermined threshold value set in advance, and the image signal may be cumulatively added when the interpolation coefficient value is within a predetermined range. Image discriminating means for performing discriminating processing of an image signal for accumulating and adding only the images that may be cumulatively added to the memory after discriminating that there is an image that may or may not be cumulatively added;
Cumulative addition control means for controlling the cumulative addition processing of the image signal corrected by the camera shake correction means according to the image discrimination means;
An image pickup apparatus comprising: An image pickup apparatus comprising means for accumulating and storing image signals taken by an image pickup unit in a memory,
Camera shake detection means for detecting a vibration amount due to camera shake of the imaging unit and outputting a camera shake signal corresponding to the detected camera shake vibration amount;
A camera shake correction amount calculating unit that calculates a camera shake correction amount for correcting the image signal based on the camera shake signal output from the camera shake detecting unit;
Interpolation coefficient calculating means for calculating an interpolation coefficient value for performing interpolation processing between pixels of the image signal captured by the imaging unit based on the camera shake correction amount calculated by the camera shake correction amount calculating means;
Based on the camera shake correction amount calculated by the camera shake correction amount calculation unit and the interpolation coefficient value calculated by the interpolation coefficient calculation unit, the camera shake correction unit that performs the camera shake correction by performing interpolation processing of the image signal captured by the imaging unit;
An image in which the interpolation coefficient value calculated by the interpolation coefficient calculation means is compared with a predetermined threshold value set in advance, and the image signal may be cumulatively added when the interpolation coefficient value is within a predetermined range. Image discriminating means for performing discriminating processing of an image signal for accumulating and adding only the images that may be cumulatively added to the memory after discriminating that there is an image that may or may not be cumulatively added;
Cumulative addition control means for controlling the cumulative addition processing of the image signal corrected by the camera shake correction means according to the image discrimination means;
Image output means for multiplying and outputting a coefficient value calculated according to the number of times of cumulative addition processing by the cumulative addition control means and an image signal accumulated and stored in the memory; and
An image pickup apparatus comprising: An image pickup apparatus comprising means for accumulating and storing image signals taken by an image pickup unit in a memory,
A camera shake detection step of detecting a vibration amount due to camera shake of the imaging unit and outputting a camera shake signal corresponding to the detected camera shake vibration amount;
A camera shake correction amount calculating step for calculating a camera shake correction amount for correcting the image signal based on the camera shake signal output in the camera shake detection step;
An interpolation coefficient calculation step for calculating an interpolation coefficient value for performing interpolation processing between pixels of the image signal captured by the imaging unit based on the camera shake correction amount calculated in the camera shake correction amount calculation step;
Based on the camera shake correction amount calculated in the camera shake correction amount calculation step and the interpolation coefficient value calculated in the interpolation coefficient calculation step, a camera shake correction step for performing camera shake correction by performing interpolation processing of the image signal captured by the imaging unit,
An image in which the interpolation coefficient value calculated in the interpolation coefficient calculation step is compared with a predetermined threshold value set in advance, and the image signal may be cumulatively added when the interpolation coefficient value is within a predetermined range. An image determination step for performing an image signal determination process in which only an image that may be cumulatively added is cumulatively added to the memory after determining that the image is not an image that may be cumulatively added;
A cumulative addition control step for controlling a cumulative addition process of the image signal corrected in the camera shake correction step according to the image determination step;
An imaging method comprising:

Images