JP2005203861A - Imaging apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an imaging apparatus capable of calculating a camera-shake correction amount, without being affected by the status of an object and the state of an optical system and realizing camera-shake correcting function, needing only a small processing load with a small memory capacity. <P>SOLUTION: The imaging apparatus is configured to include a first camera-shake detection means for detecting camera-shake vibration of an imaging section as a camera-shake signal; a second camera-shake detecting means for detecting motion vector data from an image signal imaged by the imaging section; a gain correction coefficient calculating means for calculating the gain correction coefficient value for correcting gain data for adjusting a correction amount of the camera-shake correction processing, on the basis of the first and second camera-shake amount data; a gain data generating means for generating the gain data on the basis of the gain correction coefficient value; a gain setting changeover means, capable of switching the gain setting to a prescribed setting state, in response to the discrimination of a gain discriminating means; and a camera-shake correction means for executing the camera-shake correction processing of the image signal, on the basis of the gain data generated by the gain data generating means. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to an imaging apparatus. Specifically, the present invention relates to an imaging apparatus having a function of correcting camera shake by comparing the amount of camera shake detected by a sensor with the amount of camera shake detected based on an image signal.

  In recent years, with the downsizing and higher magnification of imaging devices such as video cameras, image blurring and blurring due to the effects of camera shake have become prominent, and in order to solve this problem, imaging devices equipped with a camera shake correction function have been developed. It is getting more.

  There are two methods for detecting the amount of vibration due to camera shake (hereinafter referred to as the amount of camera shake): a method in which the vibration of the imaging unit (device itself) due to camera shake is detected by a sensor (hereinafter referred to as sensor method), and camera shake based on the captured image. There is a method for detecting the quantity (hereinafter referred to as motion vector method).

  In the sensor system, a detector such as an angular velocity sensor that applies Coriolis force is incorporated into the device, and the amount of vibration of the imaging unit (device itself) due to camera shake is detected. Since it is detected in real time, there is a feature that it is not affected by the state of the subject (type of subject, brightness, movement) when detecting the amount of camera shake.

  On the other hand, in the motion vector method, an image of a subject (light signal) projected through an imaging lens is detected by an imaging device, stored as an image signal in a memory, and compared with an image signal to be detected next. Since image blur is detected as motion vector data, that is, the difference between the two most recent image signals is detected as motion vector data, it affects the state of the optical system such as changes in shooting magnification when detecting the amount of camera shake. There is a feature of not receiving.

Many methods and apparatuses for correcting camera shake based on the above-described sensor and the amount of camera shake detected from a captured image signal have been devised.For example, the sensitivity of a shake detector that detects camera shake and the position of a correction optical system are determined. In order to prevent the sensitivity of the position detection sensor to be detected from lowering due to changes in the ambient temperature and blurring the image to be taken and preventing the desired image blur prevention effect from being obtained, the reduction in sensitivity due to temperature changes is a variable gain amplifier. By making it possible to change the amplification factor from the outside, a device has been devised that can compensate for a decrease in vibration isolation sensitivity while looking through the viewfinder and obtain a desired image blur prevention effect (for example, Patent Document 1).
Japanese Patent No. 3131431 (page 2-3, Fig. 1)

  In addition, the optical system such as the imaging lens has been miniaturized due to the downsizing of the imaging device, and the range of viewable angles (magnification range) has been increasing, so that wide-angle images or farther images can be captured. Opportunities for attaching a conversion lens for converting the magnification of the imaging lens for photographing are increasing.

  However, in an imaging device that corrects camera shake based on the amount of camera shake detected by the above-described sensor (hereinafter referred to as a sensor-type imaging device), the magnification information changes when a conversion lens or the like that converts the imaging magnification is attached. The correspondence between the detected amount of camera shake and the amount of correction (gain data) indicating how much to correct for this amount of camera shake may change, and camera shake correction may not be performed correctly.

  Further, in the case of an image pickup apparatus that corrects camera shake based on the amount of camera shake detected by the motion vector method (hereinafter referred to as a motion vector type image pickup apparatus), the image signal detected by the image pickup device is stored in a memory and then detected. Compared with the image signal, the difference between the two image signals is detected as image blur and image blur correction is performed, so that a memory capacity corresponding to the size of the image to be captured is required. In addition, since motion vector data (image blur) is detected after comparing the two most recent image signals, there is a problem that camera shake vibration cannot be detected in real time.

  Therefore, there is a method for predicting and correcting the next camera shake amount from the camera shake amount detected before one sampling (for example, one field before the video camera) without using a memory for storing the image signal. There is a problem that the processing load becomes large, and a memory for calculation is required for predicting with high accuracy, and a predicted value different from the actual amount of camera shake may be calculated.

  In addition, in the case of the correction method based on the predicted value, the calculation processing load increases as the image size to be captured increases, and the memory capacity for the calculation processing needs to be increased accordingly, and the circuit scale, cost, and power consumption are increased. The demerit of increasing will arise.

  Furthermore, it is difficult for a motion vector imaging device to detect the amount of camera shake from a low-contrast subject, a subject having a repetitive pattern, a subject from which motion vector data diverges (such as a subject during zoom operation), and Since the detection cycle of motion vector data is the same as the cycle in which an image signal is captured by the image sensor (hereinafter referred to as exposure cycle), there is a problem that camera shake at a frequency higher than the exposure cycle cannot be detected.

  Therefore, in the imaging apparatus, it is possible to calculate the camera shake correction amount without being affected by the subject state or the optical system state, and to solve the problem of realizing a camera shake correction function with a small memory capacity and a small processing load. Has challenges that must be met.

  In order to solve the above-described problems, an imaging apparatus according to the present invention is configured as follows.

(1) A first camera shake detection unit that detects camera shake vibration of the imaging unit as a camera shake signal, a second camera shake detection unit that detects motion vector data from the image signal captured by the imaging unit, and the camera shake detection unit Camera shake amount data generating means for generating first camera shake amount data based on the detected camera shake signal, and generating second camera shake amount data based on the motion vector data detected by the second camera shake detecting means; Gain correction for calculating a gain correction coefficient value for correcting the gain data for adjusting the correction amount of the camera shake correction processing based on the first camera shake amount data and the second camera shake amount data generated by the camera shake amount data generating means Based on the coefficient calculation means and the gain correction coefficient value calculated by the gain correction coefficient calculation means, the gain correction coefficient value and gain set in the apparatus Gain determining means for determining a fixed setting state, and gain data generating means for generating the gain data based on the gain correction coefficient value calculated by the gain correction coefficient calculating means in accordance with the determination of the gain determining means; A gain setting switching unit capable of switching the gain setting to a predetermined setting state in accordance with the determination of the gain determining unit; and image stabilization of the image signal based on the gain data generated by the gain data generating unit An image pickup apparatus comprising: an image stabilization unit that executes processing.
(2) The image stabilization unit controls a control signal for controlling the timing of storing the image signal in the image memory of the imaging unit and a read address of the image signal captured in the image memory based on the gain data. The imaging device according to (1), wherein a control signal for generating the control signal is generated.
(3) The camera shake correction unit generates a control signal for controlling driving of a correction lens that corrects a deviation of the optical axis of the imaging unit, based on the gain data. The imaging device described.
(4) Furthermore, a motion vector determination unit that determines the reliability of the motion vector data detected by the second camera shake detection unit based on an image signal captured by the imaging unit, and a motion amount data generation unit that generates the motion vector data. The first camera shake amount data and the second camera shake amount data are compared to determine whether or not the first camera shake detection unit and the second camera shake detection unit detect the same type of camera shake vibration. (1) The gain correction coefficient calculation means calculates the gain correction coefficient value based on the determinations of the motion vector determination means and the camera shake determination means. Imaging device.

  In the imaging apparatus having such a configuration, the first image blur amount data generated based on the camera shake signal detected by the first camera shake detection unit and the motion vector data detected by the second camera shake detection unit are generated. Based on the second camera shake amount data, a gain correction coefficient value for correcting gain data for adjusting the correction amount of the camera shake correction process is calculated, and based on the gain correction coefficient value, a gain correction factor set in the apparatus is calculated. Determine the setting status of numerical values and gain settings.

  Then, according to the determination result, the gain setting is switched to an appropriate setting, or new gain data is calculated (selected) based on the gain correction coefficient value calculated from the first camera shake amount data and the second camera shake amount data. Then, the camera shake correction processing of the image signal is executed based on the gain data.

  The imaging apparatus according to the present invention includes a first camera shake amount data generated based on a camera shake signal detected by a camera shake detection sensor, and a second camera generated based on motion vector data detected by a motion vector detection unit. By calculating the gain correction coefficient value based on the camera shake amount data, a gain correction coefficient value that is not influenced by the state of the optical system, such as the state of the subject (subject type, brightness, movement) and the change in the photographing magnification, is obtained. It becomes possible.

  Then, by generating gain data that adjusts the amount of camera shake correction based on the gain correction coefficient value, it is possible to execute the optimum camera shake correction processing, and setting the gain setting based on the gain correction coefficient value. There is an excellent effect that the state can be appropriately determined.

  In addition, since the camera shake correction processing is not performed by the motion vector data detected by the motion vector detection unit, it is not necessary to store all of the image signals captured as in the conventional motion vector imaging device in the memory. A small-capacity memory sufficient to store the amount of change in one image signal is sufficient, which makes it possible to simplify the circuit configuration and reduce costs, reduce the processing load, and reduce power consumption. There is also a merit.

  Next, an embodiment of the imaging apparatus of the present invention will be described with reference to the drawings. However, the drawings are only for explanation, and do not limit the technical scope of the present invention.

  First, an outline of the operation of the imaging apparatus of the present invention will be described with reference to FIGS.

  FIG. 1 is a block diagram illustrating simplified main parts of an imaging apparatus according to the present invention. The imaging unit 10, a camera shake detection sensor 20, a motion vector detection unit 30, a signal processing unit 40, a control microcomputer 50, and camera shake correction. The processing unit 60 and the like are provided, and processing is executed according to the flowchart of FIG.

  First, the control microcomputer 50 generates camera shake vibration data S1 (vector data) generated based on a vibration amount (hereinafter referred to as a camera shake signal) of the imaging unit 10 (device itself) due to camera shake detected by the camera shake detection sensor 20, and a motion vector. The camera shake vibration data S2 (vector data) generated based on the motion vector data detected by the detection unit 30 is compared and evaluated (ST10).

  Specifically, the motion vector detection unit is determined by determining the brightness level of the image signal captured by the imaging unit 10 and sent via the signal processing unit 40, the in-focus state, whether the zoom operation is being performed, or the like. 30. The reliability of the motion vector data detected at 30 is evaluated, and the camera shake detection sensor 20 and the motion vector detection unit 30 detect the same camera shake vibration by determining the vector directions of the camera shake vibration data S1 and the camera shake vibration data S2. Evaluate whether or not

  Next, the control microcomputer 50 is based on the camera shake vibration data S1 and the camera shake vibration data S2 when the motion vector data is reliable and the camera shake detection sensor 20 and the motion vector detection unit 30 detect the same camera shake vibration. The gain correction coefficient value R is calculated (ST20).

  Next, the control microcomputer 50 determines whether or not the gain data needs to be adjusted based on the calculated gain correction coefficient value R, and determines whether or not an appropriate gain setting has been made ( ST30).

  The gain data is data for adjusting the correction amount of the camera shake correction process. The gain G corresponds to the focal length (zoom) of the imaging unit 10 and the magnification when the conversion lens is attached to the imaging unit 10. The corresponding magnification gain Gzm, the sensitivity gain Gs corresponding to the sensitivity of the camera shake detection sensor 20, and the like.

  When the control microcomputer 50 determines that the gain data needs to be adjusted, the control microcomputer 50 calculates (or selects) the gain data based on the gain correction coefficient value R and sends it to the camera shake correction processing unit 60. For example, when it is determined that the magnification gain has been set when the conversion lens is not attached, for example, when the gain is set automatically, a message (warning, etc.) is displayed. To change the setting (ST40).

  Subsequently, the camera shake correction processing unit 60 generates a control signal based on the gain data sent from the control microcomputer 50, and based on this control signal, the image signal capturing timing and the image by the imaging device of the imaging unit 10 are generated. The signal cut-out range (address) is controlled, or the optical system of the imaging unit 10 is controlled. The image signal sent from the signal processing unit 40 is subjected to a predetermined camera shake correction process and sent to the next stage circuit (ST50).

  In this manner, the camera shake amount data S1 based on the camera shake signal detected by the camera shake detection sensor 20 and the camera shake amount data S2 based on the motion vector data detected by the motion vector detection unit 30 are evaluated. Based on the gain correction coefficient value R calculated from the camera shake amount data S1 and the camera shake amount data S2, the necessity of gain adjustment and the state of gain setting are determined, and appropriate gain data is calculated. Optimal camera shake correction is performed by executing the correction process.

[First embodiment]
As a first embodiment, a case will be described in which the processing described in FIG. 2 is executed by an imaging apparatus that electronically corrects camera shake.

  FIG. 3 is a block diagram illustrating, in a simplified manner, main parts related to the present invention in an imaging device that electronically corrects camera shake, and includes an imaging unit 10, a TG (timing generator) unit 10A, a camera shake detection sensor 20, and motion. A vector detection unit 30, a signal processing unit 40, a control microcomputer 50, a camera shake correction processing unit 60, and the like are provided.

  The imaging unit 10 includes an imaging lens, an imaging element, and the like. The imaging unit 10 detects an image of a subject (light signal) projected through the imaging lens by an imaging element such as a CCD (Charge Coupled Device), and an electrical signal (hereinafter referred to as an “electrical signal”). , An image signal) and sent to the motion vector detection unit 30.

  The TG (timing generator) unit 10 </ b> A controls the capture timing of the image signal by the imaging device of the imaging unit 10 in accordance with the control signal from the camera shake correction processing unit 60.

  The camera shake detection sensor 20 is, for example, a sensor such as an angular velocity sensor, detects the vibration amount of the imaging unit (device itself) due to camera shake, and converts the detected vibration amount into an electric signal (hereinafter referred to as a camera shake signal) for control. The data is sent to the microcomputer 50 (A / D converter 51).

  The motion vector detection unit 30 detects the motion vector data of the image from the image signal sent from the imaging unit 10 and sends it to the control microcomputer 50 (the comparison / discrimination unit 56).

  In addition, since the motion vector data to be detected is not directly used for the camera shake correction process, the motion vector detection unit 30 does not need a memory for storing all of the captured image signals, and detects the amount of change between the two most recent image signals. For example, a memory having a capacity necessary to detect and store the amount of change in the vertical direction (angle) with respect to the lens optical axis and the amount of change in the horizontal direction (angle) with respect to the lens optical axis.

  The signal processing unit 40 converts an image signal detected by the image pickup device of the image pickup unit 10 into a digital signal, loads the image signal into an image memory, performs predetermined signal processing on the image signal (digital signal) captured in the image memory, and performs control. The data is sent to the microcomputer 50 and the camera shake correction processing unit 60.

  The control microcomputer 50 is a microcomputer for controlling each unit in the image pickup apparatus. As a configuration related to the present invention, an A / D conversion unit 51, a noise removal unit 52, an integrator 53, a gain adjustment unit 54, a delay, and the like. The memory 55, the comparison / discrimination unit 56, and the like are provided.

  The A / D converter 51 of the control microcomputer 50 converts the camera shake signal detected by the camera shake detection sensor 20 into a digital signal and sends the digital signal to the noise removing unit 52.

  The noise removal unit 52 of the control microcomputer 50 converts the noise component of the camera shake signal (digital signal) sent from the A / D conversion unit 51 into HPF (High-Pass Filter) / LPF (Low-Pass Filter) / BPF ( (Band-Pass Filter) and the like are sent to the integrator 53.

  The integrator 53 of the control microcomputer 50 generates camera shake amount data S1 (vector data) by integrating the camera shake signal (digital signal) sent from the noise removing unit 52, and sends it to the gain adjusting unit 54. The integrator is used when the camera shake detection sensor 20 is an angular velocity sensor. When a sensor other than the angular velocity sensor is used, the integration calculation is not performed and the camera shake signal (digital signal) is used as the amount of camera shake. Data S1 (vector data) is converted and sent to the gain adjustment unit 54.

  The gain adjustment unit 54 of the control microcomputer 50 includes a gain G corresponding to the focal length (zoom) of the imaging unit 10, a magnification gain Gzm corresponding to the magnification when the conversion lens is attached to the imaging unit 10, and the camera shake detection sensor 20. A function of adjusting gain data such as a sensitivity gain Gs according to sensitivity is provided, and a gain correction coefficient value R sent from a comparison / discrimination unit 56 described later is set. Based on this gain correction coefficient value R Optimal gain data is calculated and sent to the camera shake correction processing unit 60. Alternatively, optimal gain data is selected from predetermined table data stored in advance based on the gain correction coefficient value R and sent to the camera shake correction processing unit 60.

  The delay memory 55 of the control microcomputer 50 is sent via the gain adjustment unit 54 until motion vector data is detected by the motion vector detection unit 30 and camera shake amount data S2 is generated from the motion vector data. The incoming camera shake amount data S1 is temporarily stored. When the camera shake amount data S2 is generated, the stored camera shake amount data S1 is sent to the comparison / determination unit 56.

  The comparison / discrimination unit 56 of the control microcomputer 50 includes the camera shake amount data S2 generated based on the motion vector data detected by the motion vector detection unit 30, and the camera shake amount data S1 sent from the delay memory 55. To determine whether or not to change the gain correction coefficient value R set in the gain adjustment unit 54, and calculates the gain correction coefficient value R based on the camera shake amount data S1 and the camera shake amount data S2. To the gain adjustment unit 54.

  Further, the control microcomputer 50 is the image signal (digital signal) sent from the signal processing unit 40, that is, the luminance level of the same image signal as the image signal when the motion vector detection unit 30 detects the motion vector data, The amount of camera shake calculated based on motion vector data by determining the reliability of motion vector data (whether it is correctly detected) by checking the in-focus state, zooming operation, etc. A function of determining the reliability of the data S2 is provided.

  Further, the control microcomputer 50 checks whether or not the vector directions of the camera shake amount data S1 and the camera shake amount data S2 coincide with each other, and detects the camera shake signal detected by the camera shake detection sensor 20 and the motion vector detection unit 30. A function is provided for determining whether the motion vector data is based on the same camera shake vibration.

  As described above, the control microcomputer 50 having the above configuration generates the camera shake vibration data S1 (vector data) based on the camera shake signal detected by the camera shake detection sensor 20, and the motion vector data detected by the motion vector detecting unit 30. Based on the above, hand vibration data S2 (vector data) is generated.

  Then, the hand vibration vibration data S1 and S2 are compared and evaluated to calculate a gain correction coefficient value R for executing the optimum hand shake correction, and gain data based on the gain correction coefficient value R is generated to correct the camera shake. The data is sent to the processing unit 60.

  The camera shake correction processing unit 60 generates a control signal based on the gain data sent from the control microcomputer 50 and sends it to the TG 10A and the signal processing unit 40, and the image capture timing by the TG 10A and the image memory of the signal processing unit 40 The readout (cutout) range (address) of the image signal taken in is controlled. The image signal sent from the signal processing unit 40 is subjected to a predetermined camera shake correction process and sent to the next stage circuit.

  FIG. 4 is a conceptual diagram schematically showing the relationship between the image signal before / after camera shake correction and the camera shake amount data S1 and the camera shake amount data S2 in the image pickup apparatus of FIG. This corresponds to the distance difference (vector amount) between the camera shake correction target range A0 (dotted line) and the cutout range A1 (solid line) after correction, and the camera shake amount data S2 includes the original image signal a0 (dotted line) and the corrected image signal ( This corresponds to a distance difference (vector amount) of a solid line).

  Next, a specific operation of the imaging apparatus for electronically correcting camera shake shown in FIG. 3 will be described.

  First, a specific operation when evaluating the camera shake vibration data S1 and the camera shake vibration data S2 will be described with reference to the flowchart shown in FIG. 5 (corresponding to the process of step ST20 in FIG. 2).

  When camera shake correction processing is executed by the imaging apparatus, the control microcomputer 50 first turns off the change flag for the gain correction coefficient value R set in the gain adjustment unit 54 as an initial setting, and also includes a built-in counter. Is reset (ST101).

  When the initial setting is completed, the camera shake signal detected by the camera shake detection sensor 20 is sent to the control microcomputer 50, and the image signal captured by the imaging unit 10 is sent to the motion vector detection unit 30 and the signal processing unit 40.

  The control microcomputer 50 converts the camera shake signal sent from the camera shake detection sensor 20 into a digital signal by the A / D converter 51, removes the noise component of the camera shake signal (digital signal) by the noise removing unit 52, and integrates the integrator. 53 generates camera shake data S1 (vector data), and temporarily stores the generated camera shake data S1 (vector data) in the delay memory 55 (ST102).

  In addition, while executing the processing related to the above-described camera shake amount data S1, the motion vector detection unit 30 detects motion vector data from the image signal captured by the imaging unit 10 and sends the motion vector data to the control microcomputer 50, and the control microcomputer. 50 generates camera-shake amount data S2 (vector data) based on the motion vector data sent from the motion vector detecting unit 30 (ST102).

  The signal processing unit 40 performs predetermined signal processing on the image signal captured by the imaging unit 10 and sends the image signal to the control microcomputer 50.

  Next, in the control microcomputer 50, the image signal (digital signal) sent from the signal processing unit 40, that is, the luminance level of the same image signal as the image signal sent to the motion vector detection unit 30, the in-focus state, Whether or not the zoom operation is being performed is checked, and the reliability of the camera shake vibration amount data S2 generated based on the motion vector data is determined (ST103).

  In the above reliability determination, (1) the luminance level of the image signal is a luminance level suitable for motion vector data detection, (2) the image is in a predetermined focus state, and (3) the zoom operation is not in progress. Only when all the above three conditions are satisfied, it is determined that the reliability of the vibration amount data S2 is “Yes”.

  When it is determined that the reliability of the camera shake vibration amount data S2 is “none”, the control microcomputer 50 resets the counter and newly restarts from the process of step ST102 (ST103 → ST108 → ST102 →...).

  When it is determined that the reliability of the camera shake vibration data S2 is “Yes”, the camera shake data is used to determine whether the camera shake detection sensor 20 and the motion vector detection unit 30 detect the same type of camera shake vibration. The vector direction of S1 and camera shake amount data S2 is determined (ST103 → ST104).

  When the vector directions of the camera shake amount data S1 and the camera shake amount data S2 are different, the control microcomputer 50 resets the counter and newly restarts from the process of step ST102 (ST104 → ST108 → ST102 →...).

  When the vector directions of the camera shake amount data S1 and the camera shake amount data S2 are the same, the control microcomputer 50 counts up the counter value and compares the count value with a predetermined threshold value (ST104 → ST105, ST106).

  If the count value is equal to or smaller than the predetermined threshold value (count value ≦ threshold value), the process returns to step ST102, and detection of the next camera shake amount data S1 and S2 is started (ST106 → ST102 →...).

  When the count value is larger than the predetermined threshold value (count value> threshold value), the control microcomputer 50 determines that the camera shake detection sensor 20 and the motion vector detection unit 30 continuously detect the same type of camera shake vibration for a predetermined time. Then, the change flag of the gain correction coefficient value R set in the gain adjustment unit 54 is turned on (ST106 → ST107).

  When the control microcomputer 50 turns on the gain correction coefficient value R change flag, the control microcomputer 50 shifts to the next process of calculating the gain correction coefficient value R in order to determine whether the gain correction coefficient value R can be changed (ST107 → Next processing).

  As described above, the control microcomputer 50 determines the reliability of the motion vector data within a predetermined time based on the counter value of the counter, and the camera shake detection sensor 20 and the motion vector detection unit 30 further detect the same kind of camera shake vibration. By determining whether or not it is detected, the gain correction coefficient value R is unnecessarily changed by vector data detected by vibrations other than camera shake (for example, motion vector data detected when the subject suddenly moves). Is prevented.

  Next, a specific operation until the camera shake correction process is executed based on the camera shake amount data S1 and the camera shake amount data S2 will be described with reference to the flowchart shown in FIG. 6 (process after step ST20 in FIG. 2).

  First, the control microcomputer 50 confirms whether or not the change flag of the gain correction coefficient value R has been turned on by the process of step ST107 of FIG. 5 described above (ST201).

  When the change flag for the gain correction coefficient value R is off, the process ends without performing the calculation process of the gain correction coefficient value R. (ST201 → end).

  When it is confirmed that the change flag of the gain correction coefficient value R is ON, the comparison / discrimination unit 56 of the control microcomputer 50 and the shake amount data S1 generated based on the shake signal detected by the shake detection sensor 20 and the motion vector A gain correction coefficient value R is calculated from the camera shake amount data S2 generated based on the motion vector data detected by the detection unit 30 by the following equation (ST201 → ST202).

  Next, the comparison / discrimination unit 56 of the control microcomputer 50 determines that the gain correction coefficient value R (hereinafter, gain correction coefficient value Rn) calculated in the process of step ST202 is within a predetermined allowable value range (minimum allowable value Vok_l < It is determined whether or not the gain correction coefficient value Rn <maximum allowable value Vok_h) (ST203).

  The comparison / discrimination unit 56 of the control microcomputer 50 does not need to change the gain correction coefficient value R set in the gain adjustment unit 54 when the gain correction coefficient value Rn is within a predetermined allowable range. Further, it is determined that the gain setting of the apparatus is also appropriate (for example, no conversion lens or the like is attached) (ST203 → ST204).

  The gain adjustment unit 54 of the control microcomputer 50 sends the gain data calculated based on the currently set gain correction coefficient value R to the camera shake correction processing unit 60 based on the determination of the comparison / determination unit 56. Alternatively, optimum gain data based on the currently set gain correction coefficient value R is selected from predetermined table data stored in advance and sent to the camera shake correction processing unit 60.

  The camera shake correction processing unit 60 generates a control signal based on the gain data sent from the control microcomputer 50, controls the signal processing unit 40 and the TG 10A, and shakes the image signal sent from the signal processing unit 40. Correction processing is executed (ST204 → ST210).

  On the other hand, when the gain correction coefficient value Rn is outside the predetermined allowable value in the process of step ST203, the comparison / discrimination unit 56 of the control microcomputer 50 further determines that the gain correction coefficient value Rn is less than or equal to the minimum allowable value. It is determined whether or not there is (minimum allowable value Vok_l ≦ gain correction coefficient value Rn) (ST203 → ST205).

  When the gain correction coefficient value Rn is equal to or less than the minimum allowable value (minimum allowable value Vok_l ≦ gain correction coefficient value Rn), it is necessary to adjust the gain correction coefficient value R currently set in the gain adjustment unit 54. It is determined that the gain setting of the apparatus is too small to be appropriate, and the gain correction coefficient value Rn is sent to the gain adjustment unit 54 (ST205 → ST206).

  The gain adjustment unit 54 sets the gain correction coefficient value Rn from the comparison / discrimination unit 56 as a new gain correction coefficient value, and sends the gain data calculated based on the gain correction coefficient value Rn to the camera shake correction processing unit 60. Alternatively, optimal gain data is selected from predetermined table data stored in advance based on the gain correction coefficient value Rn and sent to the camera shake correction processing unit 60 (ST207).

  Further, the comparison / discrimination unit 56 of the control microcomputer 50 automatically switches the gain setting of the apparatus to an appropriate setting, or issues a message prompting the appropriate setting, a message indicating the possibility of a failure, or the like. It is displayed on the display unit (ST207).

  For example, as shown in FIG. 7, when the correspondence between the zoom position L and the magnification gain Gzm is stored as table data, a message indicating that the wide-angle conversion lens is mounted based on this table data is displayed on the display unit. It is also possible to detect a failure of the camera shake detection sensor 20 or the motion vector detection unit 30 and display an alarm by displaying or by setting a predetermined failure determination value and comparing it with the gain correction coefficient value Rn.

  The camera shake correction processing unit 60 generates a control signal based on the gain data sent from the control microcomputer 50, controls the signal processing unit 40 and the TG 10A, and receives the image signal sent from the signal processing unit 40. The camera shake correction process is executed (ST207 → ST210).

  In the process of step ST205, when the gain correction coefficient value Rn is larger than the minimum allowable value (minimum allowable value Vok_l> gain correction coefficient value Rn), the gain correction coefficient value Rn is larger than the maximum allowable value Vok_h (maximum allowable value). It is determined that the value Vok_h <gain correction coefficient value Rn), and that the gain setting of the apparatus is too large to be appropriate, and sends the gain correction coefficient value Rn to the gain adjustment unit 54 (ST205 → ST208).

  The gain adjustment unit 54 sets the gain correction coefficient value Rn from the comparison / discrimination unit 56 as a new gain correction coefficient value, and sends the gain data calculated based on the gain correction coefficient value Rn to the camera shake correction processing unit 60. Alternatively, optimal gain data is selected from predetermined table data stored in advance based on the gain correction coefficient value Rn and sent to the camera shake correction processing unit 60 (ST209).

  Further, the comparison / discrimination unit 56 of the control microcomputer 50 automatically switches the gain setting of the apparatus to an appropriate setting, or issues a message prompting the appropriate setting, a message indicating the possibility of a failure, or the like. It is displayed on the display unit (ST209).

  For example, as shown in FIG. 7, when the correspondence between the zoom position L and the magnification gain Gzm is stored as table data, a message indicating that a narrow-angle conversion lens is mounted based on this table data is displayed on the display unit. Or by setting a predetermined failure determination value and comparing it with the gain correction coefficient value Rn, it is possible to detect a failure of the camera shake detection sensor 20 or the motion vector detection unit 30 and display an alarm.

  The camera shake correction processing unit 60 generates a control signal based on the gain data sent from the control microcomputer 50, controls the signal processing unit 40 and the TG 10A, and receives the image signal sent from the signal processing unit 40. Is executed (ST209 → ST210).

  As described above, in the image pickup apparatus according to the first embodiment, the control microcomputer 50 generates the camera shake amount data S1 generated based on the camera shake signal detected by the camera shake detection sensor 20, and the motion vector detected by the motion vector detection unit 30. A gain for calculating a correction coefficient value Rn based on the camera shake amount data S2 generated based on the data, and comparing and determining a currently set gain correction coefficient value to adjust the correction amount of the camera shake correction processing Generate data.

  Then, the gain setting is switched to an appropriate setting state, or a control signal is generated based on the gain data and is sent to the TG 10A and the signal processing unit 40, and is captured into the image memory of the TG 10A and the image memory of the signal processing unit 40. The image signal readout (cutout) range (address) is controlled, the image signal sent from the signal processing unit 40 is subjected to predetermined camera shake correction processing, and sent to the next-stage circuit.

[Second Embodiment]
Next, as a second embodiment, a case will be described in which the processing described in FIG. 2 is executed by an imaging apparatus that optically corrects camera shake.

  FIG. 8 is a block diagram illustrating, in a simplified manner, main portions related to the present invention in an imaging apparatus that optically corrects camera shake, and includes an imaging unit 10, a servo mechanism unit 10B, a camera shake detection sensor 20, and a motion vector detection unit. 30, a signal processing unit 40, a control microcomputer 50, a camera shake correction processing unit 60, and the like.

  The imaging unit 10 includes an imaging lens, an optical correction lens that moves according to the control of the servo mechanism unit 10B, an imaging element, and the like. The image of the subject (light signal) projected through the imaging lens and the optical correction lens is obtained by the CCD. It is detected by an image sensor such as (Charge Coupled Device), converted into an electric signal (hereinafter referred to as an image signal), and sent to the motion vector detector 30.

  It should be noted that a deviation of the lens optical axis due to camera shake vibration may be optically corrected by attaching a variable apex angle prism, a rotation / translation (eccentric) lens, or the like to a part of the optical correction lens.

  The servo mechanism unit 10B optically corrects the deviation of the lens optical axis due to camera shake vibration by moving the optical correction lens of the imaging unit 10 in accordance with a control signal from the camera shake correction processing unit 60.

  The camera shake detection sensor 20 is, for example, a sensor such as an angular velocity sensor, detects the vibration amount of the imaging unit (device itself) due to camera shake, and converts it into an electrical signal (hereinafter referred to as a camera shake signal) corresponding to the detected vibration amount. The data is sent to the control microcomputer 50 (the A / D converter 51).

  The motion vector detection unit 30 detects the motion vector data of the image from the image signal sent from the imaging unit 10 and sends it to the control microcomputer 50 (the comparison / discrimination unit 56).

  In addition, since the motion vector data to be detected is not directly used for the camera shake correction process, the motion vector detection unit 30 does not need a memory for storing all of the captured image signals, and detects the amount of change between the two most recent image signals. For example, a memory having a capacity necessary to detect and store the amount of change in the vertical direction (angle) with respect to the lens optical axis and the amount of change in the horizontal direction (angle) with respect to the lens optical axis.

  The signal processing unit 40 converts an image signal detected by the image pickup device of the image pickup unit 10 into a digital signal, loads the image signal into an image memory, performs predetermined signal processing on the image signal (digital signal) captured in the image memory, and performs control. The data is sent to the microcomputer 50 and the camera shake correction processing unit 60.

  The control microcomputer 50 is a microcomputer for controlling each unit in the image pickup apparatus. As a configuration related to the present invention, an A / D conversion unit 51, a noise removal unit 52, an integrator 53, a gain adjustment unit 54, a delay, and the like. The memory 55, the comparison / discrimination unit 56, and the like are provided.

  The A / D converter 51 of the control microcomputer 50 converts the camera shake signal detected by the camera shake detection sensor 20 into a digital signal and sends the digital signal to the noise removing unit 52.

  The noise removal unit 52 of the control microcomputer 50 converts the noise component of the camera shake signal (digital signal) sent from the A / D conversion unit 51 into HPF (High-Pass Filter) / LPF (Low-Pass Filter) / BPF ( (Band-Pass Filter) and the like are sent to the integrator 53.

  The integrator 53 of the control microcomputer 50 generates camera shake amount data S1 (vector data) by integrating the camera shake signal (digital signal) sent from the noise removing unit 52, and sends it to the gain adjusting unit 54. The integrator is used when the camera shake detection sensor 20 is an angular velocity sensor. When a sensor other than the angular velocity sensor is used, the integration calculation is not performed and the camera shake signal (digital signal) is used as the amount of camera shake. Data S1 (vector data) is converted and sent to the gain adjustment unit 54.

  The gain adjustment unit 54 of the control microcomputer 50 includes a gain G corresponding to the focal length (zoom) of the imaging unit 10, a magnification gain Gzm corresponding to the magnification when the conversion lens is attached to the imaging unit 10, and the camera shake detection sensor 20. A function of adjusting gain data such as a sensitivity gain Gs according to sensitivity is provided, and a gain correction coefficient value R sent from a comparison / discrimination unit 56 described later is set. Based on this gain correction coefficient value R Optimal gain data is calculated and sent to the camera shake correction processing unit 60. Alternatively, optimal gain data is selected from predetermined table data stored in advance based on the gain correction coefficient value R and sent to the camera shake correction processing unit 60.

  The delay memory 55 of the control microcomputer 50 is sent via the gain adjustment unit 54 until motion vector data is detected by the motion vector detection unit 30 and camera shake amount data S2 is generated from the motion vector data. The incoming camera shake amount data S1 is temporarily stored. When the camera shake amount data S2 is generated, the stored camera shake amount data S1 is sent to the comparison / determination unit 56.

  The comparison / discrimination unit 56 of the control microcomputer 50 includes the camera shake amount data S2 generated based on the motion vector data detected by the motion vector detection unit 30, and the camera shake amount data S1 sent from the delay memory 55. To determine whether or not to change the gain correction coefficient value R set in the gain adjustment unit 54, and calculates the gain correction coefficient value R based on the camera shake amount data S1 and the camera shake amount data S2. To the gain adjustment unit 54.

  Further, the control microcomputer 50 is the image signal (digital signal) sent from the signal processing unit 40, that is, the luminance level of the same image signal as the image signal when the motion vector detection unit 30 detects the motion vector data, The amount of camera shake calculated based on motion vector data by determining the reliability of motion vector data (whether it is correctly detected) by checking the in-focus state, zooming operation, etc. A function of determining the reliability of the data S2 is provided.

  Further, the control microcomputer 50 checks whether or not the vector directions of the camera shake amount data S1 and the camera shake amount data S2 coincide with each other, and detects the camera shake signal detected by the camera shake detection sensor 20 and the motion vector detection unit 30. A function is provided for determining whether the motion vector data is based on the same camera shake vibration.

  As described above, the control microcomputer 50 having the above configuration generates the camera shake vibration data S1 (vector data) based on the camera shake signal detected by the camera shake detection sensor 20, and the motion vector data detected by the motion vector detecting unit 30. Based on the above, hand vibration data S2 (vector data) is generated.

  Then, the hand vibration vibration data S1 and S2 are compared and evaluated to calculate a gain correction coefficient value R for executing the optimum hand shake correction, and gain data based on the gain correction coefficient value R is generated to correct the camera shake. The data is sent to the processing unit 60.

  The camera shake correction processing unit 60 generates a control signal based on the gain data sent from the control microcomputer 50 and sends the control signal to the servo mechanism unit 10B to control the operation of the optical correction lens of the image pickup unit 10 so as to shake the camera shake. The optical axis correction due to the lens optical axis is corrected. The image signal sent from the signal processing unit 40 is subjected to a predetermined camera shake correction process and sent to the next stage circuit.

  FIG. 9 is a conceptual diagram schematically showing the relationship between the image signal before / after camera shake correction and the camera shake amount data S1 and the camera shake amount data S2 in the image pickup apparatus of FIG. This corresponds to the distance difference (vector amount) between the original image signal b0 (two-dot chain line) and the optically corrected image signal b1 (dotted line), and the camera shake amount data S2 is the optically corrected image signal b1 (dotted line) and motion vector data. Corresponds to the distance difference (vector amount) of the image signal b2 (solid line) corrected based on the above.

  Next, a specific operation of the imaging apparatus that performs optical camera shake correction illustrated in FIG. 8 will be described.

  First, a process for evaluating the camera shake vibration data S1 and the camera shake vibration data S2 corresponding to the process of step ST20 of FIG. 2 is executed, but the specific operation is the same as the flowchart of FIG. 5 described above. The description is omitted.

  Next, specific operations until the camera shake correction process is executed based on the camera shake amount data S1 and the camera shake amount data S2, which are the processes after step ST20 in FIG. 2, will be described with reference to the flowchart shown in FIG.

  The control microcomputer 50 checks whether or not the change flag of the gain correction coefficient value R has been turned on by the process of step ST107 in FIG. 5 (ST301).

  When the change flag for the gain correction coefficient value R is off, the process ends without performing the calculation process of the gain correction coefficient value R. (ST301 → End).

  When it is confirmed that the change flag of the gain correction coefficient value R is ON, the comparison / discrimination unit 56 of the control microcomputer 50 and the shake amount data S1 generated based on the shake signal detected by the shake detection sensor 20 and the motion vector The gain correction coefficient value R is calculated from the camera shake amount data S2 generated based on the motion vector data detected by the detection unit 30 by the following equation (ST301 → ST302).

  Note that in the imaging apparatus that optically corrects camera shake, since the imaging unit 10 optically corrects camera shake, the camera shake amount data S2 is added to the camera shake amount data S1 calculated based on the detection of the camera shake detection sensor 20. The necessity of gain adjustment and the state of gain setting are appropriately determined based on the value obtained by adding (S1 + S2) and the gain correction coefficient value R calculated from the camera shake amount data S1 (see FIG. 9).

  Next, the comparison / determination unit 56 of the control microcomputer 50 determines that the gain correction coefficient value R (hereinafter, gain correction coefficient value Rn) calculated in the process of step ST302 is within a predetermined allowable value range (minimum allowable value Vok_l < It is determined whether or not the gain correction coefficient value Rn <maximum allowable value Vok_h) (ST303).

  The comparison / discrimination unit 56 of the control microcomputer 50 does not need to change the gain correction coefficient value R set in the gain adjustment unit 54 when the gain correction coefficient value Rn is within a predetermined allowable range. Further, it is determined that the gain setting of the apparatus is also appropriate (for example, no conversion lens is mounted) (ST303 → ST304).

  The gain adjustment unit 54 of the control microcomputer 50 sends the gain data calculated based on the currently set gain correction coefficient value R to the camera shake correction processing unit 60 based on the determination of the comparison / determination unit 56. Alternatively, optimum gain data based on the currently set gain correction coefficient value R is selected from predetermined table data stored in advance and sent to the camera shake correction processing unit 60.

  The camera shake correction processing unit 60 generates a control signal based on the gain data sent from the control microcomputer 50, controls the signal processing unit 40 and the TG 10A, and shakes the image signal sent from the signal processing unit 40. Correction processing is executed (ST304 → ST310).

  On the other hand, when the gain correction coefficient value Rn is outside the predetermined allowable value in the process of step ST303, the comparison / discrimination unit 56 of the control microcomputer 50 further determines that the gain correction coefficient value Rn is less than or equal to the minimum allowable value. It is determined whether or not there is (minimum allowable value Vok_l ≦ gain correction coefficient value Rn) (ST303 → ST305).

  When the gain correction coefficient value Rn is equal to or less than the minimum allowable value (minimum allowable value Vok_l ≦ gain correction coefficient value Rn), it is necessary to adjust the gain correction coefficient value R currently set in the gain adjustment unit 54. It is determined that the gain setting of the apparatus is too small to be appropriate, and the gain correction coefficient value Rn is sent to the gain adjustment unit 54 (ST305 → ST306).

  The gain adjustment unit 54 sets the gain correction coefficient value Rn from the comparison / discrimination unit 56 as a new gain correction coefficient value, and sends the gain data calculated based on the gain correction coefficient value Rn to the camera shake correction processing unit 60. Alternatively, optimal gain data is selected from predetermined table data stored in advance based on the gain correction coefficient value Rn and sent to the camera shake correction processing unit 60 (ST307).

  Further, the comparison / discrimination unit 56 of the control microcomputer 50 automatically switches the gain setting of the apparatus to an appropriate setting, or issues a message prompting the appropriate setting, a message indicating the possibility of a failure, or the like. It is displayed on the display unit (ST307).

  For example, as shown in FIG. 7, when the correspondence between the zoom position L and the magnification gain Gzm is stored as table data, a message indicating that the wide-angle conversion lens is mounted based on this table data is displayed on the display unit. It is also possible to detect a failure of the camera shake detection sensor 20 or the motion vector detection unit 30 and display an alarm by displaying or by setting a predetermined failure determination value and comparing it with the gain correction coefficient value Rn.

  The camera shake correction processing unit 60 generates a control signal based on the gain data sent from the control microcomputer 50 to control the servo mechanism unit 10B, and the camera shake of the image signal sent from the signal processing unit 40 is detected. Correction processing is executed (ST307 → ST310).

  In the process of step ST305, when the gain correction coefficient value Rn is larger than the minimum allowable value (minimum allowable value Vok_l> gain correction coefficient value Rn), the gain correction coefficient value Rn is larger than the maximum allowable value Vok_h (maximum allowable value). It is determined that the value Vok_h <gain correction coefficient value Rn), and that the gain setting of the apparatus is too large to be appropriate, and the gain correction coefficient value Rn is sent to the gain adjustment unit 54 (ST305 → ST308).

  The gain adjustment unit 54 sets the gain correction coefficient value Rn from the comparison / discrimination unit 56 as a new gain correction coefficient value, and sends the gain data calculated based on the gain correction coefficient value Rn to the camera shake correction processing unit 60. Alternatively, optimal gain data is selected from predetermined table data stored in advance based on the gain correction coefficient value Rn and sent to the camera shake correction processing unit 60 (ST309).

  In addition, the comparison / discrimination unit 56 of the control microcomputer 50 automatically switches the gain setting of the apparatus to an appropriate setting, or issues a message prompting to set it to an appropriate setting, a message indicating the possibility of a failure, or the like. It is displayed on the display unit (ST309).

  For example, as shown in FIG. 7, when the correspondence between the zoom position L and the magnification gain Gzm is stored as table data, a message indicating that a narrow-angle conversion lens is mounted based on this table data is displayed on the display unit. Or by setting a predetermined failure determination value and comparing it with the gain correction coefficient value Rn, it is possible to detect a failure of the camera shake detection sensor 20 or the motion vector detection unit 30 and display an alarm.

  The camera shake correction processing unit 60 generates a control signal based on the gain data sent from the control microcomputer 50 and controls the servo mechanism unit 10B to correct the optical axis shift in the imaging unit 10 and The image signal sent from the processing unit 40 is subjected to predetermined camera shake correction processing and sent to the next circuit (ST309 → ST310).

  As described above, in the image pickup apparatus of the second embodiment, the camera shake amount data S1 generated based on the camera shake signal detected by the camera shake detection sensor 20 by the control microcomputer 50 and the motion vector, as in the first embodiment. The correction coefficient value Rn is calculated based on the camera shake amount data S2 generated based on the motion vector data detected by the detection unit 30, and compared with the currently set gain correction coefficient value to determine the camera shake correction process. Gain data for adjusting the correction amount is generated.

  Then, the gain setting is switched to an appropriate setting state, or a control signal is generated based on the gain data and sent to the servo mechanism unit 10B to control the operation of the optical correction lens of the image pickup unit 10 to control lens light caused by camera shake vibration. The axis deviation is optically corrected, a predetermined camera shake correction process is performed on the image signal sent from the signal processing unit 40, and the image signal is sent to the next stage circuit.

It is the block diagram which simplified and showed the internal structure of the imaging device which concerns on this invention. 3 is a flowchart for explaining an outline of operation of the imaging apparatus shown in FIG. It is the block diagram which simplified and showed the internal structure of the imaging device used as 1st Example of this invention. FIG. 4 is a conceptual diagram showing a simplified relationship between image signals before / after camera shake correction and camera shake amount data S1 and camera shake amount data S2 in the imaging apparatus shown in FIG. 3; 9 is a flowchart for explaining an operation when evaluating / determining camera shake amount data detected by a camera shake detection sensor and a motion vector detection unit in the imaging apparatus of FIGS. 4 is a flowchart for explaining an operation until camera shake correction processing is executed in the imaging apparatus of FIG. 3. It is the graph which simplified and showed the correspondence of the focal distance by the conversion lens attachment to an imaging device, and magnification gain. It is the block diagram which simplified and showed the internal structure of the imaging device used as 2nd Example of this invention. FIG. 9 is a conceptual diagram showing a simplified relationship between image signals before / after camera shake correction, camera shake amount data S1, and camera shake amount data S2 in the image pickup apparatus of FIG. 8; 9 is a flowchart for explaining an operation until a camera shake correction process is executed in the imaging apparatus of FIG. 8.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10: Imaging part 10A; TG (timing generator) part 10B; Servo mechanism part 20; Shake detection sensor 30; Motion vector detection part 40; Signal processing part 50; Control microcomputer 51; A / D conversion part 52; (HPF / LPF / BPF)
53; integrator 54; gain adjustment unit 55; delay memory 56; comparison / determination unit 60; camera shake correction processing unit

Claims (4)

First camera shake detection means for detecting camera shake vibration of the imaging unit as a camera shake signal;
Second camera shake detection means for detecting motion vector data from the image signal captured by the imaging unit;
Camera shake amount data for generating first camera shake amount data based on the camera shake signal detected by the camera shake detection unit and generating second camera shake amount data based on the motion vector data detected by the second camera shake detection unit Generating means;
A gain for calculating a gain correction coefficient value for correcting the gain data for adjusting the correction amount of the camera shake correction processing based on the first camera shake amount data and the second camera shake amount data generated by the camera shake amount data generating means. Correction coefficient calculation means;
Based on the gain correction coefficient value calculated by the gain correction coefficient calculation means, gain determination means for determining a gain correction coefficient value set in the apparatus and a setting state of gain setting;
Gain data generating means for generating the gain data based on the gain correction coefficient value calculated by the gain correction coefficient calculating means in accordance with the determination of the gain determining means;
Gain setting switching means capable of switching the gain setting to a predetermined setting state in accordance with the determination of the gain determination means;
An image pickup apparatus comprising: a camera shake correction unit that performs a camera shake correction process on the image signal based on the gain data generated by the gain data generation unit. The image stabilization unit is configured to control a control signal for controlling the timing of storing an image signal in the image memory of the imaging unit and a read address of the image signal captured in the image memory based on the gain data. The imaging apparatus according to claim 1, wherein a control signal is generated. 2. The imaging according to claim 1, wherein the camera shake correction unit generates a control signal for controlling driving of a correction lens that corrects an optical axis shift of the imaging unit based on the gain data. apparatus. Furthermore, a motion vector determination unit that determines the reliability of the motion vector data detected by the second camera shake detection unit based on the image signal captured by the imaging unit;
The first camera shake amount data and the second camera shake amount data generated by the camera shake amount data generation unit are compared, and the first camera shake detection unit and the second camera shake detection unit detect the same type of camera shake vibration. A camera shake judging means for judging whether or not
The imaging apparatus according to claim 1, wherein the gain correction coefficient calculation unit calculates the gain correction coefficient value based on determinations of the motion vector determination unit and the camera shake determination unit.

Images