JP2021148917A - Image shake correction device and method and imaging apparatus - Google Patents

Abstract

To calculate a proper shake correction drive amount according to the state of an imaging apparatus.SOLUTION: An image shake correction device includes: estimation means for estimating an offset component of a shake detection signal by using the shake detection signal indicating the shake of the imaging apparatus and a motion vector signal detected from a plurality of images; detection means for detecting the state of the imaging apparatus among a plurality of predetermined states on the basis of the shake detection signal and the reliability of the motion vector signal; control means for controlling an estimation method by the estimation means on the basis of the state of the imaging apparatus; and generation means for generating a shake correction signal for driving the shake correction means on the basis of the estimated offset component.SELECTED DRAWING: Figure 3

Description

The present invention relates to an image shake correction device and method, and an image pickup device.

There is an image shake correction function for the problem that the influence of camera shake and the like on the image becomes remarkable as the image pickup device becomes smaller and the optical system becomes higher in magnification. When an angular velocity sensor is used to detect the shake of the image pickup device, movement control of a shake correction lens (hereinafter, simply referred to as a “correction lens”), an image sensor, or the like is performed according to the angular velocity detection signal.

Here, the output signal of the angular velocity sensor includes a DC component (offset component) such as a variation in the reference voltage due to individual differences. Therefore, if the output of the angular velocity sensor is integrated with the offset component included to calculate the angle signal, the integration error accumulates due to the offset component, and accurate runout correction cannot be performed.

In Patent Document 1, the output of the angular velocity sensor, the motion vector due to the difference between the frames of the image, and the velocity of the runout correction member are input, and the offset is estimated by using the Kalman filter or the sequential least squares method, and the estimation result is obtained. A method for removing the offset component is disclosed.

Japanese Unexamined Patent Publication No. 2018-25703

In Patent Document 1, the offset component is estimated using the difference between the output of the angular velocity sensor and the motion vector. However, a detection error may occur in the motion vector detection, and an appropriate offset component can be estimated depending on the state. It is possible that it cannot be done. For example, if the difference between the angular velocity sensor output and the motion vector detection value is used to estimate the offset component in the static state where the imaging device is installed on a tripod, the errors of both detection systems are superimposed, and the offset is offset only by the angular velocity sensor output. It is possible that the accuracy will be lower than when the estimation was performed.

The present invention has been made in view of the above problems, and an object of the present invention is to calculate an appropriate runout correction drive amount according to the state of the image pickup apparatus.

In order to achieve the above object, the image shake correction device of the present invention includes a shake detection signal indicating the shake of the image pickup device detected by the shake detection means, and a plurality of images captured by the image pickup device by the motion vector detection means. Using the motion vector signal detected from the above, the estimation means for estimating the offset component of the runout detection signal, the runout detection signal, and the reliability of the motion vector signal output from the motion vector detection means. Based on this, among a plurality of predetermined states, a detection means for detecting the state of the image pickup device, a control means for controlling the estimation method by the estimation means based on the state of the image pickup device, and the runout detection. It has a generation means for generating a runout correction signal for driving a runout correction means for correcting runout based on a signal and an offset component estimated by the estimation means.

According to the present invention, it is possible to calculate an appropriate runout correction drive amount according to the state of the image pickup apparatus.

A cross-sectional view of the imaging system according to the embodiment of the present invention, and a block diagram showing a schematic functional configuration. The block diagram which shows the structure related to the image shake correction control which concerns on 1st Embodiment. The flowchart which shows the calculation process of the image shake correction drive amount which concerns on 1st Embodiment. The flowchart of offset estimation processing in 1st Embodiment. The figure which shows the time waveform of the signal used for the offset estimation processing in 1st Embodiment. The block diagram which shows the structure related to the image shake correction control which concerns on 2nd Embodiment. The flowchart explaining the calculation process of the image shake correction drive amount which concerns on 2nd Embodiment.

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. The following embodiments do not limit the invention according to the claims. Although a plurality of features are described in the embodiment, not all of the plurality of features are essential to the invention, and the plurality of features may be arbitrarily combined. Further, in the attached drawings, the same or similar configurations are given the same reference numbers, and duplicate explanations are omitted.

The present invention can be applied to video cameras, digital still cameras, optical devices such as interchangeable lenses thereof, and various electronic devices having an imaging unit.

[First Embodiment]
Hereinafter, the first embodiment of the present invention will be described.
FIG. 1A is a central cross-sectional view showing an outline of the configuration of an imaging system including an image shake correction device according to the present embodiment, and FIG. 1B is a block diagram showing a functional configuration of the imaging system.

In the present embodiment, a so-called interchangeable lens type camera in which the lens unit 2 can be attached to and detached from the camera body 1 will be described, but the present invention is not limited to this, and the lens unit 2 is attached to the camera body 1. It can also be applied to a fixed imaging device.

In the imaging system shown in FIG. 1A, it is possible to take a picture with the lens unit 2 attached to the camera body 1. The camera body 1 and the lens unit 2 are electrically connected to each other via the electrical contacts 14 when the lens unit 2 is attached to the camera body 1. The lens unit 2 has an imaging optical system 3 composed of a plurality of lenses, and the optical axis 4 indicates the optical axis of the imaging optical system 3. The correction lens unit 19 in the lens unit 2 includes a correction lens for correcting the image shake of the captured image due to the user's camera shake or the like. The camera body 1 is provided with an image sensor 6 inside, and is provided with a rear display device 10a at the rear.

Next, the functional configurations of the camera body 1 and the lens unit 2 will be described with reference to FIG. 1 (b).
First, the configuration of the lens unit 2 will be described. The lens-side operation unit 16 includes an operation member operated by the user, and notifies the lens control unit 15 of the user's operation instruction. The lens control unit 15 includes a CPU (Central Processing Unit) and executes a predetermined program to control the operation of each component and perform processing. For example, the lens control unit 15 performs drive control of the correction lens unit 19, focus adjustment control by driving the focus lens, aperture control, zoom control, and the like.

The lens-side runout detection unit (hereinafter, referred to as “shake detection unit”) 17 includes an angular velocity sensor, detects the amount of runout, and outputs a runout detection signal to the lens control unit 15. For example, a gyro sensor is used to detect rotation in the pitch direction, yaw direction, and roll direction with respect to the optical axis 4 of the imaging optical system 3.

The image shake correction unit 18 performs drive control of the correction lens unit 19 in accordance with a control command from the lens control unit 15 to correct the image shake. The correction lens unit 19 includes a correction lens that is shift-driven or tilt-driven in a plane perpendicular to the optical axis 4, and a drive mechanism unit thereof. The image shake correction unit 18 drives the correction lens unit 19 by, for example, an actuator using a magnet and a flat plate coil. The lens position detection unit 20 includes, for example, a magnet and a Hall element, detects the position of the correction lens unit 19, and outputs a position detection signal to the lens control unit 15.
The focus position changing unit 22 controls the drive of the focus lens according to a control command from the lens control unit 15 to change the focal position of the imaging optical system 3.

The lens-side storage unit 23 includes a parameter holding area 23a and an offset value holding area 23b, and receives an instruction from the lens control unit 15 to write information such as a dispersion value and an offset value, which will be described later, to the lens control unit 15. Read it out.

Next, the configuration of the camera body 1 will be described. The image pickup device 6 constitutes an image pickup unit, photoelectrically converts an optical image of a subject incident on the image pickup surface of the image pickup element 6 through the image pickup optical system 3, and outputs an electric signal. Then, based on the electric signal output from the image sensor 6, an evaluation amount for focus adjustment and an appropriate exposure amount are acquired, and by adjusting the image pickup optical system 3, a subject image having an appropriate amount of light is obtained from the image sensor 6. An image can be formed on the imaging surface.

The image processing unit 7 has an analog / digital (A / D) converter, a white balance adjustment circuit, a gamma correction circuit, an interpolation calculation circuit, and the like inside, and acquires an electric signal output from the image sensor 6 to obtain an image. Processing is performed, image data for recording is generated, and stored in the storage unit 8. Further, when the image sensor 6 has a color filter of the bayer arrangement, the image processing unit 7 further includes a color interpolation processing unit, and performs color interpolation (demosizing) processing on the electric signal of the bayer arrangement to generate color image data. Then, it is recorded in the storage unit 8. Further, the image processing unit 7 compresses data such as an image, a moving image, and an audio by a predetermined compression method.

Further, the image processing unit 7 includes a motion vector detection unit 7a that calculates a motion vector which is a motion amount based on the captured image, compares images of a plurality of frames acquired by the image sensor 6, and performs motion vector detection. Get the amount of motion of the image with. Further, the motion vector detection unit 7a evaluates the reliability of the detected motion vector and outputs the vector evaluation value.
The camera control unit 5 includes a CPU (Central Processing Unit), performs operation control and processing of each configuration by executing a predetermined program, and controls the control of the entire imaging system. The camera control unit 5 and the lens control unit 15 communicate with each other via the electrical contacts 14, and can perform still image shooting and moving image shooting according to the user operation.

For example, when the user operates the shutter release button included in the camera side operation unit 9, the camera control unit 5 detects the switch signal and starts the control related to shooting. First, an electric signal output from the image sensor 6 is acquired, an appropriate focus position and aperture position of the image pickup optical system 3 are obtained by the image processing unit 7, and a control command is given to the lens control unit 15 via the electric contact 14. Send. The lens control unit 15 controls the focus position change unit 22 and the aperture drive unit (not shown) in accordance with a control command from the camera control unit 5. Then, by generating a timing signal or the like for imaging and outputting it to each component, the image sensor 6 photoelectrically converts the light from the subject incident on the subject through the imaging optical system 3 in which the aperture and the focal position are controlled. Then, the imaging process is executed. Further, the image processing unit 7 and the like are controlled to execute the image processing.

The image data recording / playback process is executed by using the image processing unit 7 and the storage unit 8 under the control of the camera control unit 5. Further, the camera control unit 5 controls to record the image data stored in the storage unit 8 on a detachable recording medium (not shown), or to read the image data from the recording medium to the storage unit 8, and is not shown. Controls transmission to an external device through an external interface.

The display unit 10 includes a rear display device 10a, a small display panel (not shown) for displaying shooting information provided on the upper surface of the main body of the device, an EVF (electronic viewfinder) (not shown), and the like, from the camera control unit 5. The image is displayed according to the control command of the above, and the image information and the like are presented to the user. The rear display device 10a is provided with a touch panel on the display screen unit, detects a user operation, and notifies the camera control unit 5. In this case, the rear display device 10a has both a function as an operating member and a display function.

In the image pickup system having the above configuration, when the image shake correction mode is set, the lens control unit 15 starts the image shake correction control when the camera control unit 5 receives a shift command to the image shake correction mode. As a specific control method, first, the lens control unit 15 acquires a shake detection signal from the shake detection unit 17. Then, the motion vector detection signal and the vector evaluation value are received from the camera body 1, and the drive amount (shake correction drive amount) of the correction lens unit 19 for performing image shake correction in combination with the shake detection signal by a method described later. Is calculated. The lens control unit 15 outputs the calculated runout correction drive amount as a command value to the image shake correction unit 18 to drive the correction lens unit 19. Further, the lens control unit 15 acquires a position detection signal from the lens position detection unit 20, and performs feedback control so that the detection position of the correction lens unit 19 follows the command value.

FIG. 2 is a block diagram showing a configuration related to image shake correction control in the first embodiment. Since each of the pitch direction, yaw direction, and roll direction axes, which are image runout correction axes, has the same configuration, only one axis will be described.

In FIG. 2, the lens control unit 15 includes a range surrounded by a broken line. The shake detection unit 17 detects the shake generated in the imaging system and outputs the shake detection signal to the subtractor 36, the LPF 38 (low-pass filter), and the camera state detection unit 32. The LPF 38 cuts off the high frequency band of the runout detection signal and outputs it to the subtractor 35.

The motion vector detection unit 7a detects a motion vector from the captured images of a plurality of frames and outputs a motion vector detection signal to the LPF 39. Further, the motion vector detection unit 7a calculates a vector evaluation value indicating the reliability of the motion vector detection at the time of motion vector detection, and outputs the vector evaluation value to the camera state detection unit 32.

The vector evaluation value is indicated by, for example, a correlation value when processing by block matching is performed at the time of motion vector detection. When the image matching process is difficult because the brightness of the image is low and the contrast is low, the correlation value is calculated to be low, and it is evaluated that the reliability of the motion vector detection is low. In addition, the vector evaluation value is calculated low even when the two images to be compared are out of focus or when the influence of camera shake is large and the image shake is large. Further, when the motion vector cannot be detected, such as during still image exposure, 0 is output as the vector evaluation value. The vector evaluation value may be other than the correlation value as long as it represents the reliability of the vector detection result.

The LPF 39 is set in the same cutoff band as the LPF 38, cuts off the high frequency band of the motion vector detection signal, and outputs the motion vector detection signal to the multiplier 37. The multiplier 37 multiplies the motion vector detection signal whose high frequency is blocked by the vector weighting coefficient αk, and outputs the signal to the subtractor 35. The vector weighting coefficient αk is controlled by the camera state detection unit 32.

The camera state detection unit 32 detects the state of the camera using the outputs of the runout detection unit 17 and the motion vector detection unit 7a, and outputs the result to the offset estimation unit 33. In the present embodiment, when the runout detection signal output from the runout detection unit 17 continues to be smaller than the predetermined statically indeterminate threshold value for a predetermined time, the camera state detection unit 32 is in the statically indeterminate state. Judge that it is in the state of. The statically indeterminate state is, for example, a state in which camera shake vibration does not occur, such as when a camera is attached to a tripod. It should be noted that the determination may be made using not only the output of the runout detection unit 17 but also the output of the motion vector detection unit 7a.

Further, when the vector evaluation value output from the motion vector detection unit 7a is lower than the predetermined value and is not in the first state, the camera state detection unit 32 determines that the camera is in the second state.

Then, in other cases, it is determined that the camera is in the third state. That is, in the third state, the runout detection signal is equal to or higher than a predetermined static threshold value, and the vector evaluation value is equal to or higher than a predetermined value. For example, in a state where the camera is gripped by the user and camera shake is generated, the motion vector detection unit 7a can detect a highly reliable motion vector.

When the camera state detection unit 32 determines that the camera state is the first or second state, the weighting coefficient αk of the multiplier 37 is set to 0, and when the camera state is determined to be the third state, the weight of the multiplier 37 is set. Change the coefficient αk to 1.
The subtractor 35 subtracts the vector detection signal of the motion vector detection unit 7a from the runout detection signal of the band-limited runout detection unit 17, and outputs the vector detection signal to the offset estimation unit 33.

The offset estimation unit 33 estimates the offset component of the runout detection signal of the runout detection unit 17 using a Kalman filter based on the output of the subtractor 35 and the result of the camera state detection unit 32, and subtracts the estimated offset estimated value. Output to 36. The details of the method of calculating the offset estimated value in the offset estimation unit 33 will be described later.

The subtractor 36 subtracts the offset estimated value estimated by the offset estimation unit 33 from the runout detection signal of the runout detection unit 17, and outputs the offset estimation value to the integrator 34.
The integrator 34 performs an integration process on the output signal of the subtractor 36 and outputs it as an image shake correction drive amount (shake correction signal) of the correction lens unit 19.

Although the camera state detection unit 32 has been described as being included in the lens control unit 15 in the present embodiment, it may be included in the camera control unit 5. In that case, the camera state detection unit of the camera control unit 5 acquires the shake detection signal of the shake detection unit 17 via the electrical contact 14, and the detection result of the camera state detected by the camera state detection unit of the camera control unit 5 is obtained. It is output to the lens control unit 15 via the electrical contact 14.

Next, the calculation process of the runout correction drive amount in the imaging system will be described with reference to the flowcharts of FIGS. 3 and 4. The process shown in the flowchart of FIG. 3 is repeatedly executed every image acquisition cycle in the image processing unit 7.

When the process starts in FIG. 3, in S1, the runout detection unit 17 detects the runout (angular velocity) and outputs the runout detection signal.

In S2, the motion vector detection unit 7a detects the motion vector using the acquired image and the image acquired one frame before the acquired image, calculates and outputs the motion vector detection signal and the vector evaluation value, and outputs the motion vector detection signal. .. The motion vector may be detected not only by one frame but also by using images before a plurality of frames.

In S3, the camera state detection unit 32 states that the camera states are in the first state, the second state, and the third state based on the shake detection signal of the shake detection unit 17 and the vector evaluation value of the motion vector detection unit 7a. Determine which is which.

In S4, the offset estimation unit 33 estimates the offset component included in the runout detection signal. The details of the offset estimation process performed in S4 will be described later with reference to FIG.

In S5, the subtractor 36 subtracts the offset estimation value estimated by the offset estimation unit 33 from the runout detection signal output from the runout detection unit 17.
In S6, the integrator 34 performs integration processing on the runout detection signal from which the offset has been removed, and outputs the runout correction drive amount to the image runout correction unit 18.

Next, the offset estimation process performed in S4 will be described with reference to FIG. 4 and a mathematical formula. First, a method of estimating the offset value used for offset processing will be described using a mathematical formula. When the offset estimation unit 33 is composed of a linear Kalman filter, the linear Kalman filter can be represented by the following equations (1) to (7).

式（１）は状態空間表現での動作モデルを表し、式（２）は観測モデルを表す。なお、式（１）及び（２）における各記号の意味は以下のとおりである。
Ａ：動作モデルでのシステムマトリクス
Ｂ：入力マトリクス
Ｃ：観測モデルでの出力マトリクス
ε_t：プロセスノイズ
δ_t：観測ノイズ
ｔ：離散的な時間。

Equation (1) represents an operation model in the state space representation, and equation (2) represents an observation model. The meanings of the symbols in the equations (1) and (2) are as follows.
A: System matrix in the operation model B: Input matrix C: Output matrix in the observation model ε _t : Process noise δ _t : Observation noise t: Discrete time.

式（３）は予測ステップにおける事前推定値を表し、式（４）は事前誤差共分散を表す。また、Σ_xは動作モデルのノイズの分散を表す。

Equation (3) represents the pre-estimated value in the prediction step, and Eq. (4) represents the pre-error covariance. In addition, Σ _x represents the noise variance of the operation model.

式（５）はフィルタリングステップにおいてカルマンゲインの算出式を表し、上付き添え字のＴは行列の転置を表している。さらに式（６）はカルマンフィルタによる事後推定値、式（７）は事後誤差共分散を表す。またΣ_zは観測モデルのノイズの分散を表す。

Equation (5) represents the calculation formula of Kalman gain in the filtering step, and the superscript T represents the transpose of the matrix. Further, Eq. (6) represents the post-estimation value by the Kalman filter, and Eq. (7) represents the post-error covariance. Σ _z represents the noise variance of the observation model.

In the present embodiment, in order to estimate the offset component of the runout detection unit 17, the offset value is x _t and the observed runout amount is z _t . Model of the offset component has no input term u _t in equation (1), and A = C = 1 in formula (1) and (2), the following equation of first order linear model (8), (9) Can be represented by.

Also, the Kalman gain k _t, when the offset observed by the shake detecting unit 17 and z _t, it is possible to constitute a Kalman filter (10) to (14).

In equations (10) to (14), the noise variance Σ _{x of the operation model in equation (4) is represented by the system noise variance SNV, and the noise variance Σ z} of the observation model in equation (5) is expressed as the system noise variance SNV. It is represented by the observed noise dispersion OMNB. The system noise dispersion SNV and the observation noise dispersion OMNV are expressed as follows in the equations (10) to (14).

Further, the pre-offset estimated value _{OFF 1 t} , the pre-error variance EV 1 _t , the post-error variance EV 2 _t , and the observed noise variance ON V _t at time t are expressed as follows in the equations (10) to (14).

In the present embodiment, the offset estimation unit 33 is configured to realize the arithmetic expressions from the equation (10) to the equation (14). As shown in the equation (10), the pre-offset estimated value OFF1 _t is calculated by _{the posterior offset estimated value OFF2 t-1} at the time t-1 of the update cycle of the estimation operation. Further, as shown in the equation (11), the pre-error variance EV1 _t is calculated by _{the post-error variance EV2 t-1 at the time t-1 and the system noise variance SNV.}

Then, as shown in the equation (12), the Kalman gain k _t is calculated _{based on the prior error variance EV 1 t} and the observed noise variance ON V _t.

Further, the equation (13), fixes the pre-offset estimate OFF1 _t by the observed offset z _t and pre offset estimate OFF1 error Kalman gain k _t a multiplied value of the _t, offset estimate OFF _t is Calculated. Further, the pre-error variance EV1 _t is corrected by the equation (14) to calculate the post-error variance EV2 _t. The offset estimation value is calculated by repeating the pre-estimation update and correction for each calculation cycle by these operations.

Next, the offset estimation process at time t will be described with reference to the flowchart of FIG.
When the offset estimation process starts, in S11, the camera state detection unit 32 determines whether the state of the camera has changed from the state at time t-1. If it is determined that the state of the camera has changed, the process proceeds to S12, and if it is determined that the state of the camera has not changed, the process proceeds to S14.

_{In S12, the posterior error variance DV2 t-1} calculated at the time t-1 when the offset estimation process was performed last time is recorded in the parameter holding region 23a. In S13, the latest posterior error variance recorded in the same state as the state of the camera after the transition is called as _{the posterior error variance DV2 t-1 at time t-1.}

For example, in the case of transition from the second state to the third state, in S12, the posterior error variance DV2 _t-1 in the second state calculated at time t-1 is recorded in the parameter holding region 23a. As described above, in S12, when the state of the camera changes, the post-error variance before the transition is recorded in the parameter holding region 23a. Then, in S13, from the posterior error variance recorded in the parameter holding region 23a, the posterior error variance recorded in the third state after the previous transition is used as the posterior error variance DV2 _{t-1 at time t-1.} Read from the parameter holding area 23a.

Then, the posterior error variance at time t is read from the parameter holding area 23a instead of the posterior error variance at time t-1 when it was in the second state, and the posterior error when it was in the third state last time. Replace with distribution. When the state after the transition is the first state after the camera is started, the initial value of the posterior error variance recorded in advance for each state in the parameter holding area 23a is called and used.

In S14, it is determined which state the camera is in. If the state of the camera is determined to be the third state, the process proceeds to S15, and if it is determined to be the first state or the second state, the process proceeds to S16.

In S15, the vector weighting coefficient αk of the multiplier 37 is set to 1. In this case, since camera shake is included and the reliability of motion vector detection is not low, it is considered that the estimation accuracy is improved by estimating the offset value using the motion vector detection signal. In the third state, the weighting coefficient αk may be changed according to the magnitude of the vector evaluation value. In that case, the larger the vector evaluation value, the closer the weighting coefficient αk is to 1.
On the other hand, in step S16, the vector weighting coefficient αk of the multiplier 37 is set to 0. That is, the offset estimation unit 33 performs offset estimation based on the difference between the runout detection signal and the motion vector signal, but since the vector weighting coefficient αk is 0, the offset estimation is substantially performed using only the runout detection signal. Become. This is because, for example, in the first state where the camera is statically indeterminate with a tripod or the like, the signal of the shake detection unit 17 does not contain a camera shake component, is stable at an amplitude close to 0, and shake detection. This is because the signal becomes an offset value.

Further, in the first state, the motion vector detection signal also stabilizes at an amplitude close to zero. Here, when the vector weighting coefficient αk is set to 1 and the vector detection signal is subtracted from the runout detection signal, the detection noise of each of the two detection systems is included. Therefore, it is better to calculate the offset value using only the runout detection signal. The accuracy is high. Therefore, when the first state is determined, the vector weighting coefficient αk is set to 0.

On the other hand, in the second state where the camera shake is included and the reliability of the motion vector detection is low, the reliability of the motion vector detection signal is low, so the vector weighting coefficient αk is set to 0 and the offset estimation is performed only by the shake detection signal. conduct. In cases other than the third state, the weighting coefficient αk may be set to a very small value that does not affect the offset estimation instead of 0, and the ratio of the motion vector detection signal may be reduced.

Next, in S17, as shown in equation (10) and (11) to calculate the pre-offset estimate OFF1 _t and advance error variance EV1 _t. As described above, when the camera state has changed in S11, the posterior error variance at time t-1 is replaced by the posterior error variance in S13 when calculating _{the prior error variance EV1 t.}

In S18, the calculated Kalman gain k _t as shown in equation (12).
In S19, it is determined whether or not the predetermined period TL has elapsed since it was determined that the camera state has changed. If the predetermined period TL has elapsed, the process proceeds to S20, and if not, the process proceeds to S21.
In S20, the post-offset estimated value OFF2 _t is calculated as shown in the equation (13).

On the other hand, in S21, the post-offset estimated value OFF2 _{t is not calculated by the equation (13), and the offset estimated value OFF t-1} at time t-1 held in the offset value holding area 23b is used as the offset estimated value at time t. Substituting as OFF _t , the offset estimated value OFF _t is retained for a predetermined period. _{The reason for holding the offset estimated value OFF t} during the predetermined period TL will be described later.

In S22, the posterior error variance EV2 _t is calculated as shown in the equation (14).
The offset estimation process is completed as described above.

As described above, in the present embodiment, the value of the vector weighting coefficient is changed according to the result of the camera state detection unit 32, and in the first state which is a statically indeterminate state and the second state where the vector evaluation value is low, Offset estimation is performed using only the runout detection signal. As a result, the accuracy of offset estimation is improved in a statically indeterminate state such as when installed on a tripod or in a state where the reliability of motion vector detection is low, and an appropriate runout correction drive amount is calculated according to the state of the imaging device. Can be done.

Next, in S13 of the flowchart of FIG. 4, the reason for calling the posterior error variance recorded when the state of the camera was the same as the state of the camera after the transition last time as the posterior error variance EV2 _{t-1 at time t-1.} Will be described.

In the second state, since the reliability of the motion vector detection is low, the offset estimation process is performed using only the runout detection signal, but unlike the first state, the second state contains a camera shake component. Therefore, the observed noise dispersion ONV _t is larger than that in the first state. As a result, the posterior error variance EV2 _t also increases.

On the other hand, in the third state, the camera shake component is included in the shake detection signal, but since the motion vector detection is performed and the offset estimation process is performed, the observed noise dispersion ONV _t is smaller than that in the second state, and as a result. The posterior error variance is also small. However, the posterior error variance EV2 _t in the third state is larger than _{the posterior error variance EV2 t} in the first state in the statically indeterminate state. _{That is, the accuracy of the observation offset value z t} is high in the order of the first state, the third state, and the second state. The error variance _{can be said to be an index showing the accuracy of the observed offset value z t} , and it can be said that the smaller the error variance is, the higher the accuracy of the observed offset value z t is, and the larger the error variance is, the lower the accuracy is.

For example, consider the case where the second state is changed at time t-1 to the third state at time t. The observation offset value z _t in the third state is relatively higher in accuracy than the observation offset value z _t in the second state, but it cannot be said to be sufficiently accurate. When the posterior error variance of the second state at time t-1 is used when calculating the pre-error variance in S17, the variance value of the second state is large, so that the Kalman gain approaches 1. Then, as shown in the equation (13), the post-offset estimated value OFF 2 _t is greatly affected by the observed offset value z _t , and the offset estimated value is likely to change.

Even after time t + 1, the effect of the observed offset value in the third state is large until the posterior error variance stabilizes. As described above, the observed offset value in the third state is more accurate than the second state, but it cannot be said to be sufficiently accurate, and the observed offset value is not stable. As a result, until the posterior error variance after time t converges, the value of the posterior offset estimated value tends to become unstable due to the influence of the observed offset value whose accuracy is not sufficient, and the runout correction drive amount is correctly adjusted. It may not be possible to calculate.

Therefore, in S13, the posterior error variance recorded in the third state last time _{is called as the posterior error variance DV2 t-1} at time t-1, and the preliminary error variance is obtained in S17. Since the value of the third state having the same posterior error variance is used, the influence of the observed offset value in the third state is reduced when calculating the Kalman gain, and the fluctuation of the pre-offset estimated value can be suppressed. In this way, when the transition from the state where the accuracy of the observation offset value is low to the state where the observation offset value is higher than the previous state but is not sufficient, the post-recorded state in the same state as the camera state after the previous transition. It is effective to call the error variance as the posterior error variance DV2 _{t-1 at time t-1.} That is, at least in the case of the present embodiment, when the transition from the second state to the third state is performed, a process of calling the ex post facto error variance in the previous second state is performed.

In the present embodiment, the camera states are classified into three states, a first state, a second state, and a third state, but they are further classified into other states according to the magnitude of the runout detection signal and the like. May be increased. For example, in the third state, it is further divided into two types according to the magnitude of the runout detection signal. Even in that case, if the observation offset value transitions from a state in which the accuracy is low to a state in which the observation offset value is more accurate than the previous state, the post-error variance recorded in the same state as the camera state after the previous transition. Is effectively called as the posterior error variance DV2 _{t-1 at time t-1.}

Further, when the transition from the first state to another state is performed as another condition, the posterior error variance in the first state may be used as it is for the posterior error variance at time t-1. This is because the accuracy of the observed offset value is good in the first state and the posterior error variance is sufficiently small, so even if the transition to other states occurs, the greater the influence of the offset estimated value estimated in the first state is, the more accurate it is. Is considered to be good.

In the above description, since it was modeled in one dimension, it was explained using the posterior error variance, but the same control may be performed by modeling in multiple dimensions and using the posterior error covariance.

Next, the reason for holding the offset estimated value in S21 will be described with reference to FIG. In FIG. 5, the dotted lines 501, 506, and 511 represent the correct offset values, and the periods 505, 510, and 515 represent the predetermined period TL.

The waveform 502 in FIG. 5A represents the time change of the output signal of the subtractor 35 when transitioning from the second state to the first state. Since the vector weighting coefficient αk = 0 in the first and second states, the output signal of the LPF 38 and the waveform 502 match. The period 503 is the second state, and at this time, the output signal of the shake detection unit 17 is large due to the camera shake component. On the other hand, the period 504 is the first state, and at this time, the output signal of the runout detection unit 17 is small. Therefore, when transitioning from the second state to the first state, a transient response is generated by the LPF 38 over a predetermined period of TL, as in the case of the waveform 502 and the correct offset value 501 in the period 505, and the deviation from the correct offset value. Occurs. Here, the transient response refers to a phenomenon in which the output signal changes only gradually even if the input signal to the LPF changes suddenly.

At this time, since the first state observation noise variance ONV _t is small, easily changes under posterior offset estimate OFF2 _t is substantially affected by the observed offset values z _t, the observed offset values z _t erroneous due to the transient response It will be pulled. Therefore, since the offset cannot be estimated correctly from the transition from the second state to the first state until the TL elapses for a predetermined period, the offset estimated value before the transition is held in the offset value holding region 23b. The held offset value is used as the output of the offset estimation unit 33.

Similarly, in the transition from the third state to the first state, a transient response by LPF38 occurs immediately after the transition to the first state, so that the transition is made before the transition until the TL elapses for a predetermined period. The offset estimated value of is held in the offset value holding region 23b, and the held offset value is used as the output of the offset estimating unit 33.

The waveform 507 of FIG. 5B represents the time change of the output signal of the subtractor 35 when transitioning from the second state to the third state. The period 508 is the second state, and as an example, represents a state in which the motion vector cannot be acquired because the image sensor 6 is under exposure. Period 509 is the third state. When transitioning from the second state to the third state, the motion vector is steeply input to the LPF 39, so that a transient response occurs in the output signal. When the output signal of the LPF 39 is input to the subtractor 35, the transient response also affects the output signal of the subtractor 35, as in the waveform 507 and the correct offset value 506 in the period 510, from the correct offset value. There is a divergence.

At this time, since the observed noise variance ONV σ _t is smaller in the third state than in the second state, the ex post facto offset estimated value OFF 2 _t is likely to change due to the influence of the observed offset value z _t , and is erroneous due to the transient response. It is pulled by the observation offset value z _t. In this way, since the offset cannot be estimated correctly from the transition from the second state to the third state until the TL elapses for a predetermined period, the offset estimated value before the transition is held in the offset value holding area 23b. Then, the held offset value is used as the output of the offset estimation unit 33.

The waveform 512 of FIG. 5C represents the time change of the output signal of the subtractor 35 when transitioning from the state before starting the offset estimation to the first state. Since the vector weighting coefficient αk = 0 in the first state, the output signal of the LPF 38 matches the waveform 512. The period 513 represents a state before starting the offset estimation, for example, before starting the power supply of the imaging device. Since the outputs of the runout detection unit 17 and the motion vector detection unit 7a are 0, the output of the subtractor 35 is also 0. be.

Offset estimation is performed in period 514, and represents the first state as an example. When the signal of the runout detection unit 17 starts to be output, the signal is suddenly input to the LPF 38 from 0, and a transient response occurs like the waveform 512 and the correct offset value 511 in the period 515 to generate the correct offset. There is a divergence from the value. Therefore, since the offset cannot be estimated correctly from the start of the signal output of the runout detection unit 17 until the TL elapses for a predetermined period, the predetermined offset estimation initial value is maintained.

As the predetermined offset estimation initial value, the last offset estimation value at the time of the previous offset estimation may be held in the offset value holding area 23b, and the held offset value may be used, or the factory may use the held offset value. A fixed offset value adjusted by the above may be used. Further, the same processing is performed even when the state before the start of offset estimation is changed to the second or third state.

The predetermined period TL is determined by the cutoff frequencies of LPF38 and LPF39.

In addition to the method of using the offset value held in the offset value holding area 23b as the output of the offset estimation unit 33 from the transition to the elapse of the predetermined period TL, as a process of setting the _{prior error variance EV1 t small.} May be good. By reducing the pre-error variance EV1 _t, the value of the Kalman gain k _t according to equation (12) is reduced, the offset estimated value x _t by formula (13) is hardly affected by the observed offset values z _t. Therefore, it is possible to prevent erroneous estimation.

As another method, it may be a process of increasing the _{observed noise dispersion ONV t.} The value of the Kalman gain k _t according to equation (12) by increasing the observation noise variance ONV _t decreases, offset estimate x _t is less susceptible to the observed offset value z _t by the formula (13). Therefore, it is possible to prevent erroneous estimation.

Note that the posterior error variance at time t-1 was saved in S12 of FIG. 4, and the posterior error variance recorded in the same state as the camera state after the previous transition was called in S13. You may save the pre-error variance of -1 and call the pre-error variance in S13. This is because there is little difference between the pre-error variance and the post-error variance at time t-1 in the same camera state.

Further, in the present embodiment, the system noise dispersion SNV when obtaining the prior error dispersion in the equation (10) is fixed, but it may be set to a unique value according to the state of the camera. For example, the system noise dispersion SNV in each camera state is stored in the parameter holding area 23a of the lens side storage unit 23, and if it is determined in S11 of FIG. 4 that the camera state has changed, the state changed after S13. The value of the system noise dispersion SNV may be changed according to the above.

In the present embodiment, the motion vector detection unit 7a calculates the vector evaluation value which is the reliability of the vector detection, and when the reliability of the vector detection is low and the vector evaluation value is smaller than the threshold value, the multiplier 37 is used to obtain the motion vector detection signal. The vector weighting coefficient αk = 0 was multiplied. However, when the image acquisition element 6 and the image processing unit 7 change the image acquisition cycle for detecting the motion vector, there are the following problems.

For example, it is assumed that the image acquisition cycle is 60 fps in the initial shooting state. When the subject becomes dark and the brightness decreases from this state, it is conceivable to automatically reduce the acquisition cycle to 30 fps or the like and perform processing so that the brightness value of the captured image in one frame increases. At this time, even if the motion vector detection unit 7a obtains an image having sufficient brightness, the exposure period of one frame is extended, so that the image in one frame may be shaken and the accuracy of the motion vector may be lowered. ..

Therefore, when the image acquisition cycle for detecting the motion vector in the image sensor 6 and the image processing unit 7 is reduced, the camera state detection unit 32 determines that the motion vector is in a second state with low reliability. You may. Alternatively, the vector evaluation value may be set to 0 and output to the camera state detection unit 32 regardless of the calculated vector evaluation value.

[Second Embodiment]
Next, a second embodiment of the present invention will be described. In the second embodiment, only the configuration and the processing portion different from those in the first embodiment will be described, and the description of the overlapping portion will be omitted.

As described above, in the first embodiment, the offset estimation is performed using the difference between the runout detection signal and the motion vector detection signal. On the other hand, the second embodiment differs in the following points.
{Daraku = OFF}
-At the time of detecting the motion vector, the image shake correction operation is performed by the image shake correction unit 18.
The motion vector detection signal and the differential value of the detection position of the correction lens unit 19 are subtracted from the amount of runout input to the offset estimation unit 33.

Since the configuration of the imaging system including the image shake correction device of the second embodiment is the same as that of FIG. 1 described in the first embodiment, the description thereof will be omitted.

FIG. 6 is a block diagram showing a configuration related to image shake correction control in the second embodiment. Compared with the configuration shown in FIG. 2 of the first embodiment, the lens position detection unit 20, the differentiator 40, and the adder 41 are added.

The position information detected by the lens position detection unit 20 is differentiated by the differentiator 40, and the lens speed information is output to the adder 41. In the differentiator 40, the difference between the position of the correction lens unit 19 acquired at the time point one cycle before and the position of the correction lens unit 19 acquired at the time point of the current cycle is calculated. The result of the differential calculation of the position of the correction lens unit 19 is the drive speed (lens drive speed) of the correction lens unit 19 driven by the image shake correction.

The motion vector detection signal calculated by the motion vector detection unit 7a is input to the adder 41 via the LPF 39.
The adder 41 outputs the addition result of the lens drive speed information and the motion vector detection signal whose high frequency is blocked to the multiplier 37. In this way, the run-out speed of the camera body 1 is calculated by adding up the run-out speed corrected by the image run-out correction and the run-out residual speed detected by the motion vector. Further, if the image shake correction is not performed, the correction lens speed becomes zero, so that the motion vector information directly represents the shake detection result of the camera body 1 without necessarily adding the correction lens speed. It will be.

Here, since the motion vector is information obtained by detecting the amount of movement of the image on the imaging surface, strictly speaking, it includes the influence of shift shake and the like in addition to the angular shake of the camera body 1. The shooting condition in which the influence of the shift runout or the like is large is when the shooting magnification is large. In this case, the reliability of the angular shake detection of the motion vector is low, so the weighting coefficient αk of the multiplier 37 is set to 0. However, under imaging conditions where the influence of shift shake is small, the influence of angular shake is dominant in the amount of movement observed on the image plane. Under such conditions, the runout speed corrected by the image runout correction and the runout remaining speed detected by the motion vector are added up and used as the runout speed of the camera body 1. By using the runout speed of the camera body 1 calculated using the motion vector, it is possible to accurately detect the runout itself of the camera body 1 having a smaller offset error than the runout detection unit 17.

The operations other than the above are the same as those of the first embodiment described with reference to FIG. 2, and thus the description thereof will be omitted.

Next, the calculation process of the runout correction drive amount in the imaging system will be described with reference to the flowchart of FIG. 7. The process shown in the flowchart of FIG. 7 is repeatedly executed every image acquisition cycle in the image processing unit 7.

When the process starts in FIG. 7, the runout detection unit 17 detects the runout (angular velocity) in S41.

In S42, the motion vector detection unit 7a detects the motion vector using the acquired image and the image acquired one frame before the acquired image, calculates and outputs the motion vector detection signal and the vector evaluation value, and outputs the motion vector detection signal. .. The motion vector may be detected not only by one frame but also by using images before a plurality of frames.

In S43, the position of the correction lens unit 19 is detected by the lens position detection unit 20.

In S44, the camera state detection unit 32 states that the camera states are in the first state, the second state, and the third state based on the shake detection signal of the shake detection unit 17 and the vector evaluation value of the motion vector detection unit 7a. Determine which is which.

In S45, the offset estimation unit 33 estimates the offset component included in the runout detection signal. In the offset estimation process performed here, as shown in FIG. 6, the motion vector detection signal acquired in S42 and the differential value of the position of the correction lens unit 19 acquired in S43 are added and used. Since it is the same as that shown in FIG. 4, the description thereof will be omitted.

In S46, the subtractor 36 subtracts the offset estimation value estimated by the offset estimation unit 33 from the runout detection signal output from the runout detection unit 17.

In S47, the integrator 34 performs integration processing on the runout detection signal from which the offset has been removed, and outputs the runout correction drive amount to the image runout correction unit 18.

As described above, according to the second embodiment, an appropriate shake correction drive amount is calculated according to the state of the imaging system even when the image shake correction operation is performed from the time before the still image exposure in which the motion vector detection is performed. can do.

[Modification example]
In the first and second embodiments described above, the lens unit 2 has the image shake correction unit 18, and the image shake correction is performed by moving the correction lens unit 19, but other methods may be used. For example, the image sensor 6 of the camera body 1 is provided with an image sensor shift drive unit capable of translating in a direction substantially orthogonal to the optical axis, and the image sensor 6 is driven in translation according to the image shake correction drive amount obtained in the embodiment. Can be considered. The translational drive amount of the image sensor 6 is determined by multiplying the translational drive amount by a coefficient corresponding to the focal length of the lens unit 2 and converting it into the movement amount on the image pickup surface due to the shake. Further, both the camera body 1 and the lens unit 2 may have a shake correction mechanism. Further, the offset estimation process of the present invention is also effective in the so-called electronic vibration isolation process of cutting out the position of each frame in moving image shooting.

[Other Embodiments]
The present invention may be applied to a system composed of a plurality of devices or a device composed of one device.

The present invention also supplies a program that realizes one or more functions of the above-described embodiment to a system or device via a network or storage medium, and one or more processors in the computer of the system or device implement the program. It can also be realized by the process of reading and executing. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

The invention is not limited to the above embodiments, and various modifications and modifications can be made without departing from the spirit and scope of the invention. Therefore, a claim is attached to make the scope of the invention public.

1: Camera body 2: Lens unit 3: Imaging optical system, 6: Imaging element, 7: Image processing unit, 7a: Motion vector detection unit, 9: Camera side operation unit, 15: Lens control unit, 16: Lens Side operation unit, 17: Lens side shake detection unit, 18: Image shake correction unit, 19: Correction lens unit, 20: Lens position detection unit, 22: Focus position change unit, 23: Lens side storage unit, 23a: Parameter retention Area, 23b: Offset value holding area, 32: Camera state detection unit, 33: Offset estimation unit, 36: Subtractor, 40: Differentiator

Claims (25)

Using the shake detection signal indicating the shake of the image pickup device detected by the shake detection means and the motion vector signal detected from a plurality of images taken by the image pickup device by the motion vector detection means, the shake detection signal can be obtained. An estimation method for estimating the offset component and
A detection means for detecting the state of the imaging device among a plurality of predetermined states based on the runout detection signal and the reliability of the motion vector signal output from the motion vector detection means.
A control means for controlling the estimation method by the estimation means based on the state of the image pickup apparatus, and
A generation means for generating a runout correction signal for driving a runout correction means for correcting runout based on the runout detection signal and an offset component estimated by the estimation means, and a generation means for generating the runout correction signal.
An image shake correction device characterized by having. The plurality of states include a first state in which the runout detection signal is smaller than a predetermined first threshold value.
The image according to claim 1, wherein the control means controls the estimation method so as to reduce the ratio of the motion vector signal used for estimating the offset component in the case of the first state. Runout correction device. The image shake according to claim 2, wherein the control means controls the estimation method so as to estimate the offset component without using the motion vector signal in the case of the first state. Correction device. The plurality of states include a second state in which the reliability is lower than a predetermined second threshold value.
Any of claims 1 to 3, wherein the control means controls the estimation method so as to reduce the ratio of the motion vector signal used for estimating the offset component in the case of the second state. The image shake correction device according to item 1. The image shake according to claim 4, wherein the control means controls the estimation method so as to estimate the offset component without using the motion vector signal in the case of the second state. Correction device. The plurality of states
A first state in which the runout detection signal is smaller than a predetermined first threshold value, and
A second state in which the reliability is lower than a predetermined second threshold value, and
Including the first state and the third state excluding the second state.
The control means controls the estimation means so that the ratio of the motion vector signal used for estimating the offset component is higher than that of the first state and the second state in the case of the third state. The image shake correction device according to any one of claims 1 to 5, wherein the image shake correction device. The estimation means estimates the offset component using a Kalman filter and estimates the offset component.
The image shake correction device further includes a parameter holding means for holding a parameter used when estimating an offset component in the estimating means.
The image shake correction device according to any one of claims 1 to 6, wherein the Kalman filter holds the posterior error variance or the posterior error covariance of each of the plurality of states by the parameter holding means. When the state of the imaging device detected by the detection means transitions, the post-error variance or post-error covariance before the transition is recorded in the parameter holding means.
The estimation means calls the posterior error variance or posterior error covariance recorded in the parameter holding means when the state is in the state before the transition before the transition, and the called posterior error variance or pre-variance. The image shake correction device according to claim 7, wherein the offset component is estimated using error covariance. The plurality of states
A first state in which the runout detection signal is smaller than a predetermined first threshold value, and
A second state in which the reliability is lower than a predetermined second threshold value, and
Including the first state and the third state excluding the second state.
When the state of the imaging device detected by the detection means transitions from the second state to the third state, the estimation means retains the parameters when it was in the third state last time. 7. Image shake correction device. The plurality of states include a first state in which the runout detection signal is smaller than a predetermined first threshold value.
When the state of the imaging device detected by the detection means transitions from the first state to another state, the estimation means is the previous first state recorded in the parameter holding means. The image shake correction device according to any one of claims 7 to 9, wherein the offset component is estimated by using the posterior error variance or the posterior error covariance obtained at the time. The estimation means
A low-pass filter that limits the high-frequency band component of the output of the runout detection means when estimating the offset component.
It has an offset value holding means for holding an estimated value of the offset component, and has
When the state of the imaging device detected by the detecting means transitions, the estimating means estimates the offset component before the transition of the state for a predetermined period after the transition of the state of the imaging device. The image shake correction device according to any one of claims 1 to 10, wherein the image is held in the offset value holding means, and the held estimated value is output as an estimated value of the offset component. The plurality of states include a first state in which the runout detection signal is smaller than a predetermined first threshold value.
When the state of the imaging device detected by the detection means transitions from the plurality of states other than the first state to the first state, the estimation means is in the state for the predetermined period. The image shake correction device according to claim 11, wherein the estimated value of the offset component before the transition is held in the offset value holding means, and the held estimated value is output as the estimated value of the offset component. .. The plurality of states
A first state in which the runout detection signal is smaller than a predetermined first threshold value, and
A second state in which the reliability is lower than a predetermined second threshold value, and
Including the first state and the third state excluding the second state.
When the state of the imaging device detected by the detection means transitions from the second state to the third state, the estimation means performs the predetermined period before the transition of the state. The image shake correction device according to claim 11 or 12, wherein the estimated value of the offset component is held and the held estimated value is output as the estimated value of the offset component. The estimated value of the offset component is held for a predetermined period after the detection means detects any of the plurality of states and the estimation means starts estimating the offset component. The image shake correction device according to any one of claims 11 to 13, wherein the held estimated value is output as an estimated value of the offset component. The estimation means
A low-pass filter that limits the high-frequency band component of the output of the runout detection means when estimating the offset component.
It has an offset value holding means for holding an estimated value of the offset component, and has
When the state of the imaging device detected by the detection means changes, the estimation means sets the posterior error variance or the pre-error covariance smaller than the other periods during a predetermined period. The image shake correction device according to any one of claims 7 to 10. The estimation means
A low-pass filter that limits the high-frequency band component of the output of the runout detection means when estimating the offset component.
It has an offset value holding means for holding an estimated value of the offset component, and has
Claims 7 to 10 are characterized in that, when the state of the image pickup apparatus detected by the detection means changes, the estimation means sets the observed noise dispersion larger than the other periods during a predetermined period. The image shake correction device according to any one of the above items. The image shake correction device according to any one of claims 11 to 16, wherein the predetermined period is determined based on the cutoff frequency of the low-pass filter. Imaging means and
Runout correction means and
The image shake correction device according to any one of claims 1 to 17,
An imaging device characterized by comprising. It further has a position detecting means for outputting a position detecting signal indicating the position of the runout correction means.
The imaging device according to claim 18, wherein the estimation means further estimates the offset component by using the position detection signal. The image pickup apparatus according to claim 18 or 19, wherein the shake correction means corrects shake by shifting a correction lens that allows light incident on the image pickup means to pass therethrough based on the shake correction signal. .. The image pickup apparatus according to claim 18 or 19, wherein the runout correction means corrects runout by shifting the image pickup means based on the runout correction signal. The image pickup apparatus according to claim 18 or 19, wherein the runout correction means corrects runout by shifting a cutout position of an image output from the image pickup means based on the runout correction signal. The first acquisition step of acquiring a shake detection signal indicating the shake of the imaging device detected by the shake detection means, and
A second acquisition step of acquiring motion vector signals detected from a plurality of images captured by the imaging device by the motion vector detecting means, and
An estimation step of estimating the offset component of the runout detection signal using the runout detection signal and the motion vector signal, and an estimation step.
A detection step of detecting the state of the imaging device among a plurality of predetermined states based on the runout detection signal and the reliability of the motion vector signal output from the motion vector detecting means.
A control step that controls the estimation method used in the estimation step based on the state of the imaging device, and
A generation step of generating a runout correction signal for driving a runout correction means for correcting runout based on the runout detection signal and the offset component estimated in the estimation step, and a generation step.
An image shake correction method characterized by having. A program for causing a computer to function as each means of the image shake correction device according to any one of claims 1 to 17. A computer-readable storage medium that stores the program according to claim 24.

Images