JP6604783B2 - Image processing apparatus, imaging apparatus, and image processing program - Google Patents

Description

  The present invention relates to an image processing technique for performing image blur correction on an image generated by imaging through an optical system.

  Conventionally proposed is a camera that performs image processing to correct image blur using motion vectors detected from multiple consecutive images (frames) and outputs from motion sensors such as gyro sensors when images are generated by imaging. Has been. Patent Document 1 discloses a camera that detects an image blur amount using a motion vector in imaging using a fisheye optical system. In the camera of Patent Document 1, in consideration of the fact that the distortion rate of the subject image varies depending on the image height, the region (cutout region) for detecting the motion vector in the fisheye image is changed according to the image height. In the following description, the high distortion rate of the subject image means that the compression rate of the image generated by capturing the subject image is high. In Patent Document 2, an image shake amount between fisheye images generated by imaging using a fisheye optical system is detected, and pixel shift is performed when the image shake amount is a predetermined value or less. A camera is disclosed. The camera of this patent document 2 changes the cutout region according to the amount of motion detected by the motion sensor only when a rotational motion of a predetermined amount or more of the camera is detected.

JP 2006-295626 A JP-A-8-307753

  However, since the camera disclosed in Patent Document 1 performs image blur correction using only a motion vector, if a motion vector is detected in a high image height region where the image compression rate is high, the motion vector detection accuracy is low. As a result, the image blur correction accuracy decreases. Further, in the camera disclosed in Patent Document 2, image blur correction is performed using both the shift amount acquired from the image and the motion amount detected by the motion sensor. The image blur correction combined with consideration is not performed.

  In the present invention, when image blur correction is performed on an image generated by capturing a distorted image using both a motion vector detection result and a motion detection result by a motion sensor, more accurate image blur correction is performed. An image processing apparatus capable of performing the above is provided.

An image processing apparatus according to an aspect of the present invention is an image processing apparatus that performs image blur correction on an input image generated by capturing a distorted image by an imaging unit, and includes a plurality of continuous images as the input image. A first motion amount acquisition means for detecting a motion vector between them and acquiring a first motion amount from the motion vector; and a second motion from an output of the motion detection means for detecting a motion of the imaging means in the imaging. A second motion amount acquisition means for acquiring the amount, the third motion amount is acquired using the first motion amount and the second motion amount at a predetermined ratio, and the third motion amount is obtained. Correction amount acquisition means for acquiring a correction amount for the image blur correction using the correction amount acquisition means, the correction amount acquisition means, the image height in the input image, the movement amount of the imaging means in the imaging, and the imaging At least the subject distance And changes the predetermined ratio in response to one.

  Note that an imaging apparatus having the above-described image processing apparatus and imaging means also constitutes another aspect of the present invention.

An image processing program according to another aspect of the present invention is a computer program that causes a computer to execute image processing for performing image blur correction on an input image generated by imaging a distorted image by an imaging unit. In the image processing, a motion vector is detected between a plurality of consecutive images as the input image, a first motion amount is obtained from the motion vector, and a motion for detecting a motion of the imaging unit in the imaging A process for obtaining a second motion amount from the output of the detection means, and using the first motion amount and the second motion amount at a predetermined ratio, obtains a third motion amount, and a correction amount acquisition processing for acquiring the correction amount for the image blur corrected using the motion amount in the correction amount acquisition processing, image height in the input image, motion of the imaging unit in the imaging And changes the predetermined ratio in response to at least one of the object distance in the amount and the imaging.

According to the present invention, when image blur correction is performed on an image generated by capturing a distorted image using both the motion vector detection result and the motion detection result by the motion sensor, a more accurate image can be obtained. Shake correction can be performed .

1 is a block diagram illustrating a configuration of an image processing apparatus that is Embodiment 1 of the present invention. 3 is a flowchart showing image processing in Embodiment 1. 7 is a flowchart illustrating processing for calculating a detection ratio of first and second motion amounts in the first embodiment. FIG. 6 is a diagram for explaining a detection ratio for each weak condition in the first embodiment. The flowchart which shows the process which calculates the detection ratio in Example 2 of this invention. FIG. 5 is a diagram for explaining a motion vector detection method according to the first embodiment. FIG. 4 is a diagram for explaining a restriction on an image blur correction amount in the first embodiment. FIG. 6 is a diagram illustrating a condition in which motion amount detection by a gyro sensor and motion detection by a motion vector are not good in the first embodiment.

  Embodiments of the present invention will be described below with reference to the drawings.

  FIG. 1 shows a configuration of an imaging apparatus including an image processing apparatus 100 that is Embodiment 1 of the present invention. The imaging optical system L forms a subject image by forming a light beam from a subject (not shown). The imaging optical system L in the present embodiment is a fisheye optical system (for example, an all-around fisheye optical system), and forms a fisheye subject image that is a distorted image of the subject. The image sensor 110 is configured by a photoelectric conversion element such as a CCD sensor or a CMOS sensor, and converts a subject image into an electrical signal (analog signal). The imaging optical system L and the imaging element 110 constitute imaging means.

  The A / D converter 120 converts the analog signal output from the image sensor 110 into an image signal as a digital signal and inputs the image signal to the image processing apparatus 100. In the image processing apparatus 100, the control unit 108 includes an image signal processing unit 110, a storage unit 112, a motion detection unit 114, a shake information acquisition unit 116, an image shake correction amount calculation unit 118, an image shake correction unit 120, and an image cutout unit 122. And the flow of data in these. The image signal processing unit 110 performs image signal processing such as synchronization processing, white balance processing, γ processing, and noise reduction processing on the imaging signal input from the A / D converter 120 to obtain image data, that is, input A captured image as an image is generated. The captured image referred to in the present embodiment is a fisheye image (for example, an all-round fisheye image) generated by imaging through the imaging optical system L that is a fisheye optical system. The captured image referred to in the present embodiment includes a plurality of continuous frame images constituting a moving image and a plurality of still images generated by continuous imaging. The storage unit 112 stores (holds) the captured image generated by the image signal processing unit 110 or stores an image that has been subjected to image blur correction by the image blur correction unit 120 (hereinafter referred to as an image blur correction image). To do.

  The motion detection unit (first motion amount acquisition unit) 114 detects the motion amount of the imaging device including the image processing device 100 using the captured image stored in the storage unit 112.

  The shake information acquisition unit (second motion amount acquisition means) 116 includes a shake sensor (motion detection means) that detects a shake (motion) of the imaging apparatus, and uses the signal from the shake sensor to determine the shake amount of the imaging apparatus. Get (motion amount) information. The shake information acquisition unit 116 of the present embodiment uses a gyro sensor.

  The image shake correction amount calculation unit (correction amount acquisition unit) 118 uses the amount of motion detected by the motion detection unit 114 and the amount of shake acquired by the shake information acquisition unit 116 as will be described later to correct image shake. Determine the amount.

  An image blur correction unit (correction unit) 120 performs image blur correction on the captured image using the image blur correction amount determined by the image blur correction amount calculation unit 118. Hereinafter, the captured image that has been subjected to the image blur correction is referred to as an image blur correction image.

  The image cutout unit (image generation unit) 122 cuts out a partial image centered on a position (image height) designated by the user from the image shake correction image generated by the image shake correction unit 120 by correcting distortion (image generation unit). Hereinafter, this partial image is referred to as a cutout image). The image processing apparatus 100 outputs this cut-out image.

  The flowchart in FIG. 2 shows image processing performed by the image processing apparatus 100. The image processing apparatus 100 configured by a computer executes this image processing according to an image processing program as a computer program. The control of the image processing is performed by the control unit 108.

  First, in step S200, the image signal processing unit 110 reads captured images (a plurality of frame images or a plurality of still images: these are collectively referred to as a continuous image hereinafter) stored in the storage unit 112.

  Next, in step S202, the motion detection unit 114 detects (calculates) a motion vector in an image region corresponding to the cutout image cut out from the image blur correction image by the image cutout unit 122 in the subsequent step S210 among the continuous images. To do.

  Various motion vector detection methods have been proposed, but the motion detection unit 114 of this embodiment uses template matching shown in FIG. The motion detection unit 114 using template matching sets a template region 601 in the reference image 610. Then, an image of the template region 601 (hereinafter referred to as a template image) is scanned on the reference image 612, and an image region (hereinafter referred to as a template image) corresponding to the template image is selected as the image region having the highest similarity with the template image. Detected as a corresponding image). As the similarity, for example, a sum of absolute differences SAD (Sum Of Difference) is used. That is, among the plurality of image areas in the reference image 612, an image area having the smallest absolute value sum of pixel value differences from the template image is set as a corresponding image for the template image. Then, the motion detection unit 114 calculates a motion vector 621 from the difference between the position (coordinates) of the template image on the standard image 610 and the position of the corresponding image on the reference image 612.

  In this embodiment, as shown by a dotted line in FIG. 6, a plurality of template regions 601 (12 in the figure) are set, and the above-described motion vector is calculated for each template region 601.

  However, a low-contrast region such as the template region 602 in the figure is excluded from the motion vector calculation target because the motion vector cannot be calculated with high accuracy. As a result, in the example of FIG. 6, as shown on the reference image 612, seven motion vectors are calculated. The mode value among the motion amounts indicated by these seven motion vectors is set as the first motion amount. Further, the horizontal component of the first motion amount is X1, and the vertical component is Y1.

  Further, in step S204, the shake information acquisition unit 116 integrates the angular velocity signal from the gyro sensor and calculates the shake angle. In this embodiment, image blur correction is performed in each of the horizontal (Yaw) direction and the vertical (Pitch) direction. Let β be the shake angle in the horizontal direction and γ be the shake angle in the vertical direction. At this time, the horizontal shake amount X2 and the vertical shake amount Y2 are calculated by the following equations (1) and (2), respectively.

X2 = f · tanβ (1)
Y2 = f · tanγ (2)
Hereinafter, the shake amounts X2 and Y2 are collectively referred to as a second motion amount.

  Next, in step S206, the image signal processing unit 110 uses the first motion amount detected by the motion detection unit 114 and the second motion amount acquired by the shake information acquisition unit 116 to perform image blur correction. A correction amount acquisition process for calculating (acquiring) the amount is performed. The flowchart of FIG. 3 shows the processing performed in this step.

  Here, as described above, the motion detection unit 114 acquires the first motion amount by detecting the motion vector, and the shake information acquisition unit 116 acquires the second motion amount by detecting the shake by the gyro sensor. There are four conditions (bad conditions) that reduce the detection accuracy of each. FIG. 8 shows these four weak conditions. In the present embodiment, among the four weak conditions, a condition relating to image height will be described, and the other three conditions will be described in a second embodiment described later.

  In this embodiment, an all-around fish-eye optical system is used as the imaging optical system L. The all-around fish-eye optical system uses four types of projection methods: orthographic projection, equidistant projection, stereoscopic projection, and equisolid angle projection. Form an image. Orthographic projection and equi-solid angle projection are projection methods in which the distortion rate of the subject image (that is, the compression rate of the image) increases as the image height increases, and stereoscopic projection is a projection method in which the distortion rate increases as the image height decreases. It is. The equidistant projection is a projection method in which the distortion rate of the subject image is not changed regardless of the image height.

  In the present embodiment, as described with reference to FIG. 6, the motion vector is calculated using template matching. However, at high image heights in the projection method in which the image compression rate increases as the image height increases, such as orthographic projection and equisolid angle projection, the amount of deformation of the subject image is large, so when performing template matching , And the motion vector cannot be detected with high accuracy. Further, even with a low image height in the projection method in which the image compression rate increases as the image height decreases as in the case of a three-dimensional projection, a motion vector cannot be detected with high accuracy for the same reason. As a result, the detection accuracy of the first motion amount using the motion vector is affected by the image height of the image area in which the motion vector is detected in the continuous image, that is, the cut-out image height that is the image height of the cut-out image. On the other hand, the detection accuracy of the second motion amount by the gyro sensor is not affected by such an image height.

  In the column of “image height” in FIG. 8, as a weak condition in the projection method in which the image compression rate increases as the image height increases, such as orthographic projection and equisolid angle projection, the cut-out image height is “high image”. The case of “high” is shown. In the present embodiment, the detection ratio described later is set in consideration of the fact that the detection accuracy of the motion vector (that is, the first motion amount) decreases when the cut-out image height is high. In the projection method in which the compression rate of the image increases as the image height decreases as in the case of the stereoscopic projection, it is considered that the detection accuracy of the motion vector (that is, the first motion amount) decreases as the clipped image height decreases. Thus, the detection ratio may be set.

  In step S300 in FIG. 3, the image blur correction amount calculation unit 118 determines the first motion amount (X1, Y1) according to the cutout image height (image height specified by the user) as a condition (parameter) for motion vector detection. ) And the second motion amount (X2, Y2) are selected (set). In the present embodiment, the third motion amount is calculated by using (combining) the first motion amount and the second motion amount at a predetermined ratio. In the present embodiment, this predetermined ratio is referred to as a detection ratio (or use ratio or synthesis ratio).

  In the present embodiment in which the detection accuracy of the first motion amount decreases as the cutout image height increases, as shown in FIG. 4A, the ratio of using the first motion amount decreases as the cutout image height increases. The detection ratio α is calculated so that the ratio of using the second motion amount is high.

  In step S302, the image blur correction amount calculation unit 118 calculates the third motion amount (the horizontal component X3 and the vertical component Y3) by the following equations (3) and (4) using the detection ratio α.

X3 = α · X1 + (1−α) · X2 (3)
Y3 = α · Y1 + (1−α) · Y2 (4)
Subsequently, in step S304, the image blur correction amount calculation unit 118 calculates the image blur correction amount (−X3, −Y3) using the third motion amount (X3, Y3). When the image blur correction amount is determined in this way, the process proceeds to step S208 in FIG.

  In step S208, the image blur correction unit 120 performs image blur correction of the captured image using the image blur correction amount determined in step S206. Specifically, the captured image is shifted in parallel by the image blur correction amount (−X3, −Y3).

  Here, in the image blur correction in step S208, pixels existing around the area corresponding to the clipped image in the captured image are used as a correction image. If the image blur correction amount is too large, correction is performed in the captured image. It becomes impossible to secure pixels for use. For this reason, as shown in FIG. 7, the increase in the image blur correction amount with respect to the increase in the third motion amount may be limited so that the image blur correction amount is equal to or smaller than the predetermined value.

  Next, in step S210, the image cutout unit 122 generates a cutout image by correcting the distortion of the partial image having the image height specified by the user from the image blur correction image (fisheye image) generated in step S208. Any method may be used for distortion correction of a partial image from a fisheye image, but a method using a normal image conversion formula is employed as disclosed in, for example, Japanese Patent Application Laid-Open No. 2010-224691. May be.

  The control unit 108 outputs the cutout image from the image processing apparatus 100. The output of the cut-out image is performed on a display device such as a personal computer (not shown), a projector, or a monitor by wired or wireless communication, or is stored as a storage medium such as a semiconductor memory or an optical disk.

  Next, in step S212, the control unit 108 determines whether to generate and output another clipped image. If so, the process returns to step S200 and repeats the processes of steps S200 to S210. When the generation and output of another clipped image are not performed, the image processing is terminated.

  In the present embodiment, the third motion amount is obtained by using the first motion amount obtained from the motion vector and the second motion amount obtained by the gyro sensor at a detection ratio corresponding to the cut-out image height. The image blur correction amount is set from the motion amount of 3. As a result, it is possible to generate a cutout image in which image blur correction is favorably performed and distortion correction is performed from a fish-eye image whose compression rate (distortion rate) changes according to the image height.

  Next, image processing performed by the image processing apparatus according to the second embodiment of the present invention will be described with reference to FIG. In the image processing in the present embodiment, as the correction amount acquisition processing performed in step S206 in the flowchart illustrated in FIG. 2 in the first embodiment, the processing illustrated in the flowchart in FIG. 5 is performed instead of the processing illustrated in the flowchart in FIG. Further, the image processing in the present embodiment is executed according to an image processing program that is a computer program by an image processing device as a computer having the same configuration as the image processing device 100 of the first embodiment (FIG. 1). Components common to the first embodiment in the image processing apparatus are denoted by the same reference numerals as in the first embodiment. In the first embodiment, it has been described that there is a weak condition regarding the image height with respect to the detection of the motion vector (that is, the first motion amount). However, the shake amount (second motion amount) using the gyro sensor is described. There are also conditions that are not good at detecting this. In addition, there are conditions other than image height that are poor for detecting motion vectors.

  First, the weak condition regarding the vibration frequency will be described. The gyro sensor is generally inferior in the detection performance of low-frequency vibration to the detection performance of high-frequency vibration. On the other hand, regarding the detection of the motion vector, there is not much decrease in detection accuracy due to such a shake frequency. For this reason, in the present embodiment, regarding the detection of the second motion amount using the gyro sensor, the detection ratios of the first and first motion amounts are set in consideration that the detection accuracy is lowered when the shake frequency is low. .

  Next, a description will be given of poor conditions regarding the shake amount of the imaging apparatus (hereinafter referred to as camera shake amount) due to camera shake or the like. Also in the present embodiment, as described in the first embodiment, the motion vector is calculated using template matching. In a fish-eye image, the compression ratio changes depending on the image height. For this reason, if the camera shake amount is large, even if the same subject is imaged, the shape of the subject changes between successive images due to the change in the compression ratio due to the image height, and as a result, the similarity in template matching decreases and moves. Vector detection accuracy decreases. On the other hand, in the shake detection by the gyro sensor, there is no particular decrease in detection accuracy due to the camera shake amount. For this reason, in this embodiment, regarding the detection of the first motion amount using the motion vector, the detection ratio of the first and first motion amounts is set in consideration that the detection accuracy is lowered when the camera shake amount is large. To do.

  Next, conditions for poor subject distance will be described. As the subject distance increases, the translation component in the image blur amount decreases and the rotation component increases. For this reason, all of the plurality of motion vectors detected at a plurality of positions in the captured image obtained by imaging a distant subject indicate movement along the rotation direction. Therefore, as described in the first embodiment, even if the mode value of the motion amount indicated by these motion vectors is obtained, it is difficult to accurately obtain a motion vector representing the image shake of the entire image. On the other hand, in the shake detection by the gyro sensor, there is no particular decrease in detection accuracy due to the subject distance. For this reason, in the present embodiment, regarding the detection of the first motion amount using the motion vector, the detection ratio of the first and first motion amounts is set in consideration of the fact that the detection accuracy is lowered when the subject distance is long. .

  As described above, in the present embodiment, the first and second conditions depend on the cut-out image height, the camera shake amount, the subject distance, which are conditions for motion vector detection, and the shake frequency, which is a condition for shake detection by the gyro sensor. Change the motion amount detection ratio.

  In step S500 of FIG. 5, the image blur correction amount calculation unit 118 calculates the detection ratio of the first and second motion amounts as described with reference to FIG. 4A in step S300 of FIG. That is, the detection ratio is calculated such that the higher the cut-out image height, the lower the ratio of using the first motion amount and the higher the ratio of using the second motion amount. The detection ratio calculated here is defined as a first detection ratio α1.

  Next, in step S502, as shown in FIG. 4B, the image blur correction amount calculation unit 118 uses the second motion amount because the ratio of using the first motion amount decreases as the shake frequency increases. The detection ratio of the first and second motion amounts is calculated so that the ratio to be increased becomes high. The detection ratio calculated here is a second detection ratio α2.

  Next, in step S504, as shown in FIG. 4C, the image blur correction amount calculation unit 118 decreases the ratio of using the first motion amount as the camera shake amount increases, and sets the second motion amount. The detection ratios of the first and second motion amounts are calculated so that the ratio to be used becomes high. The detection ratio calculated here is a third detection ratio α3.

  Next, in step S506, the image blur correction amount calculation unit 118 uses the second motion amount as the ratio of using the first motion amount decreases as the subject distance increases as illustrated in FIG. The detection ratio of the first and second motion amounts is calculated so that the ratio to be increased becomes high. The detection ratio calculated here is a fourth detection ratio α4.

Next, in step S508, the image blur correction amount calculation unit 118 calculates the final detection ratio α using the first, second, third, and fourth detection ratios α1, α2, α3, α4, A third motion amount is calculated using the calculation ratio α. Further, an image blur correction amount is determined from the third motion amount. The final detection ratio α is calculated from the product ratio of α1 to α4, for example, as shown in the following formula (5).
α = α1, α2, α3, α4
/ [Α1, α2, α3, α4 + (1-α1) (1-α2) (1-α3) (1-α4)]
(5)
Further, as shown in the following equation (6), it may be calculated from the ratio of the results of the weighted addition of α1, α2, α3, and α4.
α = (k1, α1 + k2, α2 + k3, α3 + k4, α4)
/ [K1, α1 + k2, α2 + k3, α3 + k4, α4
+ K1 · (1-α1) + k2 × (1-α2)
+ K3 · (1-α3) + k4 · (1-α4)]
= (K1 · α1 + k2 · α2 + k3 · α3 + k4 · α4) / (k1 + k2 + k3 + k4)
(6)
Here, k1, k2, k3, k4 are weighting coefficients for α1, α2, α3, α4.

  In the equations (5) and (6), the case where the final detection ratio α is calculated using all of α1, α2, α3, and α4 has been described. However, the detection ratio α may be calculated by selecting and using at least one (or at least two) of α1, α2, α3, and α4.

  In this embodiment, the first motion amount obtained from the motion vector and the second motion amount obtained by the gyro sensor are used at a detection ratio corresponding to the cut-out image height, the shake frequency, the camera shake amount, and the subject distance. Thus, the third motion amount is obtained, and the image blur correction amount is set from the third motion amount. As a result, it is possible to generate a cut-out image that has been subjected to good image blur correction and distortion correction from a fisheye image.

  In the first and second embodiments, the image processing apparatus provided in the imaging apparatus has been described. However, an image processing apparatus separate from the imaging apparatus such as a personal computer may be used.

In the first and second embodiments, the case where the image is cut out from the fisheye image has been described. However, the image is obtained from an image having another distortion (an image generated by imaging through an imaging optical system that forms a distorted image). When cutting out, the image processing described in the first and second embodiments may be performed.
(Other examples)
The present invention supplies a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in a computer of the system or apparatus read and execute the program This process can be realized. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

  Each embodiment described above is only a representative example, and various modifications and changes can be made to each embodiment in carrying out the present invention.

DESCRIPTION OF SYMBOLS 100 Image processing apparatus 114 Motion detection part 116 Shake information acquisition part 118 Image blur correction amount calculation part 120 Image blur correction part 122 Image clipping part

Claims (9)

An image processing apparatus that performs image blur correction on an input image generated by imaging a distorted image by an imaging unit,
First motion amount acquisition means for detecting a motion vector between a plurality of consecutive images as the input image and acquiring a first motion amount from the motion vector;
Second movement amount acquisition means for acquiring a second movement amount from an output of a movement detection means for detecting movement of the imaging means in the imaging;
The third motion amount is obtained using the first motion amount and the second motion amount at a predetermined ratio, and the correction amount for the image blur correction is obtained using the third motion amount. Correction amount acquisition means to
The correction amount acquisition unit changes the predetermined ratio according to at least one of an image height in the input image, a movement amount of the imaging unit in the imaging, and a subject distance in the imaging. apparatus. In the case projection method before Symbol imaging means is a method that the distortion rate increases in the distorted image higher image height, the correction amount acquisition unit, the first movement in the predetermined ratio as the image height is high The image processing apparatus according to claim 1 , wherein a ratio of the amount is decreased to increase a ratio of the second motion amount. In the case projection method before Symbol imaging means is a method that the distortion rate increases in the distorted image the lower the image height, the correction amount acquisition unit, the first movement in the predetermined ratio the image height as a lower The image processing apparatus according to claim 1 , wherein a ratio of the amount is decreased to increase a ratio of the second motion amount. Before SL correction amount acquisition unit, to claim 1, characterized in that said back in about the amount of motion is large low ratio of the first amount of movement in the predetermined ratio to increase the ratio of the second amount of movement The image processing apparatus described. Before SL correction amount acquisition unit, to claim 1, wherein the object distance is to lower the farther ratio of the first amount of movement in the predetermined ratio to increase the ratio of the second amount of movement The image processing apparatus described. 2. The image blur correction amount acquisition unit limits an increase in the image blur correction amount with respect to an increase in the third motion amount so that the image blur correction amount is smaller than a predetermined value. 6. The image processing device according to any one of items 1 to 5 . Image according to any one of claims 1 to 6, characterized in that it comprises an image generating means for cutting the distortion correction to the partial image of the specified image height of the image blur correction is made an the input image Processing equipment. An image processing apparatus according to any one of claims 1 to 7 ,
An imaging apparatus comprising the imaging means. A computer program for causing a computer to execute image processing for performing image blur correction on an input image generated by imaging a distorted image by an imaging unit,
The image processing is
Detecting a motion vector between a plurality of consecutive images as the input image, and obtaining a first motion amount from the motion vector;
A process of obtaining a second motion amount from an output of a motion detection means for detecting a motion of the imaging means in the imaging;
The third motion amount is obtained using the first motion amount and the second motion amount at a predetermined ratio, and the correction amount for the image blur correction is obtained using the third motion amount. Correction amount acquisition processing to be performed,
In the correction amount acquisition processing, the predetermined ratio is changed according to at least one of an image height in the input image, a movement amount of the imaging unit in the imaging, and a subject distance in the imaging. program.