JP6231816B2 - IMAGING DEVICE, ITS CONTROL METHOD, PROGRAM, AND STORAGE MEDIUM - Google Patents

Description

  The present invention relates to an image stabilization technique for reducing image blur of an image.

  In a conventional digital camera, a motion vector between two frame images before and after is detected in time, an amount of positional deviation between the frame images is estimated using the detected motion vector, and an image is canceled out. The electronic anti-vibration function is realized by applying geometric deformation processing to. Here, as a method of detecting a motion vector between frame images, there are a method using a gyro sensor, a method of estimating from a frame image, and the like. There is also a method for estimating a motion vector between frame images using template matching.

  In the method of estimating a motion vector using the template matching, first, one of two frame images of a captured video is defined as an original image and the other is defined as a reference image. Then, a rectangular area having a predetermined size arranged on the original image is used as a template block, and a correlation with the luminance value distribution in the template block is obtained at each position of the reference image. At this time, the position where the correlation is highest in the reference image is the destination of the template block, and the direction to the destination and the amount of movement based on the position of the template block of the original image are the motion vectors. A motion between frame images is calculated as a geometric deformation amount by performing statistical processing or the like using the plurality of motion vectors thus obtained.

  Image blur motion components generated in an image include translation, rotation, scaling (enlargement and reduction), tilt, and the like, and the type of dominant image blur varies depending on the shooting situation. In view of this, there has been proposed a method for realizing better image stabilization control by changing the content of image stabilization processing according to the type of dominant image blur that varies depending on the shooting situation (for example, Patent Document 1).

JP 2007-166269 A

  The conventional electronic image stabilization function detects a motion vector for the entire region of the frame image, estimates the geometric deformation amount using the detection result, and gives the estimation result to the geometric deformation module. Power consumption becomes enormous.

  Further, in Patent Document 1, in order to improve the stability of the image stabilization process, the motion component of the image blur to be the image stabilization target is changed according to the shooting situation, and the image stabilization geometry is based on the image blur information. Although the deformation parameter estimation method has been changed, no attention has been paid to the reduction of calculation amount and power consumption. Further, there is no suggestion of changing a motion vector detection method in accordance with a motion component of image blur that is a subject of image stabilization.

The present invention has been made in view of the above problems, its object is to reduce the calculation amount required for the anti-vibration processing, implementing the techniques to reduce power consumption.

In order to solve the above-described problems and achieve the object, an imaging apparatus of the present invention is an imaging apparatus that corrects image shake of an image, and acquires an imaging unit that captures an image, and imaging parameters by the imaging unit. Using the acquisition means, the motion vector detection means for detecting a motion vector from the first image and the second image obtained from the imaging means, and the motion vector detected by the motion vector detection means, A geometric deformation parameter estimating means for estimating an image shake between the image and the second image as a geometric deformation parameter; and the first image and the first image using the geometric deformation parameter estimated by the geometric deformation parameter estimating means. and geometric deformation means for correcting the image blur by performing geometric deformation second image, on the basis of the imaging parameters, before Symbol geometric deformation parameter estimating means by geometric variables Having a stabilization control means for changing a parameter estimation method.

According to the present invention, to reduce the calculation amount required for the anti-vibration processing, power consumption can be suppressed.

The block diagram which shows the apparatus structure of embodiment which concerns on this invention. 5 is a flowchart illustrating image stabilization processing by the imaging apparatus according to the present embodiment. The figure explaining the motion vector detection method. The figure explaining the motion vector detection method using template matching.

  Hereinafter, embodiments for carrying out the present invention will be described in detail. The embodiment described below is an example for realizing the present invention, and should be appropriately modified or changed according to the configuration and various conditions of the apparatus to which the present invention is applied. It is not limited to the embodiment. Moreover, you may comprise combining suitably one part of each embodiment mentioned later.

[Embodiment 1]
Hereinafter, an embodiment in which the present invention is applied to an imaging device such as a digital video camera that captures a moving image or a still image will be described.

  <Apparatus Configuration> First, with reference to FIG. 1, an outline of the configuration and functions of an imaging apparatus according to an embodiment of the present invention will be described.

  The imaging optical system 101 includes a lens group including a zoom lens and a focus lens, and a diaphragm mechanism. The imaging unit 102 includes an image sensor including an image sensor such as a CCD or a CMOS, and photoelectrically converts a subject image formed on the image sensor by the imaging optical system 101 to convert it into an electrical signal.

  The development processing unit 103 includes an AD conversion unit, an AGC (auto gain control unit), and an AWB (auto white balance unit). The development processing unit 103 converts an analog signal output from the image sensor into, for example, a 12-bit digital signal by an AD conversion unit, and generates a video signal by performing signal level correction, white level correction, and the like by AGC and AWB. .

  The memory 104 temporarily holds one or a plurality of frame image data of the video signal generated by the development processing unit 103.

  The imaging parameter acquisition unit 105 acquires the focal length information of the lens of the imaging optical system 101. Note that the focal length information is changed according to the zoom magnification when the imaging optical system 101 is a zoom lens.

  The image stabilization control unit 106 changes the processing contents of a motion vector detection unit 107 and a geometric deformation parameter estimation unit 108 described later based on the focal length information obtained by the imaging parameter acquisition unit 105.

  The motion vector detection unit 107 reads the frame image generated by the development processing unit 103 and held in the memory 104, and detects the motion vector from the difference between the two previous and subsequent frame images. As a motion vector detection method, there is a method using template matching described later.

  The geometric deformation parameter estimation unit 108 uses the motion vector detected by the motion vector detection unit 107 to calculate a correction amount for correcting image blur between frame images caused by camera shake of the imaging apparatus. Calculate as a parameter.

  The geometric deformation processing unit 109 performs a geometric deformation process for correcting image blur on the frame image based on the geometric deformation parameter of the image stabilization process. Here, the geometric deformation processing includes at least one of translation, rotation, scaling (enlargement and reduction), and tilting.

  The image recording unit 110 compresses the video data whose image blur has been corrected by the image stabilization process into a predetermined recording format and records the compressed data on a recording medium. Further, the image recording unit 110 reads out video data from the recording medium, expands it, and holds it in the memory 104. The recording format in the image recording unit 110 is JPEG compression for still image data, Motion JPEG or H.264 for moving image data. H.264 / AVC compression or the like is applied. In particular, H.C. The compression method of moving image data represented by H.264 / AVC can achieve a high compression rate by using interframe reference or the like. The recording medium is a built-in memory such as a hard disk or a removable memory card.

  The image display unit 111 displays the video data held in the memory 104 on a display device such as an LCD panel or an organic EL display.

  In addition to the above-described configuration, the imaging apparatus of the present embodiment controls AF (autofocus) processing and AE (automatic exposure) by controlling the lens of the imaging optical system 101 and the imaging device of the imaging unit 102. Processing, AWB (auto white balance) processing is performed.

  <Anti-Vibration Processing> Next, with reference to FIG. 2, the anti-vibration processing by the imaging apparatus of this embodiment will be described.

  2 is realized by the image stabilization control unit 106 executing a firmware program stored in a nonvolatile memory (not shown).

  In FIG. 2, in step S <b> 201, the subject image input from the imaging optical system 101 is photoelectrically converted by the imaging unit 102 and output to the development processing unit 103 as an analog signal corresponding to the subject brightness. A video signal is generated. Frame images sequentially generated at a predetermined frame rate by the development processing unit 103 are held in the memory 104, and the frame images held in the memory 104 are output to the motion vector detection unit 107 and are stored in the memory 104. Images are updated sequentially.

  In step S202, the imaging parameter acquisition unit 105 acquires the focal length information of the lens of the imaging optical system 101.

  In step S <b> 203, the image stabilization control unit 106 selects the detection area and the number of detections of the motion vector detected by the motion vector detection unit 107 based on the focal length information of the imaging apparatus obtained from the imaging parameter acquisition unit 105.

  Table 1 shows the relationship between the image blur motion component to be image-stabilized, the motion vector detection area and the number of motion vector detections necessary for accurately correcting the motion component. FIG. 3 shows the relationship between the image blur motion component and the detection area. As can be seen from Table 1, it is necessary to detect a large number of motion vectors from the entire screen when correcting the movement of the tilt as the motion component of the image blur that is the subject of image stabilization. The reason is as follows.

  FIG. 3A schematically shows the magnitude and direction of a motion vector detected when vertical tilt occurs between frame images. 301 indicates the entire frame image, and 302 indicates the motion of the tilt. A motion vector to be represented, 303, indicates a region where motion vectors are detected. As shown in FIG. 3A, since the movement of the tilt is different in the magnitude and direction of the motion vector at each position on the image, if the motion vector is not detected from the entire screen, the movement of the tilt is accurately estimated. I can't. Further, for a complicated motion such as a tilt, the estimation accuracy cannot be stabilized unless a large number of motion vectors are used. For these reasons, when the movement of the tilt is targeted for image stabilization, it is necessary to detect a large number of motion vectors from the entire frame image like the detection area 303.

  In addition, in the case where the rotational motion component is the object of image stabilization, a motion vector may be detected from an area near the center of the screen. Here, the rotational motion vector generated on the image is a point-symmetrical motion with the center of the image as the rotational center, as indicated by 304 in FIG. Therefore, in order to estimate the rotational motion, it is only necessary to detect a motion vector pair having a point-symmetric positional relationship with respect to the image center, and it is not always necessary to detect a motion vector from the entire screen. That is, a motion vector necessary for detecting a rotational motion is detected even if an area including the image center as indicated by reference numeral 305 in FIG. 3B and having a narrow vertical direction is used as a motion vector detection area. I can. In this case, since the image is input in a raster manner from the top to the bottom, the detection of the motion vector is completed earlier than the input of the entire image is completed by arranging the detection region 305 at the top of the image. be able to. In addition, the number of motion vectors to be detected can be reduced by the amount that the detection area is narrower than the tilt detection area 303. From the above, by using the rotational motion component as the object of image stabilization, the processing time can be shortened and the amount of calculation can be reduced as compared with the case where the tilt movement is the object of image stabilization.

  On the other hand, when only the translational motion as shown in the motion vector group 306 in FIG. 3C is targeted for image stabilization, a very small region on the image such as the detection region 307 is used. What is necessary is just to detect a motion vector. When the dominant motion of the imaging device is a translational motion, the detected motion vector has the same magnitude and direction at any position on the image. Therefore, there is no restriction on the position where the motion vector is detected, and the translational motion can be accurately estimated from a smaller number of motion vectors than the rotational motion. Also, unlike the case of rotational movement, it is not necessary to detect a motion vector with a point-symmetrical positional relationship, so that the detection region can be arranged at the upper part of the image. As a result, when estimating the translational movement, it is possible to further reduce the amount of calculation and the processing time.

  As described above, the amount of calculation is reduced by changing the detection area and the number of detections of the motion vector according to the type of the motion component of the image blur to be corrected, thereby shortening the processing time and power consumption. Is possible.

  Here, the image blur motion component of the detection target described at the top of Table 1 includes the image blur motion component at the lower level. For example, when a motion vector detection region and the number of detections for the tilt motion are selected, it is possible to detect a motion vector for estimating rotational and translational geometric deformation components at the same time.

  In the present embodiment, the example in which the motion vector detection region and the number of detections are changed based on tilt, rotation, and translation as the motion component of the image blur has been described. However, the present invention is not limited to this. The same applies to motion components such as reduction and shear.

  Returning to the description of FIG. 2, the information on the motion vector detection region and the number of detections set in step S <b> 203 is sent to the motion vector detection unit 107.

  In step S204, the motion vector detection unit 107 detects a motion vector between two frame images based on the motion vector detection region and the number of detections set by the image stabilization control unit 106. In this embodiment, a method using template matching will be described as an example of a motion vector detection method.

  FIG. 4 is a diagram for explaining a motion vector detection method using template matching.

  4A shows an original image, and FIG. 4B shows a reference image. These images are frame images input from the development processing unit 103 and the memory 104. As shown in the figure, a template block 401 is arranged at an arbitrary position in the original image, and a correlation value between the template block 401 and each region of the reference image is calculated. In this case, calculating the correlation value for the entire area of the reference image requires a large amount of computation, so that a rectangular area for calculating the correlation value on the reference image is actually set as the search range 402. Here, the position and size of the search range 402 are not particularly limited, but a correct motion vector cannot be detected unless the region corresponding to the movement destination of the template block 401 is included in the search range 402. .

  In this embodiment, a sum of absolute difference (hereinafter abbreviated as SAD) is used as an example of a correlation value calculation method. SAD is obtained from Equation 1.

  In Equation 1, f (i, j) represents the luminance value at the coordinates (i, j) in the template block 401, and g (i, j) represents the correlation value calculation target in the block 403 in the search range 402. Each luminance value is represented. The SAD can obtain the correlation value S_SAD by calculating the absolute value of the difference between the luminance values f (i, j) and g (i, j) in both blocks and obtaining the sum thereof. Therefore, the smaller the correlation value S_SAD, the smaller the difference in luminance value between the two blocks, that is, the texture in the block 403 in the correlation value calculation area is similar to the template block 401.

  In the present embodiment, SAD is used as an example of the correlation value, but the present invention is not limited to this, and other correlation values such as sum of squares of differences (SSD) and normalized cross-correlation (NCC) may be used. . In this case, the correlation value is calculated by moving the correlation value target block 403 for the entire region of the search range 402. Then, by calculating a correlation value between the template block 401 and the search range 402 and determining a position where the correlation is highest, the position where the template block on the original image has moved in the reference image, that is, between the images. Motion vectors can be detected.

  The motion vector detection process as described above is performed in a plurality of regions between the input frame images, and the detected motion vector group is sent to the geometric deformation parameter estimation unit 108.

  In step S <b> 205, the image stabilization control unit 106 selects a geometric deformation parameter estimation method according to the type of motion component of image blur that is the object of image stabilization. In the present embodiment, a case where a 3 × 3 determinant called a homography matrix is used as an example of a means for expressing geometric deformation will be described.

  First, a certain point a on the image,

Is the point a ′ in the next frame,

Suppose you move to. Here, the subscript T represents a transposed matrix. The correspondence between the point a in Equation 2 and the point a ′ in Equation 3 is obtained by using the homography matrix H:

It can be expressed as.

  The homography matrix H is a determinant showing the amount of deformation caused by translation, rotation, scaling, shearing, and tilting between images, and can be expressed by the following equation.

  Each element of the homography matrix H can be obtained by performing an estimation process using the motion vector group obtained in step S204, that is, the correspondence of representative points between frame images.

  Table 1 shows a method for estimating a geometric deformation parameter with respect to a motion component of image blur as an image stabilization target. From Table 1, it is necessary to estimate the geometric deformation parameter using the least-squares method with 8 degrees of freedom when the movement of the tilt is targeted for vibration isolation. Here, in the homography matrix H of Equation 5, parameters h13 and h23 represent translational motion components, h11, h12, h21 and h22 represent rotational, scaling, and shearing motion components, and h31 and h32 represent lateral motions. Represents ingredients. Therefore, when the tilt motion component is the object of image stabilization, it is necessary to use the 8-square degree least-squares method because all the 8 parameters must be estimated.

  Next, in the case where the rotational motion component is the object of image stabilization, it is not necessary to estimate h31 and h32 which are parameters representing the tilting motion component, so the other six parameters are estimated, that is, 6 It suffices to perform estimation by the least square method with degrees of freedom. Therefore, when the motion component of the image blur to be corrected is up to the rotational motion component, it is possible to obtain a stable result even if the number of vectors is small because the degree of freedom is smaller than in the case of tilting. Yes, the amount of calculation can be reduced.

  When only the translational motion component is the object of image stabilization, it is sufficient to estimate only the parameters h13 and h23 in Equation 5. In addition, since the translational motion component is a simple motion such as translation of the image, there is no need to use a statistical estimation method such as the least square method, and the estimation is performed with high accuracy even by majority processing such as histogram processing. It becomes possible.

  In addition, if only translational motion is occurring on the image, the motion will be the same in the same direction and in the same direction at any position on the image, so the translational motion can be accurately estimated with a small number of motion vectors. Is possible.

  As described above, in the estimation of the translational geometric deformation amount, the parameter can be estimated by a simple estimation method using a small number of motion vectors, so that the calculation amount can be greatly reduced.

  In the present embodiment, the least square method or the histogram processing is used as the geometric deformation amount estimation method, but the present invention is not limited to this, and other estimation methods may be used.

  In step S205, the amount of calculation is reduced by changing the estimation method in accordance with the motion component of the image blur that is the object of image stabilization, but the total amount of calculation is the detection of the motion vector set in step S203. Determined in combination with the method. From Table 1, it is necessary to detect a large number of motion vectors from the entire screen for tilt motion components, and to calculate all eight geometric deformation parameters shown in Equation 5 by statistical processing, so that the amount of computation increases. To do. On the other hand, for example, for translational geometric deformation, it is sufficient to detect a small number of motion vectors from a partial area of the screen, and the estimation method can be as simple as histogram processing. Can be reduced.

  Information on the geometric deformation amount estimation method selected as described above is sent to the geometric deformation parameter estimation unit 108.

  In step S206, the amount of geometric deformation between the frame images is estimated using the motion vector group obtained from the motion vector detection unit 107 and the geometric deformation amount estimation method selected by the image stabilization control unit 106. The amount of correction is calculated.

  The homography matrix H estimated by the method described in step S205 represents the amount of deformation of the image due to image blurring. For this reason, in order to correct the image blur of the image, it is necessary to convert the homography matrix H so that the image deformation amount cancels the deformation due to the image blur. That is, by converting the homography matrix H to the inverse matrix H, the correspondence between the points a ′ and a is

It can be expressed as. Expression 6 makes it possible to return the point a ′ after image blurring to the same coordinates as the point a before image blurring.

  According to the above-described embodiment, it is possible to reduce the amount of calculation by switching between the motion vector detection method and the geometric deformation amount estimation method according to the type of the motion component of the image blur that is the image stabilization target.

  Next, a method for determining the type of motion component of image blur that is the object of image stabilization will be described.

  In the present embodiment, since the focal length information obtained by the imaging parameter acquisition unit 105 is used as the information for performing the determination, it is necessary to consider the relationship between the magnitude of the motion component of image blur and the focal length.

  Here, the magnitude of the image blur motion component means the amount by which a pixel of interest on the image moves when the image is deformed by homography representing the image blur motion component. For example, with respect to the four corner points of the image, the movement amounts by homography representing the motion component of the image shake are respectively obtained, and the one having the maximum absolute value is set as the size of the component.

  The homography R representing the image shake movement of the image can be expressed as the following Expression 7, where α, β, and γ are image shake angles in the yaw direction, the pitch direction, and the roll direction.

  Here, for the sake of simplicity, only the rotational movement of the imaging device is considered, but the same can be said for the following when the translational movement is taken into consideration.

  The homography for correcting the movement of the image shake of the image can be expressed as follows by taking the inverse matrix of Equation 7.

  In order to simplify Expression 8, when normalized by the (3, 3) element, the following Expression 9 is obtained.

  Furthermore, in order to consider the influence of the shooting parameters, when normalization is performed using the shooting parameter matrix K of Expression 10, it can be expressed as Expression 11.

  Here, f represents the focal length in pixel units, and cx and cy represent the X coordinate and Y coordinate of the optical axis center. For simplicity, it was considered that cx = cy = 0.

  By the way, the homography H representing the deformation amount of the image can be expressed as the following Expression 12 using the translation, rotation, zooming, shear, and tilt components.

  Here, Hs is a homography that represents the amount of deformation of the image due to translation, rotation, and scaling, Ha is a homography that represents the amount of deformation of the image due to shear, and Hp is a homography that represents the amount of deformation of the image due to tilt.

  Also, vx and vy are horizontal and vertical tilt components, θ is a rotation component based on the image center, tx and ty are horizontal and vertical translation components, s is a magnification component, and a and φ are shearing Represents an ingredient.

  Expression 12 is considered to represent the homography of Expression 11 using translation, rotation, zooming, shearing, and tilt components. Since both are equivalent, vx, vy, θ, tx, and ty can be expressed as follows using the focal length f and α, β, and γ.

  Here, if α, β, and γ are constant without depending on the focal length, the following can be said with respect to the relationship between the magnitude of the image blur motion component and the focal length.

  From Equation 13, the tilt component is inversely proportional to the focal length. From Equation 14, the rotation component is constant regardless of the focal length. From Equation 15, the translational component is proportional to the focal length.

  However, when the imaging apparatus is actually used, α, β, and γ tend to become smaller as the focal length becomes longer because the photographing method changes according to the focal length.

  When the focal length is short, α, β, and γ tend to be large. This is because the angle of view (the range that can be captured by the imaging device) is large, so that even if the image shake of the image is large, the subject tends to fit on the screen, and the photographer does not intentionally suppress the image shake. For this reason, when the focal length is short, there are many photographing methods such as walk shooting, and α, β, and γ increase.

  When the focal length is long, α, β, and γ tend to be small. This is because, since the angle of view is small, if the image blur of the image is large, the subject does not fit on the screen, and the photographer often intentionally suppress the image blur. Therefore, when the focal length is long, there are many photographing methods that are firmly fixed, such as two-handed photographing, and α, β, and γ are reduced.

  When the focal length is medium, it is positioned between the short and long cases, so that the image shake is smaller than the walk shot and the image shake is larger than the two-hand hold shot. The number of methods increases, and α, β, and γ are considered to be intermediate values.

  Finally, the relationship between the magnitude of the image blur motion component and the focal length is considered as follows, considering that α, β, and γ become smaller as the focal length becomes longer.

  The tilt component is inversely proportional to the focal length from Equation 13, but α and β become smaller as the focal length becomes longer, and therefore, the tilt component becomes abruptly smaller. The rotational component is γ itself from Equation 14, and decreases as the focal length increases. Although the translational component is proportional to the focal length from Equation 15, α, β, and γ decrease as the focal length increases, so that they cancel each other, and the variation due to the focal length is small.

Table 2 shows the relationship between the motion component of image blur and the focal length, and Table 2 shows the following. That is,
When the focal length is short, the tilt, rotation, and translation components are large. On the other hand, when the focal length is medium, the rotation and translation components are large and the tilt component is small. When the focal length is long, the translation component is large and the tilt and rotation components are small.

Therefore, the determination of the type of the image blur motion component that is the object of image stabilization should be performed as follows according to the focal length.
(1) When the focal length is shorter than the predetermined threshold T1, all components of the tilt, rotation, and translation components are determined as the image stabilization targets.
(2) When the focal length is between the predetermined threshold values T1 and T2 (T1 <T2), only the dominant rotation and translation components are determined as the image stabilization target.
(3) When the focal length is longer than the predetermined threshold value T2, only the dominant translation component is determined as the image stabilization target.

  How the motion vector detection method and the geometric deformation parameter estimation method are switched with respect to the determination result is as described in Table 1.

  As described above, according to the present embodiment, the type of the dominant image shake motion component is determined from the focal length information of the imaging apparatus, and the motion vector detection method and the geometric deformation parameter estimation are performed based on the determination result. Switch methods.

  As a result, it is possible to reduce power consumption by reducing the amount of calculation required for the image stabilization process while performing excellent image stabilization with suppressed dominant image blur.

  According to this embodiment, as the focal length is longer, the motion vector detection area can be narrowed and the number of geometric deformation parameters to be estimated can be reduced, so that the amount of calculation can be reduced and the power consumption can be reduced. .

  However, if the image stabilization processing contents are switched as they are during shooting, the geometric deformation model changes before and after the switching, and therefore, discontinuous movement may occur. As an example of a method for solving this, there is a method of performing a temporal smoothing process on the geometric deformation parameter.

  This is because the geometric deformation parameters for the past several frames are stored during the image stabilization process, and the geometric deformation parameters taking into account the influence of the past geometric deformation parameters are generated for the few frames immediately after switching. This is a method for performing the image stabilization process. Thereby, it becomes possible to suppress discontinuity at the moment of switching and gradually switch to the image stabilization result by different estimation methods. Here, the smoothing method is not particularly limited, and any method such as a moving average or an IIR filter may be used. The above processing is performed in the geometric deformation parameter estimation unit 108.

  Returning to the description of FIG. 2, in step S207, the geometric deformation processing unit 109 performs geometric conversion processing on the image by using the geometric deformation amount for image stabilization obtained in step S206 to perform image stabilization.

  In step S208, the image data subjected to the image stabilization process is stored in a storage medium by the image recording unit 110 or displayed on a display device by the image display unit 111.

  As described above, according to the present embodiment, the amount of calculation required for the image stabilization process is reduced by changing the motion vector detection method and the image geometric deformation amount estimation method according to the focal length information of the imaging device. To do. As a result, it is possible to perform an effective image stabilization process in accordance with the camera shake of the imaging apparatus, thereby reducing power consumption.

  In this embodiment, the focal length is taken as an example of the shooting parameter, but the subject distance obtained by the AF sensor can also be used when the focal length cannot be obtained as the shooting parameter. In this case, when the subject distance is short, it is determined that the subject is shot with a short focal length so that the subject is captured in a size that fits on the screen, and the above-described image stabilization processing when the focal length is short I do. On the other hand, if the subject distance is long, it is determined that the subject is being photographed with a long focal length so that the subject is shown on the screen larger, and the above-described image stabilization processing is performed when the focal length is long.

  In the above-described embodiment, the case where the present invention is applied to an imaging apparatus such as a digital video camera has been described as an example. However, the present invention is not limited to this. For example, the present invention can be applied to an apparatus equipped with a photographing function such as a game machine, a mobile phone, a smartphone, and a tablet terminal. Further, the present invention can be applied even when an image processing device such as a personal computer is configured to detect a motion vector and correct image blur for image data acquired from a device having a photographing function. is there.

[Other Embodiments]
The present invention is also realized by executing the following processing. That is, software (program) that realizes the functions of this embodiment is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, etc.) of the system or apparatus reads out and executes the program. It is processing to do.

Claims (11)

An imaging apparatus that corrects image shake of an image,
Imaging means for taking an image;
Obtaining means for obtaining imaging parameters by the imaging means;
Motion vector detection means for detecting a motion vector from the first image and the second image obtained from the imaging means;
Geometric deformation parameter estimation means for estimating an image shake between the first image and the second image as a geometric deformation parameter using the motion vector detected by the motion vector detection means;
Geometric deformation means for correcting image blur by performing geometric deformation on the first image and the second image using the geometric deformation parameter estimated by the geometric deformation parameter estimation means;
Based on said imaging parameters, before Symbol imaging apparatus comprising: the image stabilization control means, for changing the geometric deformation parameter estimation means according geometric deformation parameter estimation methods. The image stabilization control unit changes the detection method of the motion vector by the motion vector detection unit based on the shooting parameter, and determines the size of the detection area of the motion vector by the motion vector detection unit according to the shooting parameter. The imaging apparatus according to claim 1, wherein the number of detections is changed. The motion component of image blur to be detected by the motion vector detection means includes at least one of translation, rotation, and tilt,
The size of the detection area of the motion vector becomes smaller as the number of types of image blur motion components to be detected by the motion vector detection means decreases.
The imaging apparatus according to claim 2, wherein the number of motion vectors detected decreases as the number of types of image blur motion components to be detected by the motion vector detection unit decreases. The shooting parameter includes either a focal length or a subject distance of the imaging device,
The image stabilization control means, the more the longer the focal length as the imaging parameters, the claim motion vector detecting means and controlling such type of motion component of image vibration detection target is reduced 2 Or the imaging device of 3 . When the focal length cannot be obtained as the imaging parameter, the image stabilization control unit changes the motion vector detection method and the geometric deformation parameter estimation method using the subject distance,
5. The control according to claim 4, wherein when the subject distance is long, control is performed so that there are fewer types of motion components of the image blur that are subject to image stabilization than when the subject distance is short. Imaging device. The image stabilization control means, in response to said imaging parameters, imaging according to any one of the geometrical transformation parameter to estimating means claims 1 and changing the number of geometric deformation parameters for calculating 5 apparatus. The shooting parameter includes either a focal length or a subject distance of the imaging device,
7. The image stabilization control unit according to claim 6, wherein the image stabilization control unit controls the number of geometric deformation parameters calculated by the geometric deformation parameter estimation unit to decrease as the focal length increases as the imaging parameter. Imaging device.   The imaging apparatus according to claim 6 or 7, wherein the geometric deformation parameter estimation means performs a smoothing process using a geometric deformation parameter calculated in the past on the calculated geometric deformation parameter. An image pickup apparatus control method for correcting image blur of an image,
An acquisition step of acquiring imaging parameters by the imaging means;
A motion vector detection step of detecting a motion vector from the first image and the second image obtained from the imaging means;
A geometric deformation parameter estimation step for estimating an image shake between the first image and the second image as a geometric deformation parameter using the motion vector detected by the motion vector detection step;
A geometric deformation step of correcting image blur by performing geometric deformation on the first image and the second image using the geometric deformation parameter estimated by the geometric deformation parameter estimating step;
Based on said imaging parameter, the control method of the preceding Symbol geometric deformation parameters and image stabilization control step of changing the estimation step by geometrical transformation parameter estimation method, an imaging apparatus characterized by having a. The program for functioning a computer as each means of the imaging device described in any one of Claims 1 thru | or 8 . A storage medium storing a program for causing a computer to function as each unit of the imaging apparatus according to any one of claims 1 to 8 .

Images