JP6960324B2 - Blur amount calculation device, blur amount correction device, image pickup device, method, and program - Google Patents

Description

The present invention relates to a blur amount calculation device, a blur amount correction device, an image pickup device, a method, and a program.

When a subject or the like is imaged by an image pickup device or the like, if the position of the image pickup device is moved to take an image, the captured image may be blurred. In such a case, a technique has been known that detects the amount of movement of the imaging device and moves the lens or the like of the imaging device based on the detected amount of movement to reduce the blurring that occurs in the captured image. (See, for example, Patent Documents 1 to 4).
Patent Document 1 Japanese Patent Application Laid-Open No. 2014-21464 Patent Document 2 Japanese Patent Application Laid-Open No. 7-22405 Patent Document 3 Japanese Patent Application Laid-Open No. 2005-114845 Patent Document 4 Japanese Patent Application Laid-Open No. 2013-54193

However, if the positions of optical elements such as lenses and camera modules and the position of the acceleration sensor that detects the moving direction of the image pickup device are separated from each other inside the image pickup apparatus, blurring can be reduced correctly. It could be difficult. In particular, in a mobile terminal or the like having a function as an image pickup device, the mounting space inside is limited, and the arrangement of optical elements and sensors cannot be adjusted appropriately. Therefore, there has been a demand for a method and an apparatus for accurately reducing blur even when the optical element and the acceleration sensor are separated from each other.

In the first aspect of the present invention, it is a blur amount calculation device that calculates the blur amount of the imaging unit relative to the lens that can move in the first direction and the second direction, and is the first based on the output of the acceleration sensor. A movement amount calculation unit that calculates the first movement amount of the acceleration sensor in one direction, an angle change calculation unit that calculates the first angle change amount of the imaging unit based on the output of the angular velocity sensor, and a first movement amount, the first A blur amount calculation device, method, and program including a blur amount calculation unit that calculates the blur amount of the image pickup unit in the first direction based on one angle change amount and the positional relationship between the image pickup unit and the acceleration sensor. offer.

In the second aspect of the present invention, the blur amount calculation device of the first aspect, the first acquisition unit for acquiring the acceleration detection result of the acceleration sensor, and the detection result of the angle change amount of the angular velocity sensor are acquired. 2 The acquisition unit, the correction signal generation unit that generates a correction signal that corrects the position of the lens according to the blur amount of the imaging unit calculated by the blur amount calculation unit, and the drive unit that moves the sensor based on the correction signal. Provided is a blur amount correction device including a drive signal generation unit that generates a drive signal to be supplied.

In the third aspect of the present invention, the lens, the acceleration sensor, the angular velocity sensor, the imaging unit that captures the image formed by the lens, the driving unit that moves the lens, and the blur correction in the second aspect. An imaging device including the device is provided.

The outline of the above invention does not list all the necessary features of the present invention. Sub-combinations of these feature groups can also be inventions.

An example of the image pickup apparatus 10 according to this embodiment is shown. An example in which the image pickup apparatus 10 according to the present embodiment _{moves in the θZ direction is shown.} A first calculation example of the amount of blurring of the image pickup apparatus 10 according to the present embodiment is shown. A second calculation example of the amount of blurring of the image pickup apparatus 10 according to the present embodiment is shown. A third calculation example of the amount of blurring of the image pickup apparatus 10 according to the present embodiment is shown. A fourth calculation example of the amount of blurring of the image pickup apparatus 10 according to the present embodiment is shown. A fifth calculation example of the amount of blurring of the image pickup apparatus 10 according to the present embodiment is shown. A sixth calculation example of the amount of blurring of the image pickup apparatus 10 according to the present embodiment is shown. An example of the case where the image pickup apparatus 10 according to the present embodiment causes blurring on the XY surface is shown. An example of the case where the image pickup apparatus 10 according to the present embodiment causes blurring on the ZX surface is shown. An example of the case where the image pickup apparatus 10 according to the present embodiment causes blurring on the YZ surface is shown. A configuration example of the image pickup apparatus 10 according to the present embodiment is shown. A configuration example of the blur amount correction device 100 according to the present embodiment is shown. A configuration example of a computer 1200 in which a plurality of aspects of the present invention can be embodied in whole or in part is shown.

Hereinafter, the present invention will be described through embodiments of the invention, but the following embodiments do not limit the inventions that fall within the scope of the claims. Also, not all combinations of features described in the embodiments are essential to the means of solving the invention.

FIG. 1 shows an example of the image pickup apparatus 10 according to the present embodiment. FIG. 1 shows an example in which the image pickup apparatus 10 images the subject 20. In FIG. 1, a lens and an image sensor are provided on a surface facing the subject 20, and a display unit for displaying an image formed by the lens on the image sensor is provided on a surface facing the user on the side opposite to the subject 20. An example of the device 10 is shown. Here, if the user holding the image pickup device 10 moves the position of the image pickup device 10, disturbance due to camera shake will occur in the captured image. For example, the user may move the imaging device 10 in directions such as the X direction, the Y direction, the Z direction, the angle θ _X , the angle θ _Y , and the angle θ _{Z during imaging.} The angle θ _X indicates the angle around the X axis, the angle θ _Y indicates the angle around the Y axis, and the angle θ _Z indicates the angle around the Z axis.

The image pickup apparatus 10 detects its own moving direction and moves an optical element such as a lens in the direction opposite to the detected moving direction to correct such camera shake. Here, when the image pickup device 10 is provided with an acceleration sensor inside in order to detect the movement of the image pickup device 10 in the X direction, the Y direction, and the Z direction, there is an error in the detection position depending on the arrangement location of the acceleration sensor. This may occur and the accuracy of camera shake correction may deteriorate. Such deterioration in accuracy tends to increase as the distance between the image pickup apparatus 10 and the subject 20 becomes closer.

Further, when the image pickup device 10 is provided as a part of a mobile terminal such as a mobile phone, a tablet, a notebook computer, and a small personal computer, the components are arranged in a limited space. Therefore, the arrangement of the optical element and the acceleration sensor of the image pickup apparatus 10 may not be suitable for the camera shake correction, and the accuracy of the camera shake correction may be deteriorated. The image pickup apparatus 10 according to the present embodiment reduces the deterioration of the accuracy of camera shake correction in consideration of the error based on the arrangement of the optical element and the acceleration sensor. Therefore, first, the error of the detection position according to the arrangement of the acceleration sensor will be described.

FIG. 2 shows an example in which the image pickup apparatus 10 according to the present embodiment _{moves in the θ Z direction.} FIG. 2 shows an example of three orthogonal directions on the XYZ axes. In FIG. 2, the image pickup device before movement is shown as the image pickup device 10. FIG. 2 shows an example in which the image pickup apparatus 10 is arranged substantially parallel to the X direction and the Y direction and substantially perpendicular to the Z direction. Further, FIG. 2 shows an example in which the long side of the display surface of the image pickup apparatus 10 is substantially parallel to the Y direction and the short side is substantially parallel to the X direction. FIG. 2 will be described focusing on the Y direction as an example of the position detection error generated in the image pickup apparatus 10. The image pickup apparatus 10 includes a lens 12, an image pickup section 30, an acceleration sensor 14, an angular velocity sensor 16, and a display section 18.

The lens 12 and the imaging unit 30 are provided on the surface of the imaging device 10 on the subject side, which is the surface on the −Z direction side. The imaging unit 30 is provided relative to the lens 12. The lens 12 may be formed so as to be movable in the X direction, the Y direction, and the Z direction. Further, the lens 12 may be formed so as to be movable in the pitch direction, the yaw direction, and the rotation direction. The lens 12 forms an image of the subject 20 on an image pickup device provided in the image pickup unit 30, for example. In the present embodiment, the position of the lens 12 and the position of the imaging unit 30 are approximated to substantially the same position, and only the reference numeral of the imaging unit 30 is shown as the position of the lens 12 and the imaging unit 30. FIG. 2 shows an example in which the lens 12 and the imaging unit 30 are provided on the surface of the imaging device 10 facing the −Z direction, but instead of or in addition to this, the surface of the imaging device 10 facing the + Z direction. It may be provided in.

The acceleration sensor 14 is provided inside the image pickup device 10 and detects the acceleration of the image pickup device 10. The acceleration sensor 14 is configured to be capable of measuring acceleration in three axial directions, for example, in the X direction, the Y direction, and the Z direction. Instead, the acceleration sensor 14 may be configured to be capable of measuring acceleration in the biaxial directions in the X and Y directions. By executing the integration twice for the acceleration in each direction detected by the acceleration sensor 14, the moving distance in each direction is calculated. The acceleration sensor 14 may be any one of sensors using capacitance, piezoelectric material, piezo resistance, strain gauge, servo, or a combination thereof.

The angular velocity sensor 16 is provided inside the image pickup device 10 and detects the angular velocity of the image pickup device 10. The angular velocity sensor 16 may be configured to be capable of measuring an angular velocity around at least one axial direction in the X, Y, and Z directions. The movement angle in each direction is calculated by executing the integration once for the angular velocity in each direction detected by the angular velocity sensor 16. The angular velocity sensor 16 may be any one of a gyro sensor using a capacitance, a piezoelectric body, a piezo resistance, an electromagnetic type, an optical fiber, or a combination thereof, or a combination thereof.

The display unit 18 displays images before and after imaging of the imaging device 10. The user who is an imager operates the image pickup apparatus 10 while checking the image of the display unit 18 to take an image of the subject 20. Here, an example will be described in which the user rotates the image pickup apparatus 10 by camera shake around the point P by an angle θ _{Z on a plane substantially parallel to the XY plane.} FIG. 2 shows an example of the image pickup device after being moved by the rotation as the image pickup device 10'.

If the angular velocity sensor 16 is provided inside the image pickup device 10, the angular velocity sensor 16 can detect the angular velocity based on the rotational operation of the image pickup device 10 regardless of the mounting position in the image pickup device 10. Since the angular velocity sensor 16 can detect, for example, the angular velocity ω _Z around the Z axis accompanying the movement of the imaging device 10, the imaging device 10 _{integrates the angular velocity ω Z} once to integrate the angular velocity ω Z once, so that the Z axis of the imaging device 10 is used. The amount of change in the surrounding angular velocity θ _Z can be calculated. Similarly, since the angular velocity sensor 16 _{can detect the angular velocity ω X} around the X axis and the angular velocity ω _Y around the Y axis, the imaging device 10 can change the angle change amount θ _X around the X axis and the angle change around the Y axis. The quantity θ _Y can be calculated respectively.

Further, the acceleration sensor 14 can detect the acceleration in each direction of the image pickup apparatus 10. _{For example, since the acceleration sensor 14 can detect the acceleration a Y} in the Y direction accompanying the movement of the image pickup device 10, the image pickup device 10 _{integrates the acceleration a Y} twice in the Y direction of the acceleration sensor 14. The amount of movement d2 _Y can be calculated. _{Here, the movement amount d2 Y} based on the detection result of the acceleration sensor 14 is the movement amount of the acceleration sensor 14 in the Y direction, and does not necessarily correspond to the movement amount of the imaging unit 30 in the Y direction.

For example, when the acceleration sensor 14 is arranged in the vicinity of the image pickup unit 30, the movement amount d2 _Y of the acceleration sensor 14 can be approximated to _{the movement amount d1 Y} of the image pickup unit 30 in the Y direction. However, as shown in FIG. 2, when the distance between the acceleration sensor 14 and the imaging unit 30 is large, the accuracy of approximation is reduced. Therefore, even if the image pickup apparatus 10 _{determines the movement amount of the imaging unit 30 based on the angle change amount θ Z} and the movement amount d2 _Y in the Y direction, and adjusts the position of the lens 12 according to the movement amount. In some cases, the user's camera shake cannot be corrected accurately.

Here, the distance between the acceleration sensor 14 and the imaging unit 30 is known at the manufacturing or design stage. For example, as shown in FIG. 2, the distance in the X direction between the acceleration sensor 14 and the imaging unit 30 and the L _X. Also, the distance in the X direction between the center of rotation P and the acceleration sensor 14 when the R _X, the following equation is established.
(Number 1)
R _X = d2 _Y / θ _Z

That is, the distance calculated in equation (1) by adding the distance L _X in R _X, it is possible to calculate the radius of rotation of the imaging unit 30. Therefore, the rotation radius _R X + _{L X} of the imaging unit 30, based on the rotation angle theta _Z, it is possible to calculate the movement amount of the imaging unit 30. However, since the equation (Equation 1) is a mathematical expression on the premise that the image pickup apparatus 10 rotates, the rotation angle becomes zero in the case of a simple translational motion without rotation, and the right side of the equation (Equation 1) is extreme. Will grow to. That is, in situations where either can not be expected to generate any shake the user, using the correction formula with R _X obtained in (Equation 1) where it may shake correction operation resulting in collapse.

Therefore, the image pickup apparatus 10 according to the present embodiment accurately calculates and corrects the movement amount of the image pickup unit 30 regardless of whether or not the camera shake is accompanied by rotation. The calculation of the amount of blurring of the image pickup apparatus 10 will be described below.

FIG. 3 shows a first calculation example of the amount of blurring of the image pickup apparatus 10 according to the present embodiment. Similar to FIG. 2, FIG. 3 shows an example of three orthogonal directions on the XYZ axes. FIG. 3 omits the description of the main body of the imaging device 10 and shows only the imaging unit 30 and the acceleration sensor 14. Further, FIG. 3 shows an example in which the imaging unit 30 and the acceleration sensor 14 are arranged on a straight line substantially parallel to the X axis. 3, such an imaging unit 30 and the acceleration sensor 14, in a plane substantially parallel to the XY plane, showing an example of a rotational positional relationship about the point P by an angle theta _Z.

The image pickup unit 30 and the acceleration sensor after movement are shown as the image pickup unit 30'and the acceleration sensor 14'. That is, in the Y direction, the moving amount of the imaging unit 30 is d1 _Y , and the moving amount of the acceleration sensor 14 is d2 _Y. Here, the movement amount d2 _Y of the acceleration sensor 14 is expressed by the following equation.
(Number 2)
d2 _Y = _{R X} · θ _Z

Further, the movement amount d1 _Y is also expressed by the following equation. Here, the following equation was modified using the equation (Equation 2).
(Number 3)
_{d1 Y = (R X + L} X) · θ Z = d2 Y + L X · θ Z

The first term of the equation (Equation 3) is the movement amount of the acceleration sensor 14, and can be calculated based on the output of the acceleration sensor 14. Further, equation (3) the second term of the angle variation theta _Z of the imaging unit 30, a multiplication of the distance L _X between the imaging unit 30 and the acceleration sensor 14. Angle variation can be calculated based on the output of the angular velocity sensor 16, the distance L _X is a predetermined be values by the position relationship between the imaging unit 30 and the acceleration sensor 14. Therefore, the movement amount d1 _Y of the imaging unit 30 can be calculated based on the detection results of the acceleration sensor 14 and the angular velocity sensor 16 and the positional relationship between the imaging unit 30 and the acceleration sensor 14.

As described above, the image pickup apparatus 10 according to the present embodiment calculates the movement amount of the image pickup unit 30 for each direction based on the detection results of the acceleration sensor 14 and the angular velocity sensor 16. That is, the image pickup apparatus 10 calculates the amount of movement of the acceleration sensor 14 in each direction, and adds a correction term to the amount of movement to calculate the amount of blurring of the image pickup unit 30 in each direction. As a result, even in the case of a simple translational motion that does not involve rotation, it is possible to prevent the calculation result of the movement amount of the imaging unit 30 from becoming extremely large and breaking.

FIG. 3 has described an example in which the point P, which is the center of rotation of the image pickup apparatus 10, is located on the −X direction side of the image pickup unit 30 and the acceleration sensor 14. Next, an example in which the point P exists between the image pickup unit 30 and the acceleration sensor 14 will be described.

FIG. 4 shows a second calculation example of the amount of blurring of the image pickup apparatus 10 according to the present embodiment. In FIG. 4, substantially the same operation as that of the image pickup apparatus 10 according to the present embodiment shown in FIG. 3 is designated by the same reference numerals, and the description thereof will be omitted. In FIG. 4, the position of the rotation center P of the image pickup apparatus 10 is different from that in FIG. 3, but as in FIG. 3, the image pickup unit 30 and the acceleration sensor 14 are centered on the point P on a plane substantially parallel to the XY plane. An example of the positional relationship rotated by the angle θ _{Z is shown.}

In this case, the movement amount d2 _Y of the acceleration sensor 14 is expressed by the following equation. In FIG. 4, since the acceleration sensor 14 moves in the −Y direction, the acceleration sensor 14 detects, for example, the acceleration in the −Y direction and outputs a negative value. That is, a negative value is calculated for the movement amount d2 _{Y of the acceleration sensor 14.}
(Number 4)
_{d2 Y = R X · (θ} Z + π) = R X · (-θ Z) = - R X · θ Z

Further, since the image pickup unit 30 moves in the + Y direction, the movement amount d1 _Y is expressed by the following equation from the positional relationship between the image pickup unit 30 and the acceleration sensor 14.
(Number 5)
d1 _Y = (L _X- R _X ) · θ _Z = d2 _Y + L _X · θ _Z

Eq. (Equation 5) is the same as Eq. (Equation 3), although the movement amount d2 _Y has a negative value. That is, even in the case of the positional relationship shown in FIG. 4, the movement amount d1 _Y of the imaging unit 30 can be calculated by using the detection results of the acceleration sensor 14 and the angular velocity sensor 16. Next, an example in which the point P exists on the + X direction side of the imaging unit 30 and the acceleration sensor 14 will be described.

FIG. 5 shows a third calculation example of the amount of blurring of the image pickup apparatus 10 according to the present embodiment. In FIG. 5, substantially the same operation as that of the image pickup apparatus 10 according to the present embodiment shown in FIGS. 3 and 4 is designated by the same reference numerals, and the description thereof will be omitted. In FIG. 5, the position of the rotation center P of the imaging device 10 is different from that in FIGS. 3 and 4, but the imaging unit 30 and the acceleration sensor 14 are on a plane substantially parallel to the XY plane, as in FIGS. shows an example of a rotational positional relationship about the point P by an angle theta _Z.

In this case, the movement amount d2 _Y of the acceleration sensor 14 is expressed by the following equation. In FIG. 5, since the acceleration sensor 14 moves in the −Y direction as in FIG. 4, a negative value is calculated for _{the movement amount d2 Y of the acceleration sensor 14.}
(Number 6)
_{d2 Y = R X · (θ} Z + π) = R X · (-θ Z) = - R X · θ Z

Further, since the image pickup unit 30 also moves in the −Y direction, the movement amount d1 _Y is expressed by the following equation from the positional relationship between the image pickup unit 30 and the acceleration sensor 14.
(Number 7)
_{d1 Y = (R X -L X} ) · (θ Z + π) = d2 Y + L X · θ Z

Equation (Equation 7) is the same as Equations (Equation 3) and (Equation 5), and the movement amount d1 _Y of the imaging unit 30 can be calculated using the detection results of the acceleration sensor 14 and the angular velocity sensor 16. .. As described above, the image pickup apparatus 10 according to the present embodiment is based on the movement amount in each direction based on the acceleration sensor 14, the corresponding detection result of the angular velocity sensor 16, and the positional relationship between the acceleration sensor 14 and the angular velocity sensor 16. By adding the correction value, the amount of movement of the imaging unit 30 can be calculated.

Although FIGS. 3 to 5 have described an example in which the imaging unit 30 is arranged on the + X direction side of the acceleration sensor 14, the arrangement is not limited to this. The positions of the acceleration sensor 14 and the imaging unit 30 in the X direction may be substantially the same. In this case, the movement amount d1 _Y in the Y direction of the imaging unit 30, since substantially coincident with the movement amount d2 _Y acceleration sensor 14, (Equation 3), (5), and (7) the correction as formula may not adding section _{L X} · _{θ Z.} Instead of this, the imaging unit 30 may be arranged closer to the −X direction than the acceleration sensor 14. The correction term in this case will be described below.

FIG. 6 shows a fourth calculation example of the amount of blurring of the image pickup apparatus 10 according to the present embodiment. In FIG. 6, substantially the same operation as that of the image pickup apparatus 10 according to the present embodiment shown in FIG. 3 is designated by the same reference numerals, and the description thereof will be omitted. FIG. 6 shows an example in which the arrangement of the imaging unit 30 and the acceleration sensor 14 is reversed as compared with FIG. Then, FIG. 6, like FIG. 3, the imaging unit 30 and the acceleration sensor 14, in a plane substantially parallel to the XY plane, showing an example of a rotational positional relationship about the point P by an angle theta _Z.

In this case, the movement amount d2 _Y of the acceleration sensor 14 is expressed by the equation (Equation 2). Further, the movement amount d1 _Y of the imaging unit 30 is expressed by the following equation from the positional relationship between the imaging unit 30 and the acceleration sensor 14.
(Number 8)
d1 _Y = (R _X- L _X ) · θ _Z = d2 _{Y −} L _X · θ _Z

FIG. 7 shows a fifth calculation example of the amount of blurring of the image pickup apparatus 10 according to the present embodiment. In FIG. 7, substantially the same operation as that of the image pickup apparatus 10 according to the present embodiment shown in FIG. 6 is designated by the same reference numerals, and the description thereof will be omitted. In FIG. 7, the position of the rotation center P of the image pickup apparatus 10 is different from that in FIG. 6, but as in FIG. 6, the image pickup unit 30 and the acceleration sensor 14 are centered on the point P on a plane substantially parallel to the XY plane. An example of the positional relationship rotated by the angle θ _{Z is shown.}

In this case, the movement amount d2 _Y of the acceleration sensor 14 is expressed by the equation (Equation 2). Further, since the image pickup unit 30 moves in the −Y direction, the movement amount d1 _Y is expressed by the following equation from the positional relationship between the image pickup unit 30 and the acceleration sensor 14.
(Number 9)
d1 _Y = (L _{X −} R _X ) · (−θ _Z ) = d2 _Y −L _X · θ _Z

FIG. 8 shows a sixth calculation example of the amount of blurring of the image pickup apparatus 10 according to the present embodiment. In FIG. 8, substantially the same operation as that of the image pickup apparatus 10 according to the present embodiment shown in FIGS. 6 and 7 is designated by the same reference numerals, and the description thereof will be omitted. In FIG. 8, the position of the rotation center P of the imaging device 10 is different from that in FIGS. 6 and 7, but the imaging unit 30 and the acceleration sensor 14 are on a plane substantially parallel to the XY plane, as in FIGS. 6 and 7. shows an example of a rotational positional relationship about the point P by an angle theta _Z.

In this case, the movement amount d2 _Y of the acceleration sensor 14 is expressed by the equation (Equation 4). Further, since the image pickup unit 30 moves in the −Y direction, the movement amount d1 _Y is expressed by the following equation from the positional relationship between the image pickup unit 30 and the acceleration sensor 14.
(Number 10)
d1 _Y = (L _X + R _X ) · (−θ _Z ) = d2 _Y −L _X · θ _Z

Equations (Equation 8) to (Equation 10) are the same equations, although the positive and negative of _{d2 Y are different.} That is, the imaging apparatus 10 according to this embodiment, the movement amount d2 _Y based on the acceleration sensor 14, the detection result of the angular velocity sensor 16, the correction term (-L _X based on the positional relationship between the imaging unit 30 and the acceleration sensor 14 -By _{adding θ Z} ), the amount of movement of the imaging unit 30 can be calculated.

The above-described example of the imaging device 10 according to the present embodiment for calculating the amount of movement of the imaging unit 30 in the Y direction based on the positional relationship between the imaging unit 30 and the acceleration sensor 14 in the X direction has been described, but the present invention is limited to this. There is nothing. The image pickup apparatus 10 may calculate the amount of movement of the image pickup unit 30 in the X direction based on the positional relationship between the image pickup unit 30 and the acceleration sensor 14 in the Y direction. That is, the image pickup apparatus 10 may be able to calculate the movement amount of the image pickup unit 30 in the X direction and the Y direction in response to the movement due to camera shake or the like on a plane substantially parallel to the XY plane.

FIG. 9 shows an example of a case where the image pickup apparatus 10 according to the present embodiment is blurred on the XY surface. FIG. 9 shows an example of three orthogonal directions on the XYZ axes as in FIGS. 2 to 8, and an example in which the image pickup apparatus 10 is arranged substantially parallel to the X direction and the Y direction and substantially perpendicular to the Z direction. show. Further, FIG. 9 shows an example in which the long side of the display surface of the image pickup apparatus 10 is substantially parallel to the Y direction and the short side is substantially parallel to the X direction, as in FIG. Further, FIG. 9, the distance between the imaging unit 30 and the acceleration sensor 14, an example spaced L _X, the Y direction by L _Y in the X-direction.

9, such an imaging device 10, in substantially a plane parallel to the XY plane, showing an example of a rotational positional relationship about the point P by an angle theta _Z. In this case, the movement amount d1 _{Y of the imaging unit 30 in the Y direction is the movement amount d2 Y} based on the acceleration sensor 14, the detection result of the angular velocity sensor 16, and the imaging unit 30 and by adding the correction term L _X · θ _Z based on the positional relationship between the acceleration sensor 14 can be calculated.

Here, since the X direction and the Y direction are orthogonal to each other, the movement amount d1 _X of the imaging unit 30 in the X direction can be calculated in the same manner. For example, as shown in FIG. 9, when the image pickup apparatus 10 is _{rotated in the + θ Z} direction, the moving directions of the image pickup unit 30 and the acceleration sensor 14 in the X direction are both −X directions, so that the sign of the correction term is also negative. _{The amount of movement d1 X} of the imaging unit 30 in the X direction can be calculated by the following equation.
(Number 11)
_{d1 X = (L Y + R} Y) · (-θ Z) = d2 X -L Y · θ Z

Further, when the position of the rotation center P of the imaging device 10 exists between the imaging unit 30 and the acceleration sensor 14, the moving direction of the imaging unit 30 is in the −X direction with respect to the rotation of _{the imaging device 10 in the + θ Z direction.} Yes, the moving direction of the acceleration sensor 14 is the + X direction. _{Therefore, the amount of movement d1 X in} the X direction of the imaging unit 30 can be calculated by the following equation.
(Number 12)
_{d1 X = (L Y -R Y} ) · (-θ Z) = d2 X -L Y · θ Z

Further, when the position of the rotation center P of the image pickup apparatus 10 is on the + X direction side of the image pickup unit 30 and the acceleration sensor 14, the image pickup unit 30 and the acceleration sensor 14 are rotated with respect to the rotation of the image _{pickup apparatus 10 in the + θ Z direction.} The moving direction is the + X direction. _{Therefore, the amount of movement d1 X in} the X direction of the imaging unit 30 can be calculated by the following equation.
(Number 13)
_{d1 X = (L Y + R} Y) · (-θ Z) = d2 X -L Y · θ Z

As described above, the movement amount d1 _{X in the X direction of the imaging unit 30 is the movement amount d2 X} in the X direction based on the acceleration sensor 14, _{the detection result θ Z} of the angular velocity sensor 16, and the imaging unit 30 and the acceleration sensor 14. by adding a correction term based on the positional relationship in the Y direction (-L Y _· θ _Z), it can be calculated. Note that FIG. 9 has described an example in which the imaging unit 30 is arranged on the + X direction side of the acceleration sensor 14, but the arrangement is not limited to this. The positions of the acceleration sensor 14 and the imaging unit 30 in the Y direction may be substantially the same, and in this case, the correction term may not be added.

Further, the imaging unit 30 may be arranged on the −Y direction side with respect to the acceleration sensor 14. The correction term in this case can be calculated by the following equation regardless of the position of the rotation center P.
(Number 14)
d1 _X = d2 _X + _LY · θ _Z

As described above, the image pickup apparatus 10 according to the present embodiment can accurately calculate the amount of movement of the image pickup unit 30 even if blurring or the like occurs on a plane substantially parallel to the XY plane. The amount of movement of the imaging unit 30 that can be calculated by the imaging device 10 is not limited to a surface substantially parallel to the XY surface. The image pickup apparatus 10 can calculate the amount of movement of the image pickup unit 30 even if blurring or the like occurs on a plane substantially parallel to the ZX plane, for example.

FIG. 10 shows an example of a case where the image pickup apparatus 10 according to the present embodiment is blurred on the ZX surface. Similar to FIG. 9, FIG. 10 shows an example of three orthogonal directions on the XYZ axes, and shows an example in which the image pickup apparatus 10 is arranged substantially parallel to the X direction and the Y direction and substantially perpendicular to the Z direction. Further, FIG. 10 shows an example in which the display surface of the image pickup apparatus 10 faces the −Z direction, the long side is substantially parallel to the Y direction, and the short side is substantially parallel to the X direction, as in FIGS. 2 and 9. .. Further, FIG. 10, the distance between the imaging unit 30 and the acceleration sensor 14, an example spaced in the Z direction by L _Z. Although FIG. 10 shows an example in which the acceleration sensor 14 is provided on the surface of the image pickup device 10, it may be provided inside the image pickup device 10.

FIG. 10 shows an example of the positional relationship in which _{such an image pickup apparatus 10 is rotated by an angle θ Y about} a point P on a plane substantially parallel to the ZX plane. In this case, the amount of movement d1 _X of the imaging unit 30 in the X direction can be calculated by the following equation as described with reference to FIGS. 6 to 8.
(Number 15)
d1 _X = d2 _X −L _Z · θ _Y

When the positional relationship between the image pickup unit 30 and the acceleration sensor 14 in the Z direction is reversed and the acceleration sensor 14 is provided on the −Z direction side of the image pickup unit 30, the amount of movement of the image pickup unit 30 in the X direction. d1 _X can be calculated by the following equation as described with reference to FIGS. 3 to 5.
(Number 16)
d1 _X = d2 _X + L _Z · θ _Y

As described above, the image pickup apparatus 10 corrects the _{movement amount d2 X based on the acceleration sensor 14 based on the detection result θ Y} of the angular velocity sensor 16 _{and the positional relationship L Z of} the image pickup unit 30 and the acceleration sensor 14 in the Z direction. By adding the term, the amount of movement d1 _X of the imaging unit 30 in the X direction can be calculated with high accuracy. Further, the image pickup apparatus 10 can calculate the amount of movement of the image pickup unit 30 even if blurring or the like occurs on a plane substantially parallel to the YZ plane.

FIG. 11 shows an example of a case where the image pickup apparatus 10 according to the present embodiment is blurred on the YZ surface. Similar to FIGS. 9 and 10, FIG. 11 shows an example of three orthogonal directions on the XYZ axes, and an example in which the image pickup apparatus 10 is arranged substantially parallel to the X direction and the Y direction and substantially perpendicular to the Z direction. show. Further, FIG. 11 shows an example in which the display surface of the image pickup apparatus 10 faces the −Z direction, the long side is substantially parallel to the Y direction, and the short side is substantially parallel to the X direction, as in FIGS. 9 and 10. .. Further, FIG. 10, the distance between the imaging unit 30 and the acceleration sensor 14, an example spaced in the Z direction by L _Z.

FIG. 11 shows an example of the positional relationship in which _{such an image pickup apparatus 10 is rotated by an angle θ X about} a point P on a plane substantially parallel to the YZ plane. In this case, the amount of movement d1 _{Y in} the Y direction of the imaging unit 30 can be calculated by the following equation as described with reference to FIGS. 3 to 5.
(Number 17)
d1 _Y = d2 _Y + L _Z · θ _X

When the positional relationship between the image pickup unit 30 and the acceleration sensor 14 in the Z direction is reversed and the acceleration sensor 14 is provided on the −Z direction side of the image pickup unit 30, the amount of movement of the image pickup unit 30 in the X direction. d1 _X can be calculated by the following equation as described with reference to FIGS. 6 to 8.
(Number 18)
d1 _X = d2 _Y- L _Z · θ _X

As described above, the image pickup apparatus 10 corrects the _{movement amount d2 Y based on the acceleration sensor 14 based on the detection result θ X} of the angular velocity sensor 16 _{and the positional relationship L Z of} the image pickup unit 30 and the acceleration sensor 14 in the Z direction. By adding the term, the amount of movement d1 _Y of the imaging unit 30 in the Y direction can be calculated with high accuracy.

The image pickup apparatus 10 may be able to adjust the blur in the Z direction by autofocus or the like. Even in this case, the image pickup apparatus 10 may calculate _{the movement amount d1 Z of the image pickup unit 30 in the Z direction.} Further, the image pickup apparatus 10 _{may supply the calculated information of the movement amount d1 Z} to the autofocus mechanism. As a result, even if camera shake or the like occurs after the autofocus mechanism detects the distance to the subject 20, it can be corrected by using the _{amount of movement d1 Z in the Z direction of the imaging unit 30.}

As described above, the image pickup apparatus 10 according to the present embodiment decomposes the amount of blurring generated in the image pickup unit 30 due to camera shake or the like into the amount of movement in each direction such as the XYZ direction. As a result, the image pickup apparatus 10 calculates the amount of movement of each based on the output of the acceleration sensor 14 and the angular velocity sensor 16 and the positional relationship between the image pickup unit 30 and the acceleration sensor 14, and the amount of blurring of the image pickup unit 30. Is calculated accurately. The configuration of the image pickup apparatus 10 capable of calculating such a blur amount will be described below.

FIG. 12 shows a configuration example of the image pickup apparatus 10 according to the present embodiment. The image pickup apparatus 10 takes an image of the subject 20. The image pickup device 10 includes a lens 12, an acceleration sensor 14, an angular velocity sensor 16, an image pickup unit 30, a storage unit 40, a drive unit 50, and a blur amount correction device 100. Since the lens 12, the acceleration sensor 14, and the angular velocity sensor 16 have already been described in FIGS. 1 to 11, duplicate description will be omitted here.

The lens 12 may be movable in the first direction and the second direction. Further, the lens 12 may be further movable in the third direction. Here, the first direction and the second direction may be orthogonal directions. Further, the first direction and the second direction may be directions orthogonal to the optical axis direction of the lens 12. As an example, the first direction is the X direction and the second direction is the Y direction. Further, the Z direction may be the third direction.

The acceleration sensor 14 detects and outputs the acceleration of the image pickup apparatus 10 in the first direction, the second direction, and the third direction, respectively. Further, the angular velocity sensor 16 detects and outputs the angular velocity of the image pickup apparatus 10 on the first surface substantially parallel to the first direction and the second direction.

The angular velocity sensor 16 detects and outputs the angular velocity of the image pickup apparatus 10 on the first plane (as an example, the XY plane) substantially parallel to the first direction and the third direction. Further, the angular velocity sensor 16 has a second surface (for example, a ZX surface) substantially parallel to the first direction and substantially orthogonal to the second direction, and a third surface substantially parallel to the second direction and substantially orthogonal to the first direction (as an example). As an example, the angular velocities of the image pickup apparatus 10 on the YZ plane) may be detected and output.

The imaging unit 30 faces the lens 12 and captures an image formed by the lens 12. The image pickup unit 30 has an image sensor made of a CCD, CMOS, or the like, and converts an input image into an electric signal. The imaging unit 30 supplies the converted electrical signal to the storage unit 40.

The storage unit 40 stores an electric signal received from the image pickup unit 30. Further, the storage unit 40 may store the set value of the image pickup apparatus 10 and the like. Further, the storage unit 40 may store data or the like processed by the image pickup apparatus 10. The storage unit 40 may store the setting data, the intermediate data, the calculation result, the parameters, and the like calculated (or used) in the process of calculating the blur amount by the image pickup apparatus 10. Further, the storage unit 40 may supply the stored data to the request source in response to the request of each unit in the image pickup apparatus 10.

The drive unit 50 moves the lens 12 in response to the drive signal. The drive unit 50 moves the lens 12 according to the drive signal for correcting the amount of blur, and corrects the blur of the image pickup unit 30. The drive unit 50 moves the lens 12 in the first direction and / or the second direction to correct the blur. Further, the drive unit 50 may move the lens 12 in the third direction.

The blur amount correction device 100 calculates the blur amount of the imaging unit 30 and generates and outputs a drive signal for correcting the blur amount. The blur amount correction device 100 generates a drive signal based on the output of the acceleration sensor 14 and the angular velocity sensor 16 and the positional relationship between the image pickup unit 30 and the acceleration sensor 14. The blur amount correction device 100 supplies the generated drive signal to the drive unit 50. A more detailed configuration of such a blur amount correction device 100 will be described below.

FIG. 13 shows a configuration example of the blur amount correction device 100 according to the present embodiment. In this embodiment, an example in which the blur amount correction device 100 is provided in the image pickup device 10 will be described, but the present invention is not limited to this. The blur amount correction device 100 may be provided separately from the image pickup device 10. The blur amount correction device 100 includes a first acquisition unit 110, a second acquisition unit 120, a blur amount calculation device 130, a correction signal generation unit 140, and a drive signal generation unit 150.

The first acquisition unit 110 acquires the acceleration detection result of the acceleration sensor 14. The second acquisition unit 120 acquires the detection result of the amount of change in the angle of the angular velocity sensor 16. Further, the first acquisition unit 110 and / or the second acquisition unit 120 may acquire the information stored in the storage unit 40. The blur amount calculation device 130 calculates the blur amount of the imaging unit 30. The blur amount calculation device 130 includes a movement amount calculation unit 210, an angle change calculation unit 220, and a blur amount calculation unit 230.

The movement amount calculation unit 210 calculates the first movement amount of the acceleration sensor 14 in the first direction based on the output of the acceleration sensor 14. _{The movement amount calculation unit 210 calculates the first movement amount d2 X} of the acceleration sensor 14, for example, by integrating the detection result of the acceleration in the X direction of the acceleration sensor 14 twice. Further, the movement amount calculation unit 210 may calculate the second movement amount of the acceleration sensor 14 in the second direction based on the output of the acceleration sensor 14. _{The movement amount calculation unit 210 calculates the second movement amount d2 Y} of the acceleration sensor 14, for example, by integrating the detection result of the acceleration in the Y direction of the acceleration sensor 14 twice. Further, the movement amount calculation unit 210 may similarly calculate the movement amount of the acceleration sensor 14 in the third direction.

The angle change calculation unit 220 calculates the first angle change amount of the imaging unit 30 based on the output of the angular velocity sensor 16. The angle change calculation unit 220 calculates the angle change amount on the first surface as the first angle change amount. _{The angle change calculation unit 220 calculates the first angle change amount θ Z} on the XY plane by, for example, integrating the detection result of the angular velocity of the angular velocity sensor 16 on the XY plane once.

Further, the angle change calculation unit 220 may calculate the second angle change amount of the angular velocity sensor 16 on the second surface. _{The angle change calculation unit 220 calculates the second angle change amount θ Y} on the XZ plane by, for example, integrating the detection result of the angular velocity of the angular velocity sensor 16 on the ZX plane once. Further, the angle change calculation unit 220 may calculate the third angle change amount of the angular velocity sensor 16 on the third surface. _{The angle change calculation unit 220 calculates the third angle change amount θ X} on the YZ plane by, for example, integrating the detection result of the angular velocity of the angular velocity sensor 16 on the YZ plane once.

The blur amount calculation unit 230 calculates the blur amount of the image pickup unit 30 in the first direction based on the first movement amount, the first angle change amount, and the positional relationship between the image pickup unit 30 and the acceleration sensor 14. The blur amount calculation unit 230 adds or subtracts the first correction value calculated based on, for example, the first angle change amount and the distance between the image pickup unit 30 and the acceleration sensor 14 to the first movement amount. The amount of blurring of the imaging unit 30 is calculated accordingly. In this case, the blur amount calculation unit 230 calculates the first correction value by multiplying the first angle change amount by the first distance between the image pickup unit 30 and the acceleration sensor 14 in the second direction.

Shake amount calculation unit 230, for example, the first movement amount d2 _X, based on the first distance _{L Y} between the first angle variation theta _Z, and the imaging unit 30 and the acceleration sensor 14, the imaging unit 30 in the X direction The amount of blur d1 _X is calculated. In this case, the blur amount calculation unit 230 calculates _{the blur amount d1 X} of the imaging unit 30 by adding or subtracting _{the first correction value LY} · θ _Z to the first movement amount d2 _{X. The blur amount calculation unit 230 has the blur amount d1 X} of the image pickup unit 30 as shown by the equations (Equation 11), (Equation 12), and (Equation 13) according to the positional relationship between the image pickup unit 30 and the acceleration sensor 14. May be calculated. _{Further, the blur amount calculation unit 230 may calculate the blur amount d1 X} of the image pickup unit 30 as shown by the equation (Equation 14) according to the positional relationship between the image pickup unit 30 and the acceleration sensor 14.

Further, the blur amount calculation unit 230 calculates the blur amount of the image pickup unit 30 in the second direction based on the second movement amount, the first angle change amount, and the positional relationship between the image pickup unit 30 and the acceleration sensor 14. You can do it. The blur amount calculation unit 230 secondly moves, for example, a second correction value calculated by multiplying the first angle change amount by the second distance between the image pickup unit 30 and the acceleration sensor 14 in the first direction. By adding or subtracting from the amount, the amount of blurring of the imaging unit 30 in the second direction is calculated.

Shake amount calculation unit 230, for example, the second movement amount d2 _Y, based on the second distance _{L X} between the first angle variation theta _Z, and the imaging unit 30 and the acceleration sensor 14, the imaging unit 30 in the Y-direction The amount of blur d1 _Y is calculated. In this case, the blur amount calculation unit 230, by adding or subtracting a second correction value _{L X} · _{θ Z} in the second movement amount d2 _Y, and calculates a blurring amount d1 _Y of the imaging unit 30. _{The blur amount calculation unit 230 has a blur amount d1 Y} of the imaging unit 30 as shown by the equations (Equation 3), (Equation 5), and (Equation 7) according to the positional relationship between the imaging unit 30 and the acceleration sensor 14. May be calculated. Further, the blur amount calculation unit 230 shakes the image pickup unit 30 as shown by the equations (Equation 8), (Equation 9), and (Equation 10) according to the positional relationship between the image pickup unit 30 and the acceleration sensor 14. You may calculate d1 _Y.

Further, the blur amount calculation unit 230 calculates the blur amount of the image pickup unit 30 in the first direction based on the first movement amount, the second angle change amount, and the positional relationship between the image pickup unit 30 and the acceleration sensor 14. You can do it. The blur amount calculation unit 230 first moves, for example, a third correction value calculated by multiplying the second angle change amount by the third distance between the image pickup unit 30 and the acceleration sensor 14 in the third direction. By adding or subtracting from the amount, the amount of blurring of the imaging unit 30 in the first direction is calculated.

The blur amount calculation unit 230 is based on, for example, the first movement amount d2 _X , the second angle change amount θ _{Y , and the third distance L Z} between the image pickup unit 30 and the acceleration sensor 14, and the image pickup unit 30 in the X direction. The amount of blur d1 _X is calculated. In this case, the blur amount calculation unit 230 calculates _{the blur amount d1 X} of the imaging unit 30 by adding or subtracting _{the third correction value LZ} · θ _Y to the first movement amount d2 _{X. The blur amount calculation unit 230 may calculate the blur amount d1 X} of the lens 12 as shown by the equation (Equation 15) according to the positional relationship between the image pickup unit 30 and the acceleration sensor 14. _{Further, the blur amount calculation unit 230 may calculate the blur amount d1 X} of the image pickup unit 30 as shown by the equation (Equation 16) according to the positional relationship between the image pickup unit 30 and the acceleration sensor 14.

Further, the blur amount calculation unit 230 calculates the blur amount of the image pickup unit 30 in the second direction based on the second movement amount, the third angle change amount, and the positional relationship between the image pickup unit 30 and the acceleration sensor 14. You can do it. The blur amount calculation unit calculates, for example, a fourth correction value calculated by multiplying the third angle change amount by the third distance between the image pickup unit 30 and the acceleration sensor 14 in the third direction, and obtains the second movement amount. The amount of blurring of the imaging unit 30 in the second direction is calculated by adding or subtracting from.

The blur amount calculation unit 230 is based on, for example, the second movement amount d2 _Y , the third angle change amount θ _{X , and the third distance L Z} between the image pickup unit 30 and the acceleration sensor 14, and the image pickup unit 30 in the Y direction. The amount of blur d1 _Y is calculated. In this case, the blur amount calculation unit 230 calculates _{the blur amount d1 Y} of the imaging unit 30 by adding or subtracting _{the fourth correction value LZ} · θ _X to the second movement amount d2 _{Y. The blur amount calculation unit 230 may calculate the blur amount d1 Y} of the lens 12 as shown by the equation (Equation 17) according to the positional relationship between the image pickup unit 30 and the acceleration sensor 14. _{Further, the blur amount calculation unit 230 may calculate the blur amount d1 Y} of the image pickup unit 30 as shown by the equation (Equation 18) according to the positional relationship between the image pickup unit 30 and the acceleration sensor 14.

The blur amount calculation unit 230 may calculate the blur amount by adding or subtracting one or a plurality of correction values for each direction. The blur amount calculation unit 230 supplies the calculated blur amount to the correction signal generation unit 140 as an output of the blur amount calculation device 130. Further, the blur amount calculation unit 230 may _{also supply information on the first angle change amount θ Z} and / or the second angle change amount θ _{Y to} the correction signal generation unit 140.

The correction signal generation unit 140 generates a correction signal for correcting the position of the lens 12 according to the blur amount of the imaging unit 30 calculated by the blur amount calculation unit 230. The correction signal generation unit 140 generates, for example, a correction signal for moving the lens 12 so as to correct the _{amount of blur d1 X in the first direction.} Further, the correction signal generation unit 140 generates a correction signal for moving the lens 12 so as to correct _{the blur amount d1 Y in the second direction.} Further, the correction signal generation unit 140 may add a signal for correcting a blur amount corresponding to _{the first angle change amount θ Z} and / or the second angle change amount θ _{Y to the generated correction signal.}

The correction signal generation unit 140 may include a limiter circuit or the like so as to correct the lens 12 within the movable range of the lens 12. The correction signal generation unit 140 supplies the generated correction signal to the drive signal generation unit 150.

The drive signal generation unit 150 generates a drive signal for moving the lens 12 based on the correction signal. The drive signal generation unit 150 supplies the generated drive signal to the drive unit 50. As a result, the drive unit 50 can move the lens 12 in response to the blur generated in the image pickup unit 30.

As described above, the blur amount calculation device 130 according to the present embodiment shakes the image pickup unit 30 based on the output of the acceleration sensor 14 and the angular velocity sensor 16 and the positional relationship between the image pickup unit 30 and the acceleration sensor 14. The amount can be calculated. Further, the blur amount correction device 100 according to the present embodiment can generate a drive signal for correcting the blur of the image pickup unit 30 based on the blur amount of the image pickup unit 30 calculated by the blur amount calculation device 130. Further, the image pickup apparatus 10 according to the present embodiment can correct the blurring of the image pickup unit 30 by using the drive signal generated by the blurring amount correction device 100. As a result, even if the image pickup unit 30 and the acceleration sensor 14 are arranged apart from each other, the amount of blur can be calculated accurately, and the blur of the image pickup unit 30 can be easily corrected.

FIG. 14 shows a configuration example of a computer 1200 in which a plurality of aspects of the present invention can be embodied in whole or in part. The program installed on the computer 1200 causes the computer 1200 to function as an operation associated with the device according to an embodiment of the present invention or as one or more "parts" of the device, or the operation or the one or more "parts". A unit can be run and / or a computer 1200 can be run a process according to an embodiment of the invention or a stage of the process. Such a program may be executed by the CPU 1212 to cause the computer 1200 to perform a specific operation associated with some or all of the blocks of the flowcharts and block diagrams described herein.

The computer 1200 according to this embodiment includes a CPU 1212, a RAM 1214, a graphic controller 1216, and a display device 1218, which are interconnected by a host controller 1210. The computer 1200 also includes input / output units such as a communication interface 1222, a hard disk drive 1224, a DVD-ROM drive 1226, and an IC card drive, which are connected to the host controller 1210 via the input / output controller 1220. The computer also includes legacy I / O units such as the ROM 1230 and keyboard 1242, which are connected to the I / O controller 1220 via the I / O chip 1240.

The CPU 1212 operates according to the programs stored in the ROM 1230 and the RAM 1214, thereby controlling each unit. The graphic controller 1216 acquires image data generated by the CPU 1212 in a frame buffer or the like provided in the RAM 1214 or in the graphic controller 1216 itself, and displays the image data on the display device 1218.

Communication interface 1222 communicates with other electronic devices via a network. The hard disk drive 1224 stores programs and data used by the CPU 1212 in the computer 1200. The DVD-ROM drive 1226 reads the program or data from the DVD-ROM 1201 and provides the program or data to the hard disk drive 1224 via the RAM 1214. The IC card drive reads the program and data from the IC card and / or writes the program and data to the IC card.

The ROM 1230 internally stores a boot program or the like executed by the computer 1200 at the time of activation, and / or a program depending on the hardware of the computer 1200. The input / output chip 1240 may also connect various input / output units to the input / output controller 1220 via a parallel port, a serial port, a keyboard port, a mouse port, and the like.

The program is provided by a computer-readable storage medium such as a DVD-ROM 1201 or an IC card. The program is read from a computer-readable storage medium, installed on a hard disk drive 1224, RAM 1214, or ROM 1230, which is also an example of a computer-readable storage medium, and executed by the CPU 1212. The information processing described in these programs is read by the computer 1200 and provides a link between the program and the various types of hardware resources described above. The device or method may be configured to implement the operation or processing of information in accordance with the use of the computer 1200.

For example, when communication is executed between the computer 1200 and an external device, the CPU 1212 executes a communication program loaded in the RAM 1214, and performs communication processing on the communication interface 1222 based on the processing described in the communication program. You may order. Under the control of the CPU 1212, the communication interface 1222 reads and reads the transmission data stored in the transmission buffer area provided in the recording medium such as the RAM 1214, the hard disk drive 1224, the DVD-ROM1201, or the IC card. The data is transmitted to the network, or the received data received from the network is written to the reception buffer area or the like provided on the recording medium.

Further, the CPU 1212 makes the RAM 1214 read all or necessary parts of a file or database stored in an external recording medium such as a hard disk drive 1224, a DVD-ROM drive 1226 (DVD-ROM1201), or an IC card. Various types of processing may be performed on the data on the RAM 1214. The CPU 1212 may then write back the processed data to an external recording medium.

Various types of information, such as various types of programs, data, tables, and databases, may be stored on recording media for information processing. The CPU 1212 describes various types of operations, information processing, conditional judgment, conditional branching, unconditional branching, and information retrieval described in various parts of the present disclosure with respect to the data read from the RAM 1214. Various types of processing may be performed, including / replacement, etc., and the results are written back to the RAM 1214. Further, the CPU 1212 may search for information in a file, a database, or the like in the recording medium. For example, when a plurality of entries each having an attribute value of the first attribute associated with the attribute value of the second attribute are stored in the recording medium, the CPU 1212 is the first of the plurality of entries. The attribute value of the attribute of is searched for the entry that matches the specified condition, the attribute value of the second attribute stored in the entry is read, and the first attribute that satisfies the predetermined condition is selected. You may get the attribute value of the associated second attribute.

The program or software module described above may be stored on a computer 1200 or in a computer-readable storage medium near the computer 1200. In addition, a recording medium such as a hard disk or RAM provided in a dedicated communication network or a server system connected to the Internet can be used as a computer-readable storage medium, whereby the program can be sent to the computer 1200 via the network. offer.

Although the present invention has been described above using the embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments. It will be apparent to those skilled in the art that various changes or improvements can be made to the above embodiments. It is clear from the description of the claims that such modified or improved forms may also be included in the technical scope of the present invention.

The order of execution of operations, procedures, steps, steps, etc. in the devices, systems, programs, and methods shown in the claims, specification, and drawings is particularly "before" and "prior to". It should be noted that it can be realized in any order unless the output of the previous process is used in the subsequent process. Even if the scope of claims, the specification, and the operation flow in the drawings are explained using "first", "next", etc. for convenience, it means that it is essential to carry out in this order. It's not a thing.

10 Imaging device, 12 lenses, 14 acceleration sensor, 16 angular velocity sensor, 18 display unit, 20 subject, 30 imaging unit, 40 storage unit, 50 drive unit, 100 blur correction device, 110 first acquisition unit, 120 second acquisition Unit, 130 blur amount calculation device, 140 correction signal generation unit, 150 drive signal generation unit, 210 movement amount calculation unit, 220 angle change calculation unit, 230 blur amount calculation unit, 1200 computer, 1201 DVD-ROM, 1210 host controller, 1212 CPU, 1214 RAM, 1216 graphic controller, 1218 display device, 1220 I / O controller, 1222 communication interface, 1224 hard disk drive, 1226 DVD-ROM drive, 1230 ROM, 1240 I / O chip, 1242 keyboard

Claims (14)

It is a blur amount calculation device that calculates the blur amount of the imaging unit relative to the lens that can move in the first direction and the second direction orthogonal to the first direction.
A movement amount calculation unit that calculates the first movement amount of the acceleration sensor in the first direction based on the output of the acceleration sensor, and a movement amount calculation unit.
An angle change calculation unit that calculates the first angle change amount of the imaging unit based on the output of the angular velocity sensor, and an angle change calculation unit.
A blur amount calculation unit that calculates the blur amount of the image pickup unit in the first direction based on the first movement amount, the first angle change amount, and the positional relationship between the image pickup unit and the acceleration sensor. equipped with a,
The first correction value calculated by multiplying the first angle change amount by the first distance between the image pickup unit and the acceleration sensor in the second direction is obtained by the blur amount calculation unit. A blur amount calculation device that calculates the blur amount of the imaging unit by adding or subtracting it to the moving amount. The blur amount calculation device according to claim 1 , wherein the angle change calculation unit calculates an angle change amount in the first surface parallel to the first direction and the second direction as the first angle change amount. The movement amount calculation unit calculates the second movement amount of the acceleration sensor in the second direction based on the output of the acceleration sensor.
The blur amount calculation unit calculates the blur amount of the image pickup unit in the second direction based on the second movement amount, the first angle change amount, and the positional relationship between the image pickup unit and the acceleration sensor. The blur amount calculation device according to claim 1 or 2, which calculates. The blur amount calculation unit obtains a second correction value calculated by multiplying the first angle change amount by the second distance between the image pickup unit and the acceleration sensor in the first direction. The blur amount calculating device according to claim 3 , wherein the blur amount of the imaging unit in the second direction is calculated by adding or subtracting to the moving amount. The angle change calculation unit calculates the amount of change in the second angle of the angular velocity sensor on the second surface parallel to the first direction and orthogonal to the second direction.
The blur amount calculation unit calculates the blur amount of the image pickup unit in the first direction based on the first movement amount, the second angle change amount, and the positional relationship between the image pickup unit and the acceleration sensor. The blur amount calculation device according to any one of claims 1 to 4, which is calculated. The blur amount calculation unit calculates by multiplying the second angle change amount by the third distance between the image pickup unit and the acceleration sensor in the first direction and the third direction orthogonal to the second direction. The blur amount calculation device according to claim 5 , wherein the blur amount of the imaging unit in the first direction is calculated by adding or subtracting the third correction value to be added to or subtracted from the first movement amount. The movement amount calculation unit calculates the second movement amount of the acceleration sensor in the second direction based on the output of the acceleration sensor.
The angle change calculation unit calculates the amount of change in the third angle of the angular velocity sensor on the third surface parallel to the second direction and orthogonal to the first direction.
The blur amount calculation unit calculates the blur amount of the image pickup unit in the second direction based on the second movement amount, the third angle change amount, and the positional relationship between the image pickup unit and the acceleration sensor. The blur amount calculation device according to any one of claims 1 to 6, which is calculated. The blur amount calculation unit calculates by multiplying the third angle change amount by the third distance between the image pickup unit and the acceleration sensor in the first direction and the third direction orthogonal to the second direction. The blur amount calculation device according to claim 7 , wherein the blur amount of the imaging unit in the second direction is calculated by adding or subtracting the fourth correction value to be added to or subtracted from the second movement amount. The blur amount calculation device according to any one of claims 1 to 8 , wherein the first direction and the second direction are orthogonal to the optical axis direction of the lens. The blur amount calculation device according to any one of claims 1 to 9,
The first acquisition unit that acquires the acceleration detection result of the acceleration sensor, and
A second acquisition unit that acquires the detection result of the amount of change in the angle of the angular velocity sensor, and
A correction signal generation unit that generates a correction signal that corrects the position of the lens according to the blur amount of the imaging unit calculated by the blur amount calculation unit.
A drive signal generation unit that generates a drive signal to be supplied to the drive unit that moves the lens based on the correction signal.
A blur amount correction device equipped with. With the lens
With the accelerometer
With the angular velocity sensor
An imaging unit that captures an image formed by the lens, and an imaging unit.
The drive unit that moves the lens and
The blur amount correction device according to claim 10,
An imaging device comprising. This is a method of calculating the amount of blurring of the imaging unit relative to the lens that can move in the first direction and the second direction orthogonal to the first direction.
To calculate the first movement amount of the acceleration sensor in the first direction based on the output of the acceleration sensor, and
To calculate the amount of change in the first angle of the imaging unit based on the output of the angular velocity sensor,
It is provided to calculate the amount of blurring of the image pickup unit in the first direction based on the first movement amount, the first angle change amount, and the positional relationship between the image pickup unit and the acceleration sensor .
To calculate the amount of blur, the first correction value calculated by multiplying the amount of change in the first angle by the first distance between the image pickup unit and the acceleration sensor in the second direction is obtained. A method including calculating the amount of blurring of the imaging unit by adding or subtracting the amount of movement to the first movement amount. A program that causes a computer to function as the blur amount calculation device according to any one of claims 1 to 9. A program that causes a computer to function as the blur amount correction device according to claim 10.

Images