JP2009246580A - Imaging apparatus, control method of imaging apparatus, and, control program for imaging apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To surely reduce various camera shakes. <P>SOLUTION: An imaging apparatus includes: input means (gyro sensors 31a-31c) each for receiving the input of a detecting signal outputted from a gyro sensor; a determination means (processor 33) for determining a camera shake rotating angle allowable around an axis during imaging by an imaging element 11; a calculation means (processor 33) for calculating a shutter speed based on a value obtained by dividing the allowable camera shake rotating angle by an angular velocity determined by the detecting signal inputted via the input means; a control means (processor 13) for performing imaging while controlling the imaging element based on the calculated shutter speed; and a camera shake reducing means (optical camera shake reduction section 50) for reducing a camera shake by controlling an optical system, that the imaging element has, when performing imaging by the control means or for reducing a camera shake by applying image processing to an image imaged by the imaging element. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an imaging apparatus, an imaging apparatus control method, and an imaging apparatus control program.

As a camera shake reduction device that prevents image degradation caused by a user's camera shake when an image is picked up by an imaging device, a technique for moving the optical system in a direction to cancel the camera shake (see Patent Document 1) and imaging There is a technique (see Patent Document 2) that cancels camera shake based on movement of a subject in a plurality of images.
Japanese Patent Laid-Open No. 07-092175 JP 2007-97195 A

  By the way, in the technique disclosed in Patent Document 1, since the optical system is driven by an actuator, there is a problem that it is difficult to reduce camera shake when the speed of camera shake is high or the amount of camera shake is large. In the technique disclosed in Patent Document 2, it is difficult to specify a subject from a captured image when the environment is dark, and thus it is difficult to reduce camera shake. Furthermore, the techniques of Patent Literature 1 and Patent Literature 2 have a problem that when camera shake occurs in the direction of rotation about the optical axis of the optical system, camera shake cannot be reduced.

  SUMMARY An advantage of some aspects of the invention is that it provides an imaging apparatus, an imaging apparatus control method, and an imaging apparatus control program that can reliably reduce various types of camera shake.

[Mode 1] In order to solve the above problem, an imaging apparatus according to mode 1 includes an imaging element and an angular velocity detection device that detects an angular velocity around an axis of the imaging element. An input unit that receives an input of a detection signal output from the angular velocity detection device when imaging, a determination unit that determines a camera shake rotation angle that is allowed around the axis at the time of imaging by the imaging device, and the determination unit Calculating means for calculating a shutter speed based on a value obtained by dividing the allowable camera shake rotation angle determined by the angular velocity obtained by the detection signal input via the input means; and the calculation Control means for controlling the image sensor based on the shutter speed calculated by the means to perform imaging, and the imaging when the control means performs imaging. A camera shake reducing means for reducing camera shake by controlling an optical system included in the device, or for reducing camera shake by performing image processing on an image captured by the image sensor. To do.
According to the above configuration, the shutter speed is calculated based on the value obtained by dividing the allowable camera shake rotation angle by the angular speed obtained from the detection signal from the angular speed detection device, and imaging is performed based on the calculated shutter speed. Control the element to take an image, and when taking an image, control the optical system of the image sensor to reduce camera shake, or apply image processing to the image taken by the image sensor To reduce camera shake. For this reason, it is possible to reliably reduce various types of camera shake.

[Mode 2] The imaging apparatus according to mode 2 is the imaging apparatus according to mode 1, wherein the angular velocity detection device detects an angular velocity around an axis in a vertical direction or a horizontal direction of the imaging element, and the determination unit includes the vertical direction. Alternatively, the camera shake rotation angle allowed around the horizontal axis is determined, and the calculating means determines the camera shake rotation angle allowed around the vertical or horizontal axis as a value around the vertical or horizontal axis. The shutter speed is calculated based on a value obtained by dividing by the corresponding angular speed.
According to the above configuration, the calculation means obtains a value obtained by dividing the camera shake rotation angle allowed around the vertical or horizontal axis of the image sensor by the corresponding angular velocity around the vertical or horizontal axis. Based on this, the shutter speed is calculated. For this reason, camera shake around the horizontal or vertical axis of the image sensor can be reliably reduced.

[Mode 3] The imaging device according to mode 3 is the imaging device according to mode 1 or 2, wherein the angular velocity detection device detects an angular velocity around an axis in a normal direction of the imaging surface of the imaging element, and the determination unit includes: The camera shake rotation angle allowed around the normal axis is determined, and the calculating means determines the camera shake rotation angle allowed around the normal axis as a corresponding angular velocity around the normal axis. The shutter speed is calculated based on a value obtained by dividing by.
According to the above configuration, the calculation means is based on a value obtained by dividing the camera shake rotation angle allowed around the normal axis of the imaging surface of the image sensor by the corresponding angular velocity around the normal axis. To calculate the shutter speed. For this reason, it is possible to reduce camera shake around the axis in the normal direction of the image sensor, which is difficult to reduce with the camera shake reduction means.

[Mode 4] In the imaging device according to mode 4, in the imaging device according to modes 1 to 3, the camera shake reduction unit reduces camera shake by controlling an optical system included in the imaging device, and the calculation unit includes the camera shake. When camera shake exceeding the reduction range by the reduction means occurs, the shutter speed is calculated based on the excess part.
According to the above configuration, when a camera shake exceeding a reduction range by the camera shake reducing unit that reduces the camera shake by controlling the optical system is generated, the calculation unit calculates the shutter speed based on the excess portion. For this reason, when camera shake that exceeds the reduction range of camera shake reduction means occurs, the shutter speed corresponding to the degree is calculated and the camera shake is reduced, so even camera shake with high speed or camera shake with large amplitude is supported. can do.

[Embodiment 5] In the imaging apparatus of Embodiment 5, in the imaging apparatuses of Embodiments 1 to 3, the camera shake reducing means reduces camera shake by performing image processing on the captured image, and the control means When the light is higher than a predetermined luminance, the image pickup device is caused to take an image based on the shutter speed set by the imaging device or the user. When the ambient light is lower than the predetermined luminance, the calculation is performed. The imaging element is caused to perform imaging based on the shutter speed calculated by the means.
According to the above configuration, when the ambient light is higher than the predetermined luminance, the image pickup device performs imaging based on the shutter speed set by the imaging device or the user, and the environmental light is lower than the predetermined luminance. In other words, the image pickup device performs imaging based on the shutter speed calculated by the calculation unit, and the camera shake reduction unit reduces image blur by performing image processing on the captured image. For this reason, since the image is taken based on the shutter speed calculated by the calculating means when the brightness of the ambient light is low, which is difficult to reduce the camera shake, the camera shake is reliably reduced even in such a state. can do.

[Mode 6] An imaging device control method according to mode 6 is an imaging device control method including: an imaging device; and an angular velocity detection device that detects an angular velocity around an axis of the imaging device. An input step for receiving a detection signal output from the angular velocity detection device, a determination step for determining a camera shake rotation angle allowed around the axis at the time of imaging by the imaging device, and the determination step. A calculation step of calculating a shutter speed based on a value obtained by dividing the determined allowable camera shake rotation angle by an angular velocity obtained by the detection signal input in the input step; and calculating in the calculation step A control step for performing control of imaging with the image sensor at the shutter speed that has been set, and the image sensor. A camera shake reduction step of reducing camera shake by controlling an optical system included in the image sensor when performing imaging, or reducing image blur by performing image processing on an image captured by the image sensor; It is characterized by having.
According to the above method, the shutter speed is calculated based on a value obtained by dividing the allowable camera shake rotation angle by the angular speed obtained from the detection signal from the angular speed detection device, and imaging is performed based on the calculated shutter speed. Control the element to take an image, and when taking an image, control the optical system of the image sensor to reduce camera shake, or apply image processing to the image taken by the image sensor To reduce camera shake. For this reason, it is possible to reliably reduce various types of camera shake.

[Mode 7] A control program for an imaging apparatus according to mode 7 is an imaging apparatus control program that includes an imaging device and an angular velocity detection device that detects an angular velocity around an axis of the imaging device. The input means for receiving the detection signal output from the angular velocity detection device, the determination means for determining the camera shake rotation angle allowed around the axis at the time of imaging by the imaging device, and the determination means Further, calculation means for calculating a shutter speed based on a value obtained by dividing the allowable camera shake rotation angle by an angular speed obtained by the detection signal input via the input means, and calculated by the calculation means. Control means for controlling the imaging device to take an image with the shutter speed, and the imaging device when imaging with the imaging device The computer is caused to function as camera shake reduction means for reducing camera shake by controlling an optical system included in the camera, or by performing image processing on an image captured by the image sensor. And
According to the above program, the shutter speed is calculated based on the value obtained by dividing the allowable camera shake rotation angle by the angular speed obtained from the detection signal from the angular speed detection device, and imaging is performed based on the calculated shutter speed. Control the element to take an image, and when taking an image, control the optical system of the image sensor to reduce camera shake, or apply image processing to the image taken by the image sensor To reduce camera shake. For this reason, it is possible to reliably reduce various types of camera shake.

  Embodiments to which the present invention is applied will be described below with reference to the drawings. In the following, (A) a configuration example of the first embodiment of the present invention, (B) an overview of the operation of the first embodiment of the present invention, (C) a detailed operation of the first embodiment of the present invention, ( D) A second embodiment of the present invention, (E) a third embodiment of the present invention, (F) a fourth embodiment of the present invention, and (G) a modified embodiment will be described in this order.

(A) Configuration Example of First Embodiment of the Present Invention FIG. 1 is a block diagram illustrating a schematic configuration of an imaging apparatus 10 according to the present embodiment. The imaging device 10 (corresponding to “imaging device” in the claims) includes an imaging device 11 (corresponding to “imaging device” in the claims), a luminance detection unit 12, a processor 13 (corresponding to “control means” in the claims), The storage unit 14, the user interface 15, the camera shake prevention unit 30, and the optical camera shake reduction unit 50 (corresponding to “camera shake reduction means” in the claims). Here, the imaging device 11 is configured by, for example, a CCD (Charge Coupled Device) or the like, and converts an optical image of a subject into a corresponding image signal and outputs the image signal. The imaging element 11 is built in an imaging unit 40 described later with reference to FIG. The luminance detection unit 12 is configured by, for example, a photodiode, detects the luminance of the subject, and outputs a signal corresponding to the detected luminance. The processor 13 sets and controls the sensitivity, focal length, aperture, exposure time, and the like of the image sensor 11 based on a program or the like stored in the storage unit 14. The storage unit 14 is constituted by, for example, a semiconductor memory, and stores codes readable by the processor 13 such as routines, programs, object components, and data structures executed by the processor 13. The user interface 15 includes operation buttons and a display unit, and generates and outputs information corresponding to the user's operation and displays the information. The camera shake prevention unit 30 calculates a shutter speed based on an angular speed detected by a gyro sensor described later and an allowable rotation angle, and outputs the shutter speed to the processor 13. Based on the outputs of the X-axis and Y-axis gyro sensors built in the camera shake prevention unit 30, the optical camera shake reduction unit 50 uses a not-shown lens 43 of the imaging unit 40 described later with reference to FIG. By driving, camera shake is reduced by an optical method.

FIG. 2 is a diagram illustrating a detailed configuration example of the camera shake prevention unit 30 illustrated in FIG. 1. As shown in this figure, the camera shake prevention unit 30 includes an X-axis gyro sensor 31a (corresponding to “angular velocity detection device” in the claims), a Y-axis gyro sensor 31b (corresponding to “angular velocity detection device” in the claims), Z Axis gyro sensor 31c (corresponding to “angular velocity detecting device” in the claims), analog-digital converter 32a (corresponding to “input means” in the claims), analog-digital converter 32b (corresponding to “input means” in the claims) Correspondence), an analog-digital conversion unit 32c (corresponding to “input means” in the claims), a processor 33 (corresponding to “determination means” and “calculation means” in the claims), and a storage unit 34. .
Here, the X-axis gyro sensor 31a is constituted by, for example, a vibration gyro sensor, and detects and outputs an angular velocity centered on the X-axis corresponding to the lateral direction (scanning line direction) of the imaging surface of the imaging device 11. . Similarly, the Y-axis gyro sensor 31b is configured by a vibration gyro sensor, and detects and outputs an angular velocity centered on the Y-axis corresponding to the vertical direction (direction orthogonal to the scanning line) of the imaging surface of the imaging element 11. Similarly, the Z-axis gyro sensor 31c is configured by a vibration gyro sensor, and detects and outputs an angular velocity centered on the Z-axis corresponding to the normal direction of the imaging surface of the imaging element 11. The analog-digital conversion unit 32a converts an analog signal indicating the angular velocity output from the X-axis gyro sensor 31a into a digital signal and outputs the digital signal. The analog-digital conversion unit 32b converts an analog signal indicating the angular velocity output from the Y-axis gyro sensor 31b into a digital signal and outputs the digital signal. The analog-digital conversion unit 32c converts an analog signal indicating the angular velocity output from the Z-axis gyro sensor 31c into a digital signal and outputs the digital signal. The processor 33 executes the program stored in the storage unit 34 and prevents camera shake based on the angular velocities around the X, Y, and Z axes output from the analog-digital conversion units 32a, 32b, and 32c. The shutter speed is calculated, and the processor 13 is notified of the obtained shutter speed. The storage unit 34 is configured by, for example, a semiconductor memory, and stores codes readable by the processor 33 such as routines, programs, object components, and data structures executed by the processor 33.

  FIG. 3 is a diagram illustrating a detailed configuration example of the imaging unit 40 in which the imaging device 11 illustrated in FIG. 1 is built. The imaging unit 40 includes a casing 41 having a cubic shape. A lens 43 for converging a light image from the subject is provided on the top surface 42 of the housing 41. An image sensor 11 for converting the optical image converged by the lens 43 into a corresponding image signal is provided on the bottom surface (not shown) inside the housing 41. The image sensor 11 is arranged so that the horizontal direction that is the direction of the scanning line coincides with the X-axis direction in the figure, and the vertical direction that is orthogonal to the scanning line is in the Y-axis direction in the figure. It arrange | positions so that it may correspond, and also arrange | positions so that the normal line of an imaging surface may correspond to the Z-axis in a figure. An X-axis gyro sensor 31a is disposed on a side surface 44 whose normal line coincides with the X-axis of the housing 41. A Y-axis gyro sensor 31b is disposed on a side surface 45 where the Y-axis of the housing 41 and the normal line thereof coincide with each other. Further, a Z-axis gyro sensor 31 c is disposed on the top surface 42. The side surface 45 has a signal line group for transmitting a control signal for controlling the image sensor 11, an image signal output from the image sensor 11, and a control signal for controlling the lens 43, and is connected to the processor 13. A flexible cable 46 is provided. Although omitted in FIG. 3, an actuator that drives the lens 43 in the vertical direction and the horizontal direction of the image pickup device 11 by a control signal from the optical camera shake reduction unit 50 is disposed inside the housing 41. ing.

  In the example of FIG. 3, the X-axis gyro sensor 31a, the Y-axis gyro sensor 31b, and the Z-axis gyro sensor 31c are provided on the side surfaces 44 and 45 and the top surface 42 of the casing 41, respectively. Is the same at any location on the imaging device 10, and can be provided at any location on the imaging device 10 as long as the axial directions to be detected match. As an example, as shown in FIG. 3, a Y-axis gyro sensor 31 b may be provided in a part of the flexible cable 46.

(B) Outline of Operation of First Embodiment of Present Invention Next, an outline of operation of the first embodiment of the present invention will be described. FIG. 4 is a diagram illustrating a state of movement of a target point on the imaging surface of the image sensor 11 when the imaging unit 40 illustrated in FIG. 3 is rotated around the X axis, the Y axis, or the Z axis. . FIG. 4A shows the movement of the target point as an arbitrary point of the subject projected on the imaging surface when the imaging element 11 is rotated around the Y axis. As shown in this figure, when the image sensor 11 is rotated about the Y axis, the target point moves in the x direction (lateral direction) that is the direction of the scanning line of the image sensor 11 as indicated by an arrow. To do. FIG. 4B shows the movement of the target point on the imaging surface when the imaging element 11 is rotated around the X axis. As shown in this figure, when the image sensor 11 is rotated around the X axis, the target point is the direction perpendicular to the scanning line of the image sensor 11 (horizontal direction) as indicated by an arrow. Move to. FIG. 4C shows the movement of the target point on the imaging surface when the imaging element 11 is rotated around the Z axis. As shown in this figure, when the image sensor 11 is rotated around the Z axis, the target point moves in the direction of rotation around the center of the image sensor 11 as indicated by an arrow.
By the way, when the blur in the Z-axis direction is not considered, the position of the target point at the start of exposure of the image sensor 11 is (x ₁ , y ₁ ), and the position of the target point at the end of exposure is (x ₂ , y _2). ), The target point moves by x ₂ −x ₁ in the x direction and y ₂ −y ₁ in the y direction between the start and end of exposure. At this time, when the pixel sizes in the x direction and the y direction are S _x and S _y , the number of moving pixels of the target point is (x ₂ −x ₁ ) / S _x and x in the x direction and y direction, respectively. (Y ₂ −y ₁ ) / S _y The smaller the number of moving pixels of the target point, the smaller the image. Therefore, in the present embodiment, in the period from the start to the end of exposure, the rotation angle of the X-axis or Y-axis of the image sensor 11 corresponding to the allowable movement amount of the target point in the x-direction or y-direction is obtained. By calculating the shutter speed based on a value obtained by dividing the rotation angle by the angular velocity detected by the gyro sensor, control is performed so that the moving amount of the moving point on the imaging surface is within the allowable moving amount. More specifically, in this embodiment, it is assumed that the amount of movement in the x and y directions is the same, and the rotation angle of the X axis of the image sensor 11 corresponding to the allowable amount of movement of the target point in the y direction. α ′ is obtained based on the equation (37) described later. Then, the allowable rotation angle α ′ is set around the X axis output from the X axis gyro sensor 31a and the Y axis gyro sensor 31b when the release button is fully pressed, as shown in equation (38) described later. The shutter speed SS is obtained by dividing by the average value of the angular velocities AngleVel_X and AngleVel_Y around the Y axis. The shutter speed SS thus obtained is supplied to the processor 13, and the processor 13 controls the shutter speed of the image sensor 11 to be SS. In the above example, rotation around the Z axis is not considered, but the principle is the same except that the direction of movement around the Z axis is different. By performing such control, the amount of movement of the target point on the imaging surface falls within the allowable amount of movement, so that an image with less camera shake can be obtained.
In the present embodiment, the camera shake reduction processing by the optical camera shake reduction unit 50 is executed together with the above-described processing by the camera shake prevention unit 30. For this reason, since the processing for reducing camera shake using the respective characteristics is executed by the processing related to these two types of camera shake, it is possible to deal with various types of camera shake. For example, the optical camera shake reduction unit 50 can effectively reduce the camera shake around the X axis and the Y axis, but cannot reduce the camera shake around the Z axis. However, since the camera shake prevention unit 30 can perform the reduction regardless of the axial direction, any camera shake can be reliably reduced by these two processing units.

(C) Detailed Operation of the First Embodiment of the Invention Next, the detailed operation of the embodiment of the present invention will be described. In the following, after describing the algorithm of the camera shake prevention process in the present embodiment, the specific operation of the embodiment will be described.
FIG. 5 is a conceptual diagram showing an imaging model defined based on a simple lens model. For ease of understanding, the focal plane is set parallel to the image plane, the lens is relatively thin, and its optical axis is perpendicular to the image plane. At this time, the lens operates according to the following law.

  Here, u represents the distance from the lens surface to the real world object, v represents the distance from the lens surface to the focused image, and f represents the focal length of the lens. Further, the relationship between the coordinates of the real world object and the coordinates of the pixels in the image is defined as follows.

F: Focal length of imaging device P _w = (x _w , y _w , z _w ): position of one point in real world object coordinates P _c = (x _c , y _c , z _c ): one point in imaging device coordinates Position P _im = (x _im , y _im ): position of one point in image plane coordinates P _focused = (x _focused , y _focused ): position of one point in focal plane coordinates P = (x, y): pixel coordinates Position S _x : x-direction pixel size S _y : y-direction pixel size x ₀ : horizontal principal point y ₀ : vertical principal point λ: model magnification α: rotation angle β around the X axis of the imaging device: Rotation angle γ around the Y axis of the imaging device: Rotation angle around the Z axis of the imaging device

ここで、ＲはＸ，Ｙ，Ｚ方向に対する回転を定義する３×３の回転マトリクスであり、また、Ｔは原点間の並進を定義する三次元並進ベクトルである。
また、関係の第２の構成要素は実世界対象座標から画像面への透視投影である。この透視投影は簡略化ピンホール・カメラ投影モデルに基づいている。

図５に示す単純レンズ・モデルにより、第３の構成要素は撮像装置１０の焦点面および画像面を（ｕ，ｖ，ｆ）の関数であるモデル倍率λと以下のように関連付ける。

ここで、（ｘ_im，ｙ_im）は二次元画像面座標に投影された点であり、（ｘ_focused，ｙ_focused）は二次元焦点面座標で対応する点であり、λはｖ／ｆに比例し、その値は“１”に近い。 The relationship between the coordinates of the real world object and the pixel coordinates includes four components. The first component of the relationship is rotation and translation between the imaging device and real world object coordinates.

Here, R is a 3 × 3 rotation matrix that defines rotation in the X, Y, and Z directions, and T is a three-dimensional translation vector that defines translation between the origins.
The second component of the relationship is perspective projection from real world object coordinates to the image plane. This perspective projection is based on a simplified pinhole camera projection model.

With the simple lens model shown in FIG. 5, the third component associates the focal plane and the image plane of the imaging device 10 with the model magnification λ, which is a function of (u, v, f), as follows.

Here, (x _im , y _im ) is a point projected on the two-dimensional image plane coordinates, (x _focused , y _focused ) is a corresponding point in the two-dimensional focal plane coordinates, and λ is v / f. It is proportional and its value is close to “1”.

The fourth component of the relationship is the conversion between the focal plane and the image elementary coordinates.

また、次のようにも表現できる。

ｘ方向画素サイズＳ_xおよびｙ方向画素サイズＳ_yは、焦点面上の撮像素子１１の解像度から計算される。なお、撮像の解像度が変更された場合には、ｘ方向画素サイズＳ_xおよびｙ方向画素サイズＳ_yは変更される。 Therefore, the relationship between the real world object coordinates and the pixel coordinates is a combination of the expressions (2), (3), (4), and (5) as follows. As a result, the coordinate projection CP can be expressed as follows.

It can also be expressed as:

The x-direction pixel size _Sx and the y-direction pixel size _Sy are calculated from the resolution of the image sensor 11 on the focal plane. Note that when the imaging resolution is changed, the x-direction pixel size _Sx and the y-direction pixel size _Sy are changed.

この結果、式（７）の撮像装置モデルは以下のように簡略化できる。

式（９）における回転マトリクスＲは次のように計算される。

Ｘ軸に対する回転を表すマトリクスは以下のように表される。

Ｙ軸に対する回転を表すマトリクスは以下のように表される。

Ｚ軸に対する回転を表すマトリクスは以下のように表される。

以上に定義したように、α、β、および、γは、それぞれ、Ｘ軸、Ｙ軸、および、Ｚ軸の周りに回転された角度で、角速度と露光時間と以下のように関係している。

ここで、Ｅは露光期間、Ｇ_x、Ｇ_y、および、Ｇ_zは、後述するようにジャイロセンサから得られる角速度である。なお、Ｎは露光期間Ｅ中に得られたジャイロセンサのサンプル数である。 In the present embodiment, it is assumed that the movement of the imaging apparatus 10 from the start to the end of exposure is a simple rotation. Under this assumption, the translation vector T is a zero vector.

As a result, the imaging device model of Expression (7) can be simplified as follows.

The rotation matrix R in equation (9) is calculated as follows.

A matrix representing the rotation about the X axis is expressed as follows.

A matrix representing the rotation about the Y axis is expressed as follows.

A matrix representing rotation about the Z axis is expressed as follows.

As defined above, α, β, and γ are angles rotated about the X axis, the Y axis, and the Z axis, respectively, and are related to the angular velocity and the exposure time as follows. .

Here, E is the exposure period, and G _x , G _y , and G _z are angular velocities obtained from the gyro sensor as will be described later. N is the number of gyro sensor samples obtained during the exposure period E.

回転がない場合、実世界対象点Ｐ_w＝（Ｐ_x，Ｐ_y，Ｐ_z）に対応する画素座標は以下の式によって表される。

ここで、（ｘ₁，ｙ₁）は、実世界対象座標の点Ｐ_wに対応する画素座標である。回転は、上述のように撮像装置１０の露光開始から終了までの間のＸ軸、Ｙ軸、および、Ｚ軸に対する単純な回転と仮定されるが、回転された角度はそれぞれα、β、および、γで表される。回転マトリクスＲは、式（１０）によって表される。さらに、撮像装置１０の回転前の実世界対象点Ｐ_w＝（ｘ_w，ｙ_w，ｚ_w）に対応する画素座標は式（１６）で表される。式（１６）から、実世界対象座標の点Ｐ_w＝（ｘ_w，ｙ_w，ｚ_w）は以下の式によって表すことができる。

説明を簡略化するために、撮像装置１０の露光中に被写体は動かないと仮定される。式（１７）および式（９）から、各画素の回転前後の関係が画像座標で得られる。

ここで、（ｘ₂，ｙ₂）は撮像装置１０の回転後の実世界対象点Ｐ_wに対応する画像の画素座標であり、（ｘ₁，ｙ₁）は撮像装置１０の回転前の実世界対象点に対応する画素座標である。
画像における各画素に対する動きベクトルは次のように表すことができる。

Prior to rotation, the imager coordinates are aligned with real world object coordinates. Therefore, the rotation matrix before the rotation is expressed as follows.

When there is no rotation, the pixel coordinates corresponding to the real world target point P _w = (P _x , P _y , P _z ) are expressed by the following equations.

Here, (x ₁ , y ₁ ) is a pixel coordinate corresponding to the point P _w of the real world target coordinate. The rotation is assumed to be a simple rotation with respect to the X axis, the Y axis, and the Z axis from the start to the end of exposure of the imaging apparatus 10 as described above, but the rotated angles are α, β, and , Γ. The rotation matrix R is represented by Expression (10). Further, the pixel coordinates corresponding to the real world target point P _w = (x _w , y _w , z _w ) before the rotation of the imaging device 10 are expressed by Expression (16). From the equation (16), the point P _w = (x _w , y _w , z _w ) in the real world object coordinates can be expressed by the following equation.

In order to simplify the explanation, it is assumed that the subject does not move during the exposure of the imaging apparatus 10. From Expression (17) and Expression (9), the relationship before and after the rotation of each pixel can be obtained in image coordinates.

Here, (x ₂ , y ₂ ) is the pixel coordinates of the image corresponding to the real world target point P _w after rotation of the imaging device 10, and (x ₁ , y ₁ ) is the actual pixel before rotation of the imaging device 10. It is a pixel coordinate corresponding to a world object point.
The motion vector for each pixel in the image can be expressed as:

In the present embodiment, in order to reduce the amount of calculation, sin α and cos α in Expression (11), Expression (12), and Expression (13) are expanded to Macrolin as follows.

  Here, if α << 1, the second and subsequent terms on the right-hand side of Equation (20) and Equation (21) can be ignored, so that it approximates sin α = α and approximates cos α = 1. can do.

When the right side of Expression (18) is expanded, the following Expression (22), Expression (23), and Expression (24) are obtained. Here, the rotation around the Z axis is negligible.

Here, in the equations (22), (23), and (24), when sin α = α and cos α = 1 are approximated and sin β = β and cos β = 1 are approximated, the following equations (25), Equations (26) and (27) are obtained.

In the equation (26), − ((x ₁ −x ₀ ) × Fc 2 × α × β) / Fc 1 is a sufficiently small value as compared with other terms. Therefore, if this is deleted, the following equation (28) Get.

式（２９）および式（３０）の右辺を整理すると、以下の式（３１）および式（３２）を得る。

ここで、（ｘ₂−ｘ₁）＝Δｘとし、（ｙ₂−ｙ₁）＝Δｙとすると、対象点の移動量であるＭｏｔｉｏｎＥｘｔｅｎｔは以下の式で表される。

Further, the following expressions (29) and (30) are obtained from the expressions (25), (28), and (19).

By rearranging the right sides of the equations (29) and (30), the following equations (31) and (32) are obtained.

Here, when (x ₂ −x ₁ ) = Δx and (y ₂ −y ₁ ) = Δy, MotionExtent, which is the amount of movement of the target point, is expressed by the following equation.

式（３４）および式（３５）はラジアン（radian）値であるため、ディグリー（degree）値に変換すると、以下の式（３６）および式（３７）を得る。

In the present embodiment, the optimum shutter speed is obtained from the movement amount of the target point. When Expression (31) and Expression (32) are modified with respect to β and α, the following Expression (34) and Expression (35) are obtained.

Since Equation (34) and Equation (35) are radian values, the following Equation (36) and Equation (37) are obtained when converted to the degree value.

In actual camera shake, since the rotation amounts of the X axis and the Y axis are different, the values of α ′ and β ′ are different, and as a result, the movement amounts (x ₂ −x ₁ ) and (y ₂ −y ₁ ) are also different. However, in this embodiment, it is assumed that they are the same in order to simplify the calculation. In that case, when Fc1 and Fc2 are the same, α ′ and β ′ have the same value.
Further, the rotation amount around the Z axis is assumed to be γ ′, and γ ′ is also assumed to have the same value. As a result, when the angular velocities detected by the X-axis gyro sensor 31a, the Y-axis gyro sensor 31b, and the Z-axis gyro sensor 31c are AngleVel_X, AngleVel_Y, and Anglevel_Z, respectively, and α ′ is used, the optimum shutter speed SS is It is obtained by the following equation (38).

In the present embodiment, α ′ is obtained by Expression (37) based on the allowable amount of movement (y ₂ −y ₁ ) in the y direction, and the X axis gyro sensor 31a, the Y axis gyro sensor 31b, and the Z axis By substituting the angular velocities AngleVel_X, AngleVel_Y, and Anglelevel_Z detected by the gyro sensor 31c into the equation (38), the optimum shutter speed SS is obtained. Then, the image sensor 11 is controlled according to the obtained optimum shutter speed SS. Further, when taking an image, the camera shake is optically reduced by the optical camera shake reduction unit 50 in the X direction and the Y direction. As a result, shooting is performed at a shutter speed at which camera shake is unlikely to occur, and image degradation due to camera shake is suppressed by the optical camera shake reduction unit 50 even when camera shake occurs during imaging. Further, the optical camera shake reduction unit 50 cannot deal with camera shake around the Z axis, which is the optical axis direction of the lens 43, but the camera shake prevention unit 30 reduces such camera shake around the Z axis. be able to.

  Next, details of processing executed in the imaging apparatus 10 will be described with reference to a flowchart of FIG. When the processing of the flowchart shown in FIG. 6 is started, the following steps are executed. That is, in step S <b> 1, the processor 33 determines the allowable movement amount Δy of the target point on the imaging surface of the imaging element 11. As a method of determining Δy, for example, a value of Δy is stored in advance in the storage unit 34, and this value may be read when the imaging device 10 is activated.

  In step S2, the processor 13 determines whether or not a release button (not shown) included in the user interface 15 is half-pressed. If the release button is half-pressed, the processor 13 proceeds to step S3. Repeat the same process. For example, when the user points the imaging device 10 toward the subject and presses the release button halfway (see S1 in FIG. 7), the determination in Step S2 is Yes and the process proceeds to Step S3.

  In step S3, the processor 13 performs AF (Auto Focus) processing and AE (Auto Exposure) processing (see “AF, AE processing” in FIG. 7). More specifically, the processor 13 detects the luminance of the subject based on the detection result of the luminance detection unit 12, and determines an optimal exposure value. As a result of the AE process, an optimum aperture value and a shutter speed corresponding to the detected luminance are determined. Further, the processor 13 controls the imaging unit 40 shown in FIG. 3 to execute AF processing according to the distance between the subject and the imaging device 10 and determine an optimum focal length. The aperture value, shutter speed, and focal length determined in this way are stored in the storage unit 14, and then supplied to the processor 33 of the camera shake prevention unit 30 in step S6 described later.

  In step S4, the processor 13 determines whether or not the release button has been fully pressed. If it is determined that the release button has been fully pressed (see S2 in FIG. 7), the process proceeds to step S5. In this case, the same processing is repeated. For example, if the user determines the composition of the subject, determines that shooting is to be performed, and presses the release button fully, the process proceeds to step S5.

  In step S5, the processor 13 notifies the camera shake prevention unit 30 that the release button has been fully pressed, and the camera shake prevention unit 30 performs gyro sampling processing (“Sampling” in FIG. 7) based on this notification. Process ”). More specifically, the processor 33 samples the outputs of the analog-digital conversion unit 32a, the analog-digital conversion unit 32b, and the analog-digital conversion unit 32c at regular intervals, and stores them in the storage unit 34 as gyro data. . The gyro sensors 31a, 31b, and 31c detect angular velocities around the X, Y, and Z axes corresponding to the camera shake amount. However, since the camera shake amount changes with time, the gyro sensors 31a, 31b, and 31c The output varies with time. Therefore, in the sampling process, sampling is performed a plurality of times instead of once, an average value of the obtained angular velocities is calculated, and the obtained values are used as angular velocities around the X, Y, and Z axes. be able to.

  In step S <b> 6, the processor 33 acquires imaging information from the storage unit 14 via the processor 13. At this time, the acquired imaging information includes an aperture value, a shutter speed, and a focal length as described above.

  In step S7, the processor 33 ends the gyro sampling process. That is, the processor 33 ends the acquisition of data from the analog-digital converters 32a, 32b, and 32c. In order to suppress power consumption in the gyro sensors 31a, 31b, and 31c and the analog-digital conversion units 32a, 32b, and 32c, supply of power to these may be stopped when the sampling process is completed. Good.

  In step S <b> 8, the processor 33 acquires gyro data stored in the storage unit 34. At this time, since the storage unit 34 stores a plurality of gyro data relating to the X axis, the Y axis, and the Z axis, as described above, the processor 33 obtains an average value of these gyro data, and calculates the X axis, AngleVel_X, AngleVel_Y, and AngleVel_Z that are angular velocities around the Y axis and the Z axis are obtained.

In step S9, the processor 33 executes a camera shake prevention process (see “calculation process” in FIG. 7). More specifically, the processor 33 first acquires the pixel size (S _y ) in the y direction of the image captured from the processor 13. For example, when the vertical / horizontal resolution of the image sensor 11 is 2304 × 2304 pixels, the vertical and horizontal lengths of one pixel are each 2.5 μm. In this case, when the substantial resolution of the image to be captured is 640 × 480 pixels, one pixel is represented by a plurality of pixels, and thus S _y = 2304/480 × 2.5 μm = 12 μm = 0.012 mm It becomes. Next, the processor 33 obtains a focal length F that is included in the acquired imaging information in step S6 from the storage unit 34, obtains the Fc2 based on Fc2 = F / S _y. Then, the processor 33 calculates the value of α ′ based on Expression (37) based on Δy (= y ₂ −y ₁ ) determined in Step S1 and Fc2. Then, the processor 33 substitutes the values of the angular velocity AngleVel_X around the X axis, the angular velocity AngleVel_Y around the Y axis, the angular velocity AngleVel_Z around the Z axis, and the α ′ described above, which are calculated in step S8, into the equation (38). A simple shutter speed SS is calculated.

In step S10, the processor 33 determines the shutter speed based on the optimum shutter speed SS calculated in step S9. That is, the processor 33 first converts the obtained shutter speed SS into a shutter speed that can be set in the imaging device 10. FIG. 8 is a diagram illustrating an example of shutter speeds that can be set in the imaging apparatus 10. In this example, the shutter speed SS can be selected from 13 types of setting values in the range of 1 sec to 1/4000 sec. “Tv” is a value for designating each shutter speed. In this example, the value of SS can be designated by a value of 0 to 12. The processor 33 searches the value closest to the SS value calculated in step S9 from FIG. 8, and sets the obtained value as a new SS value. For example, when the SS value obtained in step S9 is “1/150 sec”, “1/125 sec” in FIG. 8 is selected as the new SS value. Note that instead of selecting a close value, a value with a higher shutter speed may be selected. In this case, when the SS value obtained in step S9 is “1/150 sec”, “1/250 sec” in FIG. 8 is selected as a new SS value.
Next, the processor 33 compares the shutter speed SS included in the imaging information acquired in step S6 with the new SS obtained by the above-described processing, and determines the SS value of the faster speed as follows. Let it be the final SS value. For example, when the new SS value is “1/125 sec” and the shutter speed SS value included in the imaging information is “1/60 sec”, “1/125 sec” is set as the final SS value. Selected.
On the other hand, when the new SS value is “1/125 sec” and the shutter speed SS value included in the imaging information is “1/250 sec”, “1/250 sec” is set as the final SS value. Selected. This is because it is considered that the occurrence of camera shake is prevented when the shutter speed included in the imaging information is faster than the new SS value.
The shutter speed SS value (or Tv value) calculated as described above is supplied to the processor 13.

そして、プロセッサ１３は、求めたこれらの撮像情報（絞り値Ａｖ、シャッタ速度ＳＳ、および、感度値ＩＳＯ）に基づいて、撮像部４０の設定を行った後、光学手ぶれ軽減を伴う撮像を開始する（図７の「撮像開始」参照）。より詳細には、プロセッサ１３は、絞り値Ａｖに基づいて撮像部４０の絞りを設定し、シャッタ速度ＳＳに応じてシャッタ速度を設定し、感度値ＩＳＯに応じて感度を設定する。そして、図７のＳＯにおいて、撮像を開始する。撮像が開始されると、光学手ぶれ軽減部５０は、手ぶれ防止部３０のＸ軸ジャイロセンサ３１ａおよびＹ軸ジャイロセンサ３１ｂから出力されるＸ軸およびＹ軸を中心とするその時点における角速度を入力する。図４（Ａ）に示すように、Ｙ軸を中心とする手ぶれは対象点をｘ方向に移動させる動きとなるので、光学手ぶれ軽減部５０は、Ｙ軸ジャイロセンサ３１ｂからの出力に応じてレンズ４３をｘ方向に移動させ、対象点のｘ方向への移動をキャンセルする。また、図４（Ｂ）に示すように、Ｘ軸を中心とする手ぶれは対象点をｙ方向に移動させる動きとなるので、光学手ぶれ軽減部５０は、Ｘ軸ジャイロセンサ３１ｂからの出力に応じてレンズ４３をｙ方向に移動させ、対象点のｙ方向への移動をキャンセルする。このような処理により、撮像中において、手ぶれが生じた場合であっても、光学手ぶれ軽減部５０によって、レンズ４３が手ぶれをキャンセルする方向に移動されるので、撮像された画像の劣化を防ぐことができる。
以上のような撮像処理によって得られた画像データは、データ圧縮処理等が施された後、格納部１４に格納される。 In step S11, the processor 13 executes an imaging process based on the shutter speed SS obtained in step S10. Specifically, the processor 13 substitutes the final shutter speed SS obtained in step S10 for the following equation (39), and sets the exposure amount Ev obtained by the AE process in step S3 to the equation (39). And the sensitivity value ISO or the aperture value Av is determined. For example, when “aperture priority” is set, the sensitivity value ISO is obtained based on Expression (39) using the aperture value Av set by the user. When the sensitivity value ISO is set by the user, the aperture value Av is obtained using the set sensitivity value ISO.

Then, the processor 13 sets the imaging unit 40 based on the obtained imaging information (aperture value Av, shutter speed SS, and sensitivity value ISO), and then starts imaging with optical camera shake reduction. (Refer to “Start imaging” in FIG. 7). More specifically, the processor 13 sets the aperture of the imaging unit 40 based on the aperture value Av, sets the shutter speed according to the shutter speed SS, and sets the sensitivity according to the sensitivity value ISO. Then, imaging is started in SO of FIG. When the imaging is started, the optical camera shake reduction unit 50 inputs the angular velocity at that time centered on the X axis and the Y axis output from the X axis gyro sensor 31a and the Y axis gyro sensor 31b of the camera shake prevention unit 30. . As shown in FIG. 4 (A), camera shake centering around the Y-axis is a movement that moves the target point in the x-direction, so that the optical camera-shake reduction unit 50 determines the lens according to the output from the Y-axis gyro sensor 31b. 43 is moved in the x direction to cancel the movement of the target point in the x direction. Further, as shown in FIG. 4B, the camera shake centering around the X axis moves the target point in the y direction, so that the optical camera shake reducing unit 50 responds to the output from the X axis gyro sensor 31b. The lens 43 is moved in the y direction to cancel the movement of the target point in the y direction. By such processing, even when camera shake occurs during imaging, the optical camera shake reduction unit 50 moves the lens 43 in a direction to cancel camera shake, thereby preventing degradation of the captured image. Can do.
The image data obtained by the imaging process as described above is subjected to a data compression process or the like and then stored in the storage unit 14.

  As described above, according to the embodiment of the present invention, the allowable rotation angle α ′ and the angular velocities AngleVel_X, AngleVel_Y detected by the X-axis gyro sensor 31a, the Y-axis gyro sensor 31b, and the Z-axis gyro sensor 31c. Based on AngleVel_Z, an optimum shutter speed SS is obtained by Expression (38), and imaging is performed according to the obtained shutter speed SS. For this reason, since it is not necessary to perform trigonometric functions and determinants, it is possible to perform arithmetic processing at high speed even with an inexpensive processor. In addition, since it is not necessary to provide a lookup table, the necessary storage capacity of the storage unit 34 can be reduced.

  Further, in the present embodiment, since the output of the Z-axis gyro sensor 31c is also referred to, it is possible to deal with camera shake centered on the Z-axis that is difficult to reduce by the optical camera shake reduction unit 50. Even when correction is performed with the Z axis as the center, it is not necessary to perform trigonometric functions and determinants as in the case described above, and it is not necessary to provide a lookup table.

  Further, in the present embodiment, since the camera shake reduction processing by the optical camera shake reduction unit 50 is also executed, it is possible to reliably reduce image degradation caused by camera shake that occurs during imaging. Furthermore, in the present embodiment, the optical camera shake reduction unit 50 uses the detection signals of the X-axis gyro sensor 31a and the Y-axis gyro sensor 31b included in the camera shake prevention unit 30, and thus these gyro sensors are shared. This can prevent the apparatus from becoming complicated.

(D) Second Embodiment of the Present Invention Next, a second embodiment of the present invention will be described. Compared with the first embodiment, the second embodiment of the present invention has the same configuration of the imaging device 10 as that of the first embodiment, but the operation of the camera shake prevention unit 30 is the same as that of the first embodiment. Is different. Below, it demonstrates focusing on a different part.
FIG. 9 is a flowchart illustrating an example of processing executed when an operation for setting the camera shake prevention function is performed on the user interface 15. When the processing of the flowchart shown in FIG. 9 is started, the following steps are executed. That is, in step S31, the processor 13 displays a menu screen on the display unit of the user interface 15. FIG. 10 shows an example of a menu screen displayed on the display unit of the user interface 15 at this time. In this example, “Menu screen” as the title is displayed at the top of the screen, and “Please turn on / off the camera shake prevention function” as a message is displayed below it. “ON” and “OFF” are displayed as selection items. By selecting “ON” or “OFF” as a selection item on such a menu screen, the camera shake prevention function by the camera shake prevention unit 30 can be turned on or off.

In step S32, the processor 13 determines whether or not the operation of the camera shake prevention function is selected on the menu screen. If the operation is selected, the process proceeds to step S34, and otherwise proceeds to step S33. . More specifically, if “ON” is selected on the menu screen shown in FIG. 10, the process proceeds to step S <b> 34, and otherwise, the process proceeds to step S <b> 33.
When “OFF” is selected on the menu screen shown in FIG. 10, the process proceeds to step S <b> 33, and the processor 13 stops the processing of the camera shake prevention unit 30.
On the other hand, if “ON” is selected on the menu screen shown in FIG. 10, the process proceeds to step S <b> 34, and the processor 13 operates the camera shake prevention unit 30.

  In step S <b> 35, the processor 13 sets the angular velocity threshold value Vth of the optical camera shake reduction unit 50. Here, the angular velocity threshold value Vth indicates the angular velocity of the maximum camera shake that the optical camera shake reducing unit 50 can handle. That is, the optical camera shake reduction unit 50 reduces camera shake by driving the lens 43 with an actuator. However, when the angular velocity of camera shake is large, the operation speed of the control system cannot follow. The angular velocity threshold value Vth indicates the maximum angular velocity that can be followed.

  In step S36, the processor 13 sets the angle threshold value Ath of the optical camera shake reduction unit 50. Here, the angle threshold Ath indicates the maximum camera shake angle that the optical camera shake reduction unit 50 can handle. That is, as described above, the optical camera shake reduction unit 50 reduces camera shake by driving the lens 43 with an actuator. However, when the camera shake angle is large, the operation range (angle) of the control system can follow. Disappear. The angle threshold Ath indicates the maximum angle that can be followed.

In step S37, the processor 13 stores the angular velocity threshold value Vth and the angular threshold value Ath set in steps S35 and S37 in the storage unit 34 via the processor 33, and ends the process.
Through the above processing, the camera shake prevention function is turned on / off on the menu screen, and the angular velocity threshold value Vth and the angular threshold value Ath are set and stored in the storage unit 34.

  Next, the operation at the time of shooting in the second embodiment will be described. The operation at the time of shooting in the second embodiment is basically the same as that shown in FIG. 6, but the operation of “shutter speed determination” shown in step S10 of FIG. 6 is different. FIG. 11 is a diagram for explaining the details of the “shutter speed determination” operation shown in step S10 of FIG. When the process of this figure is started, the following steps are executed. That is, in step S51, the processor 33 searches for data having the maximum value from the data of the X-axis gyro sensor 31a sampled by the processing in steps S5 to S7, and stores the obtained data in Max_Angle_X. Thereby, in Max_Angle_X, the one having the maximum value among the angular velocity data of the X-axis gyro sensor 31a obtained by the processing of steps S5 to S7 is stored.

  In step S52, the processor 33 searches for data having the maximum value from the data of the Y-axis gyro sensor 31b sampled by the processing in steps S5 to S7, and stores the obtained data in Max_Angle_Y. Thereby, in Max_Angle_Y, the angular velocity data of the Y-axis gyro sensor 31b obtained by the processing in steps S5 to S7 is stored.

  In step S53, the processor 33 time-integrates the data of the X-axis gyro sensor 31a sampled by the processes in steps S5 to S7, and stores the obtained data in Int_Angle_X. Thereby, int_Angle_X stores the time integral value (rotational angle around the X axis during the sampling period) of the angular velocity data of the X axis gyro sensor 31a obtained by the processing of steps S5 to S7.

  In step S54, the processor 33 time-integrates the data of the Y-axis gyro sensor 31b sampled by the processes in steps S5 to S7, and stores the obtained data in Int_Angle_Y. As a result, int_Angle_Y stores the time integral value (rotational angle around the Y axis during the sampling period) of the angular velocity data of the Y axis gyro sensor 31b obtained by the processes of steps S5 to S7.

  In step S55, the processor 33 compares Max_Angle_X and Max_Angle_Y obtained in steps S51 and S52 with the Vth set in step S35 of FIG. 9, and Max_Angle_X> Vth is satisfied, or Max_Angle_Y> Vth is satisfied. If so, the process proceeds to step S56; otherwise, the process proceeds to step S57. That is, when the maximum angular velocity around the X axis exceeds the angular velocity threshold, or when the maximum angular velocity around the Y axis exceeds the angular velocity threshold, the process proceeds to step S56.

In step S56, the processor 33 calculates the shutter speed SS based on the equation (40), and ends the process. Here, ΔV_X is given by ΔV_X = Max_Angle_X−Vth when Max_Angle_X> Vth, and ΔV_X = 0 when Max_Angle_X ≦ Vth. ΔV_Y is given by ΔV_Y = Max_Angle_Y−Vth when Max_Angle_Y> Vth, and ΔV_Y = 0 when Max_Angle_Y ≦ Vth. Anglelevel_Z is an average value of angular velocities obtained by the Z-axis gyro sensor 31c. In Expression (40), the value of the shutter speed SS decreases as Max_Angle_X and Max_Angle_Y exceed the angular speed threshold Vth, and the value of the shutter speed SS decreases as the value of Anglelevel_Z increases.

  In step S57, the processor 33 compares Int_Angle_X and Int_Angle_Y obtained in steps S53 and S54 with Ath set in step S37 in FIG. 9, and Int_Angle_X> Ath is satisfied, or Int_Angle_Y> Ath is satisfied If so, the process proceeds to step S58; otherwise, the process proceeds to step S59. That is, if the integrated value of angular velocities around the X axis exceeds the angle threshold, or if the integrated value of angular velocities around the Y axis exceeds the angle threshold, the process proceeds to step S59.

In step S58, the processor 33 calculates the shutter speed SS based on the equation (41), and ends the process. Here, ΔA_X is given by ΔA_X = Int_Angle_X−Ath when Int_Angle_X> Ath, and ΔA_X = 0 when Int_Angle_X ≦ Ath. ΔA_Y is given by ΔA_Y = Int_Angle_Y−Ath when Int_Angle_Y> Ath, and ΔA_Y = 0 when Int_Angle_Y ≦ Vth. T represents the time from the start to the end of sampling. Therefore, ΔA_X / T and ΔA_Y / T are the angular velocity dimensions.

以上の処理により、シャッタ速度ＳＳが算出されると、図６のステップＳ１１に進み、前述した場合と同様に、手ぶれ軽減処理を伴う撮像処理が実行される。 In step S59, the shutter speed SS is calculated based on the equation (42). Since the process proceeds to step S59 in the case where the maximum angular velocity and angle around the X axis and the Y axis within the sample period do not exceed the threshold value, the expression (42) is based on Anglelevel_Z that is the angular velocity around the Z axis. The shutter speed is calculated.

When the shutter speed SS is calculated by the above process, the process proceeds to step S11 in FIG. 6 and an imaging process with a camera shake reduction process is executed as described above.

  As described above, in the second embodiment of the present invention, the maximum values of the angular velocities around the X axis and Y axis sampled within the sampling period of time T (in the period of steps S5 to S7 in FIG. 6). Exceeds the angular velocity threshold value Vth, the excess value is taken into account when determining the shutter speed SS. Further, when the rotation angle about the X axis and the Y axis within the sampling period exceeds the angle threshold value Ath, the excess value is taken into account when determining the shutter speed SS. The angular velocity around the Z axis is always considered and the shutter speed SS is determined. For this reason, when a camera shake exceeding the range of the camera shake reduction process by the optical camera shake reducing unit 50 occurs, the camera shake preventing process by the camera shake preventing unit 30 is executed based on the excess amount (angular velocity and angle). Is done. In addition, camera shake around the Z axis that cannot be reduced by the optical camera shake reduction unit 50 is reliably reduced by the camera shake prevention unit 30.

(E) Third Embodiment of the Present Invention Next, a third embodiment of the present invention will be described. FIG. 12 is a diagram illustrating a configuration example of an imaging apparatus 10A according to the third embodiment. In FIG. 12, parts corresponding to those in FIG. In the example of FIG. 12, compared with the case of FIG. 1, the optical camera shake reducing unit 50 is replaced with an electronic camera shake reducing unit 60 (corresponding to “camera shake reducing means” in the claims). Other configurations are the same as those in FIG. Here, the electronic camera shake reduction unit 60 moves a plurality of images continuously captured by the image sensor 11 based on the direction of camera shake and synthesizes them in software, thereby reducing image degradation due to camera shake. Generate sharp images.
FIG. 13 is a diagram for explaining the operation of the third embodiment. In FIG. 13, the same reference numerals are given to portions common to FIG. 6, and description thereof is omitted. In FIG. 13, compared with FIG. 6, “imaging processing with optical camera shake reduction processing” in step S <b> 11 is replaced with “imaging processing processing” in step S <b> 70, and “electronic camera shake reduction processing” in step S <b> 71 is added. ing. Other processes are the same as those in FIG.

  When the process shown in FIG. 13 is executed, the shutter speed SS is calculated according to the angular velocities around the X axis, the Y axis, and the Z axis by the processes of steps S1 to S10. In subsequent step S70, the processor 13 controls the imaging unit 40 based on the shutter speed SS determined in step S10 and the aperture and sensitivity determined by the equation (39), and a plurality of images (for example, 2 to 4 images). Image) is continuously taken. A plurality of images captured in this way are stored in the storage unit 14.

  In step S71, the electronic image stabilization unit 60 aligns the plurality of images stored in the storage unit 14 based on the X-axis and Y-axis gyro data sampled during imaging, Combine and superimpose multiple images. That is, the electronic camera shake reduction unit 60 refers to output data of the X-axis gyro sensor 31a and the Y-axis gyro sensor 31b at the time when an image of interest is captured among a plurality of images, detects a motion vector due to camera shake, After adjusting the positional relationship of the images based on the detected motion vector, these images are superimposed and combined. As a result, a sharp image with little deterioration in image quality due to camera shake is obtained. Then, the process ends.

  As described above, in the third embodiment of the present invention, a plurality of images are captured based on the shutter speed SS calculated by the camera shake prevention unit 30, and the electronic camera shake reduction unit 60 refers to the motion vector, A plurality of captured images are aligned and combined. For this reason, since the shutter speed SS is adjusted for the plurality of captured images, it is possible to minimize image quality degradation due to camera shake of the pre-combination image. As a result, the image quality of the combined image can be improved.

  In the third embodiment, the electronic camera shake reduction unit 60 detects a motion vector using the outputs of the X-axis gyro sensor 31a and the Y-axis gyro sensor 31b built in the camera shake prevention unit 30, and uses the detected motion vector. The image was synthesized by referring to it. For this reason, by sharing the gyro sensor, the apparatus can be simplified and the manufacturing cost can be reduced.

(F) Fourth Embodiment of the Present Invention Next, a fourth embodiment of the present invention will be described. The configuration of the fourth embodiment is the same as that of the third embodiment shown in FIG. 12, but the operation of the camera shake prevention unit 30 is different from that of the third embodiment. Below, it demonstrates focusing on a different part.
FIG. 14 is a diagram for explaining the operation of the fourth embodiment. The flowchart shown in this figure is executed when the release button is half-pressed. When the processing of this flowchart is started, the following steps are executed. That is, in step S91, the processor 13 refers to the output of the luminance detection unit 12 and measures the luminance B of the ambient light.
In step S92, the processor 13 compares the ambient light luminance B measured in step S92 with the threshold value Bth, determines whether or not B <Bth is satisfied, and proceeds to step S93 if satisfied. Otherwise, the process proceeds to step S94. For example, when the brightness B of the ambient light is smaller than the brightness threshold Bth, the determination is Yes and the process proceeds to step S93. For example, when the focal length of the optical system is F, the luminance threshold Bth can be set to Bth at which the shutter speed obtained by the AE process in step S3 is 1 / F or more. For example, when F is 50 mm, the luminance B at which the shutter speed SS obtained by the AE process in step S3 is 1/50 second can be set to Bth. Note that other methods may be used.

  When it determines with Yes in step S92, it progresses to step S93, the processor 13 operates the camera shake prevention part 30, and transfers to the process of step S3 of FIG. As a result, in the process of FIG. 13, the shutter speed SS is determined in step S10 based on the camera shake prevention process in step S9, and an image is captured in accordance with the determined shutter speed SS.

On the other hand, if it is determined No in step S92, the process proceeds to step S94, the processor 13 operates the electronic camera shake reduction unit 60, and proceeds to the process of step S3 in FIG. As a result, in the process of FIG. 13, the shutter speed is calculated according to the operation of the camera shake prevention unit 30, and a plurality of images are captured based on the calculated shutter speed. Then, as described above, the plurality of images picked up in this way are combined after being subjected to position adjustment by the electronic camera shake reduction unit 60 to obtain a sharp image with little deterioration in image quality.
If it is determined No in step S92, the electronic camera shake reduction unit 60 may be operated, the operation of the camera shake prevention unit 30 may be stopped, and the process may proceed to step S3 in FIG. In that case, the process of step S9 and step S10 is not executed, and a plurality of images are captured according to the shutter speed SS determined in the AE process of step S3, and the captured images are reduced in electronic camera shake. After the position adjustment is performed by the unit 60, the image is synthesized and a sharp image with little deterioration in image quality is obtained.

  As described above, in the fourth embodiment of the present invention, the camera shake prevention unit 30 or the optical camera shake reduction unit 50 is operated according to the luminance B of the ambient light. For this reason, when the brightness of the ambient light that cannot be sufficiently reduced by the electronic camera shake reducing unit 60 is small, the camera shake preventing unit 30 is operated, and in other cases, the electronic camera shake reducing unit 60 is operated. As a result, even if the brightness of the ambient light is small, it is possible to reliably prevent image quality deterioration due to camera shake.

また、本実施形態では、手ぶれ回転角を角速度で除算して得られる値に基づいてシャッタ速度を算出するようにしたが、この「除算」を「減算シフト」または「乗算」によって実行することも可能である。なお、前者は、各桁の計算を行うときにシフト処理と加減算の組み合わせで処理を行うものであり、後者は、掛け算で処理を行うものである。 (G) Modified Embodiment Although the present invention has been described based on the embodiment, the present invention is not limited to this. For example, in the above embodiment, three gyro sensors are provided, and the shutter speed SS is obtained based on the angular velocities around the X axis, the Y axis, and the Z axis detected by the three gyro sensors. In addition to providing 31c, only the X-axis or Y-axis gyro sensor may be provided, and the shutter speed may be obtained according to the output of these gyro sensors. For example, when only the Z-axis and X-axis gyro sensors are provided, the shutter speed SS can be calculated by using the following formula (43) instead of the formula (38). Similarly, in the case of Expression (40) and Expression (41), items other than the provided gyro sensor are excluded from the denominator, and “3” for obtaining the average value is replaced with “2”. You just have to do it.

In this embodiment, the shutter speed is calculated based on a value obtained by dividing the camera shake rotation angle by the angular speed. However, the “division” may be executed by “subtraction shift” or “multiplication”. Is possible. The former performs processing by a combination of shift processing and addition / subtraction when calculating each digit, and the latter performs processing by multiplication.

  In the above embodiment, the shutter speed SS is calculated and converted into a Tv value as necessary. However, the conversion to the Tv value may be performed on the processor 13 side, or the processor 13 It may be performed on the 33rd side. Further, the processing for obtaining the aperture value Av and the sensitivity value ISO based on the equation (39) may be performed on the processor 13 side or may be performed on the processor 33 side. When it is performed on the processor 33 side, the burden on the processor 13 can be reduced. Further, when the processing is performed on the processor 13 side, the configuration of the processor 33 can be reduced in size.

  Further, in the above embodiment, the camera shake prevention unit 30 is provided with the processor 33 and the processor 33 executes the camera shake prevention process. However, the processor 13 includes the analog-digital conversion unit 32a and the analog-digital conversion unit. 32b and the output from the analog-digital conversion unit 32c may be supplied, and the processor 13 may execute the camera shake prevention process based on a program stored in the storage unit 14. According to such a configuration, the circuit configuration can be reduced by omitting the processor 33.

  In addition, the camera shake prevention unit 30 may be configured as a one-chip, and such a camera shake prevention unit 30 may be incorporated in the imaging apparatus so that the occurrence of camera shake can be prevented. Furthermore, the X-axis gyro sensor 31a, the Y-axis gyro sensor 31b, and the Z-axis gyro sensor 31c may be configured (externally) independent from the camera shake prevention unit 30 or may be built-in. In the above embodiment, a gyro sensor is used as the angular velocity detection device. However, a sensor other than the gyro sensor (for example, a gas type rate sensor) may be used.

  In the above description, the case where a control program for realizing the function of the camera shake prevention device is stored in the storage unit 34 has been described. This control program is stored in a semiconductor recording medium such as a RAM or a ROM, an FD, It is possible to record on a magnetic storage type recording medium such as HD, an optical reading type recording medium such as CD, CDV, LD and DVD, and a magnetic recording type / optical reading type recording medium such as MO. Regardless of the reading method such as electronic, magnetic, optical, etc., any recording medium can be used as long as it is a computer-readable recording medium. Then, by reading and executing the control program recorded in these recording media by the processor 33, the imaging apparatus 10 is further provided with a network interface as a communication interface, and the control program is transmitted from the network interface via the network. It is good also as a structure which implement | achieves the function mentioned above by downloading and executing.

It is a block diagram which shows the structure of the imaging device which concerns on embodiment. It is a figure which shows the detailed structural example of a camera shake prevention part. It is a figure which shows the detailed structural example of the imaging part which has an image pick-up element. It is a figure which shows the mode of the movement on the image pick-up element of the target point by camera shake. It is a conceptual diagram which shows an imaging model. It is a flowchart which shows an example of the process performed in 1st Embodiment. It is a figure which shows the correspondence of a user's operation and the flow of a process. It is a figure which shows the relationship between shutter speed and Tv value. It is a flowchart which shows an example of the process performed in 2nd Embodiment. It is an example of the screen displayed when the process of FIG. 9 is performed. It is a flowchart which shows an example of the process performed in 2nd Embodiment. It is a block diagram which shows the structural example of 3rd Embodiment. It is a flowchart which shows an example of the process performed in 3rd Embodiment. It is a flowchart which shows an example of the process performed in 4th Embodiment.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 10 ... Imaging device, 11 ... Imaging device, 12 ... Luminance detection part, 13 ... Processor (control means), 14 ... Storage part, 15 ... User interface, 30 ... Shake prevention part, 31a ... X-axis gyro sensor (angular velocity detection) Device), 31b ... Y-axis gyro sensor (angular velocity detection device), 31c ... Z-axis gyro sensor (angular velocity detection device), 32a ... analog-digital converter (input means), 32b ... analog-digital converter (input means) 32c ... analog-digital conversion unit (input means), 33 ... processor (determination means, calculation means), 34 ... storage unit, 40 ... imaging unit, 41 ... housing, 43 ... lens, 50 ... optical shake reduction unit ( (Camera shake reducing means), 60... Electronic camera shake reducing unit (camera shake reducing means).

Claims (7)

In an imaging device having an imaging device and an angular velocity detection device that detects an angular velocity around an axis of the imaging device,
An input unit that receives an input of a detection signal output from the angular velocity detection device when an image is captured by the image sensor;
Determining means for determining a camera shake rotation angle allowed around the axis at the time of imaging by the image sensor;
Calculating means for calculating a shutter speed based on a value obtained by dividing the permissible camera shake rotation angle determined by the determining means by an angular speed determined by the detection signal input via the input means; ,
Control means for controlling the imaging device based on the shutter speed calculated by the calculating means to perform imaging;
When image capturing is performed by the control unit, camera shake is reduced by controlling an optical system included in the image sensor, or camera shake is reduced by performing image processing on an image captured by the image sensor. Camera shake reduction means,
An imaging device comprising: The imaging device according to claim 1,
The angular velocity detection device detects an angular velocity around a vertical or horizontal axis of the image sensor,
The determining means determines an allowable camera shake rotation angle around the vertical or horizontal axis;
The calculation means divides the camera shake rotation angle allowed around the vertical or horizontal axis by the corresponding angular velocity around the vertical or horizontal axis, and calculates the shutter speed based on the obtained value. calculate,
An imaging apparatus characterized by that. The imaging apparatus according to claim 1 or 2,
The angular velocity detection device detects an angular velocity around an axis in a normal direction of an imaging surface of the imaging element;
The determining means determines a camera shake rotation angle allowed around an axis in the normal direction;
The calculation means calculates a shutter speed based on a value obtained by dividing a camera shake rotation angle allowed around the normal axis by a corresponding angular velocity around the normal axis;
An imaging apparatus characterized by that. The imaging device according to any one of claims 1 to 3,
The camera shake reducing means reduces camera shake by controlling the optical system of the image sensor,
The calculation unit calculates a shutter speed based on a portion exceeding the reduction range when the shake exceeds a reduction range by the shake reduction unit.
An imaging apparatus characterized by that. The imaging device according to any one of claims 1 to 3,
The camera shake reducing means reduces camera shake by performing image processing on a captured image,
When the ambient light is higher than a predetermined brightness, the control unit causes the image sensor to take an image based on a shutter speed set by the imaging device or the user, and the ambient light is lower than the predetermined brightness. In this case, the image sensor is caused to capture an image based on the shutter speed calculated by the calculation unit.
An imaging apparatus characterized by that. In a control method for an imaging device, comprising: an imaging device; and an angular velocity detection device that detects an angular velocity around an axis of the imaging device.
An input step of receiving an input of a detection signal output from the angular velocity detection device when an image is captured by the imaging element;
A determination step of determining a camera shake rotation angle allowed around the axis at the time of imaging by the imaging device;
A calculation step of calculating a shutter speed based on a value obtained by dividing the allowable camera shake rotation angle determined in the determination step by an angular velocity obtained by the detection signal input in the input step;
A control step for performing control of imaging with an image sensor at the shutter speed calculated in the calculation step;
Camera shake that reduces camera shake by controlling an optical system included in the image sensor when imaging by the image sensor, or camera shake that reduces image blur by performing image processing on an image captured by the image sensor Mitigation steps,
A control method for an imaging apparatus, comprising: In a control program for an imaging device, comprising: an imaging device; and an angular velocity detection device that detects an angular velocity around an axis of the imaging device.
An input unit that receives an input of a detection signal output from the angular velocity detection device when an image is captured by the image sensor;
Determining means for determining a camera shake rotation angle allowed around the axis at the time of imaging by the image sensor;
Calculating means for calculating a shutter speed based on a value obtained by dividing the allowable camera shake rotation angle determined by the determining means by an angular speed obtained by the detection signal input via the input means;
Control means for performing control of imaging with the imaging device at the shutter speed calculated by the calculating means;
Camera shake that reduces camera shake by controlling an optical system included in the image sensor when imaging by the image sensor, or camera shake that reduces image blur by performing image processing on an image captured by the image sensor Mitigation measures,
An anti-shake program that allows the computer to function as a computer.

Images