JP2014099832A - Camera - Google Patents

Abstract

PROBLEM TO BE SOLVED: To achieve proper inclination correction execution control, without imposing a burden on a photographer.SOLUTION: In a camera, a camera inclination angle (θ[i]) around an optical axis is successively detected. When the inclination state of the camera is stable, a reference angle θis set based on the camera inclination angle at that time and a correction flag is turned on (steps S16 to S18). After that, an inclination correction (step 20) for correcting the inclination angle of an output image to the reference angle θis executed. When the camera inclination angle larger than a threshold θis detected or an inclination change in the same direction is continuously detected for a fixed period, during the execution of the inclination correction, the correction flag is turned off to stop the inclination correction (steps S21 to S25).

Description

  The present invention relates to a camera.

  A small camera is often used while being held in hand, and the camera is often shaken particularly when shooting while walking or when performing a shutter operation. A method of detecting and correcting tilt around the optical axis (rotational shake in the roll direction) among camera shakes has been proposed (see, for example, Patent Document 1 below). This type of tilt correction automatically detects the camera around the optical axis so that the horizontal line in the real space is horizontal in the captured image (so that it faces the horizontal direction of the captured image), regardless of user operation. Based on the tilt angle, rotation driving or image processing of the image sensor is performed.

  Also disclosed is a method of adjusting the tilt correction amount around the optical axis in accordance with the manual operation of the photographer (see, for example, Patent Document 2 below).

JP 2008-151822 A JP 2012-4899 A

  Inclination correction (hereinafter referred to as “automatic leveling process”) for automatically leveling a horizontal line in real space on a captured image regardless of a user operation functions beneficially in many situations. However, in a conventional camera having a tilt correction function, an automatic leveling process is performed even when the photographer wants to shoot with a composition tilted around the optical axis and the camera is intentionally tilted around the optical axis. , The photographer may not be able to perform the desired shooting.

  For example, as shown in FIG. 19, it is assumed that the photographer wants to photograph only the flower 911 with respect to a landscape in which a tree branch 912 extends obliquely above the flower 911. When the camera 1 is kept horizontal, the branch 912 enters the shooting area 910. Therefore, it is assumed that the photographer tilts the camera 1 around the optical axis to exclude the branch 912 from the shooting area 910. However, when the automatic leveling function is enabled, tilt correction is performed so that the tilt of the camera 1 is canceled, and the branch 912 continues to be in the captured image (even if the camera 1 is tilted, FIG. 19). The lower right shot image will be obtained). That is, the automatic leveling function hinders the photographer's operation. If the manual operation to turn off the automatic leveling function is input to the camera, the composition as intended can be obtained. Alternatively, when the tilt correction amount is adjusted manually (see Patent Document 2), the intended composition can be obtained. However, switching the tilt correction on / off by manual operation and adjusting the tilt correction amount by manual operation are troublesome for the user.

  Accordingly, an object of the present invention is to provide a camera that contributes to the realization of appropriate tilt correction execution control without imposing a burden on the photographer.

  A camera according to the present invention includes an imaging unit having an optical system and an image sensor that outputs an optical image signal formed through the optical system, and obtains an output image from an output signal of the image sensor. An inclination detector that detects a camera inclination angle around the optical axis of the optical system and an inclination correction that corrects the inclination angle of the output image due to the inclination of the camera to a reference angle based on the camera inclination angle. A tilt correction unit, and a tilt correction control unit that controls whether to perform tilt correction or sets the reference angle based on the size of the camera tilt angle or a plurality of camera tilt angles at a plurality of times. It is characterized by having.

  Based on the size of the camera tilt angle or multiple camera tilt angles at multiple times, the photographer ’s intention (whether or not he / she wants to perform tilt correction) Can be guessed). Based on the estimation results, it is possible to control whether or not to perform tilt correction and set a reference angle that can be said to be a correction target in a way that suits the photographer's intention without requiring any special manual operation. It becomes.

  Specifically, for example, the tilt correction control unit may stop the tilt correction when the camera tilt angle having a magnitude larger than a predetermined threshold is detected during the tilt correction.

  It can be inferred that a state in which a relatively large camera tilt angle is detected corresponds to a state in which the photographer tilts the camera around the optical axis with a clear intention. Therefore, when such a state is observed, it is considered that stopping the execution of the inclination correction is in line with the photographer's intention. At the time of the stop, no manual operation of the photographer is necessary, so there is no particular burden on the photographer.

  More specifically, for example, the tilt correction control unit sets the reference angle based on a plurality of camera tilt angles during the first period and starts the tilt correction, and then performs the predetermined threshold in the second period. When the camera tilt angle having a larger size is detected, the execution of the tilt correction may be stopped.

  Alternatively, for example, the tilt correction control unit may stop execution of the tilt correction when a plurality of camera tilt angles continuously changing in the same direction in time series are detected during the tilt correction. good.

  It can be inferred that the state in which a continuous tilt change in one direction is detected corresponds to a state in which the photographer rotates the camera around the optical axis with a clear intention. Therefore, when such a state is observed, it is considered that stopping the execution of the inclination correction is in line with the photographer's intention. At the time of the stop, no manual operation of the photographer is necessary, so there is no particular burden on the photographer.

  More specifically, for example, the tilt correction control unit sets the reference angle based on a plurality of camera tilt angles during the first period and starts execution of the tilt correction. When a plurality of camera tilt angles that continuously change in the same direction are detected, it is preferable to stop the tilt correction.

  Further, for example, the tilt correction control unit determines whether the tilt state of the camera around the optical axis is stable based on a plurality of camera tilt angles obtained before the tilt correction is performed, and the tilt state is stable. In this case, the tilt correction may be started by setting the reference angle based on a plurality of camera tilt angles obtained before the tilt correction is performed.

  When it is determined that the tilt state is stable based on a plurality of camera tilt angles, it is estimated that the photographer has an intention to shoot the subject with a composition having the tilt. In such a case, it is considered that it is more in line with the photographer's intention to perform the tilt correction with a reference angle based on a plurality of camera tilt angles as a target, rather than the conventional simple tilt correction. There is no particular burden on the photographer when setting the reference angle, which can also be called a correction target (manual operation that is necessary in the method of Patent Document 2 is unnecessary).

  More specifically, for example, the tilt correction unit generates the output image from the output signal of the image sensor by rotating the image sensor around the optical axis based on the camera tilt angle in the tilt correction. Alternatively, the output image may be generated by performing image processing including image rotation processing on the input image based on the output signal of the image sensor based on the camera tilt angle.

  Further, for example, the tilt correction control unit may return the once set reference angle to a predetermined initial angle according to the change rate of the camera tilt angle or the movement of the camera other than the tilt.

  According to the present invention, it is possible to provide a camera that contributes to realization of appropriate tilt correction execution control without imposing a burden on the photographer.

1 is a schematic overall block diagram of a camera according to an embodiment of the present invention. It is an internal block diagram of the imaging part which concerns on embodiment of this invention. FIG. 3 is a diagram (a) and (b) showing how an input image is obtained from the output of the image sensor of FIG. 2, and a diagram (c) showing the relationship between the input image and an XY coordinate plane. It is a figure which shows the relationship between a camera, an optical axis, and a to-be-photographed object. It is a figure which shows the relationship between a camera and an optical axis, and the horizontal reference line which is a kind of to-be-photographed object. It is a figure which shows the mode of the display screen observed on three conditions. It is a figure which shows the input image obtained on three conditions. It is a figure for demonstrating the rotation angle of an image pick-up element. FIG. 2 is an internal block diagram of a main processing / control unit in FIG. 1. It is a figure for demonstrating the content of an image rotation process. It is a figure for demonstrating the inclination angle in an output image. It is a figure which shows the relationship between a camera inclination angle and a reference angle (correction target). It is a figure for demonstrating the content of the inclination correction by IS rotation process. It is a figure for demonstrating the content of the inclination correction by an image rotation process. 5 is an operation flowchart of the camera according to the embodiment of the present invention. It is a figure which shows the mode of the camera inclination in a 1st case, and the transition of a display screen. It is a figure which shows the mode of the camera tilt in a 2nd case, and the transition of a display screen. It is a figure which shows the mode of the transition of the camera inclination in a 3rd case, and a display screen. It is a figure which shows the content of the inclination correction (automatic leveling process) in a conventional camera.

  Hereinafter, an example of an embodiment of the present invention will be specifically described with reference to the drawings. In each of the drawings to be referred to, the same part is denoted by the same reference numeral, and redundant description regarding the same part is omitted in principle. In this specification, for simplification of description, a symbol or reference that refers to information, signal, physical quantity, state quantity, member, or the like is written to indicate information, signal, physical quantity, state quantity or Names of members and the like may be omitted or abbreviated.

  FIG. 1 is a schematic overall block diagram of a camera 1 according to an embodiment of the present invention. A camera as an imaging device is a digital video camera capable of capturing and recording still images and moving images, or a digital still camera capable of capturing and recording only still images. The camera 1 is provided with each part referred by the codes | symbols 11-15.

  The imaging unit 11 captures a subject using an imaging element. FIG. 2 is an internal configuration diagram of the imaging unit 11. The imaging unit 11 includes an optical system 35 formed by a plurality of lenses including a zoom lens 30 and a focus lens 31, a diaphragm 32, a CCD (Charge Coupled Device), a CMOS (Complementary Metal Oxide Semiconductor) image sensor, and the like. An image sensor (solid-state image sensor) 33 and a driver 34 for driving and controlling the optical system 35 and the diaphragm 32 are provided. As will be described later, the image sensor 33 can also be rotationally driven by the driver 34. The main processing / control unit 12 can control the positions of the lenses 30 and 31 and the opening of the diaphragm 32 (that is, the diaphragm value) and the rotation of the image sensor 33 via the driver 34. The imaging element 33 outputs an optical image signal formed on its imaging surface via the optical system 35. The axis 300 is the optical axis of the optical system 35 (in other words, the optical axis of the imaging unit 11 or the camera 1).

  The main processing / control unit 12 is formed of a signal processing circuit, a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), and the like, and comprehensively operates each part in the camera 1. Control. The display screen 13 is a display screen such as a liquid crystal display panel, and displays an arbitrary video under the control of the main processing / control unit 12. The recording medium 14 is a nonvolatile memory such as a semiconductor memory or a magnetic disk, and records a photographed image or the like of the imaging unit 11 under the control of the main processing / control unit 12. The operation unit 15 receives various operations from the outside. The content of the operation on the operation unit 15 is transmitted to the main processing / control unit 12. The operation unit 15 may be formed using a touch panel.

  An optical image formed on the imaging surface of the imaging device 33, that is, an image formed on the imaging surface of the imaging device 33 is referred to as an input image IA (see FIG. 3A). It may be considered that an image obtained by performing predetermined signal processing (signal amplification processing, demosaicing processing, noise removal processing, etc.) on an image formed on the imaging surface of the imaging device 33 is the input image IA. FIG. 3B shows an example of the input image IA. The input image IA in FIG. 3B includes an image of the subject SUB. The input image IA or an image based on the input image IA can be displayed on the display screen 13. Further, as shown in FIG. 3C, the input image IA and the image pickup surface of the image pickup device 33 are considered to be defined on an XY coordinate plane having the X axis and the Y axis as coordinate axes. The X axis is parallel to the horizontal direction of the imaging surface of the imaging element 33 and the horizontal direction of the input image IA, and the Y axis is parallel to the vertical direction of the imaging surface of the imaging element 33 and the vertical direction of the input image IA. The X axis and the Y axis are orthogonal at the origin O of the XY coordinate plane. The origin O is located at the center of the input image IA and the center of the imaging surface.

  FIG. 4 shows the relationship between the camera 1 and the subject SUB. The movement of the camera 1 includes a movement in the roll direction that rotates the camera 1 around the optical axis 300 (in other words, rotates the camera 1 about the optical axis 300 as a rotation axis). The tilt of the camera 1 around the optical axis 300 due to the movement in the roll direction is referred to as the tilt in the roll direction. The angle of tilt in the roll direction in the camera 1 is called a camera tilt angle, which is represented by the symbol θ. When the IS rotation angle φ (see FIGS. 8A and 8B), which is the rotation angle of the image sensor 33 around the optical axis 300, is 0 °, the subject SUB in the input image IA depends on the inclination in the roll direction. Tilt. The camera 1 has a function of correcting such tilt.

  In order to clarify and clarify the description of the function, in the following, as shown in FIG. 5, a horizontal reference line HL that is a kind of subject in the real space is fixedly present, and within the imaging region of the imaging unit 11. It is assumed that the horizontal reference line HL is always within the range. The horizontal reference line HL is a line segment parallel to the horizontal direction in real space (and therefore a line segment orthogonal to the vertical direction), and one end of the horizontal reference line HL is located on the optical axis 300.

The camera 1 takes various postures, and one of the postures is referred to as a reference posture. In the reference posture, the optical axis 300 is parallel to the horizontal direction of the real space and orthogonal to the horizontal reference line HL. In the reference posture, there is no tilt in the roll direction, and therefore the camera tilt angle θ is 0 °. When viewed from the photographer who has the camera 1 facing the horizontal reference line HL, when the camera 1 is rotated clockwise by an angle θ _J from the reference posture, “θ = θ _J ” is obtained, and the camera 1 is in the reference posture. Is rotated counterclockwise by an angle θ _J, “θ = −θ _J ”. 6A to 6C show a state where the input image IA is displayed on the display screen 13 as it is when the IS rotation angle φ is 0 °. 7A to 7C show the input image IA acquired when the IS rotation angle φ is 0 °. However, “θ = 0 °” in FIGS. 6A and 7A, “θ = θ _J ” in FIGS. 6B and 7B, and FIGS. In FIG. 7C, “θ = −θ _J ” (θ _J > 0 °).

When the IS rotation angle φ is 0 ° in the reference posture, the horizontal reference line HL is parallel to the X axis (and thus orthogonal to the Y axis) on the input image IA and the imaging surface, and is positive from the origin O to the X axis. It extends in the direction (see FIG. 7A). When the IS rotation angle φ is 0 °, the horizontal reference line HL rotates counterclockwise around the origin O on the input image IA as the camera tilt angle θ increases positively from 0 °. When the IS rotation angle φ is 0 °, the angle ε _A formed by the horizontal reference line HL on the input image IA and the X axis coincides with the camera tilt angle θ including the polarity (FIG. 7B). ) And (c)). Although the camera tilt angle θ varies within a range that satisfies “−180 ° <θ ≦ −180 °”, it will be assumed below that “−90 ° <θ ≦ −90 °” holds for convenience of explanation.

  As shown in FIGS. 8A and 8B, the IS rotation angle φ is the rotation angle of the image sensor 33 around the optical axis 300 (that is, the rotation angle of the image sensor 33 with the optical axis 300 as the rotation axis). is there. In the reference posture, when the IS rotation angle φ is 0 °, the horizontal and vertical directions of the image sensor 33 coincide with the horizontal and vertical directions of the real space. A state where the IS rotation angle φ is 0 ° is referred to as an IS rotation zero state. If the IS rotation angle φ is set to a value other than 0 ° in the reference posture, the horizontal direction (and hence the X axis) of the image sensor 33 is inclined by the IS rotation angle φ from the horizontal direction of the real space. As shown in the perspective view of FIG. 8C, the angle when the image sensor 33 is rotated counterclockwise from the IS rotation zero state when viewed from the photographer having the camera 1 facing the horizontal reference line HL. Suppose that the polarity of φ is positive.

  As shown in FIG. 9, the main processing / control unit 12 includes an output image generation unit 50, an inclination detection unit 51, an inclination correction unit 52, and an inclination correction control unit 53.

  The output image generation unit 50 generates an output image IB by performing predetermined image processing (signal processing) on the input image IA. The image processing can include noise removal processing, edge enhancement processing, and the like, and can also include image rotation processing for tilt correction described later. When the image rotation process is not performed, the input image IA and the output image IB can be completely or substantially the same image. When the input image IA and the output image IB are completely the same, the output image generation unit 50 can be omitted from the camera 1. The output image IB can be displayed on the display screen 13 and can be recorded on the recording medium 14.

  Image processing including image rotation processing will be described. When the image rotation process is executed, the input image IA is acquired with the IS rotation angle φ being 0 °. Please refer to FIG. In the image rotation process, the output image generation unit 50 sets a cutout frame CF, which is a rectangular frame having a center at the center (origin O) of the input image IA and a common aspect ratio to the input image IA, as the input image IA. Then, the image in the cutout frame CF is cut out from the input image IA. At this time, the cutout frame CF is inclined by the angle α with respect to the input image IA. Assume that the polarity of the angle α when the clipping frame CF is rotated counterclockwise on the input image IA is positive with reference to the state where the clipping frame CF is not inclined with respect to the input image IA. While satisfying the condition that the entire cutout frame CF is included in the input image IA, a cutout frame CF that is as large as possible is set in the input image IA. In the image rotation process, the output image generation unit 50 performs geometric transformation that rotates the image in the cutout frame CF cut out from the input image IA clockwise by an angle of (−α), thereby outputting the output image IB. (The image in the cutout frame CF that has undergone the geometric transformation is the output image IB). An image enlargement process for matching the image size of the output image IB with a specified size may be included in the geometric transformation. Here, geometric transformation including image rotation is performed after the clipping process, but the clipping process may be performed after the entire input image IA is rotated.

Further, although the method for setting the cutout frame CF as large as possible has been described above, the following setting method may be employed instead of the setting method. A predetermined rotation limit angle α _LIM is set (0 ° <α _LIM <90 °), and the minimum inscribed in the outer frame of the input image IA is within a range where “−α _LIM ≦ α ≦ α _LIM ” is satisfied. The cutout frame CF is obtained in advance. This minimum cutout frame CF is a cutout frame CF having the smallest size among the cutout frames CF inscribed in the outer frame of the input image IA within the above range. In the image rotation process, the size of the cutout frame CF is always matched with the minimum cutout frame CF. Thereby, the angle-of-view variation of the output image IB can be eliminated within the above range.

  The tilt detection unit 51 can detect the camera tilt angle θ using a sensor. For example, the tilt detection unit 51 may detect the camera tilt angle θ using an acceleration sensor that detects the acceleration of the camera 1. At this time, the acceleration of the camera 1 in the directions of two axes orthogonal to the optical axis 300 may be detected using two acceleration sensors, and the camera tilt angle θ may be detected based on the detection results of the two acceleration sensors. . Further, for example, the tilt detection unit 51 may detect the camera tilt angle θ using a gyro sensor that detects the angular velocity of the camera 1 in the roll direction. Further, for example, the tilt detection unit 51 may use a tilt sensor that directly detects the camera tilt angle θ itself.

  The tilt detection unit 51 may detect the camera tilt angle θ based on the output signal of the image sensor 33. As a method for detecting the camera tilt angle based on the output signal of the image sensor 33, a known method (for example, a method described in Japanese Patent Application Laid-Open No. 2007-306500) can be used. When an object scene for the camera 1 is captured as an image, the object scene usually includes many edges parallel to a vertical line and a horizontal line. For example, when a building, furniture, a person in an upright position, a horizon line, and the like are captured as an image, they include many edges parallel to the vertical line and / or the horizontal line. Therefore, if the direction in which the edge extends in the input image IA is detected based on the image signal of the input image IA and the detection direction is statistically processed, the vertical line on the input image IA corresponding to the vertical line and the horizontal line in the real space. The direction of the horizontal line can be determined, and the camera tilt angle θ can be obtained from the relationship between the determined direction and the X and Y axes.

The tilt correction unit 52 is based on the camera tilt angle θ detected by the tilt detection unit 51, in other words, tilt correction that corrects the tilt angle of the output image IB due to the tilt of the camera 1 in the roll direction to the reference angle θ _REF , in other words. Inclination correction can be executed to match the inclination angle of the output image IB with the inclination of the camera 1 in the roll direction to the reference angle θ _REF . The inclination angle of the output image IB corresponds to the inclination angle ε _B of the horizontal reference line HL in the output image IB (see FIG. 11). In the state where the horizontal reference line HL is oriented in the horizontal direction in the output image IB, the angle ε _B is 0 °, and with reference to this state, as shown in FIG. 11, the output image IB around the center of the output image IB. When the horizontal reference line HL rotates in the counterclockwise direction above, the angle ε _B has a positive value corresponding to the rotation angle (conversely, when rotated in the clockwise direction, the angle is equal to the rotation angle. ε _B has a negative value).

The reference angle θ _REF in the tilt correction functions as a correction target. By executing the tilt correction, an image as if shooting was performed with the camera tilt angle θ set to the reference angle θ _REF is obtained as the output image IB. Therefore, the correction amount in the inclination correction is represented by “θ−θ _REF ” (see FIG. 12). The tilt correction unit 52 realizes the tilt correction by IS rotation processing or image rotation processing.

In the IS rotation process, the tilt correction unit 52 rotates the image sensor 33 around the optical axis 300 via the driver 34 (FIG. 2). That is, in tilt correction by IS rotation processing, the tilt correction unit 52 rotates the image sensor 33 from the IS rotation zero state by the correction amount “θ−θ _REF ” so that “φ = θ−θ _REF ”. When the tilt correction is realized by the IS rotation process, the input image IA is generated as it is as the output image IB. However, signal processing (such as noise removal processing) that does not involve image rotation may be involved in the process of generating the output image IB from the input image IA.

Specifically, for example, when θ = 25 ° and θ _REF = 0 °, when tilt correction by IS rotation processing is executed, φ = 25 ° and the image 351 in FIG. 13A is input. Obtained as an image IA and an output image IB. On the image 351, the horizontal reference line HL faces the horizontal direction of the image 351 (that is, ε _A = ε _B = 0 °).

Further, for example, when θ = 25 ° and θ _REF = 15 °, when tilt correction by IS rotation processing is executed, φ = 10 ° is set, and an image 352 in FIG. Obtained as an output image IB. On the image 352, the horizontal reference line HL is inclined by 15 ° (= θ _REF ) from the horizontal direction of the image 351 (that is, ε _A = ε _B = 15 °).

When the tilt correction is realized by the image rotation process (see FIGS. 14A and 14B), the output image generation unit 50 cuts the cutout frame CF into the input image IA obtained with the IS rotation angle φ set to 0 °. Is set, and geometrical transformation is performed to rotate the image in the cutout frame CF by the correction amount “θ−θ _REF ” to generate the output image IB. At this time, the inclination correction unit 52 controls the setting of the cutout frame CF and the cutout operation in the output image generation unit 50 based on the angles θ and _θREF .

Specifically, for example, in the case where θ = 25 ° and θ _REF = 0 °, when performing tilt correction by image rotation processing, first, the image 361 in FIG. 14A in which the angle ε _A is 25 ° is shown. Is obtained as the input image IA. The output image generation unit 50 includes a cutout frame CF (that is, a cutout frame CF with α = 25 °; see also FIG. 10) inclined by 25 ° (= θ− _θREF = 25 ° −0 °) in the image 361. The image 362 as the output image IB is obtained by executing geometric transformation that is set and rotated by 25 ° clockwise (−25 ° in the counterclockwise direction) in the cutout frame CF cut out from the image 361. . On the image 362, the horizontal reference line HL faces the horizontal direction of the image 362 (that is, ε _B = 0 °).

For example, in the case where θ = 25 ° and θ _REF = 15 °, when the inclination correction by the image rotation process is executed, first, the image 361 in FIG. 14B in which the angle ε _A is 25 ° is input. Obtained as an image IA. However, in this case, the output image generation unit 50 causes the cutout frame CF tilted by 10 ° (= θ− _θREF = 25 ° −15 °) on the image 361 (that is, the cutout frame CF of α = 10 °; 10 is also set), and an output image IB is obtained by executing a geometric transformation that rotates the image in the cutout frame CF cut out from the image 361 by 10 ° clockwise (−10 ° counterclockwise). Image 363 is obtained. On the image 363, the horizontal reference line HL is inclined by 15 ° (= θ _REF ) from the horizontal direction of the image 363 (ie, ε _B = 15 °).

  Note that when the tilt correction is realized by image rotation processing, it can be said that the tilt correction is performed by the output image generation unit 50. Therefore, it may be considered that the output image generation unit 50 is included in the inclination correction unit 52.

The tilt correction control unit 53 in FIG. 9 controls whether or not to perform the tilt correction described above or sets the reference angle θ _REF based on one or more camera tilt angles θ detected at one or more times. Details of the function of the inclination correction control unit 53 will be apparent in the description of FIG.

FIG. 15 is an operation flowchart of the camera 1 in the shooting mode. In the operation of FIG. 15, it is assumed that the tilt correction function in the camera 1 is enabled. A user as a photographer can use the operation unit 15 to switch between valid / invalid of the tilt correction function. When the camera 1 is activated and the operation mode of the camera 1 is set to the shooting mode, the control unit 53 performs an initialization process in step S11. In the initialization process, the correction flag is set to OFF, the initial angle 0 ° is set to the reference angle θ _REF , and 1 is substituted into the variable i. The correction flag is a flag for designating whether or not the tilt change of the camera 1 in the roll direction is sequentially corrected between frames.

After the initialization process, the inclination detection unit 51 detects the inclination detection angle θ [i] in step S12. In the shooting mode, the input image IA (and thus the output image IB) is sequentially obtained by sequential shooting at a predetermined frame period, and θ [i] is an inclination detection angle θ detected with respect to the i-th frame. It is. The input image IA obtained by shooting at the time t _i may be considered as the i-th frame. Time t _{i + 1} is a time after one frame period from time t _i . In step S13 subsequent to step S12, the control unit 53 calculates an inclination change amount Δθ between the previous frame and the current frame in accordance with “Δθ = θ [i] −θ [i−1]” (note that θ [0] is assumed to be zero). The control unit 53 holds the necessary values of θ [i] and Δθ in order to perform each process described later.

In step S14 following step S13, the control unit 53 checks the correction flag, and when the correction flag is on, causes the transition from step S14 to step S20 while the correction flag is off. Causes a transition from step S14 to step S15. In step S15, the control unit 53 transitions from the reference angle theta _REF confirms whether or not 0 °, if the reference angle theta _REF is not 0 ° by performing the rotation processing in step S30 to step S16 When the reference angle θ _REF is 0 °, a direct transition to step S16 is generated. Immediately after the start of the operation in FIG. 15, since the correction flag is off and the reference angle θ _REF is 0 °, the process goes through steps S14 and S15 to step S16 without going through step S30 (in step S30). The rotation process will be described later).

In step S16, the control unit 53 determines whether the camera tilt state is stable based on the plurality of camera tilt angles θ obtained during the evaluation period, and determines that the camera tilt state is stable. In step S17, the reference angle θ _REF is set or updated, and the correction flag is turned on in step S18, and then the transition to step S26 occurs. On the other hand, if it is determined that the camera tilt state is not stable, the control unit 53 causes a direct transition from step S16 to step S26. The evaluation period is a period between the current time t _i that is the time for the determination in step S16 and the time t _im that is a predetermined time before the current time t _i (m is an integer of 1 or more). The length of the evaluation period is, for example, 1 to 3 seconds. As will be apparent from the following description, since the detection of the camera tilt angle θ (step S12) is performed for each frame, a total of (m + 1) camera tilt angles θ [i−m] ˜ θ [i] is detected, and a total of m inclination change amounts Δθ based on the angles θ [i−m] to θ [i] are calculated.

For example, the control unit 53 determines that the camera tilt state is stable when the absolute values of the respective tilt change amounts Δθ calculated during the evaluation period are all equal to or less than a predetermined value (for example, 1 °), and is not so. In this case, it is determined that the camera tilt state is not stable. At this time, even if the absolute values of the respective inclination change amounts Δθ acquired during the evaluation period are all equal to or less than a predetermined value, the camera inclination angle θ changes in the same direction during the evaluation period (ie, In the case of continuous increase or decrease), it may be determined that the camera tilt state is exceptionally not stable.
Alternatively, for example, the control unit 53 determines that the camera tilt state is stable when the variance of the camera tilt angles θ [im] to θ [i] obtained during the evaluation period is equal to or less than a predetermined value. Otherwise, it may be determined that the camera tilt state is not stable.

In step S17, the average angle of the camera tilt angle θ detected during the evaluation period (that is, the average angle of θ [i−m] to θ [i]) is set as the reference angle θ _REF . When the camera tilt state is stable (for example, when Δθ is very small for a certain period), the photographer's intention to determine the shooting composition is estimated based on the camera tilt at that time. A reference angle θ _REF to be called a correction target is set based on (step S17), and a correction flag is turned on (step S18).

In step S26, the output image IB is displayed on the display screen 13. In a state where the correction flag is off and the reference angle θ _REF is zero, the input image IA in a state where neither the tilt correction in step S20 nor the rotation processing in step S30 is performed is displayed as the output image IB as it is. 13 is displayed. In step S27 following step S26, it is confirmed whether or not the operation mode of the camera 1 is maintained in the shooting mode. If the operation mode of the camera 1 is maintained in the shooting mode, the variable i is set in step S28. After adding 1, the process returns to step S12. The process from step S12 to step S28 is repeatedly executed for each frame period. When the operation mode of the camera 1 is changed to a mode other than the shooting mode (reproduction mode or the like) or when the power of the camera 1 is turned off, the operation of FIG. 15 is terminated (N in step S27).

After the reference angle θ _REF is set and the correction flag is turned on in steps S17 and S18, a transition from step S14 to step S20 occurs. In step S <b> 20, the inclination correction unit 52 performs the above-described inclination correction in which “θ [i] −θ _REF ” is set as the correction amount. The tilt correction in step S20 may be tilt correction by IS rotation processing described with reference to FIGS. 13A and 13B, or image rotation processing described with reference to FIGS. 14A and 14B. Tilt correction by means of In step S20, inclination correction is performed by replacing "θ" with "θ [i]" in FIGS. 13A, 13B, 14A, 14B, and the explanatory texts of those drawings. .

After step S20, the output image IB obtained through the tilt correction is displayed on the display screen 13 when step S26 is reached via step S21 or the like. That is, for example, if (θ [i], θ _REF ) = (25 °, 0 °), the image 351 in FIG. 13A or the image 362 in FIG. 14A is generated as the output image IB on the display screen 13. If (θ [i], θ _REF ) = (25 °, 15 °), the image 352 in FIG. 13B or the image 363 in FIG. 14B is generated as the output image IB and displayed on the display screen 13. Is displayed.

In the process from step S20 to step S26, branch determination of steps S21 and S22 is involved. That is, after step S20, in step S21, the control unit 53 compares the absolute value | θ [i] |, which is the magnitude of the current camera tilt angle θ [i], with a predetermined positive determination angle θ _TH ( For example, when θ _TH = 30 °) and | θ [i] | are larger than θ _TH , a transition to step S23 is generated, and 0 ° is set to the reference angle θ _REF in steps S23 and S25. Further, after the correction flag is turned off, a transition to step S26 is generated. When | θ [i] | is equal to or _smaller than θTH, a transition from step S21 to step S22 occurs.

In step S <b> 22, the control unit 53 determines whether or not the inclination change direction during a certain period is the same. When the inclination change direction during a certain period is the same, the control unit 53 generates a transition to step S24, updates the reference angle θ _REF in steps S24 and S25, and turns off the correction flag. The transition from step S26 to step S26 is generated. If it is determined that the inclination change directions during a certain period are not the same, a direct transition from step S22 to step S26 occurs.

In step S22, the control unit 53 sets a determination period that is a period between the current time t _i that is the time when the determination of step S22 is performed and the time t _im that is a predetermined time before the current time t _i . The determination in step S22 is performed based on a total of (m + 1) camera tilt angles θ [i−m] to θ [i] during the determination period. The length of the determination period is, for example, 1 to 3 seconds. The control unit 53, for example, when a total of m inclination change amounts Δθ based on the angles θ [i−m] to θ [i] during the determination period have the same polarity (that is, in the same direction on the time series) When a plurality of camera tilt angles θ that continuously change are detected), it is determined that the tilt change direction during a certain period is the same, and otherwise, the tilt change direction during the certain period is not the same. judge.

If it is determined at time t _i that the tilt change directions during the certain period are the same, the control unit 53 determines, at step S24, based on the plurality of camera detection angles θ during the determination period, or a plurality during the determination period. The reference angle θ _REF is set and updated based on one of the camera detection angles θ. Specifically, when the angles θ [im] to θ [i] are obtained during the determination period, in step S24 at time t _i , the latest inclination detection angle θ is set to θ [i]. Update the reference angle θ _REF or estimate the camera tilt angle θ at a time later than the time t _i from the angles θ [i−m] to θ [i], and set the reference angle θ _REF as the estimated angle. Update.

In step S24, basically, an angle other than 0 ° is set as the reference angle θ _REF , so when step S28 is reached via steps S24 and S25, step S30 is passed through step S15 in step S14 in the next frame. To. In step S <b> 30, the control unit 53 causes the tilt correction unit 52 to perform θ _REF rotation processing. The rotation process of θ _REF is also called second inclination correction. In the tilt correction executed when the correction flag is on, the tilt change of the camera 1 in the roll direction is sequentially corrected between frames based on the current camera tilt angle θ [i]. In the second tilt correction), the tilt angle of the output image IB due to the tilt of the camera 1 in the roll direction is uniformly corrected by the reference angle θ _REF without depending on the current camera tilt angle θ [i].

The second tilt correction is also realized by IS rotation processing or image rotation processing. When the second tilt correction is realized by the IS rotation process, the image sensor 33 is rotated from the IS rotation zero state by the correction amount θ _REF so that “φ = θ _REF ”. When the second tilt correction is realized by the IS rotation process, the input image IA is generated as it is as the output image IB. When the second tilt correction is realized by the image rotation process, the output image generation unit 50 sets the cutout frame CF to the input image IA obtained with the IS rotation angle φ set to 0 °, and the image in the cutout frame CF The output image IB is generated by performing geometric transformation that rotates the image by the correction amount θ _REF . At this time, the inclination correction unit 52 controls the setting of the cutout frame CF and the cutout operation in the output image generation unit 50 based on the angle θ _REF . After the second inclination correction in step S30, a transition to step S16 occurs.

  Hereinafter, the operation of FIG. 15 will be applied to several cases.

[First case]
A first case corresponding to FIG. 16 will be described. As shown in FIG. 16, after the shooting mode is started, the photographer tries to shoot the subject obliquely, and the camera 1 is tilted in the roll direction so that θ = 10 °, and the tilt state is stable. After the angle θ _REF is set to 10 °, the correction flag is turned on (steps S17 and S18), and thereafter, inclination correction (step S20) is performed with θ _REF = 10 ° as a target. That is, after that, even if the camera tilt angle θ deviates from 10 °, an image as if the image was taken while maintaining θ = 10 ° (θ _REF ) is continuously obtained as the output image IB.

When the tilt state in the roll direction is stable at θ = 10 °, it is assumed that the photographer has an intention to shoot the subject with a composition having the tilt. In such a case, It is considered that it is more in line with the photographer's intention to perform the tilt correction with θ _REF = 10 ° as a target than the correction (conventional tilt correction) in which the horizontal reference line HL always matches the horizontal direction of the image.

[Second case]
A second case corresponding to FIG. 17 will be described. In the second case, as shown in FIG. 17, as a result of the photographer maintaining the angle θ at a desired angle (for example, 0 °) during the first period after activation of the photographing mode, the desired angle is set to the reference angle θ _REF . The tilt correction is started (steps S17, S18 and S20). Thereafter, in the second period, it is assumed that the camera 1 is greatly tilted to the right (clockwise as viewed from the photographer) and “| θ [i] |> θ _TH ” is established. Then, in step S25, the correction flag is turned off and the execution of the inclination correction is stopped.

Since a relatively small camera tilt angle θ is likely to be an unintended tilt by the photographer, it is beneficial to cancel it by tilt correction. However, it can be inferred that a state in which a relatively large camera tilt angle θ exceeding θ _TH is detected corresponds to a state in which the photographer tilts the camera 1 in the roll direction with a clear intention. Therefore, when such a state is observed, it is considered that stopping the execution of the inclination correction is in line with the photographer's intention.

When the control unit 53 determines that the tilt state of the camera 1 is stabilized again based on the plurality of camera tilt angles θ during the third period after the tilt correction execution is stopped (Y in step S16; 17), the reference angle θ _REF is reset based on the plurality of camera tilt angles θ during the third period (step S17), and the tilt correction in step S20 is resumed.

[Third case]
A third case corresponding to FIG. 18 will be described. In the third case, as shown in FIG. 18, as a result of the photographer maintaining the angle θ at a desired angle (for example, 0 °) during the first period after the shooting mode is started, the desired angle is set to the reference angle θ _REF . The tilt correction is started (steps S17, S18 and S20). Thereafter, it is assumed that the transition from step S22 to step S24 in FIG. 15 occurs as a result of the camera 1 being continuously tilted to the right (clockwise as viewed from the photographer) in the second period. Then, the control unit 53 updates the reference angle θ _REF with an angle (typically the latest camera tilt angle θ) corresponding to all or part of the plurality of camera tilt angles θ obtained during the second period. At the same time, by turning off the correction flag (steps S24 and S25), the execution of the inclination correction in step S20 is stopped.

In this case, in the subsequent third period, the control unit 53 causes the tilt correction unit 52 to execute the second tilt correction in step S30. If the control unit 53 determines that the tilt state of the camera 1 is stabilized again based on the plurality of camera tilt angles θ during the third period (Y in step S16; the state of re-stable is not shown in FIG. 18), the third The reference angle θ _REF is reset based on a plurality of camera tilt angles θ during the period (step S17), and the tilt correction in step S20 is resumed instead of the second tilt correction in step S30.

  Since it is highly possible that the inclination in the roll direction that fluctuates from right to left is an inclination that the photographer does not intend, it is beneficial to cancel the inclination by inclination correction. However, it can be presumed that the state in which the continuous tilt change in the same direction is detected corresponds to the state in which the photographer rotates the camera 1 in the roll direction with a clear intention. Therefore, when such a state is observed, it is considered that stopping the execution of the inclination correction is in line with the photographer's intention.

However, if tilt correction is simply stopped in such a case, the horizontal reference line HL that faces the horizontal direction of the display screen 13 while following the tilt change of the camera 1 is displayed on the display screen 13 simultaneously with the stop of tilt correction. An action that rotates rapidly occurs, and the photographer may feel uncomfortable. In order to avoid this, after the inclination correction is stopped in the third case, the execution of the second inclination correction is started instead, and the rotation correction for the reference angle θ _REF is performed. However, the operation of FIG. 15 can be modified so that the reference angle θ _REF is always set to 0 ° in step S24 (in this case, the second inclination correction is not executed).

[Applied technology]
Several application techniques regarding the operation of the camera 1 will be described.

After setting the reference angle θ _REF in step S17 or S24, the control unit 53 monitors the change rate of the camera tilt angle θ, and when the change rate of the camera tilt angle θ exceeds a predetermined speed (that is, per unit time). The reference angle θ _REF once set may be returned to the initial angle 0 ° (when the change amount of θ exceeds a predetermined amount). When the camera 1 is rotated in the roll direction at high speed, there is a high possibility that the photographing composition desired by the photographer has been changed. In this case, the reference angle θ _REF corresponding to the composition before the change is reset and a new one is set. The reference angle θ _REF corresponding to the composition should be reset (in line with the photographer's intention).

The control unit 53 may further include a motion detection unit (not shown) that detects the motion of the camera 1 other than the motion in the roll direction. The movement of the camera 1 other than the movement in the roll direction (hereinafter referred to as the target movement) includes, for example, a movement in the yaw direction of the camera 1 (a movement that rotates the optical axis 300 in the horizontal direction with the vertical line as the rotation axis), and the camera 1 Movement in the pitch direction (movement that rotates the optical axis 300 in the vertical direction with the horizontal line as the rotation axis) and translational movement that translates the camera 1 in an arbitrary direction (for example, a parallel direction or an orthogonal direction of the optical axis 300). included. The motion detection unit can detect the direction and magnitude of the target motion by a known method based on the output signal of the gyro sensor or the image sensor 33. After setting the reference angle θ _REF in step S17 or S24, the control unit 53 sets the reference angle θ _REF once set to the initial angle 0 even when a target motion having a magnitude exceeding a predetermined amount is detected. You may make it return to °. This is also because there is a high possibility that the photographing composition desired by the photographer has been changed.

  A display control unit (not shown) included in the main control unit 12 corrects whether the current camera tilt angle θ and whether the tilt correction in step S20 or the second tilt correction in step S30 is currently being executed. Execution presence / absence information may be displayed on the display screen 13 together with the output image IB.

  A recording control unit (not shown) included in the main control unit 12 can record a plurality of output images IB sequentially obtained in the operation of FIG. 15 on the recording medium 14 as moving images. The camera tilt angle θ at each time during the shooting period and the correction execution information at each time during the shooting period of the moving image may be recorded on the recording medium 14 in association with the moving image ( For example, the camera tilt angle θ and correction execution presence / absence information may be written in the header area of the image file storing the image data of the moving image).

  The detection of the camera tilt angle θ in step S12 and the tilt correction in step S20 in FIG. 15 may be performed after the exposure of each frame is completed, or when the tilt correction is realized by IS rotation processing, each frame is detected. It may be performed a plurality of times during the exposure period.

According to this embodiment, without requiring a special manual operation by the photographer, whether or not to perform tilt correction is controlled in accordance with the photographer's intention, or a reference angle θ _REF that can be called a correction target is set. It becomes possible to do.

<< Deformation, etc. >>
The embodiment of the present invention can be appropriately modified in various ways within the scope of the technical idea shown in the claims. The above embodiment is merely an example of the embodiment of the present invention, and the meaning of the term of the present invention or each constituent element is not limited to that described in the above embodiment. The specific numerical values shown in the above description are merely examples, and as a matter of course, they can be changed to various numerical values. As annotations applicable to the above-described embodiment, annotation 1 and annotation 2 are described below. The contents described in each comment can be arbitrarily combined as long as there is no contradiction.

[Note 1]
The camera 1 may be mounted on an arbitrary electronic device. The camera 1 is also a kind of electronic device. The electronic device is an arbitrary information device that can acquire, reproduce, or process arbitrary information. For example, a digital camera, a mobile phone, an information terminal, a personal computer, an electronic book reader, an electronic dictionary, a game device, It is a navigation device.

[Note 2]
The camera 1 can be configured by hardware or a combination of hardware and software. Arbitrary specific functions that are all or part of the functions realized by the camera 1 are described as a program, the program is stored in a flash memory that can be mounted on the camera 1, and the program is executed by a program execution device. The specific function may be realized by executing on the microcomputer (for example, a microcomputer that can be mounted on the camera 1). The program can be stored and fixed in an arbitrary recording medium (not shown). A recording medium (not shown) for storing and fixing the program may be mounted or connected to a device (such as a server device) different from the camera 1.

DESCRIPTION OF SYMBOLS 1 Camera 11 Imaging part 13 Display screen 33 Image sensor 51 Inclination detection part 52 Inclination correction part 53 Inclination correction control part 300 Optical axis IA Input image IB Output image HL Horizontal reference line θ Camera inclination angle θ _REF reference angle

Claims (8)

In a camera that includes an imaging unit having an optical system and an image sensor that outputs a signal of an optical image formed through the optical system, and acquires an output image from an output signal of the image sensor,
An inclination detection unit for detecting a camera inclination angle around the optical axis of the optical system;
An inclination correction unit capable of executing inclination correction for correcting the angle of inclination of the output image due to the inclination of the camera to a reference angle based on the camera inclination angle;
A tilt correction control unit configured to control whether to perform tilt correction or to set the reference angle based on the size of the camera tilt angle or a plurality of camera tilt angles at a plurality of times. Camera. The tilt correction control unit stops the tilt correction execution when the camera tilt angle having a magnitude larger than a predetermined threshold is detected during the tilt correction. Camera. The tilt correction control unit sets the reference angle based on a plurality of camera tilt angles during a first period and starts executing the tilt correction, and then has a magnitude greater than the predetermined threshold in a second period. The camera according to claim 2, wherein execution of the tilt correction is stopped when a camera tilt angle is detected. The tilt correction control unit stops execution of the tilt correction when a plurality of camera tilt angles continuously changing in the same direction on a time series are detected during the tilt correction. The camera according to claim 1. The tilt correction control unit sets the reference angle based on a plurality of camera tilt angles during the first period and starts executing the tilt correction, and then continuously in the same direction in time series in the second period. The camera according to claim 4, wherein execution of the tilt correction is stopped when a plurality of changing camera tilt angles are detected. The tilt correction control unit determines whether the tilt state of the camera around the optical axis is stable based on a plurality of camera tilt angles obtained before the tilt correction is performed, and the tilt state is stable. 6. The tilt correction is started by setting the reference angle based on a plurality of camera tilt angles obtained before execution of the tilt correction. camera. In the inclination correction, the inclination correction unit,
Based on the camera tilt angle, the image sensor is rotated around the optical axis to generate the output image from the output signal of the image sensor, or based on the camera tilt angle, input by the output signal of the image sensor The camera according to claim 1, wherein the output image is generated by performing image processing including image rotation processing on the image. The tilt correction control unit returns the reference angle once set to a predetermined initial angle according to a change speed of the camera tilt angle or a movement of the camera other than the tilt. The camera according to claim 7.

Images