JP2010183353A - Photographing device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a photographing device which is suitable for photographing, in relation to the photographing device, including a shake correction function for correcting blurring occurring in photographing. <P>SOLUTION: The photographing device includes a detection part (28) for detecting blurring of the device to output a detection signal; a first image acquisition part (52) acquiring a first image (P<SB>1</SB>) and a second image (P<SB>2</SB>), acquired at a time which is different from the time the first image is acquired; a second image acquisition part (51) acquiring a third image which is different from the first and second images; a motion vector acquiring part (84) identifying a second region (TM1'') which corresponds to a first region (TM1) included in the first image from the second image by using the detection signal, and acquiring a motion vector using information of the first region and that of the second region; and a correction part (25) for correcting the blurring of the device by using the motion vector. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a photographing apparatus having a blur correction function for correcting a blur that occurs during photographing.

  2. Description of the Related Art There are known shake correction devices that correct optical axis shake in a shooting optical system, and shooting devices that include such a shake correction device. Some of these apparatuses correct blur by physically moving either a part of the photographing optical system or an image sensor (or film). In addition to this, there has also been proposed an apparatus that divides the exposure time and repeats imaging, and synthesizes a plurality of images obtained by the imaging based on a motion vector obtained from the optical axis blur of the imaging optical system. ing.

  An object of this invention is to provide the imaging device which can perform suitable imaging | photography.

  The present invention solves the above problems by the following means. For ease of understanding, reference numerals corresponding to the drawings showing an embodiment of the present invention are given and described, but the present invention is not limited to this.

The invention according to claim 1 is a detection unit (28) for detecting a shake of the apparatus and outputting a detection signal;
A first image acquisition unit (52) for acquiring a first image (P ₁ ) and a second image (P ₂ ) acquired at a time different from the time at which the first image (P ₁ ) was acquired; ,
A second image acquisition unit (51) for acquiring a third image different from the first image (P ₁ ) and the second image (P ₂ );
A second region (TM1 ″) corresponding to a first region (TM1) included in the first image (P ₁ ) is identified from the second image (P ₂ ) using the detection signal, and the second region (TM1 ″) is identified. A motion vector acquisition unit (84) for acquiring a motion vector using information on one area (TM1) and information on the second area (TM1 '');
An imaging apparatus comprising: a correction unit (25) that corrects a shake of the apparatus using the motion vector.

Invention of Claim 2 is the imaging device described in Claim 1,
The correction unit (25) includes at least one of the optical system (21) that forms an image acquired by the second image acquisition unit (51) and at least one of the second image acquisition unit (51). An imaging apparatus that corrects a shake of the apparatus by driving.

The invention described in claim 3 is the photographing apparatus according to claim 1 or 2, wherein
A filter unit for reducing a panning component included in the detection signal;
An operation unit (74) operable by the photographer,
When the operation unit (74) is operated by the photographer, the motion vector acquisition unit (84) uses the signal output from the filter unit to generate the second region from the second image (P ₂ ). (TM1 ″) is specified.

The invention according to claim 4 is the photographing apparatus according to any one of claims 1 to 3,
The motion vector acquisition unit (84) sets a setting area (TM1 ′) corresponding to the first area (TM1) in the second image (P ₂ ) based on the detection signal, and the setting area (TM1 When the correlation between ') and the first area (TM1) is high, the setting area (TM1') is the second area (TM1 '').

The invention described in claim 5 is the photographing apparatus described in claim 4,
When the correlation between the setting area (TM1 ′) and the first area (TM1) is low, the motion vector acquisition unit (84) and the first area (TM1) ) And the correlation between the area near the setting area (TM1 ′) and the first area (TM1) is based on the correlation between the setting area (TM1 ′) and the first area (TM1). Is higher, the area near the set area (TM1 ′) is the second area (TM1 ″).

A sixth aspect of the present invention is the imaging apparatus according to any one of the first to fifth aspects,
A main subject detection unit for detecting a main subject included in the third image;
The first device (TM1) is an imaging device characterized in that at least a part of the main subject is included.

The invention according to claim 7 is the imaging device according to any one of claims 1 to 5,
A focusing area detecting unit (35) for detecting a focusing area of the third image;
The first area (TM1) is an imaging apparatus including an area corresponding to the in-focus area detected by the in-focus area detection unit (35).

The invention according to claim 8 is the imaging apparatus according to any one of claims 1 to 7,
The first image (P ₁ ) and the second image (P ₂ ) captured by the first image acquisition unit (52) are more than the third image captured by the second image acquisition unit (51). Is a photographing apparatus characterized by a deep depth of field.

The invention according to claim 9 is the photographing apparatus according to any one of claims 1 to 8,
The first image (P ₁ ) and the second image (P ₂ ) captured by the first image acquisition unit (52) are more than the third image captured by the second image acquisition unit (51). Is a wide-angle image capturing device.

A tenth aspect of the present invention is the photographing apparatus according to any one of the first to ninth aspects,
The first image acquisition unit (52) includes a first optical system (41) for forming the first image (P ₁ ) and the second image (P ₂ ),
The second image acquisition unit (51) includes a second optical system (21) for forming the third image having a narrower angle than the first optical system (41). is there.

The invention according to claim 11 is the photographing apparatus according to any one of claims 1 to 10,
The first image acquisition unit (52), after acquiring the second image _{(P 2),} the first image _(P 1), different from the first and the second image _{(P 2)} and the third image Acquire 4 images
The motion vector obtaining unit (84), a fourth region corresponding to the second region of the second image (P ₂₎ (TM1 ''), identified from the fourth image using the detection signal, The imaging apparatus is characterized in that a motion vector is acquired using information on the second area (TM1 ″) and information on the fourth area.

  Note that the configuration described with reference numerals may be improved as appropriate, or at least a part thereof may be replaced with another component.

  ADVANTAGE OF THE INVENTION According to this invention, the imaging device which can perform suitable imaging | photography can be provided.

It is a figure which shows the structure of an electronic camera. It is a figure which shows the circuit structure of an electronic camera. It is a figure when the movement vector calculated | required by an angular velocity sensor is decomposed | disassembled into the camera shake component and the panning component. It is a figure which shows the imaging | photography range and the position of a to-be-photographed object in the 1st image and 2nd image obtained when the 1st deviation correction mode is selected. It is a figure which shows the position of a to-be-photographed object when it changes from time T1 to time T2 when the imaging | photography range is fixed. It is a figure which shows the imaging | photography range and the position of a to-be-photographed object in a 1st image and a 2nd image obtained when 2nd deviation correction mode is selected. It is a figure which shows the position of a to-be-photographed object when it changes from time T3 to time T4 when the imaging | photography range is fixed. It is a flowchart which shows the flow of the process at the time of imaging.

  FIG. 1 shows an outline of an electronic camera as an example of a photographing apparatus according to the present embodiment. The electronic camera 10 includes a camera body 11 and a lens unit 12 that houses a photographing optical system. The camera body 11 and the lens unit 12 are each provided with a pair of mounts 13 and 14 that form a male-female relationship. The lens unit 12 is attached to the camera body 11 in a replaceable manner by coupling the lens side mount 14 to the mount 13 on the camera body 11 side by a bayonet mechanism or the like. The mounts 13 and 14 are each provided with an electrical contact (not shown). When the camera body 11 and the lens unit 12 are connected, electrical connection between the two is established by contact between the electrical contacts.

  First, the configuration of the lens unit 12 will be described. The lens unit 12 includes an imaging optical system 21, a zoom encoder 22, a lens driving unit 23, a distance encoder 24, a blur correction unit 25, a diaphragm 26, a diaphragm driving unit 27, an angular velocity sensor 28, and a lens microcomputer. 29. Note that the zoom encoder 22, the lens driving unit 23, the distance encoder 24, the blur correction unit 25, and the aperture driving unit 27 are each connected to a lens microcomputer 29. Hereinafter, the imaging optical system 21 provided in the lens unit 12 will be described as the first imaging optical system 21.

  The first imaging optical system 21 has a plurality of lenses such as a zoom lens 21a and a focus lens 21b. The zoom lens 21a is a lens for adjusting the focal length, and is configured to be movable back and forth in the optical axis direction in accordance with an operation of a zoom ring (not shown). A zoom encoder 22 for detecting the position of the lens in the optical axis direction is attached to the zoom lens 21a. The focus lens 21b is a lens for adjusting the in-focus position, and is configured to be movable back and forth in the optical axis direction. The lens driving unit 23 drives the focus lens 21b by a motor (not shown). A distance encoder 24 for detecting the position of the focus lens 21b moving in the optical axis direction is attached to the focus lens 21b. The detection signal from the zoom encoder 22 and the detection signal from the distance encoder 24 are output to the lens microcomputer 29. From these detection signals, the shooting distance and focal length at the time of shooting are obtained. Note that the photographing distance and the focal distance are obtained by the lens microcomputer 29 or a CPU 58 described later, and are used at the time of parallax correction between the first imaging optical system 21 and the second imaging optical system 41.

  The blur correction unit 25 is an actuator such as a VCM (voice coil motor), for example, and at least a part of the lenses of the first imaging optical system 21 is moved in the optical axis direction based on a blur amount obtained by a correlation calculation unit 86 described later. Is moved in a direction orthogonal to the direction of the image to correct blurring that occurs during shooting.

  The diaphragm 26 adjusts the amount of light incident on the camera body 11 by opening and closing the diaphragm blades. The diaphragm drive unit 27 controls the opening degree of the diaphragm 26 by a motor (not shown).

  The angular velocity sensor 28 detects the angular velocity of the lens unit 12 when the posture of the lens unit 12 changes. For example, when so-called panning is performed in which the electronic camera 10 is moved following a moving subject, or when the camera body 11 is shaken when the user grips the electronic camera 10, Since the posture changes, in such a case, the angular velocity sensor 28 detects the angular velocity. That is, the angular velocity sensor 28 detects the angular velocities of the lens unit 12 and the camera body 11, in other words, the angular velocity of the first imaging optical system 21. This angular velocity is equal to the angular velocity of the second imaging optical system 41.

  The lens microcomputer 29 communicates with the camera body 11 through the electrical contacts of the mount 14 and executes various controls in the lens unit 12. The lens microcomputer 29 transmits lens data recorded in a ROM (not shown) to the camera body 11.

  Incidentally, the lens unit 12 shown in FIG. 1 is merely an example of a configuration of a general zoom lens unit. Therefore, in addition to the lens unit 12 described above, for example, a lens unit that does not include the lens microcomputer 29 or a lens unit of a single focus lens can be attached to the camera body 11.

  Further, an example of a so-called single-lens reflex type electronic camera in which the lens unit 12 is detachably attached to the camera body 11 is taken up. However, the present invention is not limited to this, and the lens unit 12 is fixed to the camera body 11. It may be a compact type electronic camera.

  Next, the configuration of the photographing mechanism of the camera body 11 will be described. The camera body 11 includes a quick return mirror 31, a mechanical shutter 32, a first image sensor 33, a sub mirror 34, a focus detection unit 35, a finder optical system (36 to 39), a second image pickup optical system 41, and A second image sensor 42 is provided.

  The quick return mirror 31, the mechanical shutter 32, and the first image sensor 33 are disposed along the optical axis of the first image pickup optical system 21. A sub mirror 34 is disposed behind the quick return mirror 31. Further, a finder optical system, a second imaging optical system 41, and a second imaging element 42 are disposed on the upper part of the camera body 11. Further, a focus detection unit 35 is disposed in the lower region of the camera body 11.

  The quick return mirror 31 is pivotally supported by a rotation shaft (not shown) and can be switched between an observation state and a retracted state. The quick return mirror 31 in the observation state is inclined and disposed in front of the mechanical shutter 32 and the first image sensor 33. The quick return mirror 31 in this observation state reflects the light beam that has passed through the first imaging optical system 21 upward and guides it to the finder optical system. Further, the central portion of the quick return mirror 31 is a half mirror. A part of the light beam that has passed through the quick return mirror 31 is refracted downward by the sub mirror 34 and guided to the focus detection unit 35. The focus detection unit 35 detects the amount of image shift of the subject image divided by a separator lens (not shown) for each AF area, and performs so-called phase difference detection type focus detection.

  On the other hand, the quick return mirror 31 in the retracted state is flipped upward together with the sub mirror 34 and rotated to a position outside the photographing optical path. When the quick return mirror 31 is in the retracted state, the light beam that has passed through the first imaging optical system 21 is guided to the mechanical shutter 32 and the first imaging element 33.

  The viewfinder optical system includes a diffusion screen (focus plate) 36, a condenser lens 37, a pentaprism 38, and an eyepiece lens 39. The diffusing screen 36 is positioned above the quick return mirror 31, and the light beam reflected by the quick return mirror 31 in the observation state once forms an image. The light beam formed on the diffusion screen 36 passes through the condenser lens 37 and the pentaprism 38 and is guided to the exit surface having an angle of 90 ° with respect to the incident surface of the pentaprism 38. Then, the light beam from the exit surface of the pentaprism 38 reaches the eyes of the user through the eyepiece lens 39.

  The second imaging optical system 41 and the second imaging element 42 are provided, for example, on the camera body 11. The second imaging optical system 41 is arranged so that a straight line parallel to the optical axis of the first imaging optical system 21 is an optical axis. The second image pickup optical system 41 and the second image pickup element 42 are used when acquiring an image (hereinafter referred to as a calculation image) for calculating a shake amount described later. For this reason, the second imaging optical system 41 includes an optical system that includes the angle of view of the first imaging optical system 21 and has a smaller image circle than the first imaging optical system 21. The second image sensor 42 is controlled so as to acquire a calculation image at predetermined intervals, for example, 200 images per second. As the second image sensor 42, for example, a pixel having a smaller number of pixels than the first image sensor 33, such as 10,000 pixels, is used.

  Next, the circuit configuration of the electronic camera 10 will be described with reference to FIG. The camera body 11 includes a first image acquisition unit 51, a second image acquisition unit 52, a memory 53, a recording I / F 54, a display I / F 55, a main monitor 56, an operation unit 57, a CPU 58, and a system bus 59. . Here, the first image acquisition unit 51, the memory 53, the recording I / F 54, the display I / F 55, and the CPU 58 are connected via a system bus 59. The CPU 58 is also connected to the electrical contacts of the mount 13 (not shown).

  The first image acquisition unit 51 includes a first image sensor 33, a first analog processing unit 61, and a first digital processing unit 62. The first image sensor 33 is a sensor for generating a captured image that is a recording image. The first image sensor 33 photoelectrically converts the light beam that has passed through the first imaging optical system 21 at the time of release, and outputs an analog image signal of a captured image. The output signal of the first image sensor 33 is input to the first analog processing unit 61. Note that the first image sensor 33 can also output a through image by thinning-out reading at a predetermined interval during shooting standby (non-release). For this reason, when the quick return mirror 31 is in the retracted position, the CPU 58 can determine the photographing condition based on the through image of the first image sensor 33.

  The first analog processing unit 61 is an analog front end circuit having a CDS circuit, a gain circuit, an A / D conversion circuit, and the like. The CDS circuit reduces the noise component of the output of the first image sensor 33 by correlated double sampling. The gain circuit amplifies the gain of the input signal and outputs it. In this gain circuit, the imaging sensitivity corresponding to the ISO sensitivity can be adjusted. The A / D conversion circuit performs A / D conversion of the output signal of the first image sensor 33. In FIG. 2, illustration of each circuit of the first analog processing unit 61 is omitted.

  The first digital processing unit 62 performs various types of image processing (defective pixel correction, color interpolation, gradation conversion processing, white balance adjustment, edge enhancement, etc.) on the output signal of the first analog processing unit 61 and performs shooting. Generate image data. The first digital processing unit 62 also executes compression / decompression processing of captured image data. The first digital processing unit 62 is connected to the system bus 59.

  The second image acquisition unit 52 includes a second image sensor 42, a second analog processing unit 63, and a second digital processing unit 64. In addition, since the structure of the 2nd image acquisition part 52 respond | corresponds substantially to the structure of the 1st image acquisition part 51, description is partially abbreviate | omitted about both overlapping part. The second image sensor 42 photoelectrically converts the subject image formed through the second imaging optical system 41 at predetermined intervals, thereby using an image (hereinafter referred to as a calculation image) used for calculating a blur amount described later. ) To get. The output signal of the second image sensor 42 is input to the second analog processing unit 63. As described above, the second imaging optical system 41 includes the angle of view of the first imaging optical system 21 and is composed of an optical system having an image circle smaller than that of the first imaging optical system 21. The image (calculation image) acquired by the two-image acquisition unit 52 has a deeper subject depth than the captured image acquired by the first image acquisition unit 51, and the angle of view is acquired by the first image acquisition unit 51. A wider angle than the captured image.

  The second analog processing unit 63 is an analog front end circuit having a CDS circuit, a gain circuit, an A / D conversion circuit, and the like. The second digital processing unit 64 performs color interpolation processing of the calculation image. The calculation image data output from the second digital processing unit 64 is input to the CPU 58.

  The memory 53 is a buffer memory for temporarily recording captured image data in the pre-process and post-process of image processing by the first digital processing unit 62. A connector for connecting a recording medium 66 is formed in the recording I / F 54. The recording I / F 54 executes writing / reading of captured image data with respect to the recording medium 66 connected to the connector. The recording medium 66 is composed of a hard disk, a memory card incorporating a semiconductor memory, or the like.

  The display I / F 55 controls display on the main monitor 56 based on an instruction from the CPU 58. The main monitor 56 is disposed on the back surface of the camera body 11, for example. The main monitor 56 displays various images according to instructions from the CPU 58 and the display I / F 55. For example, the main monitor 56 can display a reproduced image of a captured image, a menu screen that allows input in a GUI (Graphical User Interface) format, and the like (illustration of each image is omitted).

  The operation unit 57 includes a plurality of switches that accept user operations, such as a release button 71, a mode dial 72, a zoom switch 73, and a correction mode changeover switch 74. The release button 71 receives from the user an instruction input for starting an AF operation before photographing and an instruction input for starting an exposure operation during photographing. The mode dial 72 receives a shooting mode switching input from the user by the dial operation. The zoom switch 73 receives an operation for optically or electronically enlarging / reducing the magnification of the through image acquired by the first image sensor 33 from the user.

  The correction mode changeover switch 74 receives an operation for selecting a mode for performing blur correction (hereinafter referred to as a blur correction mode) from the user. At the time of shooting, the camera body 11 is shaken (hereinafter referred to as camera shake component), the camera body 11 is panned (hereinafter referred to as panning component), and the subject itself is shaken (hereinafter referred to as subject blur component). The image is a blurred image. In the blur correction mode, a plurality of blur correction modes are provided so that a blur component intended by the user can be corrected in advance among these blur components. These blur correction modes will be described as a first blur correction mode and a second blur correction mode. The first blur correction mode is a mode for correcting all of the camera shake component, the panning component, and the subject blur component. On the other hand, the second shake correction mode is a mode for correcting a camera shake component and a subject shake component.

  Hereinafter, the function of the correction mode changeover switch 74 will be described by taking as an example a case where a still image is acquired as a captured image.

As shown in FIG. 3, the movement vector A _ΔT on the image plane of the first imaging optical system 21 at a predetermined time can be obtained from the detection signal based on the angular velocity detected by the angular velocity sensor 28. Filtering is performed on the detection signal to separate it into a high frequency component and a low frequency component. Among these, the low frequency component is the panning component B _ΔT and the high frequency component is the camera shake component C _ΔT . That is, A _ΔT = B _ΔT + C _ΔT (Formula 1).

When the first blur correction mode is selected by the correction mode changeover switch 74, the blur correction is performed on the panning component B _ΔT , the camera shake component C _ΔT , and the subject blur component D _ΔT (see FIG. 4). On the other hand, when the second blur correction mode is selected by the correction mode changeover switch 74, the camera shake component C _ΔT and the subject blur component D _ΔT are corrected without correcting the panning component B _ΔT .

  The CPU 58 performs overall control of each part of the electronic camera 10. Further, the CPU 58 functions as a sequence control unit 81, a shooting setting unit 82, a parallax correction unit 83, and a motion vector acquisition unit 84 by a program stored in a ROM (not shown). The sequence control unit 81 executes control of operations of the lens unit 12, the quick return mirror 31, the mechanical shutter 32, the first image acquisition unit 51, the second image acquisition unit 52, and the like.

  The shooting setting unit 82 performs auto focus (AF), automatic exposure (AE) calculation, auto white balance (AWB) calculation, and the like, and various parameters (exposure time, exposure time, Aperture value, imaging sensitivity, etc.) are determined. The shooting setting unit 82 also bears processing related to AE and AWB of the second image acquisition unit 52. When the second imaging optical system 41 has a focus lens, the shooting setting unit 82 performs AF in the second imaging optical system 41.

  The parallax correction unit 83 corrects the parallax generated between the image (captured image, through image) acquired by the first image acquisition unit 51 and the image (calculation image) acquired by the second image acquisition unit 52. To do. The first image acquisition unit 51 and the second image acquisition unit 52 have different shooting conditions such as focal length and shooting magnification in the respective optical systems. For this reason, parallax occurs between the captured image acquired by the first image acquisition unit 51 and the calculation image acquired by the second image acquisition unit 52. The parallax correction unit 83 calculates the relationship between the shooting ranges of the image acquisition units 51 and 52, the ratio of the image acquired by the image acquisition units 51 and 52, and the like.

The motion vector acquisition unit 84 includes two adjacent calculation images (the calculation image P ₁ and the calculation image P ₂ , among the calculation images acquired at predetermined intervals by the second image acquisition unit 52. See FIG. 4. )) To obtain the motion vector generated in these calculation images. The motion vector acquisition unit 84 includes an estimation unit 85 and a correlation calculation unit 86.

  Next, the flow of processing when a captured image is acquired by the electronic camera 10 will be described based on the flowchart of FIG. Note that the flowchart of FIG. 8 is executed in response to the transition to the shooting standby state.

  In step S110, the CPU 58 determines whether or not the release button 71 is half-pressed. If the CPU 58 determines that the release button 71 is half-pressed, the CPU 58 proceeds to step S111. When the CPU 58 determines that the release button 71 is not half-pressed, the CPU 58 performs step S110 again after a predetermined time has elapsed.

Step S <b> 111 is a process of acquiring a calculation image P ₀ (not shown). The CPU 58 controls the second image processing unit 52 to acquire the calculation image P ₀ . In step S112, CPU 58 is a main object of the operation image _{P 0} (hereinafter, sometimes referred to as the subject) to determine the subject region TM0 (not shown) is present. The region TM0 is a region including a main subject such as a human face, for example. As a method for determining a subject region, TM0, area is in focus in the calculation image P ₀ obtained immediately before the full depression of the release button, and a method for the Tokoroigoase region, by the feature extraction face detection, etc. For example, a method of determining using the subject recognition technique, and a method of directly determining by the user using a touch panel or the like.

  Next, in step S113, the CPU 58 determines whether or not the release button 71 is fully pressed. If it is determined that the release button 71 is fully pressed, the process proceeds to step S114. If it is determined that it has not been fully pressed, the process returns to step S111. In step S114, the CPU 58 causes the first image processing unit 51 to start accumulating charges, and the process proceeds to step S121.

In step S121, the CPU 58 controls the second image processing unit 52 to acquire the calculation image P ₁ (see FIGS. 4 and 5) at time T1, and then proceeds to step S122. In step S122, CPU 58 obtains an image of the operation image _{P 1} of the template region TM1 (see FIGS. 4 and 5). Specifically, CPU 58, from the operation image _{P 1} at time T1, extracts a region including a main subject included in the subject region TM0, the template region TM1 of calculating the image _{P 1} (FIG. 4, FIG. 5 reference).

Template region TM1 is part of the calculation for the image _{P 1,} a portion corresponding to the region TM0 of calculating the image _{P 0.} For example, in the template area TM1, the same main subject as the main subject included in the area TM0 exists.

Next, in step S131 (time T2), CPU 58 obtains a temporally continuous operation image _{P 2} in the calculation image _{P 1.} Note that the CPU 58 receives an image signal that is the basis of the calculation image from the second image processing unit 52 every predetermined time (for example, every 10 ⁻⁵ to 10 ⁻¹ second), and the calculation image P _{1. The} calculation image P ₂ and the calculation image P _M (M is an arbitrary natural number) can be acquired sequentially.

  Hereinafter, with reference to FIGS. 4 to 7, steps S132 to S134 (steps of acquiring a motion vector, calculating a shake correction amount, and performing shake correction) will be described in detail.

4 and 5 illustrate a case where the first blur correction mode (a mode for correcting all of the camera shake component, panning component, and subject blur component) is selected by the correction mode changeover switch 74 (see FIG. 2). FIG. FIG. 4 is a diagram showing the positional relationship between two consecutive calculation images (calculation image P ₁ and calculation image P ₂ ), and FIG. 5 represents the state shown in FIG. 4 with a shooting range fixed. It is a thing.

In FIG. 4 and FIG. 5, the subject areas (subjects I, I ′, I ″) are indicated by “Δ”. As described above, calculating the image _{P 1} is acquired at time T1 (step S121), the template region TM1 is obtained in step S122, calculating the image _{P 2} has been acquired at time T2 (step S131).

In step S132, CPU 58, from the difference between the high from the operation image _{P 2} most correlated with the template region TM1 region 'extracts, template regions TM1 and the template region TM1 (template region TM1') '' (shift amount) Get motion vector.

4 and 5, the change in the attitude of the electronic camera 10 from the time T1 to time T2 (camera shake), shooting range of operation for the image P _2, the second imaging optical against shooting range of operation for the image P ₁ The movement vector A _ΔT on the image plane of the system 41 is shifted. Therefore, CPU 58 is to estimate the position of the object I 'of the operational image P ₂ at time T2, detects the angular velocity using the angular velocity sensor 28 (see FIG. 1), the angular velocity second imaging optical system 41 The movement vector A is converted into _ΔT .

Next, the CPU 58 (correlation calculation unit 86) sets a template area TM1 ′ having the same size as the template area TM1 at a position moved by −A _ΔT from the area TM1 using the movement vector A _ΔT , and the template area Correlation between TM1 and template region TM1 ′ is executed. If the correlation between the template region TM1 and the template region TM1 ′ is high as a result of the correlation calculation, the correlation calculation unit 86 sets the template region TM1 ′ as the template region TM1 ″. As the correlation calculation, for example, there is a template matching calculation that calculates a similarity when two images are shifted for each pixel, for example, and obtains a position (or a shift amount) where the similarity is the highest.

Next, 'when the correlation is low, the correlation calculation unit 86, calculating the image P ₂ of the template region TM1' template region TM1 and the template region TM1 for the vicinity of the template region TM1 longitudinal direction (in the plane of FIG. 5 The correlation calculation is performed while shifting in the vertical direction) and the horizontal direction (left and right direction of the paper surface of FIG. 5). Thereby, the template region TM1 ″ having the highest similarity to the template region TM1 is obtained.

Next, the CPU 58 (motion vector acquisition unit 84) obtains a subject blur component D _ΔT that is a difference (a positional deviation amount) between the template region TM1 ″ and the template region TM1 ′.

If the subject is not stationary between T1 and T2 and subject blurring has occurred, the position of the subject (subject I ″) at time T2 is the time as shown in FIGS. It is at a position shifted by D _ΔT from the subject I ′ at T2. Further, a stationary object during T1 to T2, assuming that there is no subject shake, the position of the subject in the calculation for the image P ₂ (object I '') is the subject I at time T2 'consistent with To do.

Next, the motion vector acquisition unit 84 acquires all blur components E _ΔT between times T1 and T2 as motion vectors (motion vectors on the image plane of the second imaging optical system 41). In the first blur correction mode, the total blur component E _ΔT is E _ΔT = −A _ΔT + D _ΔT = − (B _ΔT + C _ΔT ) + D _ΔT . As described above, the movement vector A _ΔT is the panning component B _ΔT and the camera shake component C _ΔT .

  Next, in step S133, the CPU 58 converts the motion vector calculated in step S132 into a motion vector on the image plane of the first imaging optical system 21, and blurs on the image plane of the first imaging optical system 21. Find the correction amount.

  Next, in step S134, the CPU 58 executes blur correction. Specifically, the CPU 58 corrects the blurring that occurs during shooting by driving the blur correction unit 25 (see FIG. 1) via the lens microcomputer 29 based on the motion vector (blur correction amount). As described above, in the first shake correction mode, the shake correction is performed so that the subject is fixed at a predetermined position in the shooting range.

  Next, referring to FIG. 6 and FIG. 7, steps when the second blur correction mode (the mode for correcting the camera shake component and the subject blur component) is selected by the correction mode changeover switch 74 (see FIG. 2). S132 to S134 (steps of acquiring a motion vector, calculating a shake correction amount, and performing shake correction) will be described in detail.

  FIG. 6 is a diagram showing the positional relationship between two consecutive calculation images, and FIG. 7 is a diagram showing blurring of the subject when the shooting range is fixed. 6 and 7, the same components as those in FIGS. 4 and 5 are denoted by the same reference numerals, and detailed description thereof is omitted.

6 and 7, the subject area (subjects J and J ′) is indicated by “Δ”. Figure 4, similarly to FIG. 5, the acquired operation image _{P 3} at time T3 (step S121), the template region TM1 is obtained in step S122, calculating the image _{P 4} is acquired at time T4 (step S131) ing. In FIGS. 6 and 7, for the sake of simplicity, the panning component B _ΔT from time T3 to time T4 is only in the horizontal direction, and the camera shake component C _ΔT is only in the vertical direction.

In step S132, CPU 58, from the difference between the high from the operation image _{P 4} most correlated with the template region TM1 region 'extracts, template regions TM1 and the template region TM1 (template region TM1') '' (shift amount) Get motion vector.

The second blur correction mode is selected, for example, when the user pans the electronic camera 10 according to the moving subject and acquires a still image of the subject as a captured image. This second blur correction mode is a mode in which the camera shake component C _ΔT and the subject blur component D _ΔT are corrected and the panning component B _ΔT is not corrected. Here, the subject blur component refers to the amount of subject movement on the image plane that still remains when the user pans according to the moving subject.

6 and 7, the shooting range of the calculation image P ₄ is shifted by the movement vector A _ΔT on the image plane of the second imaging optical system 41 with respect to the shooting range of the calculation image P ₃ due to panning by the user. .

CPU58, as a subject of panning target is fixed at a predetermined position of the imaging range, in consideration of panning by the user, the electronic camera 10 is posture change (camera shake) the time object J arithmetic image P ₄ in T4 ' In order to estimate the position, the angular velocity sensor 28 (see FIG. 1) is used to detect the angular velocity, and the panning component B _ΔT is excluded from the angular velocity and converted into the movement vector C _ΔT of the second imaging optical system 41. This is because blur correction is not performed on the panning component B _{ΔT in the} movement vector A _ΔT .

Next, the CPU 58 (correlation calculation unit 86) uses the movement vector C _ΔT to set a template area TM1 ′ having the same size as the template area TM1 at a position moved by −C _ΔT from the area TM1, and the template area Correlation between TM1 and template region TM1 ′ is executed. If the correlation between the template region TM1 and the template region TM1 ′ is high, the correlation calculation unit 86 sets the template region TM1 ′ as the template region TM1 ″.

Next, 'when the correlation is low, the correlation calculation unit 86, calculating the image P ₄ template region TM1' template region TM1 and the template region TM1 for the vicinity of the template region TM1 longitudinal direction (in the plane of FIG. 7 The correlation calculation is executed while shifting in the vertical direction) and the horizontal direction (left and right direction in FIG. 7). Thereby, the template region TM1 ″ having the highest similarity to the template region TM1 is obtained. Next, the motion vector acquisition unit 84 obtains a subject blur component D _ΔT that is a difference (a positional deviation amount) between the template region TM1 ″ and the template region TM1 ′.

If there is a subject blur component (subject movement amount on the image plane remaining after panning) between T3 and T4, as shown in FIGS. 6 and 7, the position of the subject (subject J ′) at time T4. ') Is at a position shifted by D _ΔT from the subject J' at time T4. Further, assuming that there is no subject shake component during T3 to T4, the position of the subject in the operation image P ₄ (subject J '') is subject J at time T4 'coincides with.

Next, the motion vector acquisition unit 84 acquires all blur components E _ΔT between times T3 and T4 as motion vectors (motion vectors on the image plane of the second imaging optical system 41). In the second blur correction mode, the total blur component E _ΔT is E _ΔT = −C _ΔT + D _ΔT .

  Next, in step S133, the CPU 58 converts the motion vector calculated in step S132 into a motion vector on the image plane of the first imaging optical system 21, and blurs on the image plane of the first imaging optical system 21. Find the correction amount.

  Next, in step S134, the CPU 58 executes blur correction. Specifically, the CPU 58 corrects the blurring that occurs during shooting by driving the blur correction unit 25 (see FIG. 1) via the lens microcomputer 29 based on the motion vector (blur correction amount). Thus, in the second shake correction mode, the shake correction is executed so that the subject to be panned is fixed at a predetermined position in the shooting range.

  With reference to the flowchart in FIG. 8 again, the flow of processing when acquiring a captured image will be described.

  After executing the blur correction in step S134, the CPU 58 determines whether or not the exposure time of the first image acquisition unit 51 has elapsed in step S135. When the exposure time of the first image acquisition unit 51 has elapsed, the process proceeds to step S137.

  If the exposure time of the first image acquisition unit 51 has not elapsed, the template area TM1 ″ is set to the area TM1 in step S136. Accordingly, in the same manner as described above, in step S132, the region template region TM1 ″ having the highest correlation with the template region TM1 is extracted again, the motion vector is acquired again, and the motion vector using the motion vector acquired again is acquired. Correction will be performed.

  In step S137, the CPU 58 causes the first image processing unit 51 to finish accumulating charges, and the process proceeds to step S138. In step S138, the CPU 58 stores the captured image acquired in step S137 in the storage medium 66.

  As a result, the movement direction and the movement amount between two consecutive calculation images are estimated using the movement vector obtained from the angular velocity sensor 28, and the correlation calculation is performed from the estimated movement direction and movement amount. The calculation load can be reduced, and the processing time related to the correlation calculation can be shortened. In addition, since the number of pixels of the image acquired by the second image processing unit 52 is smaller than the number of pixels of the image acquired by the first image acquisition unit 51, the time for acquiring the image can be shortened and correlation calculation can be performed. Since the required calculation load can be reduced, the processing time related to the correlation calculation can be executed in a short time.

  In the present embodiment, the blur correction is performed by moving a part of the lens constituting the first imaging optical system 21 in the direction orthogonal to the optical axis using the blur correction unit 25. It is also preferable to move the first image sensor 33 in the direction orthogonal to the optical axis using the same actuator as that of No. 25. It is also preferable to perform blur correction by tilting the first imaging optical system 21 or the first imaging element 33 using an actuator. It is also preferable to electronically correct blurring that occurs in a photographed image photographed using the first image sensor 33.

  In this embodiment, the case where a still image is acquired as a captured image is described. However, the present invention is not limited to this, and the present invention is preferably applied to an image capturing apparatus that captures a moving image.

  DESCRIPTION OF SYMBOLS 10 ... Electronic camera, 11 ... Camera body, 12 ... Lens unit, 21 ... 1st imaging optical system, 25 ... Deviation correction part, 28 ... Angular velocity sensor, 33 ... 1st imaging device, 41 ... 2nd imaging optical system, 42 ... 2nd image sensor, 51 ... 1st image acquisition part, 52 ... 2nd image acquisition part, 58 ... CPU, 74 ... Correction mode selection switch, 84 ... Motion vector acquisition part, 85 ... Estimation part, 86 ... Correlation calculation part

JP 2006-54518 A

Claims (11)

A detection unit that detects a shake of the device and outputs a detection signal;
A first image acquisition unit that acquires a first image and a second image acquired at a time different from the time at which the first image was acquired;
A second image acquisition unit for acquiring a third image different from the first image and the second image;
A second area corresponding to the first area included in the first image is identified from the second image using the detection signal, and movement is performed using the information of the first area and the information of the second area. A motion vector acquisition unit for acquiring a vector;
And a correction unit that corrects shake of the apparatus using the motion vector. An imaging apparatus according to claim 1,
The correction unit corrects a shake of the apparatus by driving at least one part of an optical system that forms an image acquired by the second image acquisition unit and at least one of the second image acquisition unit. An imaging apparatus characterized by that. The imaging device according to claim 1 or 2, wherein
A filter unit for reducing a panning component included in the detection signal;
Including a control unit that can be operated by the photographer,
The motion vector acquisition unit specifies the second region from the second image using a signal output from the filter unit when the operation unit is operated by the photographer. . It is an imaging device given in any 1 paragraph of Claims 1-3,
The motion vector acquisition unit sets a setting area corresponding to the first area in the second image based on the detection signal, and when the correlation between the setting area and the first area is high, the setting area Is the second region. An imaging apparatus according to claim 4, wherein
When the correlation between the setting area and the first area is low, the motion vector acquisition unit calculates a correlation between the area near the setting area and the first area, and the area near the setting area An imaging apparatus, wherein when the correlation with the first area is higher than the correlation between the setting area and the first area, an area in the vicinity of the setting area is set as the second area. An imaging apparatus according to any one of claims 1 to 5, wherein
A main subject detection unit for detecting a main subject included in the third image;
The photographing apparatus according to claim 1, wherein the first region includes at least a part of the main subject. An imaging apparatus according to any one of claims 1 to 5, wherein
An in-focus area detecting unit for detecting an in-focus area of the third image;
The imaging apparatus according to claim 1, wherein the first area includes the in-focus area detected by the in-focus area detection unit. It is an imaging device given in any 1 paragraph from Claim 1 to Claim 7,
The first apparatus and the second image captured by the first image acquisition unit have a depth of field deeper than the third image captured by the second image acquisition unit. . It is an imaging device given in any 1 paragraph of Claims 1-8,
The imaging apparatus according to claim 1, wherein the first image and the second image captured by the first image acquisition unit are wider-angle images than the third image captured by the second image acquisition unit. It is an imaging device given in any 1 paragraph from Claim 1 to Claim 9,
The first image acquisition unit includes a first optical system for forming the first image and the second image,
The second image acquisition unit includes a second optical system for forming the third image having a narrower angle than the first optical system. It is an imaging device given in any 1 paragraph of Claims 1-10,
The first image obtaining unit obtains a fourth image different from the first image, the second image, and the third image after obtaining the second image,
The motion vector acquisition unit identifies a fourth region corresponding to the second region of the second image from the fourth image using the detection signal, and information on the second region and the fourth region An imaging apparatus, wherein a motion vector is acquired using information.

Images