JP2017215350A - Image blur correction device, optical unit, imaging apparatus and control method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image blur correction device capable of ensuring both of image blur correction performance and object tracking performance.SOLUTION: Detecting a run-out of an apparatus, a vibration sensor 201 outputs a run-out detection signal, a subject detection part 504 detects a subject from a pick-up image, and an imaging state notification part 510 notifies the imaging state. A control unit executes subject tracking and image blur correction using a correction lens based on the tracking target position of the detected subject and blur detection signal. Depending on the imaging state, the control unit determines which of the subject tracking and the image blur correction should be executed first.SELECTED DRAWING: Figure 5

Description

  The present invention relates to an image blur correction apparatus, an optical apparatus, an imaging apparatus, and a control method.

  When an image is picked up by an image pickup apparatus such as a digital camera, the subject image may be blurred (image blur) due to the hand shake of the user holding the camera body. An imaging apparatus having an image blur correcting unit that corrects an image blur has been proposed. Image blur correction processing includes optical image blur correction processing and electronic image blur correction processing. In the optical image blur correction process, vibration applied to the camera body is detected by an angular velocity sensor or the like, and a correction lens provided in the photographing optical system is moved according to the detection result. Image blurring is corrected by moving the image formed on the light receiving surface of the image sensor by changing the optical axis direction of the photographing optical system. Further, in the electronic image blur correction process, image blur is corrected in a pseudo manner by image processing on a captured image.

  In addition to camera shake, there is a demand for tracking a subject (moving object) with a camera, but it requires a special technique of the photographer. Patent Document 1 discloses an imaging device that divides the screen into blocks, detects a specific subject such as a face by template matching, and drives subject correction by driving a blur correction unit used for image blur correction. Yes.

JP 2010-93362 A

  In the imaging apparatus disclosed in Patent Document 1, blur correction control and subject tracking control are performed independently by a shooting operation, and therefore blur correction control is not performed during subject tracking control. Therefore, it is difficult to achieve both subject tracking and image blur correction. Further, in this imaging apparatus, the blur correction unit is driven by a target signal obtained by simply combining signals for subject tracking and image blur correction, so image blur correction control and subject tracking control interfere with each other. As a result, the blur correction unit is excessively driven to exceed the drivable end, and the performance of both subject tracking and image blur correction is significantly deteriorated.

  SUMMARY OF THE INVENTION An object of the present invention is to provide an image blur correction apparatus that can achieve both image blur correction and subject tracking performance without deteriorating.

  An image blur correction apparatus according to an embodiment of the present invention includes an output unit that outputs a shake detection signal related to a shake of the apparatus, a detection unit that detects a subject from a captured image, a notification unit that notifies a shooting state, and the detection And a control unit that performs tracking of the subject and image blur correction using a correction unit based on the tracking target position of the subject and the shake detection signal. The control means determines which of the subject tracking and the image blur correction is to be executed with priority according to the shooting state.

  According to the image blur correction device of the present invention, it is possible to achieve both without reducing image blur correction and subject tracking performance.

It is a figure which shows the structural example of the imaging device of this embodiment. It is a figure which shows the structural example of an image blurring correction apparatus. It is a disassembled perspective view which shows the structural example of a 1st correction lens drive part. It is a figure which shows the positional relationship of a 1st, 2nd correction lens drive part. It is a figure which shows the control part which performs image blur correction and subject tracking. It is a figure explaining an example of the target position calculation process of a correction lens. It is a figure explaining an example of the target position calculation process of a correction lens. This is an application example of a known subject tracking method. It is an example of various signal waveforms at the time of subject tracking processing. It is an example of various signal waveforms at the time of subject tracking processing. It is an example of various signal waveforms at the time of subject tracking processing. It is a figure explaining the conditions which switch the 1st, 2nd control mode. 6 is a diagram illustrating a configuration of a control unit included in an imaging apparatus according to Embodiment 2. FIG.

Example 1
FIG. 1 is a diagram illustrating a configuration example of an imaging apparatus according to the present embodiment.
FIG. 1 illustrates an imaging apparatus as an example of an optical apparatus including an image blur correction apparatus that performs image blur correction of a captured image. The image blur correction device drives and controls a blur correction lens or the like as a correction unit, and can be mounted on an imaging device such as a video camera, a digital camera, or a silver salt still camera, or an optical device such as a binocular, a telescope, or a field scope. . The image blur correction apparatus can also be mounted on an optical device such as an interchangeable lens for a digital single lens reflex camera.

  Hereinafter, the control for performing the image blur correction on the low frequency component and the high frequency component extracted from the shake detection signal of the apparatus is referred to as “image blur correction control”. Control for tracking a subject selected as a target so that the position of the subject image detected in the captured image approaches a specific position in the captured screen is referred to as “subject tracking control”. The specific position is, for example, the center position of the shooting screen or the position designated by the photographer.

  The imaging apparatus illustrated in FIG. 1 includes a zoom unit 101 to a control unit 119. The zoom unit 101 is a part of a photographing lens that forms an imaging optical system and has a variable magnification. The zoom unit 101 includes a zoom lens that changes the shooting magnification. The zoom drive unit 102 drives the zoom unit 101 in accordance with a control signal from the control unit 119. The first correction lens 103 is a first correction member that corrects image blur.

  The first correction lens 103 is movable in a direction orthogonal to the optical axis direction of the photographing lens. The first correction lens driving unit 104 controls driving of the first correction lens 103 in accordance with a control signal from the control unit 119. The second correction lens 113 is a second correction member that corrects image blur. The second correction lens 113 is movable in a direction orthogonal to the optical axis direction of the photographing lens. The second correction lens driving unit 114 drives the second correction lens 113 in accordance with the control signal from the control unit 119.

The aperture / shutter unit 105 includes a mechanical shutter having an aperture function. The aperture / shutter driving unit 106 drives the aperture / shutter unit 105 in accordance with a control signal from the control unit 119. The focus lens 107 used for focus adjustment is a part of the photographing lens, and the position can be changed along the optical axis of the photographing lens. The focus driving unit 108 drives the focus lens 107 in accordance with a control signal from the control unit 119.
The imaging unit 109 converts an optical image formed by the photographing optical system into an electrical signal in pixel units using an imaging element such as a CCD image sensor or a CMOS image sensor. CCD is an abbreviation for Charge Coupled Device. CMOS is an abbreviation for Complementary Metal-Oxide.

  The imaging signal processing unit 110 performs A (Analog) / D (Digital) conversion, correlated double sampling, gamma correction, white balance correction, color interpolation processing, and the like on the electrical signal output from the imaging unit 109, and performs video Convert to signal. The video signal processing unit 111 processes the video signal output from the imaging signal processing unit 110 according to the application. Specifically, the video signal processing unit 111 generates video data for display, and performs encoding processing and data file formation for recording.

  The display unit 112 displays an image as necessary based on the video signal for display output from the video signal processing unit 111. The power supply unit 115 supplies power to the entire imaging apparatus according to the application. The external input / output terminal unit 116 is used for input / output of communication signals and video signals exchanged with external devices. The operation unit 117 includes buttons, switches, and the like for the user to give instructions to the imaging apparatus. The storage unit 118 stores various data including video information and the like.

  The control unit 119 controls the entire imaging apparatus. The control unit 119 includes, for example, a CPU, a ROM, and a RAM. CPU is an abbreviation for Central Processing Unit. ROM is an abbreviation for Read Only Memory. RAM is an abbreviation for Random Access Memory. The control program stored in the ROM is expanded in the RAM and executed by the CPU, whereby each unit of the imaging device is controlled, and the operation of the imaging device including various operations described below is realized.

  The operation unit 117 includes a release switch configured such that a first switch (denoted as SW1) and a second switch (denoted as SW2) are sequentially turned on in accordance with the pressing amount of the release button. SW1 is turned on when the release button is pressed halfway, and SW2 is turned on when the release button is fully pressed. When SW1 is turned on, the control unit 119 calculates an AF (autofocus) evaluation value based on the display video signal output from the video signal processing unit 111 to the display unit 112. Then, the control unit 119 performs automatic focus detection and focus adjustment control by controlling the focus driving unit 108 based on the AF evaluation value. Further, the control unit 119 performs an AE (automatic exposure) process for determining an aperture value and a shutter speed for obtaining an appropriate exposure amount based on the luminance information of the video signal and a predetermined program diagram. When the switch SW2 is turned on, the control unit 119 controls each processing unit to perform imaging with the determined aperture and shutter speed and store the image data obtained by the imaging unit 109 in the storage unit 118.

  The operation unit 117 further includes an operation switch used for selecting an image blur correction (anti-vibration) mode. When the image blur correction mode is selected as the operation mode by operating the operation switch, the control unit 119 instructs the first correction lens driving unit 104 and the second correction lens driving unit 114 to perform the image blur correction operation. The first correction lens driving unit 104 and the second correction lens driving unit 114 that have received a control command from the control unit 119 perform an image blur correction operation until an image blur correction OFF instruction is issued. The operation unit 117 includes a shooting mode selection switch that can select one of a still image shooting mode and a moving image shooting mode. A shooting mode selection process is performed by a user operation of the shooting mode selection switch, and the control unit 119 changes the operating conditions of the first correction lens driving unit 104 and the second correction lens driving unit 114.

  The first correction lens driving unit 104 and the second correction lens driving unit 114 constitute the image blur correction device of this embodiment. The operation unit 117 has a playback mode selection switch for selecting a playback mode. When the playback mode is selected by a user operation of the playback mode selection switch, the control unit 119 performs control to stop the image blur correction operation. Further, the operation unit 117 includes a magnification change switch for instructing a zoom magnification change. When an instruction to change the zoom magnification is given by a user operation of the magnification change switch, the zoom driving unit 102 that has received the instruction via the control unit 119 drives the zoom unit 101 to place the zoom lens at the designated position. Move.

FIG. 2 is a diagram illustrating a configuration example of the image blur correction apparatus according to the present embodiment.
Hereinafter, the calculation process and the position control of the driving direction and the driving amount regarding the first correction lens 103 and the second correction lens 113 will be described. The first vibration sensor 201 and the second vibration sensor 202 constitute a shake detection unit that detects a shake of the apparatus and outputs a shake detection signal. The first vibration sensor 201 is, for example, an angular velocity sensor, and detects vibration in the vertical direction (pitch direction) of the imaging device in a normal posture (a posture in which the length direction of the captured image substantially matches the horizontal direction). The second vibration sensor 202 is, for example, an angular velocity sensor, and detects vibration in the horizontal direction (yaw direction) of the imaging apparatus in a normal posture.

  The first correction control unit 203 outputs a correction position control signal for the first correction lens 103 in the pitch direction, and controls the driving of the first correction lens 103. The second correction control unit 204 outputs a correction position control signal for the second correction lens 113 in the yaw direction, and controls driving of the second correction lens 113.

  The first lens position control unit 205 acquires a correction position control signal in the pitch direction from the first correction control unit 203 and position information in the pitch direction of the first correction lens 103 from the first Hall element 209. Perform feedback control. Accordingly, the first lens position control unit 205 controls the first drive unit 207 that is an actuator. Similarly, the second lens position control unit 206 receives the correction position control signal in the yaw direction from the second correction control unit 204 and the position information in the yaw direction of the first correction lens 103 from the second Hall element 210. Obtain and perform feedback control. Accordingly, the second lens position control unit 206 controls the second drive unit 208 that is an actuator.

Next, the drive control operation of the first correction lens 103 by the first correction lens driving unit 104 will be described.
The first correction control unit 203 acquires a shake detection signal (angular velocity signal) that represents a shake in the pitch direction of the imaging apparatus from the first vibration sensor 201. In addition, the second correction control unit 204 acquires a shake detection signal (angular velocity signal) indicating the shake in the yaw direction of the imaging apparatus from the second vibration sensor 202.

  The first correction control unit 203 generates a correction position control signal for driving the first correction lens 103 in the pitch direction based on the shake detection signal, and outputs the correction position control signal to the first lens position control unit 205. Further, the second correction control unit 204 generates a correction position control signal for driving the first correction lens 103 in the yaw direction based on the shake detection signal, and outputs the correction position control signal to the second lens position control unit 206.

  The first Hall element 209 outputs a voltage signal corresponding to the strength of the magnetic field generated by the magnet provided for the first correction lens 103 as position information in the pitch direction of the first correction lens 103. The second Hall element 210 outputs a voltage signal corresponding to the strength of the magnetic field by the magnet provided for the first correction lens 103 as position information of the first correction lens 103 in the yaw direction. The position information is supplied to the first lens position control unit 205 and the second lens position control unit 206, respectively.

  The first lens position control unit 205 uses the first drive unit 207 to perform feedback so that the position detection signal value from the first Hall element 209 converges to the correction position control signal value from the first correction control unit 203. Take control. Further, the second lens position control unit 206 sets the second drive unit 208 so that the position detection signal value from the second Hall element 210 converges to the correction position control signal value from the second correction control unit 204. To perform feedback control. Since the position detection signal values output from the first Hall element 209 and the second Hall element 210 vary, the first correction lens 103 moves to a predetermined position with respect to a predetermined correction position control signal. Thus, the output adjustment of each Hall element is performed.

  Based on the shake detection signal from the first vibration sensor 201, the first correction control unit 203 outputs a correction position control signal for moving the first correction lens 103 so as to cancel the image blur of the subject image. Based on the shake detection signal from the second vibration sensor 202, the second correction control unit 204 outputs a correction position control signal for moving the first correction lens 103 so as to cancel the image blur of the subject image. For example, the first correction control unit 203 and the second correction control unit 204 may use a correction speed control signal or a correction position control signal based on a shake detection signal (angular speed signal) or a signal obtained by performing filter processing on the shake detection signal. Is generated. With the above operation, even if vibration such as camera shake is applied to the imaging apparatus during shooting, image blur can be prevented up to a certain level of vibration. Further, the first correction control unit 203 and the second correction control unit 204 acquire a shake detection signal and the like to detect the panning state of the imaging apparatus and perform panning control.

  The drive control operation of the second correction lens 113 is the same as the drive control operation of the first correction lens 103. That is, the first correction control unit 203 generates a correction position control signal for driving the second correction lens 113 in the pitch direction based on the shake detection signal, and outputs the correction position control signal to the third lens position control unit 211. Further, the second correction control unit 204 generates a correction position control signal for driving the second correction lens 113 in the yaw direction based on the shake detection signal, and outputs the correction position control signal to the fourth lens position control unit 212. The third lens position control unit 211 uses the third drive unit 214 so that the position detection signal value from the third hall element 216 converges to the correction position control signal value from the first correction control unit 203. Perform feedback control. The fourth lens position control unit 212 uses the fourth drive unit 208 so that the position detection signal value from the fourth Hall element 213 converges to the correction position control signal value from the second correction control unit 204. Feedback control.

  In the present embodiment, the first correction control unit 203, the first lens position control unit 205, and the first drive unit 207 perform image blur correction and subject tracking in the pitch direction for the low frequency component of the shake signal in the pitch direction. The first correction control unit 203, the third lens position control unit 205, and the third drive unit 214 perform image blur correction on the high frequency component of the shake signal in the pitch direction. In addition, the second correction control unit 204, the second lens position control unit 206, and the second drive unit 208 perform image blur correction and subject tracking in the yaw direction for the low frequency component of the yaw direction shake signal. The second correction control unit 204, the fourth lens position control unit 212, and the fourth drive unit 215 perform image blur correction on the high frequency component of the shake signal in the yaw direction.

FIG. 3 is an exploded perspective view illustrating a configuration example of the first correction lens driving unit.
The first correction lens driving unit 104 includes a movable lens barrel 122 that holds the first correction lens 103 and a fixed ground plate 123. The movable barrel 122 is supported on the fixed ground plate 123 via a plurality of rolling balls 124. The movable lens barrel 122 is moved by the first electromagnetic driving unit 207 and the second electromagnetic driving unit 208. The first electromagnetic drive unit 207 constitutes the first drive unit 207 shown in FIG. 2, and includes a first magnet 1251, a first coil 1252, and a first yoke 1253. The second electromagnetic drive unit 208 constitutes the second drive unit 208 shown in FIG. 2 and includes a second magnet 1261, a second coil 1262, and a second yoke 1263. The first correction lens driving unit 104 includes an urging spring 127, a first position sensor (first Hall element 209), a second position sensor (second Hall element 210), and a sensor holder 129.

  The first correction lens 103 is a correction optical member that is driven and controlled by the first correction control unit 203 and the second correction control unit 204 and corrects image blur by decentering the optical axis by movement. Image blur correction control for moving the optical image that has passed through the photographing optical system is performed, and the stability of the image on the imaging surface is ensured. In this embodiment, a correction lens is used as the correction optical system. However, it is also possible to ensure the stability of the image on the image pickup surface by driving the image pickup device with respect to the photographing optical system. In this case, the image sensor and its drive mechanism constitute image blur correction means.

  The movable barrel 122 is a first movable portion that holds the first correction lens 103 in the central opening. The movable barrel 122 holds the first magnet 1251 and the second magnet 1261. The movable lens barrel 122 includes rolling ball receiving portions at three locations, and is supported by the rolling ball 124 so as to be movable in a plane perpendicular to the optical axis. Further, the movable lens barrel 122 has spring hooking portions at three locations, and one end of the biasing spring 127 is attached.

  The fixed ground plate 123 is a first fixed member formed in a cylindrical shape, and includes followers 1231 at three locations on the outer peripheral portion. A movable barrel 122 is disposed in the central opening 123 a of the fixed ground plate 123. Thereby, the movable amount of the movable lens barrel 122 can be limited. The fixed ground plate 123 holds the first coil 1252 and the first yoke 1253 at a position facing the magnetized surface of the first magnet 1251. The fixed ground plate 123 holds the second coil 1262 and the second yoke 1263 at a position facing the magnetized surface of the second magnet 1261. The fixed ground plate 123 includes rolling ball receiving portions at three locations. The movable lens barrel 122 is rolled and supported by the fixed base plate 123 via a plurality of rolling balls 124 that are respectively arranged on the rolling ball receiving portions. The fixed ground plate 123 includes three spring hooking portions, and one end of the biasing spring 127 is attached.

  The first electromagnetic drive unit 207 is a voice coil motor. By passing a current through the first coil 1252 attached to the fixed ground plate 123, a Lorentz force is generated between the first magnet 1251 fixed to the movable lens barrel 122 and the movable lens barrel 122 is driven. The second electromagnetic drive unit 208 is configured by rotating a voice coil motor similar to the first electromagnetic drive unit 207 by 90 ° in the direction around the optical axis, and thus detailed description thereof is omitted.

  The urging spring 127 is a tension spring that generates an urging force proportional to the amount of deformation. One end of the biasing spring 127 is fixed to the movable lens barrel 122 and the other end is fixed to the fixed base plate 123, thereby generating a biasing force therebetween. By this urging force, the rolling ball 124 is held, and the rolling ball 124 can be kept in contact with the fixed base plate 123 and the movable lens barrel 122.

  The position sensor is a magnetic sensor using Hall elements 209 and 210 that detect the magnetic fluxes of the first magnet 1251 and the second magnet 1261, respectively. These sensors detect the movement of the movable lens barrel 122 in the plane from the change in magnetic flux. The sensor holder 129 has an annular shape and is fixed to the fixed ground plate 123. The sensor holder 129 holds the position sensors (Hall elements 209 and 210) at positions facing the first magnet 1251 and the second magnet 1261, respectively. The sensor holder 129 stores the movable lens barrel 122 in an internal space formed with the fixed ground plate 123. Thereby, even when an impact force is applied to the image blur correction device or when the posture difference changes, it is possible to prevent the internal components from falling off. With the above-described configuration, the first correction lens driving unit 104 can move the first correction lens 103 to an arbitrary position on a plane orthogonal to the optical axis.

FIG. 4 is a perspective view showing a positional relationship between the first correction lens driving unit and the second correction lens driving unit.
In FIG. 4, only a part of the correction lens driving unit is disassembled and shown for explanation. The movable lens barrel 132 is a second movable part provided in the second correction lens driving unit 114. The movable barrel 132 holds the second correction lens 113 at the central opening. The fixed ground plate 133 is a second fixing member provided in the second correction lens driving unit 114. The second correction lens driving unit 114 has the same configuration as that of the first correction lens driving unit 104 except for the shape of the lens and the shape of the movable lens barrel 132 that holds the lens, and a detailed description thereof will be omitted.

  FIG. 5 is a diagram illustrating a control unit that performs image blur correction and subject tracking in the pitch direction. The second correction control unit 204, the second lens position control unit 206, the fourth lens position control unit 212, the second drive unit 208, and the fourth drive unit 215 perform control to perform image blur correction and subject tracking in the yaw direction. The part is composed. Since this control unit has the same configuration as that of the control unit shown in FIG.

  The first vibration sensor 201 detects a shake applied to the imaging apparatus and outputs a shake detection signal (angular velocity signal). The first correction control unit 203 includes low-pass filters (hereinafter abbreviated as LPF) 501, 502, and 503 and a subtraction unit 500. The output signal of the first vibration sensor 201 is input to the LPF 501 and the subtraction unit 500, respectively. The output signal of the LPF 501 is input to the LPF 502 and the subtracting unit 500, respectively.

  The LPF 501 extracts a low frequency component from the shake detection signal output from the first vibration sensor 201. The low frequency camera shake signal extracted by the LPF 501 is integrated by the LPF 502 to generate a shake angle signal in which only the low frequency component is extracted. LPFs 501, 502, and 503 can change the time constant until the filter is stabilized. The time constant until the filter is stable can be changed, for example, the cutoff frequency can be changed by changing the coefficient value used for the filter operation, or the buffer holding the filter operation result (intermediate value) can be arbitrarily set. Indicates that it can be freely rewritten at the timing. Accordingly, it is possible to change the image blur correction control operation according to the shooting situation by changing the filter cutoff according to the shooting state of the imaging apparatus notified from the shooting state notification unit 510. Become.

  5 includes a subject detection unit 504, a subject tracking unit 505, a gain unit 506, and an addition unit 507. The subject detection unit 504 detects a subject (main subject) from the captured image and outputs the detection result as main subject detection information. The subject tracking unit 505 generates tracking target information based on the main subject detection information output from the subject detection unit 504. The tracking target information indicates the tracking target position of the subject. The adder 507 combines the generated tracking target information with the low-frequency component deflection angle signal generated by the LPF 502. Further, after accumulating the gain unit 506, the adding unit 511 again synthesizes it with the shake angle signal of the low frequency component to obtain the target position of the first correction lens 103. Further, the gain coefficient 506 is adjusted and the cut-off frequency of the LPF 501 is changed by information notified from the shooting state notification unit 510.

  The shooting state notification unit 510 notifies information indicating the shooting state. Specifically, the shooting state notification unit 510 includes main subject detection information from the subject detection unit 504, a shake detection signal from the vibration sensor 201, shooting mode information from the operation unit 117, first and third lens positions, and the like. Notify information. The control unit can change whether or not to perform the subject tracking operation by changing the gain coefficient in accordance with the notified shooting state. For example, when the focal length is set to be short by the operation unit 117, the control unit changes the gain coefficient to be multiplied by 1 to the tracking target position output by the subject tracking unit 505, so that the subject tracking operation is performed. I do.

  In addition, when the focal length is set to be long by the operation unit 117, the control unit sets the gain coefficient to 0 so that the subject tracking operation is not performed. Then, the gain coefficient is set to a value larger than 0 and smaller than 1 for an intermediate focal length region. As a result, in a region where the angle of view due to camera shake is large and the correction lens drive amount is large and the focal length is long, the subject tracking operation is not performed, and a movable range is secured for image blur correction. Also, in areas where the angle of view due to camera shake is small and the correction lens drive amount is small and the focal length is short, subject tracking operations are prioritized so that image blur correction operations do not interfere with subject tracking operations. Makes it easier to capture the center of the shooting screen. As a result, the operability is improved by subject tracking operation in a short focal length region, and the correction lens reaches the end of the movable range due to subject tracking operation in a long focal length region, and the image blur correction performance decreases. Can be prevented. That is, the control unit determines which of the subject tracking and the image blur correction is to be executed with priority according to the shooting state.

  The subtraction unit 500 of the first correction control unit 203 extracts the high frequency component of the shake detection signal by subtracting the low frequency component extracted by the LPF 501 from the shake detection signal from the first vibration sensor 201. The LPF 503 integrates the high-frequency component extracted by the subtracting unit 500 to convert the angular velocity information into angle information, and generates a camera shake angle signal in which only the high-frequency component is extracted. Note that the output of each filter can be output at an arbitrary magnification by changing the coefficient values of the filter operations of the LPF 502 and the LPF 503.

  Further, the cut-off frequency of the LPF 501 can be changed by information of the shooting state notification unit. Thereby, the frequency extracted as the low frequency component of the shake detection signal used for calculating the tracking target position of the subject can be changed.

  In the present embodiment, the subject detection unit 504, the subject tracking unit 505, the gain unit 506, and the addition unit 507 can simultaneously perform subject tracking and shake correction using the image blur correction unit. For example, the subject detection unit 504 detects a subject using the image signal from the imaging signal processing unit 110 and subject distance information from the focus driving unit 108, and the subject tracking unit 505 uses a known template matching to detect the subject. A tracking target position is generated. The subject distance information is information indicating a distance between the imaging apparatus and the subject, or information such as a focus lens position and a focus detection amount corresponding to the distance.

  The gain unit 506 multiplies the tracking target position obtained by the subject tracking unit 505 by a predetermined gain coefficient and outputs the result to the adding unit 507. The adding unit 507 adds the output of the gain unit 506 and the output of the LPF 502, and outputs the addition result to the adding unit 508 as the first correction lens target position. On the other hand, the corrected lens target position generated from the high-frequency component of the camera shake angle signal calculated by the LPF 503 is input to the third lens position control unit 211 via the addition unit 509.

  In the feedback control system including the first lens position control unit 205, the position information of the first correction lens 103 detected by the first Hall element 209 is subtracted from the first correction lens target position by the addition unit 508. This subtraction result is input to the first lens position control unit 205 as a drive signal for the first correction lens. The first drive unit 207 drives the first correction lens 103 in accordance with a control signal output from the first lens position control unit 205, and an image blur correction operation is executed by position feedback control. In the feedback control system including the third lens position control unit 211, the position information of the second correction lens 113 detected by the third Hall element 216 is subtracted from the third correction lens target position by the addition unit 509. The subtraction result is input to the third lens position control unit 211 as a drive signal for the second correction lens. The third drive unit 214 drives the second correction lens 113 in accordance with a control signal output from the third lens position control unit 211, and an image blur correction operation is executed by position feedback control. For the first lens position control unit 205 and the third lens position control unit 211, any control calculation unit may be used. For example, a P (proportional) I (integral) D (differential) controller can be used.

  As described above, the image blur correction and the object tracking operation for the low frequency component are performed by the first correction means using the first correction lens 103, and the second correction lens 113 is used for the image blur correction for the high frequency component. This is performed by the second correcting means used. In this configuration, since the first correction lens 103 is mainly used for correction of large shake at low frequency and subject tracking, the stroke that can be driven is large. On the other hand, since the second correction lens 113 is mainly used to correct only a small shake at a high frequency, the drivable stroke is small. A mechanism using a correction lens having a large image blur correction angle generally involves an increase in size. Therefore, in this embodiment, the role is divided between the low-frequency image stabilization and the correction lens used for subject tracking and the correction lens used for the high-frequency image stabilization. Accordingly, it is possible to achieve both the image blur correction operation and the subject tracking operation while preventing the mechanism portion from becoming excessively large and the resolution for detecting the lens position from excessively increasing.

6 and 7 are flowcharts for explaining an example of the correction lens target position calculation process.
The processing described with reference to FIGS. 6 and 7 is executed at regular time intervals according to a control program read from the memory by the CPU of the control unit 119.
First, in S101, processing starts. In S102, the first vibration sensor 201 detects camera shake or the like, and acquires a shake detection signal. Subsequently, in S <b> 103, the control unit 119 determines whether or not the shooting state has been changed based on the notification from the shooting state notification unit 510. If there is no change in the shooting state, the process proceeds to S104. If there is a change in the shooting state, the process proceeds to S118. In S118, the control unit 119 changes the cutoff frequency of the LPF 501 to a predetermined state. In S119, the control unit 119 changes the gain unit 506 to a predetermined state. Then, the process proceeds to S104. The shooting state that is the condition to be changed in step S118 and step S119 and the predetermined state of the gain unit 506 will be described later.

  Next, in S104, the LPF 501 extracts a low-frequency component from the shake detection signal. In step S105, the control unit 119 stores the calculation result of the LPF 501 in the memory. In step S106, the LPF 502 integrates the low frequency component of the shake detection signal extracted by the LPF 501, and converts the angular velocity information into angle information.

  Next, in S107 of FIG. 7, the subject detection unit 504 acquires position information on the image of the main subject. Subsequently, in S108, the subject tracking unit 505 calculates the tracking target position of the subject. In S109, the adding unit 507 adds the tracking target position and the low-frequency shake correction target position calculated in S106. The subject position obtained from the subject detection unit 504 is detected by combining the remaining shake amount when the shake of the imaging apparatus is corrected by the correction lens and the movement amount of the subject. Accordingly, in S109, only the movement amount of the subject is extracted by subtracting the correction lens shake remaining amount from the subject detection position information.

  In step S110, the control unit 119 multiplies a gain unit 506 that determines how much the correction lens is driven according to the amount of movement of the subject. In S111, the control unit 119 adds the multiplication result of the gain unit 506 and the low-frequency component extracted by the LPF 502 to calculate the first correction lens target position. In step S <b> 112, the control unit 119 inputs the first correction lens target position to the first lens position control unit 205. As a result, the first correction lens 103 is driven.

  Next, in S113, the control unit 119 acquires the low frequency component of the shake signal stored in S105. In S114, the subtracting unit 500 subtracts the low frequency component of the shake signal acquired in S113 from the shake detection signal acquired in S102. Thereby, a high frequency component of the shake detection signal is extracted. In this way, the shake detection signal is divided into a low frequency component and a high frequency component according to the cutoff frequency set by the LPF 501.

  Next, in S115, the LPF 503 integrates the high-frequency component of the extracted shake detection signal to convert the angular velocity signal into an angle signal. In S <b> 116, the high frequency component of the shake detection signal converted into the angle signal is input to the third lens position control unit 211. As a result, the second correction lens 113 is driven. In step S117, the process ends.

FIG. 8 is a control block diagram in the case where a known subject tracking method is simply applied to two correction lenses.
The control unit shown in FIG. 8 generates a subject tracking target signal by processing the subject tracking unit 505 from the subject position information obtained from the subject detection unit 504, and drives the first correction lens 103 accordingly. Further, the control unit illustrated in FIG. 8 performs image blur correction by driving the second correction lens 113 in accordance with a shake signal of the imaging device obtained from the vibration sensor 201.

FIG. 9 is an example of various signal waveforms during subject tracking processing by the control unit shown in FIG.
It is assumed that the subject moves at a constant speed. FIG. 9A shows the first correction lens target position. FIG. 9B shows the second correction lens target position. The horizontal axis represents time, and the vertical axis represents the correction lens swing angle.

  Since the second correction lens is driven so as to correct the shake of the image pickup apparatus, an operation that the photographer intentionally pans to follow the subject is also regarded as a shake and is corrected. Therefore, the composition cannot be changed to the photographer's expectation and the situation is very difficult to shoot. Ideally, the operation of following the photographer's subject is canceled by the second correction lens, and the composition is fixed. Accordingly, in the subject position information detected from the subject detection unit 504, the angle change that moves at a constant speed is directly commanded to the first correction lens as the tracking target position of the subject.

  As described above, in the control unit shown in FIG. 8, since each correction lens independently performs the shake correction operation and the subject tracking operation, the stroke that can be driven by each correction lens is easily exceeded. Therefore, both the image blur correction and subject tracking performance may be degraded.

10 and 11 are examples of various signal waveforms during subject tracking processing by the control unit of the present embodiment.
The cut-off frequency of the LPF 501 for extracting high frequency and low frequency components from the amount of camera shake detected by the vibration sensor 201 is set under the following conditions. The cutoff frequency of the LPF 501 is set lower than 5 Hz so that a frequency mainly included in the camera shake signal, for example, a signal in the vicinity of 5 Hz is included in the high frequency shake signal calculated by the LPF 503.

  FIG. 10A shows the first correction lens target position. FIG. 10B shows the second correction lens target position. FIG. 10C shows the output of the vibration sensor 201. FIG. 10D shows the output of the LPF 502. FIG. 11A shows the output of the LPF 503. FIG. 11B shows the output of the subject detection unit 504. FIG. 11C shows the output of the gain unit 506. The horizontal axis represents time, and the vertical axis represents the correction lens swing angle. The output of the gain unit 506 is a waveform example set to 1 time for subject tracking. For simplification, all waveform examples are converted using the correction lens swing angle.

  The image pickup apparatus according to the present embodiment first divides the output of the vibration sensor 201 in FIG. 10C into a low frequency component and a high frequency component at the cut-off frequency of the LPF 501. From the action of the photographer tracking the subject, the cut-off frequency of the LPF 501 is determined so that the frequency included in the movement of the tracked subject is included on the low frequency side. As a result, the extracted shake signal becomes the output of the LPF 502 in FIG. 10D and the output of the LPF 503 in FIG.

  As shown at the second correction lens target position in FIG. 10B, the high-frequency side shake signal is input to the third lens position control unit 211, and image blur correction is performed by the second correction lens 113 accordingly. . On the other hand, since the shake on the high frequency side is corrected by the second correction lens, the difference position between the low frequency side shake remaining signal including the movement of the photographer intentionally following the subject and the movement position of the subject is the subject detection unit. 504 (refer to the output of the subject detection unit 504 in FIG. 11B).

  In addition, the imaging device combines the output of the subject detection unit 504 in FIG. 11B and the output of the LPF 502 in FIG. As a result, the actual movement angle of the subject is calculated from the output of the subject detection unit 504 by removing the movement angle due to the operation in which the photographer intentionally changes the composition (the output of the gain unit 506 in FIG. 11C). See).

  Further, by calculating the difference between the calculated output of the gain unit 506 in FIG. 11C and the output of the LPF 502 in FIG. 10D, a target position signal for subject tracking by the first correction lens can be obtained. Calculated. FIG. 10A shows the first correction lens target position corresponding to the calculated target position signal.

  According to the imaging apparatus of the present embodiment, the first correction lens is compensated by an amount that compensates only for the difference position where the subject could not be tracked due to the change of the photographer's composition without preventing the change of the photographer's composition as a shake. Is driven. Further, high-frequency shake that is not related to the subject tracking operation can be prevented by the second correction lens. Therefore, the first and second correction lenses can be prevented from being excessively driven, and the respective correction lenses can be prevented from exceeding a drivable stroke, and both the image blur correction and subject tracking performance can be prevented from being deteriorated. .

  In addition, the imaging apparatus of this embodiment removes the influence of low-frequency shake of the imaging apparatus such as shake when the photographer intentionally changes the composition from the subject detection information, and extracts only the movement of the subject. . Therefore, it is possible to easily switch between the case where it is desired to emphasize the image blur correction performance and the case where the operability by the tracking operation is important depending on the shooting state by changing the coefficient of the gain unit 506. That is, if the gain coefficient is close to 1, the shake operation is not performed for the low frequency component, and the subject correction operation is mainly performed by the first correction lens and the shake correction is mainly performed for the high frequency component by the second correction lens. If the gain is close to 0, the first correction lens can perform shake correction for the low frequency component and the second correction lens can perform shake correction for the high frequency component.

FIG. 12 is a diagram illustrating conditions for switching between the first control mode and the second control mode by changing the gain.
The first control mode is a mode that prioritizes image blur correction, and the second control mode is a mode that prioritizes subject tracking. In the first control mode, for example, the control unit performs image blur correction using the first correction lens 103 and the second correction lens 113 without performing tracking of the subject. In the second control mode, for example, the control unit executes tracking of the subject using the first correction lens 103 and image blur correction using the second correction lens 113. When the gain unit 506 is set to a value larger than 0 and smaller than 1, both image blur correction and subject tracking are performed, and it is possible to freely select which one is weighted.

(Focal length)
When the focal length is changed by the zoom drive unit 102, the change in the angle of view with respect to the shake of the imaging apparatus is large in a region with a long focal length, and the image blur correction angle by the correction lens is generally small. Therefore, the imaging apparatus selects the first control mode in an area with a long focal distance, and selects the second control mode in an area with a short focal distance, with an emphasis on ease of shooting by the photographer. In the region where the focal length is between these, the first and second control modes are both weighted.

(Image sensor shake)
The imaging device selects the first control mode when the shake of the imaging device is greater than a predetermined value, and selects the second control mode when the shake is less than or equal to the predetermined value.

(Shooting situation)
The imaging device selects the first control mode when the shooting state indicates a state of shooting while walking. When the shooting state indicates a fixed point shooting, panning shooting, panning or tilting state, the second control mode is selected. During panning, panning, or tilting, the photographer intentionally changes the composition, so that the composition change operation is not hindered by image blur correction.

(Shooting mode)
The imaging device selects the second control mode at the time of composition determination before still image shooting or at the time of moving image shooting, and at the time of still image shooting, the first control mode is selected.

(Subject position in the shooting screen)
When the main subject is near the center of the shooting screen or at a position preset by the photographer, the subject tracking operation is not necessary, so the imaging apparatus selects the first control mode, and at other times. Selects the second control mode. That is, when the subject is near a predetermined position on the shooting screen, the control unit preferentially executes image blur correction. The control unit prioritizes subject tracking when the subject is at a position different from the vicinity of a predetermined position on the shooting screen.

(Physical switch is pressed)
When the photographer uses the predetermined switch to set the mode setting to the predetermined setting value, the second control mode is selected, and when the predetermined value setting is canceled, the first control mode is selected. To do.

(Photograph image degradation)
In general, optical performance such as resolution, light quantity, and distortion tends to deteriorate when the correction lens moves away from the driving center. Therefore, the imaging apparatus selects the first control mode so that the correction lens is difficult to move away from the drive center when the optical performance deterioration (photographed image deterioration) is large, and when the optical performance deterioration is not large, the second control mode. Select.

(Correction lens position)
When the correction lens is at the end of the drivable range, there is a risk that the performance of shake correction and subject tracking may be degraded. Therefore, the first control mode is selected so as not to track the subject, and it is near the center of the drivable range. Sometimes the second control mode is selected.

(Subject movement speed, imaging device shake frequency)
When the moving speed of the subject is higher than a predetermined value, or when the shake frequency of the imaging apparatus is higher than the predetermined value, it is difficult to detect the subject position or the accuracy may be lowered due to a known pattern matching process. Therefore, the imaging device selects the first control mode. The second control mode is selected when the subject moving speed is equal to or lower than a predetermined value, or when the shake frequency of the imaging apparatus is equal to or lower than the predetermined value.

  In this embodiment, subject movement information is extracted from subject tracking information and low-frequency components of image blur correction, and subject tracking is prioritized using subject movement information according to shooting conditions, or shake correction is prioritized. And choose what to do. Thereby, it is possible to achieve both the image blur correction operation and the subject tracking operation without deteriorating each other's performance by interfering with each other's operation.

(Example 2)
FIG. 13 is a diagram illustrating a configuration of a control unit included in the imaging apparatus according to the second embodiment.
The imaging apparatus of Example 2 does not have the second correction lens. The image blur correction lens target positions at low frequency and high frequency calculated by the LPF 502 and LPF 503 are added by the adding unit 700. On the other hand, the tracking target position multiplied by the gain coefficient in the gain unit 506 is similarly added by the adding unit 700. The combined signal after the addition is input to the first lens position control unit 205 via the subtraction unit 701, so that a subject tracking operation and an image blur correction operation are performed.

  According to the imaging apparatus of the second embodiment, the subject tracking operation and the image blur correction operation can be performed with one correction lens. Therefore, it is possible to realize both the image blur correction operation and the subject tracking operation while suppressing an increase in size and cost of the apparatus due to having a plurality of correction lenses.

(Other embodiments)
The present invention supplies a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in a computer of the system or apparatus read and execute the program This process can be realized. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

104 1st correction lens drive part 114 2nd correction lens drive part 119 Control part

Claims (20)

Output means for outputting a shake detection signal relating to the shake of the device;
Detection means for detecting a subject from a captured image;
A notification means for notifying the shooting state;
Control means for performing tracking of the subject and image blur correction using a correction means based on the detected tracking target position of the subject and the shake detection signal;
The control means determines which of the subject tracking and the image blur correction is to be prioritized and executed according to the shooting state. The control means includes
Using the first correction unit, the subject tracking and the image blur correction for the low frequency component of the shake detection signal are executed,
The image blur correction apparatus according to claim 1, wherein image blur correction is performed on a high frequency component of the shake detection signal using a second correction unit. In the first control mode in which priority is given to the image blur correction, the control means does not perform tracking of the subject but performs image blur correction using the first correction means and the second correction means. In the second control mode that prioritizes tracking of the subject, tracking of the subject using the first correction unit and image blur correction using the second correction unit are executed. The image blur correction device according to claim 2. The control means includes
Based on the detected position of the subject, the tracking target position of the subject is calculated,
4. The image according to claim 2, wherein a drive signal for the first correction unit is generated by adding the calculated tracking target position and a low frequency component of the shake detection signal. 5. Blur correction device. The control means multiplies the addition result of the tracking target position and the low frequency component of the shake detection signal by a predetermined gain coefficient, and subtracts the gain coefficient multiplication result from the low frequency component of the shake detection signal. The image blur correction apparatus according to claim 4, wherein a drive signal for the first correction unit is generated by the following. The image blur correction apparatus according to claim 5, wherein the control unit determines which of the tracking of the subject and the image blur correction has priority by changing the gain coefficient. The control means includes first extraction means for extracting a low frequency component from the shake detection signal, and second extraction means for extracting a high frequency component from the shake detection signal,
The image blur correction apparatus according to claim 6, wherein the first extraction unit changes a frequency to be extracted according to the shooting state. The control means prioritizes the image blur correction in an area with a long focal distance, and prioritizes the subject tracking in an area with a short focal distance. The image blur correction device according to any one of the above. The control means prioritizes the image blur correction when the shake of the apparatus is greater than a predetermined value, and prioritizes the subject tracking when the shake of the apparatus is equal to or less than the predetermined value. The image blur correction device according to claim 1, wherein: The control means prioritizes the image blur correction when the shooting state indicates a walk shooting shooting state, and the shooting state is fixed point shooting, panning shooting, panning or tilting. The image blur correction apparatus according to any one of claims 1 to 7, wherein when the state is indicated, the subject tracking is executed with priority. The control unit according to any one of claims 1 to 7, wherein the control unit prioritizes the image blur correction when capturing a still image and prioritizes the subject tracking when capturing a moving image. The image blur correction apparatus described. The control means prioritizes the image blur correction when the subject is near a predetermined position on the shooting screen, and when the subject is at a position different from the vicinity of the predetermined position on the shooting screen, The image blur correction apparatus according to any one of claims 1 to 7, wherein subject tracking is performed with priority. The control means prioritizes the image blur correction when a predetermined value is set by a predetermined switch, and prioritizes the subject tracking when the setting of the predetermined value is canceled. The image blur correction apparatus according to claim 1, wherein the image blur correction apparatus is an image blur correction apparatus according to claim 1. The control means prioritizes the image blur correction when the degradation of the captured image is large, and prioritizes the subject tracking when the degradation of the captured image is small. The image blur correction device according to any one of 1 to 7. The control means prioritizes the image blur correction when the correction means is at the end of the drivable range, and prioritizes the subject tracking when the correction means is near the center of the drivable range. The image blur correction apparatus according to claim 1, wherein the image blur correction apparatus is executed. The control means prioritizes the image blur correction when the moving speed of the subject is greater than a predetermined value, and prioritizes the subject tracking when the moving speed of the subject is equal to or less than the predetermined value. The image blur correction apparatus according to claim 1, wherein the image blur correction apparatus is an image blur correction apparatus according to claim 1. The control means prioritizes the image blur correction when the vibration frequency of the device is higher than a predetermined value, and performs tracking of the subject when the vibration frequency of the device is equal to or lower than a predetermined value. The image blur correction apparatus according to claim 1, wherein the image blur correction apparatus is executed with priority.   An optical apparatus comprising the image blur correction device according to claim 1.   An image pickup apparatus comprising the image blur correction apparatus according to claim 1. An output step of outputting a shake detection signal related to the shake of the device;
A detection process for detecting a subject from a captured image;
A notification process for notifying the shooting state;
A control step of performing tracking of the subject and image blur correction using a correction unit based on the detected tracking target position of the subject and the shake detection signal;
In the control step, it is determined which of the subject tracking and the image blur correction is to be executed with priority according to the shooting state.

Images