JP2022014068A - Imaging apparatus and control method of the same - Google Patents

Abstract

To obtain an excellent shake correction effect when performing shake correction on each of an interchangeable lens side and an imaging apparatus side.SOLUTION: An imaging apparatus 120 can be detachably attached with an interchangeable lens 100 that performs the first shake correction to the vibration. The imaging apparatus 120 comprises: correction means which performs the second shake correction to the vibration; vector detection means which detects motion vectors of a plurality of regions in an image obtained with imaging; representative decision means which decides a representative vector from the motion vectors of the plurality of regions; and control means which controls the second shake correction according to the representative vector. The representative decision means acquires a second correction amount according to an angle of the shake of the imaging apparatus and a sharing ratio of the second shake correction occupied in the shake correction by the first and second shake corrections, and decides the representative vector on the basis of the second correction amount.SELECTED DRAWING: Figure 1

Description

The present invention relates to an image pickup apparatus having a function of reducing (correcting) image shake.

Some imaging systems such as digital still cameras and video cameras (hereinafter referred to as camera systems) have a shake correction function for correcting image shake caused by camera shake such as camera shake. Some interchangeable-lens camera systems perform optical shake correction by moving the correction lens within the interchangeable lens, and also perform electronic shake correction by moving the cutout range from the image data obtained by the image pickup element in the camera.

Patent Document 1 discloses a camera system that controls the sharing ratio of optical shake correction on the interchangeable lens side and electronic shake correction on the camera side based on the amount of camera shake detected by a shake sensor provided in the interchangeable lens. Has been done.

Japanese Unexamined Patent Publication No. 2017-219635

However, the camera system of Patent Document 1 has a configuration of further correcting image shake (image shake residue) that cannot be corrected due to a detection error of the shake sensor due to a temperature factor, a tracking delay of the drive of the correction lens with respect to the camera shake, and the like. I haven't.

The present invention makes it possible to obtain a good shake correction effect when the shake correction is performed on each of the interchangeable lens side and the camera side.

The image pickup apparatus as one aspect of the present invention can be attached to and detached from an interchangeable lens that performs the first vibration correction for vibration. The image pickup apparatus is represented by a correction means for performing a second vibration correction for the vibration, a vector detection means for detecting motion vectors in a plurality of regions in an image obtained by imaging, and motion vectors in a plurality of regions. It has a representative determining means for determining a vector and a control means for controlling a second runout correction according to the representative vector. The representative determining means acquires a second correction amount according to the runout angle of the image pickup apparatus and the share of the second runout correction in the runout correction by both the first and second runout corrections, and the first It is characterized in that the representative vector is determined based on the correction amount of 2.

Further, the control method as another aspect of the present invention is applicable to an image pickup device in which an interchangeable lens that performs a first vibration correction for vibration can be attached and detached and a second vibration correction is performed for the vibration. Will be done. The control method includes a step of detecting motion vectors of a plurality of regions in an image obtained by imaging, a step of determining a representative vector from motion vectors of a plurality of regions, and a second runout correction according to the representative vector. It has a step to control. In the step of determining the representative vector, the second correction amount is acquired according to the runout angle of the image pickup apparatus and the share of the second runout correction in the runout correction by both the first and second runout corrections. , The representative vector is determined based on the second correction amount. A computer program that causes the computer of the image pickup apparatus to execute a process according to the above control method also constitutes another aspect of the present invention.

According to the present invention, a good shake correction effect can be obtained when the first runout correction is performed on the interchangeable lens side and the second runout correction is performed on the image pickup device side.

The block diagram which shows the structure of the camera system which is an Example of this invention. The figure which shows the relationship between the subject and a camera system at the time of non-shake correction in an Example. The figure which shows the relationship between the captured image and a motion vector at the time of non-shake correction in an Example. The figure which shows the relationship between the subject and a camera system at the time of shake correction in an Example. The figure which shows the relationship between the camera system and a motion vector at the time of runout correction in an Example. The figure which shows the movement amount of the subject image at the time of a non-shake correction and at the time of a shake correction in an Example. The histogram figure which shows the determination method of the representative vector in an Example. The flowchart which shows the representative vector determination process in an Example.

Hereinafter, examples of the present invention will be described with reference to the drawings.

FIG. 1 shows the configuration of an interchangeable lens camera system according to an embodiment of the present invention. The camera system is composed of an image pickup device (hereinafter, referred to as a camera) 120 such as a digital still camera or a video camera, and an interchangeable lens (hereinafter, simply referred to as a lens) 100 that can be attached to and detached from the camera 120. In this embodiment, when vibration such as camera shake occurs in the camera system (hereinafter referred to as camera shake), the correction lens (optical element) 110 described later is moved in the lens 100 to optically reduce (correct) the image shake. ) The optical shake correction as the first shake correction is performed, and the range cut out from the image data (frame image) obtained by the image pickup element 121 described later in the camera 120 is moved to electronically correct the image shake. Performs electronic runout correction as runout correction.

In the lens 100, the shake detection unit 101 is a shake sensor that detects vibration such as camera shake (hereinafter referred to as camera shake), and in this embodiment, it is an angular velocity sensor that outputs a voltage value corresponding to the angular velocity of the camera shake. The A / D converter 102 converts the voltage value output from the runout detection unit 101 into digital data (angular velocity data), and outputs the voltage value to the signal processing unit 103. The signal processing unit 103 performs signal processing to generate a runout correction amount from the angular velocity data. For signal processing, the HPF that removes the DC component superimposed on the angular velocity data, the integral filter for angle conversion from the angular velocity, and the HPF cutoff for removing the deflection component due to camera panning and tilting from the angular velocity data. Frequency change processing is included. The data converted into angle units by these signal processes is output to the division unit 104.

The dividing unit 104 determines a sharing ratio (hereinafter referred to as a correction sharing ratio) between the optical shake correction on the lens 100 side and the electronic shake correction on the camera 120 side, and the lens 100 side according to the determined correction sharing ratio. The amount of shake correction by optical shake correction (first correction amount: hereinafter referred to as optical correction amount) and the amount of shake correction by electronic shake correction on the camera 120 side (second correction amount: hereinafter referred to as electronic correction amount). Set. The correction sharing ratio is the maximum image shake amount that can be corrected by moving the correction lens 110 on the lens 100 side (hereinafter referred to as the maximum optical correction amount) and the maximum image shake amount that can be corrected by electronic shake correction on the camera 120 side (hereinafter referred to as the maximum optical correction amount). Hereinafter, it is determined based on the ratio with the maximum electronic correction amount).

Specifically, the electronic runout correction limiter 128 in the camera 120 transmits the maximum electronic correction amount held in advance to the dividing unit 104 via the camera communication processing unit 127 and the lens communication processing unit 113. The dividing unit 104 determines the correction sharing ratio from the maximum optical correction amount held in advance and the received maximum electronic correction amount, and calculates (acquires) the optical correction amount and the electronic correction amount according to the correction sharing ratio. .. The optical correction amount is calculated based on the share of the optical runout correction in the runout correction by both the optical runout correction and the electronic runout correction, and the electronic correction amount is based on the share of the electronic runout correction in the runout correction by both of the above. It is calculated.

For example, when the ratio of the maximum optical correction amount (1 deg) and the maximum electronic correction amount (3 deg) is 1: 3, the correction sharing ratio is 1: 3, and when the camera shake amount is 2 deg, the optical correction amount is set. Set 0.5 deg and the electronic correction amount to 1.5 deg. The optical correction amount set by the dividing unit 104 is output to the control filter 105, and the optical correction amount is output to the pixel unit conversion unit 112.

In the optical runout correction, the correction lens 110 included in the image pickup optical system (not shown) moves in a direction orthogonal to the optical axis of the image pickup optical system (including the case where it rotates about a point on the optical axis). Correct the runout. To drive the correction lens 110, the position of the correction lens 110 is detected, and feedback control is performed so that the deviation amount between the detected position and the target position corresponding to the optical correction amount approaches zero. A deviation amount, which is a difference between the optical correction amount output from the dividing unit 104 and the detection position of the correction lens 110, is input to the control filter 105.

The control filter 105 includes an amplifier that amplifies the input signal with a predetermined gain and a phase compensation filter. The control filter 105 amplifies and phase compensates the deviation amount input as a signal to generate motor control data, and outputs this to the pulse width modulation unit 106. The pulse width modulation unit 106 converts the input motor control data into a PWM waveform which is a waveform that changes the duty ratio of the pulse wave and outputs it to the motor driver 107.

The motor driver 107 applies a drive voltage to the motor 108 based on the input PWM waveform. The motor 108 is a voice coil type motor, and when a voltage is applied to the coil from the motor driver 107, a driving force for moving the correction lens 110 in a direction orthogonal to the optical axis is generated.

As a result of the movement of the correction lens 110, the translational shake of the subject image on the image pickup element (imaging surface) 121 caused by the camera shake is corrected. The position of the correction lens 110 is detected by the position detection unit 109. The position detection unit 109 is composed of, for example, a magnet and a magnetic sensor (Hall sensor) provided at a position facing the magnet, and outputs a voltage that changes according to the movement of the correction lens 110. The A / D converter 111 converts the output voltage from the position detection unit 109 into digital data. The deviation amount between the detection position of the digitized correction lens 110 and the target position described above is input to the control filter 105 as deviation amount data. In this way, the feedback control system that controls the drive of the correction lens 110 is configured.

On the other hand, the electronic correction amount in the electronic runout correction is an amount in pixel units indicating the position of the cutting range from the image data (hereinafter referred to as the image cutting position). The pixel unit conversion unit 112 converts the electronic correction amount (unit is an angle) from the division unit 104 into a pixel unit value. Specifically, when the focal length of the image pickup optical system is f, the electronic correction amount is x (deg), and the distance between pixels (pixel pitch) in the image sensor 121 is g (μm), the electronic correction amount is in pixel units. The value p converted to is represented by p = f × tan (x) / g. However, when x is minute with respect to the focal length f, f × tan (x) can be regarded as f × x, and p can be expressed as p = fx / g.

The pixel unit conversion unit 112 acquires information on the focal length f from the image pickup optical system, and acquires information on the pixel pitch of the image pickup element 121 from the camera 120.

The lens communication processing unit 113 receives an information transmission request from the camera 120 (camera communication processing unit 127) and transmits the corresponding information to the camera 120. The electronic correction amount (pixel unit) from the pixel unit conversion unit 121 is supplied to the camera 120 via the lens communication processing unit 113.

In the camera 120, the light that has passed through the image pickup optical system of the lens 100 is received and photoelectrically converted by the image pickup element 121 using a CCD sensor, a CMOS sensor, or the like. The image pickup signal output from the image pickup element 121 is converted into a digital signal by A / D conversion after analog signal processing such as AGC (Automatic Gain Control) processing is performed. Image data is generated by performing digital signal processing such as gamma correction and white balance processing on the digital image pickup signal. The image data is converted into a video signal (moving image) conforming to the NTSC format and stored in the image memory 122.

The motion vector detection unit (vector detection means) 123 detects the motion vector based on the current frame image (or field image) in the motion image obtained by imaging and the luminance signal included in the previous frame image. As a motion vector detection method, a known block matching method or the like is used. The block matching method is a method of dividing a frame image into a plurality of blocks and detecting similar parts between the previous frame image and the current frame image in block units. In the previous frame image, the block having the largest correlation value with the specific block in the current frame image is regarded as a similar block, and the displacement amount between the position of the specific block and the position of the similar block is the motion information between one frame. That is, it is detected as a motion vector. The motion vector detection unit 123 detects such a motion vector for each of the plurality of blocks. The motion vector may be detected by a method other than the block matching method.

The representative vector detection unit (representative determination means) 125 determines a representative vector from the motion vectors of a plurality of blocks detected by the motion vector detection unit 123 by using the electronic correction amount acquired from the lens 100. The representative vector in this embodiment is a motion vector used to determine the image cropping position in the electron runout correction among the motion vectors detected in a plurality of blocks.

The motion vector of the background area in the moving image is caused by the camera shake, and the image shake generated between the frames in the moving image can be corrected by performing the electronic shake correction based on the motion vector of the background area. .. Therefore, in this embodiment, the motion vector in the background region is calculated as a representative vector. The processing by the representative vector detection unit 125 will be described in detail later.

The image deformation amount calculation unit (control means) 129 determines the image cutout position based on the value of the representative vector determined by the representative vector detection unit 125. That is, the electronic runout correction is controlled. For example, when image shake for three upward pixels is generated due to camera shake, the representative vector detection unit 125 determines a motion vector corresponding to the three upward pixels as a representative vector. The image deformation amount calculation unit 129 moves the image cutout position by three pixels in the downward direction. As a result, it is possible to absorb the positional deviation of the subject image corresponding to the above-mentioned three upward pixels and correct the image shake. If the image shake occurs in the left or right direction due to the camera shake, the image shake can be corrected by moving the image cutout position in the opposite direction.

If the maximum electronic correction amount is obtained from the electronic runout correction limiter 128 and the amount of movement to the determined image cutout position exceeds the maximum electronic correction amount, the amount of movement of the image cutout position is limited by the maximum electronic correction amount. do.

The image deformation unit (correction means) 124 cuts out an output image from the frame image according to the image cutout position determined by the image deformation amount calculation unit 129. The output image cut out by the image transforming unit 124 is stored in a recording medium 126 such as a semiconductor memory or a hard disk, or displayed on a display device such as a liquid crystal element.

Note that the rotation image shake that occurs when the camera shake occurs in the rotation direction around the optical axis may be corrected by electronic shake correction.

Next, a method for detecting the motion vector and a method for determining the representative vector will be described with reference to FIGS. 2 (A) and 2 (B) and FIGS. 3 (A) to 3 (C). 2 (A) and 2 (B) and FIGS. 3 (A) to 3 (C) show a case where image pickup is performed without vibration correction. Here, a case where horizontal camera shake occurs in actual imaging will be described, but the same applies to a case where vertical camera shake occurs.

FIG. 2A shows the amount of camera shake due to camera shake as a fixed frequency sine wave, and the shake angle as θ. FIG. 2B shows the positional relationship between a person who is a moving subject, a building which is a background, and a camera system. The positive and negative directions of the runout angle θ in FIG. 2A show the camera shake from the center to the right and the camera shake to the left in FIG. 2B, respectively. Further, FIG. 2B shows a state in which a person is moving to the right from time T0 to T1 also shown in FIG. 2A. The amount of camera shake from the center of the camera system at time T0 is 0.

FIG. 3A shows a frame image obtained by imaging at time T0 and T1 in FIGS. 2A and 2B. FIG. 3B shows a motion vector calculated from the two frame images shown in FIG. 3A. The block matching method described above is used for motion vector detection, and the reference numerals B01 to B09 indicate nine blocks, respectively. The number of blocks may be other than nine. V01 to V09 indicate motion vectors output from the motion vector detection unit 123 for each block. The motion vector detected for a moving person is indicated by Va, and the motion vector detected for a building is indicated by Vb. The direction of the motion vector is the direction from the specific block at time T0 to the similar block at time T1.

Since the motion vectors V01, V02, V05, and V08 cannot be detected, the arrow indicating the motion vector is not described. For example, when the contrast of the subject is low or the subject has a repeating pattern, the motion vector cannot be detected because it is difficult to associate the specific block with the similar block at the time T0 and T1. If the motion vector cannot be detected, a detection error may be notified.

FIG. 3C shows a histogram generated from the detected motion vector. The horizontal axis shows the value of the motion vector, and the vertical axis shows the number of detections of the same motion vector. As shown in FIG. 3B, since the motion vector Va is detected in two blocks and the motion vector Vb is detected in three blocks, the number of detected motion vectors Va is 2 and the number of detected motion vectors Vb is 2. It is 3.

In this histogram, there is a method of determining the motion vector with the largest number of detections as the representative vector. This is because the ratio of the area occupied by the background is often larger than the ratio of the area occupied by a moving subject such as a person in an image obtained by general imaging, so a motion vector with a large number of detections is used for the background. This is because it can be inferred that it will be detected. However, in the method of determining the motion vector having the maximum number of detections as the representative vector, the representative vector may be erroneously determined as the motion vector of the moving subject area when the moving subject area is larger than the background area.

Therefore, in this embodiment, the representative vector is determined by the method described below. Here, as an example, the determination of the representative vector in the case where the moving subject area and the background area in the image have the same size as each other and the shake correction is shared between the lens 100 and the camera 120 as described above. The method will be described with reference to FIGS. 4 (A) and 4 (B) and FIGS. 5 (A) to 5 (C). 4 (A) and 4 (B) are diagrams corresponding to FIGS. 2 (A) and 2 (B), and FIGS. 5 (A) to 5 (C) are diagrams corresponding to FIGS. 3 (A) to 3 (C). be. 4 (A) and 4 (B) and FIGS. 5 (A) to 5 (C) show a case where optical shake correction is performed by the lens 100 and imaging is performed.

FIG. 4A shows the camera shake amount due to the camera shake shown in FIG. 2A by a solid line, and the optical correction amount (angle) by the correction lens 110 is shown by a dotted line. In the dotted line, the maximum optical correction amount and the maximum electronic correction amount are the same as each other, and the optics when the division unit 104 determines the correction sharing ratio of the optical shake correction and the electronic shake correction with respect to the camera shake amount by θ / 2. The amount of correction is shown.

FIG. 4B shows the positional relationship between the moving subject (person), the background (building), and the camera system, as in FIG. 2B. In FIG. 4B, the moving subject moves closer to the camera system than the positional relationship shown in FIG. 2B so that the background area and the moving subject area have the same size as described above. ing.

5 (A) shows the frame images obtained by the imaging at the time T0 and T1 of FIGS. 4 (A) and 4 (B), respectively. In these frame images, the background area and the moving subject area have the same size. Due to the optical shake correction, the amount of shake of the subject image formed on the image sensor 121 is suppressed to θ / 2, and as a result, the amount of movement of the building on the frame image is reduced.

FIG. 5B shows motion vectors of the nine blocks B01 to B09 calculated from the two frame images shown in FIG. 5A, similarly to FIG. 3B. The motion vectors V01, V04, and V07 are motion vectors Vc detected for the moving subject, and the motion vectors V03, V06, and V09 are motion vectors Vd detected for the background.

FIG. 5C shows a histogram generated from the detected motion vector, as in FIG. 3C. Since the proportion of the moving subject in the frame image is larger than that shown in FIGS. 3A to 3C, the number of detected motion vectors Vc is 3, which is the same as the number of detected motion vectors Vd. It has become. In this case, the method of determining the motion vector having the maximum number of detections as the representative vector cannot determine one representative vector. Further, when the detected number of the motion vector Vc exceeds the detected number of the motion vector Vd, the representative vector is erroneously determined. If the image cropping position is changed based on the erroneously determined representative vector, a moving image in which the imaging angle of view changes with the movement of the subject can be obtained.

Conventionally, various methods for avoiding erroneous determination of the representative vector in an imaging scene in which the motion vector for a moving subject is the maximum number of detections have been proposed. For example, a method of estimating a motion vector in a background region from a tendency of a past motion vector, a method of estimating a person or an animal as a subject from a feature amount in an image, and a method of specifying a motion vector in the background region have been proposed. There is. Although these conventional methods are not used in this embodiment, the method of this embodiment described below and the conventional method may be used in combination to avoid erroneous determination of the representative vector.

In this embodiment, a method of calculating the representative vector using the electronic correction amount acquired from the lens 110 will be described. First, the relationship between the amount of optical correction by the correction lens 110 and the amount of movement of the subject between the frame images will be described with reference to FIGS. 6A to 6C. 6 (A) to 6 (C) show the camera system (correction lens 110 and the image pickup element 121) seen from above in the actual image pickup, and the tilt in the vertical direction in each figure is the horizontal direction in the actual image pickup. Shows the slope of. For example, in FIG. 6B, the camera system is tilted downward by an angle θ in the figure, which is the same as the tilt in the negative direction in FIG. 4B. Further, FIGS. 6A, 6B and 6C show the relationship between the camera system and the subject on the left side, and the frame image obtained by imaging on the right side, respectively. The subject does not move.

FIG. 6A shows a state in which the camera shake amount of the camera system is 0, and corresponds to the state at time T0 described above. When the camera shake at an angle θ is applied to the camera system from this state, the subject image is moved by the movement amount d in the frame image, that is, on the image sensor 121 as shown in FIG. 6 (B). The amount of movement d of the subject image on the image sensor 121 is represented by d = f × tan θ, where f is the focal length of the image pickup optical system.

FIG. 6C shows a state in which optical shake correction is performed by the correction lens 110 when camera shake at an angle θ occurs in the camera system. As described above, since the correction sharing ratio of the optical shake correction is θ / 2, the movement amount d'= f × tan (θ / 2) of the subject image on the image sensor 121 is generated. The movement of the subject image corresponding to the movement amount d'is detected as a motion vector between the frame images. That is, the electronic correction amount transmitted from the lens 100 to the camera 120 and the motion vector detected between the frame images can be associated with each other.

Based on the above description, FIG. 7 shows a method in which the representative vector detection unit 125 determines the motion vector Vd detected for the background as the representative vector instead of the motion vector Vc detected for the moving subject. Will be described using. FIG. 7 is a histogram showing the same number of detected motion vectors Vc and Vd (3, respectively) as in FIG. 5 (C).

Here, as described with reference to FIG. 6C, when the camera shake at an angle θ is added to the camera system and the image shake for an angle of θ / 2 is corrected by the correction lens 110, the pixels of the lens 100 The correction sharing ratio of the electronic runout correction output from the unit conversion unit 112 is represented by d'/ g using the movement amount d'of the subject image and the pixel pitch g of the image pickup device 121. In the following description, d'/ g is referred to as an electronic correction amount E. The unit of the electronic correction amount E is the number of pixels.

When the electronic correction amount E is shown on the histogram of FIG. 7, the electronic correction amount E is a value near the motion vector Vd. This is due to the following reasons. The electronic correction amount E corresponds to an image shake component that is not corrected by the optical shake correction, appears as a movement amount of the subject image on the image sensor 121, and is detected as a motion vector. Since the movement of the subject image on the image sensor 121 is caused only by the camera shake, the detected motion vector can be regarded as equivalent to the motion vector of the background, and E≈Vd is established.

By determining the maximum number of detected motion vectors as the representative vector from the motion vectors having almost the same value as the electronic correction amount E in this way, the background motion vector can be specified without being affected by the moving subject. ..

As shown in FIG. 7, it is preferable to determine the representative vector from the motion vector whose value is included in the predetermined range E ± th including the electronic correction amount E. In other words, it is preferable to determine the representative vector from the motion vector whose difference with respect to the electronic correction amount E is a predetermined value th or less. There is a possibility that a motion vector with an electronic correction amount E or more determined by the dividing unit 104 may be detected in consideration of the shake correction residue due to the tracking performance of the optical shake correction by the correction lens 110 and the detection error by the shake detecting unit 101. Therefore, ± th is provided. th is a predetermined number of pixels. As a result, even if the detected motion vector is larger than the electronic correction amount E by a few pixels of a predetermined number of pixels or less, the motion vector detected by the representative vector detection unit 125 does not deviate from the candidates of the representative vector.

The flowchart of FIG. 8 shows a process in which the representative vector detection unit 125 determines the representative vector. The representative vector detection unit 125, which is composed of a computer together with the motion vector detection unit 123, the image deformation amount calculation unit 129, the image deformation unit 124, and the like, executes this process according to a computer program. The execution cycle of this process is synchronized with the frame rate cycle of the moving image. As a result, the execution timing of the representative vector determination process is after the motion vector detection in all the blocks in the frame image is completed.

First, in step S100, the representative vector detection unit 125 acquires the electronic correction amount E from the lens 100 via the camera communication processing unit 127.

Next, in step S101, the representative vector detection unit 125 determines whether or not the electronic correction amount E is 0. When the electronic correction amount E becomes 0, the camera shake amount detected by the shake detection unit 101 in the lens 100 is 0, or the user intentionally turns off the shake correction function of the camera system. When the electronic correction amount E is 0, the representative vector detection unit 125 proceeds to step S107 to set the value of the representative vector to 0, and then proceeds to step S108. If the electronic correction amount E is not 0, the representative vector detection unit 125 proceeds to step S102.

In step S102, the representative vector detection unit 125 acquires motion vectors of all blocks between frame images.

Next, in step S103, the representative vector detection unit 125 generates a histogram of the number of detected motion vectors as shown in FIG. 7 from the acquired motion vectors of all the blocks.
Next, in step S104, the representative vector detection unit 125 determines whether or not a motion vector having a value within the range of the electronic correction amount E ± th exists in the generated histogram. If such a motion vector exists, the representative vector detection unit 125 proceeds to step S105, and if such a motion vector does not exist, the process proceeds to step S106.

In step S105, the representative vector detection unit 125 determines the value of the motion vector having the maximum detection number within the range of the electronic correction amount E ± th as the value of the representative vector. Then, the process proceeds to step S108.

In step S106, the representative vector detection unit 125 sets the value of the representative vector to the electronic correction amount E. When a motion vector having a value within the range of E ± th is not detected, it means that the motion vector cannot be detected due to low contrast, or the moving subject is captured in the entire frame image. In such a case, even if the motion vector is detected, there is no reliability. Therefore, it is better to set the image cutout position based on the electronic correction amount E acquired from the lens 100, so that the electronic shake correction can be performed more accurately. Therefore, in step S106, the electronic correction amount E is used as the value of the representative vector. After that, the process proceeds to step S108.

In step S108, the representative vector detection unit 125 outputs the value of the representative vector determined in steps S105, S106, or S107 to the image deformation amount calculation unit 129. As a result, the electron runout correction by the image deformation unit 124 is controlled according to the representative vector. After this, the representative vector detection unit 125 ends this process.

As described above, in the present embodiment, in the camera system in which the lens 100 and the camera 120 cooperate to perform optical shake correction and electronic shake correction, the electronic correction amount calculated by the lens 100 is used as it is and electronically. Compared with the case of performing runout correction, image shake including the balance of runout correction by optical shake correction can be corrected by electronic runout correction. This makes it possible to enhance the shake correction effect of the camera system as a whole.

In the above embodiment, the case where the lens 100 is provided with the shake detection unit 101, the division unit 104, and the pixel unit conversion unit 112 has been described, but these may be provided in the camera. When the camera is provided with a division unit and a pixel unit conversion unit, the camera acquires information on the maximum optical correction amount from the lens, and the division unit (acquisition means) determines the correction sharing ratio from the maximum optical correction amount and the maximum electronic correction amount. After determining, the optical correction amount and the electronic correction amount are calculated (acquired), and the optical correction amount is transmitted to the lens 100 to perform optical runout correction. Also in this configuration, as in the above embodiment, the representative vector is determined by using the electronic correction amount calculated by the camera from the correction sharing ratio.

Further, in the above embodiment, the case where the electronic shake correction for moving the image cutout position is performed in the camera 120 has been described, but the shake correction may be performed optically by moving the image sensor in the direction orthogonal to the optical axis. .. In this case, the amount of movement of the image sensor may be set according to the representative vector. The representative vector corresponding to each frame may be accumulated and the representative vector of the corresponding frame may be predicted from the transition, or a sensor capable of acquiring a motion vector during exposure for acquiring an image may be used.

In addition to electronic image stabilization that moves the image cutout position for each frame, electronic image stabilization is performed to generate an image for one frame by synthesizing a plurality of images whose image cutout positions have been changed according to the amount of shake. You may. In this case as well, the amount of movement of the image cutout position may be set according to the representative vector. In addition, camera shake correction may be performed to generate an image for one frame by synthesizing a plurality of images that have been shake-corrected by moving the image sensor.
(Other examples)
The present invention supplies a program that realizes one or more functions of the above-described embodiment to a system or device via a network or storage medium, and one or more processors in the computer of the system or device reads and executes the program. It can also be realized by the processing to be performed. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

Each of the above-described embodiments is only a representative example, and various modifications and changes can be made to each embodiment when the present invention is implemented.

100 Interchangeable lens 101 Shake detection unit 104 Dividing unit 110 Correction lens 112 Pixel unit conversion unit 120 Imaging device (camera)
121 Image sensor 123 Motion vector detection unit 124 Image deformation unit 125 Representative vector detection unit 129 Image deformation amount calculation unit

Claims (10)

It is an image pickup device that can attach and detach an interchangeable lens that performs the first vibration correction for vibration.
A correction means for performing a second vibration correction for the vibration, and a correction means.
A vector detection means for detecting motion vectors of a plurality of regions in an image obtained by imaging, and
A representative determination means for determining a representative vector from the motion vectors of the plurality of regions,
It has a control means for controlling the second runout correction according to the representative vector.
The representative determination means is
A second correction amount is acquired according to the runout angle of the image pickup apparatus and the share of the runout correction of the second runout correction in the runout correction by both the first and second runout corrections.
An image pickup apparatus characterized in that the representative vector is determined based on the second correction amount. The representative determining means has the largest number of detected motions among the motion vectors having a value within a predetermined range including the second correction amount or the motion vectors having a value whose difference with respect to the second correction amount is equal to or less than a predetermined value. The image pickup apparatus according to claim 1, wherein the vector is determined to be the representative vector. When the control means has a value within the predetermined range or there is no motion vector whose difference is equal to or less than the predetermined value, the control means controls the second runout correction by using the second correction amount. 2. The image pickup apparatus according to claim 2. In the first runout correction, the optical element is moved.
The image pickup apparatus according to any one of claims 1 to 3, wherein in the second runout correction, a cutout position from the image is moved. In the first runout correction, the optical element is moved.
The image pickup apparatus according to any one of claims 1 to 3, wherein in the second runout correction, an image pickup device for capturing a subject image is moved. The interchangeable lens has a first correction amount and a second correction amount which are correction amounts of the first runout correction based on the share ratio between the first runout correction and the second runout correction and the angle. Get the correction amount of
The image pickup apparatus according to any one of claims 1 to 5, wherein the representative determination means acquires the second correction amount from the interchangeable lens. A detection means for detecting the angle and
Further, there is a first correction amount which is a correction amount of the first runout correction based on the angle detected by the detection means and the share ratio, and an acquisition means for acquiring the second correction amount. death,
The image pickup apparatus according to any one of claims 1 to 5, wherein the representative determination means acquires the second correction amount from the acquisition means. It can be attached to and detached from the imaging device according to any one of claims 1 to 7.
An interchangeable lens characterized by performing a first runout correction for vibration. An interchangeable lens that performs the first vibration correction for vibration can be attached and detached, and is a control method for an image pickup device that performs a second vibration correction for the vibration.
A step of detecting motion vectors of multiple regions in an image obtained by imaging, and
The step of determining the representative vector from the motion vectors of the plurality of regions,
It has a step of controlling the second runout correction according to the representative vector.
In the step of determining the representative vector,
A second correction amount is acquired according to the runout angle of the image pickup apparatus and the share ratio of the second runout correction in the runout correction by the first runout correction and the second runout correction.
A control method for an image pickup apparatus, characterized in that the representative vector is determined based on the second correction amount. An interchangeable lens that performs the first runout correction for vibration can be attached and detached, and the computer of the image pickup apparatus that performs the second runout correction for the vibration is subjected to the process according to the control method according to claim 9. A computer program characterized by letting it do.

Images