JP2018072540A - Image processing device, image processing method, and optical apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To solve the problem that sometimes correction using a motion vector cannot be sufficiently performed according to a photographing state in which, for example, a vibration-proof lens mechanism is positioned in the vicinity of an end of a motion area when motion vector information as remaining vibration is fed back to optical vibration correction means or there is only a few remaining area where an image cutout position can be corrected.SOLUTION: Optimal vibration correction is performed by changing an effective quantity of each vibration correction using a motion vector according to a photographing state.SELECTED DRAWING: Figure 4

Description

  The present invention relates to an image processing apparatus applied to an optical apparatus such as a digital still camera or a digital video camera. In particular, the present invention relates to an image processing apparatus applied to an optical apparatus equipped with optical image stabilization and electronic image stabilization.

  In recent years, various blur correction functions have been proposed for correcting blur of captured images caused by shakes applied to optical devices such as digital cameras and video cameras. By installing such blur correction functions in optical devices, You can shoot better.

  When detecting shake of an optical device and correcting the shake, an optical shake correction means (an anti-vibration lens and its holding member, hereinafter referred to as an anti-vibration lens mechanism) is movable so as to correct an image shake caused by the shake. There is known an optical apparatus including a shake correction device that drives the motor.

  An angular velocity sensor is often used for shake detection in the shake correction apparatus. The angular velocity sensor vibrates a vibration material such as a piezoelectric element at a constant frequency, and outputs a voltage corresponding to the Coriolis force generated by the rotational motion component as angular velocity information.

  The shake correction device integrates the obtained angular velocity information to determine the amount and direction of shake, and outputs a correction position control signal that drives the image stabilization lens mechanism so as to cancel the image shake.

  When the image stabilization lens mechanism is driven, the current position of the image stabilization lens mechanism is fed back to the image stabilization apparatus as an image stabilization lens mechanism position signal, and the image stabilization apparatus outputs a correction position control signal corresponding to the image stabilization lens mechanism position signal. Control is performed.

  In addition to a method using a sensor such as a gyro sensor, there is a method using image information such as a motion vector as a method for detecting shake applied to an optical device.

  When a motion vector is detected to detect a shake applied to the optical device, the amount of movement of the representative point is generally used as the amount of motion of the image, that is, the amount of motion of the camera, compared with an image one field (or one frame) or more before. .

  An electronic shake correction technique for correcting shake by moving the position of an image cut out for each frame based on this motion vector is widely used.

  In addition, the horizontal correction function for correcting the tilt of the captured image due to the tilt of the camera based on the posture information from the acceleration sensor or the like, and the ROLL correction of the rotation around the optical axis also rotate and move the position of the clipped image. It is done by.

  Electronic shake correction and horizontal correction are each performed by cutting out an output image from a captured image and moving or rotating the position of the output image. However, the periphery of the cut-out area is an extra pixel.

  Although the limit amount of correction is determined by the extra pixels, there is a limit amount of correction because the angle of view of the output image becomes narrower as more extra pixels are taken.

  Further, it is necessary to distribute the correction amounts of the electronic shake correction and the horizontal correction within the same extra pixel range. For example, if a large correction limit amount is assigned to the horizontal correction, the correction limit amount that can be assigned to the electronic shake correction is reduced accordingly.

  On the other hand, Patent Document 1 discloses a technique for performing stable blur correction by feeding back to optical blur correction means (anti-vibration lens mechanism) based on information including only a predetermined frequency band or less in motion vector information. ing.

Special registration No. 03621010

  However, let us consider a case where the position of the image stabilizing lens mechanism is near the end of the movable region when the motion vector information as the remaining vibration is fed back to the optical shake correcting means.

  Also, consider a case where there are few remaining areas where the image cutout position can be corrected. In that case, there are times when the respective corrections using the motion vectors cannot be sufficiently performed according to the shooting situation.

  According to the present invention, an electronic shake correction that electronically corrects shake by moving an image cut out in accordance with a motion vector, and feedback to an optical shake correction means based on the motion vector amount is used to enhance the effect of shake correction. The present invention relates to an image processing apparatus having the function.

  An object of the present invention is to perform an optimum shake correction by changing the amount of each shake correction according to the shooting situation.

In order to solve the above problems, an image processing apparatus of the present invention includes a vector calculation unit that calculates a motion vector of an image using an imaging signal output from an imaging element that images a light beam that has passed through an imaging optical system; First control means for controlling electronic correction means for electronically correcting image blur based on a first shake correction amount calculated using a motion vector; and a second control calculated using the motion vector An image processing apparatus comprising: a second control unit that controls an optical correction unit that optically corrects image blur based on a shake correction amount;
According to another aspect of the invention, there is provided a third control unit that performs control so as to change the correction coefficient of the first shake correction amount and the correction coefficient of the second shake correction amount in accordance with a shooting situation.

  According to the present invention, in shake correction using a motion vector, an electronic shake correction that electronically corrects shake by moving a cut-out image and feedback to an optical shake correction means to enhance the effect of shake correction. In an image processing apparatus applied to an optical apparatus having a plurality of correction techniques, optimal shake correction can be performed by changing the amount of each shake correction according to the shooting situation.

1 is a block diagram of an imaging apparatus according to an embodiment of the present invention. It is the schematic of shake correction using a motion vector. It is the schematic showing the shift lens position of optical shake correction. It is a flowchart figure in 1st Embodiment. It is a flowchart figure in 2nd Embodiment. It is a block diagram of the optical correction and electronic correction which concern on 1st Embodiment of this invention. It is a block diagram of the optical correction and electronic correction which concern on 2nd Embodiment of this invention. It is a block diagram of the optical correction and electronic correction which concern on 3rd Embodiment of this invention. It is a flowchart figure in 3rd Embodiment of this invention.

(First embodiment)
(Block diagram of imaging device)
FIG. 1 is a block diagram illustrating a configuration of an imaging apparatus as an optical apparatus according to the present embodiment.

  The imaging device shown in FIG. 1 is a digital camera that mainly captures still images and moving images.

  Of course, the present invention is not limited to the digital camera shown in FIG. 1, and can also be applied to an interchangeable lens type camera system.

  The digital camera has an imaging optical system that forms a subject image. The imaging optical system includes a zoom lens, an image shake correction lens, a diaphragm, a focus lens, and the like.

  The zoom unit 101 is a zoom unit that includes a zoom lens that is moved in the optical axis direction to change the focal length.

  The zoom drive control unit 102 controls the drive of the zoom unit 101. Further, a zoom position detection unit (not shown) detects a position (zoom position) on the optical axis of the zoom lens.

  The image blur correction lens unit 103 includes an image blur correction lens (shift lens) that moves in a direction orthogonal to the optical axis to correct image blur.

  Instead of the image blur correction lens, an image blur may be corrected by moving the imaging unit (imaging unit) 109 in a direction orthogonal to the optical axis.

  The “perpendicular direction” only needs to have a component orthogonal to the optical axis, and may be moved obliquely with respect to the optical axis.

  An optical image blur correction control unit 104 as a second control unit controls the movement of the image blur correction lens unit 103.

  The image blur correction lens unit 103 performs optical image blur correction that optically corrects the image blur caused by the shake detected by the shake detection unit 117 according to the drive amount controlled by the optical image blur correction control unit 104.

  That is, the camera system control unit 119 and the optical image shake correction control unit 104 function as a control unit that performs optical image shake correction that optically corrects the image shake caused by the shake detected by the shake detection unit 117.

The aperture / shutter unit 105 is a unit in which an aperture and a shutter are integrated.
The diaphragm adjusts the amount of light incident on the imaging unit 109, and the shutter controls the exposure amount by opening and closing. The aperture / shutter drive control unit 106 controls the drive of the aperture / shutter unit 105.

  The focus unit 107 includes a focus lens that is moved in the optical axis direction and performs focus adjustment. The focus drive control unit 108 controls the drive of the focus unit 107.

  The imaging unit (imaging unit) 109 includes an imaging element (photoelectric conversion unit) that converts a subject image formed by an imaging optical system such as a CMOS sensor into an electrical signal.

  The imaging element images the light flux that has passed through the imaging optical system.

  The imaging signal processing unit 110 converts the electrical signal output from the imaging unit 109 into a video signal. The video signal processing unit (video signal processing means) 111 processes the video signal output from the imaging signal processing unit 110 according to the application.

  The display unit 112 performs image display as necessary based on the signal output from the video signal processing unit 111.

  The power supply unit 113 supplies power to the entire system according to the application. The external input / output terminal unit 114 inputs / outputs communication signals and video signals to / from the outside.

  The operation unit 115 is a processing unit used for operating the system. The storage unit 116 stores various data such as video information.

  The shake detection unit 117 detects a shake (amount of shake) of the imaging apparatus, and detects shakes in the pitch direction, the yaw direction, and the roll direction among the shake components. As a specific example, an angular velocity sensor is used.

  An electronic image blur correction control unit 118 as a first control unit controls the video signal processing unit 111 to electronically correct image blur due to the shake detected by the shake detection unit 117. Determine the correction amount.

  Further, the motion amount (motion vector amount) of the main subject is calculated from the continuous images obtained from the video signal processing unit 111 and electronically corrected.

  Further, the calculated amount of motion is notified to the optical image shake correction control unit 104, and the amount of shake remaining is reduced by adding the optical shake correction amount, thereby further enhancing the shake correction effect. The details of the shake correction using these motion amounts will be described later.

  The correction amount set by the electronic image blur correction control unit 118 is set as an edge (margin) having a predetermined width around the captured image.

  The margin of the captured image is used for these electronic corrections and is excluded from the finally obtained captured image. Therefore, if the margin is increased, the shooting angle of view decreases accordingly.

  For this reason, the margin has a limit value (allowable value). For example, when electronic image blur correction is performed, a margin is set in the captured image, and the captured image after the electronic image blur correction is an image obtained by removing the margin from the first captured image.

  The camera system control unit 119 is a control unit that controls the entire camera system, and includes a microcomputer as an image processing unit.

  In the present embodiment, the camera system control unit 119 and the image capturing unit 109 function as an image capturing unit that captures an image using an exposure method with different exposure timings for each pixel line.

  The optical image blur correction control unit 104 and the electronic image blur correction control unit 118 may be part of the camera system control unit 119 or may be independent members.

  The operation unit 115 includes an image blur correction switch that enables selection of an image blur correction mode.

  When the image blur correction mode is selected by the image blur correction switch, the camera system control unit 119 instructs the optical image blur correction control unit 104 and the electronic image blur correction control unit 118 to perform an image blur correction operation.

  Upon receiving this instruction, the optical image blur correction control unit 104 and the electronic image blur correction control unit 118 perform an image blur correction operation until an instruction is given to turn off image blur correction. Validity and invalidity of optical image blur correction and electronic image blur correction can be individually set.

  The operation unit 115 includes a shooting mode selection switch that allows a shooting mode to be selected from a still image shooting mode and a moving image shooting mode.

  The shooting mode is selected by operating the shooting mode selection switch.

  Then, the operating conditions of the actuators of the zoom unit 101, the optical image shake correction control unit 104, the aperture / shutter unit 105, and the focus unit 107 are changed according to the selected shooting mode.

  In addition, the operation unit 115 includes a shutter release button configured such that the first switch (SW1) and the second switch (SW2) are sequentially turned on in accordance with the pressing amount.

  When the shutter release button is depressed approximately half, the switch SW1 is turned on, and when the shutter release button is depressed to the end, the switch SW2 is turned on.

  When the switch SW1 is turned on, the focus drive control unit 108 drives the focus unit 107 to perform focus adjustment, and the aperture / shutter drive control unit 106 drives the aperture / shutter 105 to obtain an appropriate exposure amount. Set.

  When the switch SW2 is turned on, image data obtained from the light image exposed to the imaging unit 109 is stored in the storage unit 116.

  The operation unit 115 has a moving image recording switch. When the moving image recording switch is pushed down, moving image shooting starts, and when the switch is pressed again during recording, the recording ends.

  Note that even during moving image shooting, the user can perform still image shooting by pressing the shutter release button.

  The operation unit 115 has a reproduction mode selection switch for selecting a reproduction mode. When the playback mode is selected by the playback mode selection switch, the camera system control unit 119 stops image blur correction.

  At this time, the energization to the actuator of the image blur correction lens unit 103 may be turned off, or the actuator may be controlled to be energized and fixed at a predetermined position.

  The operation unit 115 has a zooming switch for instructing zoom zooming.

  When an instruction for zooming magnification is input through the zooming switch, the zoom drive control unit 102 that has received this instruction via the camera system control unit 119 drives the zoom unit 101 to zoom to the instructed zoom position. The unit 101 is moved.

  Further, based on the image information processed by each signal processing unit (110, 111) sent from the imaging unit 109, the focus drive control unit 108 drives the focus unit 107 to perform focus adjustment.

  The control method of the present embodiment is realized by the function of each image processing unit included in the imaging apparatus illustrated in FIG.

(Explanation of shake correction by motion vector)
FIG. 2 is a diagram for explaining shake correction based on a motion amount (motion vector).

  In the continuous shooting such as moving image shooting, the previous and next images (frames) are compared, and the motion vector of the image is obtained by the movement amount of the representative point.

  Here, the motion vector represents the remaining vibration when performing optical hand shake correction. The effect of shake correction can be further increased by moving the position of the image cut out for each frame based on this amount of motion (electronic shake correction using a motion vector).

  In addition, as a shake correction using motion vectors, only low frequency components are extracted from continuous data of motion vectors obtained between frames, and the amount of optical hand shake correction is increased according to the amount, thereby improving the effect of shake correction. Can be strengthened. That is, the motion vector is fed back to the optical shake correction.

  These can be used singly or both can be used simultaneously. When both are used at the same time, first, the motion vector obtained in each frame is used to move and correct the shake by moving the cut out image so that there is no remaining shake in each frame by electronic shake correction.

  At the same time, the correction gain of optical shake correction corresponding to the low-frequency component amount of the continuous motion vector is increased, thereby further correcting low-frequency shake with a large shake amount.

  Thereby, the motion vector (remaining shaking) calculated after that becomes small. Here, by limiting to only the low frequency component, even if a phase delay due to the calculation occurs, the influence on the shake correction is reduced.

  By using the motion vector obtained as the remaining shake in this way, the shake correction effect can be strengthened in each of the electronic shake correction and the optical shake correction.

(Explanation of another form of shake correction using motion vectors)
However, when the motion vector is used for each shake correction, it may be better to reduce the amount of correction depending on the shooting situation. An example is shown below.

  FIG. 3 shows a schematic diagram of a shift lens position and a movable range used for optical shake correction.

  When the optical shake correction is performed, when the shift lens is near the center of the movable range as shown in (a), there is a sufficient margin for the movable range.

  In FIG. 3, x (cross) is the correction center of the movable range.

  However, when the correction lens is not near the center but near the end of the movable range (circular solid line in the figure) as shown in (b) (outside the circular dotted line in the figure), the motion vector is fed back to the optical shake correction. If the correction amount is increased, the movable end is easily hit. For this reason, steep changes in the angle of view due to edge contact, etc., occur, resulting in poor appearance.

  In such a case, the correction effect is reduced by reducing the correction gain addition amount of the optical shake correction by the motion vector so that the edge contact does not occur, and at the same time, the correction effect of the electronic shake correction using the motion vector is strengthened. Is good.

  Here, as a method of strengthening the correction effect, there is a method of increasing a correction gain of electronic correction for a motion vector or increasing an upper limit of a correction amount.

  The state of these image processes is shown in FIG. FIG. 6 is a block diagram of the calculation of the shake correction component for optical correction and the calculation of the electronic correction amount using the motion vector acquired by the electronic image shake correction control unit 118.

  In 601, the gyro signal detected in the shake detection 117 is processed to calculate a target.

  Here, target calculation is performed by removing an offset component using HPF or calculating an angle using an integration filter.

  Also, the shake correction target amount is calculated by multiplying a coefficient according to the focal length.

  At 602, shift lens control is performed using the current shift lens position and the target amount calculated at 601.

  Here, feedback control by PID control using mainly the difference (deviation) between the shift lens position and the target value is performed.

  Next, in 603, driver output is performed using the value calculated in 602, and the shift lens is driven.

  On the other hand, the motion vector calculated by the video signal processing unit 111 is acquired by the electronic image blur correction control unit 118 (604).

  Since it is desired to perform image blur correction using only the low-frequency component of the motion vector, filter processing is performed at 605.

  Here, a low-pass filter (LPF) or the like is used as a filter, or a phase advance filter (PLF) or a phase delay filter (PDF) is used for phase adjustment.

Next, the shift lens position information is acquired from the optical shake correction control unit 104, and the gain for calculating the correction addition amount to the optical correction according to the shift lens position and the correction amount for the electronic correction are calculated. Each gain is calculated (606). The correction addition amount at that time is obtained by the following equation (1).
Optical correction addition amount = (correction amount calculated based on motion vector) × optical correction gain [%] (1)

  For example, when the shift lens position is near the center (mainly the position at the center of the optical axis), the optical correction gain is multiplied by 100% in the calculation of the optical correction addition amount.

  Here, it is possible to prevent the movable end from being hit by reducing the gain when the shift lens position is more than a predetermined distance from the center.

  For example, in the movable range from the center to the movable end, the optical correction gain is set to 50% when the distance is more than 70% away from the center, and the optical correction gain is set to 100% when closer than 70%.

  That is, the feedback to the optical correction using the motion vector is optimally performed by setting the optical correction gain according to the distance from the center (or the shift lens position).

  Also, if you want to enhance the optical correction, you can increase the followability of the low range by setting the optical correction gain to 120%, but if you increase it too much, it will be overcorrected. .

  As another gain setting method, when a PLF or PDF is used in the filter processing 605, the cutoff frequency of the PDF or PLF may be changed.

  The suppression effect can be enhanced by setting the cutoff frequency of the filter so that the low-frequency gain is increased.

  The calculated optical correction addition amount is added to the target calculation 601 of the optical image blur correction control unit 104, and the total value is passed to the lens control 602.

  By performing these processes, the unbalance after the optical shake correction appears as a motion vector, and the addition amount is calculated so as to perform the optical correction that cancels the motion vector amount.

  At this time, in situations where optical correction is intensified close to the movable end, adjusting the gain as the correction coefficient to reduce the amount of addition can prevent edge contact and perform smoother shake correction Can do.

  On the other hand, similarly, in the gain calculation 606, an electronic correction gain to be multiplied with the electronic correction amount corresponding to the motion vector is also calculated.

  As described above, extra pixels are shared by electronic shake correction and horizontal correction, so when performing horizontal correction, the electronic correction is suppressed and the optical correction is strengthened, but details regarding the calculation of the electronic correction gain at that time are detailed below. I will mention it later.

The calculation of the electronic correction amount when the electronic correction amount is calculated using the electronic correction gain calculated in the gain calculation 606 (607) is expressed by Expression (2).
Electronic correction amount = (correction amount calculated based on motion vector) × electronic correction gain [%] (2)

  The calculated electronic correction amount is transferred to the video signal processing unit 111, and the shake correction is performed by moving the cut angle of view by the correction amount.

  Here, it has been described that the gain correction 606 changes the optical correction gain and the electronic correction gain according to the situation. However, the correction amount may be reduced by limiting the correction amount after the correction amount is calculated.

  Specifically, when it is desired to reduce the optical correction addition amount, the gain is calculated as it is, and the correction amount calculated in the optical correction addition amount calculation 608 is clamped at a constant value.

  Thereby, the addition amount can be reduced. The same idea can be applied to the electronic correction amount calculation 607.

(Mode to change the gain of the correction amount according to the shift lens position)
This is shown in the flowchart of FIG. When performing shake correction using a motion vector, first, in S101, a motion vector is obtained from the difference between main subjects of consecutive captured images.

  Next, it is determined whether or not the shift lens for optical shake correction is within a predetermined range of the movable range. If the shift lens is within the movable range, the process proceeds to S103, and the correction addition amount to the optical shake correction based on the motion vector amount is increased.

  Here, as a method of increasing the correction addition amount, as described above, when the optical correction gain is applied to the correction addition amount based on the motion vector amount, the gain is increased (for example, the optical correction gain is set to 100%). And). Further, instead of setting the correction gain, the correction amount may be reduced by reducing the correction amount limit value.

  In S102, when the shift lens position is outside the predetermined range and near the movable end, the process proceeds to S104 to reduce the correction addition amount to the optical shake correction.

  Here, in order to reduce the correction addition amount, the optical correction gain to the addition amount is reduced (for example, the optical correction gain is set to 50%).

  In step S105, the amount of electronic correction is increased.

  As a method of increasing the correction amount, an electronic correction gain as a correction coefficient is increased (for example, the electronic correction gain is 120%), or a limit value of the correction amount is increased.

  Then, in S106, it is determined whether or not the shake correction by the motion vector is finished, such as when the shake correction mode is turned off by the operation unit 115.

  And when it complete | finishes, these processes are complete | finished. In the case of continuing to perform shake correction using a motion vector, the process returns to S101 and repeats these processes.

  As described above, according to the present embodiment, a case where the shift lens used for optical shake correction is outside the predetermined range and near the end is considered.

  In that case, even if the optical shake correction gain is reduced to increase the remaining shake to prevent deterioration of the appearance due to edge contact, the image after shake correction can be stabilized by increasing the correction amount of electronic correction. .

  In the above description, the determination is made based on whether or not the shift lens is within a predetermined range. However, the correction gain as the correction coefficient may be changed stepwise according to the distance from the center (or the shift lens position).

(Second Embodiment)
Next, a second embodiment of the present invention will be described using the block diagram of FIG. 7 and the flowchart of FIG.

  Regarding the second embodiment, the description of the same functional part as that of the first embodiment will be simplified, and only the different part will be described in detail.

(Mode to change the correction amount ratio according to the shutter speed)
Although the overall processing in FIG. 7 is the same as that described in the first embodiment, the shutter speed information is acquired from the camera system control unit 119 by the gain calculation 606, and the optical correction gain and the electronic correction are calculated based on the values. The part for calculating the gain is different.

  Specifically, when the shutter speed is equal to or lower than a predetermined value (exposure time is long), the optical correction gain is increased and the electronic correction gain is decreased.

  This is because if the shutter speed is less than the predetermined value, the remaining exposure time (accumulation blur) due to the longer exposure time will affect the live view display and the recorded video will be blurred. This is to prevent it.

  By increasing the added amount of optical shake correction, shake during exposure can be corrected firmly, and by reducing accumulated blur, live view display and moving images with good visibility can be taken.

  If the shutter speed is higher than the specified value (exposure time is short), the optical correction gain is lowered and the electronic correction gain is increased to weaken the optical correction and make it difficult for edge contact to occur. Correct with.

  At this time, the amount of electronic correction increases, but the effect of accumulated blur is reduced because the shutter speed is large and the exposure time is short.

  These processes are shown in the flowchart of FIG.

  First, in S201, a motion vector is obtained from the difference between main subjects of consecutive captured images.

  After calculating the motion vector, in S202, it is determined whether the shutter speed during continuous shooting in live view display or moving image shooting is equal to or lower than a predetermined value (for example, slower than 1/8 second).

  If the shutter speed is equal to or lower than the predetermined value, the process proceeds to S203, the correction addition amount for optical shake correction is increased (S203), and the correction amount for electronic correction is decreased (S204).

  Increasing the correction addition amount to the optical shake correction and decreasing the correction amount of the electronic correction are performed by changing the optical correction gain and the electronic correction gain as in the first embodiment. It can also be implemented by changing the limit value of the correction amount.

  If the shutter speed is larger than the predetermined value (faster than 1/8 second), the correction amount for optical correction is decreased (S205), and the correction amount for electronic correction is increased (S206). These are similarly implemented by changing the correction gain or limit value.

  Since the shutter speed is fast, the accumulated blurring is not so noticeable even if the correction amount of the electronic correction is increased, but the contact with the movable end can be reduced by decreasing the correction amount of the optical correction.

  Further, it is possible to suppress deterioration of optical performance such as a decrease in peripheral light amount and a decrease in resolution due to the shift lens being away from the center.

  Then, in S207, it is determined whether or not the shake correction by the motion vector is to be ended, such as when the shake correction mode is turned off by the operation unit 115.

  And when complete | finishing, these processes are complete | finished. In the case of continuing to perform shake correction using a motion vector, the process returns to S101 and repeats these processes.

(Third embodiment)
Next, a third embodiment of the present invention will be described using the block diagram of FIG. 8 and the flowchart of FIG.

  In the third embodiment, the description of the same functional parts as in the first embodiment will be simplified, and only the different parts will be described in detail. Although the overall processing in FIG. 8 is the same as that described in the first embodiment, the horizontal correction amount is obtained from the horizontal correction amount calculation 609 by the gain calculation 606, and the optical correction gain and the electronic correction are calculated based on the obtained values. The part for calculating the gain is different.

  Specifically, when horizontal tilt correction using surplus pixels is performed, the correction addition amount of optical shake correction is increased (optical correction gain is increased), and the correction amount of electronic shake is reduced (electronic Reduce the correction gain).

  The above process is shown in the flowchart of FIG.

First, in S301, a motion vector is obtained from a difference between main subjects of consecutive captured images.
After calculating the motion vector, it is determined in S302 whether the horizontal correction function is ON.

  Here, if ON. Proceeding to S303, the correction addition amount to the optical shake correction is increased (S303), and the correction amount of the electronic correction is decreased (S304).

  When the correction addition amount to the optical shake correction is increased, the correction amount of the electronic correction is reduced by changing the optical correction gain and the electronic correction gain as in the first embodiment. It can also be implemented by changing the limit value of the correction amount.

  By increasing the addition amount of the optical correction, the shake correction effect is strengthened and the remaining shake is reduced, so that the motion vector amount is reduced and electronic correction is reduced.

  By using that amount in horizontal correction, it is possible to perform shake correction and horizontal correction with optimal allocation.

  If the horizontal correction function is OFF in S302, the correction amount for optical correction is reduced (S305), and the correction amount for electronic correction is increased (S306).

  These are similarly implemented by changing a correction gain as a correction coefficient or a limit value.

  Since the horizontal correction is OFF, all of the surplus pixels can be used for electronic shake correction, and by reducing the optical correction accordingly, the edge contact can be reduced and smooth shake correction can be performed.

  In step S307, it is determined whether or not the shake correction based on the motion vector is to be ended, such as when the shake correction mode is turned off by the operation unit 115.

  And when it complete | finishes, these processes are complete | finished.

  In the case of continuing to perform shake correction using a motion vector, the process returns to S101 and repeats these processes.

  Here, the condition determination is shown as an example of ON / OFF of horizontal correction, but it may be determined whether it is greater than or equal to a predetermined horizontal correction amount.

  In this way, when the surplus pixels used for electronic shake correction are used by other functions, the optical shake correction is strengthened and the electronic shake correction is weakened, so that the shake correction using the motion vector is performed according to the shooting situation. It can be done optimally.

  As described above, according to the present embodiment, optimal shake correction can be performed by changing the amount of each of optical shake correction using motion vectors and electronic shake correction according to shooting conditions. I can do it.

(Other embodiments)
The object of the present invention can also be achieved as follows. That is, a storage medium in which a program code of software in which a procedure for realizing the functions of the above-described embodiments is described is recorded is supplied to the system or apparatus. The computer (or CPU, MPU, etc.) of the system or apparatus reads out and executes the program code stored in the storage medium.

  In this case, the program code itself read from the storage medium realizes the novel function of the present invention, and the storage medium and program storing the program code constitute the present invention.

  Examples of the storage medium for supplying the program code include a flexible disk, a hard disk, an optical disk, and a magneto-optical disk. Further, a CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD-R, magnetic tape, nonvolatile memory card, ROM, or the like can also be used.

  Further, by making the program code read by the computer executable, the functions of the above-described embodiments are realized. Furthermore, when the OS (operating system) running on the computer performs part or all of the actual processing based on the instruction of the program code, the functions of the above-described embodiments are realized by the processing. Is also included.

  Furthermore, the following cases are also included. First, the program code read from the storage medium is written in a memory provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer. Thereafter, based on the instruction of the program code, the CPU or the like provided in the function expansion board or function expansion unit performs part or all of the actual processing.

  In addition, the present invention is not limited to devices such as a digital camera, but has a built-in imaging device such as a mobile phone, personal computer (laptop type, desktop type, tablet type, etc.), game machine, or external connection. It can be applied to any device. Therefore, the “imaging device” in this specification is intended to include any optical apparatus having an imaging function.

DESCRIPTION OF SYMBOLS 101 Zoom unit 102 Zoom drive control part 103 Correction lens 104 Anti-vibration control part 109 Imaging part 110 Imaging signal processing part 111 Video signal processing part 115 Operation part 117 Shake detection part 118 Electronic image shake correction control part 119 Camera system control part

Claims (6)

Based on a vector calculation unit that calculates a motion vector of an image using an imaging signal output from an imaging element that images a light beam that has passed through the imaging optical system, and a first shake correction amount calculated using the motion vector. And a first control unit that electronically corrects an image blur and an optical correction that optically corrects the image blur based on a second blur correction amount calculated using the motion vector. An image processing apparatus having second control means for controlling the means,
Image processing comprising: third control means for controlling to change the correction coefficient of the first shake correction amount and the correction coefficient of the second shake correction amount in accordance with a shooting situation apparatus.   The third control unit is configured to reduce the gain of the second shake correction amount when the correction center of the optical correction unit is separated from a predetermined value. The correction center of the optical correction unit approaches the predetermined value. The gain of the first shake correction amount when the correction center of the optical correction unit is separated from a predetermined value is made smaller than the gain of the second shake correction amount in the case of the optical correction unit. The image processing apparatus according to claim 1, wherein the correction center is set to be larger than a gain of the first shake correction amount when the correction center is closer than the predetermined value.   The third control means uses a gain of the second shake correction amount when the shutter speed is slower than a predetermined value, and a gain of the second shake correction amount when the shutter speed is faster than the predetermined value. And the gain of the first shake correction amount when the shutter speed is slower than a predetermined value is smaller than the gain of the first shake correction amount when the shutter speed is faster than the predetermined value. The image processing apparatus according to claim 1 or 2.   The third control means sets the gain of the second shake correction amount when the horizontal tilt correction amount is larger than a predetermined value, and sets the second shake correction amount gain when the horizontal tilt correction amount is smaller than the predetermined value. The gain of the first shake correction amount when the correction amount of the horizontal tilt correction is larger than a predetermined value and the correction amount of the horizontal tilt correction is larger than the predetermined value. 4. The image processing apparatus according to claim 1, wherein the image processing apparatus is smaller than a gain of the first shake correction amount when the value is also smaller.   An optical apparatus comprising the image processing apparatus according to claim 1. Based on a vector calculation step of calculating a motion vector of an image using an imaging signal output from an imaging device that images a light beam that has passed through the imaging optical system, and a first shake correction amount calculated using the motion vector. A first control step for controlling an electronic correction unit that electronically corrects the image blur, and an optical correction for optically correcting the image blur based on a second shake correction amount calculated using the motion vector. A second control step for controlling the means, and an image processing apparatus comprising:
An image processing comprising: a third control step for controlling the correction coefficient of the first shake correction amount and the correction coefficient of the second shake correction amount in accordance with a shooting situation. Method.

Images