JP6659132B2 - Image blur correction device, imaging device, control method, and program - Google Patents

Description

  The present invention relates to an image blur correction device, an imaging device, a control method, and a program.

  2. Description of the Related Art There has been proposed an image blur correction apparatus that corrects blur (image blur) generated in a captured image due to camera shake by driving an optical correction unit such as a correction lens. In addition, there is a panning shot as one of the shooting methods of the camera. The follow shot is a method of shooting while causing the camera to follow the movement of the main subject moving in the horizontal direction, for example. The user slows down the shutter speed in order to give the subject a lively feeling during panning. In order to successfully shoot a subject while shaking the camera at a slow shutter speed, experience is required, and panning is a particularly difficult shooting method for beginners.

  Patent Literature 1 discloses a camera device that supports panning by detecting a vector of a subject and driving a correction lens based on the detected motion vector.

JP 2013-110754 A

  As a method of detecting a motion vector, a correlation method based on a correlation operation, a block matching method, and the like are known. For example, in the block matching method, an input image signal is divided into a plurality of detection blocks of an appropriate size (for example, 8 pixels × 8 lines). Then, a difference from a certain range of pixels in the previous frame is calculated for each detection block, and a detection block of the frame before the sum of the absolute values of the differences is minimized is searched for. Then, the relative displacement between the screens is detected as a motion vector of the detection block.

  Regarding the arrangement of the motion vector detection areas corresponding to the plurality of detection blocks, there is an appropriate arrangement depending on the application. For example, in the case of performing electronic camera shake correction by cutting out an image, it is better to detect the shake of the entire screen, and it is better to arrange the detection blocks evenly over the entire screen. On the other hand, it is assumed that panning is performed while the camera is shaken at a relatively high angular velocity (for example, 40 dps). In this case, in order to increase the detection range of the detection block, it is preferable to reduce the size of the detection block and arrange it near the center of the screen. However, when the detection block is arranged near the center of the screen, the vector of the subject cannot be detected when the subject is located outside the motion vector detection area.

  SUMMARY OF THE INVENTION It is an object of the present invention to provide an image blur correction device that can detect a vector of a subject by placing the subject in the detection region of the motion vector when the subject is not within the detection region of the motion vector.

  An image blur correction apparatus according to an embodiment of the present invention includes a correction unit that corrects an image blur based on a shake detection signal, a detection unit that detects a motion vector from a predetermined detection area in a shooting screen, Based on the motion vector and the shake detection signal, it is determined whether the subject is in the detection area.If the subject is not in the detection area, the subject is included in the detection area. Control means for controlling the correction means.

  ADVANTAGE OF THE INVENTION According to the image blur correction device of this invention, when an object is not contained in the detection area of a motion vector, the object can be contained in the detection area of a motion vector and the vector of the object can be detected.

FIG. 1 is a diagram illustrating a configuration of an imaging device according to an embodiment. FIG. 4 is a diagram illustrating a configuration example of a subject vector selection unit. It is a figure showing the frequency distribution of the vector calculated for every frame. FIG. 5 is a diagram illustrating an example of the arrangement of detection blocks. 6 is a flowchart illustrating an operation process according to a detection result of a motion vector by the digital camera. It is a figure explaining movement of a detection block.

FIG. 1 is a diagram illustrating a configuration of an imaging device according to the present embodiment.
The imaging device shown in FIG. 1 is a digital camera. In the following description, control for correcting any one of the horizontal and vertical shakes of the image will be described, and the description of the control for correcting the other shake will be omitted. Note that the present invention is also applicable to photographing devices such as surveillance cameras, Web cameras, and mobile phones.

  The digital camera 100 includes a microcomputer (μCOM) 101 to a focus information input unit 119. The μCOM 101 controls the entire digital camera 100. The μCOM 101 includes the HPF 103 to the multiplication unit 117. The angular velocity detection unit 102 detects a shake applied to the imaging device 100 as an angular velocity signal (a shake detection signal). In the example shown in FIG. 1, the angular velocity detection unit 102 is an angular velocity sensor. The signal output from the angular velocity detection unit 102 is supplied to a high-pass filter (hereinafter, HPF) 103 inside the μCOM 101 as angular velocity data.

  The HPF 103 has a function of changing its characteristics in an arbitrary frequency band, and outputs a signal in a high frequency band after blocking a low frequency component included in the angular velocity data. Note that the HPF 103 may be configured to subtract a signal that has passed through a filter (LPF) from the output of the angular velocity detection unit 102. This filter blocks a signal in a high frequency band with respect to the output of the angular velocity detection unit 102.

  The focal length calculation unit 104 calculates the focal length of the imaging optical system (not shown) based on the zoom information input by the zoom information input unit 118, and sets the HPF 103 to an optimal value for driving the shake correction system. Correct the output of. The shake correction system is, for example, a shift lens. The μCOM 101 corrects a shake (image blur) of a captured image by driving a shake correction system based on a shake detection signal. The shake correction system polarizes the optical axis by being moved in a direction different from the optical axis (in this example, a direction perpendicular to the optical axis). Accordingly, an image in which the movement of the subject on the imaging surface caused by the shake of the apparatus is corrected is formed on the imaging device. In the present embodiment, the shake correction system is also used to support panning, which captures an image while tracking a subject.

  The integrator 105 has a function of changing its characteristics in an arbitrary frequency band, and integrates an output from the focal length calculation unit 104 to calculate a drive amount of the shake correction system. The adder 106 adds the driving amount of the shake correction system and the moving amount of the subject, which will be described later. The shake correction system position detection unit 112 detects the position of the shake correction system. The shake correction system position detection unit 112 has a magnet and a Hall sensor provided at a position facing the magnet.

  The A / D converter 113 performs A (analog) / D (digital) conversion on the output of the shake correction system position detection unit 112 and outputs digitized data. The subtracter 107 subtracts the output of the A / D converter 113 from the output of the adder 106 and supplies the result to the control filter 108.

  The control filter 108 includes an amplifier that amplifies input data with a predetermined gain, and a phase compensation filter. The deviation data supplied from the subtracter 107 is output to a pulse width modulation unit 109 after signal processing is performed by an amplifier and a phase compensation filter in a control filter 108.

  The pulse width modulator 109 modulates the data supplied through the control filter 108 into a waveform (that is, a PWM waveform) that changes the duty ratio of the pulse wave, and supplies the waveform to the motor driver 110. The motor 111 is a voice coil type motor for driving a shake correction system, and is driven by the motor driving unit 110. Thus, the shake correction system is moved in a direction perpendicular to the optical axis.

  The motion vector detection unit 114 uses a block matching method based on the luminance signal included in the current video signal generated by the signal processing unit (not shown) and the luminance signal included in the video signal one frame before. , The motion vector of the image is detected. Specifically, the motion vector detection unit 114 divides the video signal into a plurality of detection blocks of an appropriate size, calculates a difference between each block of pixels in a certain range of the previous frame, and calculates the absolute value of the difference. The detection block of the frame before the sum of the values is minimized is searched. A range corresponding to a plurality of detection blocks is a motion vector detection area. The motion vector detection unit 114 detects a relative shift between screens as a motion vector of the detected block. That is, the motion vector detection unit 114 functions as a detection unit that detects a motion vector from a predetermined detection area in the shooting screen based on the shot image. The motion vector data detected by the motion vector detection unit 114 is supplied to a subject vector selection unit 115. The subject vector selection unit 115 selects (determines) a subject vector. The integrator 116 integrates the vector of the subject to obtain angle information. The multiplier 117 multiplies the output of the integrator 116 by a gain K and supplies the result to the adder 106.

FIG. 2 is a diagram illustrating a configuration example of a subject vector selection unit.
The subject vector selection unit 115 includes a frequency distribution processing unit 201, a subject vector weighting processing unit 202, and a subject vector speed conversion unit 203. The frequency distribution processing unit 201 performs processing based on vector information in the shooting screen output from the motion vector detection unit 114 and angular velocity information output from the angular speed detection unit 102, among all vectors detected in the shooting screen. , Select the vector of the subject. The subject vector angular velocity conversion unit 203 converts the vector of the selected subject from a pixel unit to an angular velocity unit. The integrator 116 integrates the vector converted into the unit of angular velocity and outputs the result as angle information to the multiplication unit 117. Hereinafter, the operation processing of the frequency distribution processing unit 201 will be described with reference to FIG.

FIG. 3 is a diagram showing a frequency distribution of a vector calculated for each frame.
The motion vector itself detected by the motion vector detection unit 114 cannot distinguish which vector indicates the subject or the background. The frequency distribution processing unit 201 selects a subject vector using the output value of the angular velocity detection unit 102. For example, when photographing a moving subject with the camera fixed on a tripod, the output value of the angular velocity detection unit 102 is equivalent to about 0 dps. In the screen, the moving object is only the subject, and the background portion has a value substantially equal to the output value of the angular velocity detection unit 102. Therefore, the frequency distribution processing unit 201 converts the output value of the angular velocity detection unit 102 calculated at regular intervals into a pixel unit so that it can be compared with a vector in the frequency distribution.

  The frequency distribution processing unit 201 compares the output of the motion vector detection unit 114 and the output value of the angular velocity detection unit 102 with a frequency distribution. Thereby, the frequency distribution processing unit 201 distinguishes the subject from the background. In the example illustrated in the frequency distribution illustrated in FIG. 3, the frequency distribution processing unit 201 sets the vector group 302 around the output value 303 of the angular velocity detection unit 102 as a background vector group. In this example, assuming that the camera is swung in a certain direction, the frequency distribution processing unit 201 extracts a vector group 301 apart from the output value 303 as a subject vector group. That is, based on the frequency distribution of the vector obtained based on the motion vector and the shake detection signal, the frequency distribution processing unit 201 sets a vector near the output value of the shake detection signal as a background vector and a vector other than the background vector. Is determined as the vector of the subject.

  The subject vector weighting processing unit 202 weights the subject vector group extracted by the frequency distribution processing unit 201. The subject vector weighting processing unit 202 weights the subject vector group using the focus information input from the focus information input unit 120. The focus information includes the position information of the AF frame in the screen, that is, the position information of the focus detection area.

  The photographer adjusts a portion of the subject to be stopped to the AF frame. Therefore, the subject vector weighting processing unit 202 improves the reliability by weighting the vectors detected from the detection blocks around the AF frame. The subject vector weighting processing unit 202 determines the finally output subject vector from the subject vector group using the result of the weighting. The subject vector output by the subject vector weighting processing unit 202 is converted from a pixel unit to an angular velocity unit by a subject vector angular velocity conversion 203. The integrator 116 integrates the converted vector and inputs the integrated vector to the multiplier 117 as angle information. The multiplier 117 multiplies the output of the integrator 116 by the gain K and supplies the result to the adder 106.

FIG. 4 is a diagram illustrating an example of the arrangement of the detection blocks.
In the present embodiment, the arrangement of the detection blocks is changed for camera shake and panning. When a motion vector is used for correcting image blur due to camera shake, it is desired to detect a shake of the entire screen. Therefore, as shown in FIG. 4A, the detection blocks are arranged uniformly over the entire screen. On the other hand, when panning is supported by controlling the subject so that the subject comes to a predetermined position in the screen by driving the shake correction system at the time of panning, the motion vector of the subject only is used instead of the motion vector of the entire screen. is necessary. Therefore, in this case, as shown in FIG. 4B, detection blocks are densely arranged near the center of the screen.

  In panning, the photographer pans greatly (for example, equivalent to 40 dps), and the difference between the amount of movement of the subject and the amount of camera shake is vector detected as the amount of movement of the subject. Need to be detected. Therefore, the size of the detection block is reduced, and the detection range of the motion vector detection unit can be widened. Further, it is preferable that the detection blocks are arranged in a concentrated manner at a position of the subject to be stopped. However, when the detection blocks are arranged near the center of the screen, for example, when the release button is pressed while the subject is at the edge of the screen, the subject vector cannot be detected because the subject is not within the motion vector detection area 402. The image blur correction apparatus according to the present embodiment drives and controls the shake correction system according to the detection result of the vector of the subject so that the subject falls within the detection area of the motion vector.

FIG. 5 is a flowchart illustrating an operation process according to a result of detection of a motion vector by the digital camera.
In the present embodiment, when the number of subject vectors is less than the threshold, the subject vector selection unit 115 determines that the subject is not included in the detection area, and performs a shake correction system so that the subject is included in the detection area. Drive. First, in step S501, the μCOM 101 determines whether the anti-shake SW is on. Turning on the image stabilization switch indicates that the camera shake correction setting of the camera is enabled. If the image stabilization switch is not on, the process ends. If the image stabilization switch is on, the process proceeds to step S502.

  In step S502, the motion vector detection unit 114 detects a motion vector of the entire shooting screen at each interruption cycle. Subsequently, the angular velocity detection unit 102 detects an angular shake applied to the camera at each interruption cycle.

  Next, the subject vector selection unit 115 creates a frequency distribution of the vector for each interruption cycle based on the motion vector detected in step S502 and the value of the angular shake detected in step S502. Then, the subject vector selection unit 115 extracts a subject vector based on the frequency distribution of the vector.

  Next, in step S505, the subject vector selection unit 115 determines whether the number of extracted subject vectors is equal to or greater than a preset threshold value of one. The fact that the number of subject vectors is equal to or greater than the threshold value 1 indicates that the subject is included in the motion vector detection region at a predetermined ratio or more (for example, 90% or more). Therefore, when the number of subject vectors is equal to or greater than the threshold value 1, the subject vector selection unit 115 determines that the subject is within the detection area and the subject vector has been sufficiently detected with respect to the detection block. Do not drive the shake correction system for vector detection. If the number of vectors of the subject is less than the threshold value 1, the process proceeds to step S506.

  In step S506, the subject vector selection unit 115 determines whether the number of subject vectors is equal to or greater than a preset threshold value of two. The threshold 2 is set to a number smaller than the threshold 1. The fact that the number of vectors of the subject is equal to or greater than the threshold value 2 indicates that the subject is included in the motion vector detection area by a predetermined ratio or more (for example, 10% or more). If the number of vectors of the subject is equal to or larger than the threshold value 2, the process proceeds to step S508. If the number of subject vectors is less than the threshold value 2, the subject may not be within the motion vector detection area. Therefore, in this case, the process proceeds to step S507.

  In the present embodiment, when the subject does not fall within the motion vector detection region, the μCOM 101 drives the shake correction system so that the subject falls within the motion vector detection region. However, when the shake correction system is located near the drive end away from the vicinity of the optical center, a drive amount for detecting a motion vector may not be secured. Therefore, in step S507, the μCOM 101 detects the current position of the shake correction system.

  In step S508, the subject vector selection unit 115 determines whether the number of subject vectors is equal to or greater than the preset threshold value 3 and whether there is any error determination of the subject vector. The threshold value 3 is set to a value larger than the threshold value 2 and smaller than the threshold value 1. The fact that the number of vectors of the object is equal to or greater than the threshold value 3 indicates that the object is included in the detection block by a predetermined ratio or more (for example, 50% or more). If the number of subject vectors is greater than or equal to the threshold value 3 and there is no error determination of the subject vectors, the process ends without driving the shake correction system. When the number of subject vectors is less than the threshold value 3 or when there is an error determination of the subject vector, the process proceeds to step S507.

  In step S509, the μCOM 101 determines whether the current position of the shake correction system detected in step S507 is less than a preset threshold. If the current position of the shake correction system is less than the preset threshold, the μCOM 101 determines that there is room to the drive end even if the shake correction system is driven. Then, the process proceeds to step S510. If the current position of the shake correction system is equal to or greater than a preset threshold, the μCOM 101 determines that there is no room to the drive end when driving the shake correction system. In this case, it is necessary to detect the vector of the subject by a process other than the process of driving the shake correction system. Therefore, the process proceeds to step S511.

  In step S510, the μCOM 101 calculates a drive amount of the shake correction system such that the number of vectors of the subject is equal to or greater than the threshold value 2. The μCOM 101 varies the gain K (FIG. 1) according to the position of the shake correction system. For example, in the case of being near the optical center position, the μCOM 101 sets a gain K of about 1 since there is a margin to the drive end. Then, in step S512, the μCOM 101 drives the shake correction system based on the drive amount calculated in step S510. The μCOM 101 determines the moving direction of the subject from the information of the motion vector detected for each past frame, and drives the shake correction system in a direction opposite to the determined direction. As a result, the subject can be placed near the center of the motion vector detection area, and the number of vectors of the subject can be detected.

  In step S511, the μCOM 101 moves the detection area so that the subject falls within the motion vector detection area. Specifically, the μCOM 101 moves the detection block for each frame so that the subject is included in the motion vector detection area. Hereinafter, the movement of the detection block will be described.

FIG. 6 is a diagram illustrating the movement of the detection block.
FIG. 6A shows an example of the movement of the detection block. In the example shown in FIG. 6A, the μCOM 101 translates the template of the motion vector having the detection block 601 in the screen for each frame. In other words, the μCOM 101 moves the detection area such that the center of the motion vector detection area is located at the position of the AF frame 602.

  FIG. 6B shows another example of the movement of the detection block. In the example illustrated in FIG. 6B, the μCOM 101 changes the number of detection blocks and the size of the detection blocks according to the type of the subject using the subject recognition information, and moves the detection block 603 after the change so that the subject moves. Move to enter the vector detection area. For example, when the whole body of a person is panned and captured in a screen, the μCOM 101 sets the size of the detection block to be vertically long.

  As described above, according to the image blur correction apparatus of the present embodiment, even when the subject does not enter the detection area of the motion vector and the vector of the subject cannot be detected, the shake correction system is driven to detect the motion vector. The vector of the subject can be detected by being contained in the area.

101 μCOM
115 Subject Vector Selection Unit

Claims (11)

Correction means for correcting image blur based on a shake detection signal;
Detecting means for detecting a motion vector from a predetermined detection area in the shooting screen,
Based on the detected motion vector and the shake detection signal, it is determined whether the subject is in the detection area.If the subject is not in the detection area, the subject is in the detection area. Control means for controlling the correction means so as to be accommodated therein. The control means includes:
Based on the detected motion vector and the shake detection signal, select a vector of the subject from the detected motion vector,
The image blur correction device according to claim 1, wherein it is determined whether the subject is within the detection area based on the number of vectors of the subject. The control means includes:
3. The image blur correction device according to claim 2, wherein when the number of vectors of the subject is less than a threshold, the subject is determined to be out of the detection area. 4. The control means includes:
Using a frequency distribution of a vector obtained based on the detected motion vector and the shake detection signal, a background vector is determined based on an output value of the shake detection signal, and a vector other than the background vector is set as a subject. The image blur correction device according to claim 2, wherein the vector is determined as a vector. The control unit weights a motion vector detected from a region around a focus detection region among the detection regions, and determines a vector of the subject using the result of the weighting. The image blur correction device according to any one of claims 2 to 4, wherein The control means includes:
If it is determined that the subject is not within the detection area, determine whether the current position of the correction means is less than a threshold,
The image according to any one of claims 1 to 5, wherein when the current position of the correction unit is less than a threshold, the correction unit is driven so that the subject is included in the detection area. Image stabilizer. The control means includes:
The image blur correction device according to claim 6, wherein when the position of the correction unit is equal to or more than a threshold, the object moves in the detection area so that the subject falls within the detection area. The control means includes:
The image blur correction device according to claim 7, wherein when the position of the correction unit is equal to or larger than a threshold, the detection area is moved such that the center of the detection area is located in a focus detection area. The detection means detects a motion vector of a plurality of detection blocks corresponding to the detection area,
The control unit, when the position of the correction unit is equal to or larger than a threshold, moves the plurality of detection blocks and changes the arrangement number and size of the detection blocks according to the type of the subject. The image blur correction device according to claim 7 or 8, wherein An imaging device comprising the image blur correction device according to claim 1. A control method of an image blur correction device including a correction unit that corrects image blur based on a shake detection signal,
A detection step of detecting a motion vector from a predetermined detection area in the shooting screen,
Based on the detected motion vector and the shake detection signal, it is determined whether the subject is in the detection area.If the subject is not in the detection area, the subject is in the detection area. A control step of driving the correction means so that the correction means can be settled.

Images