WO2020012960A1

WO2020012960A1 - Imaging device

Info

Publication number: WO2020012960A1
Application number: PCT/JP2019/025371
Authority: WO
Inventors: 英志三家本; 篠原　隆之
Original assignee: 株式会社ニコン
Priority date: 2018-07-09
Filing date: 2019-06-26
Publication date: 2020-01-16
Also published as: TW202017355A

Abstract

Provided is an imaging device capable of further improving vibration-damping performance in optical blur correction. This imaging device 1 has: an imaging element 3 that captures an image of a subject from an optical system and outputs a signal; and a motion vector calculation unit 41 that calculates information pertaining to a motion vector for the subject, on the basis of image data having a resolution corresponding to the focal distance of the optical system and generated on the basis of the signal.

Description

Imaging device

<< The present invention relates to an imaging device.

For example, a technique has been proposed in which motion vector information of an image is detected from a captured image, and the motion vector information is fed back to the calculation of a target drive position of the blur correction lens, thereby improving the vibration reduction performance of optical blur correction. (See Patent Document 1). Conventionally, there has been a demand for improving the detection accuracy of motion vector information.

JP-A-10-145662

An imaging device according to the present invention includes an imaging device that captures an image of a subject using an optical system and outputs a signal.
A motion vector calculating unit configured to calculate information on a motion vector of the subject based on image data having a resolution corresponding to a focal length of the optical system, which is generated based on the signal.

It is sectional drawing which shows a camera typically. FIG. 2 is a block diagram illustrating a shake correction device included in the camera. FIG. 4 is a diagram illustrating basic calculation timing of motion vector information in a motion vector calculation unit. 5 is a flowchart illustrating a flow of an operation of the shake correction apparatus. FIG. 5 is a diagram illustrating a reference value calculation unit. 5 is a detailed flowchart of a reference value correction step in FIG. (A) is a graph showing the second reference value in the Yaw direction, and (b) is a graph showing the direction of the motion vector information in the X direction. FIG. 4 is a block diagram illustrating details of generation of processing image data in a signal processing unit.

Hereinafter, embodiments of the present invention will be described with reference to the drawings and the like. FIG. 1 is a sectional view schematically showing the camera 1.
As shown in FIG. 1, in this embodiment, a three-dimensional orthogonal coordinate system is set. Specifically, an axis parallel to the optical axis of the lens barrel 1B is defined as a Z-axis (horizontal direction on the paper), an axis intersecting the Z-axis in a plane perpendicular to the Z-axis is defined as an X-axis (a depth direction on the paper), and Z An axis perpendicular to the Z axis and the X axis in a plane perpendicular to the axis is defined as a Y axis (vertical direction in the drawing). The rotation direction about the Z axis is the Roll direction, the rotation direction about the Y axis is the Yaw direction, and the rotation direction about the X axis is the Pitch direction.

(Camera 1)
In the camera 1, the camera body 1A and the camera body 1A and the lens barrel 1B are integrated. However, the present invention is not limited to this, and a camera in which the lens barrel is detachable from the camera body may be used.
In the embodiment, the camera 1 has a structure that does not include a so-called quick return mirror that changes an optical path for observing a subject inside the camera. However, the camera 1 is not limited thereto, and may be a camera that has a quick return mirror. .

(Camera body 1A)
The camera body 1A includes an image sensor 3, a recording medium 13, a storage unit 14, an operation unit 15, a release switch 17, a back liquid crystal 18, and a CPU 2. Note that the CPU 2 includes a blur correction device 100 described later.

The image sensor 3 is provided on a predetermined focal plane of the photographing optical system, and is an element that generates a signal by photoelectrically converting subject light incident through the photographing

optical systems

4, 5, and 6 of the lens barrel 1B. , CMOS and the like.
From the signal output from the image sensor 3, image data is generated by a signal processing unit 40 described later included in the CPU 2.

The image sensor 3 has focus detection pixels, and the CPU 2 performs focus detection processing by a well-known pupil division type phase difference method using pixel output data from the focus detection pixels. Alternatively, focus detection may be performed by a well-known contrast method using data output from the image sensor 3.

(4) The recording medium 13 is a medium for recording captured image data, and a memory card such as an SD card or a CF card is used.

The storage unit 14 is a memory such as an EEPROM, for example. The storage unit 14 stores information on the size of image data for motion vector calculation corresponding to the zoom position (focal length) of the photographing optical system, as described later.

The operation unit 15 can perform a zoom operation, and can perform a zoom operation via the operation unit 15 to change the zoom position of the photographing optical system.

The sensor rate of the image sensor 3 is determined based on the luminance measured based on the output of the image sensor 3 in a sensor control unit 46 described below included in the CPU 2.

The release switch 17 is a member for performing a shooting operation of the camera 1 and is a switch for a user to perform a shooting instruction operation.

The back liquid crystal 18 is a color liquid crystal display provided on the back of the camera body 1A and displaying a photographed subject image (reproduced image, live view image), operation-related information (menu), and the like.

(4) The shutter controls subject light incident on the image sensor 3 in response to a shooting instruction from the release switch 17 or the like.

The CPU 2 is a central processing unit that controls the entire camera 1 and includes a shake correction device 100 described later.

(Lens barrel 1B)
Next, the lens barrel 1B will be described. The lens barrel 1B has a photographing optical system including a zoom lens 4, a focus lens 5, a blur correction lens 6, and a zoom lens drive mechanism 7, and further includes a focus lens drive mechanism 8, a blur correction lens drive mechanism 9, and a diaphragm 10. , An aperture driving mechanism 11, an angular velocity sensor 12 (blur detection sensor), and a blur correction lens position detecting section 21.

The zoom lens 4 is a lens group that is driven by a zoom lens driving mechanism 7 such as a DC motor and changes the zoom position (focal length) by moving along the optical axis direction.
When the photographer performs a zoom operation via the operation unit 15, a lens drive amount calculation unit 39 described later included in the CPU 2 calculates the drive amount of the zoom lens 4, and drives the zoom lens 4 via the zoom lens drive mechanism 7. Change the zoom position.

The focus lens 5 is a lens group that is driven by a focus lens driving mechanism 8 such as a stepping motor, moves in the optical axis direction, and focuses.

The shake correction lens 6 is a lens group that is optically shake-driven by a shake correction lens driving mechanism 9 such as a VCM (voice coil motor) and is movable on a plane perpendicular to the optical axis.

The aperture 10 is driven by an aperture drive mechanism 11 and controls the amount of subject light passing through the photographing optical system.

The angular velocity sensor 12 is a sensor that detects an angular velocity of a camera shake (a shake output signal) generated by the camera 1. The angular velocity sensor 12 is composed of two sensors, and is a sensor such as a vibration gyro that detects the angular velocity around the X axis (Pitch) and around the Y axis (Yaw). The angular velocity sensor 12 may further include a third sensor to detect the angular velocity around the Z axis (Roll).
The angular velocity sensor 12 is also connected to a later-described motion vector calculation unit 41 included in the CPU 2, and the angular velocity detected by the angular velocity sensor 12 is sent to the motion vector calculation unit 41.

(Blur correction device 100)
FIG. 2 is a block diagram illustrating the shake correction device 100 included in the camera 1. The blur correction device 100 includes an amplification unit 31, a first A / D conversion unit 32, a second A / D conversion unit 33, a reference value calculation unit 34, a subtraction unit 43, a target position calculation unit 36, a center bias calculation unit 37, A reference value correction unit 50 and a lens drive amount calculation unit 39 are provided. The blur correction device 100 further includes a signal processing unit 40, a sensor control unit 46, and a motion vector calculation unit 41.

The amplifying unit 31 amplifies the output of the angular velocity sensor 12.

(1) The first A / D converter 32 performs A / D conversion on the output of the amplifier 31.

The reference value calculator 34 calculates a reference value (first reference value, reference value before correction) of the vibration detection signal (output of the first A / D converter 32) obtained from the angular velocity sensor 12. The reference value of the angular velocity is, for example, a vibration detection signal output from the angular velocity sensor 12 when the camera 1 (camera body 1A, lens barrel 1B) is stationary. The reference value calculation unit 34 can calculate the reference value based on the output of a low-pass filter that reduces a predetermined high-frequency component from the output of the angular velocity sensor 12, for example.

The subtraction unit 43 subtracts a reference value (second reference value, corrected reference value) obtained by correcting the first reference value calculated by the reference value calculation unit 34 from the output of the first A / D conversion unit 32.

The target position calculator 36 calculates a target position for driving the blur correction lens 6 based on the output of the angular velocity sensor 12 after the subtraction of the reference value by the subtractor 43.

The center bias calculator 37 uses the centripetal force for moving the blur correction lens 6 toward the center of its movable range based on the target position of the blur correction lens 6 calculated by the target position calculator 36 as a bias amount. Calculate. Then, the control position of the blur correction lens 6 is calculated by subtracting the calculated bias amount from the target position of the blur correction lens 6.
By performing the centering bias processing in this way, it is possible to effectively prevent the blur correction lens 6 from colliding with the hard limit, and to further improve the appearance of the captured image.

The lens driving amount calculation unit 39 calculates the driving amount of the lenses of the zoom lens 4 and the blur correction lens 6.
The drive amount calculation of the blur correction lens 6 by the lens drive amount calculation unit 39 is detected by the shake correction lens position detection unit 21 and the target position from the target position calculation unit 36, and the A / D is converted by the second A / D conversion unit 33. The drive amount of the blur correction lens 6 by the blur correction lens driving mechanism 9 is calculated from the current position of the blur correction lens 6 obtained from the converted value.
Further, when the photographer performs a zoom operation from the operation unit 15, the lens driving amount calculation unit 39 instructs the zoom lens driving mechanism 7 to drive the zoom lens 4.

(Sensor control unit 46)
The sensor control unit 46 sets the sensor rate of the image sensor 3 based on the measured luminance of the subject. For example, when the subject is dark, the exposure time for obtaining one image becomes long, so the sensor rate is set to a low value of 15 fps. Since the exposure time for obtaining one image becomes shorter as the image becomes brighter, the sensor rate is set high at 30, 60, and 120 fps.

(Signal processing unit 40)
The signal processing unit 40 performs processing such as noise processing and A / D conversion on the signal acquired by the image sensor 3 and creates recording image data to be recorded as a still image on the recording medium 13. Further, the signal processing unit 40 continuously generates processing image data having a smaller image size than the recording image data in a time-series manner based on the signal acquired by the image sensor 3.
The image data for processing includes moving image display (live view display) on the rear liquid crystal 18, calculation of a motion vector by a motion vector calculation unit 41, various calculation processes such as auto focus and automatic exposure, and wireless image processing. Used for sending. The creation of the processing image data will be described later.

(Motion vector calculator 41)
The motion vector calculation unit 41 calculates motion vector information indicating a motion (movement direction and motion amount) of an image from a plurality of processing image data (first processing image data described later) processed by the signal processing unit 40. I do. The motion vector information is represented by a signed size in the X-axis direction, the Y-axis direction, the Roll direction, or the like. Further, the motion vector information includes a detection delay time and the like.

Specifically, the motion vector calculation unit 41 compares the brightness information such as a change in the position of high brightness included in the two or more pieces of image data captured by the image sensor 3 to determine the motion direction and the motion amount of the image. Is detected, and motion vector information is calculated. In addition to luminance information, motion vector information may be calculated by image pattern matching or the like.
The motion vector information may be detected from one image, may be calculated from two separate frames, or may be calculated from three images.

(Basic calculation method of motion vector)
FIG. 3 is a diagram illustrating the operation timing of the motion vector in the motion vector operation unit 41.
The motion vector calculation unit 41 calculates a motion vector from the (n-1) th image data and the next nth image data captured at a time later than the time at which the (n-1) th image data was acquired. Compute information.
For example, when image data is sent from the image sensor 3 at 30 fps, motion vector information is obtained once every 33 ms.

here,
Time t1 is the time when the exposure of the (n-1) th image data is started.
Time t2 is an intermediate time between the time when the exposure of the (n-1) th image data is started and the time when the exposure is completed.
Time t4 is a time when exposure of the n-th image data is started.
The time t5 is a time exactly intermediate between the time when the exposure of the n-th image data is started and the time when the exposure is completed.
The time t6 is the time when the motion vector information calculated from the n-1 image data and the n image data is obtained, and the generation time of the motion vector information is the time when the motion vector information is between t5 and t2. It is reasonable to think that it is just in the middle. A detection delay time of t6-t3 occurs between the time when the motion vector information is obtained and the time when the motion vector is generated.

(Reference value correction unit 50)
The reference value corrector 50 includes a center bias remover 38, a reference value correction amount calculator 35, and a reference value subtractor 42.

In the following description, correction of the reference value in the X direction will be described. Information in the X direction of the motion vector information is referred to as motion vector information X. The correction of the reference value in the Y direction is the same as that in the X direction. In the present embodiment, the Roll direction information of the motion vector information is not received. Note that the Roll vector information of the motion vector information may be received.

(Center bias removing unit 38)
The center bias removing unit 38 subtracts the bias correction amount from the motion vector information X. The bias correction amount X is calculated from the center bias amount of the X component calculated by the center bias calculator 37 (subtracted from the target position of the blur correction lens 6).

(Reference value correction amount calculation unit 35)
Based on the motion vector information X from which the bias correction amount X has been removed by the center bias removing unit 38, the reference value correction amount calculation unit 35 calculates the reference value correction amount from the output value of the angular velocity sensor around the Y axis (Yaw). Is calculated.
In the present embodiment, only the sign of the motion vector information X is determined, and if the motion vector information X is confirmed in the minus direction, the correction amount is a certain fixed amount. If the motion vector information is confirmed in the plus direction, the correction amount is Is a certain amount of minus.

(Reference value subtraction / addition unit 42)
The reference value subtraction / addition unit 42 corrects the first reference value with the correction amount calculated by the reference value correction amount calculation unit 35 to obtain a corrected second reference value.

(Operation of the image stabilizer 100)
FIG. 4 is a flowchart showing the flow of the operation of the shake correction apparatus 100.
Step 001: After the power of the camera 1 is turned on, the shake correction apparatus 100 starts a calculation for optical image stabilization. Depending on the camera, when the release switch 17 is half-pressed or when the shake correction is turned on by an instruction unit (not shown), the shake correction apparatus 100 starts the calculation for optical image stabilization.

Step 002: The blur correction device 100 amplifies the output of the angular velocity sensor 12 by the amplifying unit 31, and then performs A / D conversion by the first A / D converting unit 32.

Step 003: In the blur correction device 100, the reference value calculation unit 34 calculates the reference value of the angular velocity based on the signal after the A / D conversion of the output of the angular velocity sensor 12 (first reference value, zero deg / s). (Equivalent value). Since the reference value of the angular velocity changes due to temperature characteristics, drift characteristics immediately after startup, and the like, for example, the stationary output of the angular velocity sensor 12 at the time of factory shipment cannot be used as the reference value.

As a method of calculating the reference value, a method of calculating a moving average for a predetermined time and a method of calculating by a LPF process are known. In the present embodiment, reference value calculation by LPF processing is used.

FIG. 5 is a diagram showing the reference value calculation unit 34 (HPF). The cutoff frequency fc of the LPF 34 is generally set to a low frequency of about 0.1 [Hz]. This is because camera shake is dominant at a frequency of about 1 to 10 [Hz]. If fc is 0.1 [Hz], the effect on the camera shake component is small, and good blur correction can be performed.

However, at the time of actual photographing, movement of a low frequency such as fine adjustment of the composition (a level at which panning cannot be detected) is added, so that the reference value calculation result may have an error. Further, when fc is low (the time constant is large) and the error once increases, there is a problem that it takes time to converge to the true value. In the present embodiment, the error of the reference value is corrected.

Step 004: When the motion vector information is updated (S004: YES), the blur correction device 100 proceeds to S005, and when the information is not updated (S004: NO), the process proceeds to S006.

Step 005: When the motion vector information is updated, the blur correction device 100 corrects the first reference value calculated in S003 in the reference value correction unit 50, and calculates a second reference value. The reference value correction step will be described later.

Step 006: The blur correction device 100 controls the blur correction lens 6 in the target position calculation unit 36 based on the first reference value obtained in S003 or the second reference value obtained in S005 and the output of the angular velocity sensor 12. Calculate the target position. At this time, the target position calculation unit 36 calculates the target position of the blur correction lens 6 in consideration of the focal length, the subject distance, the shooting magnification, and the blur correction lens characteristic information.

Step 007: The blur correction device 100 performs a center bias process to prevent the blur correction lens 6 from reaching the movable end.
There are various methods of the center bias processing, such as a method of setting a bias amount according to target position information, an HPF processing, an incomplete integration processing (in S006), and the method is not limited here.

Step 008: The blur correction device 100 calculates the lens drive amount from the difference between the target position information considering the center bias component and the blur correction lens position information in the lens drive amount calculation unit 39.

Step 009: The blur correction device 100 drives the blur correction lens 6 to the target position via the blur correction lens driving mechanism 9, and returns to S002.

(Reference value correction step)
FIG. 6 is a detailed flowchart of the reference value correction step 005 in FIG.
Step 101: The blur correction device 100 sums up all the calculated motion vector information, and proceeds to step 102.
Step 102: In the shake correction apparatus 100, the center bias removing unit 38 converts the center bias component calculated in step 007 into the same scale as the motion vector information, and proceeds to step 103.
The conversion method is calculated based on focal length, subject distance, shooting magnification, and resolution information of motion vector information.

Bias_MV = Bias_θ * f * (1 + β) / MV_pitch
Bias_MV: Center bias component (same scale as motion vector information)
Bias_θ: Center bias component (angle)
f: focal length β: shooting magnification MV_pitch: motion vector pitch size

In addition, since a difference between a plurality of captured frames is obtained for a motion vector, a delay time occurs before the motion vector is detected. Therefore, it is preferable that the center bias component also have a delay time equivalent to the motion vector information. For example, if the delay time is 3 frames at 30 [fps], the delay is about 100 [ms]. Therefore, the center bias component included in the motion vector information can be calculated more accurately by using the bias information before 100 [ms].

Step 103: The blur correction device 100 subtracts the center bias component converted in step 102 from the motion vector information in the center bias removing unit 38, and proceeds to step 104. Thereby, motion vector information based on the reference value error can be obtained.

Step 104: The blur correction device 100 acquires the difference: MV_diff between the latest motion vector information (n) and the motion vector information (n−1) one frame before, and proceeds to step 105.

Step 105: In the blur correction device 100, the reference value correction amount calculation unit 35 sets an amount by which the reference value is corrected based on MV_diff. For the reference value, a correction amount is set based on the following idea, and the process proceeds to step 106.

MV_diff> 0: ω0_comp = −ω0_comp_def
MV_diff <0: ω0_comp = + ω0_comp_def
MV_diff = 0: ω0_comp = 0

ω0_comp: reference value correction amount ω0_comp_def: reference value correction constant

Step 106: The blur correction device 100 causes the reference value subtraction / addition unit 42 to subtract the ω0_comp calculated in Step 105 from the first reference value calculated in S003 (FIG. 4), and obtain a corrected second reference value. Ask. As described above, the second reference value is obtained in S005 in FIG. 3, but the blur correction process repeats the loop of returning to S002 after proceeding to S009 as shown in the flowchart of FIG. Therefore, the second reference value corrected in S005 is updated whenever the motion vector information is updated.

FIG. 7A is a graph showing a second reference value in the Yaw direction. The dotted line in the drawing indicates the first reference value when the correction is not performed according to the present embodiment, and the solid line in the drawing indicates the second reference value when the correction is performed according to the present embodiment.

FIG. 7B is a graph showing the direction of the motion vector information in the X direction.
For example, when the calculated motion vector information is in the plus direction as at time t1 in FIG. 7B, the first reference value is corrected to minus as shown in practice in FIG.
After that, the second reference value changes according to the value calculated by the reference value calculation unit 34 until time t3.
At time t3, when it is confirmed that the calculated motion vector information is in the plus direction, the first reference value is corrected to minus. In the present embodiment, the correction amount at this time is constant. That is, the correction amount at time t1 and the correction amount at time t3 are the same.
Thereafter, between the times when the motion vector information is confirmed, the reference value changes according to the value calculated by the reference value calculation unit 34 to become the corrected second reference value.
When the motion vector information is confirmed in the plus direction, the first reference value is corrected to a certain amount minus.
Further, when the motion vector information is confirmed in the minus direction as at times t22 and t25 in the drawing, the first reference value is corrected to a certain amount plus.

Next, details of creation of processing image data in the signal processing unit 40 of the embodiment will be described.
FIG. 8 is a block diagram illustrating details of generation of the processing image data in the signal processing unit 40. The signal processing unit 40 has a preprocessing unit 40A and a size adjustment unit 40B.

(Pre-processing unit 40A)
The preprocessing unit 40A performs, for example, an AFE circuit (Analog Front End circuit) on the image signal output from the image sensor 3 such as noise processing or A / D conversion to convert the image signal into digital image data. To the size adjustment unit 40B.

(Size adjustment unit 40B)
The size adjustment unit 40B converts the resolution input from the preprocessing unit 40A and adjusts the image size (number of pixels) of the captured data. For example, when the release switch 17 is fully pressed, the image size of the image data obtained from the pre-processing unit 40A is not reduced, or the reduction ratio of the image size of the image data is small, and is smaller than the processing image data described later. For example, full-size recording image data B0 is created and recorded on the recording medium 13.

The size adjustment unit 40B is not in the fully-pressed state. For example, when displaying a through image on the rear liquid crystal 18, when the release switch 17 is half-pressed to perform AF (autofocus), AE (automatic exposure) ) Is performed, a transmission image is created, a motion vector is calculated, or the like, the image size is reduced to a processing image size suitable for such processing.

Here, the image size for processing is reduced in two steps in the embodiment. First, first processing image data B1 smaller than recording image data B0 is created, and further reduced therefrom to create second processing image data B2. The relationship between the sizes of the recording image data B0, the first processing image data B1, and the second processing image data B2 is shown below.

B0 ≧ B1 ≧ B2

As an example, when the imaging size A of the imaging sensor 3 is 4608 × 3456 pixels, the size of the recording image data B0 is, for example, 4608 × 2592 pixels.
The second processing image data B2 has, for example, a VGA size and a resolution reduced to 640 × 360 pixels (pixels are thinned out), and is used for a through image, AF, AE, and transmission.

The first processing image data B1 has a size between the recording image data B0 and the second processing image data B2, and varies depending on the zoom position as described below.
The size adjustment unit 40B is connected to the lens drive amount calculation unit 39, and obtains zoom position information from the lens drive amount calculation unit 39. However, the present invention is not limited to this, and if a lens position detector is provided, the zoom position may be obtained from the lens position detector.
There are three zoom positions, for example, when the lens barrel 1 </ b> B is in a fixed area from the tele (telephoto) end, in a fixed area from the wide (wide angle) end, and in a middle area between the tele area and the wide area. It is divided into areas.

The size adjustment unit 40B is also connected to the storage unit 14.
The storage unit 14 stores the size of the first processing image data B1 corresponding to the zoom position. An example of the size of the first processing image data B1 for each area of the zoom position is shown below.

Tele area 640 × 360 pixels Middle area 1280 × 720 pixels Wide area 1920 × 1080 pixels
The motion vector detected on the image plane when the reference value of the angular velocity sensor 12 deviates from the true value is smaller as the focal length of the imaging optical system is shorter, and is larger as the focal length is longer. Therefore, to detect a motion vector, a high-resolution image is required in the wide area, but the range of motion vector detection may be small.In the tele area, a high-resolution image is not required, but the range of motion vector detection needs to be large. There is.
For example, when the zoom position is in the tele area, the image size of the first processing image data B1 is 640 × 360 pixels which is equal to the image size of the second processing image data B2. Then, as the zoom position moves from the tele area to the wide area, the image size of B1 increases, that is, the resolution increases.
As described above, the image data used in the motion vector calculation unit 41 is created in the signal processing unit 40 (size adjustment unit 40B) in different image sizes corresponding to the zoom positions.

The size adjustment unit 40B reads the image size corresponding to the zoom position obtained from the lens drive amount calculation unit 39 from the size of the first processing image data B1 stored in the storage unit 14, and reads out the image size of the image size. One processing image data B1 is created.

(Wide area)
Since the influence of camera shake becomes smaller and the detected motion vector becomes smaller as the area becomes wider, it may not be possible to detect a motion vector if the image size is small and the resolution is low.
However, according to the present embodiment, the image size of the first processing image data B1 created by the size adjustment unit 40B increases as the area becomes wider. Then, since the resolution is increased, the detection resolution (detection accuracy) of the motion vector is improved. Therefore, a motion vector can be accurately detected.

(Tele area)
The range in which the motion vector is detected is limited, and the moving amount of the feature point between two pieces of image data is, for example, up to 16 pixels. In the case of the tele region, the motion vector is larger than in the case of the wide region. Here, if the image size is large and the resolution is high like a wide area, the movement amount of the feature point for detecting the motion vector exceeds 16 pixels which is the detection limit of the motion vector, and the motion vector cannot be detected. The likelihood increases.
Therefore, in the embodiment, the image size in the tele area is smaller than that in the wide area. This makes it possible to prevent the motion vector from being undetectable in the tele area. In addition, the processing load and processing time can be reduced by reducing the image data.

(Modified form)
The present invention is not limited to the embodiments described above, and various modifications and changes as described below are possible, and these are also within the scope of the present invention.
(1) In the embodiment, the first processing image data B1 for motion vector calculation is once created, and then the second processing image data B2 used for other processing is converted from the first processing image data B1. Created. However, the present invention is not limited thereto, and the second processing image data B2 may be created directly from the full-size image without passing through the first processing image data B1.

(2) When the angular velocity is high, that is, when the camera shake amount is large, the size adjustment unit 40B may reduce the size of the first processing image data B1.
For example, when the camera shake is large, the motion vector becomes large. If the size of the first processing image data B1 used for the motion vector calculation is large, it may be out of the detection limit. Therefore, the size of the first processing image data B1 used for the motion vector calculation is reduced. As a result, it is possible to prevent the motion vector from being uncalculable.

(3) If the subject blur can be detected from, for example, captured image data, the size of the first processing image data B1 may be changed depending on the subject blur size.

(4) In the embodiment, the correction amount of the reference value is a certain amount of plus or minus. However, the correction amount may be changed depending on the zoom position. Further, the auto-focus operation may be performed by half-pressing the release switch 17, and the calculation of the motion vector may be stopped for a certain period. In such a case, the correction amount of the motion vector after the calculation of the motion vector is restarted after being stopped for a certain period may be increased.

(5) In this embodiment, the angular velocity sensor 12 is provided in the lens barrel 1B. However, the present invention is not limited to this, and the angular velocity sensor 12 may be provided in the camera body 1A.

(6) Further, instead of the angular velocity sensor, an acceleration sensor may be provided in the camera body 1A or the lens barrel 1B.

(7) In the above-described embodiment, the center bias, which is a centripetal force for moving the blur correction lens toward the center of its movable range, is determined based on the target position of the blur correction lens calculated by the target position calculation unit. Although the control using the center bias is performed, the control without using the center bias may be performed. In that case, there is no center bias calculator and no center bias remover.

1: camera, 1A: camera body, 1B: lens barrel, 2: CPU, 3: imaging sensor, 4: zoom lens, 5: focus lens, 6: blur correction lens, 7: zoom lens driving mechanism, 8: focus Lens drive mechanism, 9: blur correction lens drive mechanism, 10: aperture, 11: drive mechanism, 12: angular velocity sensor, 13: recording medium, 14: storage unit, 15: operation unit, 17: release switch, 18: rear liquid crystal , 21: blur correction lens position detector, 31: amplifier, 34: reference value calculator, 35: reference value correction amount calculator, 36: target position calculator, 37: center bias calculator, 38: center bias removal Unit, 39: lens drive amount calculation unit, 40: signal processing unit, 40A: preprocessing unit, 40B: size adjustment unit, 41: motion vector vector calculation unit, 42: reference value deduction and addition unit, 3: subtraction unit, 46: sensor control unit, 50: reference value correction unit, 100: a motion compensation device

Claims

An image sensor that captures an image of a subject by an optical system and outputs a signal;
A motion vector calculation unit that calculates information on a motion vector of the subject based on image data having a resolution corresponding to a focal length of the optical system, which is generated based on the signal.
An imaging device having:
The motion vector calculation unit calculates information about the motion vector based on image data having a higher resolution as the focal length of the optical system is shorter,
The imaging device according to claim 1.
An image generation unit that generates first image data based on the signal output from the image sensor and generates second image data having a resolution corresponding to a focal length of the optical system from the first image data. And
The motion vector calculation unit calculates information on the motion vector based on the second image data.
The imaging device according to claim 1.
An image generation unit that generates third image data having a resolution corresponding to a focal length of the optical system based on the signal output from the imaging element,
The motion vector calculation unit calculates information on the motion vector based on the third image data,
The imaging device according to claim 1.
A sensor that detects a shake of the imaging device and outputs a shake signal;
A correction element for correcting blurring of the image of the subject based on the shake signal;
A moving amount calculating unit that calculates a moving amount of the correction element using the information on the motion vector and the shake signal,
The imaging device according to claim 1.
A reference value calculation unit that calculates a first reference value serving as a reference of the shake signal based on the shake signal;
A reference value correction unit that corrects the first reference value based on the information about the motion vector to obtain a second reference value,
The movement amount calculation unit calculates the movement amount of the correction element based on the shake signal and the second reference value,
The imaging device according to claim 5.
The motion vector calculation unit calculates information on the motion vector from image data having a lower resolution as the shake signal increases,
The imaging device according to claim 5.