JP2017116924A - Zoom control device, zoom control method, and imaging apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To enable zoom control which meets user's intention more.SOLUTION: A zoom control device controlling an angle of view includes acquisition means 101 for acquiring the motion amount of an imaging optical system used for imaging a main subject, calculation means 116 for calculating the movement amount of a main subject image in a captured image on the basis of the motion amount, determination means 117 for determining whether or not the movement amount of the main subject image is a first threshold or more, and control means 118 for performing control for making the angle of view, when the movement amount of the main subject image is the first threshold or more, larger than the angle of view when the movement amount of the main subject image is smaller than the first threshold.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a zoom control device, a zoom control method, and an imaging device technology.

  Imaging devices include an optical zoom that drives a zoom lens and an electronic zoom that enlarges a part of the captured image, or a device that has both zoom functions. In recent years, even in popular machines that are not for business use, higher magnification has advanced. It is out. When a photographer who is unfamiliar with imaging performs telephoto imaging using a high-magnification model, it is difficult to capture the subject, and the captured subject can often be out of frame immediately.

  Japanese Patent Application Laid-Open No. 2004-133867 discloses a technique in which a camera automatically zooms out an imaging angle of view to a wide-angle side when a movement amount of an imaging device is detected and it is determined that a user is searching for a subject. Yes. In the following, a zooming support control function that supports the user's framing by zooming out the imaging angle of view in accordance with the movement of the imaging device will be referred to as framing assist zoom and abbreviated as “FA zoom”. It should be noted that returning to the original zoom position (zoom-in) from the state in which the imaging angle of view is zoomed out by FA zoom is also included in the FA zoom function.

JP2015-102853A

  In such a zoom control device that automatically zooms out based on the detected movement of the imaging device, it is required to perform zoom control in accordance with the user's intention.

  The present invention provides a zoom control device, a zoom control method, and an imaging device that can perform zoom control more in line with the user's intention.

  A zoom control device according to one aspect of the present invention controls an angle of view. The zoom control apparatus includes an acquisition unit that acquires a movement amount of an imaging optical system used for imaging a main subject, a calculation unit that calculates a movement amount of a main subject image in a captured image based on the movement amount, Determination means for determining whether or not the amount of movement of the subject image is equal to or greater than a first threshold; and when the amount of movement of the main subject image is equal to or greater than the first threshold, the angle of view and the amount of movement of the main subject image are the first And a control means for performing control to make the angle of view wider than the angle of view when smaller than the threshold value.

  A zoom control apparatus according to another aspect of the present invention controls the angle of view. The zoom control device includes an acquisition unit that acquires a movement amount of an imaging optical system used for imaging a main subject, and a control unit that changes a field angle. The control unit obtains a second value that is a value related to a movement amount of the movement of the imaging optical system from a time point when the first value that is the speed or acceleration as the value related to the movement amount becomes equal to or greater than the third threshold. The angle of view is widened in response to the second value becoming greater than or equal to the fourth threshold value.

  Other aspects of the invention will be described in the detailed description.

According to the present invention, it is possible to perform zoom control more in line with the user's intention.

1 is a block diagram illustrating a configuration example of an imaging apparatus according to a first embodiment. Schematic diagram explaining FA zoom Flowchart for explaining processing for calculating the movement amount of the subject image in the movement amount calculation unit according to the first embodiment. The figure explaining the moving amount calculation process of the to-be-photographed image in 1st Example The figure explaining the movement amount calculation process of the to-be-photographed image at the time of image blur correction in 1st Example The figure explaining the moving amount calculation process of the to-be-photographed image by the motion vector in 1st Example The figure explaining the detection process of the motion vector in the 1st Example, and the calculation process of a histogram The figure explaining the movement amount calculation process of the to-be-photographed image at the time of the electronic zoom effective in 1st Example. Block diagram showing a configuration example of an imaging apparatus according to the second embodiment The figure explaining the threshold value calculation process at the time of the electronic zoom effective in 2nd Example Block diagram showing a configuration example of an imaging apparatus according to the third embodiment The figure which shows the change of angular velocity and angular displacement in a comparative example The figure which shows the change of the angular velocity and angular displacement amount in 3rd Example. The flowchart which shows the process in 3rd Example. The figure which shows the method of calculating the reference value of the amount of angular displacement in the modification of 3rd Example.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. Each embodiment described below relates to a zoom control apparatus that can perform zoom control more in line with the user's intention by setting in detail the conditions for zooming out. In the first and second embodiments, an imaging apparatus including a zoom control apparatus that performs zoom control based on the amount of movement of a subject image in a captured image will be described. In the third embodiment, an imaging apparatus including a zoom control apparatus that performs zoom-out when the imaging apparatus makes a large movement will be described.

  In Embodiment 1 of the present invention, an imaging apparatus including a zoom control apparatus that performs zoom control based on a movement amount of a subject image in a captured image will be described. The FA zoom function disclosed in Japanese Patent Application Laid-Open No. 2015-102853 appropriately determines whether or not to perform zoom-out when the amount of movement of the imaging device does not match the amount of subject movement on the captured image. It is difficult. Therefore, zoom-out may not be performed at the timing desired by the photographer. For example, during electronic zoom, the movement of the subject position on the captured image is large with respect to slight movement of the imaging device, and the subject may be lost. For this reason, it is desirable to make it easier to activate the FA zoom function when electronic zoom is being performed than when electronic zoom is not being performed. Conversely, since it is relatively easy for a photographer to capture a subject moving at a constant speed in a certain direction, when the photographer is capturing the subject well, the subject position on the captured image with respect to the amount of movement of the imaging device The amount of movement becomes smaller. That is, it is desirable not to activate the FA zoom when the moving amount of the subject position on the captured image is small even if the moving amount of the imaging device is large.

  In this embodiment, by determining whether or not to perform zoom-out based on the amount of movement of the subject image in the captured image, zoom control can easily follow the photographer's intention.

  In the present embodiment, each component and its operation when the zoom control apparatus is applied to an imaging apparatus will be specifically described. FIG. 1 is a diagram illustrating a configuration example of an imaging apparatus 100 having an FA zoom function. The imaging apparatus 100 includes an image blur correction function that corrects an image blur due to camera shake or the like. The imaging apparatus 100 calculates the amount of motion of the FA zoom imaging apparatus by using shake information of the imaging apparatus detected for image blur correction. In this case, although shake information detection means is also used, since a motion amount detection sensor dedicated to FA zoom may be provided, it is not essential to provide an image blur correction function for realizing the FA zoom function.

  Before describing the FA zoom function, a configuration related to the image blur correction function will be described. The shake detection sensor 101 detects shake applied to the imaging optical system of the imaging apparatus 100. The shake sensor 101 is constituted by an angular velocity sensor such as a vibration gyroscope, and detects an angular velocity (motion velocity) as a first value relating to movement in the yaw direction and the pitch direction. A shake correction amount calculation unit 103 in the microcomputer 102 obtains an angular displacement amount in the yaw direction and the pitch direction by integrating the angular velocities in the yaw direction and the pitch direction, and acquires the shake correction amount from each displacement amount. To do. The shake detection sensor 101 outputs a detection signal to a microcomputer (μCOM) 102. The microcomputer 102 acquires the shake detection signal and performs signal processing. FIG. 1 is a functional block diagram showing processing executed by the microcomputer 102.

  The blur correction amount corresponds to an amount of driving the blur correction mechanism unit 108 to cancel the blur of the captured image. Specifically, the shake correction amount calculation unit 103 includes an integrator, integrates the angular velocity signal output from the shake detection sensor 101, converts the signal into an angle signal, and outputs the angle signal to the subtracter 104. At this time, by integrating the angular velocity in the yaw direction and the angular velocity in the pitch direction, the angle signal in the yaw direction and the angle signal in the pitch direction can be acquired.

  The subtracter 104 subtracts position data of a shake correction mechanism unit 108, which will be described later, from the output of the shake correction amount calculation unit 103, acquires deviation data, and outputs the deviation data to the control filter 105. The control filter 105 performs signal processing by the amplifier and the phase compensation filter on the deviation data acquired from the subtracter 104. The control filter 105 outputs the processed signal to the shift lens control unit 106.

  The shift lens control unit 106 drives and controls the shift lens (correction lens for image blur correction) of the blur correction mechanism unit 108 by controlling the motor 107 according to the output of the control filter 105. Specifically, the shift lens control unit 106 modulates the motor control amount into a PWM (pulse width modulation) waveform that changes the duty ratio of the pulse wave, and drives the motor 107. The motor 107 is, for example, a voice coil type motor. By driving the motor 107, the shift lens of the blur correction mechanism unit 108 moves in a direction different from the optical axis direction (for example, a direction orthogonal to the optical axis). The shift lens control unit 106 performs feedback control so that the deviation data from the subtractor 104 is closer to 0 as the shift lens moves.

  In this embodiment, as an example of the blur correction mechanism unit 108, a lens unit that can move in a direction different from the optical axis direction is illustrated. In FIG. 1, the blur correction mechanism unit 108 and the image sensor 110 are configured as separate units. . For example, the blur correction mechanism unit 108 may have a structure in which the image sensor 110 is disposed on the movable unit and can move in a direction different from the optical axis direction. In this case, image blur correction is performed by movement control of the movable unit including the image sensor 110. Further, the blur correction mechanism unit 108 may perform so-called electronic image stabilization. Image blur correction can be performed by changing the position of the cutout range of the image acquired by the image sensor 110.

  The position detection sensor 109 includes a magnet and a magnetic detection element (Hall element) provided at a position facing the magnet. The position detection sensor 109 detects the movement amount (movement amount in the direction perpendicular to the optical axis) of the correction lens of the shake correction mechanism unit 108 and outputs a detection signal to the subtractor 104. Thus, a feedback control system is configured to move the correction lens in a direction different from the optical axis direction and follow the target position with respect to the output of the blur correction amount calculation unit 103. That is, the correction lens moves based on the blur correction amount, and the blurring of the image caused by the shake of the imaging device 100 is corrected. On the imaging surface of the imaging device 110, a subject image in which the vertical or horizontal blur is corrected is formed.

  The image sensor 110 photoelectrically converts a light image of a subject formed by an image pickup optical system including the shake correction mechanism unit 108 and the zoom lens 120, and outputs an image pickup signal (image signal). The imaging optical system may be provided integrally with the imaging device, or may be provided so as to be interchangeable with the imaging device (lens exchange type). The signal processing unit 111 processes a signal acquired from the image sensor 110. The signal processing unit 111 generates a video signal (video signal) compliant with, for example, the NTSC (National Television System Committee) format and outputs the video signal to the image memory 112 and the motion vector detection unit 125. The signal processing unit 111 performs predetermined processing such as CDS (correlated double sampling), AGC (automatic gain adjustment), A / D conversion, gamma correction, and white balance on the image signal from the image sensor 110. Also good.

  The memory read control unit 113 determines the read position and read range of the image data stored in the image memory 112. For example, when the electronic zoom is executed, the memory read control unit 113 increases or decreases the image read range.

  The recording control unit 114 performs recording control of data read from the image memory 112 by the memory reading control unit 113. The recording control unit 114 performs control to output and record data read from the image memory 112 to the recording medium 115 when the user gives an instruction to record a video signal using the operation unit 130. The recording medium 115 is an information recording medium such as a semiconductor memory or a magnetic recording medium such as a hard disk.

  Next, a configuration related to the FA zoom function will be described. The zoom control apparatus according to the present embodiment can be configured by a shake detection sensor 101 and a microcomputer 102. In this embodiment, the microcomputer 102 includes a movement amount calculation unit 116, a capture determination unit 117, a zoom control unit 118, an electronic zoom control unit 122, a shake correction amount calculation unit 103, a subtractor 104, a control filter 105, and a shift lens control. Part 106 is included.

  The shake detection sensor 101 detects a shake of the imaging apparatus 100 and outputs a shake detection signal to a subject movement amount calculation unit (hereinafter referred to as a movement amount calculation unit) 116. The movement amount calculation unit 116 calculates the movement amount of the subject image. Details of the processing will be described later.

  A subject capture determination unit (hereinafter referred to as capture determination unit) 117 acquires a movement amount of the subject image from the movement amount calculation unit 116 and is a threshold value (a threshold value used for determining whether to perform zoom-out). (Also called threshold). The capture determination unit 117 determines the capture state of the subject based on whether or not the amount of movement of the subject image is greater than or equal to a threshold value. The capturing state of the subject is a state in which the determination result is one of whether the photographer captures the subject with the imaging device or searches for the subject. The movement amount of the subject image calculated by the movement amount calculation unit 116 is a value indicating, for example, a change in the position of the subject image on the image sensor 110 in units of pixels. The threshold value is a limit value of the imaging resolution.

  The zoom control unit 118 acquires the determination result of the capture determination unit 117 and outputs a zoom-out or zoom-in drive signal to the zoom motor 119 according to the determination result. When the determination result that the imaging apparatus 100 is in the zoom-out state and the capturing determination unit 117 is capturing the subject with the imaging apparatus is input, the zoom control unit 118 transmits the zoom-in drive signal to the zoom motor. It outputs to 119. On the other hand, when a determination result indicating that the imaging apparatus is in a zoom-in state (a state in which FA zoom is not functioning) and a subject is being searched for from the capture determination unit 117 is input, the zoom control unit 118 A zoom-out drive signal is output to the zoom motor 119.

  If the determination result that the imaging apparatus 100 is in the zoom-out state and the capture determination unit 117 is searching for the subject is input, the imaging angle of view is not changed. Similarly, when the determination result that the imaging device is in the zoom-in state and the capture determination unit 117 is capturing the subject with the imaging device is not changed, the imaging field angle is not changed.

  The zoom motor 119 is a stepping motor, for example. The feed screw is rotated by the rotation of the rotor, and the zoom lens 120 is moved in the optical axis direction. The zoom control unit 118 calculates a drive signal having the number of pulses necessary to move the zoom lens 120 to the target position and supplies the drive signal to the zoom motor 119. The zoom control unit 118 may input a drive signal to the zoom motor 119 until the position detection sensor 121 that detects the position of the zoom lens 120 detects that the zoom lens 120 has reached the target position. In the following description, zoom-out performed by the zoom control unit 118 when the FA function is activated is referred to as FA zoom-out, and zoom-in performed to return from the zoomed-out state by the FA zoom-out to the telephoto side is referred to as FA zoom-in. In the present embodiment, the FA zoom-out is a zoom-out performed in response to the determination result of the capture determination unit 117.

  A position scale is fixed to the holding frame of the zoom lens 120 in order to detect its position. A position detection sensor 121 is fixed to a lens barrel (not shown) at a position facing the position scale. The position scale is formed with a scale pattern such as a magnetic pattern or a light reflection pattern in the optical axis direction. The position detection sensor 121 reads the position scale magnetically or optically and outputs a position detection signal indicating the position of the zoom lens 120 in the optical axis direction. A detection signal from the position detection sensor 121 is input to the zoom control unit 118 and used for position control of the zoom lens 120. On the other hand, the position detection sensor 109 also supplies a detection signal to the movement amount calculation unit 116. In FIG. 1, A terminals 123 and 124, each indicated by A in a circular frame, indicate that they are electrically connected to each other.

  In response to the movement of the imaging apparatus 100, the shift lens control unit 106 controls the shift lens to suppress the movement of the image. The movement amount of the subject image is calculated by subtracting the movement amount of the imaging range accompanying the movement of the shift lens from the movement amount of the imaging range accompanying the movement of the imaging device.

  The imaging apparatus 100 has an electronic zoom function, and the electronic zoom control unit 122 takes charge of zoom control by image processing. When the electronic zoom function is valid, the zoom control unit 118 calculates the electronic zoom magnification and outputs it to the electronic zoom control unit 122. The electronic zoom control unit 122 instructs the memory reading control unit 113 on the image reading range based on the electronic zoom magnification. Further, the electronic zoom control unit 122 outputs the electronic zoom magnification to the movement amount calculation unit 116.

  In the electronic zoom control, a process for changing the cutout range of the image is executed. When the image cut out by this process is enlarged when displayed, the amount of movement of the subject image on the captured image is larger than when the electronic zoom control is disabled even if the movement of the imaging apparatus 100 is the same. Become. Therefore, the movement amount calculation unit 116 corrects the movement amount of the imaging range accompanying the movement of the imaging apparatus with the electronic zoom magnification, and calculates the corrected value as the movement amount of the subject image. It is assumed that the photographer can select whether to enable or disable the electronic zoom control by a menu setting on the display screen.

  The motion vector detecting unit 125 acquires the current video signal generated by the signal processing unit 111 and the video signal of the previous frame stored in the image memory 112, and based on the luminance signal included in the video signal, The motion vector of the subject at is detected. As a motion vector detection method, for example, a block matching method is used. The block matching method is a method in which a captured image is divided into regions called blocks and, for example, a similar portion between an image one frame before and a current image is detected in units of blocks. Processing for detecting a position having the largest correlation value with an arbitrary block in the current image as an analogous block position in an arbitrary range in the image one frame before is executed. Based on the amount of displacement between the position of an arbitrary block in the current image and the position of a similar block in the previous frame, a motion vector that is motion information between frame images is detected.

  The motion vector detection unit 125 may use a motion vector detection method other than the block matching method.

  The motion vector detection unit 125 outputs the detected motion vector to the movement amount calculation unit 116. In FIG. 1, B terminals 126 and 127 indicated by B in a circular frame indicate that they are electrically connected to each other. In the state where FA zoom is effective, zoom-out is automatically activated when the amount of movement of the subject image increases. On the other hand, when the photographer selects an imaging state in which FA zoom is not effective, zoom-out is not automatically performed even if the amount of movement of the subject image increases. The user can select whether to enable or disable the FA zoom by operating the operation unit 130. The operation unit 130 can be configured with operation switches, operation buttons, a touch panel, and the like, and accepts a photographer's operations on the camera, such as menu operations, zoom key operations, moving image capturing start operations, and still image capturing instruction operations. The operation unit 130 also accepts an operation for interrupting FA zoom-out. As an operation for interrupting the zoom-out, a zoom key operation for instructing zoom-in and zoom-out when the FA zoom function is not effective may be used.

  A specific example of FA zoom will be described with reference to FIGS. FIG. 2A illustrates a state in which the imaging apparatus 100 is moved from the left direction to the right direction in the drawing so as to capture the ball 201 that is the main subject that the photographer moves. Note that the imaging apparatus 100 also captures subjects 202 and 203 in addition to the ball 201. In FIG. 2A, the ball 201 is moving, and the current ball 201a is indicated by a black circle, and the past ball 201b is indicated by a shaded circle. FIG. 2B illustrates an image (captured image) captured at this time. FIG. 2C shows a temporal change in the amount of movement of the subject image. Times t1 to t3 shown in FIGS. 2B and 2C have a relationship of “t1 <t2 <t3”.

  Until the time t1, the photographer hardly moves the imaging apparatus 100, and the amount of movement of the subject image is near zero. When the ball 201 is released at time t1, the photographer moves the imaging device 100 in an attempt to catch this. At this time, the amount of movement of the subject image starts to increase as shown in FIG. At time t2, the amount of movement of the subject image exceeds the threshold value. The relationship between the threshold and the amount of movement of the subject image is as follows. As shown in FIG. 2B, the amount of movement when the main subject is likely to deviate from the imaging angle of view because the main subject moving by the imaging device cannot be captured. Is set to be equal to or greater than the threshold value. The capture determination unit 117 determines that the photographer is searching for a subject at time t2 when the amount of movement of the subject image is equal to or greater than the threshold. The zoom control unit 118 performs zoom-out control in response to the determination result. Therefore, the imaging angle of view becomes a wide angle at time t3 after time t2. That is, the movement of the picked-up image with respect to the movement of the imaging device 100 is reduced by the amount that the imaging angle of view is widened, and the movement of the imaging device 100 is also reduced. Therefore, as shown in FIG. 2C, the movement amount of the subject image gradually decreases after reaching the peak value after time t2.

With reference to the flowchart of FIG. 3 and FIGS. 4 to 8, the movement amount calculation processing performed by the movement amount calculation unit 116 will be described in detail. A subject movement amount calculation unit 116 as a microcomputer performs the processing according to a subject movement amount calculation program that is a computer program. In FIG. 3 and the following description, step is abbreviated as S.
In S100 of FIG. 3, the movement amount calculation unit 116 calculates the movement amount of the subject image from the movement amount of the imaging apparatus 100 and temporarily stores the movement amount. A specific example will be described with reference to FIGS. 4 (A), 4 (B) and FIG.

FIG. 4A is a schematic diagram illustrating a relationship between the imaging field angle θ, the imaging range d0, and the subject 202 in a state where the optical axis X of the imaging device 100 is aligned with the horizontal axis H. In FIG. 4, the imaging optical system of the imaging apparatus is schematically shown by a single lens 210. The imaging range d0 corresponds to the vertical range of the captured image. The angular velocity output of the shake detection sensor 101 is converted into an angle by integration. A value obtained by integrating the angular velocity is a displacement amount of the angle caused by the shake (hereinafter referred to as a shake angle). The shake angle may be acquired by starting integration from the timing when the FA zoom function is enabled by the photographer, or may be updated sequentially by integrating angular velocities in a predetermined period (for example, 1 second). The output (shake angle) of the shake detection sensor 101 converted to an angle is expressed as α, the current focal length of the imaging optical system is expressed as f, and the movement amount of the subject image is expressed as d. FIG. 4B is a schematic diagram illustrating a moving amount d of the subject image when the imaging apparatus 100 is shaken by the shake angle α. The optical axis X of the image pickup apparatus is inclined with respect to the horizontal axis H at a swing angle α. At this time, the movement amount d of the subject image is the same as the position P0 of the subject image before the shake of the imaging device (the state of FIG. 4A) and after the shake (the state of FIG. 4B). This is the difference from the position P1 of the subject image.
The movement amount d is calculated using the following formula (1).
d = f × tan α (1)
tan α is a tangent function of the deflection angle α. Since the movement of the position of the subject image in the captured image is suppressed by the movement of the correction lens of the shake correction mechanism unit 108, the amount of movement of the subject image in the captured image is smaller than the shake of the imaging device 100. . In FIG. 5, the correction amount based on the movement amount of the correction lens is expressed as β (unit: angle), and the movement amount of the subject image at this time is expressed as d ^* . The movement amount d ^* of the subject image is determined by shifting the lens 210 and correcting the shake in order to correct the subject image position P1 after the occurrence of the shake (the state of FIG. 4B). This is the difference from the position P2 of the subject image in the closed state (state in FIG. 5). In the case of the shake angle α and the correction amount β, the moving amount d ^* of the subject image is calculated by the following equation (2).
d ^* = f × tan (α−β) (2)
The length per pixel is expressed as u, and its unit is, for example, mm / pixel. By dividing the moving amount d ^* of the subject image by u and calculating d ^* / u, the length unit can be converted to the pixel unit. Although there is no particular limitation on the unit related to the movement amount of the subject image, when the unit of the motion vector is the pixel unit, the unit of the movement amount of the subject image is the pixel unit below. Note that the imaging apparatus 100 can be in a state where image blur correction is not performed, and in this case, the value of the correction amount β only becomes zero. That is, since it can be expressed by the above equation (2), the description will be made including the state where image blur correction is not performed.

  S101 in FIG. 3 is a process for determining whether or not a motion vector can be detected by the motion vector detection unit 125. Since the FA zoom is a function for preventing the main subject desired by the photographer from being out of frame, in this embodiment, the subject is detected based on the amount of movement of the main subject image in the actual captured image by detecting the motion vector. Judge the movement. That is, when the motion vector can be detected, the determination is basically performed based on the motion vector, and thus the process proceeds to S102. If a motion vector cannot be detected, the process proceeds to S104. As an example in which a motion vector cannot be detected, there is a case where a correct motion vector cannot be detected because the subject has a low contrast. In addition, since there are many subjects moving within one screen, it is impossible to detect a motion vector related to a specific subject, or because the shake amount of the imaging device between one frame is too large, a motion vector is detected. There are cases where imaging cannot be performed in a possible state.

  With reference to FIG. 6 and FIG. 7, the motion vector calculation processing of the main subject 204 (a, b) in S102 will be described. FIG. 6A shows the movement of the imaging device and the movement of the main subject 204. The main subject at time t1 is indicated by reference numeral 204a, and the main subject at time t2 is indicated by reference numeral 204b. An example in which the main subject 204 moves from time t1 to time t2 and the photographer moves the imaging device 100 at the swing angle α in order to capture the main subject 204 is illustrated. FIG. 6B illustrates a captured image at time t1, and FIG. 6C illustrates a captured image at time t2. FIG. 7A illustrates motion vectors V01 to V05 in a plurality of detection blocks set in the captured image. FIG. 7B illustrates a histogram of detected motion vectors. The horizontal axis represents the motion vector value, and the vertical axis represents the number of detected motion vectors (the number of detected vectors).

  Since the output of the motion vector detecting unit 125 includes the motion vector of the main subject 204 and the background motion vector, in this embodiment, the motion vectors are classified using the histogram illustrated in FIG. The motion vector detection unit 125 calculates motion vectors V01 to V05 for each detection block based on the data of the captured image at time t1 and the captured image at time t2. Processing for generating a histogram shown in FIG. 7B is executed from the detected plurality of motion vectors.

FIG. 7B illustrates a first motion vector 205 having a larger number of detected vectors and a second motion vector 206 having a smaller number of detected vectors. It can be determined that a motion vector having a value comparable to the movement amount d ^* of the subject image calculated from the movement amount of the imaging apparatus 100 is the movement of the image due to camera movement, that is, the background movement vector. In the example of FIG. 7B, since the motion vector value of the first motion vector 205 having the larger number of detected vectors has a value close to the moving amount d ^* of the moving subject image, it is the background motion vector. It is considered. Therefore, the motion vector of the main subject 204 excludes the motion vector determined as the background motion vector (first motion vector 205) from all the motion vectors, and among the remaining motion vectors, the number of detected vectors is the largest. It is determined as a motion vector with a lot of motion. In the example of FIG. 7B, the motion vector value of the second motion vector 206 is regarded as the motion vector value of the main subject 204. The magnitude of the detected motion vector of the main subject 204 is denoted by v.

In S103 of FIG. 3, the movement amount calculation unit 116 replaces the magnitude v of the motion vector of the main subject calculated in S102 with the movement amount of the subject image, and ends the process. In other words, if the motion vector can be detected, the subject image movement amount stored in S100 is updated to the motion vector of the main subject. On the other hand, if it is determined in S101 that the motion vector cannot be detected (No), the amount of movement of the subject image cannot be updated with the motion vector v of the main subject. A correction value obtained by correcting the image movement amount with the electronic zoom magnification is used as a final object image movement amount. Therefore, in S104, the movement amount calculation unit 116 determines whether or not electronic zoom control is being executed. If the electronic zoom control is being executed, the process proceeds to S105. If the electronic zoom control is not being executed, the subject image movement amount d ^* calculated in S100 is the final subject image. After the amount of movement is determined, the process is terminated.

In S105, the movement amount calculation unit 116 corrects the movement amount d ^* of the subject image calculated in S100 with the electronic zoom magnification. The corrected movement amount (denoted as e) is calculated by the following equation (3).
e = d ^* × electronic zoom magnification formula (3)
With reference to FIGS. 8A to 8C, the process of S105 will be described. FIG. 8A illustrates a state in which the main subject 207 moves from time t1 to time t2, and the photographer moves the imaging apparatus 100 with the swing angle α in order to capture the main subject 207. Let d be the amount of movement of the subject image on the captured image relative to the amount of movement of the imaging device (when β = 0, and when β is not zero, it is d ^* ). The movement amount d of the subject image is the difference between the subject position at time t1 and the subject position at time t2. FIG. 8B illustrates an image captured when the electronic zoom control is not performed, and FIG. 8C illustrates an image captured when the electronic zoom control is performed.

  When the electronic zoom is not controlled in the imaging state shown in FIG. 8A, the amount of movement of the subject image in the captured image shown in FIG. 8B is d. On the other hand, when the electronic zoom is controlled, an image enlargement or reduction process using the electronic zoom is executed. Therefore, as shown in FIG. 8C, the moving amount of the subject image within the captured image is e. That is, when electronic zoom control is performed, the amount of change in the position of the subject image in the captured image changes according to the electronic zoom magnification even if the movement of the imaging apparatus 100 is the same. In S105, the movement amount calculation unit 116 determines the movement amount e of the subject image corrected using the electronic zoom magnification as the movement amount of the subject image, and ends the processing. Even if the motion vector cannot be acquired by the correction used in the above equation (3), the movement amount of the subject image can be calculated from the movement amount of the imaging apparatus 100 in consideration of the electronic zoom.

  According to the present embodiment, the zooming support control can be performed at an appropriate timing according to the imaging state by the determination based on the movement amount of the subject image in the captured image.

  Next, a second embodiment of the present invention will be described. In the present embodiment, as in the first embodiment, an imaging apparatus including a zoom control apparatus that performs zoom control based on the amount of movement of the subject image in the captured image will be described. In the zoom control apparatus according to the present embodiment, the microcomputer 202 includes a threshold acquisition unit 228, and the subject capture determination unit 217 determines whether or not the subject is captured using the threshold acquired by the threshold acquisition unit 228. Is different from the first embodiment.

  FIG. 9 illustrates an imaging apparatus 200 having the FA zoom function of this embodiment. Note that in the imaging apparatus 200 according to the present embodiment, the same reference numerals as those used in the imaging apparatus 100 according to the first embodiment are used, and detailed descriptions thereof are omitted, and differences are mainly described. To do.

As described above, the microcomputer (μCOM) 202 in the imaging apparatus 200 includes the threshold value acquisition unit 228. The movement amount calculation unit 216 calculates the movement amount of the subject image based on the output of the shake detection sensor 101. Specifically, by subtracting the value obtained by converting the shake correction amount related to the shake correction mechanism unit 108 into an angle from the shake angle obtained by integrating the angular velocity output of the shake detection sensor 101, the subject in units of the angle. An image movement amount (denoted as d2) is calculated. Although the unit of the subject image movement amount d ^* in the first embodiment is a pixel, the basic idea of the calculation method is the same. The present embodiment is different from the first embodiment in that the amount of movement of the subject image is acquired by using an angle instead of acquiring the moving amount by using the length (in units of pixels). However, in the present embodiment, as in the first embodiment, the movement amount of the subject image may be acquired in units of pixels. In the first embodiment, the movement amount of the subject image may be acquired in units of angle. Also good. The movement amount d2 of the subject image is input to the subject capture determination unit 217.

  The subject capture determination unit 217 determines the capture state of the subject based on the movement amount d2 of the subject image and the motion vector detected by the motion vector detection unit 125 (see B terminals 126 and 127). As in the case of the first embodiment, in the determination process using the motion vector, the capture state is determined based on the motion vector of the main subject, but the threshold value is calculated by the threshold value acquisition unit 228 based on information from the zoom control unit 218. This is different from the first embodiment. The threshold acquired by the threshold acquisition unit 228 is a threshold for determining the capture state of the subject related to the movement amount d2 of the subject image. A specific example will be described with reference to FIG.

  FIG. 10A illustrates the threshold for the focal length according to the presence or absence of electronic zoom control. The horizontal axis represents the focal length of the imaging optical system, and the vertical axis represents the threshold value. In this example, the horizontal axis indicates the focal length, but it may be replaced with the distance from the camera to the subject. When electronic zoom control is not performed by the electronic zoom control unit 222 (OFF), the threshold value is denoted as TH1 and indicated by a dotted line graph. When the electronic zoom control is performed by the electronic zoom control unit 222 (ON), the threshold value is expressed as TH2 and is indicated by a solid line graph. On the telephoto side, a process of changing the determination threshold according to the focal length is performed so that the zoom-out is activated with the movement amount of the imaging device smaller than that on the wide-angle side.

In FIG. 10A, “TH2 <TH1” for the same focal length. When electronic zoom control is being executed (when the electronic zoom magnification is greater than 1), compared to when electronic zoom control is not being executed even when the amount of movement of the same imaging apparatus is (when the electronic zoom magnification is 1). The amount of movement of the subject image on the captured image increases. For this reason, when the electronic zoom magnification is larger than 1, the zoom-out is easily triggered by setting the threshold value smaller than when the electronic zoom magnification is 1. Specifically, as shown by the following formula (4), a value obtained by dividing the threshold value TH1 by the electronic zoom magnification is set as the threshold value TH2.
TH2 = TH1 / electronic zoom magnification formula (4)
TH2 may be set for each electronic zoom magnification, and may be acquired by reading information stored in the storage unit constituting the threshold acquisition unit 228, or TH1 and electronic zoom stored in the storage unit may be acquired. You may calculate from magnification.

  When the electronic zoom control is performed, zoom-out is activated when the moving amount d2 of the subject image is equal to or greater than the threshold value TH2. The threshold values TH1 and TH2 are first threshold values used for determining whether or not to perform FA zoom-out. After that, when the moving amount d2 of the subject image falls below the capture threshold TH3, zoom-in control is started, and processing for returning to the original telephoto focal length is performed. The capture threshold is a threshold for determining that the subject image is within the captured image and capturing the subject. When the subject image movement amount d2 is less than the capture threshold, the subject image is It is determined that the image is within the captured image and the subject is captured.

  In other words, the capture threshold is a threshold for determining whether or not to perform FA zoom-in, and is also referred to as a second threshold. FIG. 10B is a diagram for explaining the limitation of the threshold when electronic zoom control is performed. The horizontal axis represents the focal length of the imaging optical system (or the distance from the camera to the subject), and the vertical axis represents the threshold value.

  “TH3 <TH2” at the same focal length, and diff represents the difference between TH2 and TH3. Note that the value of TH3 also changes depending on whether electronic zoom control is performed or not. Description of the capture determination threshold value when the electronic zoom control is not executed is omitted.

  When TH2 is reduced by the electronic zoom magnification, the difference diff from TH3 is reduced. When the difference diff is too small, there are the following problems regarding the operation of the FA zoom. When the zoom-out is activated when the movement amount d2 of the subject image slightly exceeds TH2, the zoom-in starts below TH3 just after the movement amount d2 is slightly reduced. In other words, the hunting phenomenon in which zoom-out and zoom-in frequently occur may make it difficult to see the video. Therefore, in this embodiment, even if the focal length changes, a restriction is provided on TH2 so that the difference diff does not become smaller than a predetermined value (lower limit value). Specifically, the predetermined value is a value corresponding to the amount of movement when the photographer performs fixed-point imaging so as not to move the imaging device 100. The reason is to prevent the movement amount d2 from being greater than or equal to the first threshold value TH2 or less than the second threshold value TH3 only due to the influence of camera shake.

  In the present embodiment, processing for changing a determination threshold for determining the capture state of the subject is performed. According to the present embodiment, zooming support control can be performed based on a comparison result between the movement amount of the subject image in the captured image and the determination threshold value after the change.

  In this embodiment, the threshold value is changed according to the electronic zoom, but the same effect can be obtained even when the capture determination unit 217 corrects the magnitude v of the main subject motion vector with the electronic zoom magnification and then compares it with the threshold value. Can be obtained. In this embodiment, the threshold value is changed according to the focal length (optical zoom magnification) and the electronic zoom magnification. However, the threshold value may be changed only according to the focal length or the electronic zoom magnification. It is considered that changing the threshold according to both the focal length and the electronic zoom magnification can perform zoom control in accordance with the photographer's intention rather than changing the threshold according to only one of them. . On the other hand, if the change is made according to only one of them, the amount of data to be stored and the amount of calculation can be reduced.

  In Embodiment 3 of the present invention, an imaging apparatus including a zoom control apparatus that performs FA zoom-out when the imaging apparatus makes a large movement will be described. When FA zoom-out is performed based on the detected movement of the imaging device, the imaging device moves due to camera shake and the movement of the imaging device due to camera shake, and the imaging device moves by panning. In some cases, zooming out is preferable.

  In the present embodiment, a first value that is a value (speed, acceleration) related to the movement of the imaging apparatus and a second value that is a movement amount are used to set a position that is a reference for obtaining a deflection angle using acceleration. By doing so, a small movement of the imaging device due to camera shake is distinguished from a large movement of the imaging device due to panning. Note that the velocity includes angular velocity, the acceleration includes angular acceleration, and the moving amount includes a deflection angle. In this embodiment, the angular velocity is acquired as the first value using a shake detection sensor that detects the angular velocity, and the deflection angle is acquired as the second value by integrating the angular velocity.

  FIG. 11 illustrates an imaging apparatus 300 having the FA zoom function of this embodiment. Note that in the imaging apparatus 300 according to the present embodiment, the same reference numerals as those used in the imaging apparatus 100 according to the first embodiment are used to omit the detailed description thereof, and mainly the differences will be described. To do.

  The microcomputer 102 of the imaging apparatus 300 according to the present embodiment functions as a motion determination unit 128 and a threshold acquisition unit 129. The motion determination unit 128 and the threshold acquisition unit 129 determine whether or not to perform zoom-out using the FA zoom function. The motion determination unit 128 uses the angular velocity obtained from the output of the shake detection sensor 101 and the shake angle α obtained by integrating the angular velocity to change the orientation of the imaging apparatus 300 so that the photographer searches for the subject. It is determined whether (panning) is being performed. Panning is an operation of moving the camera so that its orientation in the horizontal direction changes. However, not only the horizontal direction but also the operation of moving in the vertical direction (tilting) is often included in the panning in a broad sense, and in the present invention and the present specification, the term “panning” includes tilting. .

  In general, a photographer who loses sight of a subject moves the camera greatly in order to find the subject. Therefore, when the angular velocity and the angular displacement amount of the camera are equal to or greater than a predetermined threshold (at least one of an angular velocity threshold and an angular displacement threshold described later), the motion determination unit 128 moves the camera so that the photographer searches for the subject. Judge that it is. Conversely, when the photographer is stably capturing the subject, the movement of the camera is small. For this reason, when either the angular velocity or the amount of angular displacement of the camera is smaller than the motion determination threshold, the motion determination unit 128 determines that the photographer is not looking for the subject, that is, is capturing the subject in the imaging screen. The threshold value is calculated by the threshold value acquisition unit 129. The relationship between the angular velocity and the shake angle α and the threshold will be described in detail later. A motion detection unit is configured by the shake sensor 101 that detects the angular velocity and the motion determination unit 128 that acquires a threshold from the threshold acquisition unit 129.

  The zoom control unit 118 receives a determination result of the motion determination unit 128 and supplies a drive signal for performing FA zoom-out and FA zoom-in to the zoom motor 119. The zoom control unit 116 and the motion determination unit 128 constitute field angle changing means. The shake detection sensor 101, the motion determination unit 128, and the zoom control unit 118 constitute a zoom control device.

  The threshold acquisition unit 129 acquires a threshold based on the current focal length (zoom position) obtained from the zoom control unit 118. Similar to the second embodiment, the telephoto side threshold value is set smaller than the wide angle side threshold value. In the present embodiment, since the electronic zoom is not considered, the threshold value is determined according to the focal length (TH1 of the second embodiment is used). However, in the case of a zoom control apparatus in which FA zoom-out is activated even during electronic zoom control, it is preferable to use a threshold (TH2) set according to the electronic zoom magnification and the focal length as in the second embodiment.

  Next, the relationship between the angular velocity and shake angle of the imaging apparatus 300 and the threshold value for determining whether or not to activate FA zoom out will be described. FIGS. 12 and 13A and 13B show the angular velocity and fluctuation of the camera while panning is performed following the moving main subject from the state where the subject is captured in the imaging screen (within the angle of view). An example of a corner is shown. Each figure shows a threshold value for an angular velocity (hereinafter also referred to as a third threshold value or an angular velocity threshold value) and a detected angular velocity and a threshold value for a shake angle (hereinafter also referred to as a fourth threshold value or a shake angle threshold value). The relationship with the corner is shown.

  FIG. 12 shows an example of the change between the angular velocity and the shake angle α in the comparative example in which the shake angle is acquired by integrating the angular velocity from the time when the subject is captured in the imaging screen (the time at the left end in the figure). FIG.

  In FIG. 12, while the subject is captured in the imaging screen before panning, the change in the shake angle is small, the angular velocity is temporarily increased, and the detected angular velocity is equal to or greater than the angular velocity threshold (in FIG. 12). , A), the deflection angle is less than the deflection angle threshold value, and FA zoom-out is not performed. Thereafter, when slow panning is performed so as to follow the subject, the deflection angle increases in one direction (in the positive direction in FIG. 12), and the deflection angle α eventually becomes greater than the deflection angle threshold. In this state, when the angular velocity is temporarily increased to be equal to or greater than the angular velocity threshold (time B in FIG. 12), FA zoom-out is performed even though the subject is captured on the imaging screen.

  In contrast, in this embodiment, as shown in FIG. 13A, when the angular velocity is slower than the angular velocity threshold while the subject is captured in the imaging screen before panning, the shake angle is reset to 0 and the angular velocity is set. Integration of angular velocities is started from the time when becomes equal to or higher than the angular velocity threshold (time C in FIG. 13), and the deflection angle α is acquired. Then, the deflection angle is reset to 0 in response to the angular velocity becoming smaller than the angular velocity threshold again. This is the same during the subsequent slow panning. During slow panning, since the time during which the angular velocity is equal to or greater than the angular velocity threshold is short, the deflection angle is reset to 0 before it becomes larger than the deflection angle threshold. Therefore, the FA zoom-out can be prevented from being performed when the subject can be followed by slow panning.

  On the other hand, in FIG. 13B showing the present embodiment, abrupt panning is performed in order to search for a missing subject from a state in which the subject is captured in the imaging screen before panning, as in FIG. 13A. As a result, the angular velocity becomes equal to or higher than the angular velocity threshold, and the state continues. As a result, the angular velocity integration is started from the time when the object is started to be searched, and the shake angle is acquired, and at the time E in FIG. 13B, the shake angle becomes equal to or greater than the shake angle threshold. In this case, FA zoom out is performed. As described above, in this embodiment, the timing at which the integration of the angular velocity for obtaining the deflection angle is started is the time when the angular velocity becomes equal to or higher than the angular velocity threshold. As a result, FA zoom-out is not performed when slow panning is performed, and FA zoom-out can be performed when sudden panning is performed.

  The flowchart in FIG. 14 shows processing of FA zoom control (view angle control) performed by the microcomputer 102. The microcomputer 102 performs this processing according to the FA zoom control program that is a computer program.

  In step S200, the zoom control unit 118 holds the current zoom position as the reference zoom position f0. The reference zoom position f0 is a target zoom position when performing FA zoom-in after performing FA zoom-out.

  In step S201, the zoom control unit 118 calculates a target zoom position (hereinafter referred to as a target zoom-out position) f1 for performing FA zoom-out. For example, when the zoom-out amount in FA zoom-out is set to 1 / n times in terms of focal length, the target zoom-out position f1 is f0 / n. Note that the value n for calculating the target zoom-out position may be a predetermined fixed value or a value that can be set by the photographer through a menu operation or the like. Further, the zoom-out amount may be set by the number of steps. In this case, the zoom lens moves by m steps to the wide-angle side by FA zoom-out. The value m may also be a predetermined fixed value or a value that can be set by the photographer.

  Next, in step S202, the motion determination unit 128 determines whether or not the angular velocity is greater than or equal to the angular velocity threshold. If the angular velocity is greater than or equal to the angular velocity threshold, the process proceeds to step S204, and if not, the process proceeds to step S203.

  In step S203, the deflection angle is reset to 0, the current routine is terminated, and the process returns to step S200. In the first embodiment, the shake angle acquisition (acceleration integration) is performed by the shake correction amount calculation unit 103. However, in this embodiment, the reference position of the shake angle used for the shake correction and the shake angle used for the FA zoom function are used. Therefore, the shake angle used for the FA zoom function is acquired by the motion determination unit 128, but is not limited to this. In this embodiment, the reference position of the shake angle used for shake correction is the position of the optical axis X at the time when the shake correction function is turned on.

  On the other hand, in step S204, the zoom control unit 118 calculates a shake angle. Specifically, the angular velocity obtained from the shake detection sensor 101 is integrated to calculate the shake angle. Further, when the deflection angle is calculated in step S204 of the previous routine, the previously calculated deflection angle and the currently calculated deflection angle are integrated. Thereby, the deflection angle (the integrated value of angular velocities starting from the time when the angular velocity becomes equal to or greater than the angular velocity threshold) with the position of the optical axis as the reference position when the angular velocity becomes equal to or greater than the angular velocity threshold can be calculated. it can. When the deflection angle is calculated in the previous routine, the previous deflection angle and the current deflection angle are integrated. In the previous routine, the process proceeds from step S202 to S203, and when the deflection angle is not calculated, the integration result is calculated. A certain deflection angle is set as the deflection angle calculated in this step. Then, the process proceeds to step S205.

  In step S205, the motion determination unit 128 determines whether or not the shake angle (integrated value) is equal to or greater than the shake angle threshold value (equal to or greater than the shake angle threshold value), that is, sudden panning is performed so that the photographer searches for the subject. Determine whether it is done. The zoom control unit 118 proceeds to step S206 when the shake angle is equal to or larger than the shake angle threshold, and performs FA zoom-out when the subject is captured in the imaging screen when the shake angle is smaller than the shake angle threshold. Without returning to step S200.

  In step S206, the zoom control unit 118 performs FA zoom-out toward the target zoom-out position f1 calculated in step S201.

  In step S207, the zoom control unit 118 determines whether or not the current zoom position has reached the target zoom-out position f1. If the target zoom-out position f1 has not been reached, the process returns to step S206 to continue FA zoom-out. If the target zoom-out position f1 has been reached, the process proceeds to step S208.

  In step S208, the zoom control unit 118 causes the motion determination unit 128 to determine whether the subject is captured in the imaging screen or whether rapid panning is performed so as to search for the subject. The zoom control unit 118 proceeds to step S209 when the subject is captured in the imaging screen, and repeats this step while holding the zoom position at f1 when panning is performed so as to search for the subject.

  In step S209, the zoom control unit 118 performs FA zoom-in toward the reference zoom position f0 held in step S200.

  In step S210, the zoom control unit 118 determines whether or not the current zoom position has reached the reference zoom position f0, and continues FA zoom-in when the reference zoom position has not been reached. On the other hand, if the reference zoom position f0 has been reached, the current routine is terminated and the process returns to step S200.

  Note that after the FA zoom-in is completed, the zoom control unit 118 may change the value n (or m) for calculating the target zoom-out position. If the photographer does not change the imaging angle of view through the zoom key operation, the reference zoom position f0 and the target zoom-out position f1 may be held and the next routine may be started from step S202.

  As described above, according to the present embodiment, the integrated value of the angular velocities from the time when the angular velocity becomes equal to or higher than the angular velocity threshold is used as the deflection angle, and the deflection angle is reset when the angular velocity becomes less than the angular velocity threshold. Then, FA zoom-out is performed when the deflection angle is equal to or greater than the deflection angle threshold. Thus, when the camera is moved slowly while capturing the subject in the captured image, the FA zoom-out is not performed when the camera is moved quickly and greatly so as to search for a subject that is out of the captured image. Can be done.

  In the above-described embodiment, the reference value of the shake angle is set to 0. However, as a modification of this embodiment, an example in which the shake angle for a certain period when the angular velocity is less than the angular velocity threshold is used as the reference value can be given.

  FIG. 15 shows the deflection angle reference value. In this embodiment, the average value of the deflection angles is calculated during a predetermined period (reference value calculation period) during which the angular velocity is slower than the angular velocity threshold, and this is used as the deflection angle reference value. Then, the deflection angle from the deflection angle reference value is calculated in a state where the angular velocity is equal to or greater than the angular velocity threshold value, and FA zoom-out is performed when the deflection angle becomes larger than the deflection angle threshold value. The deflection angle from the deflection angle reference value can also be obtained by taking the difference between the deflection angle with the reference value of 0 and the deflection angle reference value as in the above-described embodiment. The reference value calculation period can be set as appropriate, and may be set to 1 second, for example.

  While the subject is captured in the imaging screen without panning (and while the subject is captured in the imaging screen while performing slow panning, not shown), the time during which the angular velocity is greater than or equal to the angular velocity threshold is short. It's time. For this reason, every time the angular velocity becomes slower than the angular velocity threshold, the deflection angle is averaged with the previous deflection angle, and the averaged deflection angle never exceeds the deflection angle threshold. Therefore, it is possible to prevent the FA zoom-out from being performed while following the subject by slow panning.

  On the other hand, as shown in FIG. 15, sudden panning is performed in order to search for a subject that has been lost, so that the angular velocity becomes faster than the angular velocity threshold and the state continues. As a result, the deflection angle based on the deflection angle reference value becomes equal to or greater than the deflection angle threshold (E ′). In this case, FA zoom out is performed. As described above, even when a swing angle using the average value of the swing angles in the swing angle reference value calculation period as a reference value is used, when slow panning is performed, FA zoom-out is not performed, and rapid panning is performed. FA zoom out can be performed.

  In addition, if the average value of the deflection angle in the reference value calculation period is the deflection angle reference value, the average deflection angle in a certain stop period before the panning operation can be used as the reference value. It is possible to more accurately determine the displacement of the deflection angle associated with.

  In the present embodiment, the motion determination is performed using the angular velocity and the shake angle, but angular acceleration may be used instead of the angular velocity. Further, instead of using a shake detection sensor, a motion vector may be detected from a plurality of continuous frame images obtained by moving image capturing, and an angular velocity (angular acceleration) and a shake angle may be acquired from the motion vector. The speed, acceleration, and movement amount of the main subject image may be acquired. Even when using a motion vector, the movement amount of the main subject image is reset when the magnitude of the motion vector becomes less than a predetermined value in order to prevent FA zoom out during slow panning. Is preferred.

The fourth embodiment of the present invention is a mode in which the third embodiment is combined with the first embodiment, and a subject image movement amount acquisition method when it is determined in step S101 of the flowchart shown in FIG. 3 that a motion vector cannot be detected. Is different from the first embodiment. In this embodiment, when it is determined in step S101 that a motion vector cannot be detected, a shake angle is acquired using the time point when the angular velocity is equal to or higher than the angular velocity threshold as in the third embodiment, and the obtained shake is obtained. The movement amount d ^* of the subject image is acquired with the angle being α in Equation 2. At this time, similarly to the third embodiment, when the angular velocity becomes less than the angular velocity threshold, the deflection angle is reset to 0 (or the calculated deflection angle reference value). While the electronic zoom is being executed (when the electronic zoom magnification is not 1), the correction is made with the electronic zoom magnification as in the first embodiment.

The present invention has been described in detail based on the preferred embodiments thereof, but the present invention is not limited to these specific embodiments, and various forms within the scope of the present invention are also included in the present invention. included. A part of the above-described embodiments may be appropriately combined. That is, the imaging device does not necessarily have to be configured by a single mechanism. For example, in the application to an interchangeable lens camera, a configuration including a microcomputer, a shake detection sensor, a shake correction mechanism, and a zoom lens in a lens device that can be mounted on the camera body is also included in the technical scope of the present invention. . In addition, the detection example using the motion vector regarding the movement amount of the main subject image has been described. For example, the moving amount of the main subject image can be calculated using subject recognition, face position detection, and the like.
[Other Examples]
The present invention supplies a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in a computer of the system or apparatus read and execute the program This process can be realized. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

100, 200 Imaging device 101 Shake detection sensor 103 Blur correction amount calculation unit 108 Blur correction mechanism unit 116, 216 Movement amount calculation unit 117, 217 Capture determination unit 118, 218 Zoom control unit 122, 222 Electronic zoom control unit 125 Motion vector detection 129, 228 Threshold acquisition unit

Claims (19)

A zoom control device for controlling an angle of view,
An acquisition means for acquiring a movement amount of an imaging optical system used for imaging a main subject;
Calculating means for calculating the amount of movement of the main subject image in the captured image based on the amount of movement;
Determining means for determining whether or not the amount of movement of the main subject image is equal to or greater than a first threshold;
Control for controlling the angle of view to be wider than the angle of view when the amount of movement of the main subject image is smaller than the first threshold when the amount of movement of the main subject image is greater than or equal to the first threshold. And a zoom control device. Electronic zoom control means for changing the electronic zoom magnification by image processing;
The calculating means corrects the movement amount of the main subject image using the electronic zoom magnification when the electronic zoom magnification is changed by the electronic zoom control means, and calculates a corrected value as the movement amount. The zoom control apparatus according to claim 1, wherein: Image blur correction means for correcting blur of the captured image,
3. The calculation unit according to claim 1, wherein the calculation unit calculates a difference between a correction amount by the image blur correction unit and the movement amount, and calculates a movement amount of the main subject image based on the difference. The zoom control device described. Having detection means for detecting a motion vector from images taken at different times;
The calculation means calculates a movement amount of the main subject image based on the motion vector when the motion vector of the main subject image is detected by the detection means, and the motion vector is not detected by the detection means. 4. The zoom control apparatus according to claim 1, wherein a movement amount of the main subject image is calculated based on the movement amount.   5. The zoom control according to claim 1, wherein the first threshold value differs according to at least one of a focal length of the imaging optical system and a distance to the main subject. 6. apparatus.   The first threshold when the focal distance of the imaging optical system or the distance to the subject is large is smaller than the first threshold when the focal distance of the imaging optical system or the distance to the subject is small. The zoom control apparatus according to claim 5. Electronic zoom control means for changing the electronic zoom magnification by image processing;
The zoom control apparatus according to claim 5, wherein the first threshold value varies depending on the electronic zoom magnification.   The control means performs zoom-out control for widening the angle of view when the amount of movement of the subject image is equal to or greater than the first threshold, and the amount of movement of the subject image while the zoom-out control is being performed. The zoom control apparatus according to any one of claims 1 to 7, wherein zoom-in control is performed to narrow the angle of view when becomes smaller than a second threshold value. Threshold acquisition means for acquiring the first and second thresholds that differ depending on at least the focal length of the imaging optical system and the distance to the main subject;
Electronic zoom control means for changing the electronic zoom magnification by image processing,
9. The difference between the first threshold acquired by the threshold acquisition means and the second threshold when the electronic zoom magnification is greater than 1, is greater than or equal to a preset value. The zoom control device described in 1. The acquisition means acquires at least one of the speed and acceleration of the movement of the imaging optical system as a first value related to the movement of the imaging optical system,
The calculation means obtains a second value that is a value related to a movement amount of the movement of the imaging optical system from a time point when the first value becomes equal to or greater than a third threshold value.
Obtaining a movement amount of the main subject image based on the second value;
The second value is reset when the first value becomes less than the third threshold value, and the second value is set again based on the time point when the first value becomes equal to or greater than the third threshold value. The zoom control apparatus according to any one of claims 1 to 9, wherein the value of is acquired again. Having detection means for detecting a motion vector from images taken at different times;
The calculation means calculates a movement amount of the main subject image based on the motion vector of the main subject when the detection means detects the motion vector of the main subject, and the detection means 11. The zoom control apparatus according to claim 1, wherein when a motion vector is not detected, a movement amount of the main subject image is calculated based on the second value. A zoom control device for controlling an angle of view,
An acquisition means for acquiring a movement amount of an imaging optical system used for imaging a main subject;
Control means for changing the angle of view,
The control means is a second value that is a value related to a movement amount of the movement of the imaging optical system from a time point when a first value that is a speed or acceleration as a value related to the movement amount becomes equal to or greater than a third threshold value. And the angle of view is widened in response to the second value becoming a fourth threshold value or more.   The control unit resets the second value when the first value becomes equal to or less than the third threshold, and again sets the reference when the first value becomes equal to or greater than the third threshold. The zoom control apparatus according to claim 12, wherein the second value is reacquired as follows.   The control means uses the average value of the second values during a predetermined period during which the first value is equal to or less than the third threshold as a reference value, and the first value is equal to or greater than the third threshold. The zoom control apparatus according to claim 12, wherein the second value is acquired from a certain time point.   The zoom control apparatus according to claim 12, wherein the first value is an angular velocity or an angular acceleration, and the second value is a deflection angle. A zoom control device according to any one of claims 1 to 15,
An image pickup apparatus comprising: an image pickup device that picks up an image at the angle of view.   The imaging apparatus according to claim 16, further comprising an imaging optical system in which the angle of view is changed by the zoom control apparatus. A zoom control method executed in a zoom control device for controlling an angle of view,
Obtaining a movement amount of an imaging optical system used for imaging a main subject;
Calculating the amount of movement of the main subject image in the captured image based on the amount of movement;
Determining whether the amount of movement of the main subject image is greater than a first threshold;
Performing a control to make the angle of view wider than the angle of view when the amount of movement of the main subject image is smaller than the first threshold when the amount of movement of the main subject image is larger than the first threshold. And a zoom control method. A zoom control method executed in a zoom control device for controlling an angle of view,
Obtaining a movement amount of an imaging optical system used for imaging a main subject;
Obtaining a second value that is a value relating to a movement amount of the movement of the imaging optical system from a time point when a first value that is a speed or acceleration as a value relating to the movement amount is equal to or greater than a third threshold; And a step of widening the angle of view in response to the second value becoming greater than or equal to a fourth threshold value.