JP2019045768A - Imaging device, control method of the same, program thereof, and storage medium - Google Patents

Abstract

To provide an imaging device that, even when a plurality of subjects enter a focus detection area, correctly tracks a subject a user aims at, and enables the subject to be kept focused on.SOLUTION: An imaging device, which has a function automatically causing focus of an imaging lens to be pointed at a subject, comprises: a setting unit that sets a main subject serving as an object to be focused on by an imaging lens; a position calculation unit that calculates a position of the main subject; a velocity vector calculation unit that calculates a velocity vector of the main subject on the basis of the past positions of the main subject at a plurality of points calculated by the position calculation unit; and a space calculation unit that calculates a first subject existence space having existence of the main subject predicted, using the velocity vector calculated by the velocity vector calculation unit. The setting unit is configured to, when a distance among a plurality of subjects themselves in a screen gets close to a prescribed distance or less, select the subject serving as an object to be focused on from the plurality of subjects, using the first subject existence space calculate by the space calculation unit, and reset the selected subject as the main subject.SELECTED DRAWING: Figure 13

Description

  The present invention relates to an automatic focusing technique in an imaging apparatus.

  2. Description of the Related Art Conventionally, when photographing an object at rest using an imaging device, photographing is performed by using an autofocus (hereinafter referred to as AF) function widely known in the related art. This makes it possible to obtain a good image in which the subject is in focus.

  On the other hand, when focusing is performed on a moving subject in the same manner and then shooting is performed, a time lag occurs from the start of operation of the shutter in the imaging apparatus to exposure. Due to the time lag, an image may be captured at a position where the subject has moved further from the in-focus position, and an out-of-focus image may be obtained. Therefore, the subject position at the exposure timing is predicted from the focus adjustment information of a plurality of points in the past of the subject. Then, by moving the focusing lens to a predicted position in advance and then performing exposure, it is possible to obtain an image in focus with high probability. Such techniques are also widely used generally.

  Furthermore, an example of performing focus detection by predicting not only the depth direction (Z direction) with respect to the imaging device as described above, but also movement in a direction (XY direction) parallel to the imaging surface of the imaging device is disclosed in Is disclosed in

JP, 2009-284462, A

  However, in the prior art disclosed in Patent Document 1, for example, motion of the subject in the XYZ directions is detected, and focus detection areas in the XY directions used for focus detection are only changed. Therefore, for example, when a plurality of subjects enter the focus detection area, there arises a problem that it is not possible to know which subject is the subject of the user to be tracked.

  The present invention has been made in view of the above-described problems, and an object thereof is that a user can correctly track and continue focusing on a target subject even when a plurality of subjects enter the focus detection area. Providing an imaging device that can

  An imaging apparatus according to the present invention is an imaging apparatus having a function of automatically focusing an imaging lens on a subject, the setting unit setting a main subject to be focused on the imaging lens, and the position of the main subject Calculating the velocity of the main subject based on the positions of the main subjects at a plurality of points in the past calculated by the position calculating means; and velocity vector calculating means for calculating the velocity vector of the main subject; Calculating means for calculating a first subject existing space in which the main subject is predicted to be present, using the velocity vector calculated by the setting step, the setting means determining a distance between a plurality of subjects in the screen Is closer to or less than a predetermined distance, the plurality of objects can be captured using the first subject existing space calculated by the space calculating means. Select an object is an object to be focused from, characterized in that re-set as the main subject.

  According to the present invention, it is possible to provide an imaging device capable of correctly tracking and keeping in focus the target subject of the user even when a plurality of subjects enter the focus detection area.

FIG. 1 is a diagram showing a configuration of a camera that is a first embodiment of an imaging device of the present invention. Schematic of the pixel array of an image pick-up element. The schematic plan view and schematic sectional drawing of a pixel. Schematic explanatory drawing of a pixel and a pupil division. Schematic explanatory drawing of an image pick-up element and pupil division. Schematic relationship between defocus amount and image shift amount. 5 is a flowchart showing the flow of focus detection processing. Schematic explanatory drawing of the shading by the pupil shift of a 1st focus detection signal and a 2nd focus detection signal. The figure which shows the example of the frequency band of a filter. FIG. 6 is a view for explaining the movement of a subject and a velocity vector in the first embodiment. FIG. 6 is a view for explaining a state in which a plurality of subjects intersect in the first embodiment. FIG. 6 is a view for explaining a state in which a plurality of subjects intersect in the first embodiment. FIG. 2 is a view showing a subject existing space in the first embodiment. The figure which shows the to-be-photographed object presence space in 2nd Embodiment.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the attached drawings.

First Embodiment
In the present embodiment, a space in which a subject is expected to exist (hereinafter referred to as a subject existing space) is defined based on focus adjustment information detected by an imaging apparatus, and a subject present in the subject existing space is set as a main subject. I will explain how to do it.

[overall structure]
FIG. 1 is a view showing a configuration of a camera which is a first embodiment of an imaging device of the present invention. In FIG. 1, the first lens group 101 is disposed at the tip of the imaging optical system, and is held so as to be movable back and forth in the optical axis direction. The diaphragm / shutter 102 adjusts the light amount at the time of shooting by adjusting the aperture diameter, and also functions as an exposure time adjustment shutter at the time of still image shooting. The second lens group 103 advances and retracts in the optical axis direction together with the aperture / shutter 102, and performs a variable power operation (zoom function) in conjunction with the advancing and retracting operation of the first lens group 101.

  The third lens group 105 performs focusing by advancing and retracting in the optical axis direction. The optical low pass filter 106 is an optical element for reducing false color and moiré of a captured image. The imaging element 107 is composed of a two-dimensional CMOS photosensor and a peripheral circuit, and is disposed on the imaging surface of the imaging optical system.

  The zoom actuator 111 rotates the cam barrel (not shown) to drive the first lens group 101 and the second lens group 103 back and forth in the direction of the optical axis to perform a magnification change operation. The aperture shutter actuator 112 controls the aperture diameter of the aperture / shutter 102 to adjust the imaging light amount, and performs exposure time control at the time of still image imaging. The focus actuator 114 drives the third lens group 105 in the optical axis direction to perform focusing.

  The electronic flash 115 illuminates the subject at the time of shooting. As the electronic flash 115, a flash illumination device using a xenon tube is preferable, but an illumination device having an LED that emits light continuously may be used. The AF auxiliary light device 116 projects an image of a mask having a predetermined opening pattern onto a field through a light projection lens, and improves the focus detection capability for a dark subject or a low contrast subject.

  The CPU 121 is an in-camera CPU that controls various operations of the camera body, and includes an arithmetic unit, a ROM, a RAM, an A / D converter, a D / A converter, a communication interface circuit, and the like. Then, based on a predetermined program stored in the ROM, various circuits of the camera are driven to execute a series of operations such as AF, photographing, image processing, recording, and the like.

  The electronic flash control circuit 122 controls lighting of the electronic flash 115 in synchronization with the photographing operation. The auxiliary light drive circuit 123 controls the lighting of the AF auxiliary light device 116 in synchronization with the focus detection operation. The imaging device drive circuit 124 controls the imaging operation of the imaging device 107, A / D converts the image signal output from the imaging device 107, and transmits the image signal to the CPU 121. The image processing circuit 125 performs processing such as γ conversion, color interpolation, and JPEG compression of an image acquired by the imaging element 107.

  The focus drive circuit 126 drives and controls the focus actuator 114 based on the focus detection result to drive the third lens group 105 back and forth in the optical axis direction to perform focus adjustment. The diaphragm shutter drive circuit 128 controls driving of the diaphragm shutter actuator 112 to control the opening of the diaphragm shutter 102. The zoom drive circuit 129 drives the zoom actuator 111 according to the zoom operation of the photographer.

  The display device 131 includes an LCD or the like, and displays information on a shooting mode of the camera, a preview image before shooting and a confirmation image after shooting, an in-focus state display image at the time of focus detection, and the like. The display device 131 includes a touch panel, and the user can perform various operations by directly contacting the touch panel. The operation switch group 132 includes a power switch, a release (shooting trigger) switch, a zoom operation switch, a shooting mode selection switch, and the like. The flash memory 133 is attachable to and detachable from the camera body, and records a photographed image.

[Image sensor]
FIG. 2 is a diagram schematically showing the arrangement of imaging pixels and focus detection pixels of the imaging device in the present embodiment. In FIG. 2, a pixel (imaging pixel) array of a two-dimensional CMOS sensor which is an imaging element used in this embodiment is shown in a range of 4 columns × 4 rows, and a focus detection pixel array is shown in a range of 8 columns × 4 rows. There is.

  In the pixel group 200 of 2 columns × 2 rows shown in FIG. 2, the pixel 200R having R (red) spectral sensitivity is at the upper left, and the pixel 200 G having G (green) spectral sensitivity is at the upper right and lower left, B ( A pixel 200B having a spectral sensitivity of blue) is disposed at the lower right. Furthermore, each imaging pixel is configured by a first focus detection pixel 201 and a second focus detection pixel 202 arranged in 2 columns × 1 row.

  A large number of 4 columns × 4 rows of imaging pixels (8 columns × 4 rows of focus detection pixels) shown in FIG. 2 are arranged on the surface to enable acquisition of a captured image (focus detection signal). In the present embodiment, the period P of the imaging pixel is 4 μm, the number N of pixels of the imaging pixel is 5575 rows × 3725 rows = approximately 20.75 million pixels, the column direction period PAF of the focus detection pixels is 2 μm, and the number of pixels of the focus detection pixels In the following description, it is assumed that the NAF uses an imaging element of 11150 rows × 3725 rows = about 41.5 million pixels.

  FIG. 3A shows a plan view of one pixel 200G of the imaging device shown in FIG. 2 as viewed from the light receiving surface side (+ z side) of the imaging device, and a cross section aa in FIG. A cross-sectional view seen from the y side is shown in FIG. In the present embodiment, the horizontal direction with respect to the ground on the imaging surface is the x direction, the vertical direction is the y direction, and the depth direction of the subject is the z direction.

  As shown in FIG. 3, in the pixel 200G, a micro lens 305 for condensing incident light is formed on the light receiving side of each pixel, NH division (two divisions) in the x direction, NV division (one division in the y direction) The photoelectric conversion unit 301 and the photoelectric conversion unit 302 are formed. The photoelectric conversion unit 301 and the photoelectric conversion unit 302 correspond to the first focus detection pixel 201 and the second focus detection pixel 202, respectively.

  The photoelectric conversion unit 301 and the photoelectric conversion unit 302 may be a pin structure photodiode in which an intrinsic layer is sandwiched between a p-type layer and an n-type layer, or the intrinsic layer is omitted as necessary. It may be a photodiode.

  In each pixel, a color filter 306 is formed between the micro lens 305 and the photoelectric conversion unit 301 and the photoelectric conversion unit 302. Further, the spectral transmittance of the color filter may be changed for each focus detection pixel as needed, or the color filter may be omitted.

  Light incident on the pixel 200 </ b> G illustrated in FIG. 3 is collected by the micro lens 305, separated by the color filter 306, and then received by the photoelectric conversion unit 301 and the photoelectric conversion unit 302. In the photoelectric conversion unit 301 and the photoelectric conversion unit 302, electrons and holes are generated in a pair according to the amount of light received, and after being separated by the depletion layer, electrons of negative charge are stored in the n-type layer (not shown) The holes are discharged to the outside of the imaging device through a p-type layer connected to a constant voltage source (not shown). Electrons accumulated in the photoelectric conversion unit 301 and the n-type layer (not shown) of the photoelectric conversion unit 302 are transferred to the electrostatic capacitance unit (FD) through the transfer gate and converted into a voltage signal.

  FIG. 4 is a schematic explanatory view showing the correspondence between the pixel structure shown in FIG. 3 and pupil division. FIG. 4 shows a cross-sectional view of the aa cross section of the pixel structure shown in FIG. 3A as viewed from the + y side and an exit pupil plane of the imaging optical system. Further, in order to correspond to the coordinate axes of the exit pupil plane, the x-axis and y-axis of the sectional view are reversed with respect to FIG.

  In FIG. 4, the first pupil partial region 501 of the first focus detection pixel 201 is in a substantially conjugate relationship with the light receiving surface of the photoelectric conversion unit 301 whose center of gravity is decentered in the −x direction. A pupil region which can be received by the single focus detection pixel 201 is shown. The first pupil partial region 501 of the first focus detection pixel 201 has its center of gravity decentered on the + x side on the pupil plane.

  Similarly, the second pupil partial region 502 of the second focus detection pixel 202 is in a substantially conjugate relationship with the light receiving surface of the photoelectric conversion unit 302 whose center of gravity is decentered in the + x direction by the micro lens. The detection pixel 202 represents a light receiving pupil area. The second pupil partial region 502 of the second focus detection pixel 202 has its center of gravity decentered on the −X side on the pupil plane.

  Further, in FIG. 4, the pupil region 500 is a pupil region which can be received by the entire pixel 200 G when all of the photoelectric conversion unit 301 and the photoelectric conversion unit 302 (the first focus detection pixel 201 and the second focus detection pixel 202) are combined. It is.

  FIG. 5 is a schematic view showing the correspondence between the imaging device and the pupil division. The light beams having passed through different partial pupil regions of the first pupil partial region 501 and the second pupil partial region 502 are incident on respective pixels of the imaging device at different angles, and are divided into 2 × 1 first focus detection pixels 201 And the second focus detection pixel 202 receives light. The present embodiment is an example in which the pupil region is divided into two in the horizontal direction. If necessary, pupil division may be performed in the vertical direction.

  The imaging device of the present embodiment includes a first focus detection pixel that receives a light beam passing through a first pupil partial region of the imaging optical system, and a second pupil partial region of an imaging optical system that is different from the first pupil partial region And a second focus detection pixel for receiving a light flux passing through the light source. Furthermore, a plurality of imaging pixels that receive a light flux passing through a pupil area that is a combination of the first pupil partial area and the second pupil partial area of the imaging optical system are arranged. In the imaging device of the present embodiment, each imaging pixel is composed of a first focus detection pixel and a second focus detection pixel. As necessary, the imaging pixel, the first focus detection pixel, and the second focus detection pixel are configured as individual pixels, and the first focus detection pixel and the second focus detection pixel are partially arranged in a part of the imaging pixel array. It may be configured to

  Further, in the present embodiment, the light reception signal of the first focus detection pixel 201 of each pixel of the imaging device is collected to generate a first focus detection signal, and the light reception signal of the second focus detection pixel 202 of each pixel is collected to Two focus detection signals are generated to perform focus detection automatically. Further, by adding the signals of the first focus detection pixel 201 and the second focus detection pixel 202 for each pixel of the imaging element, an imaging signal (captured image) having a resolution of N effective pixels is generated.

[Relationship between defocus amount and image shift amount]
Hereinafter, the relationship between the defocus amount and the image shift amount of the first focus detection signal and the second focus detection signal acquired by the imaging device will be described.

  FIG. 6 is a diagram showing a schematic relationship between the defocus amount and the image shift amount between the first focus detection signal and the second focus detection signal. An imaging element is disposed on the imaging surface 800, and the exit pupil of the imaging optical system is divided into a first pupil partial region 501 and a second pupil partial region 502, as in FIGS.

  The defocus amount d sets the distance from the imaging position of the subject to the imaging plane to be | d |, and the negative sign (d <0) of the front pin state in which the imaging position of the subject is closer to the object than the imaging plane A back-pin state in which the imaging position of the subject is on the opposite side of the subject from the imaging plane is defined as a positive sign (d> 0). The in-focus state where the imaging position of the subject is on the imaging surface (in-focus position) is d = 0. In FIG. 6, the subject 801 shows an example of the in-focus state (d = 0), and the subject 802 shows an example of the front pin state (d <0). The front pin state (d <0) and the rear pin state (d> 0) are combined to be in the defocus state (| d |> 0).

  In the front pinning state (d <0), the light flux from the object 802 that has passed through the first pupil partial region 501 (second pupil partial region 502) is once condensed, and then the barycentric position G1 of the light flux The image spreads over a width Γ 1 (Γ 2) centering on (G 2) and becomes an image blurred on the imaging surface 800. The blurred image is received by the first focus detection pixel 201 (second focus detection pixel 202) constituting each pixel arranged in the imaging device, and a first focus detection signal (second focus detection signal) is generated. . Therefore, the first focus detection signal (second focus detection signal) is recorded at the gravity center position G1 (G2) on the imaging surface 800 as a subject image in which the subject 802 is blurred to a width Γ1 (Γ2). The blur width Γ1 (Γ2) of the subject image generally increases in proportion to the increase of the magnitude | d | of the defocus amount d. Similarly, the magnitude | p | of the image shift amount p of the subject image between the first focus detection signal and the second focus detection signal (= difference G1−G2 of the gravity center position of the light flux) is also the magnitude of the defocus amount d. As | d | increases, it increases roughly proportionally. Even in the rear focus state (d> 0), the image shift direction of the object image between the first focus detection signal and the second focus detection signal is opposite to that in the front focus state, but the same applies.

  Therefore, in the present embodiment, as the magnitude of the defocus amount of the imaging signal obtained by adding the first focus detection signal and the second focus detection signal or the first focus detection signal and the second focus detection signal increases. The magnitude of the image shift amount between the first focus detection signal and the second focus detection signal is increased.

[Focus detection]
Hereinafter, a phase difference focus detection method will be described. In the phase difference type focus detection, the first focus detection signal and the second focus detection signal are relatively shifted to calculate a correlation amount (first evaluation value) representing the degree of coincidence of the signals, and the correlation (degree of signal coincidence) The image shift amount is detected from the shift amount which is improved. As the magnitude of the image shift amount increases between the first focus detection signal and the second focus detection signal as the magnitude of the defocus amount of the imaging signal increases, the image shift amount is detected as the defocus amount To focus detection.

  FIG. 7 is a flowchart showing a schematic flow of focus detection processing. The operation of FIG. 7 is executed by the image sensor 107, the image processing circuit 125, and the CPU 121.

  In step S110, a focus detection area for performing focus adjustment is set from among the effective pixel areas of the imaging device. The focus detection signal generation unit configured by the image processing circuit 125 and the CPU 121 generates a first focus detection signal (image A) from the light reception signal of the first focus detection pixel of the focus detection area, and the second focus of the focus detection area. A second focus detection signal (image B) is generated from the light reception signal of the detection pixel.

  In step S120, 3-pixel addition processing is performed on the first focus detection signal and the second focus detection signal in the column direction to suppress the signal data amount, and the RGB signal is converted to a luminance Y signal. Perform Bayer (RGB) addition processing. These two addition processes are combined to make a pixel addition process.

  In step S130, shading correction processing (optical correction processing) is performed on the first focus detection signal and the second focus detection signal.

  Here, shading due to pupil shift of the first focus detection signal and the second focus detection signal will be described. FIG. 8 shows the relationship between the first pupil partial region 501 of the first focus detection pixel 201, the second pupil partial region 502 of the second focus detection pixel 202, and the exit pupil 400 of the imaging optical system at the peripheral image height of the imaging device. FIG.

  FIG. 8A shows the case where the exit pupil distance Dl of the imaging optical system and the set pupil distance Ds of the imaging device are the same. In this case, the exit pupil 400 of the imaging optical system is approximately equally pupil-divided by the first pupil partial region 501 and the second pupil partial region 502.

  On the other hand, as shown in FIG. 8B, when the exit pupil distance D1 of the imaging optical system is shorter than the set pupil distance Ds of the imaging device, the peripheral image height of the imaging device A pupil shift occurs between the exit pupil and the entrance pupil of the imaging device, and the exit pupil 400 of the imaging optical system is divided into pupils unevenly. Similarly, as shown in FIG. 8C, when the exit pupil distance D1 of the imaging optical system is longer than the set pupil distance Ds of the imaging element, the exit pupil of the imaging optical system at the peripheral image height of the imaging element As a result, pupil deviation of the entrance pupil of the imaging device occurs, and the exit pupil 400 of the imaging optical system is divided into unequal pupils. As the pupil division becomes uneven at the peripheral image height, the intensities of the first focus detection signal and the second focus detection signal also become unequal, and either one of the first focus detection signal and the second focus detection signal Shading occurs in which the intensity increases and the other intensity decreases.

  In step S130 of FIG. 7, according to the image height of the focus detection area, the F value of the imaging lens (imaging optical system), and the exit pupil distance, the first shading correction coefficient of the first focus detection signal and the second focus Second shading correction coefficients of the detection signal are generated respectively. Shading correction processing of the first focus detection signal and the second focus detection signal by multiplying the first shading correction coefficient by the first focus detection signal and multiplying the second shading correction coefficient by the second focus detection signal (optical correction processing )I do.

  In the phase difference type focus detection, the first detection defocus amount is detected based on the correlation (the degree of coincidence of the signals) between the first focus detection signal and the second focus detection signal. When shading occurs due to pupil misalignment, the correlation (the degree of coincidence of the signals) of the first focus detection signal and the second focus detection signal may decrease. Therefore, in the phase difference type focus detection, the shading correction process (optical correction process) is performed to improve the correlation (the degree of coincidence of the signals) between the first focus detection signal and the second focus detection signal and to improve the focus detection performance. It is desirable to do).

  In step S140 of FIG. 7, filter processing is performed on the first focus detection signal and the second focus detection signal. An example of the pass band of the filtering process is shown by a solid line in FIG. In this embodiment, in order to perform focus detection in a large defocus state by phase difference type focus detection, the pass band of the filter processing is configured to include a low frequency band. As necessary, when performing focus adjustment from the large defocus state to the small defocus state, the pass band of the filter processing at the time of focus detection is made as indicated by the alternate long and short dash line in FIG. It may be adjusted to a higher frequency band.

  Next, in step S150 of FIG. 7, shift processing is performed to shift the first focus detection signal and the second focus detection signal after filter processing relatively in the pupil division direction, and the correlation amount representing the degree of coincidence of the signals (evaluation Calculate the value).

  The k-th first focus detection signal after filtering is A (k), the second focus detection signal is B (k), and the range of the number k corresponding to the focus detection area is W. The correlation amount (first evaluation value) COR is calculated by the equation (1), where the shift amount by the shift processing is s1 and the shift range of the shift amount s1 is Γ1.

  The k-th first focus detection signal A (k) and the k-s1nd second focus detection signal B (k-s1) are correlated and subtracted by shift processing of the shift amount s1 to generate a shift subtraction signal . The absolute value of the generated shift subtraction signal is calculated, the sum of the numbers k is taken in the range W corresponding to the focus detection area, and the correlation amount (evaluation value) COR (s1) is calculated. As necessary, the correlation amount (evaluation value) calculated for each row may be added over a plurality of rows for each shift amount.

  In step S160, from the correlation amount (evaluation value), a shift amount of a real number value at which the correlation amount becomes a minimum value is calculated by sub-pixel calculation to obtain an image shift amount p1. The defocus amount is detected by multiplying the image shift amount p1 by the image height of the focus detection area, the F value of the imaging lens (imaging optical system), and the conversion coefficient K1 according to the exit pupil distance.

[Definition of subject existing space]
Next, the definition of the subject existing space will be described.

  First, the main subject determination means selects a main subject for focusing at the time of shooting. The main subject determination means is composed of an operation switch group 132 operated or instructed by the user or a display 131 having a touch panel, and the CPU 121, and the main subject is selected by the user's operation. Alternatively, when a subject is detected by the face detection function of the imaging apparatus, the subject may be set as a main subject, or a moving object is detected based on a correlation result of images of each frame at the time of imaging. And it may be the main subject. Furthermore, the color of the subject is detected, and the subject having a specific color is used as the main subject, or focus detection is performed on all or part of the imaging plane, and the subject detected at a predetermined distance is used as the main subject. Do. An example of selecting such a main subject is considered, and a subject detected by a widely known subject detection method may be used as the main subject.

  FIG. 10 shows that the main subject determined by the main subject determination means is obliquely directed toward the imaging device. In principle, it is desirable to detect motion in each of the X, Y, and Z directions to define the subject existing space, but here, in order to make the description easy to understand, the X and Z directions are illustrated. Here, the Z direction indicates the optical axis direction of the lens, and the vertical method in FIG. 10 is taken as the Z direction. In the X direction, the horizontal direction is taken as the X direction, and the direction orthogonal to the X direction is taken as the Y direction on the imaging surface of the imaging device 107 with the Z axis as the normal.

  FIG. 10 shows times t0 to t3 at which the subject is present, respectively. Further, velocity vectors in the X direction and Z direction at respective times are indicated by x0 to x2 and z0 to z2, respectively. The time t0 to t3 at this time can be set at an arbitrary time interval, may be sampled at every frame at an imaging frame rate cycle of the imaging apparatus, or is a cycle later than that to reduce the calculation load You may sample at.

  The position of the main subject is detected by subject position detection means configured by the image processing circuit 125 and the CPU 121. The subject position calculation means calculates the positions in the X, Y, and Z directions, respectively. Then, from the change in position in the X, Y, Z directions calculated by the object position calculation means, the object velocity vector calculation means constituted by the image processing circuit 125 and the CPU 12 calculates motion vectors in each direction. The motion velocity vector in the X direction in this case may be calculated on the basis of the object detection result in the imaging apparatus, and a generally widely known face detection function or other object tracking function may be used to calculate the X direction. Each coordinate and velocity vector may be calculated.

  Further, as the motion velocity vector in the Z direction, the one obtained by converting the object position into the image plane position is used. For example, a method using the defocus amount detected by the above-described focus detection method can be considered. The respective defocus amounts at time t0 to t3 may be detected, and the velocity vector components z0 to z2 of the defocus amount may be calculated therefrom.

  In the present embodiment, although the image plane position is converted, the conversion may be performed again at the position on the subject side, and it is only necessary to know the relative movement amount and the direction. In the case of shooting still images continuously, the focusing lens is moved each time an image is taken to focus on the subject, so the position of the subject can not be estimated from only the defocus amount. In that case, the lens driving amount for each continuous shooting may be calculated by replacing it with the position in the Z direction.

  From the change of the velocity vector in the X direction and the velocity vector in the Z direction thus obtained, it is possible to predict, for example, velocity vectors in the X direction and Z direction at time t3. In this embodiment, respective approximate expressions are created using the velocity vector information of the past three points, that is, x0 to x2 and z0 to z2. The approximation formula may use any form, but it is possible to predict with high accuracy, for example, by performing polynomial approximation using the least squares method.

  Next, the velocity vector amounts in the X direction and Z direction at time t3 are extrapolated and predicted using this approximate expression. This can define a range in which the subject is predicted to be present from the change in the past movement even if the imaging device may lose sight of the subject position on the way. A subject presence space setting means (space calculation means) configured by the CPU 121 or the like sets a range in which the subject calculated by the above method may exist (a range in which the subject is expected to exist) as the subject presence space.

  For example, when the imaging apparatus keeps tracking the subject from time t0, if the subject loses sight after time t3 or is confused with another subject and no one of the main subjects is known, time t3 to t4 The subject H1 in the middle is present in the subject existing space defined by x3 and z3 from the subject position at time t3. In the above description, the X direction and the Z direction are shown, but the present invention can naturally be applied to the Y direction, and the object existing space can be defined as a three-dimensional space by the velocity vectors in the X, Y and Z directions. It becomes.

  FIG. 11 is a diagram showing an example in which a plurality of subjects are mixed in the same focus detection area in the screen. In FIG. 11, there are two subjects H1 and H2. It is assumed that the photographer is trying to image the subject H1 using the imaging device, and the tracking function of the imaging device is treating the subject H1 as a main subject. At this time, it is assumed that the subject H1 and H2 come to a position Xn where the subjects H1 and H2 overlap with each other (a position where the distance between the subjects is equal to or less than a predetermined distance). At this time, the subjects H1 and H2 viewed from the image pickup apparatus overlap each other as shown in FIG. 12A, and it is difficult to determine which face is the main subject even if tracking is performed using a face tracking function or the like. It is in the state. Thereafter, after a predetermined time has elapsed, as shown in FIG. 12B, the subjects H1 and H2 are separated, and tracking can be started again. However, once the face of the main subject H1 is blocked by the face of another subject H2 and disappears, there is a high possibility that the face tracking function erroneously recognizes H2 as the main subject and sets the subject H2 as a tracking target .

  Therefore, it is determined which of the subjects H1 and H2 is the main subject using the subject existing space described above.

  As shown in FIG. 13, the time when the objects overlap is t5. At time t5, the two subjects H1 and H2 overlap, and the main subject H1 is hidden behind another subject H2 and can not be tracked once.

  Here, the velocity vector from time t5 to the next time is calculated by the method described above, and the subject existing space is defined. At this time, the time t5 when the subjects overlap may be detected accurately, or as another method, when another subject approaches within a predetermined range where transition of the tracking target is likely to occur from the main subject position. It may be defined as

  The area indicated by S1 in FIG. 13 is the calculated subject existing space, and the space is set starting from the position at time t5 when the subjects overlap. That is, there is a high possibility that the subject H1 exists in the subject existing space shown in S1 from time t5 to t6, so in order to determine the main subject, the subject set from the position of time t5 when the subjects overlap The face detected in the existing space S1 is reset as a main subject. In this way, if tracking is started again, it is possible to perform focusing without making a mistake in the main subject, and to obtain a good image in focus.

Second Embodiment
Next, a second embodiment will be described. In the first embodiment, since one of the subject's velocity vectors x, y, and z is reversed, the subject existing space does not overlap, and the main subject and the other subjects can be clearly divided. Met. However, for example, in a scene in which two subjects run in parallel, that is, in the case where the directions of the velocity vectors of X, Y and Z are all the same, the subject existing spaces overlap to some extent.

  FIG. 14 shows the main subject H1 and the other subject H2 running in parallel. In this case, the velocity vectors in the X, Y and Z directions of the subjects H1 and H2 are different in amount but in the same direction. Although FIG. 14 shows only in two directions of X and Z for the sake of easy understanding of the description, the present invention is applied to the three-dimensional directions of X, Y and Z.

  The motions of the subject H1 in the X and Z directions in FIG. 14 are represented by xh1 and zh1, respectively, and the subject existing space in that case is represented by SH1. Similarly, the movement in the X direction and the Z direction of the subject H2 is represented by xh2 and zh2, respectively, and the subject existing space in that case is represented by SH2.

  In this case, the subject existing spaces SH1 and SH2 overlap each other in the area indicated by SH3 in FIG. Since the subject H1 and the subject H2 can not be separated in this overlapping space, when the subject is in this space, focus adjustment is performed while tracking one of the subjects.

  As described above, while tracking is continued, the process waits for detection of an object in the difference area between SH1 and SH3. Since only the main subject H1 may exist in the difference area between SH1 and SH3, if the subject detected in this difference area is set again as the main subject and tracking is resumed, it is possible to prevent the shift of the tracking target. Is possible.

  The present invention can be variously modified without departing from the scope of the invention.

  For example, in the above embodiment, the X, Y, and Z directions of the subject are calculated using the tracking function of the imaging apparatus and the focus detection method. Then, as the focus detection method used here, the imaging surface phase difference method is used, which can perform focus detection while performing imaging as described above. However, not limited to this focus detection method, a focus detection method using a widely known contrast detection method may be used. In that case, although the defocus amount to the subject can not be detected, the velocity vector in the Z direction may be defined based on, for example, the lens driving amount which is in focus each time while performing continuous shooting. Also, in the other focus detection methods, any method may be used as long as the velocity vector in the Z direction can be defined.

  Similarly, in the X and Y directions, the above embodiment has been described to use the face tracking function of the imaging apparatus. However, not limited to the face detection of the subject, a color tracking function may be used which detects the color of the subject and detects the moving direction of the detected color from the correlation with the preceding and following frames.

  By using the method described above, even when a subject other than the main subject is mixed, tracking of the main subject is continued without causing a shift in focus, and a good image in which the main subject is in focus is obtained. It becomes possible.

(Other embodiments)
The present invention supplies a program that implements one or more functions of the above-described embodiments to a system or apparatus via a network or storage medium, and one or more processors in a computer of the system or apparatus read and execute the program. Can also be realized. It can also be implemented by a circuit (eg, an ASIC) that implements one or more functions.

101: first lens group 102: aperture / shutter combination 103: second lens group 105: third lens group 107: image pickup device 121: CPU 125: image processing circuit

Claims (14)

In an imaging device having a function of automatically focusing an imaging lens on a subject,
A setting unit configured to set a main subject to be focused by the imaging lens;
Position calculating means for calculating the position of the main subject;
Velocity vector calculating means for calculating the velocity vector of the main subject based on the positions of the main subjects at a plurality of points in the past calculated by the position calculating means;
Space calculation means for calculating a first subject existing space in which the main subject is predicted to be present, using the speed vector calculated by the speed vector calculation means;
The setting unit is configured to use the first subject existing space calculated by the space calculation unit when the distance between the plurality of subjects in the screen is less than or equal to a predetermined distance from the plurality of subjects. An image pickup apparatus characterized by selecting a subject to be focused and resetting it as the main subject.   The velocity vector calculation means calculates the direction and amount of movement of the main subject in the optical axis direction of the imaging device, and the horizontal and vertical directions on the imaging surface of the imaging device orthogonal to the optical axis. The imaging device according to claim 1, characterized in that:   3. The imaging according to claim 1, wherein the velocity vector calculating unit calculates the direction and the amount of movement of the main subject based on defocus amounts of the main subject at the plurality of points in the past. apparatus.   3. The apparatus according to claim 1, wherein the velocity vector calculating unit calculates the direction and the amount of movement of the main subject based on the amount of movement for focusing adjustment of the imaging lens at the plurality of points in the past. The imaging device according to.   The space calculation means is a vector in the optical axis direction of the imaging device calculated using the velocity vector calculation means, and a vector in the horizontal direction and a vector in the vertical direction on the imaging surface of the imaging device orthogonal to the optical axis. The image pickup apparatus according to any one of claims 1 to 4, wherein the subject existing space is calculated using three vectors of.   The space calculating means calculates a second subject existing space in which it is predicted that a subject which is not the main subject is present even for a subject which is not the main subject. The imaging device according to.   When the setting unit has an area in which the first subject existing space and the second subject existing space overlap, in the overlapping area, either the main subject or the subject which is not the main subject is selected. 7. The image pickup apparatus according to claim 6, wherein the image pickup apparatus is selected as an object to be focused.   The setting means is detected in a difference area obtained by subtracting the overlapping area from the first object existing space when the first object existing space and the second object existing space have an overlapping area. The imaging apparatus according to claim 6, wherein a subject is reset as the main subject.   The image pickup apparatus according to any one of claims 1 to 8, wherein the setting unit sets the main subject in response to an instruction from a user.   The image pickup apparatus according to any one of claims 1 to 8, wherein the setting unit sets the main subject based on an image signal output from an image pickup device.   The setting means sets the main subject using at least one of a detection result of the face of the subject, a distance of the subject, a color of the subject, and a velocity vector calculated using the velocity vector calculating means. The imaging device according to claim 10. A method of controlling an imaging apparatus having a function of automatically focusing an imaging lens on a subject, comprising:
A setting step of setting a main subject to be focused on the imaging lens;
A position calculating step of calculating the position of the main subject;
A velocity vector calculating step of calculating a velocity vector of the main subject based on the positions of the main subjects at a plurality of past points calculated by the position calculating step;
Calculating a first subject existing space in which the main subject is predicted to be present, using the speed vector calculated in the speed vector calculating step;
In the setting step, when the distance between a plurality of subjects in the screen approaches a predetermined distance or less, the first subject existing space calculated by the space calculating step is used to calculate the distance from the plurality of subjects. A control method of an image pickup apparatus, wherein an object to be focused is selected and reset as the main object.   The program for making a computer perform each process of the control method of Claim 12.   A computer readable storage medium storing a program for causing a computer to execute each step of the control method according to claim 12.

Images