JP6552408B2 - Image pickup apparatus and control method thereof - Google Patents

Description

  The present invention relates to control of continuous focus shooting.

  The imaging apparatus performs focus detection by reading out an image signal necessary for obtaining an AF (autofocus) evaluation value from the imaging device. At this time, there is a technique for performing high-speed focus detection by reading a still image signal or the like faster than a normal reading speed.

  Patent Document 1 discloses a technique for reading out only a pixel signal of a specific area from an effective pixel area from an image sensor and acquiring an AF evaluation value. Japanese Patent Application Laid-Open No. 2004-228561 discloses a technique for reading out a specific segmented area from among previously segmented still image readout areas before acquiring other segmented areas and obtaining an AF evaluation value.

JP 2012-58464 A JP 2011-166515 A

  In the prior art disclosed in Patent Document 1, a signal for the entire effective pixel area is read to detect a subject, and a reading area for acquiring an AF evaluation value is determined. When a signal is read from the entire effective pixel area, a portion of the signal that does not correspond to the subject image is also read. For this reason, useless reading time is consumed, and it may take time until the AF evaluation value is acquired. Further, in the prior art disclosed in Patent Document 2, a signal for obtaining an AF evaluation value is read out in parallel with the reading of a still image signal. The readout area for acquiring the AF evaluation value is a specific partial area within the readout area of the still image signal divided in advance. When a signal is read from this partial area, if the subject moves, there is a high possibility that the subject will not appear in the set area. Therefore, the detection accuracy and the tracking performance of the subject may be reduced.

  An object of the present invention is to realize high-speed continuous shooting by subject tracking control while shortening the time taken to determine the focus position in autofocus control.

  An apparatus according to an embodiment of the present invention is an imaging apparatus that performs autofocus control and subject tracking control in continuous shooting, and reads a first image signal related to a captured image from an imaging element. Reading means, second reading means for reading a second image signal used for calculation of the evaluation value of the autofocus control from the image sensor, and subject detection for detecting a subject using the first image signal And a position at which the second image signal is read from the subject detection information acquired by the subject detection means, and the evaluation value is determined from the second image signal read by the second readout means. And controlling means for controlling the focus adjustment. The control means calculates a detection start time by the subject detection means from a continuous shooting time interval, and the subject is being read while the first readout means is reading the first image signal from the image sensor. The detection unit starts detection of the subject at the start time, performs subject detection using the read first image signal, and the second reading unit starts from the position determined by the control unit. The second image signal is read out.

  According to the present invention, it is possible to realize high-speed continuous shooting by subject tracking control while reducing the time required for determining the focal position by autofocus control.

FIG. 1 is a schematic view showing an entire configuration of an imaging device according to an embodiment of the present invention. It is an operation | movement timing chart of the imaging process by 1st Embodiment. It is a flowchart which shows the operation | movement of the imaging process by 1st Embodiment. It is an operation | movement timing chart of the imaging process by 2nd Embodiment. It is a flowchart which shows the operation | movement of the imaging process by 2nd Embodiment. It is a flowchart which shows the operation | movement of the imaging process by 3rd Embodiment. It is an operation timing chart of an imaging process and an explanatory diagram of a reading mode according to the third embodiment. It is a flowchart which shows the operation | movement of the imaging process by 4th Embodiment. It is a flowchart which shows the operation | movement of the imaging process by 5th Embodiment. It is a flowchart which shows the operation | movement of the imaging process by 6th Embodiment. It is a flowchart which shows the calculation process of the face detection start time by 6th and 7th embodiment. It is explanatory drawing of the main face prediction of the imaging process by 6th Embodiment. It is explanatory drawing regarding the start position of the face detection of the imaging process by 6th Embodiment. It is explanatory drawing of the read-out direction of the sensor of the imaging process by 6th Embodiment. It is explanatory drawing which shows an example of the face detection method of the imaging process by 6th Embodiment. It is a flowchart which shows the operation | movement of the imaging process by 7th Embodiment. It is a flow chart which shows decision processing of AF reading field by a 7th embodiment. It is explanatory drawing of the main face prediction of the imaging process by 7th Embodiment.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.
First Embodiment
The configuration of the imaging device according to the first embodiment of the present invention will be described with reference to FIG. FIG. 1 is a diagram illustrating an overall configuration of the imaging apparatus 100. As the imaging apparatus 100, a digital camera will be exemplified, and each part will be described with the subject side defined as the front side.

  The first lens group 101 is disposed at the tip of the imaging optical system and is held so as to be able to advance and retract along the optical axis direction. The diaphragm / shutter 102 adjusts the light amount at the time of shooting by adjusting the aperture diameter, and also functions as an exposure time adjustment shutter at the time of still image shooting. The second lens group 103 is integrated with the stop / shutter 102 to advance and retract in the optical axis direction, and a magnification change operation (zoom function) is performed in conjunction with the advance / retreat operation of the first lens group 101. The third lens group 105 is a focusing lens that moves in the optical axis direction to perform focusing. The optical low-pass filter 106 is an optical element for reducing false colors and moire in the captured image. The image sensor 107 has a two-dimensional CMOS (complementary metal oxide semiconductor) photosensor and a peripheral circuit, and is disposed on the imaging plane of the image pickup optical system.

  The zoom actuator 111 is a drive unit that moves the first lens group 101 to the third lens group 105 in the optical axis direction to perform a zooming operation by rotating a cam barrel (not shown). The aperture shutter actuator 112 controls the aperture diameter of the aperture / shutter 102 to adjust the imaging light amount, and performs exposure time control at the time of still image imaging. The focus actuator 114 is a drive unit that adjusts the focus by moving the third lens group 105 in the optical axis direction.

  An illumination unit (electronic flash for subject illumination) 115 used at the time of photographing is a flash illumination device using a xenon tube or an illumination device including LEDs that emit light continuously. The electronic flash control circuit 122 controls lighting of the illumination unit 115 in synchronization with the photographing operation.

  An autofocus (hereinafter referred to as AF) auxiliary light source unit 116 projects an image of a mask having a predetermined opening pattern onto a subject via a light projection lens. Thereby, it is possible to improve the focus detection capability for a dark subject or a low contrast subject. The auxiliary light source drive circuit 123 controls the lighting of the AF auxiliary light source unit 116 in synchronization with the focus detection operation.

  The control unit 121 is a central unit that performs various controls in the imaging apparatus 100. The control unit 121 includes a CPU (Central Processing Unit) (not shown), an A (analog) / D (digital) converter, a D / A converter, a communication interface circuit, and the like. The image sensor drive circuit 124 controls the image capturing operation of the image sensor 107 and A / D converts the image signal acquired from the image sensor 107 and outputs the image signal to the control unit 121. The image processing circuit 125 performs subject detection processing using the image data acquired by the image sensor 107, γ conversion, color interpolation, JPEG (Joint Photographic Experts Group) compression, and the like on the image data.

  The focus drive circuit 126 drives and controls the focus actuator 114 based on the focus detection result by the control unit 121. The focus actuator 114 moves the third lens group 105 in the optical axis direction to perform focusing. The aperture shutter drive circuit 128 drives and controls the aperture shutter actuator 112 in accordance with a control command of the control unit 121 to control the opening of the aperture / shutter 102. The zoom drive circuit 129 drives the zoom actuator 111 according to the control command of the control unit 121 according to the zoom operation by the photographer.

  The display device 131 includes, for example, a display device such as an LCD (liquid crystal display device). The display device 131 displays information on a shooting mode of the imaging device 100, a preview image before shooting, a confirmation image after shooting, a display image of a focused state at the time of focus detection, and the like. The operation unit 132 includes a power switch, a release (shooting trigger) switch, a zoom operation switch, a shooting mode selection switch, and the like. The release switch is a two-stage switch in which the first switch (SW1) and the second switch (SW2) are sequentially turned on according to the amount of pressing. SW1 is turned on when the release switch is pressed about halfway, and SW2 is turned on when the release switch is pushed to the end.

  The recording medium 133 is a flash memory or the like that can be attached to and detached from the camera main body, and records an image file acquired by photographing. The memory 134 is a RAM (random access memory), a ROM (read only memory) or the like, stores a predetermined program, and holds image data during image processing and parameter data required for image processing. The control unit 121 drives various circuits of the imaging apparatus 100 based on a predetermined program stored in the ROM constituting the memory 134, and executes a series of operations such as AF control, imaging processing, image processing, and recording processing. Do.

  Next, with reference to FIGS. 2 and 3, high-speed continuous shooting by subject tracking control performed by the imaging apparatus 100 will be described. This continuous shooting is continuous shooting by AF control following the subject, and is hereinafter referred to as tracking AF continuous shooting. FIG. 2 is a timing chart showing the timing of the operation of the follow-up AF continuous shooting in the present embodiment. FIG. 3 is a flow chart for explaining the operation at the time of follow-up AF continuous shooting in the present embodiment. It is assumed that the shooting mode of the imaging apparatus 100 is set to the tracking AF continuous shooting mode by a user operation of a shooting mode selection switch included in the operation unit 132. In the shooting mode, a shooting operation is started by turning on a release switch included in the operation unit 132, and the process proceeds to S301.

  In step S301, exposure to the image sensor 107 is started to capture a still image, and charge is accumulated. The exposure time is determined by driving control of the aperture shutter actuator 112. At this time, the image sensor drive circuit 124 controls the image sensor 107 in a drive mode in which an image signal of an area including the entire effective pixel area of the image sensor 107 is read. The processing timing is after the start of shooting, as indicated by the still image storage 201 in FIG.

  In step S302, the imaging device driving circuit 124 A / D converts an image signal obtained from the charges accumulated in the imaging device 107 and starts processing for outputting the image signal to the control unit 121. According to the drive mode set in S301, an image signal of an area including the entire effective pixel area is read out. The timing of the processing is after completion of the processing of the still image storage 201 as shown by still image reading 202 in FIG.

  In step S303, the image signal obtained from the charge accumulated in the imaging element 107 is A / D converted, and is output to the control unit 121 one by one or in units of a plurality of pixels. The still image readout 202 in FIG. 2 is performed, and the subject detection processing starts from a certain point in time. The start timing of the subject detection process is determined so as to be the completion timing of reading up to an AF processable area. Even if a subject image is detected in an area where AF processing is not possible and AF evaluation values are calculated in the vicinity thereof, AF processing can not be performed after all. That is, when the subject is detected based on the read image signal when the reading is completed up to the area where AF processing is possible, the necessary AF evaluation value can be calculated. Therefore, the time when the reading of the image signal up to the AF processable area is completed is set as the start time of the subject detection process. Since the AF processable area varies depending on the shooting mode and the type of lens, the control unit 121 changes the start time of the subject detection process according to the conditions. Alternatively, the start point of the subject detection process may be determined based on the speed setting value of continuous shooting. When the speed setting value is large, it is necessary to further shorten the processing time between shooting of the next shooting. Thus, by performing control to advance the start point of the subject detection process, it is possible to shorten the processing time between successive shootings. In that case, since the number of image signals for which reading has been completed is reduced, the object detection accuracy is lowered, but it is possible to further increase the speed of continuous shooting.

  A subject detection process (see 203 in FIG. 2) starts in S304. The subject detection process is executed by the image processing circuit 125 based on the image signal acquired by the control unit 121. Although described later, the information of the detection result necessary for the AF process is the position information of the subject. In addition, there is a restriction on the start time of the AF process, and it is necessary that the subject detection process has been completed by the start time of the AF process. For example, assume that a large number of subjects are photographed. In this case, the subject detection process may not be completed in time before the AF process starts. In that case, the position information of the subject detection processing result acquired in the previous shooting in the continuous shooting is used instead. Alternatively, default position information set in advance is used in the AF process. The start timing of the subject detection processing 203 is in the middle of the still image readout 202. The end timing of the subject detection process is a time point at which reading of an image signal is completed up to an AF processable region or a time point at which the tracking AF continuous shooting speed being set can be guaranteed.

In S305, the readout processing of the image signal of the remaining still image and the subject detection processing are executed in parallel. The start timing of the AF processing is determined such that the following processing is completed for the next shooting start time determined by the speed of continuous shooting.
An area calculation process for reading an image signal for calculating an AF evaluation value (hereinafter referred to as an AF image signal).
Accumulation processing for reading out an AF image signal (FIG. 2: AF accumulation 204).
AF image signal readout processing (FIG. 2: AF readout 205).
AF evaluation value calculation processing and focus adjustment lens drive amount calculation processing (FIG. 2: drive amount calculation 206).
A focus adjustment lens drive process (FIG. 2: lens drive 207).
In this specification, an area for reading an AF image signal is referred to as an AF reading area.

  In S306, the AF readout area is calculated from the detection signal of the subject detection. As a premise, the reading of the AF image signal is controlled by driving the image sensor driving circuit 124 separately from the reading of the still image signal, and the image signal in the effective pixel region of the image sensor 107 is partially read out. Shall be At this time, processing is performed to determine the position where the image signal is partially read out with respect to the effective pixel area, with the subject position as the center, using the subject position information that is the subject detection result. The processing timing is included in the AF accumulation 204 of FIG. 2, but is executed at the start of the AF processing determined as described above.

  In S307, exposure to the image sensor 107 is started to calculate the AF evaluation value, and charges are accumulated. The exposure time is set by driving control of the aperture shutter actuator 112. Unlike the case of a still image, the image sensor driving circuit 124 controls the image sensor 107 by a driving method (hereinafter referred to as a partial reading method) that reads an image signal from a part of the effective pixel region of the image sensor 107. . The timing of the processing is after the AF readout area calculation processing.

  In step S <b> 308, the imaging device driving circuit 124 A / D converts an image signal obtained from the charge accumulated in the imaging device 107 and starts processing for outputting the image signal to the control unit 121. The image signal is read out from a part of the effective pixel area in accordance with the drive method set in S307. The timing of the processing is after completion of the AF accumulation 204, as indicated by the AF readout 205 in FIG.

  In step S309, the control unit 121 executes calculation processing of an AF evaluation value based on the acquired image signal, and calculates a lens driving amount for focusing on the subject in the next photographing. The timing of the process is after the completion of the AF read-out 205, as shown by the drive amount calculation 206 in FIG. In S310, drive control of the focus drive circuit 126, the aperture shutter drive circuit 128, or both is performed according to the lens drive amount calculated by the control unit 121. By driving the focus actuator 114, the focusing lens is moved to the in-focus position corresponding to the subject.

  In step S311, the control unit 121 determines whether the release switch (second switch SW2) included in the operation unit 132 is on or off. If SW2 is on, the process returns to S301, and continuous shooting is continued at the in-focus position in S310. When SW2 is off, the continuous shooting is stopped and the shooting ends. After the lens drive 207 shown in FIG. 2, the process of determining whether the switch SW2 is on or off in S311 is performed. When SW2 is on, the next shooting is started. The processing of the still image storage 208 and the still image readout 209 is executed to continue the imaging processing.

  In the present embodiment, subject detection processing is started during reading of a still image signal. Thereby, the parallel processing of the subject detection processing and the still image reading processing is executed, so that the processing time between consecutive photographing can be further shortened. In addition, the AF readout area is set to a partial area of the effective pixel area of the image pickup device based on the subject position, and the drive mode of the partial readout method is set. As a result, it is possible to further reduce the time required to read the image signal while maintaining the accuracy of subject detection and tracking.

  According to the present embodiment, it is possible to shorten the processing time between successive shootings and the readout time of the image signal for autofocus control while maintaining the accuracy of object detection and tracking. Therefore, it is possible to further increase the speed (continuous shooting speed) of continuous shooting following an object while suppressing the time taken to determine the focus position of the next shooting.

Second Embodiment
The follow-up AF continuous imaging performed by the imaging device according to the second embodiment of the present invention will be described with reference to FIGS. 4 and 5. FIG. 4 is a timing chart showing the timing of the operation of the follow-up AF continuous shooting in the present embodiment. The processes 401 and 402 correspond to the processes 201 and 202 in FIG. 2, and the processes 405 to 410 correspond to the processes 204 to 209 in FIG. 2, respectively. FIG. 5 is a flowchart showing the operation of the continuous AF continuous shooting in the present embodiment. The present embodiment is different from the first embodiment in the content of subject detection processing. About the same composition as a case of a 1st embodiment, the detailed explanation of those is omitted by using the code already used, and difference is mainly explained. Omitting such description is the same in the embodiments described later.

  The difference between FIG. 4 and FIG. 2 is that an object detection process 403 and a face detection process 404 are performed in subject detection. The thing detection process 403 is a first detection process that is less accurate than the face detection process 404 because it does not determine the content (person, object, etc.) or the number of subjects. A mono detection process is performed from color information based on the input image signal. By adjusting the detection accuracy parameter, the position and size information of the main subject can be output as a detection result by the time when the AF process is started. That is, the first detection process is a detection process that prioritizes the detection speed.

  The face detection process 404 is a second detection process with high accuracy for detecting the face of the subject when the subject is a person. For example, it is possible to determine whether or not the face is a subject as a detection result, and to acquire information such as the number of subjects, the position and size of each subject, and the order of importance. In the case of shooting when the number of faces of the subject and other objects is large, the detection process has a high load, so the time taken to output the detection result takes longer than the mono detection process 403. Face detection processes 404a and 404b shown in FIG. 4 are detection processes that prioritize detection accuracy.

  In FIG. 5, the processing of S501 and S502 is the same processing as S301 and S302 of FIG. 3, and the processing of S509 to S513 is the same processing as S307 to S311 of FIG. Therefore, the processes of S503 to S508, which are differences, will be described.

  In S503, an image signal obtained from the electric charge accumulated in the image sensor 107 is A / D converted and output to the control unit 121. In S504, subject detection is started as in S304. As shown in FIG. 4, the timing at which the mono detection process 403 and the face detection processes 404a and 404b start is in the middle of reading a still image signal. The mono detection process 403 and the face detection process 404 are started simultaneously.

  In S505, the still image signal reading process and the two types of subject detection processes are executed in parallel. The start time point of the AF process is determined as the time point when each process described above is completed with respect to the next shooting start time determined by the speed of continuous shooting, as in S305.

In step S506, the control unit 121 determines the detection results of the two types of subject detection processing started in step S504. The determination time point is the time when the start time of the AF process is reached after the processes of the thing detection process 403 and the face detection processes 404a and 404b start. At this time, determination processing is performed as to whether the detection result by the face detection processing 404 is acquired. The face detection process 404 is a detection process in which priority is given to detection accuracy, and since detection takes time compared to the thing detection process 403, there are the following cases at the determination time.
The thing detection process 403 can output the detection result, but the face detection process 404a can not output the detection result.
The case where the mono detection process 403 and the face detection process 404b can output the detection result.

  The control unit 121 determines whether or not the detection result of each subject detection can be output by the start time of the AF process, and if both the mono detection process and the face detection process can output the detection result, the process proceeds to S507. Proceed with the process. If only the product detection process can output the detection result, the process proceeds to step S508.

  In S507, processing for calculating an AF read area is executed using the detection result of the face detection processing. For example, when the number of subjects is one, the reading area is determined around the image position of the subject. When a plurality of subjects are detected, the reading area is determined with the image position of the subject having the highest order in the image as the center. When face information is used in determining the AF readout area, processing is performed to determine whether to determine the readout area around the pupil position or the readout area at the center of the face itself depending on the face size. Reading of the AF image signal is started with respect to the determined AF reading area.

  In S508, processing for calculating the AF read area is executed using the detection result of the mono detection processing. The detection result in this case is detection information for one subject, and an AF read area is determined around the image position of the subject, and reading of the AF image signal is started.

  The timing of the processing in S507 or S508 is included in the AF accumulation 405 in FIG. 4, but is the timing of starting the AF processing. After the process of S507 or S508, the process proceeds to S509. At the time of reading in S510, the image sensor 107 is driven and controlled by the partial reading method in the same manner as in S308 of FIG.

  In the present embodiment, two types of subject detection methods are performed simultaneously, and subject detection and tracking control can be performed while guaranteeing the continuous shooting speed by the first detection process that prioritizes the detection speed. In addition, when the detection result of the second detection process giving priority to detection accuracy can be acquired at the start of the process in the autofocus control, the AF process can be performed using the detection result with higher accuracy. Therefore, the subject tracking performance is improved.

Third Embodiment
Hereinafter, a third embodiment of the present invention will be described with reference to FIGS. 6 and 7. The imaging apparatus according to the present embodiment reads out an appropriate AF image signal according to the state of the subject. The condition of the subject is, for example, the luminance, the size of the subject image or the amount of movement.

  FIG. 6 is a flowchart showing the processing of tracking AF continuous shooting performed by the control unit 121 in the present embodiment. Since the processing from S601 to S608 and the processing from S613 to S617 are the same as the processing from S501 to S508 and the processing from S509 to S513 described in FIG. 5 in the second embodiment, their descriptions are omitted.

  It progresses to S609 after the process of S607 and S608. In step S609, the control unit 121 calculates a luminance value within a range corresponding to the AF reading area determined in step S607 or S608 in the still image captured in steps S601 and S602. The calculated luminance value is compared with the luminance threshold value stored in advance in the memory 134. If the calculated luminance value is smaller than the luminance threshold and it is determined that the luminance is low (YES in S609), the process proceeds to S610. On the other hand, when it is determined that the calculated luminance value is equal to or higher than the luminance threshold and not low luminance (NO in S609), the process proceeds to S611.

  In step S610, the control unit 121 determines the size of the subject image obtained from the face detection result and the thing detection result in step S604 and the amount of movement by comparing with the threshold value. The threshold values are a first threshold value for determining the size of the subject image and a second threshold value for determining the amount of movement of the subject image, and are stored in the memory 134 in advance. When the control unit 121 determines that the size of the subject image is smaller than the first threshold and the amount of motion of the subject image is smaller than the second threshold (NO in S610), the process proceeds to S612. On the other hand, if it is determined that the size of the subject image is greater than or equal to the first threshold value, or the amount of motion of the subject image is greater than or equal to the second threshold value (YES in S610), the process proceeds to S611. The luminance threshold value and the first and second threshold values used in S609 and S610 are fixed values or variable values. In the case of a variable value, the user may be able to set by the operation unit 132 before shooting.

  In step S611, the control unit 121 sets the thinning readout mode in order to reduce the readout amount of the image signal. In the thinning-out reading mode, the image sensor driving circuit 124 is set so that the AF reading process is executed in the first range including the AF reading area determined in S607 or S608.

  In step S <b> 612, the control unit 121 sets a crop reading mode that limits a reading area used for AF processing. The second range set in the crop read mode is narrower than the first range in the thinning read mode. In the crop reading mode, the reading process of the AF image signal is set to be performed in a second range including the AF reading area determined in S 607 or S 608.

  The relationship between the determination processing in S609 and S610 and the AF readout mode will be described with reference to FIG. In FIG. 7, a still image 417 has a luminance value in the AF reading area that is greater than or equal to a threshold value, a luminance value that is less than the threshold value, and the size of the subject image is greater than or equal to the first threshold value, or a subject. The image in the case where the amount of movement of an image is more than a 2nd threshold is illustrated. Each process relating to the still image 417 is shown from the still image storage 401 to the lens drive 408. The still image 419 is an image when the luminance value in the AF readout area is less than the threshold, the size of the subject image is smaller than the first threshold, and the amount of motion of the subject image is smaller than the second threshold. To illustrate. Each process relating to the still image 419 is shown from the still image storage 409 to the lens drive 416.

  In the case of the still image 417, thinning-out reading 418 is performed in the AF process. In the thinning-out read mode, the image sensor driving circuit 124 is set to read an image signal partially from a range which is narrower than the entire screen and includes the portion of the subject image. On the other hand, in the case of the still image 419, crop reading 420 is performed in which only the subject image portion is read in the AF process. That is, in the crop reading mode, the image sensor driving circuit 124 is set to read all image signals within a limited range as compared with the thinning readout mode.

  After the AF read mode setting process in S611 or S612 in FIG. 6, the process proceeds to S613, and the AF process is executed. In the present embodiment, the AF readout mode is determined by comparing the size of the subject image and the amount of movement with the threshold value in S610. If the face detection result is acquired in S606, the subsequent processing is performed with the face as the subject and when the face detection result is not acquired with the object as the subject.

  In the present embodiment, when the size of the subject image and the amount of movement are small, a crop reading mode in a narrow range is set. However, the amount of information necessary for autofocus control can be acquired as the amount of information in the reading area. For this reason, it is possible to improve the autofocus control performance even when the luminance of the subject is low without degrading the performance of the continuous AF tracking.

Fourth Embodiment
Next, a fourth embodiment of the present invention will be described with reference to FIG. FIG. 8 is a flowchart illustrating processing for continuous AF tracking in the present embodiment. The processes from S701 to S707 and the processes from S712 to S716 shown in FIG. 8 are the same as the processes from S501 to S507 and S509 to S513 in FIG.

  When the process proceeds from step S706 to step S708, the control unit 121 determines an AF readout area from the result of the thing detection by the image processing circuit 125, and proceeds to step S711. In step S707, the control unit 121 performs luminance determination processing. In the still images taken in S701 and S702, the brightness within the range corresponding to the AF readout area determined in S707 is calculated and compared with the brightness threshold stored in the memory 134 in advance. If the calculated luminance is less than the luminance threshold and it is determined that the luminance is low (YES in S709), the process proceeds to S710. On the other hand, when the calculated luminance is equal to or higher than the luminance threshold and it is determined that the luminance is not low (NO in S709), the process proceeds to S711.

  In step S710, the control unit 121 sets the mode of the imaging element drive circuit 124 to the crop reading mode. In step S711, the control unit 121 sets the mode of the imaging device drive circuit 124 to the thinning readout mode. The readout range set in the crop readout mode is narrower than the readout range set in the thinning readout mode. That is, when only the mono detection result can be acquired by subject detection, the subject tracking performance in the image processing circuit 125 is inferior to that when the face detection result is acquired. For this reason, in S711, information in a wide reading range is required to find a target in the AF process.

  According to the present embodiment, when only the detection result by the first detection process, which is lower in detection accuracy than the second detection process, can be acquired, compared to when the detection result by the second detection process is acquired. Thus, a wider range of read processing is performed. That is, since it is difficult to lose sight of the AF target by setting the thinning readout mode, the continuity of the autofocus control in the continuous AF continuous shooting can be maintained.

Fifth Embodiment
Next, a fifth embodiment of the present invention will be described with reference to FIGS. FIG. 9 is a flowchart showing processing of continuous AF continuous shooting in the present embodiment. The processes from S801 to S812 and the processes from S814 to S818 shown in FIG. 9 are the same as the processes from S601 to S612 and S613 to S617 in FIG. 6 described in the third embodiment, and thus description thereof is omitted. . It progresses to S813 after the process of S811 or S812.

  In step S813, the control unit 121 calculates the AF accumulation time in step S814 and the AF read time in step S815. In order to maintain the AF accuracy of the follow-up AF continuous shooting, the processing of the AF accumulation 413 and the AF reading 414 in FIG. 7 needs to be completed within a predetermined time until the next shooting start time comes. Although the total processing time of the AF accumulation 413 time and the AF read time 414 time is constant, the AF read time varies depending on the AF read area and the read mode. The AF read area is a read area set in the mode of S811 or S812, and the read mode is a thinning read mode or a crop read mode. When the luminance of the subject area is lower than the luminance threshold and low, the control unit 121 adjusts by extending the AF accumulation time within the total processing time when the AF read time is shorter than a predetermined time. The predetermined time is a time for determination determined by the time of imaging processing and autofocus processing and the time interval of continuous shooting. In addition, when the subject area does not have low luminance, the control unit 121 sets the AF accumulation time to the same value as in the case of reading out the still image signal or a value suitable for the AF process.

  In the present embodiment, since the AF read time is calculated according to the AF read region and the read mode and the AF accumulation time is increased when the subject region has low luminance, the processing for increasing the luminance of the AF image is performed. Accurate autofocus control is possible.

Sixth Embodiment
With reference to FIGS. 10 to 14, the following AF continuous shooting in the sixth embodiment of the present invention will be described. In the present embodiment, the position of the subject (main face) desired by the user is predicted, and processing for advancing the start timing of face detection is performed.

  FIG. 12A is a diagram illustrating an arrangement example of the main face image on the light receiving surface of the image sensor 107, for example. On the two-dimensional CMOS photosensor, the position of the photographed main face image is represented by a rectangle. The left and right direction in FIG. 12A is defined as the x direction, and the up and down direction is defined as the y direction. The first main face position 1101 of the first shot, the second main face position 1102 of the next shot, and the third main face position 1103 of the next shot are illustrated.

  FIG. 12B is an explanatory diagram of the predicted position of the main face. The vertical axis indicates the center position of the captured main face image in the y direction, and the horizontal axis indicates the number of shots. For example, the point 1104 indicates the center position of the first main face image, the point 1105 indicates the center position of the second main face image, and the point 1106 indicates the center position of the third main face image. Here, a point 1107 indicates the predicted center position of the fourth main face image. Specifically, a position predicted by a regression line from the positions of the three points 1104, 1105, and 1106 is shown by a point 1107.

  FIG. 13A shows a predicted range 1201 of the main face image position based on the center position of the main face image predicted at the point 1107 in FIG. The left and right direction in FIG. 13A is defined as the x direction, and the up and down direction is defined as the y direction. The prediction range 1201 shown in FIG. 13A is determined in consideration of the size of the main face image, variation in prediction, and the like. As an example, a case where the prediction range 1201 is a range within M (= 50)% of the maximum value in the y direction is shown.

  FIG. 13B is an explanatory diagram of a position at which face detection is started. As shown in FIG. 13A, when the prediction range 1201 is within 50% on the two-dimensional CMOS photosensor, the reading process of the still image signal at the position corresponding to 50% is completed, and then the face Detection starts. Specifically, when 50% of the still image readout 402 shown in FIG. 4 is completed, the face detection process 404a shown in FIG. 4 starts. In other words, the face detection process is not performed until the reading of the image signal in the range of 50% which is predicted to include the position of the face image is not completed.

A process for predicting the position of the main face image and advancing the start of face detection will be described with reference to the flowcharts of FIGS. 10 and 11.
First, in step S1001 of FIG. 10, the control unit 121 performs calculation processing of the face detection start time, and determines the face detection start time and the still image reading direction. Details of the processing will be described later with reference to FIG. The still image storage process of the next S1002 is the same as S501 of FIG. 5 described in the second embodiment, and thus the description thereof is omitted.

  In S1003, readout of the still image signal starts in the readout direction determined in S1001. Based on the result of S1001, the control unit 121 sets the readout direction of the pixel signal accumulated in S1002 to the image sensor driving circuit 124. The reading direction will be described later with reference to FIG. S1004 is a standby process until the face detection start time determined in S1001 elapses. The processing from S1005 to S1009 is the same as the processing from S504 to S508 in FIG. 5 described in the second embodiment, and thus description thereof is omitted.

  Following S1008, the process proceeds to S1010, and the control unit 121 stores the face detection result in the memory 134 as face detection history information. This is processing for storing the face detection result in the memory 134 for the purpose of use in S1001. Specifically, the center coordinates of the detected face image, the size in the x direction and the size in the y direction of the face image based on the center coordinates, the feature amount of the face, and the like are stored in the memory 134 as the face detection result. After storing the face detection history information, the process proceeds to S1011. The processes of S1011 to S1015 are the same as the processes of S509 to S513 of FIG. 5 described in the second embodiment, and thus the description thereof will be omitted.

  The face detection start time calculation process executed in S1001 of FIG. 10 will be described in detail with reference to the flowchart of FIG.

  In step S1016, the control unit 121 determines whether there is a face detection history. In step S1010 of FIG. 10, the face detection history information stored in the memory 134 is acquired, and it is determined whether there is a past face detection history. For example, when there is no continuous face detection result in the face detection results of the face detection history, the control unit 121 determines that there is no face detection history. In the N-th still image shooting, when there is no face detection result of the N−1th, N−2th, and N−3th images continuously captured in the past, the control unit 121 detects the face It is determined that there is no detection history. If it is determined in S1016 that there is a face detection history (YES in S1016), the process proceeds to S1017. If it is determined that there is no face detection history (NO in S1016), the process proceeds to S1020.

  In step S1017, the control unit 121 acquires a face detection history, and calculates a prediction range of the main face position. For example, the main face position prediction range 1201 in FIG. 13A is determined. In step S1018, the control unit 121 determines the reading direction of the still image signal from the predicted range of the main face position determined in step S1017.

  The process of determining the reading direction will be specifically described with reference to FIG. In FIG. 14, the left and right direction is defined as x direction, and the up and down direction is defined as y direction. With the intersection of the x axis and the y axis as the origin, the right direction is the positive direction of x, and the downward direction is the positive direction of y. FIG. 14A exemplifies the case where the prediction range of the main face position is the top 50% (0% ≦ y ≦ 50%) of the two-dimensional CMOS photosensor. FIG. 14B exemplifies the case where the prediction range of the main face position is the lower 50% (50% ≦ y ≦ 100%) of the two-dimensional CMOS photosensor.

  In the case of FIG. 14A, in the prediction range (0% ≦ y ≦ 50%), the direction is set so that the image signal is read in the positive direction in both the x and y directions starting from the origin. Direction) is set. On the other hand, in the case of FIG. 14B, the image signal is read in the negative direction in both the x and y directions starting from the lower right corner in the prediction range (50% ≦ y ≦ 100%). (Second direction) is set. The second direction is a direction opposite to the first direction. The start of face detection can be accelerated by changing the reading direction on the two-dimensional CMOS photosensor according to the prediction range.

  In S1019 of FIG. 11, the control unit 121 calculates a face detection start time. The control unit 121 calculates the time until reading of the image signal in the predicted range of the main face position determined in S1017 is completed.

  When the process proceeds from S1016 to S1020, there is no face detection history. In this case, the control unit 121 sets the reading direction of the still image signal as a default direction (first direction in FIG. 14A: the direction from the upper left to the lower right) and performs reading from the origin position. In the next S1021, the control unit 121 sets, as the face detection start time, the maximum time (MAX time) required to read out the image signal from the face detectable read area. After the processing of S1019 and S1021, the process proceeds to return processing.

According to the present embodiment, it is possible to perform autofocus control that follows the main face by predicting the position of the subject (main face) desired by the user and by accelerating the start time of face detection that requires processing time.
In the present embodiment, the main face position prediction process using the regression line has been described as an example. However, the present invention is not limited to this. For example, the main face position is predicted using an approximate expression such as a moving average method. May be. Further, as shown in FIG. 15, when the size of the image of the main face becomes a predetermined size (size threshold) (see the third image), the face is detected by detecting only the eyes by face detection. The start time of detection may be advanced.

Seventh Embodiment
Follow-up AF continuous shooting in the seventh embodiment will be described with reference to FIGS. 16 to 18. In the present embodiment, determination processing of an AF readout area when a face detection result is not obtained in a prediction range in the sixth embodiment will be described. FIG. 18 is an explanatory diagram of AF read area determination processing. The horizontal direction is defined as the x direction, and the vertical direction is defined as the y direction.

  FIG. 18A illustrates a case where the main face image is not detected in the prediction range (0% ≦ y ≦ 50%) and the face detection result is not acquired. FIG. 18B illustrates a case where face detection is successful within the prediction range, but the main face 1501 does not exist within the prediction range. The prediction range is the upper 50% range (0% ≦ y ≦ 50%), and an example is shown in which the face of another subject is detected in this range. As shown in FIGS. 18A and 18B, when the main face is not detected within the prediction range, the main face may exist outside the prediction range.

  FIG. 18C shows an example in which a region outside the prediction range is set as the AF readout region 1502 when the main face is not detected in the prediction range. FIG. 18D shows the predicted position 1503 of the main face when no main face is detected in the prediction range. Regarding the predicted position 1503 of the main face, for example, the size of the main face image in the y direction is the maximum value of the AF read area, and the center position of the AF read area in the y direction is the center position of the main face image in the y direction. In addition, with regard to the center position of the face image in the x direction, for example, the center position of the main face image in the x direction in the past consecutive N-1th, N-2th, and N-3rd images is regressed The position predicted by a straight line. Thus, the control unit 121 calculates the predicted position of the main face in an area other than the first predicted range.

  The process of determining the AF reading area will be described with reference to the flowcharts of FIGS. 16 and 17. In FIG. 16, the processes from S1401 to S1406 are the same as the processes from S1001 to S1006 of FIG. 10 described in the sixth embodiment. Further, the processing from S1408 to S1412 is the same as the processing from S1011 to S1015 described in the sixth embodiment. The calculation process of the face detection start time in S1401 is as described in FIG. Therefore, the description thereof is omitted, and the processing of S1407 which is the difference is described.

With reference to the flowchart of FIG. 17, the AF read area determination process shown in S1407 of FIG. 16 will be described in detail.
In step S1419, the control unit 121 determines whether a face detection result has been acquired in the current face detection. If the face detection result is acquired (YES in S1419), the process proceeds to S1420. If the face detection result is not acquired (NO in S1419), the process proceeds to S1424. In step S1420, the control unit 121 determines whether the previous face detection result has been acquired. Specifically, the control unit 121 obtains face detection history information stored in the memory 134, and determines whether or not there is a face detection result in the previous shooting in the face detection history. If the previous face detection result has been acquired (YES in S1420), the process moves to S1421. If the previous face detection result has not been acquired (NO in S1420), the control unit 121 determines that a new main face has been detected, and proceeds to S1422.

  In step S1421, the control unit 121 determines whether the detected face is a main face. Specifically, the control unit 121 compares the feature amount of the face image as the previous face detection result with the feature amount of the face image as the currently detected face detection result, and the face detection result detected this time is the main face. It is determined whether or not. If it is determined that the detected face is the main face (YES in S1421), the process proceeds to S1422, and if it is determined that the detected face is not the main face (NO in S1421), the process proceeds to S1424.

  In step S1422, the control unit 121 determines an AF read area from the face detection result. In step S <b> 1423, the control unit 121 stores the face detection result in the memory 134 as a face detection history. In step S1424, the control unit 121 determines whether there is a face detection history. The control unit 121 acquires face detection history information stored in the memory 134 and determines whether there is a past face detection history. For example, when there is no continuous face detection result in the face detection results of the face detection history, the control unit 121 determines that there is no face detection history. If there is a face detection history (YES in S1424), S1425 follows. If there is no face detection history (NO in S1424), S1427 follows.

  In step S1425, the control unit 121 determines an AF read area when a face detection result is not obtained in the prediction range. For example, the AF read area 1502 described with reference to FIG. In step S1426, the control unit 121 determines the predicted position of the main face. For example, the predicted position 1503 of the main face described with reference to FIG. In step S1427, the control unit 121 determines an AF read area from the mono detection result. After the processing of S1423, S1426, and S1427, the process proceeds to return processing.

According to the present embodiment, when the detection result of the main face is not obtained by the face detection in the prediction range, the auto focus that follows the main face is set by setting the AF read area in a range different from the prediction range. You can control. In the present embodiment, an example in which an area other than the prediction range is set as the AF reading area has been described. However, the present invention is not limited to this, and an area that partially overlaps the prediction range may be set as the AF read area.
As mentioned above, although preferable embodiment of this invention was described, this invention is not limited to these embodiment, A various deformation | transformation and change are possible within the range of the summary.

100 ... imaging device 105 ... third lens group 107 ... imaging device 114 ... focus actuator 121 ... control unit 124 ... imaging device drive circuit 126 ... focus drive circuit

Claims (11)

An imaging device that performs autofocus control and subject tracking control in continuous shooting,
First reading means for reading a first image signal relating to a captured image from the imaging device;
Second reading means for reading a second image signal used for calculation of the evaluation value of the autofocus control from the image sensor;
Subject detection means for detecting a subject using the first image signal;
The position for reading the second image signal is determined from the detection information of the subject obtained by the subject detection means, and the evaluation value is obtained from the second image signal read by the second readout means. And control means for controlling the focus adjustment.
The control means calculates a detection start time by the subject detection means from a continuous shooting time interval, and the subject is being read while the first readout means is reading the first image signal from the image sensor. The detection unit starts detection of the subject at the start time, performs subject detection using the read first image signal, and the second reading unit starts from the position determined by the control unit. An image pickup apparatus that reads the second image signal. The subject detection means executes, from the start time point, a first detection process for subject detection with priority on detection speed and a second detection process for subject detection with priority on detection accuracy.
The image pickup apparatus according to claim 1, wherein the control means determines a position at which the second image signal is read out from detection information by the first or second detection process.   When the detection information obtained by the first detection process and the second detection process is acquired, the control unit determines a position for reading the second image signal from the detection information obtained by the second detection process. 3. The imaging apparatus according to claim 2, wherein when the detection information obtained by the detection process of 2 is not acquired, a position from which the second image signal is read is determined from the detection information obtained by the first detection process. The control means determines the luminance, size, or amount of motion of the area including the subject image detected by the subject detection means, and sets the first or second readout mode for the second readout means. And
The first readout mode is a mode in which the second image signal is read out from a first readout range, and the second readout mode is from a second readout range that is narrower than the first readout range. The imaging apparatus according to any one of claims 1 to 3, which is a mode for reading out the second image signal.   The control means sets the first reading mode when the luminance value of the region including the subject image is equal to or higher than the luminance threshold value, and when the luminance value of the region including the subject image is smaller than the luminance threshold value. 5. The image pickup apparatus according to claim 4, wherein the size of the subject image or the amount of movement is compared with a threshold value to set the first or second readout mode. When the luminance value of the area including the subject image is smaller than the luminance threshold, the control means may
The first readout mode is set when the size of the subject image is equal to or greater than a first threshold, or the movement amount of the subject image is equal to or greater than a second threshold.
6. The second reading mode is set when the size of the subject image is smaller than the first threshold and the amount of movement of the subject image is smaller than the second threshold. The imaging device described in 1.   When the detection information of the first detection process is acquired and the detection information of the second detection process is not acquired, the control unit outputs an image signal from the reading range to the second reading unit. 4. The image pickup apparatus according to claim 3, wherein a mode for thinning out and reading is set. The control means calculates an accumulation time and a readout time of the imaging element in an imaging process for obtaining the evaluation value,
When the luminance of the region including the subject image detected by the subject detection unit is smaller than a luminance threshold, and the readout time is shorter than the time determined by the time of imaging processing and autofocus processing and the time interval of continuous shooting, The imaging apparatus according to claim 1, wherein the accumulation time is extended.   The control unit stores the detection information of the subject detection unit in the storage unit as history information, predicts the position of the subject image in the next imaging using the history information, and performs the first reading by the first reading unit. The image pickup apparatus according to claim 1, wherein a reading direction of the image signal is determined, and a detection start time by the subject detection unit is calculated.   The control means performs control to read out the second image signal from a second range different from the first range by the second reading means when the subject image is not detected in the first range. The imaging device according to claim 9, wherein the imaging device is performed. A control method executed in an imaging apparatus that performs autofocus control and subject tracking control in continuous shooting,
A first reading step of reading a first image signal relating to a captured image from the imaging device;
A second reading step of reading a second image signal used for calculating the evaluation value of the autofocus control from the image sensor;
A subject detection step of detecting a subject using the first image signal;
The position for reading the second image signal is determined from the detection information of the subject acquired in the subject detection step, and the evaluation value is acquired from the second image signal read in the second read step. And a control process for controlling the focus adjustment.
In the control step, a start time of the subject detection step is calculated from a time interval of continuous shooting, and the subject detection step is performed while the first image signal is being read from the image sensor in the first read step. Starts at the start time, subject detection is performed using the read first image signal, and in the second reading step, the second image signal is read from the determined position. A method for controlling an image pickup apparatus, characterized in that a processing to be performed is performed.

Images