JP2017026914A - Imaging apparatus and method of controlling the same - Google Patents

Abstract

PROBLEM TO BE SOLVED: To make it possible that even if a tracked subject temporarily disappears from the imaging screen, tracking is resumed when the subject reappears within the imaging screen.SOLUTION: An optical instrument 101 comprises: detection means 203 for detecting an image region corresponding to a subject in each of a plurality of images sequentially generated by photoelectrically converting an optical image of the subject formed by an imaging optical system; and control means 204 for controlling focus tracking to keep the imaging optical system focused on the subject, based on the image regions. The control means transitions from focus tracking control for a first subject to focus tracking control for a second subject which is different from the first subject as the subject in the image region changes from the first subject to the second subject. The focus tracking control for the second subject is performed only when the defocused state of the imaging optical system on the first subject satisfies predetermined conditions.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an imaging apparatus having a so-called subject tracking function.

  Some imaging devices can detect a specific subject (for example, a human face) in a moving image generated by imaging and track the movement to control focus and exposure. In addition, there is an imaging apparatus that keeps track of a registered person by registering information on the face of the person in advance.

  In the imaging apparatus disclosed in Patent Document 1, an image area of a specific subject and color information of the specific subject are acquired in the previous frame in the moving image, and the same color information as the color information acquired in the previous frame is acquired in the latest frame. The subject area that the user has is tracked. When the specific subject is a human face, the face is tracked based on the face position and color information.

  In the imaging device disclosed in Patent Document 2, a pattern of a subject (face or object) to be tracked is registered in advance, and tracking is performed by detecting the pattern from a moving image. When the subject of the tracking target registration pattern disappears from the moving image (imaging screen), the subject of the registered pattern other than the tracking target in the moving image can be newly set as the tracking target.

JP2013-101305A JP 2009-229586 A

  However, in the subject tracking method in the imaging apparatus disclosed in Patent Document 1, when a specific subject is being tracked, another subject overlaps the specific subject (the other subject is not the specific subject and the imaging device). Then, the tracking of the other subject is started. Then, even after another subject passes and the specific subject appears again, there is a possibility that the other subject may continue to be tracked.

  On the other hand, in the imaging apparatus disclosed in Patent Document 2, since the defocus amount for the subject is not considered when detecting the pattern of the subject registered in advance, the subject exists in the image. Regardless, it may not be detected correctly. That is, there is a possibility that the subject to be tracked cannot be specified and the subject cannot be tracked.

  In the present invention, even when a specific subject that has been tracked temporarily disappears from the imaging screen because it overlaps with another subject, the tracking is more reliably performed when the specific subject appears again in the imaging screen. An imaging device or the like that can be provided is provided.

  An optical apparatus according to one aspect of the present invention includes detection means for detecting an image area corresponding to a subject in each of a plurality of images sequentially generated by photoelectrically converting an optical image of the subject formed by the imaging optical system. And control means for performing focus tracking control so as to maintain the in-focus state of the imaging optical system with respect to the subject based on the image area. In response to the change of the subject from the first subject to a second subject different from the first subject in the image region, the control means performs focus tracking control on the first subject to focus tracking on the second subject. Transition to control. The focus tracking control for the second subject is performed only when the defocus state for the first subject of the imaging optical system satisfies a predetermined condition.

  According to another aspect of the present invention, there is provided an optical device control program for a subject in each of a plurality of images sequentially generated by photoelectrically converting an optical image of a subject formed by an imaging optical system in a computer of the optical device. A computer program that performs processing for detecting a corresponding image region and control processing for performing focus tracking control so as to maintain the in-focus state of the imaging optical system with respect to the subject based on the image region. In the control process, the focus on the second subject is changed from the focus tracking control on the first subject in response to the change of the subject from the first subject to the second subject different from the first subject in the image region. Transition to tracking control. The focus tracking control for the second subject is performed only when the defocus state for the first subject of the imaging optical system satisfies a predetermined condition.

  In the present invention, focus tracking control for the second subject is performed only when the defocus state for the first subject of the imaging optical system satisfies a predetermined condition. Therefore, according to the present invention, the focus tracking control for the first subject can be facilitated by re-detecting the first subject after shifting from the focus tracking control for the first subject to the focus tracking control for the second subject. It is possible to realize a subject tracking function that is easy to return to.

1 is a cross-sectional view of a digital single-lens reflex camera that is an embodiment of the present invention. FIG. 3 is a block diagram illustrating a functional configuration of the imaging apparatus according to the embodiment. 6 is a flowchart illustrating subject tracking control processing in the imaging apparatus according to the embodiment. The figure which shows the example of a scene which performs the subject tracking determination process in an Example. 6 is a flowchart illustrating subject tracking control processing in the imaging apparatus according to the embodiment. The figure which shows the example of a scene which performs the subject tracking control process in an Example. 6 is a flowchart illustrating subject tracking control processing when tracking a subject whose face cannot be detected in the embodiment. The figure which shows the example of a scene which performs subject tracking control processing in the case of tracking the subject which cannot detect the face in an Example.

  Embodiments of the present invention will be described below with reference to the drawings.

  A configuration of a digital single-lens reflex camera (hereinafter simply referred to as a camera) 101 as an imaging apparatus (optical apparatus) that is an embodiment of the present invention will be described with reference to FIGS. 1 and 2. An interchangeable lens 102 is detachably and mechanically and electrically connected (mounted) to the camera 101. FIG. 1 shows an optical configuration of the camera 101 and the interchangeable lens 102, and FIG. 2 shows an electrical configuration of the camera 101. In addition, the same code | symbol is attached | subjected about the component which is common in FIG. 1 and FIG.

  The interchangeable lens 102 has an imaging optical system including a focus lens 103 and a diaphragm 104, and receives power supply and control from the camera 101 via a lens mount contact group 105.

  In the camera 101, the main mirror 106 is a half mirror, and a sub mirror 107 is disposed behind it. As shown in FIG. 1, in a state where the main mirror 106 is disposed in the optical path (imaging optical path) of the imaging optical system, a part of the light beam from the imaging optical system is reflected by the main mirror 106 on the viewfinder screen 109. Form a subject image. The light beam transmitted through the main mirror 106 is reflected by the sub mirror 107 and enters an AF (autofocus) unit 108.

  The AF unit 108 includes an AF sensor 202 that performs focus detection by a phase difference detection method. The light flux incident on the AF unit 108 from the imaging optical system forms subject images (two images) on the pair of light receiving sensors (line sensors) in the AF sensor 202. The AF control unit 201 in the AF unit 108 calculates a phase difference between a pair of electric signals (hereinafter referred to as a pair of AF image signals) output from a pair of line sensors obtained by photoelectrically converting two images. Further, the AF control unit 201 detects the defocus amount (including the defocus direction) with respect to the subject of the imaging optical system from the phase difference.

  A system control unit 203 as a control unit calculates a drive amount of the focus lens 103 for obtaining a focused state based on defocus amount information from the AF control unit 201, and outputs a focus drive command including the drive amount. The data is transmitted to a lens control unit (not shown) in the interchangeable lens 102. The lens control unit drives the focus lens 103 according to the focus drive command. Thereby, the drive of the focus lens 103 is controlled, and AF is performed by the phase difference detection method.

  The pair of line sensors is provided for each of the plurality of focus detection areas in the imaging screen, and is a focus detection area that is automatically selected by the user or the system control unit 203 among the plurality of focus detection areas. AF is performed.

  The light beam that forms the subject image on the finder screen 109 is guided to the user's eye through the pentaprism 111 and the eyepiece 110. Thereby, the user can observe the subject image (finder image) formed on the finder screen 109. A part of the light beam (photometric image) that forms the subject image on the finder screen 109 is guided to the AE unit 112 via the pentaprism 111.

  The AE unit 112 generates a photometric image (image data) by photoelectrically converting the photometric image (optical image) by the photometric sensor 205 in order to measure the brightness of the subject. The AE control unit 204 measures the brightness of the subject using the photometric image, and performs an exposure calculation from the measurement result (photometric result).

  The system control unit 203 calculates an appropriate aperture value based on the exposure calculation result output from the AE control unit 204, and transmits an aperture drive command including the aperture value to the lens control unit in the interchangeable lens 102. The lens control unit drives the diaphragm 104 according to the diaphragm drive command. Thereby, the aperture diameter of the diaphragm 104 is controlled, and the amount of light reaching the image sensor 113 described later is adjusted.

  The AE control unit 204 as detection means performs subject detection processing for detecting a subject from the photometric image, and outputs information indicating the detected subject position (subject position information) to the system control unit 203. The system control unit 203 outputs subject position information to the AF control unit 201. The AF control unit 201 controls the driving of the focus lens 103 so that a focused state is obtained with respect to a subject (specific subject) specified by the subject position information. The system control unit 203 drives the focus lens 103 based on the subject position information acquired by the AE control unit 204 for each photometric image sequentially generated in synchronization with the frame image constituting the moving image generated by imaging. To control. In this way, the system control unit 203 performs subject tracking control processing that continues to maintain a focused state so as to follow a specific subject.

  The image sensor 113 is a photoelectric conversion element such as a CMOS image sensor or a CCD image sensor, and has a plurality of pixels arranged two-dimensionally. When imaging a subject, the main mirror 106 and the sub mirror 107 are retracted out of the imaging optical path, and the focal plane shutter 114 performs an opening / closing operation, whereby a subject image (optical image) is formed on the imaging element 113. The image sensor 113 photoelectrically converts the subject image with the plurality of pixels, and outputs an electrical signal (imaging signal) to the system control unit 203. The system control unit 203 performs various processes on the electrical signal from the image sensor 113 to generate a captured image (moving image or still image), and displays this on the display 115 provided on the back surface of the camera 101.

  The operation unit 206 includes operation members such as buttons, switches, and dials provided on the camera 101 and the interchangeable lens 102, detects a user operation of each operation member, and transmits an operation signal to the system control unit 203.

  Next, the subject detection process performed by the AE control unit 204 will be described in detail. The AE control unit 204 detects a specific subject from the photometric image acquired from the photometric sensor 205, and specifies a specific subject region that is an image region including the specific subject. An example of the specific subject is a human face.

  The AE control unit 204 sets the specific subject detected from the photometric image as the first subject as the tracking target. Here, “tracking” means that the imaging optical system is focused on the subject to be tracked by AF, and focus tracking control is performed to continue AF so as to maintain the focused state. In the following description, a specific subject other than the first subject included in the photometric image is set as the second subject.

  The first subject is a specific subject that is set as a tracking target at the start of imaging (or immediately after). The first subject can be set by various methods. For example, when there are a plurality of faces in the photometric image, the plurality of faces are displayed on the display 115, and the user selects a face to be tracked from the faces through the operation unit 206. A first subject is set. Also, information on the face of the person to be tracked is registered as face identification information, and when a face corresponding to the face identification information is detected in the photometric image, the face is set as the first subject. You may do it. Registration of the face identification information at this time may be performed by registration imaging before tracking imaging, which is imaging for detecting and tracking the first subject, or first appears in the photometric image in tracking imaging. You may make it register face information.

  Further, the face of a person existing in the photometric image may be detected and set as the first subject according to the proportion of the detected face in the photometric image and the position (coordinates) in the photometric image. Furthermore, when the user has selected a focus detection area, a face close to the focus detection area may be set as the first subject.

  The AE control unit 204 is shown in FIG. 2 so that the AE control unit 204 can detect the specific subject as described above and output the position information (subject position information) of the specific subject to the system control unit 203. As described above, a subject detection unit 207 and a subject identification unit 208 are provided. In the following description, each of the photometric images sequentially acquired by the AE control unit 204 from the photometric sensor (continuously acquired in time series) is referred to as a detection frame. In addition, the specific subject area corresponding to the first subject in each detection frame is referred to as a first subject area.

  The subject detection unit 207 performs subject detection processing (for example, face detection processing) for detecting a specific subject (first subject) in a certain detection frame. In addition, the subject detection unit 207 has a high correlation with the first subject in the vicinity of the first subject region specified in the detection frame acquired before the new detection frame in each of the detection frames newly acquired sequentially. A specific subject area corresponding to the specific subject is detected. Then, the specific subject region having a high correlation is detected as the specific subject region in the new detection frame. When the specific subject corresponding to the specific subject region in the new detection frame is the second subject that is not the first subject, the specific subject is detected as the second subject.

  For example, an object having the same color as or close to the color of the first object existing in the vicinity of the first object region in the previous detection frame in the new detection frame (but not the first object) is referred to as “correlation. Identify as “high” second subject. Thus, in this embodiment, the degree of coincidence of the subject colors is used as the correlation.

  Then, the subject detection unit 207 outputs subject position information that is position information of a specific subject area including the specific subject (second subject) in the new detection frame.

  The subject identifying unit 208 identifies the same specific subject among a plurality of detection frames based on the similarity such as the position of the detected specific subject. Further, the subject identifying unit 208 identifies a specific subject corresponding to the registered face identification information. For example, information indicating the detected facial features (shape, color, etc. of organs such as eyes, nose, mouth, ears, etc.) is extracted and compared with the feature information included in the registered face identification information A discrimination evaluation value indicating the degree is generated. Then, based on the identification evaluation value, it is determined whether or not the detected face is a face corresponding to the registered face identification information. If the face corresponds to the face identification information, the image area corresponding to the face is specified as the specific subject area. Then, the subject identification unit 208 outputs subject position information that is position information of the specific subject region in each detection frame.

  Note that the subject identification unit 208 can determine whether the face corresponds to the face identification information, that is, specify the first subject area, depending on the size and orientation of the detected face and the defocus amount with respect to the face. As a result, the subject position information may not be output. However, even in this case, the subject detection unit 207 may be able to detect the face. In this case, subject position information obtained from the subject detection unit 207 is output to the system control unit 203.

  Next, the subject tracking control process in the present embodiment will be described using the flowchart of FIG. A system control unit 203 configured by a microcomputer executes subject tracking control processing according to an optical device control program that is a computer program. Here, as shown in FIG. 4, the face of one of the two persons standing still is the first subject, and the face of the other moving person is the second subject. A case will be described in which a scene in which the second subject crosses in front of the first subject (between the first subject and the camera 101) is imaged.

  Images 401, 402, 404, and 406 in FIG. 4 are moving images generated by imaging an imaging scene in which the second subject crosses in the direction orthogonal to the imaging optical axis of the camera 101 near the first subject. These four frames (or detection frames) are shown. In addition, images 401, 402, 405, and 407 show four frames of a moving image generated by capturing a scene in which a second subject crosses near the first subject while approaching the camera 101. Furthermore, images 401, 403, and 408 show three frames of a moving image generated by capturing a scene that the second subject crosses at a position away from the first subject (position close to the camera 101). The images 401 to 408 are the same as the detection frames generated by the photometric sensor 205.

  In step S301 in FIG. 3, the system control unit 203 causes the AE control unit 204 to set the first subject surrounded by a frame in the image 401 by any of the first subject setting methods described above. The system control unit 203 starts tracking the set first subject.

  In step S302, the system control unit 203 determines whether or not the AE control unit 204 has detected the first subject in the imaging screen, that is, whether or not the first subject has disappeared from the imaging screen. The AE control unit 204 performs the face detection process described above or the first subject identification process using the registered face identification information, and confirms whether or not the first subject has been detected. When the first subject is detected as in the image 401, the system control unit 203 continues to track the detected first subject as it is and repeats the determination of this step.

  On the other hand, when the first subject is hidden behind the second subject and disappears from the imaging screen as in the images 402 and 403, the first subject is not detected. For this reason, the system control unit 203 proceeds to step S303 in order to use the specific result obtained by the subject detection unit 207 that can detect the specific subject.

  Next, in step S303, the system control unit 203 causes the subject detection unit 207 to select a first (existing) first frame existing near the first subject area specified by the AE control unit 204 in the previous frame. A subject of the same color as the subject is set as a tracking candidate. When the second subject overlaps the first subject, the first subject disappears from the imaging screen, and the second subject exists near the area where the first subject has been detected so far. The second subject is a candidate for the next tracking. At the timing when the second subject becomes a tracking target, the system control unit 203 performs subject tracking determination processing for determining whether or not to track the second subject.

  Specifically, in response to the first subject disappearing from the imaging screen due to the overlap of the second subject, the system control unit 203 moves in the optical axis direction in which the first subject was present in step S304. Is set as the reference position. Then, a reference position when the imaging optical system is focused on the second subject, that is, a defocus amount of the imaging optical system with respect to the first subject is calculated (predicted).

  In step S305, the system control unit 203 determines whether or not the defocus state expressed as the calculated defocus amount for the first subject satisfies a predetermined condition. Specifically, it is determined whether or not the defocus amount is equal to or smaller than a predetermined value (same as or smaller than the predetermined value). The predetermined value as the threshold is based on the maximum defocus amount that can detect the first subject by face detection processing or identification processing using face identification information when the first subject reappears in the imaging screen. Is set. If the defocus amount is less than or equal to the threshold value, the system control unit 203 proceeds to step S306 and starts tracking the focused second object as in the image 402. Thereafter, the process proceeds to step S308.

  On the other hand, when the defocus amount with respect to the first subject is larger than the threshold value, the system control unit 203 proceeds to step S307 and does not perform tracking for the second subject as in the image 403. Thereafter, the process proceeds to step S313. By not tracking (that is, focusing) the second subject when the defocus amount with respect to the first subject is larger than the threshold value, face detection processing after the first subject appears again in the imaging screen, The first subject can be tracked again in response to the detection of the first subject by the identification process.

  In step S308, the system control unit 203 that has started tracking the second subject in step S306 calculates a defocus amount with respect to the first subject that changes by tracking the second subject.

  In step S309, the system control unit 203 determines whether or not the calculated defocus amount with respect to the first subject is equal to or less than the predetermined value as the above-described threshold value. If the defocus amount is equal to or smaller than the threshold value, the system control unit 203 proceeds to step S310 and continues to track the second subject as it is as the image 404. Thereafter, the process proceeds to step S312.

  On the other hand, when the defocus amount with respect to the first subject becomes larger than the threshold, such as when the second subject moves to a place away from the first subject in the optical axis direction, the system control unit 203 The process proceeds to step S311. In step S <b> 311, the system control unit 203 stops tracking the second subject as in the image 405. Then, the position of the focus lens 103 is held so that the defocus amount with respect to the first subject does not change until the subject to be tracked next is set. Thereafter, the process proceeds to step S313.

  In step S312, the system control unit 203 confirms whether or not the first subject is detected in the imaging screen. When the first subject is detected in the imaging screen, the system control unit 203 proceeds to step S314, stops tracking the second subject and resumes tracking the first subject as shown in the image 406. To do. On the other hand, if the first subject is not detected in the imaging screen, the system control unit 203 returns to step S308 and continues tracking the second subject. In this case, the processes from step S308 to S312 are repeated until the first subject is detected in the subsequent frame.

  In step S313, the system control unit 203 confirms whether or not the first subject is detected in the imaging screen as in step S312. If the first subject is detected in the imaging screen, the system control unit 203 proceeds to step S314 and resumes tracking of the first subject. If the first subject is not detected in the imaging screen, the system control unit 203 returns to step S304 and repeats the processing from step S304 onward until the first subject is detected in the subsequent frame.

  As the image 403 does not shift to the tracking of the second subject, the defocus amount of the first subject becomes equal to or less than the threshold value. As a result, it is possible to return to the tracking of the first subject again in accordance with the detection of the first subject by the face detection process or the identification process after the first subject appears again in the imaging screen as in the image 408. It becomes.

  Further, even when the tracking of the second subject is stopped when the defocus amount with respect to the first subject is equal to or smaller than the threshold as in the image 405, the image can be changed according to the re-detection of the first subject in the imaging screen. As in 407, the tracking of the first subject can be resumed.

  As described above, in this embodiment, the tracking of the second subject is limited so that the defocus amount for the first subject does not become larger than the threshold when the first subject disappears from the imaging screen and is no longer detected. As a result, even when tracking is performed for the second subject, the defocus amount for the first subject is always equal to or less than the threshold value. As a result, it is possible to reliably detect the first subject by face detection processing and identification processing after the first subject appears again in the imaging screen, and to return to tracking of the first subject. . On the other hand, when the first subject disappears from the imaging screen while tracking the first subject, the second subject is detected within a range in which the first subject can be detected when it appears again in the imaging screen. Can track the subject.

  Next, a second embodiment of the present invention will be described. In this embodiment, a case where the first subject is a moving object will be described. Note that the configurations of the camera 101 and the interchangeable lens 102 of the present embodiment are the same as those of the first embodiment, and constituent elements are given the same reference numerals and description thereof is omitted. The subject detection process performed by the AE control unit 204 in the present embodiment is the same as that in the first embodiment.

  The subject tracking control process when the first subject is a moving object will be described with reference to the flowchart of FIG. 5 and FIG. Here, as shown in FIG. 6, one of the faces of the two persons, the first subject approaching the camera 101, and the other second subject approaching the camera 101. The case where the scene which crosses is imaged is demonstrated. The moving speed of the second subject in the optical axis direction is faster than that of the first subject, and the distance in the optical axis direction between the first subject and the second subject increases. .

  In step S501 in FIG. 5, the system control unit 203 causes the AE control unit 204 to perform the first framed frame in the image 601 in FIG. 6 by one of the first subject setting methods described in the first embodiment. Set the subject. The system control unit 203 starts tracking the set first subject.

  In step S502, the system control unit 203 determines whether or not the first subject has been detected in the imaging screen by the AE control unit 204 that performs the face detection processing and identification processing described in the first embodiment, that is, the first It is determined whether the subject has disappeared from the imaging screen. When the first subject is detected, the system control unit 203 continues to track the first subject detected as it is and repeats the determination of this step. On the other hand, when the first subject is hidden behind the second subject and disappears from the imaging screen as in the image 602 of FIG. 6, the first subject is not detected. Therefore, the system control unit 203 proceeds to step S504 via step S503 in order to use the specific result obtained by the subject detection unit 207.

  At this time, in step S503, the system control unit 203 moves the first subject per unit time in the optical axis direction from a plurality of consecutive past frames before the first subject is not detected on the imaging screen. The amount (that is, the moving speed) is calculated. In step S504, the system control unit 203 sets a subject of the same color as the first subject existing in the vicinity of the first subject area specified by the AE control unit 204 in the previous frame in the new (current) frame. Here, the second subject) is set as a tracking candidate.

  Next, in step S505, the system control unit 203 performs subject tracking determination processing for determining whether or not to track the second subject, and thus the first subject that changes when the second subject is focused. The defocus amount of is calculated. At this time, the system control unit 203 calculates the defocus amount using the movement amount of the first subject in the optical axis direction per unit time calculated in step S503. Specifically, the current position of the first subject is predicted from the amount of movement of the first subject in the optical axis direction per unit time, and the position is set as the reference position. Then, a reference position when the imaging optical system is focused on the second subject, that is, a defocus amount of the imaging optical system with respect to the first subject is calculated (predicted).

  In step S506, the system control unit 203 determines whether or not the calculated defocus amount (defocus state) for the first subject satisfies a predetermined condition, specifically, a predetermined value or less (same as the predetermined value or Or smaller). The predetermined value as the threshold is the maximum defocus that can detect the first subject by face detection processing or identification processing when the first subject appears again in the imaging screen, as in step S305 of the first embodiment. Set based on quantity. If the defocus amount is equal to or smaller than the threshold value, the system control unit 203 proceeds to step S507 and starts tracking the second subject focused as in the image 602. Thereafter, the process proceeds to step S509.

  On the other hand, if the defocus amount of the first subject is larger than the threshold value, the system control unit 203 proceeds to step S508 and does not perform tracking for the second subject. Thereafter, the process proceeds to step S514. The reason for not tracking the second subject is the same as that in step S307 in the first embodiment.

  In step S509, the system control unit 203 that has started tracking the second subject in step S507 calculates a defocus amount with respect to the first subject that changes by tracking the second subject. At this time, similarly to step S505, the current position of the first subject is predicted using the movement amount of the first subject in the optical axis direction per unit time calculated in step S503, and the position is determined as the reference position. Set to. Then, the defocus amount of the imaging optical system with respect to the reference position (first subject) when the imaging optical system is focused on the second subject is calculated (predicted).

  In step S510, the system control unit 203 determines whether the calculated defocus amount for the first subject is equal to or less than the above-described threshold value. If the defocus amount is equal to or smaller than the threshold value, the system control unit 203 proceeds to step S511 and continues to track the second subject as it is like the image 603 in FIG. Thereafter, the process proceeds to step S513.

  On the other hand, when the second subject whose movement speed is faster than the first subject moves away from the first subject in the optical axis direction and the defocus amount for the first subject becomes larger than the threshold, the system control unit 203 Advances to step S512. In step S512, the system control unit 203 stops tracking the second subject as shown in the image 604 of FIG. Then, the position of the focus lens 103 is controlled using the moving speed of the first subject calculated in step S503 so that the defocus amount for the first subject does not change until the subject to be tracked next is set. Thereafter, the process proceeds to step S514.

  In step S513, the system control unit 203 confirms whether or not the first subject is detected in the imaging screen. When the first subject is detected in the imaging screen, the system control unit 203 proceeds to step S515, stops tracking the second subject as shown in the image 605 in FIG. 6, and determines the first subject. Resume tracking. On the other hand, if the first subject is not detected in the imaging screen, the system control unit 203 returns to step S509 and continues tracking the second subject. In this case, the processing from steps S509 to S513 is repeated until the first subject is detected in the subsequent frame.

  In step S514, the system control unit 203 confirms whether or not the first subject is detected in the imaging screen as in step S513. When the first subject is detected in the imaging screen, the system control unit 203 proceeds to step S515 and resumes tracking of the first subject. If the first subject is not detected in the imaging screen, the system control unit 203 returns to step S504 and repeats the processing from step S504 onward until the first subject is detected in the subsequent frame.

  In this embodiment, the movement while the first subject disappears from the imaging screen is predicted from the amount of movement of the first subject in the optical axis direction before the first subject is no longer detected on the imaging screen. Thus, it is used to calculate the defocus amount for the first subject. After that, tracking of the second subject (focus tracking control) is performed so that the defocus amount with respect to the first subject is equal to or less than the threshold value. As a result, the tracking of the first subject can be resumed in response to the detection of the first subject by the face detection process or the identification process after the first subject appears again in the imaging screen as in the image 605. It becomes possible.

  As described above, in this embodiment, the tracking of the second subject is performed so that the defocus amount for the first subject when the moving first subject disappears from the imaging screen and is not detected is larger than the threshold value. Limit. Thereby, even when tracking is performed on the second subject, the defocus amount for the moving first subject is always equal to or less than the threshold value. As a result, it is possible to reliably detect the first subject after the moving first subject appears again in the imaging screen and return to tracking of the first subject.

  Next, Embodiment 3 of the present invention will be described. In this embodiment, a case where a face as the first subject is not detected will be described. Note that the configurations of the camera 101 and the interchangeable lens 102 of the present embodiment are the same as those of the first embodiment, and constituent elements are given the same reference numerals and description thereof is omitted.

  The subject tracking control process when the first subject (face) cannot be detected will be described with reference to the flowchart of FIG. 7 and FIG. Here, as shown in FIG. 8, a case will be described in which a scene crossing behind the second subject where the first subject is stationary while the first subject is being tracked is imaged. In this scene, as shown in an image 801, the face of a person moving sideways with respect to the camera 101 is set as a first subject, and the first subject is continuously imaged. In continuous imaging, the user pans the camera 101 so that the first subject is positioned at the center of the imaging screen. Since the first subject faces sideways, subject detection processing by face detection processing or identification processing cannot be performed.

  In step S701 of FIG. 7, the system control unit 203 causes the AE control unit 204 to set the first subject. Here, as exemplified in the first embodiment, the face included in (or close to) the focus detection area selected by the user is set as the first subject. Then, the system control unit 203 causes the subject detection unit 207 to acquire information on the color of the first subject.

  In this embodiment, like the images 801 and 802 in FIG. 8, the face cannot be detected even though the face that is the first subject exists in the imaging screen. Therefore, in the next step S702, the system control unit 203 causes the subject detection unit 207 to display the first subject existing near the first subject area including the first subject specified in the previous frame in the current frame. The same color is detected. Then, the first object is tracked based on the detected color. In other words, the degree of coincidence of colors is used as a correlation, and the first object is regarded as being present in an image area having a high correlation with the previous frame in the current frame, and this is tracked.

  When the first subject is hidden behind the second subject and disappears from the imaging screen as in the image 803 in FIG. 8, the second subject of the same color as the first subject is located near the first subject region so far. Since the subject exists, the second subject is a candidate for the next tracking. At the timing when the second subject becomes a tracking target, the system control unit 203 performs subject tracking determination processing for determining whether or not to track the second subject.

  In step S703, the system control unit 203 sets the position of the first subject in the optical axis direction as the reference position, and the reference position when the imaging optical system is focused on the second subject, that is, the first position. The defocus amount of the imaging optical system for the subject is calculated (predicted).

  In step S704, the system control unit 203 determines whether the calculated defocus amount (defocus state) with respect to the first subject satisfies a predetermined condition, specifically, a predetermined value or less (same as the predetermined value or Or smaller). The predetermined value as the threshold is the same as the predetermined value in the first and second embodiments. If the defocus amount is equal to or smaller than the threshold, the system control unit 203 proceeds to step S705 and starts tracking the second subject. Thereafter, the process proceeds to step S707. On the other hand, if the defocus amount of the first subject is larger than the threshold, the system control unit 203 proceeds to step S706 and does not perform tracking for the second subject as in the image 803 in FIG. Thereafter, the process proceeds to step S707.

  In step S707, the system control unit 203 confirms whether or not the first subject is detected in the imaging screen. When the first subject appears again in the imaging screen as in the image 804 in FIG. 8, the first subject cannot be detected by face detection processing or identification processing because the first subject is a profile. However, by detecting the same color as the first subject, a subject having that color can be regarded as the first subject. Note that the detection of the subject corresponding to the first subject can also be performed using information on the defocus amount and position with respect to the first subject in a plurality of past frames.

  When the first subject (subject regarded as the subject) is detected in this way, the system control unit 203 proceeds to step S708 and resumes tracking of the first subject. The subject to be tracked at this time is not determined to be the first subject, but since the defocus amount for the subject (first subject) is less than or equal to the threshold value, the subject is tracked by obtaining a focused state. be able to.

  On the other hand, if the first subject is not detected, the system control unit 203 returns to step S702.

  In this embodiment, when the first subject disappears from the imaging screen and is no longer detected, a new subject to be tracked (second subject) based on the color information of the first subject obtained in the previous frame. ) Is set. At this time, the tracking for the second subject is limited so that the defocus amount for the first subject does not exceed the threshold. Thereby, when the first subject appears again in the imaging screen, it is possible to return to the tracking of the first subject by detecting the same color as the color of the first subject.

  If the second subject is tracked regardless of the defocus amount for the first subject calculated in step S703 in the image 803, the first subject appears again in the imaging screen in the subsequent image 804. May continue to track the second subject. Even when the first subject faces the front in the state of the image 804 while tracking the second subject, the face detection process and the identification process are performed because the defocus amount of the first subject is large. The first subject cannot be detected. In contrast, in this embodiment, the tracking of the second subject in the image 803 is limited by the defocus amount with respect to the first subject, so that the first subject appears again in the imaging screen and faces the front. Sometimes, the first subject can be detected by face detection processing or identification processing.

  Thus, according to the present embodiment, the first subject can be preferentially tracked even when the face that is the first subject cannot be detected. Further, when the face that is the first subject can be detected, the first subject can be detected by the face detection process or the identification process.

  In each of the embodiments described above, the case where the tracking of the second subject is performed only when the defocus amount representing the defocus state is equal to or smaller than the predetermined value (or smaller than the predetermined value) has been described. However, an evaluation value representing a defocus state other than the defocus amount may be determined, and the tracking of the second subject may be performed only when a predetermined condition for the evaluation value is satisfied. For example, in the present embodiment, the case where the defocus amount is calculated by the phase difference detection method has been described. However, the tracking of the second subject is performed only when the contrast evaluation value corresponding to the defocus state obtained by the contrast detection method satisfies a predetermined condition of a predetermined value or higher (or higher than the predetermined value). Also good. The contrast evaluation value equal to or higher than a predetermined value (or higher than the predetermined value) corresponds to a defocus amount that is equal to or lower than the predetermined value (or lower than the predetermined value).

  In each of the above embodiments, the second subject overlaps the first subject, so that the first subject disappears from the imaging screen and the second subject temporarily exists in the imaging screen. Explained the case. However, the focus follow-up control in each embodiment is performed when the first subject disappears from the imaging screen and the second subject does not exist, or the first subject disappears from the imaging screen and the second subject appears instead. It can be applied in various scenes, such as when (frame-in).

  In each of the embodiments, the imaging of a scene in which two persons (faces) exist as subjects has been described as an example, but the number of subjects and the types of subjects (specific subjects) are not limited at all.

  In each embodiment, the case where subject detection is performed using a photometric image (detection frame) acquired by photoelectric conversion by the photometric sensor 205 has been described. However, subject detection may be performed using another image. . For example, subject detection may be performed using a captured image (moving image frame) generated using an imaging signal from the image sensor 113 that photoelectrically converts the subject image.

  In each embodiment, the case where the first subject to be tracked is determined at the start of imaging has been described. However, the determination of the first subject is not limited to at the start of imaging, and may be determined at another timing. For example, during imaging, a subject different from the first subject set at the start of imaging may be switched to the first subject.

Furthermore, in each embodiment, a digital single-lens reflex camera has been described. However, subject tracking control described in each embodiment in various optical devices including an imaging device such as a lens-integrated camera and a video camera and a lens device such as an interchangeable lens. Processing can be performed.
(Other examples)
The present invention supplies a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in a computer of the system or apparatus read and execute the program This process can be realized. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

  Each embodiment described above is only a representative example, and various modifications and changes can be made to each embodiment in carrying out the present invention.

101 Camera 102 Interchangeable Lens 103 Focus Lens 108 AF Unit 112 AE Unit 113 Image Sensor 203 System Control Unit

Claims (9)

Detecting means for detecting an image region corresponding to the subject in each of a plurality of images sequentially generated by photoelectrically converting an optical image of the subject formed by the imaging optical system;
Control means for performing focus tracking control so as to maintain the in-focus state of the imaging optical system with respect to the subject based on the image region;
The control means includes
In response to the change of the subject from the first subject to a second subject different from the first subject in the image region, the focus tracking control for the first subject causes the subject for the second subject to change. Move to focus tracking control,
An optical apparatus characterized in that the focus tracking control for the second subject is performed only when a defocus state for the first subject of the imaging optical system satisfies a predetermined condition.   2. The optical apparatus according to claim 1, wherein the predetermined condition is that a defocus amount of the imaging optical system with respect to the first subject is equal to or smaller than a predetermined value or smaller than the predetermined value.   The predetermined value is the maximum defocus amount that allows the detection unit to redetect the image area corresponding to the first subject while the focus tracking control is performed on the second subject. The optical apparatus according to claim 2, wherein the optical apparatus is set based on   When the focus tracking control for the first subject is performed, the control means determines the defocus amount when the position of the first subject does not change in the optical axis direction of the imaging optical system. The optical apparatus according to claim 2, wherein the calculation is performed with respect to the position of the first subject.   When the first subject is moving in the optical axis direction of the imaging optical system, the control means uses the defocus amount as the focus tracking control for the first subject. 5. The optical apparatus according to claim 2, wherein the calculation is performed using the position and the moving speed of the first subject.   6. The detection unit according to claim 1, wherein the detection unit detects the image region corresponding to the second subject using a correlation with the image region corresponding to the first subject. The optical device according to Item.   The optical apparatus according to claim 6, wherein the correlation is a degree of coincidence of a subject color.   The optical apparatus according to claim 1, wherein the subject is a face. In the computer of the optical equipment,
Processing for detecting an image region corresponding to the subject in each of a plurality of images sequentially generated by photoelectrically converting an optical image of the subject formed by the imaging optical system;
A computer program for performing a control process for performing a focus tracking control so as to maintain an in-focus state of the imaging optical system with respect to the subject based on the image area;
The control process is
In response to the change of the subject from the first subject to a second subject different from the first subject in the image region, the focus tracking control for the first subject causes the subject for the second subject to change. Move to focus tracking control,
An optical device control program that performs the focus tracking control for the second subject only when a defocus state for the first subject of the imaging optical system satisfies a predetermined condition.