JP5965654B2 - TRACKING DEVICE, IMAGING DEVICE, TRACKING METHOD USED FOR IMAGING DEVICE - Google Patents

Description

  The present invention relates to a tracking device that tracks a subject, an imaging device, and a tracking method used in the imaging device.

  As a tracking device for tracking a subject, there is a technique for tracking a subject using an AF evaluation value used in, for example, contrast autofocus (contrast AF). An example of such a technique is Patent Document 1. This Patent Document 1 calculates a contrast detection signal based on an output signal of an image sensor for a plurality of AF areas while driving a focus lens, and detects subject movement between AF areas from the calculated contrast detection signal. Then, it is disclosed that the in-focus position is obtained according to the detection result.

JP 2007-011054 A

  However, in Patent Document 1, when the AF area is smaller than the moving subject, the contrast tends to be low, and the moving subject may not be detected. Also, when the AF area is larger than the moving subject, Therefore, the AF evaluation value of only the moving subject may not be acquired.

  It is an object of the present invention to provide a tracking device, an imaging device, and a tracking method used for the imaging device that can improve tracking performance using an AF evaluation value appropriate for tracking.

A tracking device according to a main aspect of the present invention includes an imaging unit that images a subject, and a plurality of areas that acquire a plurality of features for tracking the subject in image data acquired by imaging of the imaging unit. A focus evaluation value acquisition area setting unit for setting a focus evaluation value acquisition area for acquiring a focus evaluation value for the subject, a tracking unit for tracking the subject, and a tracking including the subject When an area is specified, a tracking prohibition area setting unit which is an area in which tracking by the tracking unit with respect to a non-designated area other than the specified tracking area is prohibited based on the focus evaluation value in the focus evaluation value acquisition area comprising the door, the tracking prohibition area setting unit, first the distance between the tracking area and the non-designated area has been set in advance If the difference between the focus evaluation values within each focus evaluation value acquisition area is equal to or greater than a second threshold value set in advance, or the interval between the tracking area and the non-designated area is predetermined. If the area difference between the tracking area and the non-designated area is equal to or greater than a preset third threshold value within the first threshold value set, tracking of the non-designated area by the tracking unit is prohibited. .

An imaging apparatus according to a main aspect of the present invention is image-processed by the tracking apparatus, an image processing unit that performs image processing on the image data acquired by imaging of the imaging unit, and the image processing unit. A display unit for displaying the image data.
Tracking method used in the imaging apparatus according to the principal aspect of the present invention, encompassing the area a plurality of areas for obtaining a plurality of features for tracking the object Utsushitai in the image data acquired by the imaging of the imaging unit And a focus evaluation value acquisition area for acquiring a focus evaluation value for the subject is set, and when a tracking area including the subject is designated, the focus evaluation value in the focus evaluation value acquisition area is set. It prohibits pre Symbol add tails for non-designated areas other than the designated tracking area Te, first within the threshold value, and wherein the focus evaluation value distance between the tracking area and the non-designated area has been set in advance If the difference between the focus evaluation values in the acquisition area is greater than or equal to a second threshold value set in advance, or the interval between the tracking area and the non-designated area is Within the first threshold value is because set, and if the third threshold value or more area difference of the tracking area and the non-designated area has been set in advance, prohibits the tracking for the non-designated area.

  ADVANTAGE OF THE INVENTION According to this invention, the tracking method used for the tracking apparatus and imaging device which can improve tracking performance using AF evaluation value suitable for tracking, and an imaging device can be provided.

The block diagram which shows an example of the imaging device provided with the tracking apparatus which concerns on one embodiment of this invention. Explanatory drawing about the wrinkle recognition process in the same apparatus. Explanatory drawing about the tracking process using the color information in the same apparatus. Explanatory drawing about the tracking process using the luminance information in the same apparatus. The figure which shows the setting of the focus evaluation value acquisition area by the focus evaluation value acquisition area setting part in the same apparatus. The figure which shows the detection of the tracking candidate area by the tracking candidate detection part in the apparatus. The imaging | photography operation | movement flowchart in the same apparatus. The figure which shows the image data in the tracking start of the apparatus. The figure which shows the image data by which the tracking frame was set to the face part of the person by the same apparatus. The figure which shows being designated as a tracking object by the touch by the user's finger | toe to the part corresponding to the face part of the object on a touch panel in the same apparatus. The tracking process flowchart in the same apparatus. The figure which shows the setting of the focus evaluation value acquisition area by the focus evaluation value acquisition area setting part in the same apparatus. The figure which shows the tracking frame set to the face part of the 1st person in the image data by the same apparatus. The figure which shows the tracking frame set to the non-tracking object by the focus evaluation value acquisition area setting part in the same apparatus. The figure which shows the setting of the tracking prohibition area by the tracking prohibition area setting part in the same apparatus. The figure which shows prohibition of the tracking with respect to the subject of the non-tracking subject by the tracking prohibition area setting unit in the same device. The figure which shows the update of the tracking frame set to the face part of the to-be-photographed object in the apparatus. FIG. 6 is a schematic diagram of image data showing a state in which tracking by color information is about to shift from the first person to be tracked to the face of the second person to be tracked by the same apparatus. The schematic diagram of the image data which shows the prohibition state of tracking with respect to the face part of the 2nd person of the non-tracking object by the same apparatus. The schematic diagram of the image data which shows the state which returns to the tracking with respect to the 1st person of tracking object by the same apparatus. The AF evaluation value acquisition area setting flowchart in the same apparatus. The tracking prohibition area | region determination processing flowchart in the same apparatus. The schematic diagram which shows the setting of AF evaluation value acquisition area in the tracking prohibition area | region determination process in the same apparatus. The schematic diagram which shows the setting of the tracking prohibition area in the tracking prohibition area | region determination process in the same apparatus. The schematic diagram which shows the setting of the tracking frame in the tracking prohibition area | region determination process in the same apparatus. The other tracking prohibition area | region determination processing flowchart in the same apparatus. Explanatory drawing which shows the wobbling operation | movement used for the apparatus. FIG. 3 is a configuration diagram showing an AF pixel row of an image sensor in the apparatus. FIG. 3 is a configuration diagram showing an output signal of an AF pixel column of an image sensor in the same device.

Hereinafter, an embodiment of the present invention will be described with reference to the drawings.
FIG. 1 shows a configuration diagram of an example of an imaging apparatus provided with a tracking device. The imaging apparatus 100 includes an imaging optical system 102, a focus adjustment mechanism 104, a diaphragm 106, a diaphragm driving mechanism 108, a shutter 110, a shutter driving mechanism 112, an imaging element 114, and an imaging element interface (IF) circuit 116. A RAM 118, a display element 120, a display element drive circuit 122, a touch panel 124, a touch panel drive circuit 126, a recording medium 128, a system controller 130 on which a CPU 1301 is mounted, a camera operation switch 132, and a ROM 134. Have.

  Among them, the photographing optical system 102, the focus adjustment mechanism 104, the diaphragm 106, the diaphragm drive mechanism 108, the shutter 110, the shutter drive mechanism 112, the image sensor 114, and the image sensor interface (IF) circuit 116 are: The imaging unit 101 is configured.

  The photographing optical system 102 is an optical system that condenses the light flux F from the subject on the light receiving surface of the image sensor 114. The photographing optical system 102 has a plurality of lenses such as a focus lens. The focus adjustment mechanism 104 includes a motor and its drive circuit. The focus adjustment mechanism 104 drives the focus lens in the photographing optical system 102 in the optical axis direction (the one-dot chain line direction in the drawing) according to the control of the CPU 1301.

  The diaphragm 106 is configured to be openable and closable, and adjusts the amount of the light beam F incident on the image sensor 114 via the photographing optical system 102. The aperture drive mechanism 108 has a drive mechanism for driving the aperture 106. The diaphragm driving mechanism 108 drives the diaphragm 106 according to the control of the CPU 1301 in the system controller 130.

  The shutter 110 brings the light receiving surface of the image sensor 114 into a light shielding state or an exposure state. The shutter 110 adjusts the exposure time of the image sensor 114. The shutter drive mechanism 112 has a drive mechanism for driving the shutter 110 and drives the shutter 110 according to the control of the CPU 1301 in the system controller 130.

  The image sensor 114 has a light receiving surface. On this light receiving surface, the light flux F from the subject condensed through the photographing optical system 102 is imaged. The light receiving surface of the image sensor 114 is formed by arranging a plurality of pixels in a two-dimensional manner. A color filter is provided on the light incident side of the light receiving surface of the image sensor 114. The image sensor 114 converts an image (subject image) corresponding to the light flux F formed on the light receiving surface into an electrical signal (hereinafter referred to as an image signal) corresponding to the amount of light. As this imaging device 114, various types of configurations such as a CCD system or a CMOS system are known. As the color arrangement of the color filters, various arrangements such as a Bayer arrangement are known. In the present embodiment, the configuration of the image sensor 114 is not limited to a specific configuration, and image sensors having various configurations can be used.

  The image sensor IF circuit 116 drives the image sensor 114 according to the control of the CPU 1301 and reads an image signal obtained by the image sensor 114. The image sensor IF circuit 116 performs analog processing such as CDS (correlated double sampling) processing and AGC (automatic gain control) processing on the image signal read from the image sensor 114. Further, the image sensor IF circuit 116 converts the analog processed image signal into a digital signal (hereinafter referred to as image data).

In the SDRAM 118, for example, a work area 118a, an evaluation image area 118b, a reference image area 118c, a tracking position log area 118d, and a contrast information log area 118e are formed as storage areas.
In the work area 118a, for example, data generated in each unit of the imaging apparatus 100 such as image data obtained by the imaging element IF circuit 116 is temporarily stored.
Evaluation image data is temporarily stored in the evaluation image area 118b. The evaluation image data is image data of a frame including a subject to be tracked in a tracking process described later.
Reference image data is temporarily stored in the reference image area 118c. The reference image data is image data of a frame to be searched for a tracking target in tracking processing described later.

A tracking position log is temporarily stored in the tracking position log area 118d. The tracking position log is a log in which the tracking position obtained as a result of the tracking process is recorded. In the present embodiment, a tracking target is tracked using a plurality of tracking processes. Therefore, the tracking position log obtained by each tracking process (candidate tracking position) is individually recorded in the tracking position log.
A contrast information log is temporarily stored in the contrast information log area 118e. The contrast information log is a log in which contrast information obtained by contrast information calculation processing is recorded.

  The display element 120 is, for example, a liquid crystal display (LCD), and displays various images such as an image for live view and an image recorded on the recording medium 128. Hereinafter, the display element 120 is referred to as an LCD 120. The display element driving circuit 122 drives the LCD 120 based on the image data input under the control of the CPU 1301 and displays an image on the LCD 120.

  The touch panel 124 is provided integrally on the display screen of the LCD 120, detects contact (touch) of the user's finger or the like on the display screen, and outputs a contact detection signal including coordinates indicating the touch position. The touch panel drive circuit 126 drives the touch panel 124 and sends a contact detection signal from the touch panel 124 to the CPU 1301. The CPU 1301 detects a contact operation on the display screen of the user from the contact detection signal from the touch panel 124, and executes processing according to the contact operation.

  The recording medium 128 is a memory card, for example. The recording medium 128 records the image file 128a obtained by the shooting operation. The image file 128a is formed by adding a predetermined header to image data. In the header, data indicating shooting conditions, data indicating a tracking position, and the like are recorded as tag data.

  The system controller 130 controls the overall operation of the imaging apparatus 100. The system controller 130 includes the CPU 1301, an AF control circuit 1302, an AE control circuit 1303, an image processing circuit 1304, a face recognition circuit 1305, and first to third tracking processing circuits 1306 to 1308 as control circuits. A focus evaluation value acquisition area setting unit 1309, a tracking candidate detection unit 1310, a tracking area designation unit 1311, a tracking prohibition area setting unit 1312, and a memory control circuit 1313.

The CPU 1301 controls the operations of the focus adjustment mechanism 104, the aperture drive mechanism 108, the shutter drive mechanism 112, the display element drive circuit 122, the touch panel drive circuit 126, and the like, and the operations of the control circuits in the system controller 130. To control.
The AF control circuit 1302 controls contrast AF processing. Specifically, the AF control circuit 1302 extracts high-frequency components of the image data obtained by the image sensor IF circuit 116, and obtains an AF focus evaluation value by integrating the extracted high-frequency components. The CPU 1301 controls the focus adjustment mechanism 104 while evaluating the contrast of the image data in accordance with the focus evaluation value, and adjusts the focus lens of the photographing optical system 102 to a position where the focus is achieved.

  The AE control circuit 1303 controls the AE operation. Specifically, the AE control circuit 1303 calculates the luminance of the subject using the image data obtained by the imaging element IF circuit 116. The AE control circuit 1303 calculates the aperture amount (aperture value) of the aperture 106 at the time of exposure and the opening time (shutter speed value) of the shutter 110 according to the luminance of the subject.

The image processing circuit 1304 performs various image processing on the image data. This image processing includes, for example, color correction processing, gamma (γ) correction processing, compression processing, and the like. The image processing circuit 1304 also performs decompression processing on the compressed image data.
The face recognition circuit 1305 detects the buttocks of the subject (person) in the image data. Here, the eyelid recognition process will be briefly described. In the wrinkle recognition process, each correlation amount between the image data obtained in each frame and each face part 402, 404, 406 as shown in FIG. The face part 402 is image data corresponding to a shadow pattern around the nose of a person. The face part 404 is image data corresponding to a shadow pattern around the eyes of a person. The face part 406 is image data corresponding to a shadow pattern around the mouth of a person. Each correlation amount between the image data and each of the eyelid parts 402, 404, and 406 becomes maximum when a predetermined arrangement indicating the face of a person is obtained as shown in FIG. At this time, it is assumed that there is a bag in the area 408 including the bag parts 402, 404, and 406. In the figure, the heel area is a rectangular area, but it may be a circular area.

The first to third tracking processing circuits 1306 to 1308 track the subject, for example, the face of a person, in the image data acquired by the imaging unit 101.
The first tracking processing circuit 1306 performs tracking of a feature amount, for example, a subject recognized by the face recognition circuit 1305, for example, a human face, from the image data acquired by the imaging unit 101.

The second tracking processing circuit 1307 performs tracking of a subject, for example, a face of a person, based on characteristics, for example, color information, from image data acquired by imaging by the imaging unit 101. The second tracking processing circuit 1308 performs tracking processing of a human face using color information of image data acquired by imaging of the imaging unit 101. The tracking process using this color information will be briefly described. In the tracking process using color information, an area that is determined to be the same color information as the color information set in the evaluation image data, that is, a tracking color area is searched.
For example, as shown in FIG. 3A, in the (N-1) frame, when a certain position 302 of the subject is designated, the color information of the position 302 in the evaluation image data is acquired. An area having the same color information as that of the position 302 is searched for using the position 302 as a tracking process start position. Specifically, the color information is sequentially acquired so that the position 302 moves from the start position toward the periphery, and it is determined whether or not the acquired color information is the same as the color information at the position 302. As a result of this determination, if it is determined that the color information is the same as the color information at the position 302, it is included in the area, and if it is determined that the acquired color information is the same as the color information at the position 302, Do not include in the area.

  When the tracking color area is searched in this way, for example, in the case of a monochromatic subject as shown in FIG. 3A, a rectangular area 304 inscribed in the subject becomes the tracking color area. The tracking position recorded in the tracking position log area is, for example, the center of gravity position of the tracking color area 304 (same as the position 302 in the example of FIG. 3A). In the next tracking process, this tracking position is set as the starting position of the tracking process.

  As an example of the N frame tracking process, the tracking position 302 of the (N-1) frame is tracked within a predetermined search range of the N frame image data stored as the reference image data shown in FIG. An area that can be determined to be the same color information as the color information of the tracking color area 304 is sequentially searched from the periphery of the tracking position 302 as the processing start position. In the example shown in the figure, the area 306 is the tracking color area. In the example shown in the figure, the center of gravity position 308 is set as the tracking position, and the tracking position 308 is recorded in the tracking position log area. In the tracking color area log area, information indicating the range of the tracking color area 306 (for example, the positions of the four corners) is recorded.

The third tracking processing circuit 1309 tracks a subject, for example, the face of a person, based on a feature, for example, an AF focus evaluation value acquired by the AF control circuit 1302, from the image data acquired by imaging by the imaging unit 101. I do.
In the case where there are a plurality of persons as subjects in the image data acquired by imaging by the imaging unit 101, one of these persons is designated as a tracking target, and the other person is not designated as a tracking target. The tracking processing circuit 1306 tracks the person designated as the tracking target by the face portion. The second tracking processing circuit 1307 tracks the person designated as the tracking target based on the color information. The third tracking processing circuit 1309 tracks a person who is not designated as a tracking target by using a focus evaluation value for AF.

The tracking process is not limited to tracking based on face recognition and color information, but tracking using luminance information may be used. Here, the tracking process using luminance information will be briefly described. In the tracking process using luminance information, for example, when a tracking target is set in the (N-1) frame shown in FIG. 4A, the image data of the (N-1) frame is used as the evaluation image data in the RAM 118. Is stored in the evaluation image area. Image data in a predetermined range 202 including the tracking target of the evaluation image data is set as reference image data. In the subsequent tracking process, a portion corresponding to the reference image data 202 of the reference image data is searched.
As an example of N frame tracking processing, N frame image data is first stored in the reference image area of the RAM 118 as reference image data. A portion of the reference image data corresponding to the reference image data 202 is searched for by obtaining a correlation amount between the image data in the predetermined search range 204 and the reference image data 202. The correlation amount is determined from, for example, the sum of absolute differences between the standard image data and the reference image data (obtained by obtaining the absolute value of the luminance difference for each pixel and integrating it). For example, if the sum of absolute differences between the reference image data in the reference image data area 206 and the standard image data 202 shown in FIG. 4B is obtained, the reference image data area 206 and the standard image data 202 are clearly shown. It is different image data, and the sum of absolute differences is increased. On the other hand, if the sum of absolute differences between the area 208 of the reference image data and the standard image data 202 is obtained, the sum of absolute differences becomes small. Thus, the difference absolute value sum decreases as the correlation amount with the reference image data 202 increases. In the tracking process using luminance information, an area having the maximum correlation amount, that is, the minimum difference absolute value sum is searched from the reference image data. In the example of FIG. 4B, this is the area 208. In the tracking position log area, the position with the highest degree of matching in the area 208 is recorded as the tracking position. When there are a plurality of such positions, for example, a position close to the center of the area 208 is set as the tracking position. In the next tracking process, this tracking position is preferably set as the start position of the tracking process. As a result, the time required for the tracking process can be reduced.

For example, as shown in FIG. 5, the focus evaluation value acquisition area setting unit 1309 includes, for example, a human face in the image data acquired by the imaging unit 101 by the first and second tracking processing circuits 1306 and 1307. A focus evaluation value acquisition area D for acquiring a focus evaluation value for the subject Q, including a plurality of areas A and C for acquiring features such as face recognition and color information for tracking the subject Q Set.
For example, the plurality of areas for acquiring face recognition and color information are a first area A for tracking by a face part and a second area C for tracking by color information. The size of the focus evaluation value acquisition area D includes the first area A and the second area C.

The focus evaluation value acquisition area setting unit 1309 has reliability for tracking by color information, and includes a first area A for tracking by face recognition and a second area C for tracking by color information. The inclusion relationship is determined, and the focus evaluation value acquisition area D is set based on the determination result.
If the first area A for tracking by the face includes the second area C for tracking by the color information, the focus evaluation value acquisition area setting unit 1309 sets the first for tracking by the face. A focus evaluation value acquisition area D including one area A is set.
If the second area C for tracking by color information includes the first area A for tracking by face, the focus evaluation value acquisition area setting unit 1309 performs the first for tracking by color information. A focus evaluation value acquisition area D including two areas C is set.

For example, as shown in FIG. 6, the tracking candidate detection unit 1310 detects a tracking candidate area including at least one subject Q from image data acquired by imaging of the imaging unit 101, and each tracking frame is included in each tracking candidate area. CA1 and CA2 are set.
The tracking area designating unit 1311 receives an instruction of a subject to be tracked as a contact detection signal output from the touch panel 124, for example, via the CPU 1301, and at least one tracking candidate area detected by the tracking candidate detection unit 1310, for example, FIG. One face from the subjects Q1 and Q2 set by the tracking frames CA1 and CA2 shown in FIG. 6, for example, the face portion of the subject Q1 such as a person set by the tracking frame CA1 is designated as the tracking area. That is, when the user's finger touches the touch panel 124 at a position corresponding to the display position of the face of the subject Q1, the touch panel 124 outputs a contact detection signal including coordinates indicating the touch position. The designation unit 1311 designates, for example, the face portion of the subject Q1 in the tracking frame CA1 indicating the tracking area from the coordinates indicating the touch position included in the contact detection signal.

The tracking area designation unit 1311 sets the tracking frame of the tracking area that has received the instruction to be tracked to a size corresponding to the size of the face of the subject Q1.
The tracking area designation unit 1311 sets the size of the tracking candidate area that has not received an instruction to be tracked to a size corresponding to the size of the face of the subject Q2.

When the face portion of the subject Q1 in the frame CA1, for example, is designated as the tracking area including the subject, the tracking prohibition area setting unit 1312, for example, in the tracking frame CA1 based on the focus evaluation value in the focus evaluation value acquisition area D. Tracking of a non-designated area other than the designated subject Q1, for example, an area within the frame CA2 shown in FIG. 6, that is, tracking of the face of the subject Q2 is prohibited.
The tracking prohibition area setting unit 1312 includes a tracking area designated by the tracking area designating unit 1311, for example, the face of the subject Q1 in the frame CA1, and an area in the frame CA2 including, for example, the subject Q2 other than the subject Q1 (non- The subject Q2 in the non-designated area, that is, the area in the frame CA2, by the first to third tracking processing units 1306 to 1308 based on the distance from the designated area) and each focus evaluation value in each focus evaluation value acquisition area D. Tracking on the face of

  The tracking prohibition area setting unit 1312 includes, for example, the face of the subject Q1 in the tracking frame CA1 as the tracking area specified by the tracking area specifying unit 1311 and the face of the subject Q2 in the frame CA2 as the non-tracking area. If the interval is within a preset first threshold and the difference between the focus evaluation values in each focus evaluation value acquisition area D is greater than or equal to a preset second threshold, the first to third tracking As a non-designated area by the processing units 1306 to 1308, for example, tracking of the face portion of the subject Q2 in the frame CA2 is prohibited.

  The tracking prohibition area setting unit 1312 has an interval between the face of the subject Q1 in the frame CA1 as the tracking area specified by the tracking area specifying unit 1311 and the face of the subject Q2 in the frame CA2 as the non-tracking area in advance, for example. When the area difference between the face of the subject Q1 and the face of the subject Q2 is within the set first threshold and the area difference is equal to or greater than a preset third threshold, the first to third As a non-designated area by the tracking processing units 1306 to 1308, for example, tracking of the face portion of the subject Q2 in the frame CA2 is prohibited.

The memory control circuit 1313 is an interface that performs control for accessing the RAM 118, the recording medium 128, and the ROM 134 by the CPU 1301 and the like.
The camera operation switch 132 is various operation members operated by the user. Examples of the camera operation switch 132 include a release button, a moving image button, a mode dial, a selection key, a power button, and the like.
The release button button has a 1st release switch and a 2nd release switch. The 1st release switch is turned on when the user presses the release button halfway. When the 1st release switch is turned on, a shooting preparation operation such as AF processing is performed. The 2nd release switch is turned on when the user fully presses the release button. When the 2nd release switch is turned on, an exposure operation for still image shooting is performed.
The moving image button is an operation member for instructing the start or end of moving image shooting. When the moving image button is pressed by the user, the moving image shooting process is started. Further, when the moving image button is pressed during the moving image shooting process, the moving image shooting process is ended.
The mode dial is an operation member for selecting shooting settings of the imaging apparatus. In the present embodiment, for example, a still image shooting mode and a moving image shooting mode can be selected as shooting settings of the imaging apparatus. The still image shooting mode is a shooting setting for shooting a still image. The moving image shooting mode is a shooting setting for shooting a moving image.
The selection key is an operation member for selecting or determining an item on the menu screen, for example. When the selection key is operated by the user, an item is selected or determined on the menu screen.
The power button is an operation member for turning on or off the power of the imaging apparatus. When the power button is operated by the user, the imaging apparatus 100 is activated and becomes operable. If the power button is operated while the imaging apparatus is activated, the imaging apparatus 100 enters a power saving standby state.

  The flash ROM 134 stores program codes for the CPU 1301 to execute various processes. The flash ROM 134 has a shooting operation flowchart of the apparatus, control parameters necessary for the operations of the photographing optical system 102, the aperture 106, the image sensor 114, and the like, control parameters necessary for image processing in the image processing circuit 1304, and the like. The various control parameters are stored. Further, the flash ROM 134 also stores data on the hail parts used for hail detection in the hail detection circuit 1307, data for displaying the tracking frame, and the like.

Next, the operation of this apparatus will be described with reference to the flowchart of the photographing operation shown in FIG.
In step S100, the CPU 1301 determines whether or not the shooting setting of the apparatus 100 is the still image shooting mode. As a result of this determination, when determining that the shooting setting is the still image shooting mode, the CPU 1301 starts a live view operation in step S102.
In this live view operation, the CPU 1301 controls the shutter drive mechanism 112 to open the shutter 110, and then controls the image sensor IF circuit 116 to start imaging by the image sensor 114. The CPU 1301 sequentially stores each image data for each frame acquired by imaging by the imaging device 114 in the work area 118 a of the RAM 118. Subsequently, the CPU 1301 inputs the image data stored in the work area 118 a of the RAM 118 to the image processing circuit 1304 and performs image processing for live view display. The CPU 1301 inputs the image data subjected to the image processing for live view display to the display element driving circuit 122, and drives the LCD 120 by the display element driving circuit 122 to display the image data on the LCD 120. By repeating such image data display operation, the moving image of the subject is displayed on the LCD 120. The user can observe a live view of a subject displayed as a moving image on the LCD 120.

In step S104, the CPU 1301 determines whether the first release switch is turned on. If the first release switch is not turned on as a result of this determination, the CPU 1301 continues the live view operation.
When the first release switch is turned on, the CPU 1301 performs a release AF process in step S106. In the release AF, the CPU 1301 drives the focus lens to the in-focus position by scan driving. In this scan drive, the CPU 1301 controls the focus adjustment mechanism 104 to drive the focus lens in one direction within a predetermined scan range, and at the same time, an AF focus evaluation value sequentially calculated by the AF control circuit 1302. evaluate.
The CPU 1301 stops driving the focus lens at the lens position where the contrast is maximized as a result of the evaluation of the focus evaluation value for AF. Such scan driving is performed when the difference between the position of the focus lens before AF and the in-focus position is large.

  In step S108, the CPU 1301 controls the tracking candidate detection unit 1310, and, for example, as shown in FIG. 6, at least one subject, for example, two subjects Q1 and Q2 from the image data acquired by the imaging unit 101 is captured. A tracking candidate area including each face is detected, and tracking frames CA1 and CA2 are set in these tracking candidate areas, respectively.

  Specifically, FIG. 8 shows image data H1 of (N-1) frames. The image data H1 includes, for example, two first and second persons M and W as subjects. The face recognition circuit 1305 recognizes the face portions F1 and F2 of the first and second persons M and W from the image data H1. The CPU 1301 controls the display element driving circuit 122, and for example, as shown in FIG. 9, each tracking frame CA1, CA2 is arranged in an area including the face portions F1, F2 of the first and second persons M, W in the image data. And the image data in which the tracking frames CA1 and CA2 are set are displayed on the LCD 120.

  Here, as shown in FIG. 10, when the face F1 of the subject Q1 is designated as a tracking target, for example, by touching the portion corresponding to the face F1 of the subject Q1 on the touch panel 124 with the finger J, The CPU 1301 sets the face F1 of the subject Q1 for which the tracking frame CA1 is set as the tracking target, and sets the face F2 of the subject Q2 for which the tracking frame CA2 is set as the non-tracking target. That is, the tracking area designating unit 1311 receives the contact detection signal output from the touch panel 124 via the CPU 1301, and tracks the face F1 of the first person M from the coordinates indicating the touch position included in the contact detection signal. The inside is designated as a tracking area. At the same time, the tracking area designating unit 1311 sets a tracking frame other than the tracking frame CA1, here the face F2 of the second person W in the tracking frame CA2 as a similar subject.

In step S110, the CPU 1301 performs a tracking process on the face F1 of the subject Q1 within the tracking frame CA1 designated as the tracking target.
In step S112, the CPU 1301 performs AF processing so that the face F1 in the tracking frame CA1 designated as the tracking target is focused, and the exposure of the face F1 in the tracking frame CA1 is appropriate. AE processing is performed.
In the AF process after the tracking process, the CPU 1301 drives the focus lens to the in-focus position by wobbling driving. In this wobbling drive, the CPU 1301 determines whether or not the focus evaluation value calculated by the AF control circuit 1302 when the focus lens is driven has increased with respect to the focus evaluation value at the previous lens position. If the focus evaluation value has increased as a result of this determination, the CPU 1301 finely drives the focus lens in the same direction as the previous time. If the focus evaluation value decreases, the CPU 1301 finely drives the focus lens in the direction opposite to the previous time. By repeating such an operation at a high speed, the focus lens is gradually driven to the in-focus position. At the same time, in the AE process, the CPU 1301 sets the brightness of the face F1 in the tracking frame CA1 calculated by the AE control circuit 1303 to a predetermined appropriate amount (appropriate exposure amount), that is, the aperture 106 opening during the main exposure. The amount (aperture value) and the opening time of the shutter 110 (shutter speed value) are calculated.

In step S114, the CPU 1301 determines whether or not the 2nd release switch is turned on. As a result of this determination, if it is determined that the 2nd release switch is not turned on, the CPU 1301 executes processing after the tracking processing in step S110. In this way, in the still image shooting mode, the CPU 1301 continues the tracking process until the 2nd release switch is turned on.
If it is determined that the 2nd release switch is turned on, the CPU 1301 controls the display element driving circuit 122 to hide the tracking frame CA1 in step S116.

In step S118, when the 2nd release switch is turned on, the CPU 1301 controls the shutter driving mechanism 112 to close the shutter 110. Thereafter, the CPU 1301 controls the aperture driving mechanism 108 to narrow the aperture 106 to the previously calculated aperture value. Subsequently, the CPU 1301 controls the shutter driving mechanism 112 and performs imaging (exposure) by the imaging element 114 while opening the shutter 110 for the previously calculated opening time. Thereafter, the CPU 1301 processes still image data obtained via the image sensor 114 in the image processing circuit 1304. The CPU 1301 adds a header to the still image data processed by the image processing circuit 1304 to generate a still image file, and records the generated still image file on the recording medium 128.
In step S120, the CPU 1301 adds data indicating the tracking position obtained as a result of the tracking process in step S110 to the still image file previously recorded on the recording medium 128. Thereafter, the CPU 1301 ends the photographing operation shown in FIG.

On the other hand, when determining in step S100 that the shooting setting is the moving image shooting mode, the CPU 1301 proceeds to step S122 and starts a live view operation.
In step S124, the CPU 1301 determines whether or not the moving image button has been turned on. If the moving image button is not turned on as a result of this determination, the CPU 1301 continues the live view operation.
If it is determined that the moving image button is turned on, the CPU 1301 controls the display element driving circuit 122 in step S126, and is acquired by imaging of the imaging unit 101 as shown in FIG. 6, for example, as in step S108. The tracking candidate area including each face portion of at least one subject, for example, two subjects Q1 and Q2, is detected from the image data, and the tracking frames CA1 and CA2 are set in the tracking candidate areas, respectively.

Here, as described above, when the face F1 of the subject Q1 is designated as a tracking target by touching the touch panel 124, for example, the CPU 1301 tracks the face F1 of the subject Q1 in which the tracking frame CA1 is set. In addition, the face F2 of the subject Q2 in which the tracking frame CA2 is set is set as a non-tracking target.
In step S128, the CPU 1301 performs a tracking process on the face F1 of the subject Q1 in which the tracking frame CA1 designated as the tracking target is set.

  In step S130, the CPU 1301 performs AF processing so as to focus on the face F1 in the tracking frame CA1 designated as the tracking target, and AE so that the exposure of the face F1 in the tracking frame CA1 is appropriate. Process. In this AF process, the focus lens is driven to the in-focus position by wobbling driving.

  In step S132, the CPU 1301 controls the diaphragm drive mechanism 108 to record the moving image data on the recording medium 128, and narrows the diaphragm 106 to the diaphragm value calculated in the AE process. Subsequently, the CPU 1301 executes imaging (exposure) by the imaging element 114 for a time corresponding to the shutter speed value calculated in the AE process. After the exposure is completed, the CPU 1301 generates a moving image file and records it on the recording medium 128. Further, the CPU 1301 processes the moving image data obtained via the image sensor 114 in the image processing circuit 1304, and records the moving image data processed in the image processing circuit 1304 in a moving image file.

In step S134, the CPU 1301 adds data indicating the tracking position as a result of the tracking process obtained in step S128 to the moving image file previously recorded on the recording medium 128.
In step S136, the CPU 1301 determines whether or not the moving image button has been turned off. As a result of this determination, if it is determined that the moving image button is not turned off, the CPU 1301 executes processing after the tracking processing in step S128. Thus, in the moving image shooting mode, the CPU 1301 continues the tracking process and the recording of moving image data until the moving image button is turned off.
When the moving image button is turned off, the CPU 1301 controls the display element driving circuit 122 to hide the tracking frame CA1 in step S138. Thereafter, the CPU 1301 adds tracking data in step S120. Thereafter, the CPU 1301 ends the photographing operation shown in FIG.

Next, the tracking process (steps S110 and S128) will be described with reference to the tracking process flowchart shown in FIG.
First, the CPU 1301 performs tracking for a subject designated as a tracking target.
In step S <b> 200, the CPU 1301 reads the (N−1) frame image data H <b> 1 stored in the work area 118 a of the RAM 118.
Here, the image data of the (N−1) frame obtained by the first tracking process is assumed to be evaluation image data. The CPU 1301 captures the image data obtained by the first tracking process into the evaluation image area 118b of the SDRAM 118. Further, the CPU 1301 uses the N-frame image data obtained in the second and subsequent tracking processes as reference image data, and takes this reference image data into the reference image area 118 c of the SDRAM 118.

In step S202, the second tracking processing circuit 1307 of the CPU 1301 detects a feature amount, for example, color information, from the image data H1 of the (N−1) frame read from the work area 118a of the RAM 118, and in the subsequent step S204. As shown in FIGS. 3A and 3B, the tracking processing of the face F1 of the subject Q1 in the tracking frame CA1 is performed using the color information, and the tracking position of the face F1 is calculated and obtained. In the tracking process using color information, the color information set in the (N-1) frame evaluation image data obtained in the first tracking process and the color information set in the N frame evaluation image data are the same. An area determined as the same color information, that is, a tracking color area is searched.
Specifically, the second tracking processing circuit 1307 acquires the color information sequentially from the position 302 toward the periphery, for example, starting from the position 302 as shown in FIGS. It is determined whether or not the acquired color information is the same as the color information at the position 302. As a result of this determination, if it is determined that the color information is the same as the color information at the position 302, it is included in the area, and if it is determined that the acquired color information is the same as the color information at the position 302, Do not include in the area.

  The CPU 1301 stores the tracking position obtained as a result of the tracking process using the color information in the tracking log area 118d of the RAM 118 as the second candidate tracking position. In step S212, the CPU 1301 stores the tracking color area obtained as a result of the tracking processing using the color information in the tracking position log area 118d of the SDRAM 118.

  In step S206, the CPU 1301 determines the reliability of tracking based on the result of the tracking processing of the human face F1 using color information. In tracking processing using color information, for example, reliability is determined from the saturation of reference image data. Specifically, when the saturation of the tracking position of the reference image data is greater than or equal to a predetermined threshold value, it is determined that there is reliability. In this reliability determination, the threshold value can be set as appropriate. In this reliability determination, if it is determined that there is no reliability, the tracking position may not be recorded.

If the tracking processing using the color information is reliable as a result of this determination, the CPU 1301 determines that the face F1 of the subject Q1 within the tracking frame CA1 designated as the tracking target is a non-tracking target in step S208. It is determined whether or not the face F2 of the subject Q2 set with a certain tracking frame CA2 overlaps. If the face F2 of the subject Q2 in the tracking frame CA2 overlaps the face F1 of the subject Q1 in the tracking frame CA1 as a result of this determination, the CPU 1301 returns to step S200, for example.
If the face F2 of the subject Q2 in the tracking frame CA2 does not overlap the face F1 of the subject Q1 in the tracking frame CA1, the CPU 1301 stores it again in the work area 118a of the RAM 118 in step S210 (N -1) The image data H1 of the frame is read, and in the subsequent step S212, the face recognition circuit 1305 is controlled, and the first and second person M, W collar F1 in the image data acquired by the image processing circuit 1304 , F2 is detected. As shown in FIGS. 2A and 2B, this wrinkle recognition process obtains each correlation amount between the image data of each frame and each face part 402, 404, 406, and this correlation amount is maximized. Suppose that a predetermined arrangement indicating the face of a person is obtained. Here, the face recognition circuit 1305 recognizes the face portion F1 of the first person M and the face portion F2 of the second person W.

  In step S <b> 214, the CPU 1301 controls the first tracking processing circuit 1306, and the first tracking processing circuit 1306 uses the feature amount, for example, the face recognition circuit 1305, from the image data acquired by the imaging unit 101. Tracking of the recognized subject, for example, the face portion F1 of the first person M is performed.

  In step S216, the CPU 1301 again overlaps the face F1 of the subject Q2 in which the tracking frame CA2 that is the non-tracking target is set on the face F1 of the subject Q1 in the tracking frame CA1 designated as the tracking target. It is determined whether or not. If the face F2 of the subject Q2 in the tracking frame CA2 overlaps the face F1 of the subject Q1 in the tracking frame CA1 as a result of this determination, the CPU 1301 returns to step S200, for example.

  If the face F2 of the subject Q2 in the tracking frame CA2 does not overlap the face F1 of the subject Q1 in the tracking frame CA1, the CPU 1301 tracks the face F1 of the subject Q1 in the tracking frame CA1 in step S218. In step S220, the tracking position of the face F1 of the subject Q1 within the tracking frame CA1 is set.

In step S222, the CPU 1301 controls the focus evaluation value acquisition area setting unit 1309, and the focus evaluation value acquisition area setting unit 1309 uses the focus evaluation value acquisition area setting unit 1309, for example, as shown in FIG. An area including a plurality of features for tracking the face portion F1 of the person M, for example, an area (color area) G for acquiring color information and an area (face area) E for acquiring face recognition And a focus evaluation value acquisition area D1 for acquiring a focus evaluation value for the face portion F1 of the first person M is set. The focus evaluation value acquisition area setting unit 1309 acquires a focus evaluation value for the face portion F1 of the first person W in the focus evaluation value acquisition area D1.
At the same time, as shown in FIG. 12, the CPU 1301 recognizes an area and face recognition for obtaining color information, for example, for tracking the face F2 of the second person W in the image data of the (N-1) frame. A focus evaluation value acquisition area D2 is set for acquiring a focus evaluation value for the face portion F2 of the second person W and having an area including the area for acquisition. The focus evaluation value acquisition area setting unit 1309 acquires a focus evaluation value for the face portion F2 of the second person W in the focus evaluation value acquisition area D2.
Subsequently, as shown in FIG. 13, the CPU 1301 sets a tracking frame CA1 on the face portion F1 of the first person M in the image data H1 of the (N-1) frame.

Next, the CPU 1301 performs tracking for a subject (non-tracking target) that has not been designated as a tracking target.
In step S224, the CPU 1301 reads N frames of image data H2 stored in the work area 118a of the RAM 118 as shown in FIG. 14, and in the subsequent step S226, the focus set by the focus evaluation value acquisition area setting unit 1309. A focus evaluation value for AF is obtained in the evaluation value acquisition area D2.

In step S228, the CPU 1301 uses the AF focus evaluation value to track the face F2 of the subject Q2 in which the focus evaluation value acquisition area D2 is set from the image data H2 of N frames. Processing is performed to calculate and obtain the tracking position of the same face F2.
In step S230, the CPU 1301 determines the reliability of the tracking process of the face F2 of the subject Q2 in the tracking frame CA2 using the AF focus evaluation value. This determination of reliability is performed based on, for example, the degree of change in luminance of the image data H1.

In step S232, the CPU 1301 sets the tracking position of the face F2 of the subject Q2 within the tracking frame CA2.
In step S234, the CPU 1301 controls the tracking prohibition area setting unit 1312. As shown in FIG. 15, the tracking prohibition area setting unit 1312 specifies, for example, the focus evaluation value acquisition area D1 as the tracking area specified by the tracking area specifying unit 1311. For example, it is determined whether or not the distance from the focus evaluation value acquisition area D2 as a non-tracking area is within a first threshold value set in advance.
As a result of this determination, the interval between the focus evaluation value acquisition area D1 and the focus evaluation value acquisition area D2 is within a preset first threshold, and the difference in area between the focus evaluation value acquisition areas D1 and D2 is determined in advance. If it is equal to or greater than the set third threshold value, the tracking prohibition area setting unit 1312 sets the focus evaluation value acquisition area D2 in the tracking prohibition area in step S236 as shown in FIG.
In step S238, the CPU 1301 updates the tracking frame CA1 set in the face F1 of the subject Q1, as shown in FIG. If the interval between the focus evaluation value acquisition area D1 and the focus evaluation value acquisition area D2 is not within the first threshold value set in advance, the CPU 1301 moves to step S238 and is set to the face portion F1 of the subject Q1. The tracking frame CA1 is updated.

  For example, FIG. 18 shows image data H3 of (N + 1) frames. This image data H3 shows a state in which tracking by color information is about to shift to the face F2 of the second person W, but the face F2 of the second person W in the tracking frame CA2 is As shown in FIG. 19, since the tracking prohibited area setting unit 1312 sets a non-tracking target, the second tracking processing circuit 1307 tracks the color information of the face F2 of the second person W within the tracking frame CA2. Even if the result is acquired, the second tracking processing circuit 1307 does not adopt the tracking result of the color information.

At the same time, the first tracking processing unit 1306 sets a face area E for acquiring face recognition for the face F1 of the first person M as shown in FIG.
Subsequently, as shown in FIG. 20, the focus evaluation value acquisition area setting unit 1309 has an area including a color area G for acquiring color information and a face area E for acquiring face recognition. A focus evaluation value acquisition area D1 for acquiring a focus evaluation value for the face portion F1 of one person M is set.
As a result, the tracking of the face portion F1 of the first person M is continued.

Next, AF evaluation value acquisition area setting (step S222) will be described with reference to an AF evaluation value acquisition area setting flowchart shown in FIG.
In step S300, the CPU 1301 controls the focus evaluation value acquisition area setting unit 1309, and the focus evaluation value acquisition area setting unit 1309 performs tracking based on the result of the tracking processing of the human face F1 using the color information in step S206. It is determined whether or not there is reliability. If the result of this determination is that the result of the tracking processing of the person's face F1 using color information is reliable, the focus evaluation value acquisition area setting unit 1309 moves to step S302, for example, as shown in FIG. It is determined whether or not the face area E for acquiring recognition includes the color area G for acquiring color information.
As a result of this determination, if the face area E includes the color area G, the focus evaluation value acquisition area setting unit 1309 moves to step S304, and the focus evaluation value acquisition areas D1 and D2 each include the face area E. Determine the area.

If the face area E does not include the color area G, the focus evaluation value acquisition area setting unit 1309 proceeds from step S302 to step S306, and determines whether the color area G includes the face area E.
As a result of this determination, if the color area G includes the face area E, the focus evaluation value acquisition area setting unit 1309 moves to step S308, and the focus evaluation value acquisition areas D1 and D2 each include the color area G. Determine the area.
If the color area G does not include the face area E, the focus evaluation value acquisition area setting unit 1309 moves to step S309, and the focus evaluation value acquisition areas D1 and D2 include the face area E and the color area G, respectively. Determine the size area.
As a result, the sizes of the focus evaluation value acquisition areas D1 and D2 are determined.

Next, the tracking prohibited area determination processing (step S236) will be described with reference to the tracking prohibited area determination processing flowchart shown in FIG.
The tracking prohibition area setting unit 1312 is, for example, a tracking area designated by the tracking area designating unit 1311, for example, the interval between the focus evaluation value acquisition area D1 and the focus evaluation value acquisition area D2 is within a first threshold value set in advance, and If the difference between the focus evaluation values in the focus evaluation value acquisition areas D1 and D2 is equal to or larger than a second threshold value set in advance, the area D2 is tracked by the first to third tracking processing units 1306 to 1308. Set to prohibited area.

  Specifically, the CPU 1301 controls the tracking prohibition area setting unit 1312, and in step S400, the tracking prohibition area setting unit 1312 sets the (N + 1) frame image data H3 as a tracking target as shown in FIG. A focus evaluation value acquisition area D1 is set for the face F1 of the designated subject Q1. At the same time, the tracking prohibition area setting unit 1312 generates a focus evaluation value acquisition area D2 having the same area as the focus evaluation value acquisition area D1 for the face F2 of the subject Q2 not designated as a tracking target for the image data H3. Set.

In step S402, the tracking prohibition area setting unit 1312 acquires a focus evaluation value for AF in the focus evaluation value acquisition area D1 set for the face F1 of the subject Q1 in the image data H3.
Subsequently, the tracking prohibition area setting unit 1312 acquires the focus evaluation value for AF in the focus evaluation value acquisition area D2 set in the face F2 of the subject Q1 in the image data H3 in step S404.

In step S406, the tracking prohibition area setting unit 1312 sets the focus evaluation value for AF in the focus evaluation value acquisition area D1 set for the face F1 of the subject Q1 and the focus evaluation set for the face F2 of the subject Q1. The difference from the focus evaluation value for AF in the value acquisition area D2 is calculated and obtained.
In step S408, the tracking prohibition area setting unit 1312 previously sets a difference between the focus evaluation value for AF in the focus evaluation value acquisition area D1 and the focus evaluation value for AF in the focus evaluation value acquisition area D2. It is determined whether or not the second threshold is equal to or greater than the second threshold.

If the result of this determination is that the difference from the focus evaluation value for AF is equal to or greater than or equal to the second threshold value set in advance, the tracking prohibition area setting unit 1312 in FIG. As shown in FIG. 24, the focus evaluation value acquisition area D2 is set as a tracking prohibited area by the first to third tracking processing units 1306 to 1308.
Subsequently, the tracking prohibition area setting unit 1312 sets the tracking frame CA1 to the face F1 of the subject Q1 as shown in FIG.
If the difference from the focus evaluation value for AF is not equal to the second threshold value set in advance or is not equal to or greater than the second threshold value, the tracking prohibition area setting unit 1312 detects the face F2 of the subject Q2. The included tracking frame CA2 is not set as a tracking prohibited area.

Next, another tracking prohibited area determination process (step S236) will be described with reference to the tracking prohibited area determination processing flowchart shown in FIG.
The tracking prohibition area setting unit 1312 calculates, for example, the area difference between the face of the subject Q1 in the frame CA1 as the tracking area specified by the tracking area specifying unit 1311 and the face of the subject Q2 in the frame CA2 as the non-tracking area, for example. If the area difference is equal to or greater than a preset third threshold, tracking of the face of the subject Q2 in the frame CA2 is prohibited as a non-designated area by the first to third tracking processing units 1306 to 1308, for example. To do.

Specifically, the tracking prohibited area setting unit 1312 acquires the area within the tracking frame CA1 set in the face F1 of the subject Q1 specified by the tracking area specifying unit 1311 in step S500.
Subsequently, in step S502, the tracking prohibited area setting unit 1312 acquires the area within the tracking frame CA2 set in the face F2 of the subject Q2 that is not specified by the tracking area specifying unit 1311.

In step S504, the tracking prohibition area setting unit 1312 calculates the difference between the area in the tracking frame CA1 set for the face F1 of the subject Q1 and the area in the tracking frame CA2 set for the face F2 of the subject Q2. Ask.
Subsequently, the tracking prohibition area setting unit 1312 determines in step S506 whether or not the difference between the area in the tracking frame CA1 and the area in the tracking frame CA2 is greater than or equal to a preset third threshold value. .

If the result of this determination is that the difference between the area in the tracking frame CA1 and the area in the tracking frame CA2 is greater than or equal to a preset third threshold, the tracking prohibited area setting unit 1312 determines that the subject Q2 in step S508. The tracking frame CA2 including the face portion F2 is set as a tracking prohibition area, and the tracking of the face Q2 of the subject Q2 in the tracking frame CA2 by the first to third tracking processing portions 1306 to 1308 is prohibited.
Subsequently, the third tracking processing circuit 1309 sets the tracking frame CA1 to the face portion F1 of the subject Q1.
Note that if the difference between the area in the tracking frame CA1 and the area in the tracking frame CA2 is not equal to the third threshold value or greater than or equal to the third threshold value, the tracking prohibition area setting unit 1312 determines the face portion of the subject Q2. The tracking frame CA2 including F2 is not set as a tracking prohibited area.

  As described above, according to the above-described embodiment, the focus evaluation value acquisition area setting unit 1309 defines the first area A for tracking by face recognition and the second area C for tracking by color information. When the inclusion relationship is determined, the focus evaluation value acquisition area D is set based on the determination result, and the face portion of the subject Q1 within the frame CA1 is designated by the tracking prohibition area setting unit 1312, for example, the focus evaluation value acquisition area Based on the area of D, for example, tracking of the tracking frame CA2 in the non-designated area other than the subject Q1 specified in the tracking frame CA1 is prohibited. Thereby, the tracking performance can be improved by using an AF evaluation value appropriate for tracking. For example, there is no possibility that the moving subject cannot be detected due to low contrast as in the case where the AF area is smaller than the moving subject, and the periphery of the subject is included as in the case where the AF area is larger than the moving subject. Therefore, it is not possible to acquire the AF evaluation value of only the moving subject.

  The size of the focus evaluation value acquisition area D can be set to an appropriate size from the inclusion relationship between the first area A for tracking by face recognition and the second area C for tracking by color information. It is. For example, if the first area A for tracking by the face part includes the second area C for tracking by color information, the focus evaluation value acquisition area D is the first area for tracking by the face part. The size can include one area A. If the second area C for tracking by the color information includes the first area A for tracking by the face, the focus evaluation value acquisition area D is the second area for tracking by the color information. A focus evaluation value acquisition area D including the area C can be set.

Further, for example, prohibition of tracking in the tracking frame CA2 including the subject Q2 other than the tracking area designated as the tracking area is, for example, two methods, that is, the face F1 of the subject Q1 in the tracking frame CA1 as the tracking area, As the non-tracking area, the distance between the face Q2 of the subject Q2 in the tracking frame CA2 is within a preset first threshold, and the difference between the focus evaluation values in each focus evaluation value acquisition area D is set in advance. If it is equal to or greater than the second threshold, tracking of the face F2 of the subject Q2 within the tracking frame CA2 can be prohibited.
Further, an area difference between the face F1 of the subject Q1 in the tracking frame CA1 as the tracking area and the face F2 of the subject Q2 in the tracking frame CA2 as the non-tracking area is obtained, and the area difference is set in advance. If the threshold is equal to or greater than the threshold, tracking of the face portion of the subject Q2 in the frame CA2 can be prohibited as a non-designated area by the first to third tracking processing units 1306 to 1308, for example.

In the AF process after the tracking process, the CPU 1301 drives the focus lens to the in-focus position by wobbling driving. Here, the wobbling operation will be described with reference to an explanatory diagram shown in FIG.
The contrast AF method is a function for performing autofocus control that automatically adjusts the focus, and extracts the contrast and high frequency components of the AF area of the captured image to generate an AF evaluation value, and the AF evaluation value is the highest. Adjust the lens position so that

  FIG. 1 shows the change of the AF evaluation value with respect to the focus lens position. The AF evaluation value with respect to the focus lens position becomes the maximum value (peak value) at the in-focus position, and becomes smaller as the distance from the in-focus position (peak position) increases to the far side (rear pin side) or near side (front pin side). Become. Therefore, when performing autofocus control, the position of the focus lens is controlled so that the AF evaluation value becomes a peak value.

In order to move the focus lens to the position where the AF evaluation value reaches the peak value, the direction of the focus position relative to the current focus lens position, that is, the far position from the focus position It must be determined whether it is located on the short distance side.
In this specific method, the focus lens is microvibrated (wobbled) to the extent that it does not affect the image, and the variation direction of the AF evaluation value is detected.

Next, for example, phase difference AF on the imaging surface of the imaging element 114 will be described with reference to FIGS.
Phase difference AF is possible by receiving a light beam that has passed through different portions of the exit pupil by the image sensor 114. On the imaging surface of the imaging element 114, the pixel rows for AF are arranged in a Bayer array. It is possible to perform focus detection by a phase difference detection method based on the outputs of the image pickup elements 114 of the AF pixel array arranged in a part of the Bayer array on the image pickup surface of the image pickup element 114.

  FIG. 28 shows the configuration of the AF pixel column. In the AF pixel row, the light shielding portions 300a and 300b in which the positions of the openings for separating the light beam 200a from the upper portion 100a of the exit pupil of the photographing lens and the light beam 200b from the lower portion 100b are symmetrical. A plurality of pixels 400a and 400b having a plurality of pixels are arranged along the vertical direction.

  In the first AF pixel 400a, the opening of the light shielding unit 300a is biased downward with respect to the photoelectric conversion unit. In the second AF pixel 400b, the opening of the light shielding portion 300b is biased upward. The first AF pixel 400a and the second AF pixel 4b are alternately arranged in the AF pixel column. The first AF pixel 400a and the second AF pixel 400b do not need to be arranged continuously, and may be arranged discretely in a Bayer array.

  The light beam 200a from the upper portion 100a of the exit pupil passes through the openings of the microlens 500a and the light shielding unit 300a and is received by the photoelectric conversion unit of the first AF pixel 400a. The light beam 200b from the lower portion 100b of the exit pupil passes through the openings of the microlens 500b and the light shielding unit 300b and is received by the photoelectric conversion unit of the second AF pixel 400b. In this way, the light beam passing through different portions of the exit pupil is detected.

  The pixel output of the first AF pixel 400a is the a-column pixel output, and the pixel output of the second AF pixel 400b is the b-column pixel output. In FIG. 29, a column pixel output including the pixels 400a1 to 400a3 is indicated by Da. The b column pixel output including the pixels 400b1 to 400b3 is denoted by Db. Thereby, a pair of images Da and Db are generated.

A phase difference f is generated between the a-column and b-column pixel outputs in the arrangement direction of the AF pixel columns. Based on the principle of the phase difference AF method, the defocus amount and defocus direction of the photographing lens relating to the subject corresponding to the AF pixel row can be obtained from the phase difference f.
A plurality of AF pixel columns are provided at predetermined intervals in an imaging region corresponding to the entire shooting screen, and a defocus amount can be obtained over the entire shooting screen.
Further, a known TTL phase difference detection unit separate from the image sensor may be provided, and the relative distance may be obtained by detecting the defocus amount and the defocus direction. Further, an external light type phase difference detection unit that uses a light beam different from the light beam passing through the photographing lens may be provided, and the subject distance may be detected to obtain the relative distance.

Although the present invention has been described above based on the embodiments, the present invention is not limited to the above-described embodiments, and various modifications and applications are naturally possible within the scope of the gist of the present invention.
Further, the above-described embodiments include various stages of the invention, and various inventions can be extracted by appropriately combining a plurality of disclosed constituent elements. For example, even if some configuration requirements are deleted from all the configuration requirements shown in the embodiment, the above-described problem can be solved, and this configuration requirement is deleted when the above-described effects can be obtained. The configuration can also be extracted as an invention.

  DESCRIPTION OF SYMBOLS 100: Imaging device, 102: Imaging optical system, 104: Focus adjustment mechanism, 106: Aperture, 108: Aperture drive mechanism, 110: Shutter, 112: Shutter drive mechanism, 114: Image sensor, 116: Image sensor interface (IF) Circuit: 118: SDRAM, 120: Display element, 122: Display element driving circuit, 124: Touch panel, 126: Touch panel driving circuit, 128: Recording medium, 1301: CPU, 130: System controller, 132: Camera operation switch, 134: ROM, 101: imaging unit, 118a: work area, 118b: evaluation image area, 118c: reference image area, 118d: tracking position log area, 118e: contrast information log area, 128a: image file, 1302: AF control circuit, 1303 : AE control circuit, 304: Image processing circuit, 1305: Face recognition circuit, 1306: First tracking processing circuit, 1307: Second tracking processing circuit, 1308: Third tracking processing circuit, 1309: Focus evaluation value acquisition area setting unit, 1310 : Tracking candidate detection unit, 1311: tracking area designation unit, 1312: tracking prohibition area setting unit, 1313: memory control circuit, 402, 404, 406: face parts.

Claims (16)

An imaging unit for imaging a subject;
A focus having an area including a plurality of areas for acquiring a plurality of features for tracking the subject in the image data acquired by imaging of the imaging unit, and for acquiring a focus evaluation value for the subject A focus evaluation value acquisition area setting unit for setting an evaluation value acquisition area;
A tracking unit for tracking the subject;
When a tracking area including the subject is designated, the tracking unit prohibits tracking by the tracking unit for non-designated areas other than the designated tracking area based on the focus evaluation value in the focus evaluation value acquisition area. Tracking area setting section,
Equipped with,
The tracking prohibition area setting unit is configured such that an interval between the tracking area and the non-designated area is within a preset first threshold, and a difference between the focus evaluation values in the focus evaluation value acquisition areas is determined in advance. If it is greater than or equal to the set second threshold, or the distance between the tracking area and the non-designated area is within a preset first threshold, and the area difference between the tracking area and the non-designated area Is a third threshold value set in advance , the tracking device prohibits tracking of the non-designated area by the tracking unit. An imaging unit for imaging a subject;
A tracking unit that tracks the subject according to different characteristics from image data acquired by imaging of the imaging unit;
A focus evaluation value acquisition area setting that includes an area including a plurality of tracking areas for performing the plurality of tracking by the tracking unit, and sets a focus evaluation value acquisition area for acquiring a focus evaluation value for the subject And
A tracking area designating unit for designating a tracking area including the subject to be tracked from at least one of the subjects included in the image data acquired by imaging of the imaging unit;
Based on the interval between the tracking area designated by the tracking area designation unit and the non-designated area including the other subject, and the focus evaluation values in the focus evaluation value acquisition areas, the tracking unit performs the tracking evaluation by the tracking unit. A tracking prohibited area setting section which is an area for which tracking is prohibited for a non-designated area;
Equipped with,
The tracking prohibition area setting unit includes an interval between the tracking area designated by the tracking area designating unit and the non-designated area within a preset first threshold, and within each focus evaluation value acquisition area. If the difference between the focus evaluation values is greater than or equal to a preset second threshold, or the interval between the tracking area and the non-designated area is within a preset first threshold and the tracking area When the area difference between the non-designated area and the non-designated area is equal to or greater than a preset third threshold , the tracking device prohibits the tracking of the non-designated area by the tracking unit. The tracking unit includes tracking of the face portion of the subject and tracking based on the color information of the subject,
The focus evaluation value acquisition area setting unit includes an area including a first area for tracking the face portion and a second area for tracking the color information, and the focus evaluation value is Set the focus evaluation value acquisition area to acquire,
The tracking device according to claim 1, wherein: When the focus evaluation value acquisition area setting unit has reliability for tracking the color information, the first area for tracking the face portion and the second area for tracking the color information; The tracking device according to claim 3 , wherein the focus evaluation value acquisition area is set based on the determination of the inclusion relation. The focus evaluation value acquisition area setting unit includes the focus including the area for tracking the face if the area for tracking the face includes the area for tracking the color information. The tracking device according to claim 4 , wherein an evaluation value acquisition area is set. The focus evaluation value acquisition area setting unit, if the area for tracking of the color information if encompasses an area for tracking of the face portion, the focus encompasses an area for tracking of the color information The tracking device according to claim 4 , wherein an evaluation value acquisition area is set . The tracking unit according to claim 3, wherein the perform tracking by said color information and said face portion of said object with respect to the tracking area, performing tracking by the focus evaluation value with respect to the non-designated area The tracking device described in 1. A tracking candidate detection unit that detects a tracking candidate area including at least one of the subjects from the image data acquired by imaging of the imaging unit, and sets a frame in the tracking candidate area;
When the tracking area designating unit receives an instruction for the tracking candidate area as the tracking target for the tracking candidate area in which the frame is set, the tracking area designating unit designates the tracking candidate area that has received the instruction as a tracking area. ,
The tracking device according to claim 2. The tracking device according to claim 8 , wherein the tracking area specifying unit sets the frame of the tracking area that has received the instruction to be tracked to a size according to the size of a face . 10. The tracking according to claim 9 , wherein the tracking area designating unit sets the size of the tracking candidate area that has not received an instruction to be the tracking target to the same size as the size of the frame of the tracking area. apparatus. The tracking device according to any one of claims 1 to 10 ,
An image processing unit that performs image processing on the image data acquired by imaging of the imaging unit;
A display unit for displaying the image data image-processed by the image processing unit;
An imaging apparatus comprising: Focus for having the encompassing area of the plurality of areas for obtaining a plurality of features for tracking the object Utsushitai in the image data acquired by the imaging of the imaging unit, and obtains a focus evaluation value for the object Set the evaluation value acquisition area,
When the tracking area including the object is specified, and prohibit the previous SL add tails for said non-designated areas other than the tracking area where the specified based on the focus evaluation value of the focus evaluation value acquisition area,
The interval between the tracking area and the non-designated area is within a preset first threshold, and the difference between the focus evaluation values in the focus evaluation value acquisition area is equal to or greater than a preset second threshold. Or the interval between the tracking area and the non-designated area is within a preset first threshold, and the area difference between the tracking area and the non-designated area is set in advance. If it is greater than or equal to the threshold, tracking for the non-designated area is prohibited.
The tracking method used for the imaging device characterized by the above-mentioned. The focus evaluation value acquisition area, claim 12, characterized in that it comprises a first area for tracking the face of the subject, and a second area for tracking color information of the object The tracking method used for the imaging device of description. When the focus evaluation value acquisition area has reliability for tracking the color information, the first area for tracking the face portion and the second area for tracking the color information The tracking method used for the imaging apparatus according to claim 13 , wherein the focus evaluation value acquisition area is set based on the determination of the inclusion relation. The focus evaluation value acquisition area is set such that if the area for tracking the face includes the area for tracking the color information, the focus including the area for tracking the face is included. The tracking method used for the imaging apparatus according to claim 13 , wherein an evaluation value acquisition area is set. The focus evaluation value acquisition area is set so that the color information tracking area includes the color information tracking area if the color information tracking area includes the face tracking area. The tracking method used for the imaging apparatus according to claim 13 , wherein an evaluation value acquisition area is set.

Images