JP2010074315A - Object tracking method and imaging device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To more accurately determine success or failure of object tracking in various photographic scenes and photographic sensitivities. <P>SOLUTION: An object tracking function in an imaging device detects a partial area corresponding to a partial area of a tracking object of the preceding image frame in the current image frame on the basis of a correlation in partial areas of a plurality of image frames that are imaged to thereby track the partial area between the image frames. Then, the object tracking function stores correlations acquired about a corresponding partial area in the current image frame in the tracking processing to thereby store a correlation about a corresponding partial area of a plurality of past image frames, and determines whether to succeed in tracking the partial area by the tracking processing on the basis of a change in a plurality of correlations that are stored. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an imaging apparatus having a subject tracking function and a subject tracking method thereof.

  2. Description of the Related Art Conventionally, there is an imaging system that performs subject tracking in a monitoring device using a digital camera or a monitoring camera. In this type of imaging system, an appropriate shooting condition is set for a subject or signal processing is performed by using a subject tracking function.

  Japanese Patent Application Laid-Open No. 2004-151561 describes an automatic subject tracking device that calculates a motion vector of a subject from a photographing signal and rotates a camera according to the motion of the subject. The subject automatic tracking device of Patent Document 1 can quickly track the subject and take an image when the zoom position is telephoto, and when the zoom position is wide, the camera rotation speed is quick and the subject is loosened when the zoom position is wide. It is what I did.

  Patent Document 2 describes an imaging apparatus that detects the skin color area and controls the WB and exposure using the detected area as a tracking frame to improve the optimal white balance and exposure control performance. Yes.

  Various proposals have been made, such as subject tracking systems and white balance and exposure control using the subject tracking function. As these subject tracking control methods, a tracking method using image correlation between frames in a subject area in which a motion vector is calculated, and a tracking method that performs subject tracking based on image feature values are generally used. Among these, in the subject tracking method using the image correlation, for example, when another moving object crosses in front of the subject, the image correlation of the main subject becomes low, and there is a possibility of tracking the crossed subject. high. Also, with the main subject tracking method based on image feature amounts such as color, it is difficult to keep track of the main subject when similar subjects exist around the main subject.

That is, in both methods of tracking using image correlation and tracking using image features, there are scenes that are unsatisfactory, so it is effective to use a tracking method that combines these methods. For example, tracking is basically performed by face detection (tracking using image features), and only when face detection becomes impossible, such as when the face is turned sideways, a tracking method using image correlation is used. Can improve the tracking performance.
Japanese Patent Laid-Open No. 06-054326 JP 2007-104200 A

  However, even in subject tracking using a combination of image features and image correlation as described above, tracking continues if face detection fails and image correlation of the main subject becomes low during tracking by image correlation. Becomes impossible. For example, in the case where another moving object crosses in front of the subject while tracking by image correlation after face detection fails, tracking may not be continued. Therefore, when tracking is performed using image correlation, it is necessary to determine whether tracking has succeeded or whether tracking has failed due to the influence of an intruder or the like.

  In general, when determining whether tracking by image correlation is successful, it is determined that subject tracking has failed (subject has been lost) when the image correlation of the subject region is small (when the interframe difference is large). . Such a determination method is considered to be sufficiently effective when tracking is performed on a photographic subject such as a surveillance camera or a subject or scene whose photographing sensitivity is relatively constant. However, when the shooting scene and shooting sensitivity are not limited as in the case of a digital camera, it is difficult to determine whether the subject tracking is successful only with the image correlation value. For example, when there is a lot of noise during high-sensitivity shooting, or when the subject itself contains moving objects, the subject area moves more than a certain value. It is difficult to judge whether the subject tracking is successful.

  The present invention has been made in view of the above problems, and an object of the present invention is to make it possible to more accurately determine the success or failure of subject tracking in various shooting scenes and shooting sensitivities.

In order to achieve the above object, an imaging apparatus according to an aspect of the present invention has the following arrangement. That is,
Based on the degree of correlation in the partial areas of the plurality of captured image frames, the partial area corresponding to the tracking target partial area of the previous image frame is detected in the current image frame, so that the part between the image frames is detected. A tracking means for tracking the area;
Storing means for storing the correlation degrees related to the corresponding partial areas of a plurality of past image frames by storing the correlation degrees acquired for the corresponding partial areas in the current image frame in the tracking means; ,
Determination means for determining whether or not the tracking unit has successfully tracked the partial area based on a plurality of correlation degree changes stored by the storage unit.

In addition, a subject tracking method according to another aspect of the present invention for achieving the above object is as follows:
Based on the degree of correlation in the partial areas of the plurality of captured image frames, the partial area corresponding to the tracking target partial area of the previous image frame is detected in the current image frame, so that the part between the image frames is detected. A tracking process to track the area;
By storing in the memory the correlation degree acquired for the corresponding partial area in the current image frame in the tracking step, the correlation degree for the corresponding partial area of a plurality of past image frames is stored in the memory. A saving process to save;
And a determination step of determining whether or not the tracking of the partial region by the tracking step is successful based on a plurality of changes in the correlation degree stored in the memory.

  According to the present invention, the success or failure of subject tracking can be determined more accurately in various shooting scenes and shooting sensitivities.

  Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

  FIG. 1 is a block diagram illustrating a configuration example of an imaging apparatus according to the present embodiment. In FIG. 1, when a user gives a shooting instruction via the user interface 103, the system controller 107 performs a shooting process. In the photographing process, the system controller 107 controls the focal position of the lens barrel device 112, the aperture / mechanical shutter, the image sensor 101 which is a sensor such as a CCD, and the light emitting device (not shown) when performing flash photographing. Take a picture. When shooting is performed, an image signal is output from the image sensor 101 and stored in the buffer memory 102 as CCD image data. Thereafter, signal processing for image generation is performed by the signal processing circuit 111 to generate a YUV image, which is stored in the buffer memory 102.

  When image recording is performed, the YUV image in the buffer memory 102 is sent to the compression / decompression unit 104. The compression / decompression unit 104 performs image compression processing on the YUV image to generate a JPEG file. The generated JPEG file is recorded on the recording medium 106 by the recording control unit 105.

  In addition, the display control unit 108 generates display image data according to the YUV image stored in the buffer memory 102 or a command from the system controller 107. The D / A unit 109 converts the generated display image data into an analog signal and displays it on the monitor device 110.

  Further, the face detection unit 120 performs face detection from YUV or CCD image data stored in the buffer memory 102, and outputs face detection information including the coordinate position in the image. When face detection is successful in the face detection unit 120, the system controller 107 instructs the display control unit 108 to display a face frame (subject frame) at the face position.

  Also, when subject tracking is performed in moving image shooting or electronic viewfinder (EVF), the subject tracking unit 121 acquires the position of the subject frame on the image from the system controller 107. The subject tracking unit 121 tracks the subject by tracking the partial area based on the degree of correlation between the captured image frames in the partial area specified by the subject frame (also referred to as the subject area). Realize. The subject tracking unit 121 calculates a motion vector of the subject area indicated by the subject frame and outputs the motion vector to the system controller 107. The system controller 107 changes the position of the subject frame based on the motion vector and notifies the display control unit 108 of the change.

  When the system controller 107 determines a subject frame for the first time, a result (face detection information) detected by the face detection unit 120 may be used, or a user specified area specified by the user interface 103 may be used. May be. Further, the position of the object recognition result other than the face detection may be used. In the present embodiment, a method for tracking a face detection result area based on face detection information will be described.

  Note that some or all of the functions of the respective units described above may be realized by a computer (CPU) executing a program stored in a program memory (not shown).

  FIG. 6 is a flowchart for explaining subject tracking processing based on image correlation in the imaging apparatus of the present embodiment.

  In step S201, the system controller 107 acquires an image (current frame image) acquired from the image sensor 101 and held in the buffer memory 102, and develops it on an image memory (not shown). The face detection unit 120 performs face detection processing (described later with reference to FIG. 4 and the like) on the developed image. In step S202, it is determined whether face detection by the face detection unit 120 is successful for the image. When the face detection by the face detection unit 120 is successful, the result of the face detection is used as a tracking result, and thus tracking by image correlation is terminated. On the other hand, if face detection fails, the process proceeds to step S203 in order to perform tracking processing based on image correlation.

  In step S <b> 203, the system controller 107 acquires a subject area (partial area specified by the subject frame) in the previous frame and sets it in the subject tracking unit 121. In step S204, the subject tracking unit 121 calculates an image correlation related to a partial region between the previous frame and the current frame. In step S205, the subject tracking unit 121 detects, from the current frame, a partial region that has a tracking relationship with the subject region of the previous frame based on the calculated image correlation, and calculates a motion vector thereof. In step S206, the system controller 107 calculates the coordinates where the subject frame has been moved by the amount of motion vector calculated by the subject tracking unit 121, and corrects the display position of the subject frame.

  In step S207, the system controller 107 calculates the inter-frame difference value between the subject area image of the previous frame and the subject area after tracking detection in the current frame. The inter-frame difference value may be the inverse of the image correlation calculated in step S204. In step S208, the system controller 107 stores the inter-frame difference value calculated in step S207 for a plurality of frames. In step S209, the system controller 107 calculates an intruder detection evaluation value from the past inter-frame difference value accumulated in step S208. For example, a maximum value and a minimum value of a difference value are selected in a range of a predetermined number of frames from a frame under evaluation (current frame), the difference is calculated as a motion evaluation value, and this is used as an intruder detection evaluation value.

  FIG. 8 is a diagram illustrating an example of a method for calculating an intruder detection evaluation value. As a specific example, the inter-frame difference value of the subject area of the past plural frames (22 frames in FIG. 8) is calculated, and the maximum value and the minimum value are selected from among the difference values. Of course, the motion evaluation value is not limited to such a difference, and the maximum and minimum magnification values of the inter-frame difference values may be used. In FIG. 8, the vertical axis uses the inter-frame difference value, and the smaller the value, the more similar the images in the subject frame (the higher the image correlation) (in FIGS. 9 and 10). The same).

  In step S210, the system controller 107 determines a detection threshold used to determine whether tracking has failed (whether tracking is lost) or not based on the evaluation value calculated in step S209. This detection threshold is adaptively set according to the shooting conditions of the camera, the shooting scene, and the like.

  FIG. 9 is a graph showing the transition of each frame difference value between ISO100 and ISO400. As shown in this graph, as the sensitivity increases, the light shot noise and random noise due to the dark current component are amplified, so that the inter-frame difference value increases.

  As can be seen from this example, since the amount of noise changes according to the sensitivity, it is impossible to detect the success or failure of tracking with the absolute value of the inter-frame difference value. Therefore, by comparing the difference between the maximum value and the minimum value of the image difference values in a predetermined number of recent past frames with the detection threshold, the scene in which tracking has failed (the image difference value has increased). Scene) can be detected with high accuracy. As a comparison target, a difference between a steady value of inter-frame difference values immediately after tracking start, such as an average value of inter-frame difference values for a predetermined number of frames immediately after tracking start, and an inter-frame difference value in the target frame is set as a detection threshold. You may make it compare.

  Similarly, the detection threshold value may be determined depending on the noise amount of the camera (sensor individual) or a value corresponding to the noise amount. For example, the noise amount includes dark noise and optical noise. For the dark noise, for example, a standard deviation value of an image signal value photographed while being shielded from light can be used. For optical noise, for example, charts can be photographed, and the standard deviation value of the chart portion can be used as the amount of noise. These noise amounts may be set in advance at the time of shipment or the like, or a mode for measuring these noises may be provided in the imaging apparatus. Further, for example, since the noise amount depends on the gain amount with respect to the sensor output, the threshold value may depend on a gain setting value for adjusting the sensitivity of the imaging apparatus.

  Further, in a general camera, the photographing sensitivity depends on the subject luminance value (Bv value) obtained by weighting the luminance obtained from the image frame for each region, so that the detection threshold is set according to the Bv value. Needless to say, it may be variable. The detection threshold may be set according to an image luminance value (Y value) that is the luminance of only the subject area in the image frame. When the luminance value of the subject area is high, the value of the inter-frame difference value is also large, so it goes without saying that the detection threshold value may be varied according to the image luminance value of the subject area. Conversely, the image difference value may be normalized with the average value of the luminance values of the subject area without changing the detection threshold. In addition to this, for example, when the focal length becomes long, the subject is more likely to come out of the angle of view, so the detection threshold may be lowered. Further, since the inter-frame difference value increases as the size of the subject tracking frame (subject frame) increases, a detection threshold value may be set for each image size or pixel of the subject frame. The average value may be obtained by the number of pixels. Further, since the degree to which the subject can move between frames decreases as the frame rate increases, the detection threshold may be set lower as the frame rate increases. As described above, the detection threshold can be determined by at least one of the imaging sensitivity, the gain setting value, the noise amount of the individual sensor, the subject luminance (Bv value), the image luminance value (Y value) of the subject region, and the frame rate. .

  FIG. 10 shows an example in which the image correlation between frames decreases when the number of frames elapses (an example in which the inter-frame difference value increases). This indicates that if the subject tracking was successful immediately after the start of tracking, the image correlation was significantly lower than that. Specifically, the subject tracking failed and another object or plane There is a high possibility that the tracking frame follows the target.

  As can be seen, while the subject tracking is successful, it is expected that the inter-frame difference value is at a constant level or becomes smaller as time elapses after the tracking is started.

  As in this example, when the maximum value is delayed in the temporal relationship between the maximum value and the minimum value of the image difference, there is a high possibility that tracking has failed. Therefore, a detection threshold smaller than the detection threshold (detection threshold used in FIG. 8) when the inter-frame difference value is decreasing (when the image correlation is increasing) is set, and the inter-frame difference value is temporally determined. Is increased, it is determined that tracking has failed by comparison with the detection threshold.

  FIG. 11 shows an example of setting the intruder detection threshold as described above. The detection threshold increases as the sensitivity increases, and the detection threshold is set according to the image luminance value (Y value) of the subject area. It shows how it is getting higher. In FIG. 11, when the inter-frame difference value is decreasing in time series (when the image correlation is increasing) and when the inter-frame difference value is increasing in time series Both threshold values (rising system) are set (when the image correlation is falling).

  As described above, when the image frame that has obtained the maximum value of the inter-frame difference value is later in time than the image frame that has obtained the minimum value, the image frame that has obtained the maximum value of the inter-frame difference value is the smallest The detection threshold is changed depending on the time before the image frame from which the value was obtained. For example, a new value for a temporally old inter-frame difference value is larger than a first detection threshold, and a new one for a temporally old one is larger than a first detection threshold. 2 is detected to be smaller than the detection threshold value. For example, change the threshold to detect that an intruder has entered the subject area when going up, and that an intruder that entered the subject area has come out or that movement of the subject area has stopped moving down. Is preferred.

  Returning to FIG. 6, in step S212, the system controller 107 compares the evaluation value calculated in step S209 with the detection threshold value determined in step S210, and determines whether or not the tracking is lost. In this embodiment, the difference between the maximum value and the minimum value in the last 22 frames of the difference value between the frames in the last 22 frames is compared with the detection threshold value. Is determined to be lost. As described above, the system controller 107 determines whether or not to continue tracking the subject (whether or not the tracking is successful) based on the plurality of correlation degree changes stored for the tracking region.

  If it is determined in step S213 that the tracking has been successful, the process returns to step S201. If it is determined that the tracking has failed, the process proceeds to step S214. In step S214, the system controller 107 instructs the display control unit 108 to delete the subject frame, and ends the tracking. The display control unit 108 erases the subject frame displayed on the monitor device 110 in response to an instruction to erase the subject frame.

Next, a face detection method by the face detection unit 120 will be described. As a face detection method,
・ Methods using learning represented by neural networks,
・ A method using template matching for parts with physical shape features such as eyes and nose from the image area,
・ A method using statistical analysis to detect image features such as skin color and eye shape,
Many methods have been proposed, and face recognition is generally performed by combining a plurality of these methods.

  Typical products currently proposed include a face detection method using wavelet transform and image feature amounts, a method combining template matching, and the like.

  In this embodiment, any method may be adopted as the face detection method, but an example thereof will be described below. FIG. 4 is a flowchart showing face detection processing according to the present embodiment, and FIG. 3 is a diagram for explaining specific contents of face detection. The target image for face detection in FIG. 3 is assumed to be (a) in FIG. In step S <b> 101, the face detection unit 120 performs a skin color area extraction process for extracting a skin color area in the image. FIG. 2 is a chromaticity diagram showing representative colors in the CIELAB Lab color space, and an ellipse 201 in the chromaticity diagram is an area that is highly likely to be a skin color. FIG. 3B is a diagram illustrating a state in which an area that is the chromaticity of the skin color area is extracted from the image. In step S102, the face detection unit 120 applies a high-pass filter to the image. (C) of FIG. 3 is an image obtained by performing processing using a high-pass filter on the image of (b). For such high-pass filter processing, for example, a two-dimensional high-pass filter having coefficients as shown in FIG. 5 may be used.

  In step S103, the face detection unit 120 performs template matching to detect eyes in the image. In step S <b> 104, the face detection unit 120 performs face recognition based on the positional relationship between the detected eye regions, and extracts feature amounts such as direction and size. Thus, the face detection unit 120 detects a face region from the image, generates face detection information, and outputs it.

  Next, a method for calculating an inter-frame difference value used for subject tracking according to the present embodiment will be described. Many methods for detecting the correlation and motion vector of a subject have been proposed, and any method can be used. Here, the simplest method will be described.

  FIG. 7 is a flowchart illustrating a process of calculating an image correlation by the subject tracking unit 121 according to the present embodiment.

  In step S301, the subject tracking unit 121 acquires image data of a reference partial region that is a source of correlation. In the present embodiment, the image is a partial area specified in the subject frame of the previous frame, and the subject frame corresponding to the face area detected by the face detection unit 120 or the user-specified area at the start of tracking The subject frame moved by the motion vector later corresponds to this. Next, in step S302, the subject tracking unit 121 acquires a current frame for which image correlation is performed.

  In step S303, the position of the subject frame in the current frame is shifted to any one of vertical, horizontal, and diagonal directions, and the image data in the subject frame in the current frame is used as image data of the tracking target partial area (target partial area). get. Here, the movement of the position of the subject frame is performed within a predetermined vector movement amount detection range in units of a predetermined number of pixels. Therefore, a wide range can be seen by setting the vector movement amount detection range to be large, but detection takes time due to an increase in comparison objects. Further, if the vector movement amount detection range is narrowed, the detection speed is improved but the detection capability is lowered.

  In step S304, the subject tracking unit 121 calculates an image difference value between the image of the target partial area and the image of the reference partial area obtained by performing pixel shifting. In step S305, the subject tracking unit 121 acquires the integral value of the absolute value of the image difference in step S304. Needless to say, the smaller the absolute value of the integral value of the image difference, the higher the image correlation.

  In step S306, when the correlation value acquired in step S305 is the highest among the correlation values acquired so far (the difference value between the pixels is the smallest), the subject tracking unit 121 determines the correlation value and the time at that time. Save the subject frame position (shift amount).

  In step S308, the subject tracking unit 121 determines whether the movement of the subject frame has been completed for the entire range of the vector movement amount detection range. If not, the process returns to step S303. On the other hand, if image correlation values have been calculated for the entire vector movement amount detection range, the image correlation value having the highest correlation and the shift amount (motion vector) at that time are calculated in step S309. A determination that the tracking has failed may be added when the image correlation value having the highest correlation does not reach a predetermined threshold value.

  As described above, according to the subject tracking function of the present embodiment, it is possible to accurately perform tracking performance and failure determination regardless of the subject, environment, and camera sensitivity setting. That is, when subject tracking, the correlation value varies greatly depending on the noise component at the time of high sensitivity and the movement of the subject's background, so it is generally difficult to accurately determine whether the subject can be tracked using only the subject correlation value between frames. Solve the problem of being. Further, in the subject tracking process of the present embodiment, whether or not the tracking is successful is determined not only by the absolute value of the inter-frame difference value, but also by the change of the inter-frame difference value in a plurality of frames, the scene, the subject, and the imaging device. A threshold depending on the setting is used. For this reason, it is possible to detect whether the subject has entered with high accuracy and to determine whether or not the tracking state is successful. Of course, it is also possible to make a determination using image correlation instead of the inter-frame difference value.

  In addition, an imaging apparatus using such a subject tracking function can appropriately perform optimal shooting settings and signal processing for a subject and display / erase of a subject frame indicating that the subject is being tracked. In particular, according to the subject tracking function of the present embodiment, it is possible to adaptively determine tracking based on fixed pattern noise or image correlation in a dark place, so it is possible to control the setting of shooting conditions that can be placed in the imaging apparatus with high accuracy. It becomes. Even in the case of such a subject, the tracking performance can be improved, and user convenience can be improved.

  Although the embodiment has been described in detail above, the present invention can take an embodiment as a system, apparatus, method, program, storage medium, or the like. Specifically, the present invention may be applied to a system composed of a plurality of devices, or may be applied to an apparatus composed of a single device.

  In the present invention, the functions of the above-described embodiments are achieved by supplying a software program directly or remotely to a system or apparatus, and the computer of the system or apparatus reads and executes the supplied program code. Including the case. In this case, the supplied program is a computer program corresponding to the flowchart shown in the drawings in the embodiment.

  Accordingly, since the functions of the present invention are implemented by computer, the program code installed in the computer also implements the present invention. In other words, the present invention includes a computer program itself for realizing the functional processing of the present invention.

  In that case, as long as it has the function of a program, it may be in the form of object code, a program executed by an interpreter, script data supplied to the OS, or the like.

  Examples of the computer-readable storage medium for supplying the computer program include the following. For example, floppy (registered trademark) disk, hard disk, optical disk, magneto-optical disk, MO, CD-ROM, CD-R, CD-RW, magnetic tape, nonvolatile memory card, ROM, DVD (DVD-ROM, DVD- R).

  As another program supply method, a client computer browser is used to connect to a homepage on the Internet, and the computer program of the present invention is downloaded from the homepage to a recording medium such as a hard disk. In this case, the downloaded program may be a compressed file including an automatic installation function. It can also be realized by dividing the program code constituting the program of the present invention into a plurality of files and downloading each file from a different homepage. That is, a WWW server that allows a plurality of users to download a program file for realizing the functional processing of the present invention on a computer is also included in the present invention.

It is a block diagram which shows the structural example of the imaging device in embodiment. FIG. 5 is a diagram illustrating a skin color region used for face detection in a typical chromaticity diagram in a CIELab color space. It is a figure explaining the face recognition method in an embodiment. It is a flowchart explaining the face detection method in embodiment. It is a figure which shows the example of the high pass filter used for the face detection process in embodiment. 6 is a flowchart illustrating subject tracking by image correlation in the embodiment. It is a flowchart which shows the process which calculates the image correlation value and motion vector amount in embodiment. It is a figure explaining calculation of the intruder detection evaluation value in an embodiment. It is a figure which shows the example of the intrusion detection evaluation value according to ISO sensitivity in embodiment. It is a figure explaining calculation of the intruder detection evaluation value in an embodiment. It is a figure which shows the example of a setting of the detection threshold value in embodiment.

Claims (8)

Based on the degree of correlation in the partial areas of the plurality of captured image frames, the partial area corresponding to the tracking target partial area of the previous image frame is detected in the current image frame, so that the part between the image frames is detected. A tracking means for tracking the area;
Storing means for storing the correlation degrees related to the corresponding partial areas of a plurality of past image frames by storing the correlation degrees acquired for the corresponding partial areas in the current image frame in the tracking means; ,
An image pickup apparatus comprising: a determination unit configured to determine whether or not the tracking unit has successfully tracked the partial region based on a plurality of correlation degree changes stored by the storage unit.   The determination unit determines whether the tracking for the partial area has succeeded by comparing a difference value between a minimum value and a maximum value of the plurality of correlation values stored in the storage unit and a predetermined threshold value. The imaging apparatus according to claim 1, wherein:   The determination unit determines whether the tracking for the partial region has succeeded by comparing a minimum value and a maximum magnification value of the plurality of correlation values stored in the storage unit with a predetermined threshold value. The imaging apparatus according to claim 1, wherein:   The determination means includes a case in which the image frame that has obtained the maximum value of the correlation degree among the correlation degrees stored in the storage means is later in time than the image frame that has obtained the minimum value of the correlation degree; 4. A different threshold is used for the comparison in a case where an image frame having a maximum correlation value is temporally preceding an image frame having a minimum correlation value. The imaging device described in 1.   And a determination unit that determines the threshold value used in the determination unit based on at least one of a gain setting value for the sensor output, a noise amount of the individual sensor, a subject luminance value, a luminance value of a partial region, and a frame rate. The imaging device according to any one of claims 2 to 4, wherein   The imaging apparatus according to claim 1, further comprising a setting unit that detects a face from the captured image and sets a partial area to be tracked based on the detected face area. Based on the degree of correlation in the partial areas of the plurality of captured image frames, the partial area corresponding to the tracking target partial area of the previous image frame is detected in the current image frame, so that the part between the image frames is detected. A tracking process to track the area;
By storing in the memory the correlation degree acquired for the corresponding partial area in the current image frame in the tracking step, the correlation degree for the corresponding partial area of a plurality of past image frames is stored in the memory. A saving process to save;
And a determination step of determining whether or not the tracking of the partial area by the tracking step is successful based on a plurality of correlation degree changes stored in the memory.   A program for causing a computer to execute the subject tracking method according to claim 7.

Images