JP2010154374A - Image capturing apparatus and subject tracking method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image capturing apparatus and a subject tracking method capable of reducing a wrong subject from being tracked when detecting and tracking a specific subject or partial subjects included in a moving image. <P>SOLUTION: The image capturing apparatus has a subject tracking means for detecting a specific subject or partial subjects included in the moving image and tracking the subject on the basis of similarity extracted by matching processing with a reference image registered beforehand. The subject tracking means includes: a reference image registering means for registering the reference image; a matching processing means for performing the matching processing; and a tracking end decision means for ending tracking of the subject. The tracking end decision means is configured to end the tracking of the subject in accordance with the information other than the information used for the matching processing. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an imaging device and a subject tracking method, and more particularly, to an imaging device and a subject tracking method for detecting and tracking a specific subject or a part of a subject such as a person, an animal, or an object included in an image.

An image processing method for automatically detecting and tracking a specific subject from an image is very useful, and can be used, for example, for specifying a human face area in a moving image.
Such methods can be used in many areas such as teleconferencing, man-machine interfaces, security, monitor systems for tracking human faces, image compression, and the like.
In digital cameras and digital video cameras, a face is detected from a captured image, and the focus and exposure are optimized by using the detection result as a control target.
For example, Patent Document 1 discloses a photographing apparatus that detects the position of a face in an image, focuses on the face, and photographs the face with an optimal exposure.

Also, an apparatus and a method are known that enable stable control in time series by tracking the face detected in this way.
For example, Patent Document 2 proposes an apparatus and method using a template matching technique as an apparatus and method for automatically tracking a specific subject. Such template matching is a method of registering a partial image obtained by cutting out an image region to be tracked as a reference image, estimating a region having the highest correlation with the reference image in the image, and tracking a specific subject.
FIG. 8 shows a flowchart of an example of subject tracking using template matching.
FIG. 9 illustrates an example of subject tracking using template matching.
Here, an example is shown in which a face is tracked as a target subject.
In FIG. 9, 1101 is an input image at frame t = 0, 1102 is a subject detection result in the input image at frame t = 0, 1103 is a reference image registered in the input image at frame t = 0, and 1104 is frame t = 1. It is an input image at.
1105 is a matching result in the input image of the frame t = 1, 1106 is a reference image updated in the input image of the frame t = 1, and 1107 is an input image in the frame t = 2.
1108 is a matching result in the input image of frame t = 2, and 1109 is a reference image updated in the input image of frame t = 2.

Next, a method for tracking the specific subject described above will be described with reference to FIGS.
First, an input image 1101 at frame t = 0 is read by an imaging device such as a video camera (S1001).
Next, a subject area is extracted from the input image 1101 by subject detection processing, and a subject detection result such as 1102 is obtained (S1002).
An initial reference image 1103 is registered from the subject detection result (S1003). Subsequently, the input image 1104 at the frame t = 1 is read (S1004).
A matching process is performed between the input image 1104 and the reference image 1103 registered in the input image 1101 of the frame t = 0 (S1005).
If the matching process is not completed in the predetermined matching area (NO in S1006), the matching process is subsequently performed (S1005).
If completed (YES in S1006), the matching result 1105 is obtained with the region having the highest degree of correlation as the subject region at frame t = 1 (S1007).
Then, the reference image 1106 is updated based on the estimated subject area (S1008).

Subsequently, the input image 1107 in the frame t = 2 is read (S1004).
A matching process is performed between the input image 1107 and the reference image 1106 updated in the input image 1104 of the frame t = 1 (S1005).
If the matching process is not completed in the predetermined matching area (NO in S1006), the matching process is subsequently performed (S1005).
If completed (YES in S1006), the matching result 1108 is obtained with the region having the highest degree of correlation as the subject region at frame t = 2 (S1007).
Then, the reference image 1109 is updated based on the estimated subject area (S1008).
As described above, the target object is tracked by correlating the continuously input image with the reference image obtained from the matching result in the previous frame.
JP 2005-318554 A JP 2001-60269 A

However, the method of tracking a specific subject according to the above-described conventional example has the following problems because it is based on the similarity of the pattern between the current frame image and the reference image.
That is, in the tracking method of the subject according to the conventional example, when the tracking target is hidden behind an obstacle, the size of the tracking target changes due to the scaling operation,
Alternatively, when the screen itself is unstable due to panning or camera shake, an area different from the actual tracking target is extracted as a subject area from similarity.
That is, wrong subject tracking occurs in subject tracking.

  In view of the above-described problems, the present invention can reduce the occurrence of tracking of an erroneous subject when detecting and tracking a specific subject or a part of a subject included in a moving image. An object is to provide an apparatus and a subject tracking method.

The present invention provides an imaging apparatus and a subject tracking method configured as follows.
The imaging apparatus of the present invention detects a specific subject or a part of subjects included in a moving image, and tracks the subject based on similarity extracted by a matching process with a reference image registered in advance. An imaging apparatus having subject tracking means,
The subject tracking unit includes a reference image registration unit that registers the reference image, a matching processing unit that performs the matching process, and a tracking end determination unit that ends the tracking of the subject.
The tracking end determination unit is configured to end tracking of the subject by information other than information used in the matching process.
In addition, the subject tracking method of the present invention detects a specific subject or a part of subjects included in a moving image, and subjects the subject to be tracked based on similarity extracted by matching processing with a reference image. A tracking method,
As the reference image, a reference image registration step of previously registering a subject area in any frame image in the moving image;
A matching step for matching a frame image generated at a timing different from the reference image with the reference image and extracting a region similar to the reference image;
A tracking end determination step of ending tracking of the subject by information other than the information used in the matching process.

  According to the present invention, it is possible to reduce occurrence of tracking of an erroneous subject when detecting and tracking a specific subject or a part of a subject included in a moving image.

  The best mode for carrying out the present invention will be described by the following examples.

Hereinafter, a specific subject or a part of subjects included in the moving image in the embodiment of the present invention is detected, and the subject is determined based on similarity extracted by matching processing with a reference image registered in advance. The imaging device to be tracked will be described in detail with reference to the drawings.
FIG. 1 is a block diagram illustrating an imaging apparatus such as a video camera according to an embodiment of the present invention.
In FIG. 1, 101 is a zoom lens, 102 is a focus lens, 103 is an image sensor, 104 is an analog signal processing unit, 105 is an A / D conversion unit, 106 is a camera signal processing unit, 107 is a display unit, and 108 is a recording unit. A medium, 109 is a camera control unit, and 110 is a subject detection unit.
Reference numeral 111 denotes a subject tracking unit, 112 a reference image registration unit, 113 a matching processing unit, 114 a feature pixel determination unit, 115 an end determination unit, and 116 a vibration gyro.

In the image pickup apparatus of the present embodiment, the size of the subject image picked up by the zoom lens 101 is changed, and the light beam representing the subject image is condensed by the focus lens 102 to pick up an image like a CCD image sensor or a CMOS image sensor. Incident on the element 103.
The image sensor 103 outputs an electrical signal corresponding to the intensity of the incident light beam in units of pixels. This electrical signal is a video signal.
The analog signal processing unit 104 performs analog signal processing such as correlated double sampling (CDS) on the video signal output from the image sensor 103.
The video signal output from the analog signal processing unit 104 is converted into a digital data format by the A / D conversion unit 105 and input to the camera signal processing unit 106.
The camera signal processing unit 106 performs camera signal processing such as gamma correction, white balance processing, and AF / AE evaluation value generation.
Here, the AF evaluation value is generally obtained by extracting a high-frequency component of the luminance of the camera signal and adding it for a specific area in the screen.
The AE evaluation value is generally obtained by adding the luminance of the camera signal for a specific area in the screen.
In addition to normal camera signal processing, the camera signal processing unit 106 for camera signals performs camera signal processing using information on a specific subject area in an image supplied from a subject detection unit 110 and a subject tracking unit 111 described later. It has a function.
The video signal output from the camera signal processing unit 106 is sent to the display unit 107. The display unit 107 is, for example, an LCD or an organic EL display, and displays a video signal.
By sequentially displaying images taken continuously in time series on the display unit 107, the display unit 107 can function as an electronic viewfinder (EVF).
The video signal is recorded on a recording medium 108, for example, a removable memory card.
The recording destination may be a built-in memory of the camera or a connected external device capable of communication.

The video signal output from the camera signal processing unit 106 is also supplied to the subject detection unit 110.
The subject detection unit 110 is for detecting a specific target subject in the image and specifying the number of subjects and the subject region.
A typical subject is a human face or the like. A known face detection method is used as the detection method.
Known techniques for face detection include a method of using knowledge about the face (skin color information, parts such as eyes, nose, mouth, etc.) and a method of configuring a classifier for face detection by a learning algorithm represented by a neural network. is there.
In order to improve the recognition rate, face recognition is generally performed by combining them.
Specifically, there is a method for detecting a face using wavelet transform and an image feature amount described in JP 2002-251380 A.
The subject tracking unit 111 tracks a specific subject from video signals with different times output from the camera signal processing unit 106 based on the similarity of the video signal patterns.
The subject tracking unit 111 includes a reference image registration unit 112, a matching processing unit 113, a feature pixel determination unit 114, and an end determination unit 115.

The reference image registration unit 112 registers a partial area of the video signal output from the camera signal processing unit 106 as a reference image based on the result of the subject detection unit 110 or the subject tracking unit 111.
The reference image registration unit 112 registers the subject area based on the result of the subject detection unit 110 as a reference image because the result of the subject tracking unit 111 does not exist during the initial operation of the subject tracking unit 111.
After the initial operation of the subject tracking unit 111, the result of the subject tracking unit 111 can be registered in the reference image.
In the result of the subject detection unit 110 and the result of the subject tracking unit 111, the accuracy of the subject tracking unit 111 can be improved by registering the reference image based on the more reliable result.
The matching processing unit 113 performs matching processing between the frame image of the video signal in the current frame and the reference image registered by the reference image registration unit 112.
By the matching process, a region having the highest degree of correlation with the reference image in the image of the video signal in the current frame is extracted as a target subject region.
Here, it is assumed that the matching processing unit 113 operates only when a reference image having a different time from the video signal of the current frame is registered.
In addition, the matching processing unit 113 divides the image into a number of regions, and extracts a region having the smallest difference from the luminance value in the region included in the reference image as a region having the highest correlation.

In the region extracted by the matching processing unit 113, the feature pixel determination unit 114 determines that a pixel is a feature pixel when the color information of each pixel is included in a predetermined color model indicating the target feature of the subject.
In this embodiment, since the target subject is a face, the color model is a skin color model. The predetermined color model may be fixed or may change dynamically according to the extracted subject.
The next end determination unit 115 is characterized by the present invention. In addition to the tracking result of the subject tracking unit, the next end determination unit 115 determines the end of tracking based on various information (AF evaluation value, variable magnification, blur information) in the imaging apparatus. is there.
Based on the AF / AE evaluation value signal output from the camera signal processing unit 106 and the shake signal output from the vibration gyroscope 116 (to be described later), the camera control unit 109 detects the focus lens 102, an exposure control circuit (not shown), Controls the shake correction circuit.
The camera control unit 109 uses information on the extraction result of the target subject area supplied from the subject detection unit 110 and the subject tracking unit 111 for the focus lens control and the exposure control.
Therefore, the imaging apparatus according to the present exemplary embodiment has a function of performing imaging processing in consideration of information on a specific subject area in the captured image.
The vibration gyroscope 116 is an angular velocity sensor, and the camera control unit detects an image blur amount from the sensor output.
Although an angular velocity sensor such as a vibration gyroscope is used as a shake detection method in the present embodiment, motion vector detection by image detection without using a sensor may be used.

FIG. 2 is a flowchart illustrating the subject tracking method according to the present embodiment.
FIG. 3 shows an explanatory diagram of the subject tracking method according to the present embodiment.
In the following description, an example of tracking a person's face as a subject will be described.
In FIG. 2, S202 is processing performed by the matching processing unit 113, S203 is processing by the characteristic pixel determination unit 114, and S204 is processing performed by the reference image registration unit 112.
In FIG. 3, reference numeral 301 denotes a reference image registered by the reference image registration unit 112, and 302 denotes an input image of the subject tracking unit 111.
Reference numeral 303 denotes a subject extraction result by the matching processing unit 113, 304 denotes a feature pixel determination result by the feature pixel determination unit 114, and 306 denotes a reference image registered by the reference image registration unit 112.
First, a captured image such as 302 is read as an input image by an imaging device such as a video camera (S201).
Next, a matching process is performed between the input image and a reference image such as 301 registered in advance (S202).
Such a matching processor, for example, matches a frame image generated at a timing different from that of the reference image with the reference image, and extracts a region similar to the reference image.
In the matching process, a difference sum between the luminance of each pixel of the partial area having the same size as the reference image in the input image and the luminance value of each pixel of the reference image is calculated.
At that time, as shown in the flow chart of an example of subject tracking using the template matching of the conventional example,
It is assumed that the position of the partial area having the same size as the reference image in the input image is changed and the position of the partial area where the calculated difference sum is the minimum is the area having the highest degree of correlation (from S1005 in FIG. 8). (See S1007).
Since various methods can be considered as a method of matching two images, the processing example of this embodiment is only an example, and the present invention is not limited to this matching processing method.

Here, since the matching process is based on the similarity between the reference image and the input image, a correct subject area is not always a matching process result.
As a result of the matching process shown in 303, an area that is partially deviated from the correct subject area is extracted as a result.
This is because the reference image includes an area different from the subject area as indicated by 301, and the appearance of the subject changes between the reference image indicated by 301 and the input image indicated by 302.
Since the correlation between the region different from the subject region in the reference image 301 and the partial region of the input image shown in 302 is high, a matching processing result like 303 is obtained.
Next, in the region obtained by the matching process, a feature pixel indicating a target subject characteristic is determined (S203).

Next, details of the feature pixel determination process will be described.
FIG. 4 is a flowchart for explaining the feature pixel determination process.
First, the color information which the pixel of the predetermined position in the area | region obtained by the matching process has is acquired (S401).
Next, it is determined whether the acquired color information is information included in a color model that satisfies a predetermined condition (S402).
Next, when the color information is information included in a predetermined color model (YES in S402), the pixel is determined as a feature pixel (S403).
If the image color information is not information included in the predetermined color model (NO in S402), the pixel is not determined as a feature pixel.
Next, it is determined whether processing has been performed for all pixels in the region obtained by the matching processing (S404).
If the processing is not performed for all the pixels (NO in S404), the color information of the pixels that are not processed is acquired (S401). If processing has been performed for all pixels (YES in S404), the processing ends.

By the feature pixel determination, a feature pixel determination result such as 304 is obtained. In 304, feature pixels are painted white and pixels other than the feature pixels are painted black.
Here, the predetermined color model is a color model indicating the characteristics of a target subject, and when the target subject is a person's face, it is a skin color model.
FIG. 5 shows an example of a color model.
The color information acquired from the pixel is the color difference CbCr of the YCbCr data, the vertical axis in FIG. 5 is Cb, and the horizontal axis is Cr.
The shaded portion in FIG. 5 is a skin color model, and it is determined whether or not the acquired CbCr component is included in the shaded portion.
It is assumed that the color model as shown in FIG. 5 is set in advance.
It should be noted that since various types of color information and color models can be considered, the example of this embodiment is only an example, and the present invention is not limited to this format.
For example, the format of the acquired color information may be RGB data or hue H data converted into HSV color system data.
In addition, since various methods are conceivable regarding the feature pixel determination method based on the color model and the color information of the pixel, the example of this embodiment is only an example, and the present invention is not limited to this method. .
In addition, the present invention can be applied to whether the region for performing the feature pixel determination is the same as the region extracted based on the matching processing result or a predetermined region centered on the center of gravity of the extracted region. It is.

Next, an area extracted based on the matching process is registered as a reference image as indicated by 306 (S204).
For example, using the extraction result of the matching means, an area where the subject is estimated to be present is selected, and the selected area is registered as a new reference image.
Such a reference image 306 is used in the matching process of the next frame.
By sequentially updating the reference image, the subject tracking process can be performed even when the appearance of the subject changes in time series, such as when the orientation of the subject changes.
When both the subject detection unit 110 and the subject tracking unit 111 operate, the reference image may be registered based on a result with higher reliability.
On the other hand, when the change in appearance of the subject in time series is not taken into consideration, the present invention can be applied even if the reference image registered in the initial stage is maintained without updating the reference image.
Although the shape of the reference image 306 is shown as a rectangle, various shapes can be considered as the shape of the reference image. Therefore, the example of this embodiment is only an example, and the present invention is limited to this shape. is not.
For example, the present invention can be applied regardless of whether the shape of the reference image is a circle or a polygon.

Next, the tracking end determination is performed based on various information (S205).
FIG. 6 is a flowchart for explaining the tracking end determination process based on various information.
FIG. 7 shows an example of a scene adapted to the end of tracking (stopping the tracking operation) according to the embodiment of the present invention.
First, when the number of feature pixels obtained by the feature pixel determination unit 114 is smaller (below) the predetermined value th1 (YES in S601), the subject tracking is ended (S609).
As shown in FIG. 7A, when the tracking target of the subject tracking unit 111 is hidden by an obstacle, the number of feature pixels in the extracted region based on the matching processing result is reduced, and the subject tracking unit 111 is terminated. Is done.
Next, in accordance with the size of the subject being tracked, a threshold th2 of the tracking movement amount from the previous time to this time used later is set (S602).
As shown in FIG. 7B, it is determined whether the movement amount of the tracked subject from the previous time to the current time is larger than the predetermined value th2, and if it is larger than the predetermined value th2 (YES in S603), the subject tracking is interrupted. The reference image is reset (S609).
At this time, the predetermined amount th2 is large when the subject is large due to the size of the subject detected by the subject detection unit 110, and is small when the subject is small.
This is because the movement of the subject is considered to be less than or equal to the amount determined by the size of the subject, and there is a high possibility of mistracking when a small subject has a large movement amount.
Next, a tracking movement amount threshold th3 to be used later is set according to the size of the subject being tracked (S604).
As shown in FIG. 7C, it is determined whether the movement amount in the screen of the subject that has been successfully tracked continuously from the start of tracking of the tracked subject to this time is larger than the predetermined value th3, and is larger than the predetermined value th3 (in S605). YES) ends the subject tracking and resets the reference image (S609).
At this time, the predetermined value th3 is changed according to the size of the subject detected by the subject detection unit 110, and is large when the subject is large and small when the subject is small.
This is because the movement of the subject is considered to be less than or equal to the amount determined by the size of the subject, and if the small subject has a large amount of movement, there is a higher possibility of performing mistracking.
These are examples of a case where the subject tracking operation is stopped when a distance from a similar region extracted by matching exceeds a predetermined amount.

Next, as shown in FIG. 7D, it is determined whether the AF evaluation value extracted from the area of the tracked subject is smaller than the predetermined value th4. If it is smaller than the predetermined amount th4 (YES in S606), the subject tracking is finished. The reference image is reset (S609).
This is because the subject is being tracked, and a certain AF evaluation value should be obtained. If the level is small, there is a high possibility that erroneous tracking is being performed.
As these means, a focus evaluation value generating means for extracting a high frequency component of the image signal of the area selected by the tracking end determination means and generating a focus evaluation value in the area is configured.
The tracking end determination unit stops the tracking operation of the subject when the focus evaluation value becomes smaller than a predetermined amount.
Next, as shown in FIG. 7E, it is determined whether or not the amount of magnification change by the magnification lens has changed by the predetermined amount th5. If the amount of change has changed by the predetermined amount th5 (YES in S607), the subject tracking is finished, and the reference The image is reset (S609).
This is because if there is a difference in magnification between the two images to be compared, there is no point in matching and there is a high possibility of mistracking.
Next, as shown in FIG. 7F, when the camera shake detected by the camera shake detection means exceeds the predetermined amount th6 (YES in S608), the subject tracking is ended and the reference image is reset ( S609).
This is because if two images to be compared are different due to camera shake, there is no point in taking matching, and there is a high possibility of mistracking.
Accordingly, it is possible to reduce the possibility of erroneously tracking a region different from the target subject region.

According to the configuration of the above embodiment, in the subject tracking process, the tracking end determination process can be performed based on the tracking status and various information of the imaging device in addition to the luminance matching information used for tracking, and the tracking can be ended.
As a result, the accuracy of subject tracking processing can be improved, and the possibility of erroneously tracking a region different from the target subject region can be reduced.
Further, although the above embodiment has been described by taking a video camera as an example, it goes without saying that it can be applied to a digital still camera having a moving image function.
Further, even in an application that runs on a general-purpose computer such as a personal computer, it is possible to track the subject by performing the same processing as in the above-described embodiment.

1 is a block diagram of an image pickup apparatus according to an embodiment of the present invention. 6 is a flowchart for explaining a subject tracking method according to the embodiment of the present invention. Explanatory drawing about the subject tracking method which concerns on the Example of this invention. The flowchart explaining the determination process of the feature pixel based on the Example of this invention. An example of the color model which concerns on the Example of this invention. The flowchart explaining the tracking end determination process which concerns on the Example of this invention. An example of the scene applied to the end of tracking according to an embodiment of the present invention. 10 is a flowchart for explaining subject tracking by template matching in a conventional example. Explanatory drawing about the subject tracking by template matching in a prior art example.

Explanation of symbols

101: Variable magnification lens (zoom lens)
102: Focus lens 103: Image sensor 104: Analog signal processing unit 105: A / D conversion unit 106: Camera signal processing unit 107: Display unit 108: Recording medium 109: Camera control unit 110: Subject detection unit 111: Subject tracking unit 112: reference image registration unit 113: matching processing unit 114: feature pixel determination unit 115: end determination unit 116: vibration gyro 301: reference image registered by the reference image registration unit 112 302: input image 303: matching processing unit 113 Subject extraction result 304: Feature pixel determination result 306: Feature pixel determination result 306: Reference image 501 registered by reference image registration unit 112: Example of skin color model in feature pixel determination unit 114

Claims (12)

An imaging apparatus having a subject tracking unit that detects a specific subject or a part of a subject included in a moving image and tracks the subject based on similarity extracted by matching processing with a reference image registered in advance Because
The subject tracking unit includes a reference image registration unit that registers the reference image, a matching processing unit that performs the matching process, and a tracking end determination unit that ends the tracking of the subject.
The imaging apparatus according to claim 1, wherein the tracking end determination unit is configured to end tracking of the subject by information other than information used in the matching process. While the reference image registration unit has a configuration for registering a subject region in any frame image in the moving image as a reference image,
2. The system according to claim 1, further comprising: selecting an area where the subject is estimated to exist using the extraction result of the matching means, and registering the selected area as a new reference image. The imaging device described in 1.   The matching processing means is configured to perform matching with the reference image for a frame image generated at a timing different from the reference image, and extract a region similar to the reference image. The imaging device according to claim 1 or 2. Characteristic pixel determination means for determining color information in the region extracted by the matching means;
The tracking end determination unit is configured such that when the area including color information satisfying a predetermined condition is less than a predetermined amount by the feature pixel determination unit,
The imaging apparatus according to claim 1, wherein the imaging apparatus is configured to be able to stop the tracking operation of the subject. The tracking end determination means, when the distance between the subject area registered as the reference image and the similar area extracted by the matching means exceeds a predetermined amount,
The imaging apparatus according to claim 1, wherein the imaging apparatus is configured to be able to stop the tracking operation of the subject. When the tracking end determination unit continuously tracks the subject, the distance between the subject region initially registered as the reference image and the similar region extracted by the matching unit is a predetermined amount. When exceeding
The imaging apparatus according to claim 1, wherein the imaging apparatus is configured to be able to stop the tracking operation of the subject.   The imaging apparatus according to claim 5, wherein the predetermined amount is configured to be able to be changed according to a size of the extracted subject. A focus evaluation value generating unit that extracts a high-frequency component of the image signal of the region selected by the tracking end determination unit and generates a focus evaluation value in the region;
4. The tracking end determination unit is configured to be able to stop the tracking operation of the subject when the focus evaluation value becomes smaller than a predetermined amount. The imaging apparatus according to item 1. A scaling unit for scaling the image of the subject;
4. The tracking end determination unit is configured to be able to stop the tracking operation of the subject when a scaling amount by the scaling unit reaches a predetermined amount. The imaging device according to any one of the above. A blur detection means for extracting a blur amount of the image of the subject;
The tracking end determination unit is configured to be able to stop the tracking operation of the subject when the blur amount reaches a predetermined amount. The imaging device described in 1.   11. The imaging apparatus according to claim 1, wherein the matching unit extracts a region having a minimum difference from a luminance value of the reference image as a region similar to the reference image. . A subject tracking method for detecting a specific subject or a part of a subject included in a moving image and tracking the subject based on similarity extracted by matching processing with a reference image,
As the reference image, a reference image registration step of previously registering a subject area in any frame image in the moving image;
A matching step for matching a frame image generated at a timing different from the reference image with the reference image and extracting a region similar to the reference image;
A tracking end determination step for ending tracking of the subject by information other than the information used for the matching process;
An object tracking method characterized by comprising:

Images