JP2008299834A - Photographic subject tracking program and photographic subject tracking device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To track a photographic subject in frames by carrying out template matching. <P>SOLUTION: A template matching part 104a calculates similarity levels of an image within a search frame in each search frame position with a template image while moving the search frame in a search subject region set in each input image inputted in time series. Then, the position in the input image in the search frame with highest calculated similarity level is specified as the photographic subject position. The photographic subject tracking part 104b tracks the specified photographic subject position in a plurality of input images. Furthermore, a template matching part 104a judges reliability in the similarity level in the photographic subject position. Next, content of at least one process in calculation of the similarities, specifying of the photographic subject position or tracking of the photographic subject position is changed. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a subject tracking program and a subject tracking device for tracking a subject between frames.

  The following pattern matching methods are known. In this pattern matching method, an image is divided into a plurality of regions, a template matching process is performed for each region, and a region having the highest similarity is extracted as a matching region (for example, Patent Document 1).

JP-A-5-81433

  However, if the subject position in the image is specified by template matching using the conventional method and the subject position is to be tracked between frames, if there is an image similar to the template in the background, pseudo-matching will occur and it will be accurate. The subject may not be tracked.

The present invention calculates the similarity between the image in the search frame and the template image at each search frame position while moving the search frame within the search target area set in each input image input in time series, The position in the input image of the search frame with the highest calculated similarity is specified as the position where the subject exists (subject position), and the specified subject position is tracked between a plurality of input images, and the similarity at the subject position is determined. When the reliability is determined and it is determined that the reliability of the similarity at the subject position is low, the content of at least one of the similarity calculation process, the subject position specifying process, and the subject tracking process is determined. It is characterized by changing.
In the present invention, when the absolute value of the difference between the average value of the similarity between the image in the search frame at each search frame position and the template image and the highest similarity is larger than a predetermined threshold, the subject position It is determined that the reliability of the similarity at is high, and if it is equal to or less than a predetermined threshold, it is determined that the reliability of the similarity at the subject position is low.
When it is determined that the reliability of the similarity at the subject position is low, the subject position specified by the input image may be excluded from the subject tracking target.
If it is determined that the reliability of the similarity at the subject position is low, the size of the search target area for moving the search frame at the time of template matching of the next input image is changed according to the similarity at the subject position. It may be.
If it is determined that the reliability of the similarity at the subject position is low, the size of the search target area for moving the search frame is changed according to the similarity at the subject position and set in the current input image. In the search target area after changing the size, template matching between the image in the search frame and the template image is performed while moving the search frame, and the image in the search frame and the template image at each search frame position are matched. The degree of similarity may be recalculated, and the position in the input image of the search frame with the highest degree of similarity recalculated may be re-specified as the subject position.
When the absolute value of the difference between the maximum value of the similarity in the search target area in the past input image and the maximum value of the similarity in the search target area in the current input image is greater than a predetermined threshold The reliability of the similarity at the subject position may be determined to be high, and if the similarity is equal to or less than a predetermined threshold value, the reliability of the similarity at the subject position may be determined to be low.
If it is determined that the reliability of the similarity at the subject position is low, the size of the search target area to which the search frame is moved at the time of template matching of the next input image may be enlarged.
If it is determined that the reliability of the similarity at the subject position is low, the size of the search target area for moving the search frame is enlarged, and within the search target area after enlargement set in the current input image, Perform template matching between the image in the search frame and the template image while moving the search frame, recalculate the similarity between the image in the search frame and the template image at each search frame position, and the recalculated similarity The position of the highest search frame in the input image may be specified again as the subject position.

  According to the present invention, the subject can be accurately tracked.

-First embodiment-
FIG. 1 is a block diagram showing a configuration of an embodiment in which a camera is used as the subject tracking device in the first embodiment. The camera 100 includes an operation member 101, a lens 102, an image sensor 103, a control device 104, a memory card slot 105, and a monitor 106. The operation member 101 includes various input members operated by the user, such as a power button, a release switch, a zoom button, a cross key, a determination button, a playback button, and a delete button.

  The lens 102 is composed of a plurality of optical lens groups, but is represented by a single lens in FIG. The image sensor 103 is, for example, a CCD or a CMOS, and captures a subject image formed by the lens 102 to acquire an image. The acquired image data (image data) is output to the control device 104. The control device 104 compresses the image data acquired by the image sensor 103 into a predetermined image format, for example, JPEG format, and generates an image file in a predetermined format such as Exif (Exchangeable Image File Format Digital Still Camera). Output to the memory card slot 105.

  The memory card slot 105 is a slot for inserting a memory card as a storage medium, and the image file output from the control device 104 is written and recorded on the memory card. Further, based on an instruction from the control device 104, an image file stored in the memory card is read.

  The monitor 106 is a liquid crystal monitor (rear monitor) mounted on the back surface of the camera 100, and the monitor 106 displays an image stored in a memory card, a setting menu for setting the camera 100, and the like. . Further, the control device 104 acquires images in time series from the image sensor 103 and outputs them to the monitor 106. As a result, the image of each frame is sequentially displayed on the monitor 106 at predetermined time intervals. That is, a through image is displayed on the monitor 106.

  The control device 104 includes a CPU, a memory, and other peripheral circuits, and functionally includes a template matching unit 104a and a subject tracking unit 104b. The memory constituting the control device 104 includes a ROM used as a buffer memory, such as a flash memory, and a RAM, such as an SDRAM. In the present embodiment, the template matching unit 104a performs a template matching process on each frame of the through image input from the image sensor 103, and identifies an area in which the subject is shown in the frame. Then, the subject tracking unit 104b tracks the specified area between frames. Specifically, the process is as follows.

  The template matching unit 104 a performs a matching operation between a template image serving as a reference for template matching processing and image data (target image) of each frame input in time series from the image sensor 103. Specifically, as shown in FIG. 2, the subject position in the target image I is specified using the template image A. The template image A is acquired based on an instruction from the user. For example, when the first frame of the through image is displayed on the monitor 106, the user operates the operation member 101 to specify a range including a subject to be tracked between frames in the first frame. The template matching unit 104a extracts an image within the range specified by the user as the template image A and stores it in the ROM.

  When the input of the through image from the image sensor 103 is started, the template matching unit 104a sets the image of each frame (frame) as the target image I and has the same size as the template image at a predetermined position on the target image I. The search frame B is set. The template matching unit 104a performs a matching operation between the image (search partial image) in the search frame B at each position and the template image A while moving the set search frame B in the target image I. As a result of the matching calculation, the coordinate value in the target image I of the matching region having the highest similarity between the template image A and the image in the search frame B is specified as the subject position.

  Note that the template matching unit 104a performs template matching not on the entire target image I but on a predetermined range (in the search target area) including the position where the template image A is extracted from the image. As a result, the template matching range can be limited to a range where there is a high possibility that the subject in the target image I exists, and the processing speed can be increased. Since the template matching method is well known, detailed description thereof is omitted. For example, the similarity may be calculated by the residual sum shown in the following equation (1) or the normalized correlation shown in the following equation (2).

When the similarity is calculated by the residual sum represented by the following equation (1), the smaller the calculated r is, the higher the similarity between the template image A and the search partial image is. On the other hand, when the similarity is calculated by the normalized correlation represented by the following equation (2), the larger the calculated r is, the higher the similarity between the template image A and the search partial image is.

  FIG. 3 is a diagram illustrating a change in similarity calculated when template matching is performed while moving the search frame B within a predetermined range of the target image I. In FIG. 3, the vertical axis represents similarity and the horizontal axis represents the number of calculations. That is, when template matching is performed a plurality of times while moving the search frame B, the result of plotting the similarity for each time is shown. Note that the vertical axis represents the degree of similarity normalized to a numerical value from 0 to 1, and in this embodiment, a value indicating the degree of similarity represented by 0 to 1 is referred to as a similarity value. The similarity value indicates that the similarity is lower as it is closer to 0, and the similarity is higher as it is closer to 1.

  For example, when the similarity changes as shown in FIG. 3A as a result of template matching, the position of the search frame B on the target image I when the point 3a having the highest similarity value is calculated. Is identified as the subject position. On the other hand, when the similarity changes as shown in FIG. 3B, the position of the search frame B on the target image I when the point 3c having the highest similarity value is calculated is specified as the subject position. Is done.

  Comparing the results shown in FIG. 3A with the results shown in FIG. 3B, in FIG. 3A, the similarity value (maximum similarity value) at the point 3a is significantly large, and the similarity value While the difference from the average value (similarity candidate average) 3b is also as large as 0.94, in FIG. 3B, the highest similarity value at the point 3c is not significantly large, and the similarity candidate average 3d The difference between is as small as 0.12. As described above, when the result shown in FIG. 3A is obtained, it can be estimated that the subject position specifying result is highly reliable, but the result shown in FIG. 3B is obtained. In this case, it is estimated that the result of specifying the subject position is low in reliability.

In the present embodiment, the template matching unit 104a determines that the subject position specifying result is high when the following equation (3) is satisfied, and determines that the subject position specifying result is low when the following result is not satisfied.
｜ Maximum similarity value−Similarity candidate average |> Threshold (3)

  When the template matching unit 104a determines that the reliability of the subject position specifying result is high, the template matching unit 104a specifies a range corresponding to the search frame B corresponding to the subject position as a range where the subject exists in the target image I. . Then, the subject tracking unit 104b displays a target frame on the frame for clearly indicating the range where the subject exists, thereby indicating the position where the subject exists in the frame to the user.

  On the other hand, when it is determined that the reliability of the subject position specifying result is low, the subject tracking unit 104b does not display the target frame corresponding to the subject position, and the template matching unit 104a does not display the next frame. Move on to template matching. That is, if the subject tracking unit 104b determines that the reliability of the result of specifying the subject position is low, the subject tracking unit 104b excludes the subject position from the subject tracking target.

  By performing the above-described processing for each frame of the input through image, only subject positions having high similarity at the time of matching among the subject positions specified by template matching by the template matching unit 104a are set as tracking targets. A target frame can be displayed and subject tracking can be performed between frames.

  FIG. 4 is a flowchart showing the processing of the camera 100 in the first embodiment. The processing shown in FIG. 4 is executed by the control device 104 as a program that is activated when input of a through image from the image sensor 103 is started.

  In step S1, the template matching unit 104a reads the first frame of the through image, and proceeds to step S2. In step S2, the template matching unit 104a extracts the template image A from the first read frame based on an instruction from the user as described above. Then, it progresses to step S3.

  In step S3, the template matching unit 104a reads the next frame of the through image, and proceeds to step S4. In step S4, the template matching unit 104a matches the image in the search frame B and the template image A at each position while moving the search frame B within the predetermined range in the target image I as described above. Then, the similarity calculation is performed to calculate the subject position. Then, it progresses to step S5.

  In step S5, the template matching unit 104a determines whether or not the subject position specifying result is highly reliable using the equation (3). If it is determined that the reliability is low, the process returns to step S3 to read the next frame. On the other hand, if it is determined that the reliability is high, the process proceeds to step S6, and the subject tracking unit 104b displays a target frame for clearly indicating the range where the subject exists on the frame. Thereafter, the process proceeds to step S7.

  In step S7, the template matching unit 104a sets various parameters necessary for performing template matching of the next frame. For example, in accordance with the similarity value calculated at the specified subject position, a search area parameter for specifying the position and size of the range (search range) in which the search frame B is moved in the next frame is set.

  Specifically, when the similarity value calculated at the specified subject position is high, there is a high possibility that the subject is surely captured. Therefore, in the next frame, a narrow range centered on the specified subject position. The search area parameter is set so as to search. On the other hand, when the reliability of the similarity calculated at the specified subject position is low, the subject may be removed, so in the next frame, a wide range centering on the specified subject position. The search area parameter is set so as to search. Thereafter, the process proceeds to step S8.

  In step S8, the template matching unit 104a determines whether or not the input of the through image has been completed. If a negative determination is made, the process returns to step S3, and the template matching unit 104a reads the next frame. On the other hand, if an affirmative determination is made, the process is terminated.

According to the first embodiment described above, the following operational effects can be obtained.
(1) While matching the search frame within the search target area set in each frame of the input image input in time series, template matching between the image in the search frame and the template image is performed, and each search frame position The similarity between the image in the search frame and the template image was calculated. Then, the position in the input image of the search frame having the highest calculated similarity is specified as the subject position, the reliability of the similarity at the subject position is determined, and the reliability of the similarity at the subject position is determined to be low. In this case, the subject position specified by the frame is excluded from the subject tracking target. As a result, the target frame can be displayed only at a subject position with high reliability of similarity, and subject tracking can be performed with high accuracy.

(2) The reliability of similarity is determined by equation (3). That is, if the absolute value of the difference between the average value of the similarity value between the image in the search frame and the template image at each search frame position and the highest similarity value is larger than a predetermined threshold value, It is determined that the reliability of the similarity is high. When the similarity is equal to or lower than a predetermined threshold, it is determined that the reliability of the similarity at the subject position is low. As a result, the reliability of similarity can be uniformly determined, and the processing can be simplified.

-Second embodiment-
In the first embodiment described above, when the template matching unit 104a determines that the reliability of the result of specifying the subject position is low based on the determination using the expression (3), the template matching unit 104a includes the target frame at the subject position. Moved to next frame processing without displaying. That is, when it is determined that the reliability of the subject position specifying result is low, the subject position is excluded from the subject tracking target.

  On the other hand, in the second embodiment, the template matching unit 104a specifies the frame even when it is determined that the reliability of the subject position specifying result is low by the determination using the expression (3). A target frame is displayed at the subject position to be a subject tracking target. Then, the template matching unit 104a resets the search area parameters described above so that a wider range is targeted for searching in order to increase the reliability of the subject position specifying result in the next frame. In the second embodiment, since FIGS. 1 to 3 are the same as those in the first embodiment, description thereof will be omitted.

  Specifically, when the template matching unit 104a determines that the reliability of the subject position specifying result is low, the template matching unit 104a expands the search target area in which the search frame B is moved in the target image I. For example, as shown in FIG. 5, a search target area C is set in the target image I. At this time, the template matching unit 104a may enlarge the search target area at a predetermined ratio set in advance, or may change the enlargement ratio of the search target area according to the similarity value at the subject position. Good.

  FIG. 6 is a flowchart illustrating processing of the camera 100 according to the second embodiment. The process shown in FIG. 6 is executed by the control device 104 as a program that starts when input of a through image from the image sensor 103 is started. In FIG. 6, the same step numbers are assigned to the same processes as those in FIG. 4 described in the first embodiment, and differences will be mainly described.

  In step S4, as described above, the template matching unit 104a performs similarity calculation to identify the subject position. In the present embodiment, the template matching unit 104a uses the range corresponding to the search frame B set at the subject position as the range in which the subject exists in the target image I, regardless of the reliability of the identification result. As specified. Thereafter, the process proceeds to step S6, and the subject tracking unit 104b displays a target frame for clearly indicating the range in which the subject exists on the frame, and then proceeds to step S7.

  In step S7, the template matching unit 104a sets various parameters necessary for template matching of the next frame, for example, the search area parameters described above. Thereafter, the process proceeds to step S5, and the template matching unit 104a determines whether or not the subject position specifying result is highly reliable using Expression (3). If it is determined that the reliability is high, the process proceeds to step S8.

  On the other hand, if it is determined that the reliability is low, the process proceeds to step S11, and the search area parameter set in step S7 is reset based on the similarity value at the subject position. For example, as described above, if the reliability of the similarity of the specified subject position is low, the subject may have been removed, so in the next frame, a wide range centering on the specified subject position is used. Need to explore. For this reason, here, the search area parameter is set so as to search a wider range than the range set in step S7. Thereafter, the process proceeds to step S8.

According to the second embodiment described above, the following effects can be obtained in addition to the functions and effects of the first embodiment.
(1) The target frame is displayed at the specified subject position even when the reliability of the similarity of the specified subject position is low. Thus, regardless of whether the reliability is high or low, the target frame can be displayed on each frame of the through image, and the user can grasp the change in the subject position.

(2) When the reliability of the similarity of the specified subject position is low, the search area parameter is reset. As a result, if the reliability of the similarity of the specified subject position is low, the subject may be removed, so in the next frame, it is necessary to search a wide range centering on the specified subject position. In consideration of the fact, the search target area to be searched in the next frame can be set to an optimum size.

-Third embodiment-
In the above-described first and second embodiments, the template matching unit 104a determines that the reliability of the subject position specifying result in the current frame is low based on the determination result using Expression (3). I made it. In contrast, in the third embodiment, the difference between the average value of the maximum similarity values in the search target area in the past frame and the maximum similarity value in the search target area in the current frame. If the calculation result is greater than a predetermined threshold value, it is determined that the subject position identification result in the current frame is highly reliable, and if the calculation result is less than the predetermined threshold value, An example will be described in which it is determined that the reliability of the result of specifying the subject position in the frame is low. If the template matching unit 104a determines that the reliability of the subject position specifying result in the current frame is low, the template matching unit 104a expands the search target area in the current frame.

  Specifically, the template matching unit 104a sets a search frame B having the same size as the template image at a predetermined position on the target image I, as in the first and second embodiments. Then, the template matching unit 104a performs a matching operation between the image (search partial image) in the search frame B at each position and the template image A while moving the set search frame B in the target image I. As a result of the matching calculation, the coordinate value in the target image I of the matching area having the highest similarity between the template image A and the image in the search frame B is specified as the subject position, and the similarity value at that time is calculated. Then, a search frame B centered on the specified subject position is newly set, and a matching calculation in the next frame is performed using the newly set search frame B.

  For example, as shown in FIG. 7, a case where the maximum value of similarity values is calculated for the image of the fourth frame will be described. In this case, the template matching unit 104a calculates an average value 7a of maximum values of similarity values calculated in the past three frames, and the similarity between the average value 7a and the image of the current frame (fourth frame). The absolute value 7c of the difference between the maximum values 7b is calculated. As a result, when the calculated absolute value 7c of the difference is larger than a preset threshold value, it is determined that the reliability of the subject position specifying result in the current frame is high. On the other hand, when the absolute value 7c of the difference is equal to or less than a preset threshold value, it is determined that the reliability of the subject position specifying result in the current frame is low.

That is, in the third embodiment, the template matching unit 104a determines that the subject position specifying result is high when the following equation (4) is satisfied, and determines that the subject position specifying result is low when the following result is not satisfied.
| Average value of maximum values of similarity values calculated in past frames−Maximum value of similarity values of current frame |> Threshold (4)

  If the template matching unit 104a determines that the reliability of the subject position specifying result in the current frame is low by not satisfying the expression (4), as shown in FIG. In the frame 5), the template matching process is executed in an area obtained by enlarging the search target area. At this time, the subject position (the center of the search target area) in the next frame may be the same position as the previous frame (frame 3 in FIG. 8).

  For example, as shown in FIGS. 8A to 8D, as a result of performing template matching by setting the search target region C in the target image I from the first frame to the fourth frame (current frame), A case where it is determined that the reliability of the result of specifying the subject position in the frame is low will be described. In this case, the template matching unit 104a enlarges the search target area C and sets a new search target area D with respect to the current frame, and uses the search target area D as a target for the next frame (frame 5). ) To execute the template matching process.

  FIG. 9 is a flowchart illustrating processing of the camera 100 according to the third embodiment. The processing shown in FIG. 9 is executed by the control device 104 as a program that is activated when input of a through image from the image sensor 103 is started. In FIG. 9, the same step numbers are assigned to the same processes as those in FIG. 4 described in the first embodiment and FIG. 6 described in the second embodiment, and differences are mainly described. To do.

  In step S51, the template matching unit 104a determines whether or not the subject position specifying result is highly reliable using Expression (4). If it is determined that the reliability is high, the process proceeds to step S6, and the subject tracking unit 104b displays the target frame in a range corresponding to the specified subject position.

  On the other hand, if it is determined in step S51 that the reliability is low, the process proceeds to step S11, and the template matching unit 104a resets the search parameters so as to widen the search target area, and then proceeds to step S8.

  According to the third embodiment described above, the following effects can be obtained in addition to the operational effects of the first and second embodiments. That is, the absolute value of the difference between the average value of the maximum similarity value calculated in the past frame and the maximum value of the similarity value of the current frame is calculated, and this calculation result is compared with the threshold value to calculate the current value. The reliability of the result of specifying the subject position in the frame is determined to be high or low. This makes it possible to determine with high accuracy whether the reliability of the result of specifying the subject position in the current frame is high or low using the calculation result of the similarity value of the past frame.

-Modification-
Note that the subject tracking device of the above-described embodiment can be modified as follows.
(1) In the second embodiment described above, even when the reliability of the similarity of the specified subject position is low, the target frame is displayed at the specified subject position, and is used for template matching of the next frame. An example of resetting search parameters has been described. However, when the reliability of the similarity of the specified subject position is low, the template matching unit 104a resets the search parameter and calculates the similarity for the current frame using the reset search parameter. May be performed again to recalculate the subject position. Then, the subject tracking unit 104b may display the target frame at the subject position specified as a result.

  Specifically, the control device 104 may execute the process shown in FIG. In FIG. 10, the same processing as that in FIG. 4 described in the first embodiment, FIG. 6 described in the second embodiment, and FIG. 9 described in the third embodiment is the same. A step number is assigned. As shown in FIG. 10, in step S5, the template matching unit 104a determines whether or not the subject position specifying result is highly reliable using Expression (3). If it is determined that the reliability is high, the process proceeds to step S6, and the subject tracking unit 104b displays the target frame in a range corresponding to the specified subject position.

  On the other hand, if it is determined in step S5 that the reliability is low, the process proceeds to step S11, and the template matching unit 104a resets the search parameters so as to widen the search target area, and then proceeds to step S12. In step S12, the template matching unit 104a recalculates the subject position by performing similarity calculation again for the current frame using the reset search parameter. Thereafter, the process proceeds to step S6, and the subject tracking unit 104b displays the target frame in a range corresponding to the newly specified subject position.

  As a result, when the template matching for the current frame is performed and the reliability of the subject position identification result is low, the template matching is performed again after resetting the search area parameter. Can be improved.

  In the third embodiment described above, template matching may be performed again after expanding the search target area for the current frame (frame 4 in FIG. 8). For example, as shown in FIG. 11, the template matching unit 104a enlarges the search target area C and sets a new search target area D for the current frame (frame 4). That is, the search target area D is set as shown in the frame 4 ′. Then, the template matching process is executed again in the current frame (frame 4 ′) with this search target area D as a target.

(2) As in the modification (1) described above, in the case where the search target area is expanded for the current frame and template matching is performed again, the first template matching, that is, the range searched in the process in step S4, There is no need to process again. For this reason, here, it is only necessary to perform a search only for a range that is not processed in the first template matching, that is, an enlarged range.

  For example, when the search target area at the first template matching is the range 8a shown in FIG. 12A, the range to be re-searched is expanded to the range obtained by adding the range 8b to the range 8a. Re-searches only the range 8b. Specifically, as shown in FIG. 12B, the template matching unit 104a performs the template matching by moving the search frame B in a spiral shape from the outside of the range (8a + 8b) to be re-searched, and enters the range 8a. The process may be stopped immediately before entering.

(3) In the third embodiment described above, the template matching unit 104a uses the expression (4) to calculate the average value of the maximum similarity values in the search target area in the past frame and the search target area in the current frame. An example has been described in which the absolute value of the difference between the maximum similarity value and the threshold value is compared with the threshold value to determine the reliability of the subject position specifying result in the current frame. However, in calculating the average value of the maximum similarity values in the search target area in the past frame, the maximum reliability value of the past frame has a low reliability, that is, the similarity value is a predetermined value. If less than that is included, the average value may be calculated by removing the similarity value. Thereby, the accuracy of the determination result of the reliability of the result of specifying the subject position in the current frame can be improved.

(4) In the above-described first to third embodiments, the example in which the camera is used as the subject tracking device and the subject is tracked on the through image has been described. However, when the camera 100 has a moving image shooting function, template matching may be performed for each frame of a captured moving image, and subject tracking may be performed between the frames constituting the moving image. In addition, a program for executing the processing shown in FIG. 4, 6 or 7 is recorded in another device capable of reproducing moving images, such as a personal computer (personal computer), and the personal computer is executed by executing the program. The subject tracking may be performed above.

  Note that the present invention is not limited to the configurations in the above-described embodiments as long as the characteristic functions of the present invention are not impaired.

It is a block diagram which shows the structure of one Embodiment of a camera. It is a figure which shows the specific example of the target image I, the template image A, and the search frame B. FIG. It is a figure which shows the change of the similarity calculated when template matching is performed, moving the search frame B within the predetermined range of the target image. It is a flowchart figure which shows the process of the camera 100 in 1st Embodiment. It is a figure which shows the example of a setting of the search object area | region in 2nd Embodiment. It is a flowchart figure which shows the process of the camera 100 in 2nd Embodiment. It is a figure which shows the change of the similarity for every flame | frame in 3rd Embodiment. It is a figure which shows the example of a setting of the search object area | region in 3rd Embodiment. It is a flowchart figure which shows the process of the camera 100 in 3rd Embodiment. It is a flowchart figure which shows the process of the camera 100 in a modification. It is a figure which shows the example of a setting of the search object area | region in a modification. It is a figure which shows the example of a movement of the search frame within the search object area | region in a modification.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 Camera, 101 Operation member, 102 Lens, 103 Image pick-up element, 104 Control apparatus, 104a Template matching part, 104b Subject tracking part, 105 Memory card slot, 106 Monitor

Claims (9)

On the computer,
Similarity calculation procedure for calculating the similarity between the image in the search frame and the template image at each search frame position while moving the search frame in the search target area set in each input image input in time series When,
A subject position specifying procedure for specifying a position in the input image of the search frame having the highest similarity calculated in the similarity calculation procedure as a position where the subject exists (subject position);
A subject tracking procedure for tracking the subject position specified in the subject position specifying procedure between the plurality of input images;
A reliability determination procedure for determining the reliability of the similarity at the subject position;
When it is determined in the reliability determination procedure that the reliability of the similarity at the subject position is low, at least one of the similarity calculation procedure, the subject position specifying procedure, and the subject tracking procedure is performed A subject tracking program for executing a processing control procedure for changing processing contents. The subject tracking program according to claim 1,
The reliability determination procedure is performed when the absolute value of the difference between the average value of the similarity between the image in the search frame at each search frame position and the template image and the highest similarity is larger than a predetermined threshold. Determines that the reliability of the similarity at the subject position is high, and determines that the reliability of the similarity at the subject position is low when it is equal to or less than a predetermined threshold. program. The subject tracking program according to claim 2,
In the process control procedure, when it is determined in the reliability determination procedure that the reliability of the similarity at the subject position is low, the subject position specified by the input image is determined with respect to the subject tracking procedure. A subject tracking program characterized in that it is excluded from the subject tracking target. The subject tracking program according to claim 2,
In the process control procedure, when it is determined that the reliability of the similarity at the subject position is low in the reliability determination procedure, the similarity calculation procedure is performed when the template matching of the next input image is performed. A subject tracking program, wherein a size of the search target area to which a search frame is moved is changed according to the similarity at the subject position. The subject tracking program according to claim 2,
In the case where it is determined in the reliability determination procedure that the reliability of the similarity at the subject position is low, the processing control procedure is
With respect to the similarity calculation procedure, the size of the search target area for moving the search frame is changed according to the similarity at the subject position,
A template of an image in the search frame and a template image while moving the search frame in the search target area after changing the size set in the current input image for the similarity calculation procedure Performing matching to recalculate the similarity between the image in the search frame and the template image at each search frame position,
A subject tracking program for causing the position in the input image of the search frame having the highest similarity recalculated in the similarity calculation procedure to be re-specified as the subject position with respect to the subject position specifying procedure. The subject tracking program according to claim 1,
In the reliability determination procedure, the absolute value of the difference between the maximum value of the similarity in the search target area in the past input image and the maximum value of the similarity in the search target area in the current input image is predetermined. When the threshold value is larger than the threshold value, it is determined that the reliability of the similarity level at the subject position is high. Subject tracking program. The subject tracking program according to claim 6,
In the process control procedure, when it is determined that the reliability of the similarity at the subject position is low in the reliability determination procedure, the similarity calculation procedure is performed when the template matching of the next input image is performed. A subject tracking program for enlarging the size of the search target area for moving a search frame. The subject tracking program according to claim 6,
In the case where it is determined in the reliability determination procedure that the reliability of the similarity at the subject position is low, the processing control procedure is
For the similarity calculation procedure, enlarge the size of the search target area to move the search frame,
For the similarity calculation procedure, performing template matching between the image in the search frame and the template image while moving the search frame in the search target area after being set in the current input image, Recalculate the similarity between the image in the search frame and the template image at each search frame position,
A subject tracking program for causing the position in the input image of the search frame having the highest similarity recalculated in the similarity calculation procedure to be re-specified as the subject position with respect to the subject position specifying procedure.   A subject tracking apparatus comprising execution means for executing the subject tracking program according to claim 1.

Images