JP2014093711A - Tracking apparatus, tracking method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To prevent wrongly tracking of a similar object when losing sight of a target object and performing restoration processing.SOLUTION: When it is determined that the sight of the target object is lost, a tracking apparatus searches the frame image for an object with a feature quantity similar to that of the target object, and stores the result of the search in a first memory 112. Then, in the next frame image, the similar object is tracked, and the restoration processing of the target object is performed. At this time, the apparatus searches, for the target object, areas except an area including the similar object tracked in the process of tracking the similar object.

Description

  The present invention relates to a tracking device, a tracking method, and a program, and more particularly, to a technique suitable for use in searching for a target object that has been lost.

  Conventionally, the following methods are known as methods for tracking a target object. First, a region having a similar feature amount in the current frame image is searched for the feature amount calculated from the region of the target object in the previous frame image in terms of time. As a result of the search, tracking is realized between frames by setting a region having a similar feature amount as a target object region of the current frame image.

  However, depending on the scene, the target object may be lost from the image, for example, the target object may be outside the image, or another object may be crossed and the target object may be hidden. Conventionally, as a countermeasure for the above problem, if the feature value of the target object fluctuates greatly between frames, it is judged that the target object has been lost from the image, and it takes several frames until the target object is found. In the image, a region having a feature quantity similar to the target object is searched from the image. Then, when a region whose feature amount is similar to that of the target object region is found, a process of returning to the tracking from the next frame image as the target object region is performed, so-called return processing is performed from within the image. It solved the problem of losing sight of the target object. Further, for example, Patent Document 1 discloses a countermeasure when two target objects intersect when tracking a plurality of target objects.

JP 2010-117946 A

  However, depending on the scene, there may be a similar object having a region whose feature quantity is similar to that of the target object region in the frame image. In such a case, as a result of losing sight of the target object in the current frame image and performing the return processing on the next frame image, there is a possibility that a similar object is tracked by mistake as the target object. In addition, the method described in Patent Document 1 uses a difference in feature amount between regions of target objects that intersect each other as a countermeasure when two target objects intersect. There is a possibility of tracking by mistake.

  In view of the above-described problems, an object of the present invention is to prevent a similar object from being tracked by mistake when performing a return process while losing sight of a target object.

  The tracking device of the present invention includes a target object tracking unit that tracks a target object between frame images, a search unit that searches for an object having a feature quantity similar to the target object from the frame image, and the search between the frame images. Similar object tracking means for tracking a similar object whose feature quantity is similar to the object searched by the means, determination means for determining whether the target object is lost as a result of tracking by the target object tracking means, and the determination means And when the target object is determined to have lost sight, a frame image determined to have been lost or a return means for searching for the target object from a temporally subsequent frame image, the return means includes the similar object The target object is searched for except for a region including a similar object tracked by the tracking means.

  ADVANTAGE OF THE INVENTION According to this invention, when losing sight of a target object and performing a return process, it can prevent tracking a similar object accidentally.

It is a block diagram which shows the structural example of the imaging device 100 which concerns on embodiment. It is a flowchart which shows an example of the process sequence by the tracking process part in embodiment. It is a figure for demonstrating the feature-value of an area | region. It is a figure for demonstrating the search range used by the return process of a target object. It is a figure for demonstrating the search range with respect to the area | region of a target object. It is a flowchart which shows an example of the detailed process sequence of step S207 of FIG. It is a flowchart which shows an example of the detailed process sequence of step S215 of FIG.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.
FIG. 1 is a block diagram illustrating a configuration example of an imaging apparatus 100 according to the present embodiment.
In FIG. 1, the CPU 101 controls each block connected by the internal bus 102. The operation unit 103 connected to the CPU 101 accepts input from various buttons by the user, and the CPU 101 can perform various processes in accordance with user instructions. When tracking is started, a subject in the image can be set as a target object by inputting information from the operation unit 103 to the CPU 101.

Next, processing for acquiring an image will be described.
The light that has passed through the focus lens 104 forms an image on the imaging surface of the imaging element 106, and the optical image is converted into an electrical signal (analog signal). The lens control unit 107 drives the focus lens 104 and the diaphragm 105 to adjust the focus and the diaphragm amount. The A / D converter 108 converts the analog signal sent from the image sensor 106 into a digital signal (image signal). The image processing unit 109 reads an image signal from the A / D conversion unit 108 and performs various signal processing.

  First, the tracking processing unit 110 first outputs the current frame image (hereinafter, the current frame image) output from the image processing unit 109 and the temporally previous frame image (hereinafter, the previous frame image) stored in the frame memory 111. ) And. Further, the data of the processing result of the tracking processing unit 110 that is stored in the first memory 112 and performed in the previous frame in time is read out. Then, between the frame images, the target object (hereinafter referred to as target object) is tracked based on the read data, and the processing result data of the tracking processing unit 110 of the current frame is stored in the first memory 112. Remember. Detailed processing of the tracking processing unit 110 will be described later with reference to FIG.

  When recording the input image signal, the image signal output from the image processing unit 109 is encoded by the CODEC 113, and the encoded data is written in the second memory 114. Then, the recording medium control unit 115 records the encoded data on the recording medium 116 such as an SD card. When reproducing the recorded image signal, the recording medium control unit 115 reads out the encoded data from the recording medium 116 and writes it into the second memory 114, and decodes the encoded data by the CODEC 113. Then, the decoded image signal is sent to the drawing processing unit 117.

  The drawing processing unit 117 receives an image signal from the image processing unit 109 or the CODEC 113, draws characters or figures on the image based on an instruction from the CPU 101, and writes the image signal after drawing into the third memory 118. The drawing processing unit 117 also performs processing for drawing a tracking frame in the target object region of the current frame image based on the processing result of the tracking processing unit 110 for the current frame image. The display unit 119 performs control to read out an image signal from the third memory 118 and output and display the image signal on an LCD (not shown) or the like.

Next, the operation of the tracking processing unit 110 will be described with reference to FIG. FIG. 2 is a flowchart illustrating an example of a processing procedure of the tracking processing unit 110 in the present embodiment.
First, in step S201, an image signal of the current frame is input from the image processing unit 109. Next, in step S202, the image signal of the previous frame is read from the frame memory 111. In step S203, data of the processing result of the tracking processing unit 110 of the previous frame is read from the first memory 112.

  Here, the data of the processing result of the tracking processing unit 110 of the previous frame is the coordinates of the region of the target object on the previous frame image, or the region of the object (hereinafter referred to as a similar object) whose feature quantity is similar to the target object region. , And Flag information. The value of Flag indicates the following state.

  A flag value of 0 indicates that the target object has not been lost. A flag value of 1 indicates that the target object has been lost and no similar object is present in the image. . A flag value of 2 indicates that the target object has been lost, but a similar object exists in the image. Note that the Flag value at the time of starting tracking of the target object, that is, the initial value of Flag is set to 0. Also, it is assumed that there is no flag value other than 0, 1, and 2.

  After step S204, processing of the current frame image is determined based on the value of Flag, which is one of the data read in step S203. First, a process performed on the current frame image when the value of Flag is 0 and the target object is not lost will be described.

  In step S204, it is determined whether or not the Flag value of the previous frame image is zero. As a result of the determination, if the Flag value is 0, the process proceeds to Step S205, and if the Flag value is not 0, the process proceeds to Step S212. In step S205, the target object is searched on the current frame image using the feature amount of the target object region in the previous frame image.

  Here, a method for searching for a target object will be described with reference to FIG. In the previous frame image shown in FIG. 5A, the position of the target object region 502 with respect to the target object 501 to be tracked is the upper left coordinates (X ′s, Y ′s) and the lower right coordinates (X′e). , Y′e). Therefore, first, a search range is set on the current frame image.

  As shown in FIG. 5B, the position of the area 506 serving as the center of the search range is the upper left coordinates (X ′s, Y ′s) and lower right coordinates ( X′e, Y′e). Hereinafter, a region that is the center of the search range in the current frame image is referred to as a central region.

  Based on the position of the center region 506, the search range 505 is based on predetermined values S_X and S_Y, and the upper left coordinates (X′s−S_X, Y′s−S_Y) and lower right coordinates (X′e + S_X, Y′e + S_Y). Next, a position having a feature quantity most similar to the target object area 502 in the previous frame image is searched for the search range 505 set in the current frame image, and a motion vector indicating the position is detected. In the present embodiment, a luminance signal is used as the feature amount.

Here, feature amounts used in the present embodiment will be described with reference to FIG. If the feature amount of the region 302 of the target object 301 in the previous frame image shown in FIG. 3A is P0, the feature amount P0 indicates all the luminance signals 303 in the region, and the feature amount P0 (x, y) is , The feature value of the luminance signal of the pixel at the position of the coordinates (x, y) on the target object region. When the number of horizontal pixels in the target object region is W and the number of vertical pixels is H, the feature amount P0 is expressed by the following equation (1).
P0 = {P0 (0,0), P0 (1,0),... P0 (W-1, H-1)}
(1)

  In the present embodiment, an absolute difference sum, that is, a so-called SAD value is used as a method for searching for a region whose feature amount is most similar to the region of the target object. In the search range 304 of the current frame image shown in FIG. 3B, if the feature amount of the region 305 that is the current search target is P1, the feature amount P1 indicates all the luminance signals 306 in the region. The feature amount P1 (x, y) is the feature amount of the luminance signal of the pixel at the position of the coordinate (x, y) on the search target area. Similar to the target object area, if the horizontal pixel number of the current search target area is W and the vertical pixel number is H, the SAD value DP between the current search target area 305 and the target object area 302. Is calculated by the following equation (2).

  The SAD value DP between the search target area 302 and the target object area 302 is calculated while shifting the area 305 one pixel at a time from the upper left of the search range 304. The region where the SAD value DP calculated within the search range 304 is the smallest is the region whose feature quantity is most similar to the target object region. The amount of deviation between the region 305 to be searched at this time and the region 302 of the target object is taken as the motion vector of the target object.

  Finally, by adding the detected motion vector component to the coordinates of the target object region in the previous frame image, the position of the region having the most similar feature amount to the target object region in the previous frame image is obtained. . Hereinafter, an area having a feature amount most similar to the area of the target object in the previous frame image is referred to as a target object candidate area.

  As shown in FIG. 5C, for the upper left coordinates (X ′s, Y ′s) and the lower right coordinates (X′e, Y′e) representing the position of the center region 511, the following expression (3 The components (VX, VY) of the motion vector 517 detected in accordance with () are added. Accordingly, the position of the target object candidate region 514 in the current frame image is represented by the upper left coordinates (Xst, Yst) and the lower right coordinates (Xet, Yet).

  Returning to the description of FIG. 2, after the target object tracking process is performed in step S205, it is determined in step S206 whether or not the target object has been lost on the current frame image. In the determination, the feature amount of the target object region in the previous frame image and the feature amount of the target object candidate region in the current frame image are used.

  Here, the feature amount of the target object candidate region in the current frame image is P1, and the feature amount of the target object region in the previous frame image is P0. Then, the threshold value TH is determined for the absolute difference DP of the feature amounts P1 and P0 according to the following equation (4). If the absolute difference DP is greater than or equal to the threshold value TH, it is determined that the target object has been lost, and the process proceeds to step S207. On the other hand, if the difference absolute sum DP is less than the threshold value TH, it is determined that the target object is not lost, and the target object candidate area is the target object area on the current frame image. Proceed to In step S211, the data of the flag having a value of 0 and the coordinates of the target object are stored in the first memory 112 as the processing result of the current frame image, and the processing of the tracking processing unit 110 of the current frame image is terminated. To do.

  Here, the threshold TH is a value obtained by averaging the SAD values of the feature amounts of the target object area of the previous frame image and the target object candidate area of the current frame image, which are obtained by the target object tracking process of the past several frames. It is good to use.

  In step S207, it is searched whether there is a similar object on the current frame image. This similar object search process will be described later in detail with reference to FIG. In step S208, it is determined whether a similar object exists as a result of the search in step S207. As a result of the determination, if a similar object exists, the process proceeds to step S209. On the other hand, if no similar object exists, the process proceeds to step S210.

  In step S209, the data of the flag having a value of 2 and the coordinates of the similar object are stored in the first memory 112 as the processing result of the current frame image, and the processing of the tracking processing unit 110 of the current frame image is ended. . On the other hand, in step S210, Flag data with a value of 1 is stored in the first memory 112 as the processing result of the current frame image, and the processing of the tracking processing unit 110 of the current frame image ends.

  On the other hand, if the flag value is not 0 as a result of the determination in step S204, it is determined whether the flag value is 1 or 2 in step S212. As a result of this determination, if the Flag value is 2 in Step S212, the process proceeds to Step S213, and if the Flag value is 1, the process proceeds to Step S215.

  In step S213, the similar object tracking process is performed by searching for a similar object on the current frame image using the feature amount of the similar object region in the previous frame image. The similar object search method is the same as the target object tracking process in step S205, and the description is omitted because only the target object is replaced with the similar object.

  Next, in step S214, it is determined whether a similar object has been lost on the current frame image. The determination method is the same as the process of determining whether or not the target object is lost in step S206, and the description is omitted because the target object is simply replaced with a similar object. This determination result is used in a target object return process in step S215 to be described later, and in either case, the process proceeds to the next step S215.

  Next, in step S215, the target object is searched on the current frame image. The target object return process will be described in detail later with reference to FIG. In step S216, it is determined whether the target object exists as a result of the target object return process in step S215. As a result of the determination, if the target object exists, the process proceeds to step S211. If the target object does not exist, the process proceeds to step S217.

  In step S217, it is determined whether a similar object exists in the current frame image. As a result of the determination, if a similar object exists, the process proceeds to step S218. If a similar object does not exist, the process proceeds to step S219.

  In step S218, data of the flag having a value of 2 and the coordinates of the region of the similar object are stored in the first memory 112 as the processing result of the current frame image, and the processing of the tracking processing unit 110 of the current frame is terminated. To do. On the other hand, in step S219, Flag data with a value of 1 is stored in the first memory 112 as the processing result of the current frame image, and the processing of the tracking processing unit 110 for the current frame is terminated.

Next, the similar object search process in step S207 of FIG. 2 will be described in detail with reference to FIG. FIG. 6 is a flowchart showing an example of a detailed processing procedure in step S207 of FIG.
First, in step S601, a similar object search range used in step S602 described later is set. First, a range that is arbitrarily wider than the search range used when searching for the target object in the target object tracking process in step S205 is excluded from the previous frame image that is the frame before losing sight of the target object. To do. Then, the range after exclusion is set as a similar object search range, and the process proceeds to step S602.

  Next, in step S602, a search is performed as to whether an object similar to the target object on the same frame image exists within the search range set in step S601 on the previous frame image. The search method at this time performs the same calculation as the target object tracking process in step S205 of FIG. 2, and searches for a region where the SAD value DP between the target object region and the target object region is less than the threshold value TH. After searching for a similar object, the process proceeds to step S603.

  Next, in step S603, it is determined whether the area of the search result obtained in step S602 is a similar object area. Specifically, first, the feature amount P1 of the target object candidate region of the previous frame image calculated in step S205 by the equation (5) and the target object region feature amount P0 of the second previous frame image The absolute difference sum DP1 is calculated. Next, the absolute difference sum DP2 between the feature amount P2 of the area of the search result of the previous frame image calculated in step S602 and the feature quantity P0 of the target object region of the previous frame image is calculated. Then, the calculated difference absolute sum DP1 and DP2 are compared. As a result of this comparison, if the value of the absolute difference sum DP2 is larger than the value of the absolute difference sum DP1, it is determined that the area of the search result is not an area of a similar object, and the similar object search process is terminated. On the other hand, if the value of the absolute difference sum DP2 is equal to or smaller than the value of the absolute difference sum DP, the search result area is determined to be an area of a similar object, and the process proceeds to step S604.

  In step S604, the region of the similar region determined in step S603 is excluded from the search range when searching for a similar object on the previous frame image used in step S602, and the search range is reset. . The search range excluding the similar region determined in step S603 is used when another similar object is searched again in step S602.

Next, details of the target object return processing in step S215 in FIG. 2 will be described with reference to FIG. FIG. 7 is a flowchart illustrating an example of a detailed processing procedure of step S215 in FIG.
First, in step S701, it is determined from the Flag value whether a similar object exists. As a result of this determination, if a similar object exists, that is, if the Flag value is 2, the process proceeds to step S702. If the other state, that is, the Flag value is not 2, the process proceeds to step S705.

  In step S702, the coordinate data of one or more similar object regions obtained in step S213 is acquired. In step S703, a range obtained by excluding the region of the similar object based on the coordinates of the similar object acquired in step S702 from the current frame image is set as a search range used when searching for the target object in step S704 described later. .

FIG. 4 is a diagram illustrating an example of the search range set in step S703.
As shown in FIG. 4, the target object region search range 401 set in the current frame image 404 in step S703 is a range obtained by excluding similar object regions 402 and 403 from the current frame image. When searching for a target object in step S705 to be described later, the target object area is searched in such a manner as to avoid the similar object areas 402 and 403 as indicated by arrows. Therefore, the conventional problem of erroneously recognizing a similar object as a target object as a result of searching for the target object can be solved.

  Returning to the description of FIG. 7, in step S <b> 704, the feature amount data of the target object region stored in advance in the first memory 112 is read. The feature amount data of the target object region is used when searching for the target object in step S705 to be described later. Further, the feature amount data of the target object region used in the present embodiment is the feature amount of the target object region on the frame image at the start of tracking.

  Next, in step S705, it is searched whether the target object exists within the search range set in step S703 on the current frame image. When searching, the feature quantity of the area of the target object read from the first memory 112 in step S704 is used. The search method is the same as the target object tracking process in step S206 of FIG. Then, after performing the search, the process proceeds to step S706.

  Next, in step S706, it is determined whether the target object candidate area obtained by searching the target object area in step S705 is the target object area. The determination method is determined by using the same method as the target object lost determination process in step S206, that is, the equation (4). However, P0 is the feature quantity of the target object of the frame at the start of tracking. The threshold value TH may be the same value as in the lost determination process. And if it determines by step S706, a target object return process will be complete | finished.

  As described above, according to the present embodiment, the tracking of the similar object is performed and the region of the similar object is excluded to perform the return processing of the target object. It is possible to prevent a similar object from being tracked by mistake.

(Other embodiments)
As mentioned above, although preferable embodiment of this invention was described, this invention is not limited to these embodiment, A various deformation | transformation and change are possible within the range of the summary. The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, or the like) of the system or apparatus reads the program. It is a process to be executed.

101 CPU
110 Tracking processing unit 112 First memory

Claims (7)

Target object tracking means for tracking the target object between frame images;
Search means for searching for an object having a feature quantity similar to the target object from the frame image;
Similar object tracking means for tracking similar objects whose feature amounts are similar to the objects searched by the search means between the frame images;
As a result of tracking by the target object tracking means, a determination means for determining whether or not the target object has been lost,
When it is determined by the determination means that the target object has been lost, a frame image determined to have lost sight, or a return means for searching for the target object from a temporally subsequent frame image,
The tracking device according to claim 1, wherein the return means searches for the target object excluding a region including the similar object tracked by the similar object tracking means.   The tracking device according to claim 1, wherein the search unit searches for an object similar to the target object from a frame image determined by the determination unit to have lost sight of the target object.   The tracking device according to claim 1, further comprising a storage unit that stores information when the target object is not lost as a result of the determination by the determination unit.   4. The tracking device according to claim 3, wherein the storage unit further stores information on a result of a search by the search unit when it is determined that the target object has been lost as a result of the determination by the determination unit. .   The target object tracking unit tracks the target object based on an absolute difference between the target object and the feature amount of the previous frame image in time. The tracking device according to item. A target object tracking process for tracking the target object between frame images;
A search step for searching for an object having a feature quantity similar to the target object from the frame image;
A similar object tracking step of tracking a similar object having a feature quantity similar to the object searched in the search step between the frame images;
As a result of tracking in the target object tracking step, a determination step of determining whether or not the target object has been lost,
When it is determined that the target object has been lost in the determination step, a frame image determined to have lost sight, or a return step for searching for the target object from a temporally subsequent frame image,
In the return step, the target object is searched for except for a region including the similar object tracked in the similar object tracking step. A target object tracking process for tracking the target object between frame images;
A search step for searching for an object having a feature quantity similar to the target object from the frame image;
A similar object tracking step of tracking a similar object having a feature quantity similar to the object searched in the search step between the frame images;
As a result of tracking in the target object tracking step, a determination step of determining whether or not the target object has been lost,
When it is determined that the target object has been lost in the determination step, the computer executes a frame image determined to have been lost or a return step for searching for the target object from a temporally subsequent frame image,
In the return step, the target object is searched for excluding a region including the similar object tracked in the similar object tracking step.

Images