KR101284815B1 - Target tracking apparatus - Google Patents

Abstract

In accordance with another aspect of the present invention, a target tracking device includes: a sub block generator configured to generate a plurality of sub reference blocks in a reference block including a target to be tracked in a reference image frame; A block matching unit searching for matching sub blocks corresponding to each of the plurality of sub reference blocks in a current image frame using a block matching algorithm; An outlier removing unit removing an outlier of the matching subblocks based on a distance between each of the matching subblocks and a tracking point of the target to be tracked in the reference image frame; And a tracking point determiner configured to determine a tracking point of a target to be tracked in the current image frame using the matching subblocks from which the outliers are removed.

Description

Target tracking apparatus

The present invention relates to a target tracking method and apparatus, and more particularly, to a target tracking method and apparatus for tracking a moving target in a frame-based input image.

The block matching algorithm (BMA) is a representative technique for tracking a moving target in an input image.

The BMA is a method of assigning one motion vector per block, assuming that all pixels in any block of an image frame to estimate motion have the same motion. In general, motion can be represented by rotation, movement, and enlargement or reduction in size.BMA estimates the motion estimation on the premise that all movements can be approximated only by the movement of an object because the deformation caused by the movement is not large between frames close in time. Will be done. Therefore, the motion vector is represented as one x, y coordinate per block. In the BMA, a search block having the highest similarity is found by moving the search block in the current video frame and comparing with the reference block of the reference video frame, which is one of the previous video frames. The position difference of the search block having the highest similarity with the reference block is the motion vector of the reference block.

Target tracking by BMA is vulnerable to changes in the shape of the target because it uses rigid body assumptions. In particular, tracking may fail when an object becomes larger or smaller in an image according to a distance from the camera, or when an object rotates with respect to the camera.

SUMMARY OF THE INVENTION The present invention has been made in an effort to provide a target tracking method and apparatus capable of performing target tracking effectively even with a change in shape of a target, and a computer readable recording medium having recorded thereon a program for executing the method.

In order to solve the above technical problem, a target tracking method according to the present invention includes: (a) generating a plurality of sub reference blocks in a reference block including a target to be tracked in a reference image frame; (b) finding matching subblocks corresponding to each of the plurality of sub-reference blocks in a current image frame using a block matching algorithm; (c) removing an outlier of the matching subblocks based on a distance between each of the matching subblocks and a tracking point of the target to be tracked in the reference image frame; And (d) determining a tracking point of a target to be tracked in the current image frame using the matching subblocks from which the outliers are removed.

In one embodiment, step (a) may divide the reference block into a predetermined size to generate the sub reference blocks.

In an embodiment, the step (a) may detect corner points in the image in the reference block and generate blocks of a predetermined size including each of the corner points as the sub reference blocks.

In an embodiment, the step (a) may detect the edge in the image in the reference block, and generate the sub reference blocks by generating blocks of a predetermined size along the edge.

In an embodiment, the step (c) may include obtaining a distance between each matching subblock and the tracking point in the reference image frame, and determining and removing the matching subblock whose distance is out of a threshold. have.

In one embodiment, the distance may be an Euclidean distance.

In one embodiment, the distance may be a Mahalanobis distance.

In one embodiment, step (d) may determine the center of gravity of the matching subblocks from which the outliers have been removed as the tracking point.

In order to solve the above technical problem, a target tracking device according to the present invention includes: a sub block generation unit generating a plurality of sub reference blocks in a reference block including a target to be tracked in a reference image frame; A block matching unit searching for matching sub blocks corresponding to each of the plurality of sub reference blocks in a current image frame using a block matching algorithm; An outlier removing unit removing an outlier of the matching subblocks based on a distance between each of the matching subblocks and a tracking point of the target to be tracked in the reference image frame; And a tracking point determiner configured to determine a tracking point of a target to be tracked in the current image frame using the matching subblocks from which the outliers are removed.

In an embodiment, the sub block generator may generate the sub reference blocks by dividing the reference block into a predetermined size.

In an embodiment, the sub-block generator may detect corner points in an image in the reference block, and generate blocks of a predetermined size including each of the corner points as the sub-reference blocks.

In an embodiment, the sub-block generator may detect the edge in the image in the reference block, and generate the sub-reference blocks by generating blocks of a predetermined size along the edge.

In one embodiment, the outlier removing unit may obtain a distance between each matching subblock and the tracking point in the reference image frame, and determine and remove the outlier matching subblock whose distance is out of a threshold. .

In one embodiment, the tracking point determiner may determine the center of gravity of the matching subblocks from which the outliers are removed as the tracking point.

In order to solve the above technical problem, there is provided a computer-readable recording medium recording a program for executing the target tracking method described above.

According to the present invention described above, even when a change in shape such as a change in size or rotation of a target to be tracked or an obstruction by another object occurs, the target tracking can be robustly performed.

1 shows a block diagram of a target tracking device according to an embodiment of the present invention.
2 shows an example of generating a sub-reference block according to embodiments of the present invention.
3 illustrates an example of a method of removing an outlier among matching subblocks according to an embodiment of the present invention.
4 illustrates an example in which a tracking point of a target to be tracked is determined according to an embodiment of the present invention.
5 and 6 show target tracking results according to an embodiment of the present invention.
7 is a flowchart of a target tracking method according to an embodiment of the present invention.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the drawings. In the following description and the accompanying drawings, substantially the same components are denoted by the same reference numerals, and redundant description will be omitted. In the following description of the present invention, a detailed description of known functions and configurations incorporated herein will be omitted when it may make the subject matter of the present invention rather unclear.

According to the existing target tracking method using the BMA, the target is tracked by finding the most similar block in the current image frame using the BMA for a specific block including the target to be tracked in the reference image frame. This method is vulnerable to the shape change of the target to be tracked. In the present invention, a block including a target to be tracked is divided into a plurality of sub-blocks, and the target is tracked by finding the most similar block in the current video frame using the BMA for each sub-block. Detailed features are relatively maintained even when the overall size and shape of the target change, and according to the present invention, the tracking performance is prevented from being degraded due to the change in the overall shape of the target.

1 shows a block diagram of a target tracking device according to an embodiment of the present invention. The target tracking device 100 includes a sub block generation unit 110, a block matching unit 120, an outlier removing unit 130, and a tracking point determination unit 140.

The target tracking device 100 receives each image frame as a current image frame, and simultaneously receives a reference image frame to be compared with the current image frame. The reference video frame is one of the previous video frames of the current video frame. The reference video frame is an image frame adjacent to the current video frame, or a video frame at the time of the last tracking in which the tracking target is specified within the video frame, or a predetermined number of times than the current video frame. It can be a video frame or the like as far as advance. The reference image frame may be updated every image frame, may be updated every few image frames are processed, or may be fixed to the image frame at the time of initial tracking. How to set the reference video frame and how to update follows the existing BMA algorithm.

The sub block generator 110 receives the reference image frame and generates a plurality of sub reference blocks having a smaller size than the reference block in the reference block, which is a predetermined area including the target to be tracked, in the reference image frame.

The reference block including the target to be tracked is a target area using a known technique (for example, a technique using statistical characteristics of the target area and the background area) based on the tracking point of the target to be tracked in the corresponding image frame. Can be determined by extracting

The sub-reference block is a block present in the reference block and smaller in size than the reference block as described above. However, some of the sub reference blocks may deviate from the reference blocks.

According to a specific embodiment, sub-reference blocks may be generated by dividing the entire reference block into a predetermined size. 2 (a) shows an example of how a sub-reference block is generated accordingly. Referring to FIG. 2A, the reference block is divided into 8 × 5 = 40 sub-reference blocks.

In another specific embodiment, corner points may be detected in an image in the reference block, and blocks of a predetermined size including each of the detected corner points may be generated as sub blocks. Corner point detection can be performed using existing representative corner detectors such as Harris and SUSAN. The corner point corresponds to the boundary between the target to be traced and the background, and thus, the corner point is less affected by the shape change of the target, which is effective for tracking. Referring to FIG. 2B, a corner point is detected and sub reference blocks are generated at each corner point.

In another specific embodiment, sub-blocks may be generated by detecting an edge in an image in the reference block and generating blocks of a predetermined size along the edge. Edge detection can be performed using conventional edge detection operators such as Robert, Sobel, and Kirsh. The edges, like the corner points, correspond to the boundary between the target to be tracked and the background, which is effective for tracking because the edge is less affected by the shape change of the target. Referring to FIG. 2B, an edge is detected and sub-reference blocks of a predetermined size are generated along the edge.

Referring back to FIG. 1, the block matching unit 120 receives a current image frame. The block matching unit 120 searches for matching sub blocks corresponding to each of the sub reference blocks in the current image frame by using a block matching algorithm, targeting the sub reference blocks generated by the sub block generating unit 110. In detail, the block matching unit 120 searches for a search block having the most similarity to the corresponding sub reference block by moving the search block having the same size as the corresponding sub reference block in the current image frame for each sub reference block. The matching subblock corresponding to the block is determined.

The outlier removing unit 130 removes an outlier that is determined to deviate from the tracking target among the matching subblocks found by the block matching unit 120. Ideally, the matching sub-blocks would all be inliers that do not leave the target being tracked, but some matching sub-blocks may occur because the shape of the target may change or an occlusion may occur where other objects obscure the target. The trace may have failed.

The outlier removing unit 130 removes the outliers among the matching subblocks based on the distance between each matching subblock and the tracking point of the target to be tracked in the reference image frame. This is because a matching subblock that is excessively deviated from the tracking point of the reference video frame is more likely to be obtained incorrectly than other matching subblocks. The outlier remover 130 obtains the distance between each matching subblock and the tracking point of the reference image frame, and determines the matching subblock whose out of range is out of the threshold as an outlier. Here, the threshold may be a predetermined value or may be a value determined according to the obtained distance (for example, a value determined based on the average and standard deviation of the obtained distances).

According to a specific embodiment, the outlier removing unit 130 may select an outlier among the matching subblocks based on an Euclidean distance, that is, a general distance between the matching subblock and the tracking point in the reference image frame. Remove That is, the Euclidean distance is obtained for each matching subblock, and the matching subblock whose threshold is out of the threshold is determined as an outlier and removed.

In another specific embodiment, the outlier removing unit 130 may output the outliers of the matching subblocks based on a probabilistic distance between the matching subblocks and the tracking points in the reference image frame, that is, the mahalanobis distance. Remove the liars. In other words, the Mahalanobis distance is obtained for each matching subblock, and the matching subblock whose threshold is out of the threshold is determined as an outlier and removed. The Mahalanobis distance is the distance concept used in cluster analysis. It is not only the distance between two points but also the distance from which the standard deviation and the correlation coefficient which characterize the variable are considered.

In this embodiment, the outlier remover 130 calculates a distance vector between the matching subblock and the tracking point in the reference video frame, and obtains statistical information such as an average, a standard deviation, and a correlation coefficient. Based on this, the Mahalanobis distance of each matching subblock is calculated, and the subblock whose threshold is out of the threshold is determined as an outlier and removed. Using the Mahalanobis distance in this way, the outlier can be more accurately removed when the target shape is not uniform around the tracking point, such as the shape of the target is elongated in a specific direction.

3 illustrates an example of a method for removing an outlier among matching subblocks according to an embodiment of the present invention. 3A illustrates a tracking point and a sub reference block of a reference video frame. 3 (b) shows how a matching subblock is found for each subblock. In FIG. 3B, an arrow indicates a motion vector of a matching subblock based on the subreference block. 3 (c) shows an outlier of the matching subblocks removed based on the Mahalanobis distance. As a result, the outlier becomes a matching subblock outside the valid gate, which is a certain area containing the tracking points. If you use Euclidean distance, the effective gate will be concentric with the tracking point, but if you use Mahalanobis distance, the effective gate will be oval in shape, as shown. .

Referring back to FIG. 2, the tracking point determiner 140 determines the tracking point of the target to be tracked in the current image frame by using the matching subblocks from which the outliers of the matching subblocks are removed. The tracking point can be determined, for example, by the center of gravity of the matching subblocks. Referring to FIG. 4D, the tracking point of the target to be tracked in the current image frame is determined based on the center of gravity of five matching subblocks except the outlier.

4 illustrates an example in which a tracking point of a target to be tracked is determined according to an embodiment of the present invention. 4 (a) shows a state in which a plurality of sub reference blocks are generated in a reference image frame. Referring to FIG. 4B, the matching subblocks are found for each sub-reference block. In FIG. 4 (b), the tracking target is blocked due to the tree, and the outer five matching sub-blocks in the outlier removing unit 130 are determined as the outliers. As shown in FIG. 4B, the tracking point in the current video frame is determined based on the center of gravity of the remaining matching subblocks except for the outliers.

5 and 6 show target tracking results according to an embodiment of the present invention.

Referring to FIG. 5, a reference block including a target to be tracked is divided into six sub-reference blocks (a), and a matching sub-block is found for each sub-reference block (b, c) to perform target tracking. This is shown. As shown in FIG. 5, even if the size of the target to be tracked changes considerably, it can be confirmed that the tracking was successfully performed.

Referring to FIG. 6, a reference block including a target to be tracked is divided into 15 sub-reference blocks (a). In (b), the tracking point is determined after two matching subblocks among the matching subblocks found for each sub-reference block are removed as outliers. In (c), the tracking point is determined after four matching subblocks of the matching subblocks are removed as outliers. In (c), the tracking point is determined after four matching subblocks are removed as outliers.

7 is a flowchart of a target tracking method according to an embodiment of the present invention. Target tracking method according to an embodiment of the present invention, it is composed of the steps that are processed in the target tracking device 100 described above. Therefore, even if omitted below, the above description of the target tracking device 100 is also applied to the target tracking method according to an embodiment of the present invention.

In operation 710, the target tracking device 100 receives a reference image frame and a current image frame.

In operation 720, the target tracking apparatus 100 generates a plurality of sub reference blocks in a reference block including a target to be tracked in a reference image frame.

In operation 730, the target tracking apparatus 100 searches for matching subblocks corresponding to each of the plurality of sub-reference blocks in the current image frame by using a block matching algorithm.

In operation 740, the target tracking apparatus 100 removes an outlier among the matching subblocks based on the distance between each of the matching subblocks and the tracking point of the target to be tracked in the reference image frame.

In operation 750, the target tracking apparatus 100 determines a tracking point of the target to be tracked in the current image frame using the matching subblocks from which the outliers are removed.

The target tracking device according to the embodiment of the present invention described above may include a bus coupled to respective units of the device as shown in FIG. 1, and at least one processor coupled to the bus. It may also include a memory coupled to the bus for storing instructions, received messages or generated messages and coupled to the at least one processor for performing the instructions as described above.

The above-described embodiments of the present invention can be embodied in a general-purpose digital computer that can be embodied as a program that can be executed by a computer and operates the program using a computer-readable recording medium. The computer-readable recording medium includes a storage medium such as a magnetic storage medium (e.g., ROM, floppy disk, hard disk, etc.), optical reading medium (e.g., CD ROM,

So far I looked at the center of the preferred embodiment for the present invention. It will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims. Therefore, the disclosed embodiments should be considered in an illustrative rather than a restrictive sense. The scope of the present invention is defined by the appended claims rather than by the foregoing description, and all differences within the scope of equivalents thereof should be construed as being included in the present invention.

Claims (8)

A sub block generator configured to generate a plurality of sub reference blocks in a reference block including a target to be tracked in the reference image frame;
A block matching unit searching for matching sub blocks corresponding to each of the plurality of sub reference blocks in a current image frame using a block matching algorithm;
An outlier removing unit removing an outlier of the matching subblocks based on a distance between each of the matching subblocks and a tracking point of the target to be tracked in the reference image frame; And
And a tracking point determiner configured to determine a tracking point of a target to be tracked in the current image frame using the matching subblocks from which the outliers are removed. delete delete delete The method of claim 1,
The outlier removal unit obtains a distance between each matching subblock and the tracking point in the reference video frame, and removes the matching subblock whose threshold is out of a threshold by determining the outlier. Device. The method of claim 5,
The distance is a Euclidean distance. The method of claim 5,
And said distance is a Mahalanobis distance. The method of claim 1,
And the tracking point determiner determines the center of gravity of the matching subblocks from which the outliers are removed as the tracking point.

Images