KR20200104267A - Object tracking method and apparatus performing the same - Google Patents

Abstract

Disclosed are an object tracking method capable of providing improved object tracking performance and a device performing the same. According to one embodiment of the present invention, the object tracking method comprises the steps of: setting a second bounding box by expanding a first bounding box for tracking a target object; setting an area of the second bounding box excluding the first bounding box as an outer area; determining whether or not the first bounding box is reset based on a feature point in the first bounding box and a feature point in the outer area; and resetting the first bounding box to the second bounding box if it is determined that the first bounding box is reset.

Description

Object tracking method and device that performs it {OBJECT TRACKING METHOD AND APPARATUS PERFORMING THE SAME}

The following embodiments relate to an object tracking method and an apparatus for performing the same.

Most of the currently used tracking algorithms use a bounding box-based tracking technology. Tracking is performed by designating a bounding box containing the object to be tracked in the image, dividing the background and the object within the box, and adjusting the position of the bounding box while learning the characteristics of the object.

The Boosting tracking algorithm is an improvement of the AdaBoost machine learning algorithm, and while tracking an object, it adjusts the classifier in real time and naturally handles changes in appearance such as rotation of an object and changes in brightness. In addition, it provides a stable tracking result by grasping the most distinguishing features of objects according to the background. Additionally, fast tracking is achieved by tracking using features that can be easily calculated, such as a direction histogram.

In the case of the detection-based tracking method used in the past, when the tracker is not accurate, an error occurs in generating training data, which degrades the performance of the classifier. To improve this, a Multiple Instance Learning (MIL) tracking algorithm was proposed. The algorithm uses multi-MIL learning rather than the conventional learning method. By using the image set to update the appearance model, it has been shown that tracking performance is improved even when using smaller parameters.

The above tracking methods are examples of tracking algorithms that have emerged to improve the existing tracking methods. These are relatively recently developed algorithms, and continuous research is ongoing. In addition, it has the advantage of showing fast and stable performance in common.

However, these methods initially use a fixed bounding box and do not resize the rectangle boundary. Therefore, when the object size changes, there is a disadvantage in that the area of the bounding box cannot be accurately matched.

For example, when an object gets closer to the camera, the weight of the object within the rectangular boundary increases. In this case, since all objects cannot be contained in the bounding box area, the characteristics of the object that the tracker can learn is reduced.

Conversely, as the object moves away from the camera, the weight of the target within the rectangular boundary decreases. In this case, a phenomenon in which another object appears on a rectangular boundary may occur, and an error in which the tracker learns characteristics of another object may occur.

Embodiments may provide a technique for determining whether there is an object other than an object to be tracked in a bounding box using a feature point.

In addition, embodiments may provide a technique for adjusting the size of a bounding box by using clustering analysis of feature points inside and outside the bounding box.

Accordingly, embodiments may provide improved object tracking performance in an image in which the size of an object changes.

An object tracking method according to an embodiment includes the steps of setting a second bounding box by expanding a first bounding box for tracking a target object, and removing an area from the second bounding box excluding the first bounding box into an outer area. And determining whether to reset the first bounding box based on a feature point in the first bounding box and a feature point in the outer region; and, when it is determined to reset, the first bounding box is set to the first bounding box. 2 It may include the step of resetting to the bounding box.

The determining may include calculating a clustering degree of the feature points in the outer region, and between-cluster variance, which is a variance between the central position of the feature point in the first bounding box and the central position of the feature point in the outer region. ), and determining whether to reset the first bounding box based on the clustering degree and the variance between the colonies.

The determining may include determining whether the object in the external area is an object different from the target object.

The resetting may include resetting the first bounding box to the second bounding box when the object is not determined to be an object different from the target object.

The central position of the feature point in the outer area may be a feature point in which a distance dispersion to the remaining feature points is minimized based on one of the plurality of feature points in the outer area.

The clustering degree may be a variance calculated based on a central position of a feature point in the outer region.

An object tracking device according to an embodiment includes a memory including instructions and a processor for executing the instructions, and when the instructions are executed by the processor, the processor includes a first for tracking a target object. A second bounding box is set by expanding a bounding box, and an area other than the first bounding box in the second bounding box is set as an outer area, and based on a feature point in the first bounding box and a feature point in the outer area It is determined whether to reset the first bounding box, and when it is determined to reset, the first bounding box is reset to the second bounding box.

The processor calculates a clustering degree of the feature points in the outer region, calculates a between-cluster variance, which is a variance between the central position of the feature point in the first bounding box and the central position of the feature point in the outer region, , It may be determined whether to reset the first bounding box based on the clustering degree and the variance between the colonies.

The processor may determine whether the object in the external area is an object different from the target object.

When the object is not determined to be an object different from the target object, the processor may reset the first bounding box to the second bounding box.

The central position of the feature point in the outer area may be a feature point in which a distance dispersion to the remaining feature points is minimized based on one of the plurality of feature points in the outer area.

1 is a diagram illustrating an object tracking device according to an exemplary embodiment.
2 is a schematic block diagram of the object tracking apparatus shown in FIG. 1.
3 is a flow chart for describing in detail an object tracking operation performed by the object tracking device.
4 is a diagram for describing an example of a first bounding box and a second bounding box set in a frame.
5A and 5B are diagrams for explaining an example of an operation in which the object tracking device adjusts the size of a bounding box.
6A and 6B are diagrams for explaining an example of an operation in which the object tracking device adjusts the size of a bounding box.

Hereinafter, exemplary embodiments will be described in detail with reference to the accompanying drawings. However, since various changes may be made to the embodiments, the scope of the rights of the patent application is not limited or limited by these embodiments. It should be understood that all changes, equivalents, or substitutes to the embodiments are included in the scope of the rights.

The terms used in the examples are used for illustrative purposes only and should not be interpreted as limiting. Singular expressions include plural expressions unless the context clearly indicates otherwise. In the present specification, terms such as "comprise" or "have" are intended to designate the presence of features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, but one or more other features. It is to be understood that the presence or addition of elements or numbers, steps, actions, components, parts, or combinations thereof, does not preclude in advance.

Terms such as first or second may be used to describe various components, but the components should not be limited by terms. The terms are only for the purpose of distinguishing one component from other components, for example, without departing from the scope of rights according to the concept of the embodiment, the first component may be named as the second component, and similarly The second component may also be referred to as a first component.

Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which the embodiment belongs. Terms as defined in a commonly used dictionary should be interpreted as having a meaning consistent with the meaning in the context of the related technology, and should not be interpreted as an ideal or excessively formal meaning unless explicitly defined in this application. Does not.

In addition, in the description with reference to the accompanying drawings, the same reference numerals are assigned to the same components regardless of the reference numerals, and redundant descriptions thereof will be omitted. In describing the embodiments, when it is determined that a detailed description of related known technologies may unnecessarily obscure the subject matter of the embodiments, the detailed description thereof will be omitted.

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. However, the scope of the patent application is not limited or limited by these embodiments. The same reference numerals in each drawing indicate the same members.

1 is a diagram illustrating an object tracking device according to an exemplary embodiment, and FIG. 2 is a schematic block diagram of the object tracking device shown in FIG. 1.

1 and 2, the object tracking device 100 may receive a video 110. The object tracking device 100 may adjust the size of the bounding box of the frame 130 by using the frame 130 of the video 110. The object tracking device 100 may be implemented in an electronic device.

Electronic devices include a data server, a computer, a laptop computer, a mobile phone, a smart phone, a tablet PC, a mobile internet device (MID), a personal digital assistant (PDA). ), EDA (enterprise digital assistant), digital still camera, digital video camera, portable multimedia player (PMP), personal navigation device or portable navigation device (PND), handheld game console game console), an e-book, or a smart device. For example, the smart device may be implemented as a smart watch or a smart band.

The object tracking device 100 includes a memory 210, a receiver 230, and a controller 250.

The memory 210 may store instructions (or programs) executable by the controller 250. For example, the instructions may include instructions for executing an operation of the controller 250 and/or an operation of each component of the controller 250.

The receiver 230 may receive the video 110. The video 110 may include one or more frames 130.

The controller 250 may set the second bounding box 330 by extending the first bounding box 310 for tracking the target object in the frame 130. In addition, the controller 250 may set the outer area 350 of the second bounding box 330 excluding the area of the first bounding box 310.

The controller 250 may determine whether to reset the first bounding box 310 using the first bounding box 310 and the second bounding box 330. For example, the controller 250 is based on the feature points in the first bounding box 310 and the feature points in the outer region 350 excluding the region of the first bounding box 310 in the second bounding box 310. Whether to reset the bounding box 310 may be determined.

In this case, the controller 250 may maintain the first bounding box 310 as it is or reset the second bounding box 330 to the first bounding box 310. That is, the reset first bounding box may be the first bounding box 310 or the second bounding box 330.

The controller 250 may adjust the size of the reset first bounding box 310 or 330 based on the reset first bounding box 310 or 330. For example, the controller 250 may adjust the size of the reset first bounding box 310 or 330 based on the feature points in the reset first bounding box 310 or 330.

The object tracking apparatus 100 may improve tracking performance in a video in which the size of an object changes by adjusting the size of the bounding box.

3 is a flow chart for describing in detail an object tracking operation performed by an object tracking device, and FIG. 4 is a diagram for describing an example of a first bounding box and a second bounding box set in a frame.

3 and 4, the controller 250 may set the second bounding box 330 and the outer region 350 based on the first bounding box 310 (410). For example, the controller 250 extends the width (w) and height (h) of the first bounding box 310 by a, respectively, and the width (w+a) and the height ( h+a), and the outer area 350 excluding the first bounding box 310 from the second bounding box 330 may be set.

The controller 250 may extract feature points from the first bounding box 310 and the outer region 350. Also, the controller 250 may calculate a clustering degree of feature points in the first bounding box 310 and the outer region 350 (420).

)를 계산할 수 있다.The clustering degree of the feature points in the first bounding box 310 may be variance. For example, the cluster diagram of the feature points in the first bounding box 310 may be a variance value of a distance from the center coordinates of the feature points to each feature point. The controller 250 uses Equation 1 and Equation 2 to determine the center coordinates of the feature points (

) Can be calculated.

Here, K _i is a set of feature points in the first bounding box 310, N is the number of elements in K _i , and K _i (x _t ) and K _i (y _t ) are each of the t-th feature points in K _i It means x-coordinate and y-coordinate.

The controller 250 may calculate variance of horizontal coordinates and variance of vertical coordinates of the feature points in the first bounding box 310 by using Equations 3 and 4.

)를 구할 수 있다.The controller 250 may calculate a clustering degree of feature points within the outer region 350. The clustering degree of the feature points within the outer region 350 and the outer region 350 may be a variance calculated based on the central position of the feature points within the outer region 350. The central position of the feature point in the outer region 350 may be a central position of the feature point at which distance dispersion to the remaining feature points is minimized based on one of the plurality of feature points in the outer region 350. The controller 250 uses Equation 5 and Equation 6 to determine the central position of the feature points (

) Can be obtained.

Here, K _out is a set of feature points in the outer region 350, N is a coefficient of an element of K _out , and n has a value from 0 to (N-1).

) 및 외부 영역(350) 내 특징점의 중심위치(

)에 대한 집락 간 분산(variance between clusters)을 계산할 수 있다.The controller 250 is the central position of the feature point in the first bounding box 310 (

) And the central position of the feature point in the outer area 350 (

The variance between clusters for) can be calculated.

The controller 250 is based on at least one of the distribution between clusters of the central position of the feature point in the first bounding box 310 and the central position of the feature point in the outer region 350 and the cluster diagram of the feature points in the outer region 350. Whether to reset the first bounding box 310 may be determined.

For example, the controller 250 may determine whether to reset the first bounding box 310 by sequentially comparing the distribution between the colonies and the clustering degree of the feature points of the outer region 350.

First, the controller 250 may determine whether to reset the first bounding box 310 by comparing the variance between the colonies with a threshold (430 ). For example, the controller 250 determines how far away from the central point between the central position of the feature point in the first bounding box 310 and the central position of the feature point in the outer region 350 through dispersion between colonies, It may be determined whether an object different from the target object is included in the bounding box 350.

When the variance between the colonies is less than the threshold value, since the clusters of feature points between the first bounding box 310 and the outer region 350 are adjacent, it may be determined that the object in the outer region 350 is the same object as the target object. Accordingly, when the variance between the colonies is less than the threshold value, the controller 250 may reset the second bounding box 310 to the first bounding box 310 (460).

When the variance between the colonies is greater than or equal to the threshold value, the controller 250 may determine whether to reset the first bounding box 310 by comparing the clustering degree of the feature points in the outer region 350 with the threshold value (440). For example, the controller 250 determines that there are several objects in the second bounding box 310 when the variance between the colonies is greater than or equal to the threshold value and the clustering degree of the feature points in the outer region 350 is less than the threshold value. 1 It is determined not to reset the bounding box 310, and the first bounding box 310 may be maintained (450 ).

The controller 250 may maintain the first bounding box 310 as it is when the variance between the colonies is greater than or equal to the threshold value and the clustering degree of the feature points of the outer region 350 is greater than the threshold value (460 ).

That is, the controller 250 may maintain the first bounding box 310 as it is or reset the second bounding box 330 to the first bounding box 310.

Thereafter, the controller 250 may adjust the size of the reset first bounding box 310 or 330 based on the cluster diagram of the feature points in the reset first bounding box 310 or 330. The controller 250 compares the variance in each coordinate direction of the reset feature points of the first bounding box 310 or 330 based on the cluster diagram of the feature points in the reset first bounding box 310 or 330 Can be calculated. For example, the controller 250 may calculate the relative variance by comparing the variance of the horizontal coordinates and the variance of the vertical coordinates of the feature points in the reset first bounding box 310 or 330, and the controller 250 The size of the reset first bounding box 310 or 330 may be adjusted based on the variance.

5A and 5B are diagrams for explaining an example of an operation in which the object tracking device adjusts the size of a bounding box.

5A and 5B are a case in which the size of the target object decreases as the target object moves away from the camera while the object tracking device 100 tracks the pedestrian. In FIG. 5B, when the object tracking operation of the object tracking apparatus 100 is used compared to the case of using the existing algorithm, it can be seen that the bounding box is adjusted to an appropriate size as the target object becomes smaller.

6A and 6B are diagrams for explaining an example of an operation in which the object tracking device adjusts the size of a bounding box.

6A and 6B illustrate a case in which the size of the target object increases as the target object approaches the camera while the object tracking device 100 tracks the vehicle. In FIG. 6B, when the object tracking operation of the object tracking apparatus 100 is used rather than using the existing algorithm, it can be seen that the bounding box is adjusted to an appropriate size as the target object increases.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, and the like alone or in combination. The program instructions recorded on the medium may be specially designed and configured for the embodiment, or may be known and usable to those skilled in computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. -A hardware device specially configured to store and execute program instructions such as magneto-optical media, and ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine language codes such as those produced by a compiler but also high-level language codes that can be executed by a computer using an interpreter or the like. The hardware device described above may be configured to operate as one or more software modules to perform the operation of the embodiment, and vice versa.

The software may include a computer program, code, instructions, or a combination of one or more of these, configuring the processing unit to behave as desired or processed independently or collectively. You can command the device. Software and/or data may be interpreted by a processing device or to provide instructions or data to a processing device, of any type of machine, component, physical device, virtual equipment, computer storage medium or device. , Or may be permanently or temporarily embodyed in a transmitted signal wave. The software may be distributed over networked computer systems and stored or executed in a distributed manner. Software and data may be stored on one or more computer-readable recording media.

As described above, although the embodiments have been described by the limited drawings, a person of ordinary skill in the art can apply various technical modifications and variations based on the above. For example, the described techniques are performed in a different order from the described method, and/or components such as a system, structure, device, circuit, etc. described are combined or combined in a form different from the described method, or other components Alternatively, even if substituted or substituted by an equivalent, an appropriate result can be achieved.

Therefore, other implementations, other embodiments and claims and equivalents fall within the scope of the following claims.

Claims (12)

Setting a second bounding box by expanding a first bounding box for tracking a target object;
Setting a region of the second bounding box excluding the first bounding box as an external region;
Determining whether to reset the first bounding box based on a feature point in the first bounding box and a feature point in the outer region; And
When it is decided to reset, resetting the first bounding box to the second bounding box
Containing, object tracking method.
The method of claim 1,
The determining step,
Calculating a clustering degree of feature points in the outer region;
Calculating between-cluster variance, which is a variance between the central position of the feature point in the first bounding box and the central position of the feature point in the outer area; And
Determining whether to reset the first bounding box based on the cluster diagram and the variance between the colonies
Containing, object tracking method.
The method of claim 1,
The determining step,
Determining whether the object in the external area is an object different from the target object
Containing, object tracking method.
The method of claim 3,
The resetting step,
When the object is not determined to be an object different from the target object, resetting the first bounding box to the second bounding box
Containing, object tracking method.
The method of claim 2,
The central position of the feature point in the outer region is,
The object tracking method, wherein the distance variance to the remaining feature points is minimized based on one of the plurality of feature points in the external area.
The method of claim 5,
The cluster diagram,
The object tracking method, which is a variance calculated based on the center position of the feature point in the outer area.
A memory containing instructions; And
A processor for executing the instructions
Including,
When the instructions are executed by the processor, the processor,
A second bounding box is set by expanding a first bounding box for tracking a target object, a region other than the first bounding box is set as an external region, and a feature point in the first bounding box and Determining whether to reset the first bounding box based on the feature point in the external area, and resetting the first bounding box to the second bounding box when it is determined to reset the first bounding box.
The method of claim 7,
The processor,
Calculate the clustering degree of the feature points in the outer region, calculate the between-cluster variance, which is a variance between the central position of the feature point in the first bounding box and the central position of the feature point in the outer region, and the cluster diagram And determining whether to reset the first bounding box based on the variance between the colonies.
The method of claim 7,
The processor,
An object tracking device that determines whether an object in the external area is an object different from the target object.
The method of claim 9,
The processor,
When the object is not determined to be an object different from the target object, resetting the first bounding box to the second bounding box.
The method of claim 8,
The central position of the feature point in the outer region is,
The object tracking device, wherein the distance distribution to the remaining feature points is minimized based on one of the plurality of feature points in the external area.
The method of claim 11,
The cluster diagram,
The object tracking device, which is a variance calculated based on the center position of the feature point in the external area.

Images