KR102552968B1 - Method of tracking multiple objects and apparatus for the same - Google Patents

Abstract

One embodiment of the present invention, the object detection unit detecting a plurality of objects; generating a tracker for tracking a target object, which is any one of the objects, by a tracker generating unit; The tracker learning unit sets the target object as a positive sample for the tracker, sets one or more objects different from the target object among the objects as a negative sample, and uses the samples to learning the tracker; and tracking the target object by an object tracking unit using the tracker.

Description

Multi-object tracking method and device therefor {METHOD OF TRACKING MULTIPLE OBJECTS AND APPARATUS FOR THE SAME}

Embodiments of the present invention relate to a multi-object tracking method and an apparatus therefor, and more particularly, to a multi-object tracking method for tracking multiple objects by learning a tracker using positive samples and negative samples, and device for this purpose.

Technology for tracking people and objects is a representative application in the field of computer vision, and is a key technology required for a surveillance system using a camera that is emerging these days.

In recent years, in a situation where the demand for a surveillance system is increasing, when a user directly checks each image with his/her eyes, a lot of time and effort is required. The above effort can be saved by using a tracker that automatically tracks the target in such a surveillance system.

Korean Patent Publication No. 10-2010-0026123

Embodiments of the present invention set the object tracked by a specific tracker as a positive sample, and set the objects tracked by the other trackers as negative samples, by training all trackers, so that all objects in an image Provided is a multi-object tracking method capable of reliably discriminating and tracking each of the objects even in a complicated situation where objects overlap each other, and a device therefor.

One embodiment of the present invention, the object detection unit detecting a plurality of objects; generating a tracker for tracking a target object, which is any one of the objects, by a tracker generating unit; The tracker learning unit sets the target object as a positive sample for the tracker, sets one or more objects different from the target object among the objects as a negative sample, and uses the samples to learning the tracker; and tracking the target object by an object tracking unit using the tracker.

In the step of generating the tracker, the tracker creation unit further creates an additional tracker for tracking an additional target object that is different from the target object among the objects, and in the step of learning the tracker, the tracker learning unit generates the tracker For , the additional target object is set as a negative sample, and for the additional tracker, the additional target object is set as a positive sample, and the target object is set as a negative sample to mutually learn the tracker and the additional tracker. there is.

In the step of learning the tracker, the tracker learning unit calculates a log likelihood value of each of the classifiers included in the tracker or a combination of two or more of the classifiers to determine a weight of each of the classifiers. By doing so, the tracker can be learned.

Another embodiment of the present invention, an object detection unit for detecting a plurality of objects; a tracker generating unit generating a tracker for tracking a target object that is one of the objects; For the tracker, set the target object as a positive sample, set any one or more objects different from the target object among the objects as negative samples, and learn the tracker using the samples. Tracker Learning Unit; and an object tracking unit for tracking the target object using the tracker.

Other aspects, features and advantages other than those described above will become apparent from the following drawings, claims and detailed description of the invention.

According to the multi-object tracking method and apparatus therefor according to embodiments of the present invention, an object tracked by a specific tracker is set as a positive sample, and objects tracked by other trackers are set as negative samples. By learning all trackers in this way, it is possible to discriminate and track each object more reliably even in a complicated situation where all objects in the image overlap each other.

1 is a diagram showing the configuration of a multi-object tracking device according to an embodiment of the present invention.
2 is an operational flowchart illustrating a multi-object tracking method according to an embodiment of the present invention.
3 is a diagram illustrating a method for setting and learning a tracker according to an embodiment of the present invention.
4 is a diagram illustrating a method of calculating an overlapping ratio between objects according to an embodiment of the present invention.
5 is a diagram illustrating an operation process of a tracker according to an embodiment of the present invention.

Since the present invention can apply various transformations and have various embodiments, specific embodiments will be illustrated in the drawings and described in detail in the detailed description. However, it should be understood that this is not intended to limit the present invention to specific embodiments, and includes all transformations, equivalents, and substitutes included in the spirit and scope of the present invention. In describing the present invention, if it is determined that a detailed description of related known technologies may obscure the gist of the present invention, the detailed description will be omitted.

In the following embodiments, terms such as first and second may be used to describe various components, but the components should not be limited by the terms. Terms are only used to distinguish one component from another.

Terms used in the following examples are only used to describe specific examples, and are not intended to limit the present invention. Singular expressions include plural expressions unless the context clearly dictates otherwise. In the following embodiments, the terms "include" or "have" are intended to indicate that there is a feature, number, step, operation, component, part, or combination thereof described in the specification, but one or more It should be understood that the presence or addition of other features, numbers, steps, operations, components, parts, or combinations thereof is not precluded.

Embodiments of the invention may be presented as functional block structures and various processing steps. These functional blocks may be implemented with any number of hardware or/and software components that perform specific functions. For example, embodiments of the present invention may be directed to memory, processing, logic, look-up tables, etc., which may perform various functions by means of the control of one or more microprocessors or other control devices. Circuit configurations may be employed. Similar to the components of an embodiment of the invention may be implemented as software programming or software elements, an embodiment of the invention may include various algorithms implemented as data structures, processes, routines, or combinations of other programming constructs. , C, C++, Java (Java), can be implemented in a programming or scripting language such as assembler (assembler). Functional aspects may be implemented in an algorithm running on one or more processors. In addition, embodiments of the present invention may employ conventional techniques for electronic environment setting, signal processing, and/or data processing. Terms such as mechanism, element, means, and configuration may be used broadly and are not limited to mechanical and physical configurations. The term may include a meaning of a series of software routines in connection with a processor or the like.

In addition, in this specification and drawings, the same code|symbol is used for the component which has substantially the same structure, and redundant description is abbreviate|omitted.

1 is a diagram showing the configuration of a multi-object tracking device according to an embodiment of the present invention.

Referring to FIG. 1 , a multi-object tracking device according to an embodiment of the present invention includes an object detection unit 110, a tracker generation unit 120, a tracker learning unit 130, and an object tracking unit 140.

The object detection unit 110 detects a plurality of objects in an image.

For example, the object detector 110 may detect a local motion of an object from a difference image between a current image and a reference image obtained while the camera is looking in one direction. The reference image is captured and determined at the moment when a motion detection function for object detection is activated. The current image is an image captured after the reference image is determined. The reference video and the current video are determined by the angle of view of the camera.

The object detector 110 may detect an object by simple comparison between a reference image generated in the same direction and a current image. The method of detecting the local motion of the object by the object detector 110 of the present invention is not limited to the method using the above-described image difference, and various known methods may be applied.

In an optional embodiment, the object detection unit 110 may generate metadata that is a set of object detection results. Metadata may include the number of objects detected in the image and the coordinates of each object. The coordinates of the object may include upper left coordinates and lower right coordinates of the object block.

In an optional embodiment, the object detector 110 may detect objects using an integral channel feature (ICF). The integral channel feature may be composed of a plurality of images on which color-related and direction-related integral operations have been performed, and by using the integral channel feature, the sum of feature values for a desired region can be obtained with only 4 addition or subtraction operations. .

The tracker generation unit 120 creates trackers for tracking objects detected by the object detection unit 110 . The tracker generation unit 120 may create a tracker for tracking any one of the objects as a target object.

A tracker may consist of a combination of a plurality of classifiers. The classifier performs a test on an input image and provides a probability that the corresponding input image is positive or negative as a result value. When the specific image is positive, it means that the corresponding image corresponds to the target object, and when the specific image is negative, it means that the corresponding image does not correspond to the target object.

A classifier may include multiple binary tests. For example, the classifier may be a randomized fern that randomly selects and includes a plurality of binary tests.

A binary test is a test in which the result is set to one of two values. The binary test may be a test that compares sums of values in two arbitrary rectangular areas and determines whether the sum of values in a specific area is greater. For example, the binary test may be a binary test such as Haar-like.

A plurality of binary tests may be connected in a hierarchical structure and performed. Binary tests can be linked in a hierarchical structure in the form of a binary tree. For example, when the first binary test is a root node and the second and third binary tests are child nodes of the first binary test, the first binary test is performed and the result value is the first result value, the second A binary test may be performed, and a third binary test may be performed in case of a second result value. In each of the binary tests, a result value of a child node or classifier may be mapped to a result value.

As an optional embodiment, the tracker generator 120 may create an additional tracker for tracking one of the objects different from the target object as an additional target object. Furthermore, the tracker generator 120 may generate a plurality of trackers that track each of the objects. For example, when three objects are detected by the object detector 110, three trackers for tracking each object may be generated.

The tracker learning unit 130 trains the tracker generated by the tracker generator 120 using positive and negative samples. The positive sample may be information obtained from a target area tracked by the tracker, and the negative sample may be information obtained from an area other than the target area. For example, the tracker learning unit 130 may learn the tracker using a multiple-instance learning boosting (MIL boosting) scheme.

For the tracker, the tracker learning unit 130 may set a target object as a positive sample, set one or more objects different from the target object among the objects as negative samples, and train the tracker using the corresponding samples.

In an optional embodiment, when an additional tracker is generated by the tracker generating unit 120, the target object is set as a positive sample for the tracker, the additional target object is set as a negative sample, and the additional target object is set for the additional tracker. It is possible to mutually learn the tracker and the additional tracker by setting P as a positive sample and learning by setting a target object as a negative sample. Furthermore, even when three or more trackers are generated, the tracker learning unit 130 sets its own target object as a positive sample for each tracker and sets the target object of other trackers as a negative sample, so that the corresponding tracker can learn from each other.

The tracker learning unit 130 may learn the tracker by determining weights for each of the classifiers included in the tracker. The tracker provides a final probability that the input image is positive, and the final probability may be a sum or an average value obtained by multiplying each of the positive probabilities by classifiers included in the corresponding tracker by a weight. For example, when the weight is a value within the range of 0 to '1/(number of trackers)', the final probability is the sum of values obtained by multiplying each of the positive probabilities by the weight, and when the weight is a value within the range of 0 to 1, the final probability is It may be an average value of values obtained by multiplying each of the positive probabilities by a weight.

The tracker learning unit 130 may calculate a log likelihood value for each classifier and determine a weight using the log likelihood value.

For example, the weight of each of the classifiers may be determined as 0 or 1 according to the log likelihood value. That is, when the weight is 0, the result value of the classifier is not reflected in the tracker, and when the weight is 1, the result value of the classifier is reflected in the tracker.

More specifically, a method of determining the weight of each of the classifiers will be described below.

First, a log likelihood value is calculated for each of the classifiers, and the weight of the classifier having the largest log likelihood value is determined to be 1.

Next, after generating combinations using one classifier whose weight is determined to be 1 and each of the remaining classifiers, a log-likelihood value is calculated for each of the combinations, and the remainder used for the combination with the largest log-likelihood value is used. Set the weight of the classifier to 1.

Next, after generating combinations using the two classifiers for which the weight is determined to be 1 and each of the remaining classifiers, a log likelihood value is calculated for each of the combinations, and the remainder used for the combination with the largest log likelihood value is used. Set the weight of the classifier to 1. Each time this process is repeated, the classifier whose weight is determined to be 1 can be increased by one. That is, by repeating this process, the weights of a predetermined number of classifiers may be determined to be 1, and the weights of the remaining classifiers may be determined to be 0.

In an alternative embodiment, the tracker learning unit 130 may train the tracker using a positive bag and a negative bag. A positive bag is a sample group including at least one positive sample, and a negative bag is a sample group including only negative samples.

The object tracking unit 140 tracks an object using the tracker learned in the tracker learning unit 130 . According to the learning result of the tracker learning unit 130, a specific tracker can more reliably discriminate and track a corresponding object even if a specific object overlaps with other objects.

In an optional embodiment, when overlapping occurs between objects, the object tracking unit 140 calculates the overlapping ratio by calculating the ratio of the width of the union region to the width of the intersection region with respect to regions corresponding to the objects. can do.

That is, in the multi-object tracking device according to an embodiment of the present invention, for a specific tracker, an object tracked by the corresponding tracker is set as a positive sample and the remaining objects are set as negative samples. By mutually learning all trackers, each object can be discriminated and tracked more reliably even in a complicated situation where all objects in an image overlap each other.

2 is an operational flowchart illustrating a multi-object tracking method according to an embodiment of the present invention.

Referring to FIG. 2 , in the multi-object tracking method according to an embodiment of the present invention, an object detector detects a plurality of objects (S210), and a tracker generator generates a tracker for tracking a target object (S220). , the tracker learning unit using the samples to learn the tracker (S230), and the object tracking unit tracking the target object (S240).

First, an object detection unit detects a plurality of objects in an image (S210). In step S210, the object detector may detect objects using the integral channel feature.

Next, the tracker generation unit creates a tracker for tracking the objects detected in step S210 (S220). In step S220, the tracker generation unit may create a plurality of trackers for tracking each of a plurality of objects.

Next, the tracker learning unit learns the tracker generated in step S220 using positive and negative samples (S230). In step S230, the tracker learning unit may learn the tracker by setting a target object, which is a target for the tracker to track, as a positive sample for a specific tracker, and setting at least one object different from the target object as a negative sample. . When a plurality of trackers are generated in step S220, the tracker learning unit sets its target object as a positive sample for each tracker and sets the target object of another tracker as a negative sample, thereby mutually learning the corresponding trackers. can

Finally, the object detector tracks the object using the tracker learned in step S230 (S240). When overlapping occurs between objects in step S240, the object detection unit may calculate an overlapping ratio by calculating a ratio of the width of the union region to the width of the intersection region with respect to regions corresponding to the objects.

As such, in the multi-object tracking method according to an embodiment of the present invention, for a specific tracker, an object tracked by the corresponding tracker is set as a positive sample, and the remaining objects are set as negative samples. By mutually learning all trackers with , each object can be discriminated and tracked more reliably even in a complicated situation where all objects in an image overlap each other.

3 is a diagram illustrating a method for setting and learning a tracker according to an embodiment of the present invention.

Referring to FIG. 3 , the tracker setting method according to an embodiment of the present invention can be explained through a process of setting the trackers 331, 332, and 333 for each of the objects 311, 312, and 313. .

For example, as shown in FIG. 3(a), the object detection unit recognizes people 301, 302, and 303 in the image 310, and as shown in FIG. 3(b), an area including people 301, 302, and 303 Can be obtained as objects (311, 312, 313). Although FIG. 3 illustrates an example of acquiring an object by recognizing people, the recognition object of the present invention is not limited, and objects may be acquired for various recognition objects according to known recognition algorithms.

Next, as shown in FIG. 3(c), the tracker generation unit may create a tracker for an object to be tracked among all objects 320. 3 shows a case in which the trackers 331, 332, and 333 are generated for all of the objects 320 (311, 312, and 313), but some of the objects 320 (eg, 311 and 312) ), trackers (eg, 331 and 332) may be generated. In this case, the tracker 331 may track the object 311, the tracker 332 may track the object 312, and the tracker 333 may track the object 313 as a target object.

As such, the tracker according to an embodiment of the present invention may be set to track each of the recognized objects in the image.

Referring to FIG. 3 , the tracker learning method according to an embodiment of the present invention includes positive samples 341a, 342a, and 343a and negative samples 341b, 342b, and 343b for trackers 331, 332, and 333, respectively. This can be explained through the setting process.

For example, the tracker learning unit sets the target object 311 as the positive sample 341a for the tracker 331 and sets the target objects 312 and 313 of the other trackers 332 and 333 as the negative sample 341b. can In addition, the tracker learning unit sets the target object 312 as the positive sample 342a for the tracker 332, and sets the target objects 311 and 313 of the other trackers 331 and 333 as the negative sample 342b. can In addition, the tracker learning unit sets the target object 313 as the positive sample 343a for the tracker 333, and sets the target objects 311 and 312 of the other trackers 331 and 332 as the negative sample 343b. can

In this way, a tracker according to an embodiment of the present invention can mutually learn by setting its own target object as a positive sample and setting the target object of another tracker as a negative sample for each of the trackers.

4 is a diagram illustrating a method of calculating an overlapping ratio between objects according to an embodiment of the present invention.

Referring to FIG. 4( a ) , it can be seen that overlap between two objects 311 and 312 among the objects 311 , 312 and 313 has occurred.

As shown in FIG. 4(b), the object detection unit measures the area of the union area A∪B and the area of the intersection area A∩B with respect to the areas A and B corresponding to the objects 311 and 312. By calculating the ratio of , the overlapping ratio can be calculated.

5 is a diagram illustrating an operation process of a tracker according to an embodiment of the present invention.

Referring to FIG. 5 , the tracker according to an embodiment of the present invention takes an image (x) as an input and provides a probability value (H(x)) that the image (x) is positive as a result.

The tracker includes a plurality of classifiers (h1, hi, hr), and each of the classifiers (h1, hi, hr) performs a plurality of binary tests on the image (x) to obtain a probability value (eg, image ( x) is either positive (c0) or negative (c1). The tracker multiplies the probability values calculated from the classifiers (h1, hi, hr) by the weights (a1, ai, ar) and finally provides the probability value (H(x)) that the image (x) is positive. .

Embodiments according to the present invention described above may be implemented in the form of a computer program that can be executed on a computer through various components, and such a computer program may be recorded on a computer-readable medium. At this time, the medium is a magnetic medium such as a hard disk, a floppy disk and a magnetic tape, an optical recording medium such as a CD-ROM and a DVD, a magneto-optical medium such as a floptical disk, and a ROM hardware devices specially configured to store and execute program instructions, such as RAM, flash memory, and the like. Furthermore, the medium may include an intangible medium implemented in a form transmittable on a network, and may be, for example, a medium implemented in the form of software or an application and capable of transmission and distribution through a network.

Meanwhile, the computer program may be specially designed and configured for the present invention, or may be known and usable to those skilled in the art of computer software. An example of a computer program may include not only machine language code generated by a compiler but also high-level language code that can be executed by a computer using an interpreter or the like.

Specific implementations described in the present invention are examples and do not limit the scope of the present invention in any way. For brevity of the specification, description of conventional electronic components, control systems, software, and other functional aspects of the systems may be omitted. In addition, the connection of lines or connecting members between the components shown in the drawings are examples of functional connections and / or physical or circuit connections, which can be replaced in actual devices or additional various functional connections, physical connection, or circuit connections. In addition, if there is no specific reference such as “essential” or “important”, it may not be a component necessarily required for the application of the present invention.

Therefore, the spirit of the present invention should not be limited to the above-described embodiments and should not be determined, and all scopes equivalent to or equivalently changed from the claims as well as the claims described below are within the scope of the spirit of the present invention. will be said to belong to

110: object detection unit 120: tracker generation unit
130: tracker learning unit 140: object tracking unit

Claims (5)

learning a tracker using positive and negative samples obtained from an image; and
Including; tracking the object using the tracker;
The positive sample is obtained from a target area tracked by the tracker in the image, and the negative sample is obtained from an area other than the target region in the image;
The tracker learning step includes learning a plurality of trackers,
The object tracking method of the object tracking device, wherein the positive samples to be learned by each of the plurality of trackers are different from each other. According to claim 1,
The plurality of trackers include a first tracker and a second tracker,
The object tracking method of the object tracking device, wherein the positive sample of the first tracker is the negative sample of the second tracker. The method of claim 1, wherein the tracker learning step,
The object tracking method of the object tracking device, wherein each of the plurality of trackers mutually learns the plurality of trackers so that they are learned using positive samples, which are objects tracked by themselves, and negative samples, which are objects tracked by other trackers. The method of claim 1, wherein the tracker learning step,
Learning a plurality of positive samples and a plurality of negative samples for each of the plurality of trackers. The method of claim 1, wherein the tracker learning step,
Characterized in that the tracker is learned by calculating a log likelihood value of each of the classifiers included in the tracker or a combination of any two or more of the classifiers to determine a weight of each of the classifiers, Object tracking method of object tracking device.

Images