WO2012074362A1

WO2012074362A1 - Method and system for tracking object using adaptive attention regions

Info

Publication number: WO2012074362A1
Application number: PCT/MY2011/000128
Authority: WO
Inventors: Kim Meng Liang
Original assignee: Mimos Bhd
Current assignee: Mimos Bhd
Priority date: 2010-12-02
Filing date: 2011-06-22
Publication date: 2012-06-07
Anticipated expiration: 2013-06-02
Also published as: MY162243A

Abstract

The present invention provides a method of tracking a target object in a scene, where the method is based on the use of the adaptive attention regions of the target object. The method does not use complete profile of the target object, saving computer power. More importantly, the method allows track the target object even when the target object has high similarity with the scene background and other objects in the scene. The present invention also provides a system of automatic tracking a target object in a scene.

Description

METHOD AND SYSTEM FOR TRACKING OBJECT USING ADAPTIVE

ATTENTION REGIONS

Field of the Invention

[0001] The present invention generally relates to automatic object recognition and tracking technologies, and more particularly to a system and method for automatically tracking a target object based on adaptive attention regions of the target object Background of the Invention

[0002] Object tracking is an important part of video surveillance and video analysis systems, and is best described as an automatic process to assign to a target object appearing in a scene the same label from the beginning when the target object enters the scene until it leaves the scene, where the target object refers to any object that is to be tracked in the scene.

[0003] Current object tracking systems and methods are diverse. For example, US

Patent No 7,684,592 discloses a real-time computer vision system tracks one or more objects moving in a scene using a target location technique which does not involve searching. The imaging hardware includes a color camera, frame grabber and processor. The software consists of the low-level image grabbing software and a tracking algorithm. The system tracks objects based on the color, motion and/or shape of the object in the image. A color matching function is used to compute three measures of the target's probable location based on the target color, shape and motion. The method then computes the most probable location of the target using a weighting technique. Once the system is running, a graphical user interface displays the live image from the color camera on the computer screen. The operator can then use the mouse to select a target for tracking, fhe system will then keep track of the moving target in the scene in real-time.

[0004] US patent application No 2009/0175500 Λ1 discloses an object tracking apparatus that tracks an object on image data captured continuously. The object tracking apparatus includes an object color adjusting unit and a particle filter processing unit. The object color adjusting unit calculates tendency of color change in regions on image data and adjusts a color of the object set as an object color based on the tendency of color change to obtain a reference color. The particle filter processing unit estimates a region corresponding to the object on image data based on likelihood of each particle calculated by comparing a color around each particle with the reference color, using particles which move on image data according to a predefined rule.

[0005] It is a still unsolved issue in object tracking where the properties of a target object in a scene have high similarity with the scene background or other objects in the scene. In this case, the known tracking algorithm that is mainly based on the complete profile of the target object to perform tracking will likely predict the wrong location for the target object.

Summary of the Invention

[0006] One aspect of the present invention provides a method of tracking a target object in a scene. In one embodiment, the method comprises selecting a target object that is to be tracked in a scene, predicting possible locations of the target object on next frame: calculating the properties of the background and other objects present at the predicted locations; determining the attention regions of the target object: and determining the tracked location of the target object on the next frame; whereby the method allows to track the target object even when the larget object has high similarity with the scene background and other objects in the scene.

[0007] In one embodiment of the method, the step of selecting a target object comprises identifying new motion blobs that are calculated by image differencing technique, where the target object is selected automatically.

[0008] In another embodiment of the method, the step of selecting a target object comprises manual selection via a graphic user interface (GUT).

[0009] In another embodiment of the method, the step of predicting the possible locations of the target object on next frame comprises using a prediction algorithm based on the history locations of the target object at the previous frames.

[0010] In another embodiment of the method, the step of calculating the properties of the background and other objects present at the predicted locations comprises predicting other objects in the predicted location where the target object is predicted on the next frame; determining whether other objects are found in the predicted location; calculating the properties of the other objects at the predicted location, and storing the information of the properties of the other objects into the storage if the other objects arc found (Yes) at the predicted location; and calculating the properties of the background at the predicted location regardless of whether other objects arc found in the predicted location, and stored into the storage if no other object is found (No) at the predicted location. In a further embodiment of the method, the step of calculating the properties of the other objects at the predicted locations further comprises determining the regions of the other objects that do not overlap with each other at the predicted locations; deciding if non-overlap regions are found: wherein if non-overlap regions are found (Yes), the properties of the non-overlap regions of the other objects are calculated as stored information; and wherein if non- overlap regions are not found (No), the size and bottom position of the overlapped other objects are determined; and identifying the regions that belong to the other objects that arc bigger in size and lower in terms of bottom position.

[0011] In another embodiment of the method, the step of determining the attention regions of the target object comprises calculating the color shade properties of the target object; grouping the areas having almost similar color or with the difference of the median color below a predefined threshold together as a similar shade region; calculating the color shades of the overlap other objects and background at the predicted location; and calculating the matching score between each color shade properties of the target object and the color shade properties of the overlap other objects and background; thereby, if the calculated matching score is be-low the predefined threshold value, the respective color shade properties are extracted as attention regions. In a further embodiment of the method, the color shade properties are calculated based on clustering technique.

[0012] In another embodiment of the method, the tracked location is calculated using appearance modeling, motion modeling or searching methods.

Ϊ0013] Another aspect of the present invention provides a system for automatically tracking an object in one scene. In one embodiment, the system comprises a visual information module for acquiring and presenting visual images frame by frame; a microprocessor being electronically coupled with the visual information module; and a graphic user interface (GUI) being electronically coupled with the microprocessor. 100141 The objectives and advantages of the invention will become apparent from the following detailed description of preferred embodiments thereof in connection with the accompanying drawings. Brief Description of the Drawings

[0015] Preferred embodiments according to the present invention will now be described with reference to the Figures, in which like reference numerals denote like elements.

[0016] FIG { is a functional block flowchart that illustrates the method of tracking a target object, in a scene in accordance with one embodiment of the present invention.

[0017] FIG 2 is a functional block flowchart that illustrates the calculation of the ptx>perties of the background and other objects within the predicted locations 30 in accordance with one embodiment of the present in vention.

[0018] FIG 3 is a functional block flowchart that illustrates the operation of the calculation of the propcilics of the other objects at the predicted location 33 in accordance with one embodiment of the present invention.

[0019] FIG 4 shows the illustration of calculating the properties of the other objects at the predicted locations.

[0020] FIG 5 is a functional block flowchart thai illustrates the operation of determining the attention regions of the target object 40 in accordance with one embodiment of the present invention.

[0021] FIG 6 shows a graph illustrating the output of the clustering technique to obtain color shade in accordance with one embodiment of the present invention.

[0022] FIG 7 is an exemplary functional block diagram o f the system in accordance with one embodiment of the present invention.

Detailed Description of the Invention [0023] The present invention may be understood more readily by reference to the following detailed description of certain embodiments of the invention. [0024] Throughout this application, where publications are referenced, the disclosures of these publications arc hereby incorporated by reference, in their entireties, into this application in order to more fully describe the state of art to which this invention pertains.

[0025] One aspect of the present invention provides a ttiethod of tracking a target object in a scene that have high similarity with the scene background and other objects in the scene. Here, the target object is the one to be tracked in the scene; the other objects refer to the objects that move within the scene, likely interfering with the tracking of the target object. It is to be noted that the method and system of the present invention are not limited to tracking one object at one time; instead, they are suitable for simultaneously tracking more than one object within the scene. For the sake of brevity, the description of the present invention uses one target object ιο illustrate the principles of the present invention.

[0026] The method is performed by an algorithm that is stored in a computer executable medium, and the algorithm can be implemented in any computer system for surveillance and visual analysis. In principle, the method of the present invention for tracking a target object is based on the use of the adaptive attention regions of the target object. Briefly, the target object is divided into two more regions based on their color shade properties; then one or more of the regions are adaptively selected in light of the background information and other objects within each frame of the scene; then the adaptively selected regions are used to track the target object within each frame. The selected regions for tracking are designated as attention regions; since the attention regions for each frame arc adaptively selected, they arc designated as adaptive attention regions throughout this application. Evidently, the method of the present invention does not use the complete profile of a target object to perform the tracking of the target object; it is advantageous in consuming less computing power and more critically reducing or eliminating the tracking errors when the target object has high similarity with the background of or other objects in the scene.

[0027] Now referring to FIG 1. there is provided a functional block flowchart that illustrates the method of tracking a target object in a scene in accordance with one embodiment of the present invention. As shown in FIO I. after Start, the method of tracking a target object 1 first selects a target object that is to be tracked in a scene (target object) 10; then predicts the possible locations of the target object on next frame (predicted locations) 20, each location within the next frame is referred to an area in which the target object might be present; then calculates the properties of the background and other objects at the predicted locations 30; then determines the attention regions of the target object 40; and finally determines the location of the target object on the next frame (tracked location) 50.

[0028] In one embodiment, the step 10 of selecting a target object is done automatically by for example identifying new motion blobs that are calculated by image differencing technique. In another embodiment, it is done by manual selection via a graphic user interface (GUI). After the target object is selected, its basic properties are calculated and stored, where the basic properties include color shade, texture, edge information and shape.

[0029] In one embodiment, the step 20 of predicting the possible locations of the target object on next frame is done using for example a prediction algorithm based on the history locations of the target object at the previous frames.

[0030] Now referring to FIG 2, there is provided a functional block flowchart that illustrates the calculation of the properties of the background and other objects within the predicted locations 30 in accordance with one embodiment of the present invention. As shown in FIG 2, the calculation of the properties of the background and other objects at the predicted locations 30 first predicts other objects that likely move into the predicted location where the target object is predicted on the next frame 31 ; then determines whether other objects are found in the predicted location 32; if the other objects are found (Yes) at the predicted location, then calculates the properties of the other objects at the predicted location 33, and stores the information of the properties of the other objects into the storage 34; if no other object is found (No) at the predicted location, the properties of the background at the predicted location are calculated regardless of whether other objects are found in the predicted location, and stored into the storage 35.

[0031] Now referring to FIG 3, there is provided a functional block flowchart that illustrates the operation of the calculation of the properties of the other objects at the predicted location 33 in accordance with one embodiment of the present invention. As shown in FIG 3, the regions of the other objects that do not overlap with each other at the predicted location are first determined 321. At the decision step 322, if non-overlap regions are found (Yes), the properties of the non-overlap regions of the other objects are calculated as stored information 323; if non-overlap regions arc not found (No), the size and bottom position of the overlapped other objects are determined 324. The regions thai belong to the other objects that are bigger in size and lower in terms of bottom position are identi fied as stored information 325. FIG 4 shows the illustration of calculating the properties of the other objects at the predicted locations.

{0032] Now referring to FIG 5, there is provided a functional block flowchart that illustrates the operation of determining the attention regions of the target object 40 in accordance with one embodiment of the present invention. As shown in FIG 5, the color shade properties of the target object are first calculated 41. In one embodiment, the color shade properties are calculated based on clustering technique. The output of the clustering technique to obtain color shade is illustrated in FIG 6. The areas having almost similar color or with the difference of the median color below a predefined threshold are grouped together as a similar shade region. In the following step 42, the color shades of the overlap other objects and background at the predicted location are calculated. Next 43 is calculating the matching score between each color shade properties of the target object and the color shade properties of the overlap other objects and background. If the calculated matching score is below the predefined threshold value 44, the respective color shade properties arc extracted as attention regions.

[0033] The tracked location of the target object on the next frame 50 is determined based on the extracted attention regions. In one embodiment, the tracked location is calculated using appearance modeling, motion modeling and searching methods.

[0034] Another aspect of the present invention provides a system for automatically tracking an object in one scene. As shown in FIG 7, the system 100 comprises a visual information module 110 for acquiring and presenting visual images frame by frame, a microprocessor 120 being electronically coupled with the visual information module 110, and a graphic user interface (GUI) 130 being electronically coupled with the microprocessor. The visual information module 110 may comprise a video camera for recording the images of the scene. The microprocessor 120 comprises a computer executable medium for embedding the programs of automatic tracking and performing the automatic tracking as described above. The GUI can be any screen for displaying the images and results and receiving inputs from a user. The system 100 can be any device or apparatus for surveillance and visual analysis.

[0035] While the present invention has been described with reference to particular embodiments, it will be understood that the embodiments are illustrative and thai the invention scope is not so limited. Alternative embodiments of the present invention will become apparent to those having ordinary skill in the art to which the present invention pertains. Such alternate embodiments arc considered to be encompassed within the scope of the present invention. Accordingly, the scope of the present invention is defined by the appended claims and is supported by the foregoing description.

Claims

What is claimed is: I . A method of tracking a target object in a scene, comprising:

selecting a target object that is to be tracked in a scene;

predicting possible locations of the target object on next frame;

calculating the properties of the background and other objects present at the predicted locations;

determining the attention regions of the target object; and

determining the tracked location of the target object on the next frame;

whereby the method allows to track the target object even when the target object has high similarity with the scene background and other objects in the scene.

2. The method of claim 1 , wherein the step of selecting a target object comprises identifying new motion blobs that are calculated by image differencing technique, where the target object is selected automatically.

3. The method of claim 1 , wherein the step of selecting a target object comprises manual selection via a graphic user interface (GUI).

4. The method of claim I , wherein the step of predicting the possible locations of the target object on next frame comprises using a prediction algorilhm based on the history locations of the target object at the previous frames.

5. The method of claim 1 , wherein the step of calculating the properties of the background and other objects present at the predicted locations comprises:

predicting other objects in the predicted location where the target object is predicted on the next frame;

determining whether other objects arc found in the predicted location; calculating the properties of the other objects at the predicted location, and storing the information of the properties of the other objects into the storage if the other objects are found (Yes) at the predicted location; and

calculating the properties of the background at the predicted location regardless of whether other objects arc found in the predicted location, and stored into the storage if no other object is found (No) at the predicted location.

6. Hie method of claim 5, wherein the step of calculating the properties of the other objects at the predicted locations further comprises:

determining the regions of the other objects that do not overlap with each other at the predicted locations;

deciding if non-overlap regions are found; wherein if non-overlap regions arc found (Yes), the properties of the non-overlap regions of the other objects arc calculated as stored information; and wherein if non-overlap regions are not found (No), the size and bottom position of the overlapped other objects are determined; and

identifying the regions that belong to the other objects that are bigger in size and lower in terms of bottom position.

7. The method of claim 1 , wherein the step of determining the attention regions of the target object comprises:

calculating the color shade properties of the target object;

grouping the areas having almost similar color or with the difference of the median color below a predefined threshold together as a similar shade region;

calculating the color shades of the overlap other objects and background at the predicted location: and

calculating the matching score between each color shade properties of the target object and the color shade properties of the overlap other objects and background;

thereby, if the calculated matching score is below the predefined threshold value, the respective color shade properties are extracted as attention regions.

8. The method of claim 7, wherein the color shade propenies are calculated based on clustering technique.

9. The method of claim I , wherein the tracked location is calculated using appearance modeling, motion modeling or searching methods.

10. A system for automatically tracking an object in one scene, said system comprising: a visual inforniaiion module for acquiring and presenting visual images frame by frame;

a microprocessor being electronically coupled with the visual inibmiation module; and

a graphic user interlace (GUI) being electronically coupled with the microprocessor;

wherein the microprocessor comprises a computer executable medium for embedding the programs of automatic tracking and performing the method of the automatic tracking of claim 1.