US20070133840A1 - Tracking Using An Elastic Cluster of Trackers - Google Patents

Tracking Using An Elastic Cluster of Trackers Download PDF

Info

Publication number
US20070133840A1
US20070133840A1 US11/556,896 US55689606A US2007133840A1 US 20070133840 A1 US20070133840 A1 US 20070133840A1 US 55689606 A US55689606 A US 55689606A US 2007133840 A1 US2007133840 A1 US 2007133840A1
Authority
US
United States
Prior art keywords
tracking
target
trackers
voting
elastic matrix
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Abandoned
Application number
US11/556,896
Other languages
English (en)
Inventor
Andrew Cilia
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Clean Earth Technologies LLC
Original Assignee
Clean Earth Technologies LLC
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Clean Earth Technologies LLC filed Critical Clean Earth Technologies LLC
Priority to US11/556,896 priority Critical patent/US20070133840A1/en
Assigned to CLEAN EARTH TECHNOLOGIES, LLC reassignment CLEAN EARTH TECHNOLOGIES, LLC ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: CILIA, ANDREW
Publication of US20070133840A1 publication Critical patent/US20070133840A1/en
Assigned to WARPET, LLC reassignment WARPET, LLC SECURITY AGREEMENT Assignors: CET, LLC (F/K/A CLEAN EARTH TECHNOLOGIES, L.L.C.)
Abandoned legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
    • G06T7/248—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments involving reference images or patches
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/24—Aligning, centring, orientation detection or correction of the image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/103—Static body considered as a whole, e.g. static pedestrian or occupant recognition
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30196—Human being; Person
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30232—Surveillance
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30241—Trajectory

Definitions

  • the present invention is in the field of methods for tracking objects, which may be non-rigid objects, and that may be moving in complex, cluttered environments, especially in multi-dimensional situations where the object being tracked, any of the tracked targets, may be occluded by another object between the viewer or sensor and the target.
  • objects to be tracked are humans, animals, vehicles, tactical military equipment, parts in a factory, and in vivo objects in tissue.
  • methods are those that pertain to tracking humans or parts thereof, or groups of humans in images, video scenes, or maps, which are generated by optical, electro-optical, radar and other sensor systems and devices.
  • Tracking objects by optical, electro-optical, radar systems, and other sensors is important in security, surveillance and reconnaissance, traffic and flow control, industrial and healthcare applications.
  • Common problems encountered in tracking objects referred to as the targets, are the occlusion of the target object when another object is situated between the sensor and the target, the dynamic variation of the target morphology, e.g., the relative motion of limbs, head and torso while walking or during other movement, variation and diversity in lighting, and the non-uniformity of motion of the target and its parts.
  • Prior art has addressed many of these problems with various degrees of success, complexity, and accuracy. For some situations, such as for synthetic aperture radar mapping of a large area and the detection and tracking of numerous moving targets, very elaborate tracking methods have shown considerable success.
  • the automatic tracking of individual persons as they move through a dynamically changing scene poses the challenges to avoid loss of ‘locking’ on the target, to maintain or reacquire tracking as the target moves erratically, adds or subtracts garments or picks up packages or performs other actions that will change appearance and form, and to perform accurate tracking with sufficiently efficient and rapid information processing so that real-time or near-real-time use of the tracking information can be made, e.g., the graphical display of the track in an image.
  • Prior art includes many examples of tracking methods, schemes, and techniques, which include motion prediction, pixel correlations, probabilistic data association, association or clustering of sets of objects or features, cost minimization function methods, expectation-maximization methods, and looping or iteration through a sequence of algorithms and process steps. Some of these steps include thresholding, filtering (including multiple particle filtering), track association, and multiple layers of objects, e.g., foreground and background. Tracking of clusters of features has been extensively applied to variations and extensions of the classic Kanade, Lucas, Tomasi (often called the “KLT”) tracking scheme, which allows for translation, rotation, and deformation of a target. KLT trackers work well for small displacements and for a limited amount of occlusion.
  • Chiba et al used the sum of squared differences (SSD) method applied to patches of the image, selected high confidence patches, estimated the optical flow, and then applied the KLT hierarchy to tracking.
  • SSD sum of squared differences
  • This invention is a method of tracking target objects, which may be non-rigid target objects, e.g., humans, in complex, cluttered environments in which the view of the target may be subject to severe or complete occlusion by objects between the viewer (i.e., imaging sensor, camera, or radar) and the target.
  • the method called the Elastic Cluster Tracker, uses insulated small patches as features that are identified and retained or discarded according to the correlation of their motion and spatial relationship with the track of the target.
  • the tracking process is initiated with two successive video frames. In the first frame, a target designation window is constructed around the target to define a region of interest of the image containing the target. This window may be constructed by a human operator or may be generated from the results of an automated target recognition (ATR) algorithm. Multiple targets may be designated by constructing multiple windows, one enclosing each target.
  • the subsequent tracking process then comprises the following three steps:
  • the motion of patches that are image segments of a grid within an initially designated target window is determined by calculating the pixel-by-pixel convolution values to construct correlation surfaces for candidate matches (patches) in the succeeding video frame with each patch in the preceding frame.
  • the segments correspond to ‘kernels’ of specified size in contrast to many prior tracking schemes in which specific shapes, textures, colors, or other characteristics are selected a priori as tracking features.
  • a weighted, layered, four-dimensional (4-D, e.g., “phase space” comprising 2 spatial components, x, y, and 2 velocity components, u,v) voting scheme is used with voting resolution that collects votes in a limited neighborhood of each kernel in the image grid to determine the highest quality kernel track and accordingly the velocity vector in the Motion Field.
  • Elastic Matrix Creation is performed.
  • a target cluster is generated by segmentation (partition) of the target designation window in the first frame and Candidate Targets are evaluated for the quality of their track and correlation with the expected motion as predicted by the Motion Field.
  • Individual members of the target cluster are referred to as “Trackers”.
  • Background and target segments are identified, e.g., background segments may be static, so that background segments may not be considered further.
  • Deviant Trackers also are dismissed from further consideration.
  • the remaining Trackers are grouped by the similarity of their motion and the Elastic Matrix is generated for the member set of Trackers and their nearest neighbors.
  • Each node of the matrix contains the position in phase space and information about the quality and persistence of each member.
  • an Elastic Matrix Relationship is determined that predicts the position of either member of the pair in case it is occluded or disappears.
  • Recurring Tracking is then performed for successive video frames.
  • Intensity based convolution operations provide correlation surfaces.
  • Least Square Error tracking is performed by minimization of errors on the correlation surfaces to find the best match.
  • Weighted amalgamation is used to combine the tracking data of a small cluster of Trackers to obtain a larger effective aperture. The weights are a function of the amplitude of the corresponding correlation peak.
  • Vote tallying is performed to identify nodes in the Elastic Matrix with similar motion vectors. The velocity pair layer with the most support decides the tracking results for each Tracker. Next the Motion Field, Elastic Matrix, and Elastic Matrix Relationships are updated.
  • the Elastic Cluster Tracker has several important characteristics. These are:
  • FIG. 1 The Elastic Cluster Tracker is used to track a person. Shown is a person, the object to be tracked, with several candidate targets (features) with motion and attributes that are captured in an Elastic Matrix that describes their temporal-spatial correspondences.
  • FIG. 2 Motion Fields are used to evaluate candidate members of a target cluster.
  • the target object is shown with motion vectors that comprise the local motion fields on and around the target.
  • FIG. 3 A set of Candidate Targets is created. Salient features of the target are selected for tracking based on the feature's target quality indicators.
  • FIG. 4 An Elastic Matrix is created. Temporal-spatial relationships among candidate targets are established and maintained in a data structure called the Elastic Matrix. Relationships are displayed as lines between the candidate targets.
  • FIG. 5 Tracking based on the Elastic Matrix is performed. This sequence illustrates the tracking process using an Elastic Matrix.
  • the Matrix maintains the cohesiveness of the cluster of trackers while allowing each tracker to follow its marker.
  • FIG. 6 Tracking is performed through foreground occlusions. This sequence illustrates the Elastic Cluster Tracker's ability to track through several foreground occlusions.
  • FIG. 7 Tracking is performed through an obscuration. Shown are two real-life tracking sequences of maneuvering targets in an uncontrolled occluded environment.
  • FIG. 8 A pedestrian is tracked outdoors. This tracking sequence shows the tracker following a pedestrian through a severe hard occlusion and through a severe partial occlusion.
  • FIG. 9 Persons shopping at a mall and in a skating rink are tracked. Shown are four real-life examples of the simultaneous tracking of individuals in an uncontrolled crowded environment.
  • Motion Fields extraction is an optical flow process that calculates the local motion at all points of the input video frame.
  • the local motions are used to validate targets during the cluster creation and to guide the trackers during the recurring tracking of the target.
  • the target acquisition process is initiated by either a human operator or by an Automatic Target Recognition (ATR) process external to the tracker.
  • Inputs to the algorithm consist of two consecutive frames of video plus a region-of-interest (ROI) designator that encloses the area where the target is present.
  • ROI region-of-interest
  • the scene's local motion is extracted by a process called Motion Fields Extraction.
  • the results are stored in an array of local motion descriptors used to both validate extracted candidate targets and to guide the tracking process during the initial phases of target tracking.
  • the Motion Fields Extraction process is composed of the following steps: Generate Candidate Matches, Localized Motion Voting, and Voting Resolution.
  • each ROI of the first video frame subdivide each ROI of the first video frame using a fine grid comprising segments that contain several contiguous pixels. For every segment of the grid, execute a convolution search (based on intensity and or color) on the second video frame.
  • Using several kernel sizes can yield better defined motion descriptors. This may result by the greater suitability of smaller kernels in regions close to motion boundaries, and by the greater suitability of larger kernels for larger areas of little texture.
  • the best results are obtained by using the three kernel sizes: 5 ⁇ 5, 7 ⁇ 7, and 9 ⁇ 9 pixels.
  • the entire set of results of the convolution search are retained as the entire correlation surfaces of all the candidate matches, in contrast to the method of Nicolascu in which only the correlation peaks are retained. Keeping the entire correlation surfaces, a greatly simplified tensor voting scheme can be used effectively.
  • the aperture of several neighboring kernels are combined to reinforce common traits while eliminating the noise inherent of low aperture trackers. This avoids a well-known difficulty that is otherwise encountered when using small kernels to deduce the local motion in a scene. This difficulty results because the smaller kernels don't contain enough pixels to uniquely locate the corresponding image on the new frame.
  • a variety of voting schemes are available to exploit such combined apertures.
  • a preferred embodiment uses a voting framework that is modeled on a simplified version of the Layered 4-D Tensor voting framework.
  • each potential match is encoded into a 4-D tensor as follows: the tensor is located in the 4-D space given by the point (x, y, v x , v y ) and described by a set of eigenvectors and eigenvalues where each potential match is encoded into a 4-D ball tensor.
  • the ball tensor does not show preference for any particular direction.
  • each token propagates its preferred information to its neighbors through several steps of voting; the voting range is determined by a scale factor controlled by the operator.
  • the vote strength decays with distance and orientation in a way such that smooth surface continuities are encouraged.
  • the vote orientation corresponds to be best possible local surface continuation from voter to recipient.
  • the voting process gives strong support to tokens with similar motion parameters, that is, they lie on the same or on close-by layers (velocity descriptors) while communication among tokens with different motion attributes is inhibited by the layer separation in the 4-D space. Wrong matches appear as isolated points that receive little support.
  • the computational framework must be able to infer local motion information from the available data while taking into account and handling the restrictions caused by the limited aperture of the small kernels.
  • the simplest voting scheme consists of adding the correlation surfaces of neighboring kernels, which is equivalent to using a larger kernel:
  • Some kernels will provide higher-quality tracking data while others provide little or even erroneous data, because the quality of the motion information is related to intensity gradient by the optical flow constrain equation (1) and therefore highly dependent of imagery content assigned to each kernel, A kernel assigned an area of little texture will not be able to discern any motion, while a kernel tracking a prominent feature will provide the most accurate measurements. A kernel whose target goes into occlusion most likely will provide an erroneous output as it attempts to match its template to an image that does not contain the target.
  • the quality of the kernel track can be measured in several ways, for example, the magnitude of the Least Sum of Square Errors can be used to segregate kernels with poor image matches. Alternatively, the number and the slope of the correlation peaks can be analyzed to identify kernels with sufficient optical flow.
  • the probability of finding the kernel in the search area lowers and therefore the weight should be lower.
  • the distance to each of the neighboring kernels in the voting group is also important since the influence of the kernel diminishes with distance, so for larger distances the weight value diminishes rapidly.
  • the resulting correlation surface C uv is the product of the quality-weighted voting function of the surrounding kernels.
  • Each of the values in C uv holds the votes that support a target track to the location [u,v]. It should be noted that Equation (4) is calculated independently for every possible pixel velocity within the search range [u,v], maintaining the layer separation as defined in the layered 4-D voting algorithm. 5) Voting Resolution
  • Each kernel in the image grid collects votes from its neighboring kernels up to a maximum distance defined by the weight function scale factor. After its own vote is added, a search is made for the velocity pair with the most support. Since the votes are derived from the sum of squared errors, higher votes signify more errors and lower vote values are indicative of successful matches; correct velocity pairs receive the votes with lower error values while incorrect ones receive votes with higher errors rates.
  • This voting scheme presents several advantages. For example, regions of low texture may have a higher quality indicator because the probability of finding the reference image somewhere in the search area is high, these regions will cast votes for all velocities equally so it will not affect the voting result. An interesting effect comes for a given kernel, when an attempt is made to find a low texture region on a mixed search area. In this situation, it may not be possible to identify the location of the region, but the search will provide a very strong indication of where the region is not. Its vote will be counted and used to provide a strong rejection for the incorrect velocity pairs.
  • the creation of the Elastic Matrix is composed of three major processes:
  • the first step to creating a Target Cluster is to simply divide the area enclosed by the target designator into many small sub-images; each image is assigned its own tracker. Subsequently, the tracker evaluates the quality of each of the sub-images; areas of low texture and areas limited by the optical flow constrain are eliminated from consideration.
  • the system performs two major tasks: runs the Motion Fields algorithm and performs a Least Square Error tracking on all the remaining Candidate Targets.
  • the quality of the track and its correlation to the expected motion is evaluated; trackers that did not produce a sharp correlation peak or whose track was outside of the expected motion are dismissed.
  • the remaining trackers are grouped by motion similarity and an Elastic Matrix is created by linking each tracker with its closest neighbors.
  • the quality of each of the targets is assessed in an effort to reduce the number of features to track to the set that is most likely to produce an unambiguous location during subsequent video frames.
  • the target quality is assessed by tracking the target on the same video frame from which it was extracted by using a convolution-based tracker.
  • the results of the convolution operation at target locations surrounding the original position are saved into an array called the Correlation Surface; the shape of the surface can be analyzed to determine the quality of the target.
  • We use a fairly easy and quick analysis technique by counting the number of correlation peaks and their slope. If we find more than one peak we look at the statistical distribution of the peaks on the correlation surface. If the peaks are few and tightly clustered, the standard deviation is small and the target is assigned a higher quality than a target with the same number of peaks but widely distributed.
  • an Elastic Matrix Node For every candidate target, an Elastic Matrix Node is created.
  • the node stores information relevant to tracking the individual target feature such as its position, velocity, track quality, number of frames tracked, number of frames of lost track, the voting database, the list of elastic relationships to other nodes, and a link to a Least Squares Error tracker dedicated to tracking the target from frame to frame.
  • an Elastic Matrix Relationship For every pair of candidate targets, an Elastic Matrix Relationship is created; the Relationship keeps track of the data needed to predict the position of either one of targets in case of occlusion where only one of them can be located on the video frame. Each Relationship stores the offset position and speeds, as well as the distance and the weight of each of the trackers based on their track quality indicators. Each of the nodes in the Elastic Matrix keeps a list of the Relationships that links it to other nodes, and the list is kept sorted by relevance (closer higher quality nodes are kept first, followed by further high quality nodes, and finally low quality nodes).
  • the tempo-spatial relations between the trackers in the cluster are established. These relationships are used to create local support groups where the apertures are combined, and to provide an elastic reference frame that trackers use to maintain cohesion.
  • the Elastic Matrix is especially important for trackers that temporarily lost their targets, as they can use it to maintain their orientation and position as the target moves. For example, when the subject being tracked walks behind a partial foreground occlusion such as a column or another person, the trackers following the obscured features will maintain their position in relation to the features still visible. When the obscured features emerge on the opposite side of the obscuration, the corresponding trackers will be positioned correctly to reacquire track and help support the Elastic Matrix during subsequent frames.
  • the Motion Fields results are used to guide the search algorithm. For example, if the Motion Fields indicate that a particular area of the image is moving with certain velocity and direction, the trackers working on that area will bias their search knowing that the target they are looking for is most probably moved in the direction indicated.
  • the Cluster Tracker One of the most powerful features of the Cluster Tracker is its ability to perform a weighted aperture amalgamation of the trackers linked by the Elastic Matrix. During the amalgamation process, trackers with a higher track quality have a higher influence on tracking decisions than trackers that can't find their targets. Since the track qualities (and therefore the weights) are calculated during the initial phase of tracking, the Cluster Tracker automatically ignores features obscured by background or foreground interferences while seamlessly tracking the target using the combined aperture of the higher quality trackers.
  • the Elastic Matrix maintains the low-quality trackers in position as the targets moves by extrapolating their location and velocity from their elastic relationships to the higher quality nodes. This ability allows the Cluster Tracker to maintain track lock even when the target goes through severe occlusion environments. As long as some part of the target is visible, the tracker can extrapolate the position of the rest of the trackers.
  • Each of the trackers of the cluster produces a correlation surface by performing an intensity-based convolution operation of a reference image template and the current video frame.
  • the Vector Distance method can be implemented on hardware efficiently, requiring only integer registers and Arithmetic Logic Units (ALU). Least Square Error Tracking
  • the correlation surface is built it is fairly straightforward to scan it by looking for the minimum value. Because each of the correlation surface values is derived from the number of errors between the reference template and the current video frame, the lowest value on the correlation surface corresponds to the least errors and therefore the best match. Once the correlation peak is found, it is necessary to determine the quality of the tracking operation, and so the operations are repeated as described in the above section describing how to extract a value that can be used when assessing the ‘trustworthiness’ of the tracker.
  • the correlation peak value is used because it is a direct indicator of how closely matched are the reference template and the video frame. Because of the large dynamic range of the correlation peak, and because of the particular interest in and importance of the weight when the values are low, compression of the correlation value is obtained by using a logarithmic operator. In a preferred embodiment, it is desirable to have values between 0 and 1, and so, a negative exponential operation is used to force values of good correlation to be 1 and poor correlation values to asymptotically approach zero.
  • the amalgamation process is used to combine the tracking data of the small cluster trackers in order to increase their effective aperture. After selecting which trackers will participate on the voting process, the amalgamation process collects their votes in the form of their squared sum of errors at each of the possible motion vectors multiplied by a weight factor derived from their track quality. Effectively the trackers linked by the Elastic Matrix add their correlation surfaces, with the highest quality trackers having the most impact on the results.
  • the system is able to cope with highly dynamic target imagery where features change appearance rapidly, sometimes a changing feature does not always match the template best, but if its motion is not corroborated by the other trackers the false peak can be overwritten by the strength of the neighbors' votes.
  • the votes are collected on a second correlation surface called the Voting Surface.
  • This second correlation surface is kept at the Elastic Matrix Node and is used exclusively by the Elastic Matrix to calculate the most probable position of the target feature.
  • the Elastic Matrix Node runs a second tracking algorithm on the Voting Surface; since the Voting Surface has a much higher aperture than that of the individual trackers it is more robust to local obscurations and background interferences.
  • the weighting operation is what enables the tracker to automatically and seamlessly switch tracking references from its own tracker to the combined reference supported by the other trackers in its Elastic Matrix vicinity.
  • each of the velocity pairs (vx, vy) can be viewed as a layer on a 4-Dimensional array of dimensions (x, y, vx, vy).
  • the layered view allows the segregation of trackers of similar motion characteristics because those nodes will have components on similar layers; votes take place on layers, one layer per velocity pair. For example, a tracker with a strong peak at (vx 1 , vy 1 ) places a strong vote on that layer on its neighbor's Voting Surface.
  • the layers are analyzed and groups of nodes with similar motion vectors that reinforce each other quickly dominate while insulated votes that receive little support are dismissed.
  • the velocity pair layer with the most support decides the final tracking results for each of the trackers.
  • the first step is to rank the trackers relative to each other according to their track quality. For this we simply find the maximum and minimum values and assign the relative quality from 0 to 10 according to where in the range a tracker's quality value falls. If all the trackers are of very similar quality we assign all of them the maximum value.
  • the second step is to initialize the nodes of the matrix corresponding to the highest-ranking trackers, so we move all the nodes with a quality rank of 8 or better to their tracker own locations.
  • the last step is to iteratively approximate the node locations to the tracker's using a weighted voting scheme and the known relationships of each of the nodes in the matrix to each other. For each node in the matrix a weighted vote is taken from all its selected neighbors. The vote consists on the predicted location of the node multiplied by the weight calculated during the amalgamation process.
  • Each of the links in the Elastic Matrix stores the position and speed offsets between its two endpoint nodes. The prediction process takes the first node's current position and speed and using the offsets it calculates the second node's position. As the iterative process moves the nodes it converges to an equilibrium point usually in less than four iterations.
  • the resulting node position is a balance of the node's own tracking results and the predicted position from the linked nodes, heavily influenced by all the node's weights.
  • the iterative voting has several effects; it provides support to nodes with poor tracking, and keeps the matrix as a coherent unit by keeping nodes from floating away.
  • the Cluster Tracker was tested in a variety of controlled and uncontrolled environments, including a mall and a skating rink.
  • the controlled tests consisted of a sequence with a static complex background and a single subject walking perpendicularly to the camera; a assortment of synthetic foreground obscurations were superimposed to the video prior to tracking in order to observe the tracker performance under several degrees of occlusion severity.
  • FIG. 6 The performance of the Cluster Tracker when part of the target is obscured by different foreground obstacles is seen in FIG. 6 . It is found that the tracker is able to maintain track even in the presence of very severe line of sight obscurations where only small portions of the target are visible. Similar performance was observed when severe obscurations occur with obstacles of similar coloration and texture as the target as shown in FIGS. 7 and 8 .
  • FIG. 8 the tracking of a pedestrian as he walks behind a couple of severe obscurations is shown.
  • the Cluster Tracker mitigates the problem because is able to ignore the background and it can maintain lock as long as some part of the target remains visible as shown in FIG. 9

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Image Analysis (AREA)
  • Closed-Circuit Television Systems (AREA)
  • Radar Systems Or Details Thereof (AREA)
US11/556,896 2005-11-04 2006-11-06 Tracking Using An Elastic Cluster of Trackers Abandoned US20070133840A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US11/556,896 US20070133840A1 (en) 2005-11-04 2006-11-06 Tracking Using An Elastic Cluster of Trackers

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US73339805P 2005-11-04 2005-11-04
US11/556,896 US20070133840A1 (en) 2005-11-04 2006-11-06 Tracking Using An Elastic Cluster of Trackers

Publications (1)

Publication Number Publication Date
US20070133840A1 true US20070133840A1 (en) 2007-06-14

Family

ID=37806711

Family Applications (1)

Application Number Title Priority Date Filing Date
US11/556,896 Abandoned US20070133840A1 (en) 2005-11-04 2006-11-06 Tracking Using An Elastic Cluster of Trackers

Country Status (5)

Country Link
US (1) US20070133840A1 (fr)
EP (1) EP1949339A2 (fr)
AU (1) AU2006311276A1 (fr)
CA (1) CA2628611A1 (fr)
WO (1) WO2007056711A2 (fr)

Cited By (39)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080002856A1 (en) * 2006-06-14 2008-01-03 Honeywell International Inc. Tracking system with fused motion and object detection
US20100007717A1 (en) * 2008-07-09 2010-01-14 Prime Sense Ltd Integrated processor for 3d mapping
US20100033568A1 (en) * 2008-08-08 2010-02-11 Hon Hai Precision Industry Co., Ltd. Surveillance system and surveillance method thereof
US20100118123A1 (en) * 2007-04-02 2010-05-13 Prime Sense Ltd Depth mapping using projected patterns
US20100201811A1 (en) * 2009-02-12 2010-08-12 Prime Sense Ltd. Depth ranging with moire patterns
US20100225746A1 (en) * 2009-03-05 2010-09-09 Prime Sense Ltd Reference image techniques for three-dimensional sensing
US20100265316A1 (en) * 2009-04-16 2010-10-21 Primesense Ltd. Three-dimensional mapping and imaging
US20100290698A1 (en) * 2007-06-19 2010-11-18 Prime Sense Ltd Distance-Varying Illumination and Imaging Techniques for Depth Mapping
US20110025827A1 (en) * 2009-07-30 2011-02-03 Primesense Ltd. Depth Mapping Based on Pattern Matching and Stereoscopic Information
US20110211044A1 (en) * 2010-03-01 2011-09-01 Primesense Ltd. Non-Uniform Spatial Resource Allocation for Depth Mapping
JP2013027452A (ja) * 2011-07-27 2013-02-07 Hitachi Aloka Medical Ltd 超音波画像処理装置
CN102982555A (zh) * 2012-11-01 2013-03-20 江苏科技大学 基于自适应流形粒子滤波的制导红外小目标跟踪方法
US20130108177A1 (en) * 2011-11-01 2013-05-02 Google Inc. Image matching using motion manifolds
US8699748B2 (en) 2010-10-13 2014-04-15 Industrial Technology Research Institute Tracking system and method for regions of interest and computer program product thereof
US20140233798A1 (en) * 2013-02-21 2014-08-21 Samsung Electronics Co., Ltd. Electronic device and method of operating electronic device
US8830227B2 (en) 2009-12-06 2014-09-09 Primesense Ltd. Depth-based gain control
US20150096352A1 (en) * 2013-10-07 2015-04-09 Google Inc. Smart-home system facilitating insight into detected carbon monoxide levels
US20150110369A1 (en) * 2012-01-10 2015-04-23 Koninklijke Philips N.V. Image processing apparatus
US9030528B2 (en) 2011-04-04 2015-05-12 Apple Inc. Multi-zone imaging sensor and lens array
US20150131859A1 (en) * 2007-02-07 2015-05-14 Samsung Electronics Co., Ltd. Method and apparatus for tracking object, and method and apparatus for calculating object pose information
US20150146915A1 (en) * 2012-12-18 2015-05-28 Intel Corporation Hardware convolution pre-filter to accelerate object detection
US9066087B2 (en) 2010-11-19 2015-06-23 Apple Inc. Depth mapping using time-coded illumination
US9063283B2 (en) 2005-10-11 2015-06-23 Apple Inc. Pattern generation using a diffraction pattern that is a spatial fourier transform of a random pattern
US9066084B2 (en) 2005-10-11 2015-06-23 Apple Inc. Method and system for object reconstruction
US9098931B2 (en) 2010-08-11 2015-08-04 Apple Inc. Scanning projectors and image capture modules for 3D mapping
US9131136B2 (en) 2010-12-06 2015-09-08 Apple Inc. Lens arrays for pattern projection and imaging
US9157790B2 (en) 2012-02-15 2015-10-13 Apple Inc. Integrated optoelectronic modules with transmitter, receiver and beam-combining optics for aligning a beam axis with a collection axis
EP2474163A4 (fr) * 2009-09-01 2016-04-13 Behavioral Recognition Sys Inc Détection d'objet de premier plan dans un système de surveillance vidéo
US9330324B2 (en) 2005-10-11 2016-05-03 Apple Inc. Error compensation in three-dimensional mapping
JP2016157166A (ja) * 2015-02-23 2016-09-01 富士通株式会社 画像処理プログラム、画像処理装置、及び画像処理方法
EP3121791A1 (fr) * 2015-07-24 2017-01-25 Ricoh Company, Ltd. Procédé et système de suivi d'objets
EP3382643A1 (fr) * 2017-03-30 2018-10-03 The Boeing Company Suivi automatique d'objet dans un flux vidéo à l'aide d'apprentissage machine
US20180286054A1 (en) * 2017-03-31 2018-10-04 Canon Kabushiki Kaisha Image processing apparatus, image processing method, and program
US20190244371A1 (en) * 2018-02-04 2019-08-08 Applied Research, LLC Target Detection, Tracking, and Classification in Compressive Measurement Domain
US10417487B2 (en) * 2014-08-26 2019-09-17 Canon Kabushiki Kaisha Pattern recognition apparatus, pattern recognition method, and storage medium
CN110363791A (zh) * 2019-06-28 2019-10-22 南京理工大学 一种融合单目标跟踪结果的在线多目标跟踪算法
US10855918B2 (en) * 2018-05-09 2020-12-01 Canon Kabushiki Kaisha Image processing device, image processing method, image pickup apparatus, and program storage medium that calculates a matching degree between an estimated target of interest and tracked feature points, then selects a feature point to which tracking is continued according to the matching degree
CN115542308A (zh) * 2022-12-05 2022-12-30 德心智能科技(常州)有限公司 基于毫米波雷达的室内人员探测方法、装置、设备及介质
US11727250B2 (en) 2019-09-06 2023-08-15 International Business Machines Corporation Elastic-centroid based clustering

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP2729916A4 (fr) * 2011-07-04 2015-04-08 Lee Vincent Streeter Compensation de mouvement dans une imagerie de distance
CN103810460B (zh) * 2012-11-09 2017-05-03 株式会社理光 对象跟踪方法和装置

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6496592B1 (en) * 1998-07-13 2002-12-17 Oerlikon Contraves Ag Method for tracking moving object by means of specific characteristics

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6496592B1 (en) * 1998-07-13 2002-12-17 Oerlikon Contraves Ag Method for tracking moving object by means of specific characteristics

Cited By (65)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9330324B2 (en) 2005-10-11 2016-05-03 Apple Inc. Error compensation in three-dimensional mapping
US9066084B2 (en) 2005-10-11 2015-06-23 Apple Inc. Method and system for object reconstruction
US9063283B2 (en) 2005-10-11 2015-06-23 Apple Inc. Pattern generation using a diffraction pattern that is a spatial fourier transform of a random pattern
US8467570B2 (en) * 2006-06-14 2013-06-18 Honeywell International Inc. Tracking system with fused motion and object detection
US20080002856A1 (en) * 2006-06-14 2008-01-03 Honeywell International Inc. Tracking system with fused motion and object detection
US10049277B2 (en) * 2007-02-07 2018-08-14 Samsung Electronics Co., Ltd. Method and apparatus for tracking object, and method and apparatus for calculating object pose information
US20150131859A1 (en) * 2007-02-07 2015-05-14 Samsung Electronics Co., Ltd. Method and apparatus for tracking object, and method and apparatus for calculating object pose information
US20100118123A1 (en) * 2007-04-02 2010-05-13 Prime Sense Ltd Depth mapping using projected patterns
US8493496B2 (en) 2007-04-02 2013-07-23 Primesense Ltd. Depth mapping using projected patterns
US8494252B2 (en) 2007-06-19 2013-07-23 Primesense Ltd. Depth mapping using optical elements having non-uniform focal characteristics
US20100290698A1 (en) * 2007-06-19 2010-11-18 Prime Sense Ltd Distance-Varying Illumination and Imaging Techniques for Depth Mapping
US20100007717A1 (en) * 2008-07-09 2010-01-14 Prime Sense Ltd Integrated processor for 3d mapping
US8456517B2 (en) 2008-07-09 2013-06-04 Primesense Ltd. Integrated processor for 3D mapping
US8248469B2 (en) * 2008-08-08 2012-08-21 Hon Hai Precision Industry Co., Ltd. Surveillance system and surveillance method thereof
US20100033568A1 (en) * 2008-08-08 2010-02-11 Hon Hai Precision Industry Co., Ltd. Surveillance system and surveillance method thereof
US8462207B2 (en) 2009-02-12 2013-06-11 Primesense Ltd. Depth ranging with Moiré patterns
US20100201811A1 (en) * 2009-02-12 2010-08-12 Prime Sense Ltd. Depth ranging with moire patterns
US20100225746A1 (en) * 2009-03-05 2010-09-09 Prime Sense Ltd Reference image techniques for three-dimensional sensing
US8786682B2 (en) 2009-03-05 2014-07-22 Primesense Ltd. Reference image techniques for three-dimensional sensing
US20100265316A1 (en) * 2009-04-16 2010-10-21 Primesense Ltd. Three-dimensional mapping and imaging
US8717417B2 (en) 2009-04-16 2014-05-06 Primesense Ltd. Three-dimensional mapping and imaging
US9582889B2 (en) * 2009-07-30 2017-02-28 Apple Inc. Depth mapping based on pattern matching and stereoscopic information
US20110025827A1 (en) * 2009-07-30 2011-02-03 Primesense Ltd. Depth Mapping Based on Pattern Matching and Stereoscopic Information
EP2474163A4 (fr) * 2009-09-01 2016-04-13 Behavioral Recognition Sys Inc Détection d'objet de premier plan dans un système de surveillance vidéo
US8830227B2 (en) 2009-12-06 2014-09-09 Primesense Ltd. Depth-based gain control
US8982182B2 (en) 2010-03-01 2015-03-17 Apple Inc. Non-uniform spatial resource allocation for depth mapping
US20110211044A1 (en) * 2010-03-01 2011-09-01 Primesense Ltd. Non-Uniform Spatial Resource Allocation for Depth Mapping
US9098931B2 (en) 2010-08-11 2015-08-04 Apple Inc. Scanning projectors and image capture modules for 3D mapping
US8699748B2 (en) 2010-10-13 2014-04-15 Industrial Technology Research Institute Tracking system and method for regions of interest and computer program product thereof
US9066087B2 (en) 2010-11-19 2015-06-23 Apple Inc. Depth mapping using time-coded illumination
US9167138B2 (en) 2010-12-06 2015-10-20 Apple Inc. Pattern projection and imaging using lens arrays
US9131136B2 (en) 2010-12-06 2015-09-08 Apple Inc. Lens arrays for pattern projection and imaging
US9030528B2 (en) 2011-04-04 2015-05-12 Apple Inc. Multi-zone imaging sensor and lens array
JP2013027452A (ja) * 2011-07-27 2013-02-07 Hitachi Aloka Medical Ltd 超音波画像処理装置
US9373040B2 (en) * 2011-11-01 2016-06-21 Google Inc. Image matching using motion manifolds
US20130108177A1 (en) * 2011-11-01 2013-05-02 Google Inc. Image matching using motion manifolds
US20150110369A1 (en) * 2012-01-10 2015-04-23 Koninklijke Philips N.V. Image processing apparatus
US9157790B2 (en) 2012-02-15 2015-10-13 Apple Inc. Integrated optoelectronic modules with transmitter, receiver and beam-combining optics for aligning a beam axis with a collection axis
US9651417B2 (en) 2012-02-15 2017-05-16 Apple Inc. Scanning depth engine
CN102982555A (zh) * 2012-11-01 2013-03-20 江苏科技大学 基于自适应流形粒子滤波的制导红外小目标跟踪方法
US9342749B2 (en) * 2012-12-18 2016-05-17 Intel Corporation Hardware convolution pre-filter to accelerate object detection
US20150146915A1 (en) * 2012-12-18 2015-05-28 Intel Corporation Hardware convolution pre-filter to accelerate object detection
US9406143B2 (en) * 2013-02-21 2016-08-02 Samsung Electronics Co., Ltd. Electronic device and method of operating electronic device
US20140233798A1 (en) * 2013-02-21 2014-08-21 Samsung Electronics Co., Ltd. Electronic device and method of operating electronic device
US10991213B2 (en) 2013-10-07 2021-04-27 Google Llc Smart-home device installation guidance
US10546469B2 (en) * 2013-10-07 2020-01-28 Google Llc Smart-home system facilitating insight into detected carbon monoxide levels
US10529195B2 (en) 2013-10-07 2020-01-07 Google Llc Smart-home device installation guidance
US20150096352A1 (en) * 2013-10-07 2015-04-09 Google Inc. Smart-home system facilitating insight into detected carbon monoxide levels
US10049280B2 (en) 2013-10-07 2018-08-14 Google Llc Video guidance for smart-home device installation
US10417487B2 (en) * 2014-08-26 2019-09-17 Canon Kabushiki Kaisha Pattern recognition apparatus, pattern recognition method, and storage medium
JP2016157166A (ja) * 2015-02-23 2016-09-01 富士通株式会社 画像処理プログラム、画像処理装置、及び画像処理方法
CN106651901A (zh) * 2015-07-24 2017-05-10 株式会社理光 对象跟踪方法和设备
EP3121791A1 (fr) * 2015-07-24 2017-01-25 Ricoh Company, Ltd. Procédé et système de suivi d'objets
EP3382643A1 (fr) * 2017-03-30 2018-10-03 The Boeing Company Suivi automatique d'objet dans un flux vidéo à l'aide d'apprentissage machine
KR20180111630A (ko) * 2017-03-30 2018-10-11 더 보잉 컴파니 머신 러닝을 사용하는 비디오 피드 내에서의 자동화된 오브젝트 추적
US11068721B2 (en) 2017-03-30 2021-07-20 The Boeing Company Automated object tracking in a video feed using machine learning
KR102543508B1 (ko) * 2017-03-30 2023-06-13 더 보잉 컴파니 머신 러닝을 사용하는 비디오 피드 내에서의 자동화된 오브젝트 추적
US20180286054A1 (en) * 2017-03-31 2018-10-04 Canon Kabushiki Kaisha Image processing apparatus, image processing method, and program
US10776927B2 (en) * 2017-03-31 2020-09-15 Canon Kabushiki Kaisha Image processing apparatus, image processing method, and program
US10529079B2 (en) * 2018-02-04 2020-01-07 Applied Research, LLC Target detection, tracking, and classification in compressive measurement domain
US20190244371A1 (en) * 2018-02-04 2019-08-08 Applied Research, LLC Target Detection, Tracking, and Classification in Compressive Measurement Domain
US10855918B2 (en) * 2018-05-09 2020-12-01 Canon Kabushiki Kaisha Image processing device, image processing method, image pickup apparatus, and program storage medium that calculates a matching degree between an estimated target of interest and tracked feature points, then selects a feature point to which tracking is continued according to the matching degree
CN110363791A (zh) * 2019-06-28 2019-10-22 南京理工大学 一种融合单目标跟踪结果的在线多目标跟踪算法
US11727250B2 (en) 2019-09-06 2023-08-15 International Business Machines Corporation Elastic-centroid based clustering
CN115542308A (zh) * 2022-12-05 2022-12-30 德心智能科技(常州)有限公司 基于毫米波雷达的室内人员探测方法、装置、设备及介质

Also Published As

Publication number Publication date
WO2007056711A2 (fr) 2007-05-18
CA2628611A1 (fr) 2007-05-18
EP1949339A2 (fr) 2008-07-30
AU2006311276A1 (en) 2007-05-18
WO2007056711A3 (fr) 2007-12-21

Similar Documents

Publication Publication Date Title
US20070133840A1 (en) Tracking Using An Elastic Cluster of Trackers
Bunyak et al. Flux tensor constrained geodesic active contours with sensor fusion for persistent object tracking
US7706571B2 (en) Flexible layer tracking with weak online appearance model
US8599252B2 (en) Moving object detection apparatus and moving object detection method
Cannons A review of visual tracking
US11138742B2 (en) Event-based feature tracking
US20170350956A1 (en) System and method for multi-sensor multi-target 3d fusion using an unbiased measurement space
US10636190B2 (en) Methods and systems for exploiting per-pixel motion conflicts to extract primary and secondary motions in augmented reality systems
Bunyak et al. Geodesic active contour based fusion of visible and infrared video for persistent object tracking
Jiang et al. Multiple pedestrian tracking using colour and motion models
US10042047B2 (en) Doppler-based segmentation and optical flow in radar images
Deng et al. Infrared small target detection based on the self-information map
CN113223045A (zh) 基于动态物体语义分割的视觉与imu传感器融合定位系统
Tawab et al. Efficient multi-feature PSO for fast gray level object-tracking
US20160084953A1 (en) Doppler-based segmentation and optical flow in radar images
Lee et al. Particle filters and occlusion handling for rigid 2D–3D pose tracking
Zhao et al. Robust multiple object tracking in RGB-D camera networks
US20080198237A1 (en) System and method for adaptive pixel segmentation from image sequences
CN117036404A (zh) 一种单目热成像同时定位与建图方法和系统
Liu et al. A joint optical flow and principal component analysis approach for motion detection
CN109271854B (zh) 基于视频处理方法及装置、视频设备及存储介质
Pham et al. Fusion of wifi and visual signals for person tracking
CN113158816B (zh) 面向室外场景物体的视觉里程计二次曲面路标构建方法
Jones et al. Moving target indication and tracking from moving sensors
Rodriguez et al. Automatic recognition of multiple targets with varying velocities using quadratic correlation filters and Kalman filters

Legal Events

Date Code Title Description
AS Assignment

Owner name: CLEAN EARTH TECHNOLOGIES, LLC, MISSOURI

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:CILIA, ANDREW;REEL/FRAME:018903/0209

Effective date: 20070212

STCB Information on status: application discontinuation

Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION

AS Assignment

Owner name: WARPET, LLC, MISSOURI

Free format text: SECURITY AGREEMENT;ASSIGNOR:CET, LLC (F/K/A CLEAN EARTH TECHNOLOGIES, L.L.C.);REEL/FRAME:026962/0498

Effective date: 20110919