CN111986231A - Multi-target tracking method and system - Google Patents
Multi-target tracking method and system Download PDFInfo
- Publication number
- CN111986231A CN111986231A CN202010794120.9A CN202010794120A CN111986231A CN 111986231 A CN111986231 A CN 111986231A CN 202010794120 A CN202010794120 A CN 202010794120A CN 111986231 A CN111986231 A CN 111986231A
- Authority
- CN
- China
- Prior art keywords
- tracking
- detection
- tracker
- track
- result
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
- G06T7/248—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments involving reference images or patches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/103—Static body considered as a whole, e.g. static pedestrian or occupant recognition
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30196—Human being; Person
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a multi-target tracking method and a multi-target tracking system, when the multi-target tracking is realized, firstly, a detection area is arranged in a fisheye image from top to bottom, and multiple targets are detected in the detection area; secondly, matching the obtained detection result with the tracking result of the tracker, and if the matching is successful, updating the tracking track by adopting the detection result; if the matching is unsuccessful, establishing a new tracking track by adopting the detection result, and deleting the track which disappears in the visual field; and finally, processing the obtained tracking track in a pedestrian Re-identification (ReID) mode. Therefore, the embodiment of the invention can realize the accurate tracking of multiple targets based on the top-down fisheye image.
Description
Technical Field
The invention relates to an image processing technology, in particular to a multi-target tracking method and a multi-target tracking system.
Background
In recent years, algorithms based on deep learning have been widely applied and have been developed in a breakthrough manner in the field of image processing. The method comprises the step of achieving target tracking by adopting graphic processing. The process of realizing target tracking is as follows: firstly, detecting a target appearing in an image by using a detection algorithm; then, matching the detected target with the existing track through a matching strategy, creating a new track for the target which is not matched, and deleting the track which disappears in the visual field; and finally, obtaining the track of the detection target. It can be seen that the target tracking is implemented by detecting a target from an image and matching the track, and the picture quality of the image determines the accuracy of detecting the target and the accuracy of matching the track, especially when performing multi-target tracking. The images used by the multi-target tracking method may be acquired in a variety of ways, such as by acquiring top-down fisheye images using a device with a fisheye lens.
The fisheye lens is an extreme wide-angle lens, the visual angle of the fisheye lens is close to or exceeds 180 degrees, and the maximum visual angle can reach 220 degrees, so that the fisheye lens provides possibility for shooting scenes with a large visual angle range at a short distance. The fisheye lens has the advantages of an extremely wide angle, and has limitation, the imaging distortion is severe at the edge of fisheye imaging, and certain difficulty is brought to the processing in the aspect of vision.
The multi-target tracking method has important significance for analyzing the behaviors of the targets. In public places such as museums, exhibition halls or various shops, the target behavior is analyzed, the stay time of the target in a specific area is recorded, and then the target interesting product is analyzed, so that the method has a guiding function on the placement of articles, goods sales promotion and the like in the public places such as the museums, the exhibition halls or various shops. Products interested by the target are analyzed according to the statistics of the stay time of the target in the designated area, and a marketing strategy more matched with a specific target is formulated, so that the marketing success can be greatly improved; the target density degree of a designated area is counted through track tracking, the visiting route of the target is intelligently guided, and the satisfaction degree of the target is improved; by complete track tracking, the behavior of the target is analyzed, so that lawless persons can hide everywhere, and the safety sense is greatly improved.
At present, a multi-target tracking method is mostly based on detection, a detection algorithm is used for detecting a target in each frame of image, and then a detected result is associated with a predicted result so as to obtain a complete track of the target. Wherein for a new target, a new trajectory is generated; and for the track which disappears in the image visual field range, terminating the tracking and deleting the track. The difficulty of the multi-target tracking method is that frequent shielding among targets may cause frequent switching of the tracked targets; aiming at performing multi-target tracking by adopting a fisheye image from top to bottom, the similarity between targets is large, and the feature matching of the targets is difficult; meanwhile, due to the problem of the visual angle of the target detection in the fisheye image from top to bottom, the detected target is seriously shielded, and the recall rate is insufficient.
Disclosure of Invention
In view of this, the embodiment of the present invention provides a multi-target tracking method, which can implement accurate multi-target tracking based on a top-down fisheye image.
The embodiment of the invention also provides a multi-target tracking system, which can realize the accurate tracking of multiple targets based on the top-down fisheye image.
The embodiment of the invention provides a multi-target tracking method, which comprises the following steps:
acquiring a top-down fisheye image, setting a detection area from the top-down fisheye image, and detecting multiple targets in the detection area;
matching the obtained detection result with the tracking result tracked by the tracker, judging whether the matching is successful, and if so, updating the tracking track by adopting the detection result of detecting multiple targets; if not, establishing a new tracking track by adopting a detection result of detecting multiple targets, and deleting the track disappeared in the visual field;
and correcting the obtained tracking track in a pedestrian re-identification ReID mode.
Preferably, the set detection area is larger than the set region of interest; the detecting multiple targets within the detection area includes:
and detecting multiple targets in the detection area by adopting the set target detection algorithm, and detecting the multiple targets by adopting the detection inclined rectangular frame when detecting the multiple targets in the detection area.
Preferably, before the matching the obtained detection result with the tracking result tracked by the tracker, the method further includes:
judging whether the detection result is larger than a set detection threshold value or not, and if so, executing the step of matching the obtained detection result with a tracking result tracked by a tracker; and if not, deleting the detection result.
Preferably, the matching the obtained detection result with the tracking result tracked by the tracker includes:
and calculating the squared Mahalanobis distance between the tracking result tracked by the tracker and the position of the detection result and the cosine distance between the target appearance characteristic tracked by the tracker and the target appearance characteristic of the detection result, and determining whether the target appearance characteristic is matched or not based on the calculated distance value.
Preferably, the modifying the obtained tracking track by the ReID method includes:
and judging the relevant state of the tracking track, and if the tracking track of the previous frame is in the shielding state and the tracking track of the current frame is in the non-shielding state, correcting the tracking track by adopting a ReiD mode.
The embodiment of the invention also provides a multi-target tracking system, which comprises a detection module, a matching module and a ReID module, wherein,
the detection module is used for acquiring a top-down fisheye image, setting a detection area from the top-down fisheye image, and detecting multiple targets in the detection area;
the matching module is used for matching the obtained detection result with a tracking result tracked by the tracker, judging whether the matching is successful, and if so, updating the tracking track by adopting the detection result of detecting multiple targets; if not, establishing a new tracking track by adopting a detection result of detecting multiple targets, and deleting the track disappeared in the visual field;
and the ReiD module is used for correcting the obtained tracking track in a ReiD mode.
Preferably, the detection module is further configured to set a detection area larger than the set region of interest; the detecting multiple targets within the detection area includes: and detecting multiple targets in the detection area by adopting the set target detection algorithm, and detecting the multiple targets by adopting the detection inclined rectangular frame when detecting the multiple targets in the detection area.
Preferably, the matching module is further configured to determine whether the detection result is greater than a set detection threshold, and if so, perform a step of matching the obtained detection result with a tracking result tracked by the tracker; if not, deleting the detection result.
Preferably, the matching module is further configured to calculate a squared mahalanobis distance between a tracking result tracked by the tracker and a position of the detection result, and a cosine distance between an appearance feature of the target tracked by the tracker and an appearance feature of the target of the detection result, and determine whether to match based on the calculated distance value.
Preferably, the ReID module is further configured to process the obtained tracking track in a ReID manner, including: and judging the relevant state of the tracking track, and if the tracking track of the previous frame is in the shielding state and the tracking track of the current frame is in the non-shielding state, correcting the tracking track by adopting a ReiD mode.
As can be seen from the above, when multi-target tracking is implemented in the embodiment of the present invention, first, a detection area is set in a top-down fisheye image, and multiple targets are detected in the detection area; secondly, matching the obtained detection result with the tracking result of the tracker, and if the matching is successful, updating the tracking track by adopting the detection result; if the matching is unsuccessful, establishing a new tracking track by adopting the detection result, and deleting the track which disappears in the visual field; and finally, processing the obtained tracking track in a pedestrian Re-identification (ReID) mode. Therefore, the embodiment of the invention can realize the accurate tracking of multiple targets based on the top-down fisheye image.
Drawings
FIG. 1 is a flow chart of a multi-target tracking method provided by an embodiment of the invention;
FIG. 2 is a schematic structural diagram of a multi-target tracking system according to an embodiment of the present invention;
FIG. 3 is a flowchart illustrating a specific example of a multi-target tracking method according to an embodiment of the present invention;
fig. 4 is a schematic diagram of a top-down fisheye image according to an embodiment of the invention;
fig. 5 is a schematic diagram of the obtained tracking track corrected by the ReID method according to the embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is further described in detail below with reference to the accompanying drawings and examples.
In order to realize accurate multi-target tracking based on the top-down fisheye image, when the multi-target tracking is realized, firstly, a detection area is arranged in the top-down fisheye image, and multiple targets are detected in the detection area; secondly, matching the obtained detection result with the tracking result of the tracker, and if the matching is successful, updating the tracking track by adopting the detection result; if the matching is unsuccessful, a new tracking track is established by adopting the detection result, and the track which disappears in the visual field is deleted.
The embodiment of the invention can solve the problem of insufficient recall rate of the edge of the detection area by adjusting the set detection area; the problem of shielding between targets is solved by detecting the inclined frame when the targets are detected; the problem of frequent exchange of target identifications in the tracking process is solved by a ReID mode. Therefore, accurate tracking of multiple targets can be realized.
Fig. 1 is a flowchart of a multi-target tracking method according to an embodiment of the present invention, which includes the following specific steps:
101, acquiring a top-down fisheye image, setting a detection area from the top-down fisheye image, and detecting multiple targets in the detection area;
102, matching the obtained detection result with a tracking result tracked by a tracker, judging whether the matching is successful, and if so, executing a step 103; if not, go to step 104;
step 103, updating a tracking track by adopting a detection result of detecting multiple targets;
and 105, correcting the obtained tracking track in a ReID mode.
In the method, the set detection area is larger than the set region of interest; the detecting multiple targets within the detection area includes:
and detecting multiple targets in the detection area by adopting the set target detection algorithm, and detecting the multiple targets by adopting the detection inclined rectangular frame when detecting the multiple targets in the detection area.
In the method, before performing step 102, the method further includes: judging whether the detection result is larger than a set detection threshold value, if so, executing a step 102; if not, deleting the detection result. Here, the detection results at the edges of the detection area are filtered by judging whether the center of the target position in the detection results is within the region of interest.
In the method, step 102 mainly filters out the detection result with a large overlap degree by calculating the overlap degree between the detection result and the tracking result of the tracker. Here, the tracker predicts the position of the tracker of the next frame by using a kalman filter algorithm, and continuously updates the tracking result.
In the method, the matching in step 102 is: and calculating the squared Mahalanobis distance between the tracking result tracked by the tracker and the position of the detection result and the cosine distance between the target appearance characteristic tracked by the tracker and the target appearance characteristic of the detection result, and determining whether the target appearance characteristic is matched or not based on the calculated distance value.
In the method, the step 105 of processing the obtained tracking track in a ReID manner includes: and judging the relevant state of the tracking track, and if the tracking track of the previous frame is in the shielding state and the tracking track of the current frame is in the non-shielding state, correcting the tracking track by adopting a ReiD mode.
The method described in fig. 1 is embodied as follows.
The first step is as follows: the method comprises the steps of obtaining a fisheye image from top to bottom, setting a detection area in the fisheye image, detecting a plurality of targets moving in the detection area by using a set target detection algorithm, and obtaining the position coordinates of each target.
When the detection area is set, due to the consideration of the randomness of the target position and the particularity of the fish-eye view angle, at the edge of the original region of interest, only part of information of the target may appear in the detection area, and meanwhile, the mutual shielding of the targets is serious. In order to solve the problem, the original region of interest is expanded, and the expanded region is used as a detection region, so that only half of objects appear at the original edge of interest and can be contained in all the detected regions, and the recall rate of the target at the edge of the region of interest can be improved; meanwhile, a lightweight detection model based on a target detection algorithm such as yolov3 is modified, a horizontal detection rectangular frame is modified into a slant detection rectangular frame, and the detection rate is greatly improved.
The second step is as follows: and filtering the detection result with a small threshold value by setting a detection threshold value, and deleting the edge detection result by judging whether the center of the target position is in the region of interest.
The third step: and matching the detection result with the tracking result tracked by the tracker.
And calculating the square Mahalanobis distance between the position of the current target and the position of the detection result and the cosine distance between the appearance characteristic of the current target and the appearance characteristic of the detection result. Here, the current target refers to a tracking result tracked by the tracker. Mahalanobis distance, proposed by the indian statistician mahalanobis, represents the covariance distance of the data, with u ═ for one mean (u ═ for u)1,u2,…,up)TThe covariance matrix is ∑ with the p-dimensional vector x ═ x (x)1,x2,…,xp)TP denotes that its dimension is p-dimension, u1,u2,…,upRepresenting the mean, x, over each dimension1,x2,…,xpThe value of the vector in each dimension is represented, T represents the vector transposition, and the calculation formula of the Mahalanobis distance is as follows:
the mahalanobis distance is not affected by dimension, the mahalanobis distance between two points is independent of the unit of measurement of the raw data, and the two are calculated from the normalized data and the centralized data (i.e. the difference between the raw data and the mean value)The mahalanobis distance between points is the same, which can eliminate correlation interference between variables. Cosine distance uses the cosine value of the included angle of two vectors as the size for measuring the difference between two individuals, and compared with Euclidean distance, the cosine distance pays more attention to the difference of the two vectors in the direction. Let the two p-dimensional vectors be a ═ x, respectively1,x2,…,xp)Ta＝(x1+x2+…+xp)T，b＝(y1,y2,…,yp)Tb＝(y1+y2+…+yp)TWherein x is1,x2,…,xpRepresents the value of the a vector in each dimension, y1,y2,…,ypThe value of the b vector in each dimension is represented, T represents the transposition of the vector, and theta represents the included angle between the two vectors. x is the number of1x1+x2+…+xpThe formula for calculating the cosine distances of vectors a, b is as follows:
and combining the squared mahalanobis distance of the position with the cosine distance of the appearance characteristic to serve as a difference measure of the detection result and the tracking result, wherein due to the existence of a plurality of detection results and a plurality of tracking results, the optimal matching between the detection result and the tracking result needs to be searched next. And (3) matching the detection result with the tracking result, and giving a loss metric according to the difference degree of the detection result and the tracking result, wherein the larger the difference is, the larger the loss of the matching of the detection result and the tracking result is, and the smaller the difference is, the smaller the loss of the matching of the detection result and the tracking result is. The embodiment of the invention uses the Hungarian function to calculate the optimal assignment between the tracking result and the detection result.
The embodiment of the invention allows the disappeared tracking result to continuously exist, so that when the Kalman filtering is used for predicting the tracking result of the next frame tracker, the longer the disappearance time is, the worse the result accuracy is, and a staged matching strategy is adopted in the matching process. The tracker in the working state (secured) at the present stage is preferentially matched, and then matching is performed from small to large according to the number of vanishing frames, wherein the secured indicates that the tracker is in the working state, which is described later.
And obtaining three groups of results after the matching results, if the matching between the tracking results and the detection results is successful, storing the tracking results and the detection results in set matches, storing unmatched detection results in set unmachined detections, and storing unmatched tracking results in unmachined tracks. And for the result in the matches, updating the track (track) by using the detection result, updating the related attribute value of the track, if the tracker is in a disappearing state, setting the state of the tracker to be confirmed, and if the tracker is in an unconfirmed state, judging whether the state of the tracker is set to be confirmed or not according to the condition. And initializing a new tracker for the detection result in the unadapted detections, wherein the state of the initial tracker is an unconfirmed state, and the state of the initial tracker is set to be a disappeared state for the tracking result in the unadapted tracks.
And thirdly, due to the frequent shielding problem in the multi-target tracking process, the tracked target identification can be frequently changed, and the changes can influence the effect of the multi-target tracking algorithm. Therefore, after the matching is finished, the wrong target identification exchange in the tracking process is corrected in a ReId mode.
In the ReID process of the embodiment of the invention, the ReID mode is not used for all tracking sequences, but the ReID mode is adopted to correct the track of the current frame if the tracking of the previous stage is in the shielding state and the tracking of the current frame is in the non-shielding state by judging the related state of the tracking sequences.
Fig. 2 is a schematic structural diagram of a multi-target tracking system provided in an embodiment of the present invention, including: a detection module, a matching module and a ReID module, wherein,
the detection module is used for acquiring a top-down fisheye image, setting a detection area from the top-down fisheye image, and detecting multiple targets in the detection area;
the matching module is used for matching the obtained detection result with a tracking result tracked by the tracker, judging whether the matching is successful, and if so, updating the tracking track by adopting the detection result of detecting multiple targets; if not, establishing a new tracking track by adopting a detection result of detecting multiple targets, and deleting the track disappeared in the visual field;
and the ReiD module is used for correcting the obtained tracking track in a ReiD mode.
In the system, the detection module is also used for setting the detection area to be larger than the set region of interest; the detecting multiple targets within the detection area includes: and detecting multiple targets in the detection area by adopting the set target detection algorithm, and detecting the multiple targets by adopting the detection inclined rectangular frame when detecting the multiple targets in the detection area.
In the system, the matching module is further configured to determine whether the detection result is greater than a set detection threshold, and if so, perform a step of matching the obtained detection result with a tracking result tracked by a tracker; if not, deleting the detection result.
In the system, the matching module is further configured to calculate a squared mahalanobis distance between a tracking result tracked by the tracker and a position of the detection result, and a cosine distance between an appearance feature of the target tracked by the tracker and an appearance feature of the target of the detection result, and determine whether to match based on the calculated distance value.
In this system, the ReID module is further configured to process the obtained tracking track in a ReID manner, including: and judging the relevant state of the tracking track, and if the tracking track of the previous frame is in the shielding state and the tracking track of the current frame is in the non-shielding state, correcting the tracking track by adopting a ReiD mode.
The following describes embodiments of the present invention in detail with reference to a specific example.
Before describing the process provided by the embodiment of the present invention in detail, the relevant identifier is defined.
Four states of the Track (Track):
tentative ═ 1: in the experimental state of the tracker, there are some problems such as error detection, so that some trackers are initialized with the result of error detection, and when hits > is n _ init, the track state is changed from Tentative to Confirmed.
Secured 2: the confirmation state and the track state of the tracker can be changed from the Tentative or miss state to the Confirmed state.
Deleted-3: a delete state of the tracker, the delete state indicating that the tracker is to be deleted. And if the tracker in the Confirmed state does not match the detection result, setting the state of the tracker in the Tentative state as Deleted.
Miss ═ 4: and the tracker in the Confirmed state is not matched with the detection result, and the tracker state is set to be Miss.
Track type correlation attribute:
track _ id: the identification number of the tracker, the unique identification of the tracker, the same track _ id representing the same tracker.
n _ init: tracker changed from Tertive to Confirmed
hits: number of frames tracker was updated
age: number of frames tracker from first appearance to current
time _ sequence _ update: last time tracker is updated to current frame number
max _ age: the longest number of frames that the tracker disappears, if greater, is deleted.
And (5) state: status of tracker
feature: appearance characteristics of tracker
reid _ feature: reid feature of tracker
Initialization:
loading a detection model, reid model and initializing a multi-target tracker (tracker).
Fig. 3 shows a specific process of the entire solution, and fig. 3 is a flowchart of a specific example of the multi-target tracking method provided in the embodiment of the present invention, where the specific process includes:
1) and acquiring a top-down fisheye image, and setting a detection area from the fisheye image, wherein the detection area is slightly larger than the region of interest in the prior art. As shown in fig. 4, fig. 4 is a schematic diagram of a top-down fisheye image according to an embodiment of the invention.
In the step, in consideration of the particularity of the fisheye image, only half of the targets at the boundary of the region of interest often appear in the region of interest.
2) And setting a detection model to detect the target in the detection area.
In this step, the horizontal rectangular frame of detection of prior art is revised to detecting the slope rectangular frame, very big improvement the relevance ratio.
Further, filtering a part of detection results through a threshold, then filtering targets at edges by judging whether the position center is in the detection area, and finally filtering out detection results with large overlapping degree by calculating the overlapping degree between the detection results and the prediction results of the tracker. The tracker uses a kalman filtering algorithm to predict the position of the tracker for the next frame.
3) And matching the detection result with the tracking result tracked by the tracker.
And calculating the square Mahalanobis distance between the position of the current target and the position of the detection result and the cosine distance between the appearance characteristic of the current target and the appearance characteristic of the detection result. Here, the current target refers to a tracking result tracked by the tracker. Mahalanobis distance, proposed by the indian statistician mahalanobis, represents the covariance distance of the data, with u ═ for one mean (u ═ for u)1,u2,…,up)TThe covariance matrix is ∑ with the p-dimensional vector x ═ x (x)1,x2,…,xp)TP denotes that its dimension is p-dimension, u1,u2,…,upRepresenting the mean, x, over each dimension1,x2,…,xpThe value of the vector in each dimension is represented, T represents the vector transposition, and the calculation formula of the Mahalanobis distance is as followsShown in the figure:
the mahalanobis distance is not influenced by the dimension, the mahalanobis distance between two points is independent of the measurement unit of the original data, the mahalanobis distance between two points calculated by the normalized data and the centralized data (i.e. the difference between the original data and the mean value) is the same, and the mahalanobis distance can exclude the correlation interference between variables. Cosine distance uses the cosine value of the included angle of two vectors as the size for measuring the difference between two individuals, and compared with Euclidean distance, the cosine distance pays more attention to the difference of the two vectors in the direction. Let the two p-dimensional vectors be a ═ x, respectively1,x2,…,xp)Ta＝(x1+x2+…+xp)T，b＝(y1,y2,…,yp)Tb＝(y1+y2+…+yp)TWherein x is1,x2,…,xpRepresents the value of the a vector in each dimension, y1,y2,…,ypThe value of the b vector in each dimension is represented, T represents the transposition of the vector, and theta represents the included angle between the two vectors. x is the number of1x1+x2+…+xpThe formula for calculating the cosine distances of vectors a, b is as follows:
and combining the squared mahalanobis distance of the position with the cosine distance of the appearance characteristic to serve as a difference measure of the detection result and the tracking result, wherein due to the existence of a plurality of detection results and a plurality of tracking results, the optimal matching between the detection result and the tracking result needs to be searched next. And (3) matching the detection result with the tracking result, and giving a loss metric according to the difference degree of the detection result and the tracking result, wherein the larger the difference is, the larger the loss of the matching of the detection result and the tracking result is, and the smaller the difference is, the smaller the loss of the matching of the detection result and the tracking result is. The embodiment of the invention uses the Hungarian function to calculate the optimal assignment between the tracking result and the detection result.
The embodiment of the invention allows the tracker after disappearance to continuously exist, so that when the Kalman filtering is used for predicting the result of the tracker of the next frame, the longer the disappearance time is, the worse the result accuracy is, and a staged matching strategy is adopted in the matching process of the embodiment of the invention. Preferentially matching the tracker in a confirmed state at the present stage, and then matching according to the number of vanishing frames from small to large.
4) And after matching is finished, obtaining three groups of results, if the tracking result is successfully matched with the detection result, storing the tracking result and the detection result in matches, storing unmatched detection results in unmatched detections, and keeping unmatched tracking results in unmatched tracks. And for the result in the matches, updating the track by using the result of detection, simultaneously updating the related attribute value of the track, if the tracker is in the miss state, setting the state of the tracker to be confirmed, and if the tracker state is in the transient state and the hits value is greater than n _ init, setting the state of the tracker to be confirmed. For the result in unmachined _ detections, a new tracker is initialized, the state of the initial tracker is tentative, and for the result in unmachined _ tracks, it is set to miss state.
5) And correcting the obtained tracking track by a ReID mode.
The tracker has two states of occlusion and non-occlusion, and because the embodiment of the invention adopts the ReID mode, the characteristics of the ReID mode must be clean and relatively correct, so that not every frame of characteristics of the tracker can be stored in the ReID _ feature of the ReID mode of the tracker, and only when the state of the tracker is in the non-occlusion state, the characteristics of the tracker can be stored in the ReID _ feature of the ReID mode.
The tracker is in an unshielded state initially, then the overlapping degree of the tracker and other trackers is judged, and when the overlapping degree exceeds a certain threshold value, the state of the tracker is set to be an unshielded state and the tracking track in the shaded state.
Firstly, the ReID of the initialization phase is performed, when the state of the tracker changes from latent to confirmed, the tracker is confirmed to be valid, if the tracker disappears before, that is, the tracker is in the same sequence of the tracking track sequence in the miss state, the discontinuity between the sequences is caused, and therefore, the ReID mode processing needs to be performed on the tracker in the initial phase and the tracker in the miss state. Considering that the trackers in the initial stage and the trackers in the miss state are multiple, the results of the ReiD mode processing are assigned by using the Hungarian algorithm. The tracker, after successful assignment, modifies its track _ id and sets some attributes of the tracker.
In addition, the ReID mode processing is also required for a tracker that changes from occlusion to non-occlusion. After the ReID mode processing result is processed, the identifier of the tracker needs to be processed correspondingly, because the state of the tracker is confirmed only after the tracker continuously appears for several frames, but at this time, the tracker in the tentative state also has an id, if the identifier of the tracker, the occurrence frequency of which does not reach the state change, is not recovered, the identifier of the tracker jumps, and if a plurality of trackers are in the miss state, the identifier of the tracker may be recovered, so that after the ReID mode processing is finished, all newly generated identifiers of the trackers need to be checked, and the problems of the identifier of the tracker and the identifier of the jump tracker are processed. As shown in fig. 5, fig. 5 is a schematic diagram of a tracking track obtained by performing a correction process in a ReID manner according to an embodiment of the present invention.
6) After the steps are completed, the information of each tracker is stored in a database, the target flow, the detention condition and the track of the detection area are counted, and the result is displayed. And then repeating the five steps to continue the execution.
The embodiment of the invention can realize multi-target tracking under the fisheye visual angle. The embodiment of the invention integrates the ReiD mode into the multi-target tracking process, thereby greatly improving the recovery effect of the shielded target. According to the embodiment of the invention, the detection rate of the edge of the region of interest is effectively improved by enlarging the detection region and then filtering the detection result. According to the embodiment of the invention, the output layer of the detection model is modified, and the inclined frame of the target under the fisheye lens is detected by improving the output layers of the regression rectangles bbox and anchor into the regression inclined bbox and anchor, so that the shielding condition is relieved, and the detection rate is improved.
An object of an embodiment of the invention may be a pedestrian.
Therefore, the embodiment of the invention can completely track the track of the pedestrian under the fish-eye lens without shielding; for pedestrians with slight shielding, such as mutual shielding of the pedestrians and shielding between the pedestrians and other objects, the tracking track can be restored immediately after shielding, and the tracking track mark cannot be switched or lost due to shielding; for the situation that the tracking track identification is switched due to various situations during the occlusion period when the occlusion is serious, due to the introduction of the ReiD mode, the embodiment of the invention can recover the tracking track after the occlusion is not generated.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like made within the spirit and principle of the present invention should be included in the scope of the present invention.
Claims (10)
1. A multi-target tracking method is characterized by comprising the following steps:
acquiring a top-down fisheye image, setting a detection area from the top-down fisheye image, and detecting multiple targets in the detection area;
matching the obtained detection result with the tracking result tracked by the tracker, judging whether the matching is successful, and if so, updating the tracking track by adopting the detection result of detecting multiple targets; if not, establishing a new tracking track by adopting a detection result of detecting multiple targets, and deleting the track disappeared in the visual field;
and correcting the obtained tracking track in a pedestrian re-identification ReID mode.
2. The method of claim 1, wherein the set detection area is larger than the set region of interest; the detecting multiple targets within the detection area includes:
and detecting multiple targets in the detection area by adopting the set target detection algorithm, and detecting the multiple targets by adopting the detection inclined rectangular frame when detecting the multiple targets in the detection area.
3. The method of claim 1, wherein before matching the obtained detection result with a tracking result tracked by a tracker, further comprising:
judging whether the detection result is larger than a set detection threshold value or not, and if so, executing the step of matching the obtained detection result with a tracking result tracked by a tracker; and if not, deleting the detection result.
4. The method of claim 1, wherein matching the obtained detection result with a tracking result tracked by a tracker comprises:
and calculating the squared Mahalanobis distance between the tracking result tracked by the tracker and the position of the detection result and the cosine distance between the target appearance characteristic tracked by the tracker and the target appearance characteristic of the detection result, and determining whether the target appearance characteristic is matched or not based on the calculated distance value.
5. The method according to claim 1, wherein the modifying the obtained tracking track by ReID comprises:
and judging the relevant state of the tracking track, and if the tracking track of the previous frame is in the shielding state and the tracking track of the current frame is in the non-shielding state, correcting the tracking track by adopting a ReiD mode.
6. A multi-target tracking system, comprising a detection module, a matching module and a ReID module, wherein,
the detection module is used for acquiring a top-down fisheye image, setting a detection area from the top-down fisheye image, and detecting multiple targets in the detection area;
the matching module is used for matching the obtained detection result with a tracking result tracked by the tracker, judging whether the matching is successful, and if so, updating the tracking track by adopting the detection result of detecting multiple targets; if not, establishing a new tracking track by adopting a detection result of detecting multiple targets, and deleting the track disappeared in the visual field;
and the ReiD module is used for correcting the obtained tracking track in a ReiD mode.
7. The system of claim 6, wherein the detection module is further configured to set a detection area larger than a set region of interest; the detecting multiple targets within the detection area includes: and detecting multiple targets in the detection area by adopting the set target detection algorithm, and detecting the multiple targets by adopting the detection inclined rectangular frame when detecting the multiple targets in the detection area.
8. The system of claim 6, wherein the matching module is further configured to determine whether the detection result is greater than a set detection threshold, and if so, perform the step of matching the obtained detection result with the tracking result tracked by the tracker; if not, deleting the detection result.
9. The system of claim 6, wherein the matching module is further configured to calculate a squared mahalanobis distance between the tracking result tracked by the tracker and the position of the detection result, and a cosine distance between the target appearance feature tracked by the tracker and the target appearance feature of the detection result, and determine whether to match based on the calculated distance values.
10. The system of claim 6, wherein the ReID module further configured to process the obtained trace track by ReID comprises: and judging the relevant state of the tracking track, and if the tracking track of the previous frame is in the shielding state and the tracking track of the current frame is in the non-shielding state, correcting the tracking track by adopting a ReiD mode.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010794120.9A CN111986231A (en) | 2020-08-10 | 2020-08-10 | Multi-target tracking method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010794120.9A CN111986231A (en) | 2020-08-10 | 2020-08-10 | Multi-target tracking method and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111986231A true CN111986231A (en) | 2020-11-24 |
Family
ID=73444633
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010794120.9A Pending CN111986231A (en) | 2020-08-10 | 2020-08-10 | Multi-target tracking method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111986231A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112561963A (en) * | 2020-12-18 | 2021-03-26 | 北京百度网讯科技有限公司 | Target tracking method and device, road side equipment and storage medium |
| CN112686921A (en) * | 2021-01-08 | 2021-04-20 | 西安羚控电子科技有限公司 | Multi-interference unmanned aerial vehicle detection tracking method based on track characteristics |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101860729A (en) * | 2010-04-16 | 2010-10-13 | 天津理工大学 | Target tracking method for omnidirectional vision |
| CN106530239A (en) * | 2016-09-24 | 2017-03-22 | 上海大学 | Large-visual field bionic fish eye-based small unmanned aerial rotorcraft moving target low-altitude tracking method |
| CN110555377A (en) * | 2019-07-27 | 2019-12-10 | 华南理工大学 | pedestrian detection and tracking method based on fisheye camera overlook shooting |
| CN111144247A (en) * | 2019-12-16 | 2020-05-12 | 浙江大学 | Escalator passenger reverse-running detection method based on deep learning |
| CN111784746A (en) * | 2020-08-10 | 2020-10-16 | 上海高重信息科技有限公司 | Multi-target pedestrian tracking method and device under fisheye lens and computer system |
-
2020
- 2020-08-10 CN CN202010794120.9A patent/CN111986231A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101860729A (en) * | 2010-04-16 | 2010-10-13 | 天津理工大学 | Target tracking method for omnidirectional vision |
| CN106530239A (en) * | 2016-09-24 | 2017-03-22 | 上海大学 | Large-visual field bionic fish eye-based small unmanned aerial rotorcraft moving target low-altitude tracking method |
| CN110555377A (en) * | 2019-07-27 | 2019-12-10 | 华南理工大学 | pedestrian detection and tracking method based on fisheye camera overlook shooting |
| CN111144247A (en) * | 2019-12-16 | 2020-05-12 | 浙江大学 | Escalator passenger reverse-running detection method based on deep learning |
| CN111784746A (en) * | 2020-08-10 | 2020-10-16 | 上海高重信息科技有限公司 | Multi-target pedestrian tracking method and device under fisheye lens and computer system |
Non-Patent Citations (3)
| Title |
|---|
| GUAN’AN WANG等: "High-Order Information Matters: Learning Relation and Topology for Occluded Person Re-Identification", 《ARXIV:2003.08177V4》, pages 1 - 10 * |
| NICOLAI WOJKE等: "SIMPLE ONLINE AND REALTIME TRACKING WITH A DEEP ASSOCIATION METRIC", 《ARXIV:1703.07402V1》, pages 1 - 5 * |
| 张笑谋: "基于深度学习的网球比赛实时视频语义分析关键技术的研究与实现", 《中国优秀硕士学位论文全文数据库 信息科技辑》, no. 08, pages 138 - 872 * |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112561963A (en) * | 2020-12-18 | 2021-03-26 | 北京百度网讯科技有限公司 | Target tracking method and device, road side equipment and storage medium |
| CN112686921A (en) * | 2021-01-08 | 2021-04-20 | 西安羚控电子科技有限公司 | Multi-interference unmanned aerial vehicle detection tracking method based on track characteristics |
| CN112686921B (en) * | 2021-01-08 | 2023-12-01 | 西安羚控电子科技有限公司 | Multi-interference unmanned aerial vehicle detection tracking method based on track characteristics |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP6944598B2 (en) | Target tracking method and device, storage medium | |
| JP6759411B2 (en) | Object tracking method and equipment | |
| JP4971191B2 (en) | Identification of spurious areas in video frames | |
| CN101142593B (en) | Method of tracking objects in a video sequence | |
| EP1859411B1 (en) | Tracking objects in a video sequence | |
| EP1844443B1 (en) | Classifying an object in a video frame | |
| US7620207B2 (en) | Linking tracked objects that undergo temporary occlusion | |
| CN112883819A (en) | Multi-target tracking method, device, system and computer readable storage medium | |
| Di Lascio et al. | A real time algorithm for people tracking using contextual reasoning | |
| CN110288627B (en) | Online multi-target tracking method based on deep learning and data association | |
| CN109035295B (en) | Multi-target tracking method, device, computer equipment and storage medium | |
| CN106934817B (en) | Multi-attribute-based multi-target tracking method and device | |
| CN112836640A (en) | Single-camera multi-target pedestrian tracking method | |
| KR101139930B1 (en) | Apparatus and method for object recognition based on part-template matching | |
| CN105374049B (en) | Multi-corner point tracking method and device based on sparse optical flow method | |
| CN111626194A (en) | Pedestrian multi-target tracking method using depth correlation measurement | |
| CN111986231A (en) | Multi-target tracking method and system | |
| CN111639570B (en) | Online multi-target tracking method based on motion model and single-target clue | |
| CN115063441A (en) | Multi-target tracking method and system for reducing jump and related equipment | |
| CN114926859A (en) | Pedestrian multi-target tracking method in dense scene combined with head tracking | |
| CN114820765A (en) | Image recognition method and device, electronic equipment and computer readable storage medium | |
| CN111815682B (en) | Multi-target tracking method based on multi-track fusion | |
| US20080198237A1 (en) | System and method for adaptive pixel segmentation from image sequences | |
| US7773771B2 (en) | Video data tracker | |
| KR101595334B1 (en) | Method and apparatus for movement trajectory tracking of moving object on animal farm |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |