CN110675432A - Multi-dimensional feature fusion-based video multi-target tracking method - Google Patents

Multi-dimensional feature fusion-based video multi-target tracking method Download PDF

Info

Publication number
CN110675432A
CN110675432A CN201910964726.XA CN201910964726A CN110675432A CN 110675432 A CN110675432 A CN 110675432A CN 201910964726 A CN201910964726 A CN 201910964726A CN 110675432 A CN110675432 A CN 110675432A
Authority
CN
China
Prior art keywords
track
detection target
target
historical
calculating
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201910964726.XA
Other languages
Chinese (zh)
Other versions
CN110675432B (en
Inventor
满庆奎
徐晓刚
李冠华
管慧艳
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hangzhou Yunqi Smart Vision Technology Co ltd
Original Assignee
Smart Vision Hangzhou Technology Development Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Smart Vision Hangzhou Technology Development Co Ltd filed Critical Smart Vision Hangzhou Technology Development Co Ltd
Priority to CN202211130193.3A priority Critical patent/CN115311329B/en
Priority to CN201910964726.XA priority patent/CN110675432B/en
Priority to CN202211130202.9A priority patent/CN115311330B/en
Publication of CN110675432A publication Critical patent/CN110675432A/en
Application granted granted Critical
Publication of CN110675432B publication Critical patent/CN110675432B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a multi-target video tracking method based on multi-dimensional feature fusion, and relates to the technical field of videos and images. The method comprises the following steps: recording time information, position information and content characteristics of the detection target; calculating the content feature similarity of the detection target and all track containers, and if the maximum similarity value is greater than a first threshold value, determining that the detection target and all track containers are effectively matched; for unmatched track containers, further calculating the change proportion of the historical track speed to the temporary track speed, the movement included angle between the detection target and the motion direction of the historical track and the width change proportion, and if the three are within a given threshold interval, determining that the three are effectively matched; and storing the effectively matched detection target into a corresponding track container. The invention has real and reliable matching result and effectively reduces the serial number phenomenon caused by track fragments, shielding and other conditions.

Description

Multi-dimensional feature fusion-based video multi-target tracking method
Technical Field
The invention relates to the technical field of video and image processing, in particular to a multi-target video tracking method based on multi-dimensional feature fusion.
Background
The multi-target tracking technology is an important link of a video analysis technology, and the technology obtains a motion direction and a predicted position through target historical track analysis according to the appearance time sequence of input Detection target Detection images and combines the content characteristic similarity between targets for matching connection. How to effectively match and concatenate the target in each video frame with the track target in the historical frame is the key of the technology. The technical field of multi-target tracking at present generally adopts target prediction (a selection scheme of Kalman trajectory prediction in a relatively mature way) position, then matches the actual position of a detected target with the predicted position, and then matches the trajectory and the target which are not matched with each other by feature similarity; or preferentially matching the track and the target by using the feature similarity to ensure that the long-distance moving target can be matched, and then matching the actual position of the detected target with the predicted position.
Aiming at the problems that in a daily monitoring video, targets are overlapped and lost due to mutual shielding of the targets, and the phenomenon of rapid movement of the targets is common; therefore, the problems that simple position matching and feature similarity matching cannot be well solved are caused, the final track fragments are excessive, and the phenomenon of serial numbers (IDSwitch) in the track is serious.
Disclosure of Invention
The invention aims to provide a video multi-target tracking method based on multi-dimensional feature fusion, which is real and reliable in matching result and effectively reduces the serial number phenomenon caused by track fragments, shielding and the like.
In order to achieve the purpose, the invention provides the following technical scheme:
a video multi-target tracking method based on multi-dimensional feature fusion is characterized by comprising the following steps:
s1, recording the time information, the position information and the content characteristics of the detection target;
s2, calculating the content feature similarity of the detection target and all track containers, and if the maximum similarity value is greater than a first threshold value, determining that the detection target and all track containers are effectively matched;
s3, for unmatched track containers, further calculating the change proportion of the historical speed and the temporary speed of the track, the movement included angle between the detection target and the motion direction of the historical track and the width change proportion of the track, and if the three are within a given threshold interval, determining that the three are effectively matched;
and S4, storing the detection target which is effectively matched into the corresponding track container.
Further, the specific content of S2 is as follows:
extracting content characteristics of the detection target, and performing content characteristic similarity calculation with the content characteristics stored in all track containers to obtain a corresponding similarity value sequence; finding out the maximum similarity value from the similarity value sequence to be compared with a first threshold value, calculating the distance between the detection target and the last frame in the track container, and comparing the distance with a distance threshold value; if the maximum similarity value is greater than the first threshold and the distance is within the distance threshold, a valid match is identified.
Further, the specific content of S2 is as follows:
if the track container has historical record information of the detection target, predicting the position information of the detection target which should appear in the current frame in real time to obtain a predicted rectangular area;
detecting a rectangular area of an actual detection target, calculating an intersection ratio of a prediction rectangular area and the rectangular area, and taking the difference between 1 and the intersection ratio as the distance between the actual detection target and the prediction detection target; and calculating the similarity of the content characteristics of the track container corresponding to the minimum distance and the detection target, and comparing the similarity with a first threshold value, wherein if the maximum similarity value is greater than the first threshold value, the track container is determined to be effectively matched.
Further, the specific content of S3 is as follows:
for unmatched track containers, the track container PTrack with the maximum similarity to the content features of the detection target is selectedContSimultaneously recording the corresponding similarity values SimiCont(ii) a Calculating the distances between the detection target and all track containers, and taking the track container Ptrack corresponding to the minimum distanceEucCalculating the track container PtrackEucSimilarity value Simi with content feature of detection targetEuc；
S31 if PTrackContAnd PtrackEucSame, and SimiContIf the three are within the corresponding given threshold interval, the effective matching is determined;
s32 if PTrackContAnd PtrackEucSame as when SimiContAnd SimiEucWhen the larger value is larger than or equal to the given parameter threshold, calculating the width change proportion, and if the width change proportion is within the given threshold interval, determining that the width change proportion is effectively matched; when SimiContAnd SimiEucWhen the larger value of the two values is smaller than a given parameter threshold value, taking the track container corresponding to the larger value of the two values, calculating the change proportion of the historical track speed and the temporary speed, and the movement included angle and the width change proportion of the detection target and the historical track movement direction, and if the three values are in the corresponding given threshold value interval, determining that the three values are effectively matched;
s33, if the detection target still does not find a track container with effective matching, taking SimiContAnd SimiEucAnd (4) calculating the change proportion of the historical track speed and the temporary speed, and the movement included angle and the width change proportion of the detection target and the historical track movement direction, and if the three values are in the corresponding given threshold interval, determining that the three values are effectively matched.
Further, the calculation method of the historical track speed and the temporary track speed is as follows:
Vhist＝(Rt-Rt-Δt) [ Delta ] t, wherein RtIs position information of the last to last frame of the history information, Rt-ΔtIs the position information of the penultimate frame of the history information, and Δ t is the time interval of two frames of history information;
Vcur＝(Rcur-Rt)/Δtcurwherein，RtIs position information of the last to last frame of the history information, RcurIs position information of the detection target, Δ tcurIs the time interval between the detection target time and the last but one frame of the history information.
Further, the method for calculating the motion included angle between the detection target and the motion direction of the historical track is as follows:
Figure RE-GDA0002274217770000041
wherein the content of the first and second substances,
Figure RE-GDA0002274217770000042
is RtCenter point location to RcurVector representation of the center point position;is Rt-ΔtCenter point location to RtVector representation of the location of the center point.
Further, in the S31, if PTrackContIf the history information of (2) has only one frame of information, the width change ratio is directly calculated, and if the width change ratio is within a given threshold interval, the matching is determined to be valid.
Further, the distance is a euclidean distance.
Further, the content features are extracted by adopting a deep convolutional neural network.
Further, the position information includes a center position, an aspect ratio, and a height of the detection target.
Compared with the prior art, the invention has the beneficial effects that: according to the invention, from the perspective of multi-feature fusion, feature matching is preferred, and in the first two links, content feature similarity constraint is added to the matching result of each link, so that the matching result is output truly and reliably; and selecting the optimal matching logic for the Euclidean distance space and the content characteristic space for the last remaining detection targets which are not successfully matched in a discrete mode, and then ensuring the truth and reliability of matching by using the multidimensional constraints of time, position, speed and angle. The invention has high operation efficiency and far-exceeding real-time requirements, effectively reduces track fragments and the serial number phenomenon caused by shielding and the like, and obtains good application effect.
Drawings
Fig. 1 is an overall schematic block diagram of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention are clearly and completely described below, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Referring to fig. 1, the present invention provides a video multi-target tracking method based on multi-dimensional feature fusion, which is characterized by comprising the following steps:
and S1, recording the time information, the position information and the content characteristics of the detection target.
Specifically, for a newly added Detection target Detection, a new track container PathTrack is opened, the life limit MaxAge of the track is initialized, and the preferential MaxAge is 56; initializing a state matrix of a Kalman filter of the new track container PathTrack to be an initialization state; and recording time information, position information, corresponding speed information in image coordinates and image content characteristics of the Detection target Detection, and storing the time information, the position information, the corresponding speed information and the image content characteristics of the Detection target Detection in the MsgBox unit stack. The position information includes a center position, an aspect ratio, and a height of the detection target. Preferably, the image content features adopt abstract features extracted by a deep convolutional neural network.
And S2, calculating the content feature similarity of the detection target and all track containers, and if the maximum similarity value is greater than a first threshold value, determining that the detection target and all track containers are effectively matched. And the content feature matching is adopted, so that the matched result is output really and reliably.
There are three specific embodiments of step S2, which are as follows:
in the first embodiment, the specific content of S2 is as follows:
and extracting image content characteristics of the Detection target Detection, and performing content characteristic similarity calculation with the image content characteristics of the MsgBox information stored in all track containers PathTrack. Wherein, it is noted that two N-dimensional image content features X, Y are described as: x (X)1,x2,...,xN)，Y(y1,y2,...,yN) The corresponding image content similarity calculation formula between X and Y is as follows:
Figure RE-GDA0002274217770000061
and according to a similarity calculation formula of the content features, obtaining a similarity result sequence of the image content features corresponding to the Detection and MsgBox information stored in the PathTrack. Finding out the maximum similarity value and the corresponding relation between the Detection and PathTrack from the similarity value sequence, wherein the central position of the Detection is (x)det,ydet) The position information of the last MsgBox center in PathTrack is (x)last,ylast) The corresponding distance calculation formula:
Figure RE-GDA0002274217770000062
an invalid match is considered if the ratio of the corresponding distance D to the width dimension information of Detection is greater than the corresponding distance threshold condition distThr. Otherwise, comparing the maximum similarity value with a first threshold value, if the maximum similarity value is greater than the first threshold value SimiThr, determining that the maximum similarity value is a valid match, and jumping to S4; otherwise, an invalid match is considered. Preferably, the distance threshold value distThr is 1.1 and the first threshold value simitr is 0.5.
In the second embodiment, the specific content of S2 is as follows:
if the track container PathTrack has history information of the Detection target Detection, a Kalman filter is used to predict the history information in real time for the position information (including the central position, the aspect ratio and the height of the Detection target Detection and the corresponding speed information predictDetection in the image coordinate) of the Detection target Detection to appear in the current frame, so as to obtain a predicted rectangular area PreRect. Wherein, the Kalman filter adopts a uniform velocity model and a linear observation model.
Detecting a rectangular area Rect where an actual Detection target Detection is located, and respectively carrying out position distance calculation with a prediction rectangular area PreRect of all track containers where the Detection target Detection exists to obtain corresponding distance relation sequences. Here, the method for calculating the position distance according to the present invention includes: calculating the intersection ratio of the prediction rectangular region PreRect and the rectangular region Rect, and then taking the difference between 1 and the intersection ratio as the distance dist between the actual detection target and the prediction detection target; the specific formula is as follows:
Figure RE-GDA0002274217770000071
from the distance relation sequence, calculating the content feature similarity of the track container PathTrack corresponding to the minimum distance and the Detection target Detection, comparing the content feature similarity with the first threshold value SimiThr in the first embodiment, if the maximum similarity is greater than the first threshold value SimiThr, determining that the content feature similarity is effective matching, and jumping to S4; otherwise, an invalid match is considered.
And in the second embodiment, the Kalman prediction is utilized to analyze and predict the position of the historical track data, the predicted position is used for carrying out position matching with the target, and the matched pair is subjected to image content similarity constraint, so that the matching reliability is further improved.
In the third embodiment, the specific content of S2 may also be a superposition of the first embodiment and the second embodiment, specifically as follows:
if the track container PathTrack has history information of the Detection target Detection, a Kalman filter is used to predict the history information in real time for the position information (including the central position, the aspect ratio and the height of the Detection target Detection and the corresponding speed information predictDetection in the image coordinate) of the Detection target Detection to appear in the current frame, so as to obtain a predicted rectangular area PreRect. Wherein, the Kalman filter adopts a uniform velocity model and a linear observation model.
Extracting image content characteristics of Detection target Detection and all railsAnd performing content feature similarity calculation on the image content features of the MsgBox information stored in the track container PathTrack. Wherein, it is noted that two N-dimensional image content features X, Y are described as: x (X)1,x2,...,xN)，Y(y1,y2,...,yN) The corresponding image content similarity calculation formula between X and Y is as follows:
Figure RE-GDA0002274217770000081
and according to a similarity calculation formula of the content features, obtaining a similarity result sequence of the image content features corresponding to the Detection and MsgBox information stored in the PathTrack. Finding out the maximum similarity value and the corresponding relation between the Detection and PathTrack from the similarity value sequence, wherein the central position of the Detection is (x)det,ydet) The position information of the last MsgBox center in PathTrack is (x)last,ylast) The corresponding distance calculation formula:an invalid match is considered if the ratio of the corresponding distance D to the width dimension information of Detection is greater than the corresponding distance threshold condition distThr. Otherwise, comparing the maximum similarity value with a first threshold value, if the maximum similarity value is greater than the first threshold value SimiThr, determining that the maximum similarity value is a valid match, and jumping to S4; otherwise, an invalid match is considered. Preferably, the distance threshold value distThr is 1.1 and the first threshold value simitr is 0.5.
If the Detection target Detection is invalid and matched, further detecting the rectangular area Rect where the actual Detection target Detection is located, and respectively calculating the position distance with the predicted rectangular area Rect of all the track containers where the Detection target Detection exists to obtain the corresponding distance relation sequence. Here, the method for calculating the position distance according to the present invention includes: calculating the intersection ratio of the prediction rectangular region PreRect and the rectangular region Rect, and then taking the difference between 1 and the intersection ratio as the distance dist between the actual detection target and the prediction detection target; the specific formula is as follows:
Figure RE-GDA0002274217770000091
calculating the content feature similarity of the track container PathTrack corresponding to the minimum distance from the distance relation sequence and the Detection target Detection, comparing the content feature similarity with the first threshold value SimiThr, if the maximum similarity is greater than the first threshold value SimiThr, determining that the content feature similarity is effective matching, and jumping to S4; otherwise, an invalid match is considered.
Compared with the first embodiment and the second embodiment, the third embodiment adopts two links of feature matching, and content feature similarity constraint is added to the matching result of each link, so that the matching result is output truly and reliably.
And S3, for the unmatched track container PathTrack, further calculating the change proportion of the historical speed and the temporary speed of the track, the movement included angle between the detection target and the motion direction of the historical track and the width change proportion, and if the three are within a given threshold interval, determining that the three are effectively matched.
The specific contents are as follows:
for the unmatched track container PathTrack, on one hand, the track container PTTrack with the maximum content feature similarity with the Detection target Detection is takenContSimultaneously recording the corresponding similarity values SimiCont(ii) a It is worth mentioning that if the similarity value SimiContIf the threshold value is less than 0.3, the PTrack is judgedContAbsent, jump to S1.
On the other hand, the Euclidean distances between the detection target and all the track containers are calculated, and the track container Ptrack corresponding to the minimum distance is selectedEucCalculating the track container PtrackEucSimilarity value Simi with content feature of detection targetEuc(ii) a If the ratio of the minimum distance to the width of Detection is greater than a given threshold value MoveDist, preferably 3.3, then Ptrack is determinedEucAbsent, jump to S1. It is worth mentioning that the euclidean distance dist herein is calculated as:
Figure RE-GDA0002274217770000101
wherein dist is the Euclidean distance; (x)new,ynew) The coordinate of the central point of the rectangular region Rect; (x)hist,yhist) Coordinates of the center point of the rectangular area preret are predicted.
S31 if PTrackContAnd PtrackEucSame, and SimiContGreater than or equal to a similarity threshold ValidSimiThr for effective matching, preferably, ValidSimiThr ═ 0.65. Then, if the PTrackContWhen the history information MsgBox exists for two frames or more, calculating the history speed V of the trackhistWith a temporary speed VcurIf the three are within the corresponding given threshold interval, the change proportion of the motion Angle and the width change proportion of the motion direction of the detection target and the historical track is determined as effective matching, and the step is carried out to S4. In particular, computational analysis VcurAnd VhistVariation amplitude, variation multiple less than a given parameter threshold [1/VThr, VThr]Preferably, VThr ═ 2; meanwhile, the Angle is smaller than a given parameter threshold value anglerthr, and the preferred Angle thr is 45 °. If not, an invalid match is considered; otherwise, calculating the width ratio change, namely the ratio of the width size information in the specific first-last MsgBox to the width of the rectangular region Rect, and if the ratio is in a given change threshold interval [1/changeRate, changeRate]Then a valid match is considered, the preferred ChangeRate is 1.5; then the match is deemed a valid match and a jump is made to S4.
If the PTrackContIf the history information of (1) has only one frame, the width change ratio is directly calculated, and if the width change ratio is in a given threshold interval [1/ChangeRate, ChangeRate ]]If yes, the process goes to S4. The following is the historical speed VhistTemporary velocity VcurAnd a specific calculation method of the Angle of motion.
The calculation method of the track historical speed and the temporary speed is as follows:
Vhist＝(Rt-Rt-Δt) [ Delta ] t, wherein RtIs the position information of the last frame of the MsgBox information, Rt-ΔtIs MsgBox penultimatePosition information of two frames, note RtCenter (x) of (c)t,yt)，Rt-ΔtCenter (x) of (c)t-Δt,yt-Δt) Then (R)t-Rt-Δt) The calculation formula of (2):
Figure RE-GDA0002274217770000111
at is the time interval of two frames of history information.
Vcur＝(Rcur-Rt)/ΔtcurWherein R istIs the position information of the last frame of the MsgBox information, RcurIs position information of Detection, note RcurCenter (x) of (c)cur,ycur) Then (R)cur-Rt) The calculation formula of (2):Δtcuris the time interval between the detection target time and the last but one frame of the history information.
The calculation method of the motion included angle between the detection target and the motion direction of the historical track is as follows:
Figure RE-GDA0002274217770000113
wherein the content of the first and second substances,is RtCenter point location to RcurVector representation of the center point position;
Figure RE-GDA0002274217770000115
is Rt-ΔtCenter point location to RtVector representation of the location of the center point.
S32 if PTrackContAnd PtrackEucSame as when SimiContAnd SimiEucWhen the larger value is equal to or greater than the given parameter threshold value directmatcthr, the width change ratio is preferably calculated such that directmatcthr is 0.85. If the width variation ratio is within a given threshold range [1/ChangeRate, ChangeRate]Interior, then it is determined to haveMatching, jumping to S4; when SimiContAnd SimiEucWhen the larger value of the two values is smaller than a given parameter threshold value, the track container corresponding to the larger value of the two values is taken to be recorded as a PTrackBetter. Like S31, when PTrackBetterExtracting V when the medium history track MsgBox exists for more than two framescurAnd VhistAngle information. Calculating the historical velocity V of the trackhistWith a temporary speed VcurIf the three are within the corresponding given threshold interval, the change proportion of the motion Angle and the width change proportion of the motion direction of the detection target and the historical track is determined as effective matching, and the step is carried out to S4.
S33, if the Detection target Detection still does not find a track container which is effectively matched, taking SimiContAnd SimiEucThe track container corresponding to the smaller value (i.e. the track container remaining for matching in S32) is marked as PTrackLast. Like S31, when PTrackLastExtracting V when the medium history track MsgBox exists for more than two framescurAnd VhistAngle information. Calculating the historical velocity V of the trackhistWith a temporary speed VcurIf the change proportion, the motion included Angle between the detection target and the motion direction of the historical track and the width change proportion are all in the corresponding given threshold value interval, effective matching is determined, and S4 is skipped. Otherwise, the process jumps to S1 to open a new track container.
And S4, storing the detection target which is effectively matched into the corresponding track container. Specifically, if the track container is matched with a new Detection target Detection, the MsgBox correspondingly recording the Detection information is stored in a stack of the track container PathTrack and is stored in sequence; meanwhile, the existence life counter of the track is 0, and simultaneously, the state of the Kalman state matrix is updated; if no new Detection target Detection is matched, the life counter is increased by 1 until the track life counter reaches MaxAge, and the track is ended.
It will be evident to those skilled in the art that the invention is not limited to the details of the foregoing illustrative embodiments, and that the present invention may be embodied in other specific forms without departing from the spirit or essential attributes thereof. The present embodiments are therefore to be considered in all respects as illustrative and not restrictive, the scope of the invention being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein.

Claims (10)

1. A video multi-target tracking method based on multi-dimensional feature fusion is characterized by comprising the following steps:
s1, recording the time information, the position information and the content characteristics of the detection target;
s2, calculating the content feature similarity of the detection target and all track containers, and if the maximum similarity value is greater than a first threshold value, determining that the detection target and all track containers are effectively matched;
s3, for unmatched track containers, further calculating the change proportion of the historical speed and the temporary speed of the track, the movement included angle between the detection target and the motion direction of the historical track and the width change proportion of the track, and if the three are within a given threshold interval, determining that the three are effectively matched;
and S4, storing the detection target which is effectively matched into the corresponding track container.
2. The multi-target video tracking method based on multi-dimensional feature fusion of claim 1, wherein the specific contents of the step S2 are as follows:
extracting content characteristics of the detection target, and performing content characteristic similarity calculation with the content characteristics stored in all track containers to obtain a corresponding similarity value sequence; finding out the maximum similarity value from the similarity value sequence to be compared with a first threshold value, calculating the distance between the detection target and the last frame in the track container, and comparing the distance with a distance threshold value; if the maximum similarity value is greater than the first threshold and the distance is within the distance threshold, a valid match is identified.
3. The multi-dimensional feature fusion based video multi-target tracking method according to claim 1 or 2, wherein the specific contents of the step S2 are as follows:
if the track container has historical record information of the detection target, predicting the position information of the detection target which should appear in the current frame in real time to obtain a predicted rectangular area;
detecting a rectangular area of an actual detection target, calculating an intersection ratio of a prediction rectangular area and the rectangular area, and taking the difference between 1 and the intersection ratio as the distance between the actual detection target and the prediction detection target; and calculating the similarity of the content characteristics of the track container corresponding to the minimum distance and the detection target, and comparing the similarity with a first threshold value, wherein if the maximum similarity value is greater than the first threshold value, the track container is determined to be effectively matched.
4. The multi-target video tracking method based on multi-dimensional feature fusion of claim 1, wherein the specific contents of the step S3 are as follows:
for unmatched track containers, the track container PTrack with the maximum similarity to the content features of the detection target is selectedContSimultaneously recording the corresponding similarity values SimiCont(ii) a Calculating the distances between the detection target and all track containers, and taking the track container Ptrack corresponding to the minimum distanceEucCalculating the track container PtrackEucSimilarity value Simi with content feature of detection targetEuc；
S31 if PTrackContAnd PtrackEucSame, and SimiContIf the three are within the corresponding given threshold interval, the effective matching is determined;
s32 if PTrackContAnd PtrackEucSame as when SimiContAnd SimiEucWhen the larger value is larger than or equal to the given parameter threshold, calculating the width change proportion, and if the width change proportion is within the given threshold interval, determining that the width change proportion is effectively matched; when SimiContAnd SimiEucWhen the larger value is smaller than the given parameter threshold value, the two are takenCalculating the change proportion of the historical speed and the temporary speed of the track, the movement included angle and the width change proportion of the movement direction of the detection target and the historical track, and if the change proportion of the historical speed and the temporary speed of the track, the movement included angle and the width change proportion of the detection target and the historical track are within a corresponding given threshold interval, determining that the change proportion is effective matching;
s33, if the detection target still does not find a track container with effective matching, taking SimiContAnd SimiEucAnd (4) calculating the change proportion of the historical track speed and the temporary speed, and the movement included angle and the width change proportion of the detection target and the historical track movement direction, and if the three values are in the corresponding given threshold interval, determining that the three values are effectively matched.
5. The multi-dimensional feature fusion-based video multi-target tracking method according to claim 1 or 4, wherein the calculation method of the track historical speed and the temporary speed is as follows:
Vhist＝(Rt-Rt-Δt) [ Delta ] t, wherein RtIs position information of the last to last frame of the history information, Rt-ΔtIs the position information of the penultimate frame of the history information, and Δ t is the time interval of two frames of history information;
Vcur＝(Rcur-Rt)/Δtcurwherein R istIs position information of the last to last frame of the history information, RcurIs position information of the detection target, Δ tcurIs the time interval between the detection target time and the last but one frame of the history information.
6. The multi-dimensional feature fusion-based video multi-target tracking method according to claim 1 or 4, wherein the calculation method of the motion included angle between the detection target and the motion direction of the historical track is as follows:
Figure FDA0002230111090000031
wherein the content of the first and second substances,
Figure FDA0002230111090000032
is RtCenter point location to RcurVector representation of the center point position;
Figure FDA0002230111090000033
is Rt-ΔtCenter point location to RtVector representation of the location of the center point.
7. The multi-target video tracking method based on multi-dimensional feature fusion of claim 4, wherein in the step S31, if PTrackContIf the history information of (2) has only one frame of information, the width change ratio is directly calculated, and if the width change ratio is within a given threshold interval, the matching is determined to be valid.
8. The multi-target video tracking method based on multi-dimensional feature fusion of claim 4, wherein the distance is Euclidean distance.
9. The multi-target video tracking method based on multi-dimensional feature fusion of claim 1, wherein the content features are extracted by a deep convolutional neural network.
10. The multi-dimensional feature fusion based video multi-target tracking method according to claim 1, wherein the position information comprises a center position, an aspect ratio and a height of a detection target.
CN201910964726.XA 2019-10-11 2019-10-11 Video multi-target tracking method based on multi-dimensional feature fusion Active CN110675432B (en)

Priority Applications (3)

Application Number Priority Date Filing Date Title
CN202211130193.3A CN115311329B (en) 2019-10-11 2019-10-11 Video multi-target tracking method based on double-link constraint
CN201910964726.XA CN110675432B (en) 2019-10-11 2019-10-11 Video multi-target tracking method based on multi-dimensional feature fusion
CN202211130202.9A CN115311330B (en) 2019-10-11 2019-10-11 Video multi-target tracking method based on position prediction

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910964726.XA CN110675432B (en) 2019-10-11 2019-10-11 Video multi-target tracking method based on multi-dimensional feature fusion

Related Child Applications (2)

Application Number Title Priority Date Filing Date
CN202211130193.3A Division CN115311329B (en) 2019-10-11 2019-10-11 Video multi-target tracking method based on double-link constraint
CN202211130202.9A Division CN115311330B (en) 2019-10-11 2019-10-11 Video multi-target tracking method based on position prediction

Publications (2)

Publication Number Publication Date
CN110675432A true CN110675432A (en) 2020-01-10
CN110675432B CN110675432B (en) 2022-11-08

Family

ID=69081635

Family Applications (3)

Application Number Title Priority Date Filing Date
CN201910964726.XA Active CN110675432B (en) 2019-10-11 2019-10-11 Video multi-target tracking method based on multi-dimensional feature fusion
CN202211130202.9A Active CN115311330B (en) 2019-10-11 2019-10-11 Video multi-target tracking method based on position prediction
CN202211130193.3A Active CN115311329B (en) 2019-10-11 2019-10-11 Video multi-target tracking method based on double-link constraint

Family Applications After (2)

Application Number Title Priority Date Filing Date
CN202211130202.9A Active CN115311330B (en) 2019-10-11 2019-10-11 Video multi-target tracking method based on position prediction
CN202211130193.3A Active CN115311329B (en) 2019-10-11 2019-10-11 Video multi-target tracking method based on double-link constraint

Country Status (1)

Country Link
CN (3) CN110675432B (en)

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111914754A (en) * 2020-08-03 2020-11-10 杭州云栖智慧视通科技有限公司 Image content similarity measurement method and device and computer equipment
CN112184769A (en) * 2020-09-27 2021-01-05 上海高德威智能交通系统有限公司 Tracking abnormity identification method, device and equipment
CN113112526A (en) * 2021-04-27 2021-07-13 北京百度网讯科技有限公司 Target tracking method, device, equipment and medium
CN113971216A (en) * 2021-10-22 2022-01-25 北京百度网讯科技有限公司 Data processing method and device, electronic equipment and memory
CN114155273A (en) * 2021-10-20 2022-03-08 浙江大立科技股份有限公司 Video image single-target tracking method combined with historical track information
CN114329063A (en) * 2021-10-29 2022-04-12 腾讯科技（深圳）有限公司 Video clip detection method, device and equipment
CN117315421A (en) * 2023-09-26 2023-12-29 中国人民解放军91977 部队 Method and device for predicting flight path of offshore target
CN117495917A (en) * 2024-01-03 2024-02-02 山东科技大学 Multi-target tracking method based on JDE multi-task network model
CN114155273B (en) * 2021-10-20 2024-06-04 浙江大立科技股份有限公司 Video image single-target tracking method combining historical track information

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115965657B (en) * 2023-02-28 2023-06-02 安徽蔚来智驾科技有限公司 Target tracking method, electronic device, storage medium and vehicle

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104094279A (en) * 2014-04-30 2014-10-08 中国科学院自动化研究所 Large-range-first cross-camera visual target re-identification method
CN104915970A (en) * 2015-06-12 2015-09-16 南京邮电大学 Multi-target tracking method based on track association
CN105261035A (en) * 2015-09-15 2016-01-20 杭州中威电子股份有限公司 Method and device for tracking moving objects on highway
CN107291216A (en) * 2016-04-05 2017-10-24 中兴通讯股份有限公司 A kind of mobile terminal method for tracking target, device and mobile terminal
CN109191497A (en) * 2018-08-15 2019-01-11 南京理工大学 A kind of real-time online multi-object tracking method based on much information fusion
CN109344712A (en) * 2018-08-31 2019-02-15 电子科技大学 A kind of road vehicle tracking
CN110135314A (en) * 2019-05-07 2019-08-16 电子科技大学 A kind of multi-object tracking method based on depth Trajectory prediction

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8116527B2 (en) * 2009-10-07 2012-02-14 The United States Of America As Represented By The Secretary Of The Army Using video-based imagery for automated detection, tracking, and counting of moving objects, in particular those objects having image characteristics similar to background
CN104424648B (en) * 2013-08-20 2018-07-24 株式会社理光 Method for tracing object and equipment
CN104517275A (en) * 2013-09-27 2015-04-15 株式会社理光 Object detection method and system
CN106845385A (en) * 2017-01-17 2017-06-13 腾讯科技（上海）有限公司 The method and apparatus of video frequency object tracking
WO2019006632A1 (en) * 2017-07-04 2019-01-10 深圳大学 Video multi-target tracking method and device
CN108460787B (en) * 2018-03-06 2020-11-27 北京市商汤科技开发有限公司 Target tracking method and apparatus, electronic device, program, and storage medium
CN110276783B (en) * 2019-04-23 2021-01-08 上海高重信息科技有限公司 Multi-target tracking method and device and computer system
CN110084836B (en) * 2019-04-26 2022-03-04 西安电子科技大学 Target tracking method based on deep convolution characteristic hierarchical response fusion

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104094279A (en) * 2014-04-30 2014-10-08 中国科学院自动化研究所 Large-range-first cross-camera visual target re-identification method
CN104915970A (en) * 2015-06-12 2015-09-16 南京邮电大学 Multi-target tracking method based on track association
CN105261035A (en) * 2015-09-15 2016-01-20 杭州中威电子股份有限公司 Method and device for tracking moving objects on highway
CN107291216A (en) * 2016-04-05 2017-10-24 中兴通讯股份有限公司 A kind of mobile terminal method for tracking target, device and mobile terminal
CN109191497A (en) * 2018-08-15 2019-01-11 南京理工大学 A kind of real-time online multi-object tracking method based on much information fusion
CN109344712A (en) * 2018-08-31 2019-02-15 电子科技大学 A kind of road vehicle tracking
CN110135314A (en) * 2019-05-07 2019-08-16 电子科技大学 A kind of multi-object tracking method based on depth Trajectory prediction

Non-Patent Citations (4)

* Cited by examiner, † Cited by third party
Title
NICOLAI WOJKE ET AL: "SIMPLE ONLINE AND REALTIME TRACKING WITH A DEEP ASSOCIATION METRIC", 《ARXIV:1703.07402V1》 *
NICOLAI WOJKE ET AL: "SIMPLE ONLINE AND REALTIME TRACKING WITH A DEEP ASSOCIATION METRIC", 《ARXIV:1703.07402V1》, 21 March 2017 (2017-03-21), pages 1 - 5, XP080758706, DOI: 10.1109/ICIP.2017.8296962 *
杜永胜: "基于运动特征的视频网络多目标匹配和跟踪方法研究", 《中国优秀博硕士学位论文全文数据库（硕士） 信息科技辑》 *
杜永胜: "基于运动特征的视频网络多目标匹配和跟踪方法研究", 《中国优秀博硕士学位论文全文数据库（硕士） 信息科技辑》, 15 February 2017 (2017-02-15), pages 19 - 25 *

Cited By (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111914754A (en) * 2020-08-03 2020-11-10 杭州云栖智慧视通科技有限公司 Image content similarity measurement method and device and computer equipment
CN112184769B (en) * 2020-09-27 2023-05-02 上海高德威智能交通系统有限公司 Method, device and equipment for identifying tracking abnormality
CN112184769A (en) * 2020-09-27 2021-01-05 上海高德威智能交通系统有限公司 Tracking abnormity identification method, device and equipment
CN113112526B (en) * 2021-04-27 2023-09-22 北京百度网讯科技有限公司 Target tracking method, device, equipment and medium
WO2022227771A1 (en) * 2021-04-27 2022-11-03 北京百度网讯科技有限公司 Target tracking method and apparatus, device and medium
CN113112526A (en) * 2021-04-27 2021-07-13 北京百度网讯科技有限公司 Target tracking method, device, equipment and medium
CN114155273A (en) * 2021-10-20 2022-03-08 浙江大立科技股份有限公司 Video image single-target tracking method combined with historical track information
CN114155273B (en) * 2021-10-20 2024-06-04 浙江大立科技股份有限公司 Video image single-target tracking method combining historical track information
CN113971216B (en) * 2021-10-22 2023-02-03 北京百度网讯科技有限公司 Data processing method and device, electronic equipment and memory
CN113971216A (en) * 2021-10-22 2022-01-25 北京百度网讯科技有限公司 Data processing method and device, electronic equipment and memory
CN114329063A (en) * 2021-10-29 2022-04-12 腾讯科技（深圳）有限公司 Video clip detection method, device and equipment
CN117315421A (en) * 2023-09-26 2023-12-29 中国人民解放军91977 部队 Method and device for predicting flight path of offshore target
CN117495917A (en) * 2024-01-03 2024-02-02 山东科技大学 Multi-target tracking method based on JDE multi-task network model
CN117495917B (en) * 2024-01-03 2024-03-26 山东科技大学 Multi-target tracking method based on JDE multi-task network model

Also Published As

Publication number Publication date
CN110675432B (en) 2022-11-08
CN115311330B (en) 2023-04-07
CN115311329B (en) 2023-05-23
CN115311329A (en) 2022-11-08
CN115311330A (en) 2022-11-08

Similar Documents

Publication Publication Date Title
CN110675432B (en) Video multi-target tracking method based on multi-dimensional feature fusion
Luo et al. Fast and furious: Real time end-to-end 3d detection, tracking and motion forecasting with a single convolutional net
CN110084831B (en) Multi-target detection tracking method based on YOLOv3 multi-Bernoulli video
Ren et al. Deep reinforcement learning with iterative shift for visual tracking
CN110400332B (en) Target detection tracking method and device and computer equipment
CN105335986B (en) Method for tracking target based on characteristic matching and MeanShift algorithm
CN110853078B (en) On-line multi-target tracking method based on shielding pair
Tsintotas et al. Modest-vocabulary loop-closure detection with incremental bag of tracked words
CN102147851A (en) Device and method for judging specific object in multi-angles
US9014422B2 (en) Method of recognizing a moving feature in a video sequence
CN113052873B (en) Single-target tracking method for on-line self-supervision learning scene adaptation
CN107464256A (en) A kind of target detection differentiates the correlating method of amendment with possibility
Tsintotas et al. DOSeqSLAM: Dynamic on-line sequence based loop closure detection algorithm for SLAM
CN115423846A (en) Multi-target track tracking method and device
CN112037257B (en) Target tracking method, terminal and computer readable storage medium thereof
CN110766715B (en) Multi-target tracking method combined with single target track
CN116645396A (en) Track determination method, track determination device, computer-readable storage medium and electronic device
CN116343080A (en) Dynamic sparse key frame video target detection method, device and storage medium
Gao et al. Beyond group: Multiple person tracking via minimal topology-energy-variation
Tang et al. Place recognition using line-junction-lines in urban environments
CN117173217A (en) Video multi-target tracking method based on multi-scale channel feature aggregation
CN112990119B (en) Video multi-target facial expression recognition method and system
Song et al. A cross frame post-processing strategy for video object detection
Yan et al. Improved SiamFC Target Tracking Algorithm Based on Anti-Interference Module
Zhang et al. AIPT: Adaptive information perception for online multi-object tracking

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
TA01 Transfer of patent application right
TA01 Transfer of patent application right

Effective date of registration: 20220913

Address after: 310000 Room 401, building 2, No.16, Zhuantang science and technology economic block, Xihu District, Hangzhou City, Zhejiang Province

Applicant after: Hangzhou yunqi smart Vision Technology Co.,Ltd.

Address before: 310000 room 279, building 6, No. 16, Zhuantang science and technology economic block, Zhuantang street, Xihu District, Hangzhou City, Zhejiang Province

Applicant before: Smart vision (Hangzhou) Technology Development Co.,Ltd.

GR01 Patent grant
GR01 Patent grant