CN111080673A - Anti-occlusion target tracking method - Google Patents

Anti-occlusion target tracking method Download PDF

Info

Publication number
CN111080673A
CN111080673A CN201911261618.2A CN201911261618A CN111080673A CN 111080673 A CN111080673 A CN 111080673A CN 201911261618 A CN201911261618 A CN 201911261618A CN 111080673 A CN111080673 A CN 111080673A
Authority
CN
China
Prior art keywords
target
tracking
detection
candidate
candidate item
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201911261618.2A
Other languages
Chinese (zh)
Other versions
CN111080673B (en
Inventor
张盛
易梦云
徐赫
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenzhen International Graduate School of Tsinghua University
Original Assignee
Shenzhen International Graduate School of Tsinghua University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenzhen International Graduate School of Tsinghua University filed Critical Shenzhen International Graduate School of Tsinghua University
Priority to CN201911261618.2A priority Critical patent/CN111080673B/en
Publication of CN111080673A publication Critical patent/CN111080673A/en
Application granted granted Critical
Publication of CN111080673B publication Critical patent/CN111080673B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/22—Matching criteria, e.g. proximity measures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
    • G06V2201/07—Target detection
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
    • Y02T10/00—Road transport of goods or passengers
    • Y02T10/10—Internal combustion engine [ICE] based vehicles
    • Y02T10/40—Engine management systems

Abstract

The invention provides an anti-occlusion target tracking method, which comprises the steps of firstly, for an input video or an image sequence, detecting each frame of image in the video by adopting a target detector to obtain a candidate item based on detection; and according to the target detection result of the current frame, predicting the position of the target in the next frame by using a Kalman filter to obtain a candidate item based on tracking. Calculating the confidence of the candidate item according to a confidence score formula, and obtaining a final candidate item by adopting a non-maximum suppression algorithm; and inputting the candidate items of the adjacent frames into a feature matching network, and calculating the matching degree between the targets through a cascade matching algorithm. Extracting features of the candidate items based on detection through a deep neural network, and matching similarity between the features; and performing IOU coincidence degree matching based on the tracked candidate items. And determining the position of the target in the current frame according to the target matching result of the adjacent frame so as to output a target motion track. The target is detected and tracked under the condition that the target is shielded, and the tracking precision and performance are improved.

Description

Anti-occlusion target tracking method
Technical Field
The invention relates to the technical field of target tracking, in particular to an anti-occlusion target tracking method.
Background
In recent years, with the continuous development of deep neural networks and the continuous improvement of GPU computing power, methods based on deep learning make breakthrough progress on computer vision tasks. Computer vision technologies such as target detection, target recognition, target tracking, pedestrian re-recognition and the like are rapidly developed and widely applied to various industries and fields such as intelligent monitoring, human-computer interaction, virtual reality and augmented reality, medical image analysis and the like.
Multi-Object Tracking (Multi Object Tracking) is a classic computer vision task, a region of interest obtained by target Tracking is the basis for further high-level vision analysis, and the accuracy of target Tracking directly affects the performance of a computer vision system. Most of the existing multi-target Tracking methods adopt Tracking-by-Detection (Tracking-by-Detection), that is, under the Detection result of a target detector, the motion track association is carried out on the Detection result from the same target between frames. Such detection methods depend to a large extent on the detection result. However, in many practical applications, especially in crowded scenes, the detection result of the detector is usually not accurate enough due to the interaction between objects, the appearance similarity and frequent occlusion of the objects, thereby seriously affecting the accuracy and performance of tracking.
In the existing multi-target tracking algorithm, a target detector is retrained through a large-scale data set to obtain a more accurate detection result, but motion information in a video image is ignored, and the method is not efficient enough. Some methods carry out feature extraction by designing and training deeper neural networks to obtain more robust target features, however, the appearance similarity problem is difficult to solve by appearance-based features, and the real-time performance of the algorithm is difficult to guarantee. In view of the above, it is desirable to provide a new anti-occlusion target tracking method for solving the target occlusion interaction.
Disclosure of Invention
The invention provides an anti-occlusion target tracking method for solving the existing problems.
In order to solve the above problems, the technical solution adopted by the present invention is as follows:
an anti-occlusion target tracking method comprises the following steps: s1: inputting a video or an image sequence into a target detector according to frames to obtain a target detection result, wherein the target detection result is a candidate item based on detection and comprises a bounding box and detection confidence of all targets in each frame of image; s2: generating a candidate item based on tracking for each frame of image by utilizing a joint detection and tracking frame according to the target detection result, wherein the joint detection and tracking frame carries out tracking motion estimation on the detection result through a Kalman filter and camera motion compensation so as to obtain the candidate item based on tracking; s3: screening the detection-based candidate items and the tracking-based candidate items by using a non-maximum suppression algorithm according to the confidence degrees of the detection-based candidate items and the tracking-based candidate items to obtain the screened detection-based candidate items and the screened tracking-based candidate items; s4: extracting apparent features of all screened candidate items based on detection and screened candidate items based on tracking of the current frame by utilizing a pre-trained deep neural network; s5: calculating the target matching degree of the adjacent frames by using a cascade matching algorithm, wherein the method comprises the following steps: the screened candidate items based on detection perform apparent feature similarity matching on the existing tracks of the adjacent frames; the screened candidate items based on the tracking are subjected to boundary frame intersection with a target boundary frame of the existing track of the adjacent frame and are matched with the matching degree; s6: and determining the position of the target in the current frame according to the target matching degree of the adjacent frames, thereby outputting a target motion track.
Preferably, the object detector is an SDP object detector.
Preferably, the confidence is given by the following confidence score formula:
Figure BDA0002311749100000021
wherein the content of the first and second substances,
Figure BDA0002311749100000022
for the detection confidence of the t-1 th frame,
Figure BDA0002311749100000023
for the confidence of the tracking of the t-th frame,
Figure BDA0002311749100000024
the number N of the candidate items based on detection in the track to be associatedtrkFor the number of candidates based on tracking in the last trajectory to be associated, I (-) is a binary function, when the function is true, the value is 1, otherwise, the value is 0, and the parameter α is a constant.
Preferably, the screening the detection-based candidate item and the tracking-based candidate item by using a non-maximum suppression algorithm to obtain the screened detection-based candidate item and the screened tracking-based candidate item includes the following steps: s21: sorting according to the confidence score of all the candidate items based on detection and the candidate items based on tracking to obtain a candidate list; s22: selecting the detection-based candidate item and the tracking-based candidate item with the highest confidence level to be added into a final output list and deleted from the candidate item list; s23: calculating the detection-based candidate item and the tracking-based candidate item with the highest confidence coefficient to be in a border intersection ratio with other candidate items, and deleting the detection-based candidate item and the tracking-based candidate item with the border intersection ratio larger than a preset threshold value; s24: and repeating the process until the candidate item list is empty, wherein the final output list is the screened candidate items based on detection and the candidate items based on tracking.
Preferably, the preset threshold is 0.3-0.5.
Preferably, the deep neural network is a google lenet based network, comprising from the input layer to the initiation 4e layer, and then connected by a 1 × 1 convolutional layer.
Preferably, the loss function of the training of the neural network is:
ltriplet(Ii，Ij，Ik)＝m+d(Ii，Ij)-d(Ii，Ik)
wherein, Ii，IjFor pictures from the same identity, Ii，IkFor pictures from different identities, d represents the euclidean distance and m is a constant.
Preferably, the step of calculating the target matching degree of the adjacent frames by using the cascade matching algorithm comprises the following steps: s51: obtaining the target detection result of the first frame, and generating a track for each target to obtain an initial track set
Figure BDA0002311749100000031
Candidate item set composed of the filtered detection-based candidate items and the filtered tracking-based candidate items
Figure BDA0002311749100000032
The apparent characteristics
Figure BDA0002311749100000033
And constructing all matched candidate item sets
Figure BDA0002311749100000034
All candidate item sets that are not matched
Figure BDA0002311749100000035
S52: selecting the candidate item based on detection after screening
Figure BDA0002311749100000036
With the initial trajectory set
Figure BDA0002311749100000037
Calculating the feature similarity, and updating the matched candidate item set according to the matching result
Figure BDA0002311749100000038
The set of unmatched candidate items
Figure BDA0002311749100000039
The initial trajectoryCollection
Figure BDA00023117491000000310
S53: selecting the selected tracking-based candidate item
Figure BDA00023117491000000311
And the updated initial track set
Figure BDA00023117491000000312
The target bounding box carries out bounding box intersection and proportion matching, and the matched candidate item set is updated according to the matching result
Figure BDA00023117491000000313
The set of unmatched candidate items
Figure BDA00023117491000000314
Preferably, the matched candidate item set is
Figure BDA00023117491000000315
Each candidate item bounding box in the set of initial trajectories and the initial trajectory set matched thereto
Figure BDA00023117491000000316
The track segments in (1) are connected; grouping the unmatched candidate items
Figure BDA00023117491000000319
Initializing the track into a new track; for the initial track set
Figure BDA00023117491000000317
The track section which is not matched in the initial track set is set as a temporary track section, if the track section is not matched in the following continuous N frames, the temporary track section is considered to be finished, and the temporary track section is selected from the initial track set
Figure BDA00023117491000000318
Is deleted. And N is 5-8.
The invention also provides a computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the steps of the method according to any one of the above.
The invention has the beneficial effects that: the anti-blocking target tracking method is provided, and through the combined action of the combined detection tracking frame and the cascade matching algorithm, when the target is blocked interactively and the detection result of the detector is inaccurate, a better candidate item can be generated through the combined detection tracking frame, and the target cascade matching is carried out. The problem of inaccurate detection during target interaction shielding is solved, and the influence of target shielding on the tracking effect is reduced, so that accurate tracking during target shielding is realized.
Furthermore, the method is very simple to implement, the calculation cost is low, the algorithm can reach the operation speed of 30 frames/second on the GPU, and real-time tracking can be realized. Compared with the traditional target tracking method, the method has the advantages of low required calculation cost, strong anti-blocking capability and high real-time property.
Drawings
FIG. 1 is a schematic diagram of an anti-occlusion target tracking method in an embodiment of the present invention.
Fig. 2 is a schematic diagram of a method for obtaining filtered detection-based candidate items and filtered tracking-based candidate items according to an embodiment of the present invention.
Fig. 3 is a schematic diagram of a method for calculating a target matching degree of adjacent frames by using a cascade matching algorithm in the embodiment of the present invention.
Detailed Description
In order to make the technical problems, technical solutions and advantageous effects to be solved by the embodiments of the present invention more clearly apparent, the present invention is further described in detail below with reference to the accompanying drawings and the embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
It will be understood that when an element is referred to as being "secured to" or "disposed on" another element, it can be directly on the other element or be indirectly on the other element. When an element is referred to as being "connected to" another element, it can be directly connected to the other element or be indirectly connected to the other element. In addition, the connection may be for either a fixing function or a circuit connection function.
It is to be understood that the terms "length," "width," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," and the like are used in an orientation or positional relationship indicated in the drawings for convenience in describing the embodiments of the present invention and to simplify the description, and are not intended to indicate or imply that the referenced device or element must have a particular orientation, be constructed in a particular orientation, and be in any way limiting of the present invention.
Furthermore, the terms "first", "second" and "first" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of the embodiments of the present invention, "a plurality" means two or more unless specifically limited otherwise.
As shown in FIG. 1, the present invention provides an anti-occlusion target tracking method, which comprises the following steps:
s1: inputting a video or an image sequence into a target detector according to frames to obtain a target detection result, wherein the target detection result is a candidate item based on detection and comprises a bounding box and detection confidence of all targets in each frame of image;
s2: generating a candidate item based on tracking for each frame of image by utilizing a joint detection and tracking frame according to the target detection result, wherein the joint detection and tracking frame carries out tracking motion estimation on the detection result through a Kalman filter and camera motion compensation so as to obtain the candidate item based on tracking;
taking the nth frame as an example, the position of the target boundary frame output by the SDP target detector of the current frame is taken as a candidate for the nth frame based on detection. Meanwhile, the position of the target boundary box is input into a Kalman filter, and the position of the target boundary box in the next frame is estimated to be used as a candidate item of the N +1 th frame based on tracking.
S3: screening the detection-based candidate items and the tracking-based candidate items by using a non-maximum suppression algorithm according to the confidence degrees of the detection-based candidate items and the tracking-based candidate items to obtain the screened detection-based candidate items and the screened tracking-based candidate items;
s4: extracting apparent features of all screened candidate items based on detection and screened candidate items based on tracking of the current frame by utilizing a pre-trained deep neural network;
the apparent feature obtained in one embodiment of the invention is a 512-dimensional depth feature;
s5: calculating the target matching degree of the adjacent frames by using a cascade matching algorithm, wherein the method comprises the following steps: the screened candidate items based on detection perform apparent feature similarity matching on the existing tracks of the adjacent frames; the screened candidate items based on the tracking are subjected to boundary frame intersection with a target boundary frame of the existing track of the adjacent frame and are matched with the matching degree;
s6: and determining the position of the target in the current frame according to the target matching degree of the adjacent frames, thereby outputting a target motion track.
In one embodiment of the invention, the object detector is an SDP object detector.
The confidence is given by the following confidence score formula:
Figure BDA0002311749100000051
wherein the content of the first and second substances,
Figure BDA0002311749100000052
for the detection confidence of the t-1 th frame,
Figure BDA0002311749100000053
for the tracking confidence of the t-th frame, NdetTo be closedThe number of candidate items based on detection in the joint track, NtrkFor the number of candidates based on tracking in the last trajectory to be associated, I (-) is a binary function, when the function is true, the value is 1, otherwise, the value is 0, and the parameter α is a constant.
In one embodiment of the invention, the value of α is 0.05.
As shown in fig. 2, the step of screening the detection-based candidate and the tracking-based candidate by using a non-maximum suppression algorithm to obtain the screened detection-based candidate and the screened tracking-based candidate includes the following steps:
s21: sorting according to the confidence score of all the candidate items based on detection and the candidate items based on tracking to obtain a candidate list;
s22: selecting the detection-based candidate item and the tracking-based candidate item with the highest confidence level to be added into a final output list and deleted from the candidate item list;
s23: calculating the detection-based candidate item and the tracking-based candidate item with the highest confidence coefficient to be in a border intersection ratio with other candidate items, and deleting the detection-based candidate item and the tracking-based candidate item with the border intersection ratio larger than a preset threshold value;
s24: and repeating the process until the candidate item list is empty, wherein the final output list is the screened candidate items based on detection and the candidate items based on tracking.
In one embodiment of the invention, the predetermined threshold is 0.3-0.5.
The deep neural network is a google lenet-based network, which includes layers from the input layer to the initiation 4e, and then connected by a 1 × 1 convolutional layer. The network input picture size is 160 x 80, and the output target feature is 512 dimensions. The network is pre-trained on a large-scale pedestrian re-identification data set, and the loss function is as follows:
ltriplet(Ii，Ij，Ik)＝m+d(Ii，Ij)-d(Ii，Ik)
wherein, Ii，IjFor pictures from the same identity, Ii，IkFor pictures from different identities, d represents the euclidean distance and m is a constant.
As shown in fig. 3, the step of calculating the target matching degree of the adjacent frames by using the cascade matching algorithm includes the following steps:
s51: obtaining the target detection result of the first frame, and generating a track for each target to obtain an initial track set
Figure BDA0002311749100000061
Candidate item set composed of the filtered detection-based candidate items and the filtered tracking-based candidate items
Figure BDA0002311749100000062
The apparent characteristics
Figure BDA0002311749100000063
And constructing all matched candidate item sets
Figure BDA0002311749100000064
All candidate item sets that are not matched
Figure BDA0002311749100000065
S52: selecting the candidate item based on detection after screening
Figure BDA0002311749100000066
With the initial trajectory set
Figure BDA0002311749100000067
Calculating the feature similarity, and updating the matched candidate item set according to the matching result
Figure BDA0002311749100000071
The set of unmatched candidate items
Figure BDA0002311749100000072
The initial set of trajectories
Figure BDA0002311749100000073
In an embodiment of the invention, a Hungarian algorithm is used for feature similarity matching.
S53: selecting the selected tracking-based candidate item
Figure BDA0002311749100000074
And the updated initial track set
Figure BDA0002311749100000075
The target bounding box carries out bounding box intersection and proportion matching, and the matched candidate item set is updated according to the matching result
Figure BDA0002311749100000076
The set of unmatched candidate items
Figure BDA0002311749100000077
In one embodiment of the invention, Hungarian algorithm is used for bounding box intersection and fitness matching.
Further, matching the candidate item set
Figure BDA0002311749100000078
Each candidate item bounding box in the set of initial trajectories and the initial trajectory set matched thereto
Figure BDA0002311749100000079
The track segments in (1) are connected; grouping the unmatched candidate items
Figure BDA00023117491000000710
Initializing the track into a new track; for the initial track set
Figure BDA00023117491000000711
The track section which is not matched in the initial track set is set as a temporary track section, if the track section is not matched in the following continuous N frames, the temporary track section is considered to be finished, and the temporary track section is selected from the initial track set
Figure BDA00023117491000000712
And deleting, wherein N is generally 5-8.
On the MOT17 public multi-target pedestrian tracking data set, the tracking results of the present invention are shown in the table below. It can be seen that in most metrics, particularly in F1 scores, tracking rates, ID exchange times and accuracy, are superior to other existing techniques and can be run at real-time speeds. The improvement of the ID exchange times shows that the apparent features extracted by the method enhance the recognition capability of the tracker and reduce the inaccuracy of tracking under the condition that the target is interacted and occluded. The improvement of false positive and tracking rate indicates the effectiveness of the anti-occlusion target tracking method of the invention.
TABLE 1 test results
Method of producing a composite material Accuracy of measurement F1 score Tracking rate Loss rate False positive False negative Number of ID exchanges Speed of rotation
HISP 44.6 38.8 15.1％ 38.8％ 25,478 276,395 10,617 4.7
SORT 43.1 39.8 12.5％ 42.3％ 28,398 287,582 4,852 143.3
FPSN 44.9 48.4 16.5％ 35.8％ 33,757 269,952 7,136 10.1
MASS 46.9 46 16.9％ 36.3％ 25,773 269,116 4,478 17.1
OTCD 44.6 38.8 15.1％ 38.8％ 25,478 276,359 3,573 46.5
The invention 47.4 50.1 16.8％ 37.2％ 26,910 267,331 2,760 35.7
All or part of the flow of the method of the embodiments may be implemented by a computer program, which may be stored in a computer readable storage medium and executed by a processor, to instruct related hardware to implement the steps of the embodiments of the methods. Wherein the computer program comprises computer program code, which may be in the form of source code, object code, an executable file or some intermediate form, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, usb disk, removable hard disk, magnetic disk, optical disk, computer Memory, Read-only Memory (ROM), Random Access Memory (RAM), electrical carrier wave signals, telecommunications signals, software distribution medium, etc. It should be noted that the computer readable medium may contain content that is subject to appropriate increase or decrease as required by legislation and patent practice in jurisdictions, for example, in some jurisdictions, computer readable media does not include electrical carrier signals and telecommunications signals as is required by legislation and patent practice.
The foregoing is a more detailed description of the invention in connection with specific preferred embodiments and it is not intended that the invention be limited to these specific details. For those skilled in the art to which the invention pertains, several equivalent substitutions or obvious modifications can be made without departing from the spirit of the invention, and all the properties or uses are considered to be within the scope of the invention.

Claims (10)

1. An anti-occlusion target tracking method is characterized by comprising the following steps:
s1: inputting a video or an image sequence into a target detector according to frames to obtain a target detection result, wherein the target detection result is a candidate item based on detection and comprises a bounding box and detection confidence of all targets in each frame of image;
s2: generating a candidate item based on tracking for each frame of image by utilizing a joint detection and tracking frame according to the target detection result, wherein the joint detection and tracking frame carries out tracking motion estimation on the target detection result through a Kalman filter and camera motion compensation to obtain the candidate item based on tracking;
s3: screening the detection-based candidate items and the tracking-based candidate items by using a non-maximum suppression algorithm according to the confidence degrees of the detection-based candidate items and the tracking-based candidate items to obtain the screened detection-based candidate items and the screened tracking-based candidate items;
s4: extracting apparent features of all screened candidate items based on detection and screened candidate items based on tracking of the current frame by utilizing a pre-trained deep neural network;
s5: calculating the target matching degree of the adjacent frames by using a cascade matching algorithm, wherein the method comprises the following steps: the screened candidate items based on detection perform apparent feature similarity matching on the existing tracks of the adjacent frames; the screened candidate items based on the tracking are subjected to boundary frame intersection with a target boundary frame of the existing track of the adjacent frame and are matched with the matching degree;
s6: and determining the position of the target in the current frame according to the target matching degree of the adjacent frames, thereby outputting a target motion track.
2. The anti-occlusion target tracking method of claim 1, wherein the target detector is an SDP target detector.
3. The anti-occlusion target tracking method of claim 1, wherein the confidence is given by the following confidence score formula:
Figure FDA0002311749090000011
wherein the content of the first and second substances,
Figure FDA0002311749090000012
for the detection confidence of the t-1 th frame,
Figure FDA0002311749090000013
for the tracking confidence of the t-th frame, NdetThe number N of the candidate items based on detection in the track to be associatedtrkFor the number of candidates based on tracking in the last trajectory to be associated, I (-) is a binary function, when the function is true, the value is 1, otherwise, the value is 0, and the parameter α is a constant.
4. The anti-occlusion target tracking method of claim 1, wherein the step of screening the detection-based candidate and the tracking-based candidate using a non-maximum suppression algorithm to obtain the screened detection-based candidate and the screened tracking-based candidate comprises the steps of:
s21: sorting according to the confidence score of all the candidate items based on detection and the candidate items based on tracking to obtain a candidate list;
s22: selecting the detection-based candidate item and the tracking-based candidate item with the highest confidence level to be added into a final output list and deleted from the candidate item list;
s23: calculating the detection-based candidate item and the tracking-based candidate item with the highest confidence coefficient to be in a border intersection ratio with other candidate items, and deleting the detection-based candidate item and the tracking-based candidate item with the border intersection ratio larger than a preset threshold value;
s24: and repeating the process until the candidate item list is empty, wherein the final output list is the screened candidate items based on detection and the candidate items based on tracking.
5. The anti-occlusion target tracking method of claim 4, wherein the preset threshold is 0.3-0.5.
6. The anti-occlusion target tracking method of claim 1, wherein the deep neural network is a google lenet based network comprising from an input layer to an initiation-4 e layer, and then connected by a 1 x 1 convolutional layer.
7. The anti-occlusion target tracking method of claim 6, wherein a loss function of the training of the neural network is:
ltriplet(Ii，Ij，Ik)＝m+d(Ii，Ij)-d(Ii，Ik)
wherein, Ii，IjFor pictures from the same identity, Ii，IkFor pictures from different identities, d represents the euclidean distance and m is a constant.
8. The anti-occlusion target tracking method of claim 1, wherein calculating the target matching degree of adjacent frames using a cascade matching algorithm comprises the steps of:
s51: obtaining the target detection result of the first frame, and generating a track for each target to obtain an initial track set
Figure FDA0002311749090000021
Candidate item set composed of the filtered detection-based candidate items and the filtered tracking-based candidate items
Figure FDA0002311749090000022
The apparent characteristics
Figure FDA0002311749090000023
And constructing all matched candidate item sets
Figure FDA0002311749090000024
All candidate item sets that are not matched
Figure FDA0002311749090000025
S52: selecting the candidate item based on detection after screening
Figure FDA0002311749090000026
With the initial trajectory set
Figure FDA0002311749090000027
Calculating the feature similarity, and updating the matched candidate item set according to the matching result
Figure FDA0002311749090000031
The set of unmatched candidate items
Figure FDA0002311749090000032
The initial set of trajectories
Figure FDA0002311749090000033
S53: selecting the selected tracking-based candidate item
Figure FDA0002311749090000034
And the updated initial track set
Figure FDA0002311749090000035
The target bounding box carries out bounding box intersection and proportion matching, and the matched candidate item set is updated according to the matching result
Figure FDA0002311749090000036
The set of unmatched candidate items
Figure FDA0002311749090000037
9. The anti-occlusion target tracking method of claim 8, wherein the matched candidate item set is
Figure FDA0002311749090000038
Each candidate item bounding box in the set of initial trajectories and the initial trajectory set matched thereto
Figure FDA0002311749090000039
The track segments in (1) are connected; grouping the unmatched candidate items
Figure FDA00023117490900000310
InitialForming a new track; for the initial track set
Figure FDA00023117490900000311
The track section which is not matched in the initial track set is set as a temporary track section, if the track section is not matched in the following continuous N frames, the temporary track section is considered to be finished, and the temporary track section is selected from the initial track set
Figure FDA00023117490900000312
Deleting; and N is 5-8.
10. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 9.
CN201911261618.2A 2019-12-10 2019-12-10 Anti-occlusion target tracking method Active CN111080673B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201911261618.2A CN111080673B (en) 2019-12-10 2019-12-10 Anti-occlusion target tracking method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201911261618.2A CN111080673B (en) 2019-12-10 2019-12-10 Anti-occlusion target tracking method

Publications (2)

Publication Number Publication Date
CN111080673A true CN111080673A (en) 2020-04-28
CN111080673B CN111080673B (en) 2023-04-18

Family

ID=70313832

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201911261618.2A Active CN111080673B (en) 2019-12-10 2019-12-10 Anti-occlusion target tracking method

Country Status (1)

Country Link
CN (1) CN111080673B (en)

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112016445A (en) * 2020-08-27 2020-12-01 重庆科技学院 Monitoring video-based remnant detection method
CN112509338A (en) * 2020-09-11 2021-03-16 博云视觉（北京）科技有限公司 Method for detecting traffic jam event through silent low-point video monitoring
CN112734800A (en) * 2020-12-18 2021-04-30 上海交通大学 Multi-target tracking system and method based on joint detection and characterization extraction
CN112883819A (en) * 2021-01-26 2021-06-01 恒睿(重庆)人工智能技术研究院有限公司 Multi-target tracking method, device, system and computer readable storage medium
CN112990072A (en) * 2021-03-31 2021-06-18 广州敏视数码科技有限公司 Target detection and tracking method based on high and low dual thresholds
CN113223051A (en) * 2021-05-12 2021-08-06 北京百度网讯科技有限公司 Trajectory optimization method, apparatus, device, storage medium, and program product
CN113689462A (en) * 2020-05-19 2021-11-23 深圳绿米联创科技有限公司 Target processing method and device and electronic equipment
CN114972418A (en) * 2022-03-30 2022-08-30 北京航空航天大学 Maneuvering multi-target tracking method based on combination of nuclear adaptive filtering and YOLOX detection

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170286774A1 (en) * 2016-04-04 2017-10-05 Xerox Corporation Deep data association for online multi-class multi-object tracking
CN109919981A (en) * 2019-03-11 2019-06-21 南京邮电大学 A kind of multi-object tracking method of the multiple features fusion based on Kalman filtering auxiliary

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170286774A1 (en) * 2016-04-04 2017-10-05 Xerox Corporation Deep data association for online multi-class multi-object tracking
CN109919981A (en) * 2019-03-11 2019-06-21 南京邮电大学 A kind of multi-object tracking method of the multiple features fusion based on Kalman filtering auxiliary

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
周维;陈听海;邱宝鑫: "引入特征重检的抗遮挡目标跟踪方法研究" *

Cited By (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113689462A (en) * 2020-05-19 2021-11-23 深圳绿米联创科技有限公司 Target processing method and device and electronic equipment
CN112016445A (en) * 2020-08-27 2020-12-01 重庆科技学院 Monitoring video-based remnant detection method
CN112509338A (en) * 2020-09-11 2021-03-16 博云视觉（北京）科技有限公司 Method for detecting traffic jam event through silent low-point video monitoring
CN112734800A (en) * 2020-12-18 2021-04-30 上海交通大学 Multi-target tracking system and method based on joint detection and characterization extraction
CN112883819A (en) * 2021-01-26 2021-06-01 恒睿(重庆)人工智能技术研究院有限公司 Multi-target tracking method, device, system and computer readable storage medium
CN112883819B (en) * 2021-01-26 2023-12-08 恒睿(重庆)人工智能技术研究院有限公司 Multi-target tracking method, device, system and computer readable storage medium
CN112990072A (en) * 2021-03-31 2021-06-18 广州敏视数码科技有限公司 Target detection and tracking method based on high and low dual thresholds
CN113223051A (en) * 2021-05-12 2021-08-06 北京百度网讯科技有限公司 Trajectory optimization method, apparatus, device, storage medium, and program product
CN114972418A (en) * 2022-03-30 2022-08-30 北京航空航天大学 Maneuvering multi-target tracking method based on combination of nuclear adaptive filtering and YOLOX detection
CN114972418B (en) * 2022-03-30 2023-11-21 北京航空航天大学 Maneuvering multi-target tracking method based on combination of kernel adaptive filtering and YOLOX detection

Also Published As

Publication number Publication date
CN111080673B (en) 2023-04-18

Similar Documents

Publication Publication Date Title
CN111080673B (en) Anti-occlusion target tracking method
CN109360226B (en) Multi-target tracking method based on time series multi-feature fusion
CN110516556B (en) Multi-target tracking detection method and device based on Darkflow-deep Sort and storage medium
US11094070B2 (en) Visual multi-object tracking based on multi-Bernoulli filter with YOLOv3 detection
Sun et al. A survey of multiple pedestrian tracking based on tracking-by-detection framework
Feng et al. Dynamic saliency-aware regularization for correlation filter-based object tracking
Zhou et al. Improving video saliency detection via localized estimation and spatiotemporal refinement
Xiang et al. Online multi-object tracking based on feature representation and Bayesian filtering within a deep learning architecture
WO2009109127A1 (en) Real-time body segmentation system
CN111862145B (en) Target tracking method based on multi-scale pedestrian detection
CN110991397B (en) Travel direction determining method and related equipment
CN110349188B (en) Multi-target tracking method, device and storage medium based on TSK fuzzy model
Kim et al. Multiple player tracking in soccer videos: an adaptive multiscale sampling approach
Al-Shakarji et al. Robust multi-object tracking with semantic color correlation
Iraei et al. Object tracking with occlusion handling using mean shift, Kalman filter and edge histogram
CN112541441A (en) GM-PHD video multi-target tracking method fusing related filtering
CN111161325A (en) Three-dimensional multi-target tracking method based on Kalman filtering and LSTM
Naik et al. YOLOv3-SORT: detection and tracking player/ball in soccer sport
Poiesi et al. Tracking multiple high-density homogeneous targets
Lu et al. Multi-target tracking by non-linear motion patterns based on hierarchical network flows
Lit et al. Multiple object tracking with gru association and kalman prediction
Wan et al. Tracking beyond detection: learning a global response map for end-to-end multi-object tracking
Khan et al. Foreground detection using motion histogram threshold algorithm in high-resolution large datasets
Saada et al. A multi-object tracker using dynamic Bayesian networks and a residual neural network based similarity estimator
Truong et al. Single object tracking using particle filter framework and saliency-based weighted color histogram

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant