CN111368938A - Multi-target vehicle tracking method based on MDP - Google Patents
Multi-target vehicle tracking method based on MDP Download PDFInfo
- Publication number
- CN111368938A CN111368938A CN202010196916.4A CN202010196916A CN111368938A CN 111368938 A CN111368938 A CN 111368938A CN 202010196916 A CN202010196916 A CN 202010196916A CN 111368938 A CN111368938 A CN 111368938A
- Authority
- CN
- China
- Prior art keywords
- target vehicle
- state
- tracking
- target
- mdp
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 75
- 238000012549 training Methods 0.000 claims abstract description 57
- 230000008569 process Effects 0.000 claims abstract description 44
- 230000007704 transition Effects 0.000 claims abstract description 24
- 230000004913 activation Effects 0.000 claims abstract description 12
- 230000002787 reinforcement Effects 0.000 claims abstract description 12
- 238000001514 detection method Methods 0.000 claims description 50
- 230000003287 optical effect Effects 0.000 claims description 31
- 230000006870 function Effects 0.000 claims description 22
- 238000005070 sampling Methods 0.000 claims description 20
- 230000009471 action Effects 0.000 claims description 16
- 230000000875 corresponding effect Effects 0.000 claims description 15
- 238000012360 testing method Methods 0.000 claims description 13
- 239000013598 vector Substances 0.000 claims description 13
- 238000012886 linear function Methods 0.000 claims description 5
- 238000005457 optimization Methods 0.000 claims description 4
- 238000004364 calculation method Methods 0.000 claims description 3
- 230000005484 gravity Effects 0.000 claims description 3
- 238000010606 normalization Methods 0.000 claims description 3
- 230000000391 smoking effect Effects 0.000 claims description 3
- 230000008685 targeting Effects 0.000 claims description 3
- 230000003213 activating effect Effects 0.000 claims description 2
- 230000001276 controlling effect Effects 0.000 claims description 2
- 238000005259 measurement Methods 0.000 claims description 2
- 230000000694 effects Effects 0.000 abstract description 9
- 238000005516 engineering process Methods 0.000 abstract description 5
- 238000012546 transfer Methods 0.000 description 5
- 241001239379 Calophysus macropterus Species 0.000 description 4
- 230000008859 change Effects 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 238000012545 processing Methods 0.000 description 4
- 238000004458 analytical method Methods 0.000 description 3
- 238000010276 construction Methods 0.000 description 2
- 230000007613 environmental effect Effects 0.000 description 2
- 238000012706 support-vector machine Methods 0.000 description 2
- 208000033962 Fontaine progeroid syndrome Diseases 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000001364 causal effect Effects 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 238000005314 correlation function Methods 0.000 description 1
- 238000002790 cross-validation Methods 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 230000001788 irregular Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 238000009877 rendering Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 230000001629 suppression Effects 0.000 description 1
- 238000010200 validation analysis Methods 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/41—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items
- G06V20/42—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items of sport video content
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/07—Target detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/08—Detecting or categorising vehicles
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Bioinformatics & Computational Biology (AREA)
- General Engineering & Computer Science (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a multi-target vehicle tracking method based on Markov Decision Process (MDP). A target vehicle is obtained through a video sequence and detected, and the target vehicle enters an activation state; wherein the state of the target vehicle includes: an active state, a tracking state, an inactive state, and a lost state; training a second classifier for the target vehicle in the activated state in an off-line mode, and judging the state transition of the target vehicle according to the second classifier; for the target vehicle entering the tracking state, constructing an appearance template for the target vehicle in an online mode, and judging the state transition of the target vehicle according to the appearance template; for the target vehicle in the lost state, the state transition of the target vehicle is judged by the reinforcement learning of the data association two classifiers; the method can overcome the main technical difficulties faced by the vehicle tracking technology, reduce the tracking error probability and improve the tracking effect.
Description
Technical Field
The invention relates to the technical field of target tracking, in particular to a multi-target vehicle tracking method based on MDP.
Background
According to the characteristics of the input video sequence when the target tracking algorithm operates, the input data can be divided into fixed camera shooting data and moving camera shooting data. For data obtained by shooting through a fixed camera, the background of a shooting scene is almost unchanged because a shooting point is fixed, and the most obvious change in the whole input data is the target to be tracked, so the data has low algorithm requirements. For the data obtained by the moving camera, the shooting point moves at the moment, which causes the whole shooting picture to be in the non-stop motion, and can cause great challenge to the correct operation of the tracking algorithm. Environmental factors also present more challenges to vehicle tracking technology during the shooting process. The environmental factor not only includes the influence of outside weather environment vehicle definition of shooing in to the video, and more importantly the vehicle probably appears irregular motion in the driving process, and this can cause the influence to vehicle self characteristic, and the main technical difficulty that vehicle tracking technology faced includes: the algorithm has high time complexity, fast vehicle movement, camera movement, vehicle occlusion, vehicle dimension change and similar object interference, and the technical difficulties can put high requirements on the tracking algorithm.
Furthermore, the realization of multi-target tracking through video sequences is an important subject in the field of computer vision, and can be widely applied to various video analysis scenes, such as robot navigation, automatic driving, video monitoring, motion analysis and the like. In recent years, the field of multi-target tracking mainly uses a strategy of realizing tracking through detection, and in order to solve the problems of detection target data association errors and detection failures which may occur in the tracking process, a plurality of tracking algorithms adopt a batch processing mode for an input image sequence. Batch processing means that images at a future time are also used to solve the problem of data association anomalies when tracking is performed, i.e. the tracking algorithm is a non-causal system. Therefore, the use scenarios of such multi-target tracking algorithms are limited, and the multi-target tracking algorithms cannot be applied to the application scenarios such as robot navigation and automatic driving. The main challenge of achieving multi-target vehicle tracking in an online manner is how to correlate the current detection result with the previous target in a data manner. The basis for realizing data association is to calculate a similarity function between the detection result and the target, and the probability of errors in data association can be greatly reduced by introducing different information such as vehicle appearance, motion state, position and the like when the similarity function is calculated. Many of the previous online multi-vehicle tracking algorithms mainly adopt a heuristic method to select a parameter model of a similar function, and then adjust parameters in a cross validation mode, which is not suitable for the situation with more features and cannot effectively ensure the generalization capability of the algorithms.
Many technicians have recently attempted to add learning capabilities to multi-objective vehicle tracking algorithms; according to the sequence of the algorithm learning process and the tracking process, the algorithm can be divided into an off-line learning tracking algorithm and an on-line learning tracking algorithm. For the offline learning algorithm, the learning process occurs before actual tracking, and the algorithm needs to learn a similar function in advance according to real calibration to realize data association. As a result, the off-line learning tracking algorithm is static, and it cannot effectively use dynamic information in the input image sequence and past state information of the tracking target in the data association process, and these pieces of information have very important significance for improving the data association effect, especially playing a decisive role in the case that the vehicle disappears or is partially occluded for a short time. The corresponding online learning tracking algorithm learns while tracking. A strategy commonly adopted by an online learning tracking algorithm is to determine a tracked target vehicle through real calibration data, and then to establish a positive training data set and a negative training data set for the target vehicle respectively so as to train a similar function used in data association. The tracking algorithm of online learning fully applies dynamic information in an image sequence and past state information of a tracking target, but no other real calibration data can be referred to in the tracking process, so that tracking errors are easily accumulated, and the phenomenon of tracking drift is caused finally due to tracking failure.
Therefore, how to overcome the main technical difficulties faced by the vehicle tracking technology, reduce the tracking error probability, and improve the tracking effect becomes a technical problem to be solved urgently by practitioners in the same industry.
Disclosure of Invention
The invention aims to provide an MDP-based online multi-target tracking algorithm, which solves the main technical difficulties and poor tracking effect of the vehicle tracking technology.
The MDP (Markov decision) process is an optimal decision process of a random dynamic system, and a decision maker analyzes and selects from a plurality of available decisions according to the current state of the system at each moment; when the decision maker takes a decision, the system will receive a reward, which is related to the system state and the decision and affects the state of the system at the next moment; and (4) in the state of the system at the next time point, a decision maker still needs to observe the state of the system and take a decision according to the process, and the process is repeated until the end condition is met.
In order to solve the above technical problem, an embodiment of the present invention provides an MDP-based multi-target vehicle tracking method, including:
s1, acquiring a target vehicle through a video sequence, detecting the target vehicle, and activating the target vehicle to enable the target vehicle to be in an activated state; wherein the state of the target vehicle includes: an active state, a tracking state, an inactive state, and a lost state;
s2, training a second classifier for the target vehicle in the activated state in an off-line mode, and judging the state transition of the target vehicle according to the second classifier; if the tracking state is transferred, the step S3 is carried out, otherwise, the activation state is entered, and the tracking is stopped;
s3, constructing an appearance template for the target vehicle entering the tracking state in an online mode, and judging the state transition of the target vehicle according to the appearance template; if the state is transferred to the lost state, performing step S4, otherwise, continuing to perform step S3;
s4, for the target vehicle in the lost state, the state transition of the target vehicle is judged by the reinforcement learning of a data association two classifier; stopping tracking when the state is switched to the inactive state, performing step S3 when the state is switched to the tracking state, and continuing to perform step S4 when the state is lost;
s5, tracking the target vehicle through the steps S1 to S4 until the target vehicle is in an inactive state or tracks the target vehicle to the last frame of a video sequence.
In one embodiment, the step S2 includes:
feature vector phi through five dimensionsActive(s) training the classifiers to obtain:
through training the obtained classifier, the learned reward function in the activated state is as follows:
wherein (w)Active,bActive) Representing the SVM hyperplane in the activated state; if y (a) is +1, the corresponding action a is a1The target vehicle enters a tracking state in an activation state; if y (a) is equal to-1, then corresponding action a is equal to a2When the target vehicle is in the activated state, the target vehicle enters the non-activated state; the five-dimensional feature vector phiActive(s) includes coordinates in two dimensions, width, height and detection score.
In one embodiment, the step S3 includes:
s31, initializing the data of the target template when the target vehicle is transferred to the tracking state and becomes the tracking target vehicle;
s32, uniformly and densely acquiring a sampling point set in the target template through an optical flow tracking algorithm, and calculating an optical flow from the sampling point set to a new frame;
s33, after optical flows of all the sampling point sets are calculated, calculating the median of forward and reverse errors of the sampling point sets in the target template as a decision-making measurement standard;
the forward and reverse errors of the sampling point set in the target template are pre-estimated by using the original sampling point set and the forward and reverse errorsAnd (3) representing the Euclidean distance between the sampling point sets: e (u) | | u-u' | non-smoking circuitry2，：emedFB＝median({e(ui)}n ＝1) Representing the median of the forward and reverse errors of the sample point set, wherein n represents the number of the point sets; said emedFBA metric for making a decision; wherein u represents a set of sample points within the target template and u' represents a set of predicted sample points for the target template;
s34, calculating the average boundary box overlapping rate of the target vehicle in the past K frames:
said omeanTo make a decision, another metric, the reward function in the tracking state is finally defined as:
wherein e is0And o0Indicates a preset threshold, and if y (a) +1, corresponds to action a ═ a3If y (a) is equal to-1, the corresponding action a is equal to a4The target vehicle will transition to the lost state when e of said target vehiclemedFBLess than a predetermined threshold value omeanIf the target vehicle is larger than the preset threshold value, the target vehicle is continuously in the tracking state, otherwise, the target is transferred to the loss state.
In one embodiment, the step S4 includes:
the determination of the state transition of the target vehicle in step S4 includes:
using a real-valued linear function f (t, d) of wTCompleting two classifications by phi (t, d) + b, wherein (w, b) represents a parameter for controlling a linear function, and phi (t, d) represents a feature vector of similarity between the target vehicle and the detection result; in the lost state, the reward function for the data association problem is:
the parameters (w, b) are obtained by two-classifier reinforcement learning, and if y (a) is +1, the corresponding action a is a6If y (a) is equal to-1, corresponding to action a is equal to a5The target vehicle keeps a lost state, and M represents the number of detection results needing to be matched in the data association process;
if the time that the target vehicle is in the loss state is more than TLostIf the target is marked as inactive and the tracking is stopped, TLostRepresenting a preset fixed threshold.
In one embodiment, the two classifier reinforcement learning process comprises:
training the training algorithm through all targets in the video sequence, tracking the targets according to a strategy pi in the MDP, and taking different actions by the MDP according to real calibration;
when the MDP generates errors in the data association process, reconstructing a training data set S, and retraining the secondary classifier according to the new data set S;
updated training data set
Classifier for obtaining maximum interval by solving soft interval optimization problem in data association
ξ thereinkK is 1, …, M is a relaxation variable, C is a normalization parameter;
after the classifier is updated, the parameters (w ', b') and the strategy pi are updated and used in the next round of training process; the training algorithm is continuously circulated and the strategy is updated until all target vehicles are successfully tracked;
in one embodiment, the feature vector in step S4 includes:
FB error: respectively representing the whole, the left half part, the right half part and the upper half part of the target template, wherein the average forward and reverse errors when the lower half part calculates the optical flow;
NCC: calculating the mean value of the median of the normalized correlation coefficient of the image blocks around the matching point in the optical flow; or normalized correlation coefficients between image blocks from the detection results and bounding boxes from the optical flow calculations;
height ratio: the average value of the height ratios of all detection results and the bounding box obtained by an optical flow method; or the height ratio between the target and the detection result;
the overlapping rate: the overlapping rate between the detection result and the boundary frame calculated by the optical flow method;
scoring: a normalized detection score;
distance: the Euclidean distance between the tracking target and the detection is based on the center of gravity.
In one embodiment, the reconstructing the training data set comprises:
when MDP is in process of the ith frame viJ-th target vehicle t in (1)ijTracking, and the MDP entering the lost state at frame l, takes into account two important errors:
(1) MDP has been targeting vehicles
And the detection result dkPerforming correlation, but is indeed wrong according to the true calibration; the target is erroneously associated with the detection result,
adding a training set S as a negative training sample;
(2) the MDP does not correlate the target vehicle with any test results, the target is visible and correctly detected by the detector according to the actual calibration,
the training set S will be added as a negative training sample.
The invention has the advantages that the invention provides an online multi-target tracking algorithm based on MDP, which divides the state space of the MDP into four sub-state spaces of activation, tracking, loss and non-activation according to various states of a target possibly appearing in the target tracking process, so that when the tracked target appears, disappears for a short time and appears again, the state transition among the four sub-state spaces in the MDP is corresponded, the similar function learned in the data association process for realizing correct tracking uses the strategy learned in the MDP to replace, and the strategy is learned by adopting the reinforcement learning mode, the advantages of the online learning tracking algorithm and the offline learning tracking algorithm can be effectively integrated, the proposed algorithm has sufficient rationality on model construction and feature selection, and the multi-target tracking algorithm provided by the invention has good effects on time complexity and tracking effect, the tracked target has higher priority than the target in a lost state in the tracking process, and the detection result blocked by the tracked target in the data association process is inhibited to reduce the error probability.
Additional features and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. The objectives and other advantages of the invention will be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.
The technical solution of the present invention is further described in detail by the accompanying drawings and embodiments.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description serve to explain the principles of the invention and not to limit the invention. In the drawings:
FIG. 1 is a flow chart of an MDP-based online multi-target tracking algorithm according to an embodiment of the present invention;
fig. 2 is a schematic state space diagram of an MDP tracking algorithm according to an embodiment of the present invention;
fig. 3 is a flowchart of step S3 according to an embodiment of the present invention;
FIG. 4 illustrates a forward error and a reverse error provided by an embodiment of the present invention;
FIG. 5 is a block diagram of determining a boundary by an optical flow method according to an embodiment of the present invention;
FIG. 6 is a comparison graph of data of the impact of different numbers of templates on the performance index of the tracking algorithm according to the embodiment of the present invention;
FIG. 7 is a graph comparing the performance of different functions in the traceback removal algorithm provided by an embodiment of the present invention;
fig. 8 is a data diagram for comparing the performance of the MDP multi-target tracking algorithm in the KITTI data set according to the embodiment of the present invention;
fig. 9 is a data diagram of a performance comparison graph of the MDP multi-target tracking algorithm in the MOT data set according to the embodiment of the present invention.
Detailed Description
Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
The embodiment of the invention provides an MDP-based multi-target vehicle tracking method, which is shown in the reference of fig. 1-2 and comprises the following steps:
s1, acquiring a target vehicle through a video sequence, detecting the target vehicle, and enabling the target vehicle to enter an activated state; wherein the state of the target vehicle includes: an active state, a tracking state, an inactive state, and a lost state;
s2, training a second classifier for the target vehicle in the activated state in an off-line mode, and judging the state transition of the target vehicle according to the second classifier; if the tracking state is transferred, the step S3 is carried out, otherwise, the activation state is entered, and the tracking is stopped;
s3, constructing an appearance template for the target vehicle entering the tracking state in an online mode, and judging the state transition of the target vehicle according to the appearance template; if the state is transferred to the lost state, performing step S4, otherwise, continuing to perform step S3;
s4, for the target vehicle in the lost state, the state transition of the target vehicle is judged by the reinforcement learning of a data association two classifier; stopping tracking when the state is switched to the inactive state, performing step S3 when the state is switched to the tracking state, and continuing to perform step S4 when the state is lost;
s5, tracking the target vehicle through the steps S1 to S4 until the target vehicle is in an inactive state or tracks the target vehicle to the last frame of a video sequence.
In this embodiment, in step S1, the Active state is any initial state of the tracking target vehicle, and when an object is detected by the detector, it first enters the Active state.
In step S2, the tracking target vehicle in the activated state may shift to the tracking or deactivated state. Under ideal conditions, the target vehicle from the detector should enter a tracking state if it is a positive sample, i.e. for vehicle tracking problems, it should transition from an active state to a tracking state if the detected target vehicle is indeed a vehicle, which may be one of a car, a bicycle and a pedestrian, and to an inactive state if it is a negative sample, i.e. it should enter an inactive state if the detected target vehicle is not a car, such as a container similar in size to a vehicle on a road surface, a projection of a vehicle onto a road surface, etc.
In step S3, in the tracking state, a target in the tracking state may keep tracking all the time or fail to track due to some reason, and transition to a lost state, such as other objects being blocked or temporarily disappearing from the camera capture area.
In step S4, in the loss (Lost) state, the target vehicle in the Lost state may continue to be in the Lost state or transition to the tracking state because it occurs again, or transition to the Inactive state because it is in the Lost state for a sufficiently long time, and the final Inactive (Inactive) state is the final state of all the tracking target vehicles, that is, the target vehicle in the Inactive state will always be in the Inactive state.
In step S5, by the steps S1-S4, the tracking is completed for the target vehicle tracking to the last frame of the video sequence or the target vehicle being in the inactive state.
In one embodiment, in step S2, in the activated state SActiveThe MDP decides whether the detected target vehicle is transferred to a tracking state or an inactive state, the transfer to the tracking state indicates that the detected target vehicle is determined to be the target vehicle to be tracked, and the transfer to the inactive state realizes effective processing of error detection; the decision process can be considered as a preprocessing process carried out before formal tracking, non-maximum value suppression and division of score threshold values of the detected target vehicle are mostly adopted in the traditional tracking method for processing, and an SVM is used for carrying out classification judgment in the invention;
using SVM techniques in the present invention, a five-dimensional feature vector phi is usedActive(s) training a secondary classifier in an off-line mode to realize the judgment of the target vehicle transferring to a tracking or non-activation state; five-dimensional feature vector phiActive(s) including coordinates, width, height and detection scores in two dimensions, which training data is obtainable from a training video sequence; for such a binary classification problem, to ensure efficient classification and ensure the maximum classification interval, i.e. sufficient generalization ability, it can be expressed as:
this is equivalent to the reward function learned in the active state:
wherein (w)Active,bActive) Defining an SVM hyperplane in an activated state; if y (a) +1, the corresponding action a ═ a1If y (a) is equal to-1, the operation corresponds to a2(ii) a a denotes an action that the target vehicle can perform, a1Representing that the target vehicle is executed with the action when the target vehicle is converted from the activated state to the non-activated state; a is2Representing that the target vehicle is executed when the target vehicle is converted from the activation state to the tracking state;
the invention uses SVM to realize binary classification discrimination, which shows many advantages in solving the problems of nonlinearity and small sample identification, mainly expressed in the following aspects:
(1) the method is characterized in that the complexity of the classifier is determined by the number of the support vectors only, so that the fast training speed can be ensured under the condition of more training data, and the sufficient generalization capability can be ensured under the condition of less training data.
(2) And (4) global optimization. The SVM can classify linear and nonlinear problems by solving an optimal hyperplane. The SVM converts data in the low-dimensional space into the high-dimensional space, so that the hyperplane in the high-dimensional space actually corresponds to the nonlinear classification surface in the low-dimensional space, and the problem of finding the optimal hyperplane can be converted into a quadratic programming problem by using a Lagrange multiplier method to realize the solution of the global optimal hyperplane. The finally obtained global optimal hyperplane can ensure to obtain higher accuracy during classification.
(3) The generalization ability is strong. The SVM is based on a statistical learning theory, a structural risk minimization principle is adopted to replace a traditional experience risk minimization principle, reasonable balance can be carried out between experience risks and model complexity, and generalization capability is improved as much as possible. Even with decision rules derived from fewer training samples, a smaller error can still be derived for an unknown input test set. The generalization capability of the SVM is strong, so that the classifier based on the SVM is effective to various types of data as well as specific class targets. In this context, it is shown that it is not only able to discriminate vehicles, but also effective to discriminate pedestrians and even bicycles.
(4) The learning and prediction time is short. For the same type of problems, the training time and the prediction time of the neural network are 34 times different compared with those of the support vector machine, and when the number of training samples is increased, the advantages of the support vector machine algorithm in terms of time complexity and space complexity are more obvious, so that the SVM has a wider application prospect.
As shown in fig. 3, in one embodiment, in the tracking state, the MDP needs to decide whether to keep track of the target vehicle or to transfer the target vehicle to the loss state. The tracking algorithm should keep track all the time as long as the target vehicle is not completely occluded or is out of the camera's capture area, otherwise the target vehicle should be marked as a lost state target vehicle. The decision making process is consistent with the idea of the single-target vehicle tracking algorithm, and is inspired by the single-target vehicle tracking algorithm, in the invention, an appearance template is constructed for the target vehicle in an online mode, and tracking is realized according to the template; if the appearance template can successfully track the target vehicle in the next frame, the target vehicle is kept in a tracking state, otherwise, the target vehicle is transferred to a loss state;
the appearance of the tracking target vehicle is represented by a template, and the template is an image block of the video containing the tracking target vehicle. When a detection result is transferred to a tracking state to become a tracking target vehicle, the target template is initialized by using data in a detection result bounding box (bounding box). When the target vehicle is in the tracking state, the MDP collects templates of tracking targets as the tracking history of the target vehicle, and the templates of the targets are used for judging whether the target vehicle can be transferred from the loss state to the tracking state or not in the loss state;
in order to realize tracking through the target template, a tracking algorithm uniformly and densely acquires sampling points in the target template and calculates the optical flow from the sampling points to a new frame; the set of points for i on the target template, u ═ is known (u)x,uy) The algorithm calculates the corresponding position v ═ u + d ═ on the next frame j by the LK optical flow method (u ═ u + d ═x+dx,uy+dy) Wherein d ═ dx+dy) Is the optical flow at u.
As shown in FIG. 4, after calculating the optical flows of all sampling points, the algorithm evaluates the predicted stability by calculating forward and reverse errors; knowing the prediction v of the set of points u in the target template, the algorithm can derive a new prediction u' by computing its inverse optical flow against the set of predicted points v of the target template. If under the condition of using the optical flow method to stabilize the prediction, u and u' should be very close, so the forward-backward error of the point set in the target template is represented by the Euclidean distance between the original point set and the forward-backward prediction point set: e (u) | | u-u' | non-smoking circuitry2And the stability of the tracking target vehicle uses the median of the forward and reverse errors of all sampling points: e.g. of the typemedFB＝median({e(ui)}n ＝1) Representing, wherein n is the number of point sets; e.g. of the typemedFBA metric for making a decision;
if emedFBIf the target vehicle boundary frame is larger than a certain threshold value, the tracking process is considered to be unstable, otherwise, the algorithm filters out points with excessive forward and backward errors in the point set according to a specific threshold value, then the boundary frame of the target vehicle is predicted by using the remaining stable matching in the point set, and the newly obtained boundary frame of the target vehicle is considered to be the new position of the tracked target vehicle.
As shown in FIG. 5, the quality of the optical flow is an important criterion for determining whether to continue tracking the target vehicle, howeverThe decision making by optical flow alone is a great risk because a target vehicle being tracked may be a false detection from a detector whose appearance is constant, such as a container similar in size to the vehicle on the road, a projection of the vehicle on the road, etc. In such a case, the optical flow tracking algorithm will continue to track the wrong target vehicle. In order to solve such a situation, it is highly likely that a target vehicle being tracked is a false detection if it does not match the result of detection for a long period of time, considering that it is impossible for the detector to continuously detect a false detection target vehicle. Therefore, the algorithm checks the historical state of the target vehicle and calculates the average boundary box overlapping rate of the target vehicle in the past K frames:
as another metric for decision making, the reward function in tracking state s is finally defined as:
wherein e is0And o0Represents a predetermined threshold value, omeanIf y (a) is +1, the target vehicle corresponds to an action a ═ a3，a3Indicating that the target vehicle is performing an action while the target vehicle is in a tracking state; if y (a) is equal to-1, the corresponding action a is equal to a4，a4Representing that the target vehicle is executed when the target vehicle in the tracking state is converted into the loss state; when tracking target vehicle emedFBLess than a predetermined threshold value omeanIf the target vehicle is larger than the preset threshold value, the target vehicle can be continuously tracked in the tracking state, otherwise, the target vehicle can be transferred to the loss state.
In the tracking process, the appearance of the tracked target vehicle is constantly in the process of changing due to various factors. The contour template of the target vehicle therefore needs to be continuously updated to accommodate the contour changes of the tracked target vehicle. For a common tracking algorithm, after a tracker successfully tracks a target vehicle, a template is continuously updated to adapt to the change of the vehicle appearance, and as a result, an accumulated error is generated in the template updating process to cause a drift phenomenon.
In the present invention, a slow update scheme is used in combination with the detection result of the detector to prevent the tracking drift phenomenon from occurring. Specifically, the target template remains unchanged when the tracking algorithm is able to successfully track the target vehicle, and when the target vehicle profile changes too much, resulting in template tracking failure, the tracked target vehicle will be transferred to the lost state, and a new tracking template is obtained when the target vehicle successfully transfers from the lost state to the tracking state. Meanwhile, the tracking algorithm saves K templates as historical templates of the tracking target vehicle, the tracking template in use is one of the K historical templates, but the tracking template in use is not necessarily the last of the K historical templates because the tracking algorithm adopts a slow updating strategy, and the K historical templates are also used for data association in a lost state. The tracking error is not accumulated in the tracking process, but the appearance change problem of the tracking target vehicle is solved by means of data association in a lost state.
In one embodiment, in the loss state, the MDP needs to decide whether to transfer the target vehicle to the tracking state or continue to leave the target vehicle in the loss state, or mark the target vehicle as inactive. The problem of transitioning to an inactive state is simple, if a target vehicle is in a lost state for a period of time greater than TLostThe target vehicle is directly marked as inactive and tracking is stopped, wherein TLostFor a predetermined fixed threshold, e.g. TLostAnd may be 3 seconds. The real challenge is to determine whether the target vehicle should transition to a tracking state or continue to be in a lost state, which is considered a data correlation problem in the present invention. In order to shift the target vehicle from the loss state to the tracking state, the target vehicle must match one of the detection results obtained by the detector, otherwise the target vehicle will continue to be in the loss state.
For the data correlation problem, t denotes a target vehicle in a loss state, d denotes a detection result obtained by a detector, the target vehicle of the tracking algorithm is a label y ∈ { +1, -1} of a predicted data pair (t, d), if y { +1 denotes that the data pair matches, otherwise y ═ 1 denotes that the data pair does not matchTCompleting two classifications by phi (t, d) + b, wherein (w, b) is a parameter of a control linear function, and phi (t, d) is a feature vector representing the similarity between the target vehicle and the detection result; in the lost state, the reward function for the data association problem is:
if y (a) ═ 1, the action a ═ a6 is corresponded, a6 indicates that the target vehicle is executed when the target vehicle is converted from the loss state to the tracking state, if y (a) ═ 1, the action a ═ a5 is corresponded, a5 indicates that the target vehicle is executed when the target vehicle is kept in the loss state, and M represents one of detection results which need to be matched and appear in the data association process; therefore, the learning of the state to the strategy pi is simplified into the learning of the parameters (w, b) in the decision function;
in the invention, a two-classifier is trained in a reinforcement learning mode; by using
Representing the video sequence used for training, N representing the length of the video sequence, assuming that v is the ith frameiIn which is NiIndividual real calibration target
The final goal is to train the MDP to successfully track all target vehicles.
The training process starts with initial parameters (w)0,b0) Harmony training set
Initially, when the parameters (w ', b') of the two classifiers are determined, the MDP has a complete set of strategies pi, the objective of which is to maximize the number of states in a given stateGeneralizing the reward after performing action a. Therefore, the training algorithm trains through all target vehicles in the video sequence, and tracks the target vehicles according to the existing strategy pi in the MDP, and at the moment, the MDP takes different actions according to the real calibration. Updating the policy pi or the parameter (w ', b') only when the MDP generates an error in the data association process; when the MDP is working on the jth target vehicle t in the ith frame viijTracking, and at the l-th frame the MDP goes into the missing state, two important errors are considered:
(1) MDP has been targeting vehicles
And the detection result dkPerforming correlation, but is indeed wrong according to the true calibration; the target vehicle is erroneously associated with the detection result,
adding a training set S as a negative training sample;
(2) the MDP does not correlate the target vehicle with any detection results, the target vehicle is visible and correctly detected by the detector according to the true calibration,
the training set S will be added as a negative training sample.
After the reconstruction of the training data set S is completed, the secondary classifier is retrained according to the new data set S;
updated training data set
Obtaining a classifier with a maximum interval by solving a soft interval optimization problem during data association;
ξ thereinkK is 1, …, M is a relaxation variable, C is a normalization parameter; after the classifier is updated, the parameters (w ', b') and the strategy pi are updated and used in the next round of training process; the training process continues to loop and update the strategy until all target vehicles are successfully tracked.
Wherein, the used feature vectors include:
FB error: respectively representing the whole, the left half part, the right half part and the upper half part of the target template, wherein the average forward and reverse errors when the lower half part calculates the optical flow;
NCC: calculating the mean value of the median of the normalized correlation coefficient of the image blocks around the matching point in the optical flow; or normalized correlation coefficients between image blocks from the detection results and bounding boxes from the optical flow calculations;
height ratio: the average value of the height ratios of all detection results and the bounding box obtained by an optical flow method; or the height ratio between the target vehicle and the detection result;
the overlapping rate: the overlapping rate between the detection result and the boundary frame calculated by the optical flow method;
scoring: a normalized detection score;
distance: and the Euclidean distance between the tracking target vehicle and the detection result is based on the gravity center.
In order to evaluate the algorithm proposed in the present invention, the present invention performs experimental verification on the algorithm, and includes the following aspects:
with respect to the test data set and the evaluation index:
the experimental phase uses the KITTI and MOT target tracking data sets. The KITTI target tracking data set consists of about 19000 frames (about 32 minutes) of images. The data set consisted of 21 training and 29 test video sequences, all recorded by a camera mounted above the moving vehicle. Each video sequence was recorded at 10FPS and the video sequence frame number was not fixed (containing from 78 to 1176 frames) and each sequence had a variable number of tracked objects (including cars, pedestrians, cyclists).
The MOT data sets consisted of 111286 frames (approximately 16.5 minutes) and all had different FPSs. MOT data sets are widely used video sequences and some new challenging sequences collected by MOT communities, which are then divided into training and test sets, each with 11 sequences. Since the true calibration of the test set is not provided, the validation set of 6 sequences out of the 11 training sequences is taken out separately for analyzing the tracking algorithm proposed by the present invention.
The data set selected by the invention is widely used at present and very challenging, and mainly represents three aspects:
(1) the scene is very complicated, the phenomenon that the tracking target is partially shielded and disappears temporarily exists, and the video shooting environment is complicated.
(2) The camera is not fixed, the background environment can be changed greatly, and meanwhile, the tracking target can be blurred, so that a great challenge is brought to a tracking algorithm.
(3) Tracking targets occur anywhere in the video, rendering many conventional techniques employed in multi-target tracking unsuitable for such situations, such as fixed entry/exit locations, background subtraction, etc.
Multiple indicators are used in the present invention to evaluate the performance of a multi-target vehicle tracking algorithm. These include multi-Object Tracking Accuracy (abbreviated MOTA), multi-Object Tracking Precision (abbreviated MOTP), primary Tracking target (Mostly Track, abbreviated MT, indicating that the Tracking result is more than 80% identical to the real calibration result), primary missing target (Mostly Lost, abbreviated ML indicating that the Tracking result is less than 20% identical to the real calibration result), number of positive samples (FP), number of negative samples (FN), number of ID exchanges (IDs), total number of times the Track is segmented (Frag), and number of processed video sequences in one second (Hz). The meanings of the indexes are intuitively understood, the higher the MOTA, MOTP, MT and HZ is, the lower the ML, FP, FN and IDS are, the better the performance of the tracking algorithm is represented;
for experimental results and analysis:
firstly, analyzing the influence of the number of historical templates on the performance of the tracking algorithm in a lost state based on the MDP multi-target tracking algorithm, and determining the optimal number of the historical templates through experiments. Then, considering that the tracking algorithm provided by the invention comprises a plurality of modules in order to realize effective tracking of multiple targets, the algorithm also relates to a plurality of parameters and characteristics in each module, and in order to verify the correctness of the algorithm using model and the selected characteristics, the influence of each part on the algorithm performance is verified by removing one part of the algorithm and reserving the operation of the rest part. Finally, the tracking algorithm is verified through the KITTI and MOT data sets and data shot by authors.
The number of historical templates has an influence on the performance of the tracking algorithm: when more target templates are reserved in the algorithm, more historical information of the target can be captured, and the accuracy of the tracking algorithm is improved; however, more templates can also increase the complexity of the algorithm in time and space, and reduce the computational efficiency of the algorithm. Therefore, it is necessary to find out the appropriate number of templates, so that the algorithm can be well balanced between the tracking precision and the operation complexity.
As shown in fig. 6, it can be seen that the performance of the tracking algorithm has two peaks at template numbers of 6 and 10, respectively. The method effectively proves that more target state information can be acquired when the multi-target template is used for data association, and the method is very helpful for improving the performance of the tracking algorithm. When the number of the templates is 10, the ML and MT indexes are obviously improved, which means that when a larger tracking template value is selected, the tracking effect of the tracking algorithm is better and the efficiency of the algorithm operation is also ensured in the data association process. And finally setting the number of target templates saved by the tracking algorithm to be 10.
As shown in fig. 7, each part in MDP has an impact on the tracking algorithm performance: in order to realize effective tracking of multiple targets, the MDP-based multi-target vehicle tracking algorithm provided by the invention comprises a plurality of modules, and each module is related to a plurality of parameters and characteristics. In order to verify the correctness of the selection of each module and each parameter feature in the algorithm, the effect of each part on tracking the performance of the algorithm is tested in the section by removing one part of the algorithm every time and reserving the rest part for operation.
(1) Action a when removing tracking State3Then, the corresponding template tracking function is disabled, so that an object in the tracking state will directly transition to the lost state. However, it can be seen from the figure that the tracking performance is not significantly reduced in this case, and since the algorithm can continue to achieve effective tracking of the target through data association in the lost state, not only the importance of using the template tracking method in the tracking state is proved, but also the data association function in the lost state has good performance.
(2) When the action a6 in the lost state is removed, the data association function in the lost state is turned off. Under such conditions, a significant reduction in tracking performance can be seen in the figure, thus proving that the data correlation function is a very important component of the MDP-based multi-target vehicle tracking algorithm.
(3) The performance of the 4 th to 7 th trackers in the figure is to analyze the influence of each characteristic in the data association process on the performance of the tracking algorithm. The influence of the forward error and the backward error, the normalized correlation coefficient, the height ratio and the distance feature on the tracking performance is shown in the figure, and it can be seen from the figure that the performance of the tracking algorithm is seriously influenced by removing each feature independently, and particularly the performance is obviously reduced after removing the distance feature. Therefore, the effectiveness and correctness of the selected features in the data association are also proved.
And respectively testing the KITTI data set and the MOT data set and comparing the data set with the currently popular tracking algorithm under the condition that the number of the finally selected historical templates is 10 and all functions and characteristics in the algorithm are reserved.
As shown in fig. 8, a test for vehicle tracking alone based on the KITTI dataset is shown, the comparison of the algorithm of the present invention with other tracking algorithms. It can be seen from the table that the MDP tracking algorithm obtains the best results in the three test indexes of MOTA, MOTP and Frag, and obtains good results in the two indexes of MT and running time.
As shown in fig. 9, which shows a test of pedestrian tracking alone based on MOT data set, it can be seen that the MOTA index of the multi-target tracker proposed in the present invention is improved by 7% compared to the second one, and the best performance is obtained on both MT and ML indexes.
According to the test results, the MDP-based multi-target tracking algorithm provided by the invention has great advantages in the aspects of modeling tracking problems and strategy learning, and achieves good performances in tests aiming at individual tracking of people and vehicles.
The invention has the advantages that the MDP-based online multi-target tracking algorithm is provided, and the state space of the MDP is divided into four sub-state spaces of activation, tracking, loss and non-activation according to various possible states of the target in the target tracking process. This corresponds to a state transition between the four substate spaces in the MDP when the trace object appears, disappears briefly and appears again. The similar function learned in the data association process for realizing correct tracking is replaced by the strategy learned in the MDP, and the strategy is learned by adopting a reinforcement learning mode, so that the advantages of an online learning tracking algorithm and an offline learning tracking algorithm can be effectively integrated. The algorithm provided by the invention has sufficient rationality in model construction and feature selection, and the comparison with other algorithms proves that the multi-target tracking algorithm provided by the invention has good performances in time complexity and tracking effect, and obtains the best performance in multiple indexes.
Finally, it should be noted that the above embodiments are only used for illustrating the technical solutions of the present invention and are not limited. Although the present invention has been described in detail with reference to the embodiments, it will be understood by those skilled in the art that various changes may be made and equivalents may be substituted without departing from the spirit and scope of the invention as defined in the appended claims.
Claims (7)
1. A multi-target vehicle tracking method based on MDP is characterized by comprising the following steps:
s1, acquiring a target vehicle through a video sequence, detecting the target vehicle, and activating the target vehicle to enable the target vehicle to be in an activated state; wherein the state of the target vehicle includes: an active state, a tracking state, an inactive state, and a lost state;
s2, training a second classifier for the target vehicle in the activated state in an off-line mode, and judging the state transition of the target vehicle according to the second classifier; if the tracking state is transferred, the step S3 is carried out, otherwise, the activation state is entered, and the tracking is stopped;
s3, constructing an appearance template for the target vehicle entering the tracking state in an online mode, and judging the state transition of the target vehicle according to the appearance template; if the state is transferred to the lost state, performing step S4, otherwise, continuing to perform step S3;
s4, training a secondary classifier for the target vehicle in the lost state in a reinforcement learning mode, and judging the state transition of the target vehicle; stopping tracking when the state is switched to the inactive state, performing step S3 when the state is switched to the tracking state, and continuing to perform step S4 when the state is lost;
s5, tracking the target vehicle through the steps S1 to S4 until the target vehicle is in an inactive state or tracks the target vehicle to the last frame of a video sequence.
2. The MDP-based multi-target vehicle tracking method according to claim 1, wherein the step S2 includes:
feature vector phi through five dimensionsActive(s) training the classifiers to obtain:
the classifier obtained by the reinforcement learning training has the following learning reward functions in the activated state:
wherein (w)Active,bActive) Representing the SVM hyperplane in the activated state; if y (a) is +1, the corresponding action a is a1The target vehicle enters a tracking state in an activation state; if y (a) is equal to-1, then corresponding action a is equal to a2When the target vehicle is in the activated state, the target vehicle enters the non-activated state; the five-dimensional feature vector phiActive(s) includes coordinates in two dimensions, width, height and detection score.
3. The MDP-based multi-target vehicle tracking method according to claim 1, wherein the step S3 includes:
s31, initializing the data of the target template when the target vehicle is transferred to the tracking state and becomes the tracking target vehicle;
s32, uniformly and densely acquiring a sampling point set in the target template through an optical flow tracking algorithm, and calculating an optical flow from the sampling point set to a new frame;
s33, after optical flows of all the sampling point sets are calculated, calculating the median of forward and reverse errors of the sampling point sets in the target template as a decision-making measurement standard;
the forward and reverse errors of the sampling point set in the target template are represented by Euclidean distances between an original sampling point set and a forward and reverse prediction sampling point set: e (u) | | u-u' | non-smoking circuitry2，
Representing the median of the forward and reverse errors of the sample point set, wherein n represents the number of the point sets; said emedFBA metric for making a decision; wherein u represents the target modeA set of sampling points within the plate, u' representing a set of predicted sampling points for the target template;
s34, calculating the average boundary box overlapping rate of the target vehicle in the past K frames:
the omean is another metric for decision making, and finally defines the reward function in the tracking state as:
wherein e is0And o0Indicates a preset threshold, and if y (a) +1, corresponds to action a ═ a3If y (a) is equal to-1, the corresponding action a is equal to a4The target vehicle will transition to the lost state when e of said target vehiclemedFBLess than a predetermined threshold value omeanIf the target vehicle is larger than the preset threshold value, the target vehicle is continuously in the tracking state, otherwise, the target is transferred to the loss state.
4. The MDP-based multi-target vehicle tracking method according to claim 1, wherein the determination of the state transition of the target vehicle in step S4 includes:
using a real-valued linear function f (t, d) of wTCompleting two classifications by phi (t, d) + b, wherein (w, b) represents a parameter for controlling a linear function, and phi (t, d) represents a feature vector of similarity between the target vehicle and the detection result; in the lost state, the reward function for the data association problem is:
the parameters (w, b) are obtained by two-classifier reinforcement learning, and if y (a) is +1, the corresponding action a is a6If y (a) is equal to-1, corresponding to action a is equal to a5Target vehicle remaining in lost state, M tableIndicating the number of detection results needing to be matched in the data correlation process;
if the time that the target vehicle is in the loss state is more than TLostIf the target is marked as inactive and the tracking is stopped, TLostRepresenting a preset fixed threshold.
5. The MDP-based multi-target vehicle tracking method of claim 4, wherein the two-classifier reinforcement learning process comprises:
training the training algorithm through all targets in the video sequence, tracking the targets according to a strategy pi in the MDP, and taking different actions by the MDP according to real calibration;
when the MDP generates errors in the data association process, reconstructing a training data set S, and retraining the secondary classifier according to the new data set S;
updated training data set
Classifier for obtaining maximum interval by solving soft interval optimization problem in data association
ξ thereinkK is 1, …, M is a relaxation variable, C is a normalization parameter;
after the classifier is updated, the parameters (w ', b') and the strategy pi are updated and used in the next round of training process; the training algorithm continues to loop and update the strategy until all target vehicles are successfully tracked.
6. The MDP-based multi-target vehicle tracking method according to claim 5, wherein the feature vectors in step S4 include:
FB error: respectively representing the whole, the left half part, the right half part and the upper half part of the target template, wherein the average forward and reverse errors when the lower half part calculates the optical flow;
NCC: calculating the mean value of the median of the normalized correlation coefficient of the image blocks around the matching point in the optical flow; or normalized correlation coefficients between image blocks from the detection results and bounding boxes from the optical flow calculations;
height ratio: the average value of the height ratios of all detection results and the bounding box obtained by an optical flow method; or the height ratio between the target and the detection result;
the overlapping rate: the overlapping rate between the detection result and the boundary frame calculated by the optical flow method;
scoring: a normalized detection score;
distance: the Euclidean distance between the tracking target and the detection is based on the center of gravity.
7. The MDP-based multi-target vehicle tracking method of claim 5, wherein reconstructing the training data set comprises:
when MDP is in process of the ith frame viJ-th target vehicle t in (1)ijTracking, and the MDP entering the lost state at frame l, takes into account two important errors:
(1) MDP has been targeting vehicles
And the detection result dkPerforming correlation, but is indeed wrong according to the true calibration; the target is erroneously associated with the detection result,
adding a training set S as a negative training sample;
(2) the MDP does not correlate the target vehicle with any test results, the target is visible and correctly detected by the detector according to the actual calibration,
the training set S will be added as a negative training sample.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010196916.4A CN111368938A (en) | 2020-03-19 | 2020-03-19 | Multi-target vehicle tracking method based on MDP |
| PCT/CN2020/103235 WO2021184621A1 (en) | 2020-03-19 | 2020-07-21 | Multi-object vehicle tracking method based on mdp |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010196916.4A CN111368938A (en) | 2020-03-19 | 2020-03-19 | Multi-target vehicle tracking method based on MDP |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111368938A true CN111368938A (en) | 2020-07-03 |
Family
ID=71207702
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010196916.4A Pending CN111368938A (en) | 2020-03-19 | 2020-03-19 | Multi-target vehicle tracking method based on MDP |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN111368938A (en) |
| WO (1) | WO2021184621A1 (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112699854A (en) * | 2021-03-22 | 2021-04-23 | 亮风台（上海）信息科技有限公司 | Method and device for identifying stopped vehicle |
| WO2021184621A1 (en) * | 2020-03-19 | 2021-09-23 | 南京因果人工智能研究院有限公司 | Multi-object vehicle tracking method based on mdp |
| CN114283175A (en) * | 2021-12-28 | 2022-04-05 | 中国人民解放军国防科技大学 | Vehicle multi-target tracking method and device based on traffic video monitoring scene |
| CN114973169A (en) * | 2022-08-02 | 2022-08-30 | 山东建筑大学 | Vehicle classification counting method and system based on multi-target detection and tracking |
| CN115077549A (en) * | 2022-06-16 | 2022-09-20 | 南昌智能新能源汽车研究院 | Vehicle state tracking method, system, computer and readable storage medium |
| CN116091552A (en) * | 2023-04-04 | 2023-05-09 | 上海鉴智其迹科技有限公司 | Target tracking method, device, equipment and storage medium based on deep SORT |
| CN118405130A (en) * | 2024-06-26 | 2024-07-30 | 江苏智能无人装备产业创新中心有限公司 | Method, system and medium for sensing and tracking guided vehicles in off-road environment |
Families Citing this family (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114061616A (en) * | 2021-10-22 | 2022-02-18 | 北京自动化控制设备研究所 | Self-adaptive peak detection step counting method |
| CN114092516B (en) * | 2021-11-08 | 2024-05-14 | 国汽智控(北京)科技有限公司 | Multi-target tracking detection method, device, equipment and medium |
| CN114187510B (en) * | 2021-12-02 | 2022-09-02 | 中南大学 | Small sample remote sensing scene classification method based on metanuclear network |
| CN114663977A (en) * | 2022-03-24 | 2022-06-24 | 龙港市添誉信息科技有限公司 | Long-time span video image pedestrian monitoring accurate tracking method |
| CN114882450A (en) * | 2022-04-13 | 2022-08-09 | 南京大学 | Method for detecting reversing behavior of high-speed ramp junction under unilateral cruising of unmanned aerial vehicle |
| CN115052131A (en) * | 2022-06-22 | 2022-09-13 | 北京国信互通科技有限公司 | Special operation site safety on-line monitoring system based on remote video monitoring |
| CN117011768B (en) * | 2023-08-07 | 2024-07-23 | 哈尔滨工程大学 | Multi-ship target detection and tracking method based on robust data association |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108921873A (en) * | 2018-05-29 | 2018-11-30 | 福州大学 | The online multi-object tracking method of Markovian decision of filtering optimization is closed based on nuclear phase |
| CN109285180A (en) * | 2018-08-31 | 2019-01-29 | 电子科技大学 | A kind of road vehicle tracking of 3D |
| CN109344712A (en) * | 2018-08-31 | 2019-02-15 | 电子科技大学 | A kind of road vehicle tracking |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106023650B (en) * | 2016-07-01 | 2018-11-30 | 南京邮电大学 | Real-time pedestrian's method for early warning based on traffic intersection video and computer parallel processing system |
| CN108446585B (en) * | 2018-01-31 | 2020-10-30 | 深圳市阿西莫夫科技有限公司 | Target tracking method and device, computer equipment and storage medium |
| CN111368938A (en) * | 2020-03-19 | 2020-07-03 | 南京因果人工智能研究院有限公司 | Multi-target vehicle tracking method based on MDP |
-
2020
- 2020-03-19 CN CN202010196916.4A patent/CN111368938A/en active Pending
- 2020-07-21 WO PCT/CN2020/103235 patent/WO2021184621A1/en active Application Filing
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108921873A (en) * | 2018-05-29 | 2018-11-30 | 福州大学 | The online multi-object tracking method of Markovian decision of filtering optimization is closed based on nuclear phase |
| CN109285180A (en) * | 2018-08-31 | 2019-01-29 | 电子科技大学 | A kind of road vehicle tracking of 3D |
| CN109344712A (en) * | 2018-08-31 | 2019-02-15 | 电子科技大学 | A kind of road vehicle tracking |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2021184621A1 (en) * | 2020-03-19 | 2021-09-23 | 南京因果人工智能研究院有限公司 | Multi-object vehicle tracking method based on mdp |
| CN112699854A (en) * | 2021-03-22 | 2021-04-23 | 亮风台（上海）信息科技有限公司 | Method and device for identifying stopped vehicle |
| CN114283175A (en) * | 2021-12-28 | 2022-04-05 | 中国人民解放军国防科技大学 | Vehicle multi-target tracking method and device based on traffic video monitoring scene |
| CN114283175B (en) * | 2021-12-28 | 2024-02-02 | 中国人民解放军国防科技大学 | Vehicle multi-target tracking method and device based on traffic video monitoring scene |
| CN115077549A (en) * | 2022-06-16 | 2022-09-20 | 南昌智能新能源汽车研究院 | Vehicle state tracking method, system, computer and readable storage medium |
| CN115077549B (en) * | 2022-06-16 | 2024-04-26 | 南昌智能新能源汽车研究院 | Vehicle state tracking method, system, computer and readable storage medium |
| CN114973169A (en) * | 2022-08-02 | 2022-08-30 | 山东建筑大学 | Vehicle classification counting method and system based on multi-target detection and tracking |
| CN116091552A (en) * | 2023-04-04 | 2023-05-09 | 上海鉴智其迹科技有限公司 | Target tracking method, device, equipment and storage medium based on deep SORT |
| CN118405130A (en) * | 2024-06-26 | 2024-07-30 | 江苏智能无人装备产业创新中心有限公司 | Method, system and medium for sensing and tracking guided vehicles in off-road environment |
| CN118405130B (en) * | 2024-06-26 | 2024-09-03 | 江苏智能无人装备产业创新中心有限公司 | Method, system and medium for sensing and tracking guided vehicles in off-road environment |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2021184621A1 (en) | 2021-09-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111368938A (en) | Multi-target vehicle tracking method based on MDP | |
| US8548195B2 (en) | Tracking method and device adopting a series of observation models with different life spans | |
| CN111310583B (en) | Vehicle abnormal behavior identification method based on improved long-term and short-term memory network | |
| US8885876B2 (en) | Visual tracking system and method thereof | |
| EP1844443B1 (en) | Classifying an object in a video frame | |
| CN110084831A (en) | Based on the more Bernoulli Jacob's video multi-target detecting and tracking methods of YOLOv3 | |
| CN113139620A (en) | End-to-end multi-target detection and tracking joint method based on target association learning | |
| US20120275649A1 (en) | Foreground object tracking | |
| Maag et al. | Time-dynamic estimates of the reliability of deep semantic segmentation networks | |
| CN111292355A (en) | Nuclear correlation filtering multi-target tracking method fusing motion information | |
| CN111582349B (en) | Improved target tracking algorithm based on YOLOv3 and kernel correlation filtering | |
| CN107844739B (en) | Robust target tracking method based on self-adaptive simultaneous sparse representation | |
| Huang et al. | Siamsta: Spatio-temporal attention based siamese tracker for tracking uavs | |
| García et al. | Adaptive multi-cue 3D tracking of arbitrary objects | |
| CN113379795B (en) | Multi-target tracking and segmentation method based on conditional convolution and optical flow characteristics | |
| CN111931654A (en) | Intelligent monitoring method, system and device for personnel tracking | |
| CN106127766B (en) | Method for tracking target based on Space Coupling relationship and historical models | |
| CN112116629A (en) | End-to-end multi-target tracking method using global response graph | |
| CN110349184B (en) | Multi-pedestrian tracking method based on iterative filtering and observation discrimination | |
| Huang et al. | A bayesian hierarchical detection framework for parking space detection | |
| CN110660084A (en) | Multi-target tracking method and device | |
| Spampinato et al. | Evaluation of tracking algorithm performance without ground-truth data | |
| CN111881775B (en) | Real-time face recognition method and device | |
| CN113920155A (en) | Moving target tracking algorithm based on kernel correlation filtering | |
| CN116665097A (en) | Self-adaptive target tracking method combining context awareness |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200703 |