CN105957105B - The multi-object tracking method and system of Behavior-based control study - Google Patents
The multi-object tracking method and system of Behavior-based control study Download PDFInfo
- Publication number
- CN105957105B CN105957105B CN201610258466.0A CN201610258466A CN105957105B CN 105957105 B CN105957105 B CN 105957105B CN 201610258466 A CN201610258466 A CN 201610258466A CN 105957105 B CN105957105 B CN 105957105B
- Authority
- CN
- China
- Prior art keywords
- target
- frame
- behavior
- tracking
- prediction
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
Abstract
The present invention proposes that a kind of multi-object tracking method of Behavior-based control study and system, this method include：Target video sequence is obtained, target video sequence is detected, and obtains the size and location information of tracking target candidate frame according to testing result；Multiple target real-time tracking problem is modeled, and establishes the generative statistical model of multiple target real-time tracking problem；For the global conditions probability item in generative statistical model, off-line training is carried out on the training set correctly marked, the pervasive global behavior in various scenes is carried out to predict, tracking data of each target before present frame, local behavior prediction of the real-time online training suitable for the target are utilized for local condition's probability item in generative statistical model；The behavior prediction of target is obtained in conjunction with global behavior prediction and local behavior prediction, and multiple target tracking is carried out according to the goal behavior of prediction.The present invention can keep tracking rate in multiple target tracking, while can significantly reduce tracking error rate.
Description
Technical field
The present invention relates to computer vision and robot navigation's technical field, more particularly to a kind of Behavior-based control learns more
Method for tracking target and system.
Background technology
Target following is always the major issue and research hotspot of computer vision field, and target following according to tracking simultaneously
Number of targets be divided into monotrack and multiple target tracking (MOT, Multi-Object Tracking).In recent years, due to
The video analysis scene such as video monitoring, motion analysis, automatic Pilot and robot navigation has wider application, for more mesh
The research of mark tracking becomes more important and possesses more realistic meanings.The important application of some multiple target trackings of brief description
Scene：
(1) intelligent video monitoring：(such as mankind's identification based on footwork, automatic object detection etc.) is identified based on movement,
Automatic monitoring (one scene of monitoring is to detect suspicious actions)；(collecting traffic data in real time is used for directing traffic traffic monitoring
Flowing).
(2) human-computer interaction：Conventional human's interaction is carried out by calculating keyboard and mouse.And it is intended that computer more
Intelligently exchanged with people with natural way.One of the mode for realizing this target is so that computer is had identification and understand the appearance of people
The abilities such as state, action, gesture, tracking are the key that complete these one steps of task.
(3) robot navigation：Visual sensor is a kind of important information source of intelligent robot, for energy autokinetic movement, intelligence
Energy robot must recognize and the object in tracking environmental.In Robot Hand-eye application, tracking technique, which is used, is mounted on robot
On video camera shoot object, calculate its movement locus, optimum posture selected to capture object.
(4) virtual reality：3D interactions and virtual role action simulation directly have benefited from video human movement in virtual environment
The achievement in research of analysis can provide abundanter interactive form to participant.Human body movement data is obtained from video, with new
Virtual portrait or object with similar joint model replace the personage in original video, to obtain unexpected special effect
Fruit.Its key technology is mans motion simulation analysis.
Multiple target tracking refers to the position that interested multiple targets (being usually that certain is a kind of) is obtained from video sequence, size
And the process of movement locus.As the more advanced algorithm such as deep learning is successfully applied to object detecting areas, object
The accuracy of detection further increases.This also just for the multiple target tracking algorithm based on detection generation provide the foundation and according to
According to.Multiple target tracking based on detection is exactly to be correspondingly connected with the testing result of each frame to form each target in simple terms
At " track (trajectories) " of sdi video.In order to determine this track it is necessary to solve detection between ambiguity link with
And erroneous detection problem (more inspections --- false sun, missing inspection --- false cloudy).
In order to preferably solve the above problems, relatively straightforward idea is to use whole (batch) pattern, i.e., in whole section of video
Loop iteration is carried out in sequence, continues to optimize the track acquired.It, can not since this method can only handle existing whole section of video
Real-time tracking, also referred to as offline (offline) pattern.It is clear that the practicability of one-piece pattern is not strong, can not be applied to
The field that the needs such as real time monitoring and automatic Pilot are handled in real time.Therefore, it is on the other side in real time (online) pattern just at
For the Main way studied now.
Invention content
The present invention is directed to solve at least to a certain extent it is above-mentioned in the related technology the technical issues of one of.
For this purpose, an object of the present invention is to provide a kind of multi-object tracking method of Behavior-based control study, this method
Tracking rate can be kept in multiple target tracking, while can significantly reduce tracking error rate.
It is another object of the present invention to propose a kind of multiple-target system of Behavior-based control study.
To achieve the goals above, the embodiment of first aspect present invention proposes a kind of multiple target of Behavior-based control study
Tracking includes the following steps：S1：Target video sequence is obtained, the target video sequence is detected, and according to inspection
Survey the size and location information that result obtains tracking target candidate frame；S2：Multiple target real-time tracking problem is modeled, and is built
Found the generative statistical model of the multiple target real-time tracking problem；S3：For the global item in the generative statistical model
Part probability item carries out off-line training on the training set correctly marked, carries out the pervasive global behavior in various scenes
Prediction utilizes tracking number of each target before present frame for local condition's probability item in the generative statistical model
According to local behavior prediction of the real-time online training suitable for the target；And S4：In conjunction with global behavior prediction and the office
Portion's behavior prediction obtains the behavior prediction of target, and carries out multiple target tracking according to the goal behavior of prediction.
The multi-object tracking method of Behavior-based control study according to the ... of the embodiment of the present invention, applied to multiple target tracking scene
When, tracking rate can be kept, while can significantly reduce tracking error rate, improve tracking accuracy.
In addition, the multi-object tracking method of Behavior-based control according to the above embodiment of the present invention study can also have it is as follows
Additional technical characteristic：
In some instances, in the S2, it is as follows that the generative statistical model establishes process：
It indicates to solve multiple target tracking problem using probabilistic model shown in public formula (I)：
Wherein,Indicate the set of the state of tracked target,Indicate the information aggregate of all testing results；
Specific to the state of all targets of each frame t in the target video sequence, the public affairs formula (I) can be deformed into：
Wherein, ZtAnd XtIndicate that all observations and the dbjective state of t frames, the formula (II) are equivalent to frame by frame respectively
Acquire optimal solution；
According to the Markov property of multiple target tracking problem, XtIt can be counted as the dbjective state X by former framet-1With
And the prediction about goal behavior or prioriIt generates jointly, then the formula (II) can further be derived
For：
Wherein, the prioriAnd the state X of former frame targett-1It is independent from each other, specially：
Using the progress of single order Markov property, recursion, the formula (III) can further be deformed into frame by frame：
Wherein, P (X1) it is in first frame initialized target state, P (Zt|Xt) it is observed object frame and dbjective state letter
The characteristics of image similitude of determined target frame is ceased,It is the global priori of all target Common behaviors
Prediction to dbjective state,The probability of current frame state is generated in the state of former frame for each target,
It is equivalent to the status predication to the target according to the behavior of target itself.
In some instances, in the S3, the global behavior prediction specifically includes：Using the method for transformation of data,
The annotation results of original detection data combined training collection are generated into training data, and nerve net is trained using the training data
Network, so that the neural network trained can be according to T before targetgThe status information of frame predicts the state letter of target present frame
Breath, wherein the structure of the training data is：T before targetgThe status information of the status information of frame and corresponding present frame.
In some instances, in the S3, the part behavior prediction specifically includes：In the process for carrying out target following
In, it is trained in real time, and the tracking information before target is deformed, produces training data, and utilize the training
Data train neural network, so that the neural network trained can be according to T before targetlThe status information of frame predicts target and works as
The status information of previous frame, wherein the structure of training data is T before targetlThe shape of the status information of frame and corresponding present frame
State information.
To achieve the goals above, the embodiment of second aspect of the present invention additionally provides a kind of more mesh of Behavior-based control study
Tracking system is marked, including：Acquisition module, the acquisition module is for obtaining target video sequence, to the target video sequence
It is detected, and obtains the size and location information of tracking target candidate frame according to testing result；Modeling module, the modeling mould
Block establishes the production probabilities mould of the multiple target real-time tracking problem for being modeled to multiple target real-time tracking problem
Type；Prediction module, the prediction module are used to, for the global conditions probability item in the generative statistical model, carry out
Off-line training is carried out on the training set correctly marked, is carried out the pervasive global behavior in various scenes and is predicted, for the generation
Local condition's probability item in formula probabilistic model utilizes tracking data of each target before present frame, real-time online training suitable
Local behavior prediction for the target；And tracking module, the tracking module be used in conjunction with the global behavior prediction and
The part behavior prediction obtains the behavior prediction of target, and carries out multiple target tracking according to the goal behavior of prediction.
The multiple-target system of Behavior-based control study according to the ... of the embodiment of the present invention, applied to multiple target tracking scene
When, tracking rate can be kept, while can significantly reduce tracking error rate, improve tracking accuracy.
In addition, the multiple-target system of Behavior-based control according to the above embodiment of the present invention study can also have it is as follows
Additional technical characteristic：
In some instances, it is as follows to establish process for the generative statistical model：
It indicates to solve multiple target tracking problem using probabilistic model shown in public formula (I)：
Wherein,Indicate the set of the state of tracked target,Indicate the information aggregate of all testing results；
Specific to the state of all targets of each frame t in the target video sequence, the public affairs formula (I) can be deformed into：
Wherein, ZtAnd XtIndicate that all observations and the dbjective state of t frames, the formula (II) are equivalent to frame by frame respectively
Acquire optimal solution；
According to the Markov property of multiple target tracking problem, XtIt can be counted as the dbjective state X by former framet-1With
And the prediction about goal behavior or prioriIt generates jointly, then the formula (II) can further be derived
For：
Wherein, the prioriAnd the state X of former frame targett-1It is independent from each other, specially：
Using the progress of single order Markov property, recursion, the formula (III) can further be deformed into frame by frame：
Wherein, P (X1) it is in first frame initialized target state, P (Zt|Xt) it is observed object frame and dbjective state letter
The characteristics of image similitude of determined target frame is ceased,It is the global priori of all target Common behaviors
Prediction to dbjective state,The probability of current frame state is generated in the state of former frame for each target,
It is equivalent to the status predication to the target according to the behavior of target itself.
In some instances, the global behavior prediction specifically includes：Using the method for transformation of data, by original detection
The annotation results of data combined training collection generate training data, and train neural network using the training data, so that training
The neural network gone out can be according to T before targetgThe status information of frame predicts the status information of target present frame, wherein described
The structure of training data is：T before targetgThe status information of the status information of frame and corresponding present frame.
In some instances, the local behavior prediction specifically includes：During carrying out target following, carry out in real time
Training, and the tracking information before target is deformed, training data is produced, and nerve is trained using the training data
Network, so that the neural network trained can be according to T before targetlThe status information of frame predicts the state letter of target present frame
Breath, wherein the structure of training data is T before targetlThe status information of the status information of frame and corresponding present frame.
The additional aspect and advantage of the present invention will be set forth in part in the description, and will partly become from the following description
Obviously, or practice through the invention is recognized.
Description of the drawings
The above-mentioned and/or additional aspect and advantage of the present invention will become in the description from combination following accompanying drawings to embodiment
Obviously and it is readily appreciated that, wherein：
Fig. 1 is the flow chart of the multi-object tracking method of Behavior-based control study according to an embodiment of the invention；
Fig. 2 is the overall flow figure of the multi-object tracking method of Behavior-based control study according to an embodiment of the invention；
Fig. 3 is the flow diagram of the global follow of the multi-object tracking method of the Behavior-based control study of one embodiment of the invention；
Fig. 4 is the flow chart element that the multi-object tracking method of the Behavior-based control study of one embodiment of the invention locally tracks
Figure；
Fig. 5 is the pre- of the target sizes of the multi-object tracking method of the Behavior-based control study of one specific embodiment of the present invention
Survey result and actual result comparison schematic diagram；
Fig. 6 is the pre- of the target location of the multi-object tracking method of the Behavior-based control study of one specific embodiment of the present invention
Survey result and actual result comparison schematic diagram；And
Fig. 7 is the structure diagram of the multiple-target system of the Behavior-based control study of one embodiment of the invention.
Specific implementation mode
The embodiment of the present invention is described below in detail, examples of the embodiments are shown in the accompanying drawings, wherein from beginning to end
Same or similar label indicates same or similar element or element with the same or similar functions.Below with reference to attached
The embodiment of figure description is exemplary, and is only used for explaining the present invention, and is not considered as limiting the invention.
The multi-object tracking method and system of Behavior-based control study according to the ... of the embodiment of the present invention are described below in conjunction with attached drawing.
Fig. 1 is the flow chart of the multi-object tracking method of Behavior-based control study according to an embodiment of the invention.Fig. 2 is
The overall flow figure of the multi-object tracking method of Behavior-based control study according to an embodiment of the invention.As shown in Figure 1, and tying
Fig. 2 is closed, the multi-object tracking method of Behavior-based control study according to an embodiment of the invention includes the following steps：
Step S1：Target video sequence is obtained, target video sequence is detected, and is tracked according to testing result
The size and location information of target candidate frame.
Step S2：Multiple target real-time tracking problem is modeled, and establishes the production of multiple target real-time tracking problem
Probabilistic model.
Wherein, in step s 2, it is as follows to establish process for generative statistical model：
It indicates to solve multiple target tracking problem using probabilistic model shown in public formula (I)：
Wherein,Indicate the set of the state of tracked target,Indicate the information aggregate of all testing results, wherein
The state of tracked target includes size and location.Therefore multiple target tracking problem is exactly to ask so as to the general of observed result
Rate reaches maximum dbjective state.
Specific to the state of all targets of each frame t in target video sequence, public formula (I) can be deformed into：
Wherein, ZtAnd XtAll observations and the dbjective state of t frames are indicated respectively, and formula (II) is equivalent to be acquired frame by frame
Optimal solution.Since real-time multi-target tracer request obtains the optimal solution of present frame, this characteristic of formula (II) can be effective
Processing real-time multi-target tracking problem, therefore can be applied in wider scene, such as automatic Pilot and robot
Navigation etc..
According to the Markov property of multiple target tracking problem, XtIt can be counted as the dbjective state X by former framet-1With
And the prediction about goal behavior or prioriIt generates jointly, then formula (II) can be further derived as：
Wherein, prioriAnd the state X of former frame targett-1It is independent from each other, specially：
Using the progress of single order Markov property, recursion, formula (III) can be further deformed into frame by frame：
Wherein, P (X1) it is in first frame initialized target state, P (Zt|Xt) it is observed object frame and dbjective state letter
The characteristics of image similitude of determined target frame is ceased,It is the global priori of all target Common behaviors
Prediction to dbjective state,The probability of current frame state is generated in the state of former frame for each target,
It is equivalent to the status predication to the target according to the behavior of target itself.
Step S3：For the global conditions probability item in generative statistical model, in the training set correctly marked
Upper carry out off-line training, carrying out the pervasive global behavior prediction in various scenes, (i.e. the position of target and the variation of size becomes
Gesture is predicted), utilize tracking number of each target before present frame for local condition's probability item in generative statistical model
According to local behavior prediction of the real-time online training suitable for the target.
Wherein, in conjunction with shown in Fig. 3, global behavior prediction specifically includes：Using the method for transformation of data, by original detection
The annotation results of data combined training collection generate training data, and train neural network using training data, so as to train
Neural network can be according to T before targetgThe status information of frame predicts the status information of target present frame, wherein training data
Structure be：T before targetgThe status information of the status information of frame and corresponding present frame.Specific training process is as follows：
Pass through the preceding T of generationgThe status information of frame calculates predicted value in the forward process of neural networkIt will prediction
Value is brought into the loss function of following formula：
Residual error is obtained to process by rear, and constantly iteration is until reaching setting value.Nerve net is trained using these data
Network so that the neural network trained can be according to T before targetgThe status information of frame predicts the state letter of target present frame
Breath.
On the other hand, in conjunction with shown in Fig. 4, local behavior prediction specifically includes：During carrying out target following, in real time
It is trained, and the tracking information before target is deformed, produce training data, and nerve is trained using training data
Network, so that the neural network trained can be according to T before targetlThe status information of frame predicts the state letter of target present frame
Breath, wherein the structure of training data is T before targetlThe status information of the status information of frame and corresponding present frame.With it is complete
Unlike office's off-line training, trained data set is only produced from current tracking target in real time, is to currently tracking target
The study that the behavior of itself carries out.
Step S4：The behavior prediction of target is obtained in conjunction with global behavior prediction and local behavior prediction, and according to prediction
Goal behavior carries out multiple target tracking.Specifically, the prediction for obtaining two ways (i.e. global prediction and local prediction)
It is weighted combination, obtains the final predicted state of target.It is pre- that the result that Fig. 5 and Fig. 6 are shown demonstrates the embodiment of the present invention
The accuracy of survey.Using the Euclidean distance of testing result and predicted state as a standard on data of similar features, in conjunction with image
Other features such as feature carry out multiple target tracking.
To sum up, the multi-object tracking method of Behavior-based control according to the ... of the embodiment of the present invention study, applied to multiple target with
When track scene, tracking rate can be kept, while can significantly reduce tracking error rate, improve tracking accuracy.
Further embodiment of the present invention additionally provides a kind of multiple-target system of Behavior-based control study.
Fig. 7 is the structure diagram of the multiple-target system of Behavior-based control study according to an embodiment of the invention.Such as
Shown in Fig. 7, which includes：Acquisition module 110, modeling module 120, prediction module 130 and tracking module 140.
Specifically, acquisition module 110 is used to obtain target video sequence, is detected to target video sequence, and according to
Testing result obtains the size and location information of tracking target candidate frame.
Modeling module 120 establishes multiple target real-time tracking problem for being modeled to multiple target real-time tracking problem
Generative statistical model.
Wherein, it is as follows to establish process for generative statistical model：
It indicates to solve multiple target tracking problem using probabilistic model shown in public formula (I)：
Wherein,Indicate the set of the state of tracked target,Indicate the information aggregate of all testing results, wherein
The state of tracked target includes size and location.Therefore multiple target tracking problem is exactly to ask so as to the general of observed result
Rate reaches maximum dbjective state.
Specific to the state of all targets of each frame t in target video sequence, public formula (I) can be deformed into：
Wherein, ZtAnd XtAll observations and the dbjective state of t frames are indicated respectively, and formula (II) is equivalent to be acquired frame by frame
Optimal solution.Since real-time multi-target tracer request obtains the optimal solution of present frame, this characteristic of formula (II) can be effective
Processing real-time multi-target tracking problem, therefore can be applied in wider scene, such as automatic Pilot and robot
Navigation etc..
According to the Markov property of multiple target tracking problem, XtIt can be counted as the dbjective state X by former framet-1With
And the prediction about goal behavior or prioriIt generates jointly, then formula (II) can be further derived as：
Wherein, prioriAnd the state X of former frame targett-1It is independent from each other, specially：
Using the progress of single order Markov property, recursion, formula (III) can be further deformed into frame by frame：
Wherein, P (X1) it is in first frame initialized target state, P (Zt|Xt) it is observed object frame and dbjective state letter
The characteristics of image similitude of determined target frame is ceased,It is the global priori of all target Common behaviors
Prediction to dbjective state,The probability of current frame state is generated in the state of former frame for each target,
It is equivalent to the status predication to the target according to the behavior of target itself.
Prediction module 130 is used to, for the global conditions probability item in generative statistical model, correctly be marked
Training set on carry out off-line training, carry out the pervasive global behavior in various scenes and predict, in generative statistical model
Local condition's probability item utilize tracking data of each target before present frame, real-time online training is suitable for the target
Local behavior prediction.
Wherein, global behavior prediction specifically includes：Using the method for transformation of data, by original detection data combined training
The annotation results of collection generate training data, and train neural network using training data, so that the neural network trained can
According to T before targetgThe status information of frame predicts the status information of target present frame, wherein the structure of training data is：Target
Preceding TgThe status information of the status information of frame and corresponding present frame.
Specific training process is as follows：
Pass through the preceding T of generationgThe status information of frame calculates predicted value in the forward process of neural networkIt will prediction
Value is brought into the loss function of following formula：
Residual error is obtained to process by rear, and constantly iteration is until reaching setting value.Nerve net is trained using these data
Network so that the neural network trained can be according to T before targetgThe status information of frame predicts the state letter of target present frame
Breath.
On the other hand, local behavior prediction specifically includes：During carrying out target following, it is trained in real time, and
Tracking information before target is deformed, training data is produced, and neural network is trained using training data, so that instruction
The neural network practised can be according to T before targetlThe status information of frame predicts the status information of target present frame, wherein instruction
The structure for practicing data is T before targetlThe status information of the status information of frame and corresponding present frame.With global off-line training
Unlike, trained data set is only produced from current tracking target in real time, is the behavior to currently tracking target itself
The study of progress.
Tracking module 140 is used to combine global behavior prediction and local behavior prediction to obtain the behavior prediction of target, and root
It is predicted that goal behavior carry out multiple target tracking.Specifically, two ways (i.e. global prediction and local prediction) is obtained
To prediction be weighted combination, obtain the final predicted state of target.
To sum up, the multiple-target system of Behavior-based control according to the ... of the embodiment of the present invention study, applied to multiple target with
When track scene, tracking rate can be kept, while can significantly reduce tracking error rate, improve tracking accuracy.
In the description of the present invention, it is to be understood that, term "center", " longitudinal direction ", " transverse direction ", " length ", " width ",
" thickness ", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom" "inner", "outside", " up time
The orientation or positional relationship of the instructions such as needle ", " counterclockwise ", " axial direction ", " radial direction ", " circumferential direction " be orientation based on ... shown in the drawings or
Position relationship is merely for convenience of description of the present invention and simplification of the description, and does not indicate or imply the indicated device or element must
There must be specific orientation, with specific azimuth configuration and operation, therefore be not considered as limiting the invention.
In addition, term " first ", " second " are used for description purposes only, it is not understood to indicate or imply relative importance
Or implicitly indicate the quantity of indicated technical characteristic.Define " first " as a result, the feature of " second " can be expressed or
Implicitly include at least one this feature.In the description of the present invention, the meaning of " plurality " is at least two, such as two, three
It is a etc., unless otherwise specifically defined.
In the present invention unless specifically defined or limited otherwise, term " installation ", " connected ", " connection ", " fixation " etc.
Term shall be understood in a broad sense, for example, it may be being fixedly connected, may be a detachable connection, or integral；Can be that machinery connects
It connects, can also be electrical connection；It can be directly connected, can also can be indirectly connected through an intermediary in two elements
The interaction relationship of the connection in portion or two elements, unless otherwise restricted clearly.For those of ordinary skill in the art
For, the specific meanings of the above terms in the present invention can be understood according to specific conditions.
In the present invention unless specifically defined or limited otherwise, fisrt feature can be with "above" or "below" second feature
It is that the first and second features are in direct contact or the first and second features pass through intermediary mediate contact.Moreover, fisrt feature exists
Second feature " on ", " top " and " above " but fisrt feature be directly above or diagonally above the second feature, or be merely representative of
Fisrt feature level height is higher than second feature.Fisrt feature second feature " under ", " lower section " and " below " can be
One feature is directly under or diagonally below the second feature, or is merely representative of fisrt feature level height and is less than second feature.
In the description of this specification, reference term " one embodiment ", " some embodiments ", " example ", " specifically show
The description of example " or " some examples " etc. means specific features, structure, material or spy described in conjunction with this embodiment or example
Point is included at least one embodiment or example of the invention.In the present specification, schematic expression of the above terms are not
It must be directed to identical embodiment or example.Moreover, particular features, structures, materials, or characteristics described can be in office
It can be combined in any suitable manner in one or more embodiments or example.In addition, without conflicting with each other, the skill of this field
Art personnel can tie the feature of different embodiments or examples described in this specification and different embodiments or examples
It closes and combines.
Although the embodiments of the present invention has been shown and described above, it is to be understood that above-described embodiment is example
Property, it is not considered as limiting the invention, those skilled in the art within the scope of the invention can be to above-mentioned
Embodiment is changed, changes, replacing and modification.
Claims (8)
1. a kind of multi-object tracking method of Behavior-based control study, which is characterized in that include the following steps：
S1：Target video sequence is obtained, the target video sequence is detected, and obtains tracking target according to testing result
The size and location information of candidate frame；
S2：Multiple target real-time tracking problem is modeled, and establishes the production probabilities of the multiple target real-time tracking problem
Model；
S3：It is enterprising in the training set correctly marked for the global conditions probability item in the generative statistical model
Row off-line training carries out the pervasive global behavior in various scenes and predicts, for the partial strip in the generative statistical model
Part probability item utilizes tracking data of each target before present frame, local behavior of the real-time online training suitable for the target
Prediction；And
S4：The behavior prediction of target is obtained in conjunction with global behavior prediction and the local behavior prediction, and according to prediction
Goal behavior carries out multiple target tracking.
2. the multi-object tracking method of Behavior-based control study according to claim 1, which is characterized in that in the S2, institute
Stating generative statistical model, to establish process as follows：
It indicates to solve multiple target tracking problem using probabilistic model shown in public formula (I)：
Wherein,Indicate the set of the state of tracked target,Indicate the information aggregate of all testing results；
Specific to the state of all targets of each frame t in the target video sequence, the public affairs formula (I) can be deformed into：
Wherein, ZtAnd XtIndicate that all testing results and dbjective state of t frames, the formula (II) are equivalent to frame by frame respectively
Acquire optimal solution；
According to the Markov property of multiple target tracking problem, XtIt can be counted as the dbjective state X by former framet-1And about
The prediction of goal behavior or prioriIt generates jointly, then the formula (II) can be further derived as：
Wherein, the prioriAnd the state X of former frame targett-1It is independent from each other, specially：
Using the progress of single order Markov property, recursion, the formula (III) can further be deformed into frame by frame：
Wherein, P (X1) it is in first frame initialized target state, P (Zt|Xt) it is that testing result and target status information are determined
The characteristics of image similitude of fixed target frame,It is the global priori of all target Common behaviors to target
The prediction of state,The probability for generating current frame state in the state of former frame for each target, is equivalent to
According to the behavior of target itself to the status predication of the target.
3. the multi-object tracking method of Behavior-based control study according to claim 1, which is characterized in that in the S3,
The global behavior prediction specifically includes：
Using the method for transformation of data, the annotation results of original detection data combined training collection are generated into training data, and profit
Neural network is trained with the training data, so that the neural network trained can be according to T before targetgThe status information of frame is pre-
Measure the status information of target present frame, wherein the structure of the training data is：T before targetgThe status information of frame and right
The status information for the present frame answered.
4. the multi-object tracking method of Behavior-based control study according to claim 1, which is characterized in that in the S3,
The part behavior prediction specifically includes：
It during carrying out target following, is trained in real time, and the tracking information before target is deformed, produce
Training data, and neural network is trained using the training data, so that the neural network trained can be according to T before targetl
The status information of frame predicts the status information of target present frame, wherein the structure of training data is T before targetlThe state of frame
The status information of information and corresponding present frame.
5. a kind of multiple-target system of Behavior-based control study, which is characterized in that including：
Acquisition module, the acquisition module are detected the target video sequence, and root for obtaining target video sequence
The size and location information of tracking target candidate frame is obtained according to testing result；
Modeling module, the modeling module is for modeling multiple target real-time tracking problem, and it is real to establish the multiple target
When tracking problem generative statistical model；
Prediction module, the prediction module are used for for the global conditions probability item in the generative statistical model, into
Off-line training is carried out on the training set that row correctly marks, and is carried out the pervasive global behavior in various scenes and is predicted, for the production
Local condition's probability item in raw formula probabilistic model utilizes tracking data of each target before present frame, real-time online training
Local behavior prediction suitable for the target；And
Tracking module, the tracking module are used to obtain target in conjunction with global behavior prediction and the local behavior prediction
Behavior prediction, and multiple target tracking is carried out according to the goal behavior of prediction.
6. the multiple-target system of Behavior-based control study according to claim 5, which is characterized in that the production is general
Rate model foundation process is as follows：
It indicates to solve multiple target tracking problem using probabilistic model shown in public formula (I)：
Wherein,Indicate the set of the state of tracked target,Indicate the information aggregate of all testing results；
Specific to the state of all targets of each frame t in the target video sequence, the public affairs formula (I) can be deformed into：
Wherein, ZtAnd XtIndicate that all testing results and dbjective state of t frames, the formula (II) are equivalent to frame by frame respectively
Acquire optimal solution；
According to the Markov property of multiple target tracking problem, XtIt can be counted as the dbjective state X by former framet-1And about
The prediction of goal behavior or prioriIt generates jointly, then the formula (II) can be further derived as：
Wherein, the prioriAnd the state X of former frame targett-1It is independent from each other, specially：
Using the progress of single order Markov property, recursion, the formula (III) can further be deformed into frame by frame：
Wherein, P (X1) it is in first frame initialized target state, P (Zt|Xt) it is that testing result and target status information are determined
The characteristics of image similitude of fixed target frame,It is the global priori of all target Common behaviors to target
The prediction of state,The probability for generating current frame state in the state of former frame for each target, is equivalent to
According to the behavior of target itself to the status predication of the target.
7. the multiple-target system of Behavior-based control study according to claim 5, which is characterized in that the global behavior
Prediction specifically includes：
Using the method for transformation of data, the annotation results of original detection data combined training collection are generated into training data, and profit
Neural network is trained with the training data, so that the neural network trained can be according to T before targetgThe status information of frame is pre-
Measure the status information of target present frame, wherein the structure of the training data is：T before targetgThe status information of frame and right
The status information for the present frame answered.
8. the multiple-target system of Behavior-based control study according to claim 5, which is characterized in that the part behavior
Prediction specifically includes：
It during carrying out target following, is trained in real time, and the tracking information before target is deformed, produce
Training data, and neural network is trained using the training data, so that the neural network trained can be according to T before targetl
The status information of frame predicts the status information of target present frame, wherein the structure of training data is T before targetlThe state of frame
The status information of information and corresponding present frame.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610258466.0A CN105957105B (en) | 2016-04-22 | 2016-04-22 | The multi-object tracking method and system of Behavior-based control study |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610258466.0A CN105957105B (en) | 2016-04-22 | 2016-04-22 | The multi-object tracking method and system of Behavior-based control study |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN105957105A CN105957105A (en) | 2016-09-21 |
| CN105957105B true CN105957105B (en) | 2018-10-02 |
Family
ID=56916246
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610258466.0A Active CN105957105B (en) | 2016-04-22 | 2016-04-22 | The multi-object tracking method and system of Behavior-based control study |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105957105B (en) |
Families Citing this family (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106447698B (en) * | 2016-09-28 | 2019-07-02 | 深圳市鸿逸达科技有限公司 | A kind of more pedestrian tracting methods and system based on range sensor |
| CN107092869B (en) * | 2017-04-05 | 2019-11-26 | 武汉大学 | A kind of point target tracking of video satellite |
| CN108932731B (en) * | 2017-05-24 | 2021-02-05 | 上海云从企业发展有限公司 | Target tracking method and system based on prior information |
| WO2019036938A1 (en) * | 2017-08-23 | 2019-02-28 | 深圳前海达闼云端智能科技有限公司 | Robot monitoring method and apparatus, server, and terminal device |
| CN107505951B (en) * | 2017-08-29 | 2020-08-21 | 深圳市道通智能航空技术有限公司 | Target tracking method, unmanned aerial vehicle and computer readable storage medium |
| CN108388586A (en) * | 2017-12-03 | 2018-08-10 | 广东鸿威国际会展集团有限公司 | A kind of information management system and method |
| CN108153519B (en) * | 2018-01-16 | 2020-06-12 | 中国人民解放军海军航空大学 | Universal design framework for target intelligent tracking method |
| CN108447076B (en) * | 2018-03-16 | 2021-04-06 | 清华大学 | Multi-target tracking method based on deep reinforcement learning |
| CN108875588B (en) * | 2018-05-25 | 2022-04-15 | 武汉大学 | Cross-camera pedestrian detection tracking method based on deep learning |
| CN109493364A (en) * | 2018-09-26 | 2019-03-19 | 重庆邮电大学 | A kind of target tracking algorism of combination residual error attention and contextual information |
| CN112889068A (en) | 2018-10-26 | 2021-06-01 | 英特尔公司 | Neural network object recognition for image processing |
| CN111127510B (en) * | 2018-11-01 | 2023-10-27 | 杭州海康威视数字技术股份有限公司 | Target object position prediction method and device |
| CN111797785B (en) * | 2020-07-09 | 2022-04-29 | 电子科技大学 | Multi-aircraft tracking method based on deep learning |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101325691A (en) * | 2007-06-14 | 2008-12-17 | 清华大学 | Method and apparatus for tracing a plurality of observation model with fusion of differ durations |
| CN103093199A (en) * | 2013-01-15 | 2013-05-08 | 中国科学院自动化研究所 | Certain face tracking method based on online recognition |
| CN103942536A (en) * | 2014-04-04 | 2014-07-23 | 西安交通大学 | Multi-target tracking method of iteration updating track model |
| CN105467382A (en) * | 2015-12-31 | 2016-04-06 | 南京信息工程大学 | SVM (Support Vector Machine)-based multi-sensor target tracking data fusion algorithm and system thereof |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9171049B2 (en) * | 2002-06-13 | 2015-10-27 | Salesforce.Com, Inc. | Offline simulation of online session between client and server |
-
2016
- 2016-04-22 CN CN201610258466.0A patent/CN105957105B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101325691A (en) * | 2007-06-14 | 2008-12-17 | 清华大学 | Method and apparatus for tracing a plurality of observation model with fusion of differ durations |
| CN103093199A (en) * | 2013-01-15 | 2013-05-08 | 中国科学院自动化研究所 | Certain face tracking method based on online recognition |
| CN103942536A (en) * | 2014-04-04 | 2014-07-23 | 西安交通大学 | Multi-target tracking method of iteration updating track model |
| CN105467382A (en) * | 2015-12-31 | 2016-04-06 | 南京信息工程大学 | SVM (Support Vector Machine)-based multi-sensor target tracking data fusion algorithm and system thereof |
Non-Patent Citations (4)
| Title |
|---|
| Semi-Supervised On-line Boosting for Robust Tracking;Helmut Grabner et al.;《Computer Vision-ECCV 2008》;20081031;第1-14页 * |
| SlidingWindow Optimization on an Ambiguity-Clearness Graph for Multi-object Tracking;Qi Guo et al.;《arXiv》;20151130;第1-9页 * |
| 基于计算机视觉的复杂场景下目标跟踪研究;贺文骅;《中国博士学位论文全文数据库 信息科技辑》;20160315;第1.2节 * |
| 视觉计算中的离线和在线样本学习研究;刘嘉;《中国科学院电子学研究所机构知识库》;20051231;第5章 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN105957105A (en) | 2016-09-21 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105957105B (en) | The multi-object tracking method and system of Behavior-based control study | |
| Frossard et al. | End-to-end learning of multi-sensor 3d tracking by detection | |
| Chiu et al. | Probabilistic 3d multi-modal, multi-object tracking for autonomous driving | |
| CN109800689B (en) | Target tracking method based on space-time feature fusion learning | |
| Choi et al. | A general framework for tracking multiple people from a moving camera | |
| CN110472554A (en) | Table tennis action identification method and system based on posture segmentation and crucial point feature | |
| Mutis et al. | Real-time space occupancy sensing and human motion analysis using deep learning for indoor air quality control | |
| CN111310659B (en) | Human body action recognition method based on enhanced graph convolution neural network | |
| CN108222749B (en) | Intelligent automatic door control method based on image analysis | |
| CN106169188A (en) | A kind of method for tracing object based on the search of Monte Carlo tree | |
| CN109919974A (en) | Online multi-object tracking method based on the more candidate associations of R-FCN frame | |
| CN102447835A (en) | Non-blind-area multi-target cooperative tracking method and system | |
| US20110208685A1 (en) | Motion Capture Using Intelligent Part Identification | |
| Li et al. | Robust multiperson detection and tracking for mobile service and social robots | |
| Beetz et al. | Camera-based observation of football games for analyzing multi-agent activities | |
| Dong et al. | Dynamic gesture recognition by directional pulse coupled neural networks for human-robot interaction in real time | |
| CN109993770A (en) | A kind of method for tracking target of adaptive space-time study and state recognition | |
| CN105809718A (en) | Object tracking method with minimum trajectory entropy | |
| CN114283355A (en) | Multi-target endangered animal tracking method based on small sample learning | |
| Yu et al. | Towards robust human trajectory prediction in raw videos | |
| CN105894008A (en) | Target motion track method through combination of feature point matching and deep nerve network detection | |
| Mittal et al. | Pedestrian detection and tracking using deformable part models and Kalman filtering | |
| CN109493370A (en) | A kind of method for tracking target based on spatial offset study | |
| Wang et al. | Simple but effective: Upper-body geometric features for traffic command gesture recognition | |
| CN115880620A (en) | Personnel counting method applied to cart early warning system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |