CN105957105A - Multi-target tracking method and system based on behavior learning - Google Patents

Abstract

The invention provides a multi-target tracking method and system based on behavior learning. The method comprises steps of: acquiring and detecting a target video sequence, and acquiring the size and the positional information of a tracked target candidate frame depending on a detection result; modeling a multi-target real-time tracking problem and creating a production probability model of the multi-target real-time tracking problem; for the global conditional probability items in the production probability model, performing offline training on a correctly-marked training set in order to perform global behavior prediction applicable to various scenarios, and for local conditional probability items in the production probability model, on-line training local behavior prediction for each target in real time by using the tracking data of the target prior to a current frame; obtaining the behavior prediction of the target in combination with the global behavior prediction and the local behavior prediction, and tracking the multiple targets depending on the predicted target behavior. The method and the system may keep a tracking rate while tracking the multiple targets and may significantly reduce a tracking error rate.

Description

The multi-object tracking method of Behavior-based control study and system

Technical field

The present invention relates to computer vision and robot navigation's technical field, the many mesh learnt particularly to a kind of Behavior-based control Mark tracking and system.

Background technology

Target following is always major issue and the study hotspot of computer vision field, and target following is according to the mesh followed the tracks of simultaneously Mark number is divided into monotrack and multiple target tracking (MOT, Multi-Object Tracking).In recent years, owing to regarding The video analysis scenes such as frequency monitoring, motion analysis, automatic Pilot and robot navigation have wider application, for many mesh The research that mark is followed the tracks of becomes more important and has more realistic meaning.The important application of some multiple target trackings of brief description Scene:

(1) intelligent video monitoring: based on Motion Recognition (such as based on footwork the mankind identify, automatic object detection etc.), automatically Change monitoring (monitoring that a scene is to detect questionable conduct)；Traffic monitoring (collecting traffic data in real time is used for commanding traffic flow).

(2) man-machine interaction: conventional human is carried out by calculating keyboard and mouse alternately.And it is intended that computer is more intelligent Ground exchanges with people with natural way.One of mode realizing this target be make computer have identify and understand people attitude, The ability such as action, gesture, tracking has been key one step of these tasks.

(3) robot navigation: vision sensor is a kind of important information source of intelligent robot, for energy autonomic movement, intelligent machine Device people must understanding and the object in tracking environmental.In Robot Hand-eye is applied, tracking technique is with being arranged on robot Video camera shooting object, calculates its movement locus, selects optimum posture to capture object.

(4) virtual reality: in virtual environment, 3D is mutual and virtual role action simulation directly has benefited from video human motion analysis Achievement in research, more abundant interactive form can be provided to participant.Human body movement data is obtained, with new from video Virtual portrait or there is the object of similar joint model replace the personage in former video, to obtain beyond thought special-effect. Its key technology is mans motion simulation analysis.

Multiple target tracking refer to obtain from video sequence the position of multiple targets (the most a certain class) interested, size with And the process of movement locus.Along with the more advanced algorithms such as degree of depth study are successfully applied to object detecting areas, object is examined The accuracy surveyed improves further.This also just provides the foundation and foundation for generation based on the multiple target tracking algorithm detected. The each target of formation that the testing result correspondence of each frame coupled together exactly in simple terms based on the multiple target tracking detected is regarding Frequently " track (trajectories) " in space.In order to determine this track it is necessary to solve ambiguity link and mistake between detection Test problems (the false sun of many inspections, missing inspection vacation is cloudy).

In order to preferably solve the problems referred to above, relatively straightforward idea is by overall (batch) pattern, i.e. whole section of video sequence Carry out loop iteration on row, continue to optimize the track tried to achieve.Owing to this method can only process existing whole section of video, it is impossible to Real-time tracking, also referred to as off-line (offline) pattern.It is clear that the practicality of one-piece pattern is the strongest, it is impossible to application The field processed in real time is needed to monitoring and automatic Pilot etc. in real time.Therefore, (online) pattern in real time on the other side is just Become the Main way of research now.

Summary of the invention

It is contemplated that one of technical problem solved the most to a certain extent in above-mentioned correlation technique.

To this end, it is an object of the present invention to propose the multi-object tracking method of a kind of Behavior-based control study, the method is many Tracking rate can be kept during target following, can significantly reduce tracking error rate simultaneously.

Further object is that the multiple-target system proposing the study of a kind of Behavior-based control.

To achieve these goals, the embodiment of first aspect present invention proposes the multiple target tracking of a kind of Behavior-based control study Method, comprises the following steps: S1: obtains target video sequence, detects described target video sequence, and according to inspection Survey result and obtain the size and location information following the tracks of target candidate frame；S2: multiple target real-time tracking problem is modeled, and Set up the generative statistical model of described multiple target real-time tracking problem；S3: for the overall situation in described generative statistical model Conditional probability item, carries out off-line training in the training set the most correctly marked, and carries out pervasive in the overall row of various scenes For prediction, utilize the tracking before present frame of each target for the local condition's probability item in described generative statistical model Data, real-time online training is applicable to the local behavior prediction of this target；And S4: combine the prediction of described global behavior and institute State local behavior prediction and obtain the behavior prediction of target, and carry out multiple target tracking according to the target behavior of prediction.

The multi-object tracking method of Behavior-based control study according to embodiments of the present invention, when being applied to multiple target tracking scene, Tracking rate can be kept, can significantly reduce tracking error rate simultaneously, improve and follow the tracks of accuracy.

It addition, the multi-object tracking method of Behavior-based control according to the above embodiment of the present invention study can also have following additional Technical characteristic:

In some instances, in described S2, it is as follows that described generative statistical model sets up process:

Public probabilistic model shown in formula (I) is used to represent the multiple target tracking problem that solves:

Wherein,Represent the set of the state of followed the tracks of target,Represent the information aggregate of all of testing result；

Specific to the state of all targets of frame t each in described target video sequence, described public formula (I) can be deformed into:

$X_t^{*}=\underset{X_t}{\mathrm{argmax}}P\left(Z_t\right|X_t)P\left(X_t\right)---\left(II\right),$

Wherein, Z_tAnd X_tRepresent that all observations and the dbjective state of t frame, described public formula II are equivalent to frame by frame respectively Try to achieve optimal solution；

According to the Markov property of multiple target tracking problem, X_tCan be counted as by dbjective state X of former frame_t-1And close Prediction or priori in target behaviorCommon generation, the most described public formula II just can be derived as further:

Wherein, described prioriAnd state X of former frame target_t-1It is separate, particularly as follows:

Utilizing single order Markov property to carry out recursion frame by frame, described public formula III can further be deformed into:

Wherein, P (X₁) it is at the first frame initialized target state, P (Z_t|X_t) it is observed object frame and target status information The characteristics of image similarity of the target frame determined,It it is the overall priori pair of all target Common behaviors The prediction of dbjective state,Present frame shape probability of state, phase is produced in the state of former frame for each target When in the status predication to this target of the behavior according to target self.

In some instances, in described S3, the prediction of described global behavior specifically includes: the method utilizing transformation of data, will The annotation results of original detection data combined training collection produces training data, and utilizes described training data to train neutral net, So that the neutral net trained can be according to T before target_gThe status information of frame dopes the status information of target present frame, its In, the structure of described training data is: T before target_gThe status information of frame and the status information of the present frame of correspondence.

In some instances, in described S3, described local behavior prediction specifically includes: during carrying out target following, It is trained in real time, and the tracking information before target is deformed, produce training data, and utilize described training number According to training neutral net, so that the neutral net trained can be according to T before target_lIt is current that the status information of frame dopes target The status information of frame, wherein, the structure of training data is, T before target_lThe status information of frame and the shape of the present frame of correspondence State information.

To achieve these goals, the embodiment of second aspect present invention additionally provide a kind of Behavior-based control study multiple target with Track system, including: acquisition module, described acquisition module is used for obtaining target video sequence, enters described target video sequence Row detection, and the size and location information following the tracks of target candidate frame is obtained according to testing result；MBM, described modeling mould Block is for being modeled multiple target real-time tracking problem, and sets up the production probabilities mould of described multiple target real-time tracking problem Type；Prediction module, described prediction module, for for the global conditions probability item in described generative statistical model, is being entered Carry out off-line training in the training set of the correct mark of row, carry out the pervasive global behavior in various scenes and predict, for described product Local condition's probability item in raw formula probabilistic model utilizes each target tracking data before present frame, and real-time online is trained It is applicable to the local behavior prediction of this target；And tracking module, described tracking module is used for combining the prediction of described global behavior And described local behavior prediction obtains the behavior prediction of target, and carry out multiple target tracking according to the target behavior of prediction.

The multiple-target system of Behavior-based control study according to embodiments of the present invention, when being applied to multiple target tracking scene, Tracking rate can be kept, can significantly reduce tracking error rate simultaneously, improve and follow the tracks of accuracy.

It addition, the multiple-target system of Behavior-based control according to the above embodiment of the present invention study can also have following additional Technical characteristic:

In some instances, to set up process as follows for described generative statistical model:

Public probabilistic model shown in formula (I) is used to represent the multiple target tracking problem that solves:

Wherein,Represent the set of the state of followed the tracks of target,Represent the information aggregate of all of testing result；

Specific to the state of all targets of frame t each in described target video sequence, described public formula (I) can be deformed into:

$X_t^{*}=\underset{X_t}{\mathrm{argmax}}P\left(Z_t\right|X_t)P\left(X_t\right)---\left(II\right),$

Wherein, Z_tAnd X_tRepresent that all observations and the dbjective state of t frame, described public formula II are equivalent to frame by frame respectively Try to achieve optimal solution；

According to the Markov property of multiple target tracking problem, X_tCan be counted as by dbjective state X of former frame_t-1And close Prediction or priori in target behaviorCommon generation, the most described public formula II just can be derived as further:

Wherein, described prioriAnd state X of former frame target_t-1It is separate, particularly as follows:

Utilizing single order Markov property to carry out recursion frame by frame, described public formula III can further be deformed into:

Wherein, P (X₁) it is at the first frame initialized target state, P (Z_t|X_t) it is observed object frame and target status information The characteristics of image similarity of the target frame determined,It it is the overall priori pair of all target Common behaviors The prediction of dbjective state,Present frame shape probability of state, phase is produced in the state of former frame for each target When in the status predication to this target of the behavior according to target self.

In some instances, the prediction of described global behavior specifically includes: the method utilizing transformation of data, by original detection number Produce training data according to the annotation results of combined training collection, and utilize described training data to train neutral net, so that training Neutral net can be according to T before target_gThe status information of frame dopes the status information of target present frame, wherein, described instruction The structure practicing data is: T before target_gThe status information of frame and the status information of the present frame of correspondence.

In some instances, described local behavior prediction specifically includes: during carrying out target following, instruct in real time Practice, and the tracking information before target is deformed, produce training data, and utilize described training data training nerve Network, so that the neutral net trained can be according to T before target_lThe status information of frame dopes the state letter of target present frame Breath, wherein, the structure of training data is, T before target_lThe status information of frame and the status information of the present frame of correspondence.

The additional aspect of the present invention and advantage will part be given in the following description, and part will become bright from the following description Aobvious, or recognized by the practice of the present invention.

Accompanying drawing explanation

Above-mentioned and/or the additional aspect of the present invention and advantage will be apparent from from combining the accompanying drawings below description to embodiment With easy to understand, wherein:

Fig. 1 is the flow chart of the multi-object tracking method of Behavior-based control study according to an embodiment of the invention；

Fig. 2 is the overall flow figure of the multi-object tracking method of Behavior-based control study according to an embodiment of the invention；Fig. 3 It it is the FB(flow block) of the global follow of the multi-object tracking method of the Behavior-based control study of one embodiment of the invention；

Fig. 4 is the FB(flow block) of the multi-object tracking method local tracking of the Behavior-based control study of one embodiment of the invention；

Fig. 5 is the prediction knot of the target sizes of the multi-object tracking method of the Behavior-based control study of one specific embodiment of the present invention Fruit compares schematic diagram with actual result；

Fig. 6 is the prediction knot of the target location of the multi-object tracking method of the Behavior-based control study of one specific embodiment of the present invention Fruit compares schematic diagram with actual result；And

Fig. 7 is the structured flowchart of the multiple-target system of the Behavior-based control study of one embodiment of the invention.

Detailed description of the invention

Embodiments of the invention are described below in detail, and the example of described embodiment is shown in the drawings, the most identical Or similar label represents same or similar element or has the element of same or like function.Retouch below with reference to accompanying drawing The embodiment stated is exemplary, is only used for explaining the present invention, and is not considered as limiting the invention.

Below in conjunction with accompanying drawing, multi-object tracking method and the system that Behavior-based control according to embodiments of the present invention learns is described.

Fig. 1 is the flow chart of the multi-object tracking method of Behavior-based control study according to an embodiment of the invention.Fig. 2 is basis The overall flow figure of the multi-object tracking method of the Behavior-based control study of one embodiment of the invention.As it is shown in figure 1, and combine figure 2, the multi-object tracking method of Behavior-based control study according to an embodiment of the invention, comprise the following steps:

Step S1: obtain target video sequence, target video sequence is detected, and obtain tracking mesh according to testing result The size and location information of mark candidate frame.

Step S2: multiple target real-time tracking problem is modeled, and sets up the production probabilities of multiple target real-time tracking problem Model.

Wherein, in step s 2, it is as follows that generative statistical model sets up process:

Public probabilistic model shown in formula (I) is used to represent the multiple target tracking problem that solves:

Wherein,Represent the set of the state of followed the tracks of target,Represent the information aggregate of all of testing result, wherein, institute The state following the tracks of target includes size and location.Therefore multiple target tracking problem is exactly to ask so that arrive the probability of observed result Reach maximum dbjective state.

Specific to the state of all targets of frame t each in target video sequence, public formula (I) can be deformed into:

$X_t^{*}=\underset{X_t}{\mathrm{argmax}}P\left(Z_t\right|X_t)P\left(X_t\right)---\left(II\right),$

Wherein, Z_tAnd X_tRepresent that all observations and the dbjective state of t frame, public formula II are equivalent to try to achieve frame by frame respectively Optimal solution.Owing to real-time multi-target tracer request obtains the optimal solution of present frame, this characteristic of public formula II just can have The process real-time multi-target tracking problem of effect, therefore may apply to widely in scene, such as automatic Pilot and machine People's navigation etc..

According to the Markov property of multiple target tracking problem, X_tCan be counted as by dbjective state X of former frame_t-1And close Prediction or priori in target behaviorCommon generation, then public formula II just can be derived as further:

Wherein, prioriAnd state X of former frame target_t-1It is separate, particularly as follows:

Utilizing single order Markov property to carry out recursion frame by frame, public formula III can further be deformed into:

Wherein, P (X₁) it is at the first frame initialized target state, P (Z_t|X_t) it is observed object frame and target status information The characteristics of image similarity of the target frame determined,It it is the overall priori pair of all target Common behaviors The prediction of dbjective state,Present frame shape probability of state, phase is produced in the state of former frame for each target When in the status predication to this target of the behavior according to target self.

Step S3: for the global conditions probability item in generative statistical model, enterprising in the training set the most correctly marked Row off-line training, carries out the pervasive global behavior in various scenes and predicts that (i.e. the position of target and the variation tendency of size is pre- Survey), utilize each target tracking data before present frame for the local condition's probability item in generative statistical model, real Time on-line training be applicable to the local behavior prediction of this target.

Wherein, shown in Fig. 3, global behavior prediction specifically includes: the method utilizing transformation of data, by original detection number Produce training data according to the annotation results of combined training collection, and utilize training data to train neutral net, so that the god trained Through network can according to target before T_gThe status information of frame dopes the status information of target present frame, wherein, training data Structure is: T before target_gThe status information of frame and the status information of the present frame of correspondence.Concrete training process is as follows:

By the front T produced_gThe status information of frame, calculates predictive value in the forward process of neutral netBy predictive value It is brought in the loss function of following formula:

$J(W,b)=\frac{1}{m}\underset{i=1}{\overset{m}{\Σ}}\left(\frac{1}{2}{||h_{W,b}\left(data\right)-label||}^2\right)+\frac{1}{2}{\mathrm{\λW}}^2---\left(V\right),$

Obtain residual error by backward process, and continuous iteration is until reaching setting value.These data are utilized to train neutral net, Make the neutral net trained can be according to T before target_gThe status information of frame dopes the status information of target present frame.

On the other hand, shown in Fig. 4, locally behavior prediction specifically includes: during carrying out target following, enter in real time Row training, and the tracking information before target is deformed, produce training data, and utilize training data training nerve Network, so that the neutral net trained can be according to T before target_lThe status information of frame dopes the state letter of target present frame Breath, wherein, the structure of training data is, T before target_lThe status information of frame and the status information of the present frame of correspondence.With Unlike overall situation off-line training, the data set of training is only produced from current tracking target in real time, is to currently following the tracks of mesh Mark the study that the behavior of self is carried out.

Step S4: combine the behavior prediction that global behavior is predicted and locally behavior prediction obtains target, and according to the target of prediction Behavior carries out multiple target tracking.Specifically, will the prediction that obtains of two ways (i.e. global prediction and local prediction) enter Row weighted combination, obtains the predicted state that target is final.The result that Fig. 5 and Fig. 6 is shown demonstrates embodiment of the present invention prediction Accuracy.Using the Euclidean distance of testing result and predicted state as a standard on data of similar features, special in conjunction with image Other features such as levy and carry out multiple target tracking.

To sum up, the multi-object tracking method of Behavior-based control study according to embodiments of the present invention, it is being applied to multiple target tracking field Jing Shi, can keep tracking rate, can significantly reduce tracking error rate simultaneously, improves and follows the tracks of accuracy.

Further embodiment of the present invention additionally provides the multiple-target system of a kind of Behavior-based control study.

Fig. 7 is the structured flowchart of the multiple-target system of Behavior-based control study according to an embodiment of the invention.Such as Fig. 7 institute Showing, this system 100 includes: acquisition module 110, MBM 120, prediction module 130 and tracking module 140.

Specifically, acquisition module 110 is used for obtaining target video sequence, detects target video sequence, and according to detection Result obtains the size and location information following the tracks of target candidate frame.

MBM 120 is for being modeled multiple target real-time tracking problem, and sets up the generation of multiple target real-time tracking problem Formula probabilistic model.

Wherein, to set up process as follows for generative statistical model:

Public probabilistic model shown in formula (I) is used to represent the multiple target tracking problem that solves:

Wherein,Represent the set of the state of followed the tracks of target,Represent the information aggregate of all of testing result, wherein, institute The state following the tracks of target includes size and location.Therefore multiple target tracking problem is exactly to ask so that arrive the probability of observed result Reach maximum dbjective state.

Specific to the state of all targets of frame t each in target video sequence, public formula (I) can be deformed into:

$X_t^{*}=\underset{X_t}{\mathrm{argmax}}P\left(Z_t\right|X_t)P\left(X_t\right)---\left(II\right),$

Wherein, Z_tAnd X_tRepresent that all observations and the dbjective state of t frame, public formula II are equivalent to try to achieve frame by frame respectively Optimal solution.Owing to real-time multi-target tracer request obtains the optimal solution of present frame, this characteristic of public formula II just can have The process real-time multi-target tracking problem of effect, therefore may apply to widely in scene, such as automatic Pilot and machine People's navigation etc..

According to the Markov property of multiple target tracking problem, X_tCan be counted as by dbjective state X of former frame_t-1And close Prediction or priori in target behaviorCommon generation, then public formula II just can be derived as further:

Wherein, prioriAnd state X of former frame target_t-1It is separate, particularly as follows:

Utilizing single order Markov property to carry out recursion frame by frame, public formula III can further be deformed into:

Wherein, P (X₁) it is at the first frame initialized target state, P (Z_t|X_t) it is observed object frame and target status information The characteristics of image similarity of the target frame determined,It it is the overall priori pair of all target Common behaviors The prediction of dbjective state,Present frame shape probability of state, phase is produced in the state of former frame for each target When in the status predication to this target of the behavior according to target self.

Prediction module 130 is for for the global conditions probability item in generative statistical model, in the training the most correctly marked Carry out off-line training on collection, carry out the pervasive global behavior in various scenes and predict, for the local in generative statistical model Conditional probability item utilizes each target tracking data before present frame, real-time online training to be applicable to the partial row of this target For prediction.

Wherein, global behavior prediction specifically includes: the method utilizing transformation of data, by original detection data combined training collection Annotation results produce training data, and utilize training data train neutral net so that the neutral net trained can root According to T before target_gThe status information of frame dopes the status information of target present frame, and wherein, the structure of training data is: target Front T_gThe status information of frame and the status information of the present frame of correspondence.

Concrete training process is as follows:

By the front T produced_gThe status information of frame, calculates predictive value in the forward process of neutral netBy predictive value It is brought in the loss function of following formula:

$J(W,b)=\frac{1}{m}\underset{i=1}{\overset{m}{\Σ}}\left(\frac{1}{2}{||h_{W,b}\left(data\right)-label||}^2\right)+\frac{1}{2}{\mathrm{\λW}}^2---\left(V\right),$

Obtain residual error by backward process, and continuous iteration is until reaching setting value.These data are utilized to train neutral net, Make the neutral net trained can be according to T before target_gThe status information of frame dopes the status information of target present frame.

On the other hand, locally behavior prediction specifically includes: during carrying out target following, be trained in real time, and will Tracking information before target deforms, and produces training data, and utilizes training data to train neutral net, so that instruction The neutral net practised can be according to T before target_lThe status information of frame dopes the status information of target present frame, wherein, instruction The structure practicing data is, T before target_lThe status information of frame and the status information of the present frame of correspondence.With overall situation off-line training Except for the difference that, the data set of training is only produced from current tracking target in real time, is to the current behavior following the tracks of target self The study carried out.

Tracking module 140 is used for combining the behavior prediction that global behavior is predicted and locally behavior prediction obtains target, and according to prediction Target behavior carry out multiple target tracking.Specifically, will two ways (i.e. global prediction and local prediction) obtain Prediction is weighted combining, and obtains the predicted state that target is final.

To sum up, the multiple-target system of Behavior-based control study according to embodiments of the present invention, it is being applied to multiple target tracking field Jing Shi, can keep tracking rate, can significantly reduce tracking error rate simultaneously, improves and follows the tracks of accuracy.

In describing the invention, it is to be understood that term " " center ", " longitudinally ", " laterally ", " length ", " width ", " thickness ", " on ", D score, "front", "rear", "left", "right", " vertically ", " level ", " top ", " end " " interior ", " outward ", Orientation or the position relationship of the instruction such as " clockwise ", " counterclockwise ", " axially ", " radially ", " circumferential " are based on shown in the drawings Orientation or position relationship, be for only for ease of describe the present invention and simplify describe rather than instruction or hint indication device or Element must have specific orientation, with specific azimuth configuration and operation, be therefore not considered as limiting the invention.

Additionally, term " first ", " second " are only used for describing purpose, and it is not intended that instruction or hint relative importance or Person implies the quantity indicating indicated technical characteristic.Thus, define " first ", the feature of " second " can express or Implicitly include at least one this feature.In describing the invention, " multiple " are meant that at least two, such as two, Three etc., unless otherwise expressly limited specifically.

In the present invention, unless otherwise clearly defined and limited, term " install ", " being connected ", " connection ", the art such as " fixing " Language should be interpreted broadly, and connects for example, it may be fixing, it is also possible to be to removably connect, or integral；Can be machinery Connect, it is also possible to be electrical connection；Can be to be joined directly together, it is also possible to be indirectly connected to by intermediary, can be two units Connection within part or the interaction relationship of two elements, unless otherwise clear and definite restriction.Ordinary skill for this area For personnel, above-mentioned term concrete meaning in the present invention can be understood as the case may be.

In the present invention, unless otherwise clearly defined and limited, fisrt feature second feature " on " or D score can be One directly contacts with second feature, or the first and second features are by intermediary mediate contact.And, fisrt feature is Two features " on ", " top " and " above " but fisrt feature directly over second feature or oblique upper, or be merely representative of first Characteristic level height is higher than second feature.Fisrt feature second feature " under ", " lower section " and " below " can be fisrt feature Immediately below second feature or obliquely downward, or it is merely representative of fisrt feature level height less than second feature.

In the description of this specification, reference term " embodiment ", " some embodiments ", " example ", " concrete example ", Or specific features, structure, material or the feature bag that the description of " some examples " etc. means to combine this embodiment or example describes It is contained at least one embodiment or the example of the present invention.In this manual, to the schematic representation of above-mentioned term necessarily It is directed to identical embodiment or example.And, the specific features of description, structure, material or feature can be arbitrary Individual or multiple embodiment or example combine in an appropriate manner.Additionally, in the case of the most conflicting, the skill of this area The feature of the different embodiments described in this specification or example and different embodiment or example can be combined by art personnel And combination.

Although above it has been shown and described that embodiments of the invention, it is to be understood that above-described embodiment is exemplary, Being not considered as limiting the invention, those of ordinary skill in the art within the scope of the invention can be to above-described embodiment It is changed, revises, replaces and modification.

Claims (8)

1. the multi-object tracking method of a Behavior-based control study, it is characterised in that comprise the following steps:

S1: obtain target video sequence, described target video sequence is detected, and obtain tracking mesh according to testing result The size and location information of mark candidate frame；

S2: multiple target real-time tracking problem is modeled, and set up the production probabilities of described multiple target real-time tracking problem Model；

S3: for the global conditions probability item in described generative statistical model, enterprising in the training set the most correctly marked Row off-line training, carries out the pervasive global behavior in various scenes and predicts, for the partial strip in described generative statistical model Part probability item utilizes each target tracking data before present frame, real-time online training to be applicable to the local behavior of this target Prediction；And

S4: combine the prediction of described global behavior and described local behavior prediction obtains the behavior prediction of target, and according to prediction Target behavior carries out multiple target tracking.

The multi-object tracking method of Behavior-based control the most according to claim 1 study, it is characterised in that in described S2, It is as follows that described generative statistical model sets up process:

Public probabilistic model shown in formula (I) is used to represent the multiple target tracking problem that solves:

Wherein,Represent the set of the state of followed the tracks of target,Represent the information aggregate of all of testing result；

Specific to the state of all targets of frame t each in described target video sequence, described public formula (I) can be deformed into:

$X_t^{*}=\underset{X_t}{\mathrm{argmax}}P\left(Z_t\right|X_t)P\left(X_t\right)---\left(II\right),$

Wherein, Z_tAnd X_tRepresent that all observations and the dbjective state of t frame, described public formula II are equivalent to frame by frame respectively Try to achieve optimal solution；

According to the Markov property of multiple target tracking problem, X_tCan be counted as by dbjective state X of former frame_t-1And close Prediction or priori in target behaviorCommon generation, the most described public formula II just can be derived as further:

Wherein, described prioriAnd state X of former frame target_t-1It is separate, particularly as follows:

Utilizing single order Markov property to carry out recursion frame by frame, described public formula III can further be deformed into:

Wherein, P (X₁) it is at the first frame initialized target state, P (Z_t|X_t) it is observed object frame and target status information The characteristics of image similarity of the target frame determined,It it is the overall priori pair of all target Common behaviors The prediction of dbjective state,Present frame shape probability of state, phase is produced in the state of former frame for each target When in the status predication to this target of the behavior according to target self.

The multi-object tracking method of Behavior-based control the most according to claim 1 study, it is characterised in that in described S3, The prediction of described global behavior specifically includes:

The method utilizing transformation of data, produces training data, and profit by the annotation results of original detection data combined training collection Neutral net is trained, so that the neutral net trained can be according to T before target with described training data_gThe status information of frame is pre- Measuring the status information of target present frame, wherein, the structure of described training data is: T before target_gThe status information of frame and The status information of corresponding present frame.

The multi-object tracking method of Behavior-based control the most according to claim 1 study, it is characterised in that in described S3, Described local behavior prediction specifically includes:

During carrying out target following, it is trained in real time, and the tracking information before target is deformed, produce Go out training data, and utilize described training data to train neutral net, so that the neutral net trained can be according to T before target_l The status information of frame dopes the status information of target present frame, and wherein, the structure of training data is, T before target_lThe shape of frame The status information of the present frame of state information and correspondence.

5. the multiple-target system of a Behavior-based control study, it is characterised in that including:

Acquisition module, described acquisition module is used for obtaining target video sequence, detects described target video sequence, and The size and location information following the tracks of target candidate frame is obtained according to testing result；

MBM, described MBM is for being modeled multiple target real-time tracking problem, and it is real to set up described multiple target Time tracking problem generative statistical model；

Prediction module, described prediction module, for for the global conditions probability item in described generative statistical model, is being entered Carry out off-line training in the training set of the correct mark of row, carry out the pervasive global behavior in various scenes and predict, for described product Local condition's probability item in raw formula probabilistic model utilizes each target tracking data before present frame, and real-time online is trained It is applicable to the local behavior prediction of this target；And

Tracking module, described tracking module obtains target for combining the prediction of described global behavior and described local behavior prediction Behavior prediction, and carry out multiple target tracking according to the target behavior of prediction.

The multiple-target system of Behavior-based control the most according to claim 5 study, it is characterised in that described production It is as follows that probabilistic model sets up process:

Public probabilistic model shown in formula (I) is used to represent the multiple target tracking problem that solves:

Wherein,Represent the set of the state of followed the tracks of target,Represent the information aggregate of all of testing result；

Specific to the state of all targets of frame t each in described target video sequence, described public formula (I) can be deformed into:

$X_t^{*}=\underset{X_t}{\mathrm{argmax}}P\left(Z_t\right|X_t)P\left(X_t\right)---\left(II\right),$

Wherein, Z_tAnd X_tRepresent that all observations and the dbjective state of t frame, described public formula II are equivalent to frame by frame respectively Try to achieve optimal solution；

According to the Markov property of multiple target tracking problem, X_tCan be counted as by dbjective state X of former frame_t-1And close Prediction or priori in target behaviorCommon generation, the most described public formula II just can be derived as further:

Wherein, described prioriAnd state X of former frame target_t-1It is separate, particularly as follows:

Utilizing single order Markov property to carry out recursion frame by frame, described public formula III can further be deformed into:

Wherein, P (X₁) it is at the first frame initialized target state, P (Z_t|X_t) it is observed object frame and target status information The characteristics of image similarity of the target frame determined,It it is the overall priori pair of all target Common behaviors The prediction of dbjective state,Present frame shape probability of state, phase is produced in the state of former frame for each target When in the status predication to this target of the behavior according to target self.

The multiple-target system of Behavior-based control the most according to claim 5 study, it is characterised in that described overall situation row Specifically include for prediction:

The method utilizing transformation of data, produces training data, and profit by the annotation results of original detection data combined training collection Neutral net is trained, so that the neutral net trained can be according to T before target with described training data_gThe status information of frame is pre- Measuring the status information of target present frame, wherein, the structure of described training data is: T before target_gThe status information of frame and The status information of corresponding present frame.

The multiple-target system of Behavior-based control the most according to claim 5 study, it is characterised in that described partial row Specifically include for prediction:

During carrying out target following, it is trained in real time, and the tracking information before target is deformed, produce Go out training data, and utilize described training data to train neutral net, so that the neutral net trained can be according to T before target_l The status information of frame dopes the status information of target present frame, and wherein, the structure of training data is, T before target_lThe shape of frame The status information of the present frame of state information and correspondence.