CN103699908A - Joint reasoning-based video multi-target tracking method - Google Patents

Abstract

The invention discloses a joint reasoning-based video multi-target tracking method in the technical field of video processing. The method comprises the following steps: firstly, reading a frame of image of a video file, carrying out image rasterization processing, and then calibrating a candidate position of a target by adopting an online detector and a KLT tracking algorithm as a tracker, respectively screening and then integrating the results; secondly, carrying out quantitative grading on the obtained candidate position result; finally, describing the target tracking condition by using a joint function, and taking the optimal solution based on the joint function as the position of the target in the frame, so as to achieve target tracking. By adopting the joint reasoning-based video multi-target tracking method, the processing of the treatment method combined with a detection tracking algorithm and multi-target relationship in the tracking technology under multi-target tracking can be achieved, and the multi-target relationship is described by using the joint function, so that not only is the result fusion of detection and tracking achieved, but also the relationship between the targets is integrated according to overall situation, so that a global optimal resolution is obtained.

Description

Video multi-target tracking based on cooperate reasoning

Technical field

What the present invention relates to is a kind of method of technical field of video processing, specifically a kind of video multi-target tracking based on cooperate reasoning.

Background technology

Along with the development of camera device and universal, video tracking produce and life in occupied more and more consequence.Particularly, in video monitoring system, track algorithm can effectively reduce cost of labor, saves time.Yet due to the imperfection of tracking technique itself and tracking environmental complicated and changeable, affected the accuracy of tracking results, the application of track algorithm is restricted.

Target following in video is very challenging problem, because there is a lot of uncertain factors in tracing process, such as: complicated background, if background is similar to tracked target, can disturb the judgement of track algorithm to target location; Obvious object blocks, the quick variation of target shape, and these all can cause the outward appearance of target in picture to have obvious variation, causes algorithm to lose tracked target.For multiple target tracking, except above-mentioned problem, the similarity between each target, interaction and mutually block all and can bring difficulty to correct tracking.

The method that adopts often detection algorithm to combine with track algorithm for the general disposal route of these problems, namely utilizes detection algorithm to improve the effect of final tracking.Yet, in the correctness of detection algorithm, be difficult under the prerequisite of assurance, whether can improve tracking rate and be still a problem.

Literature search through prior art is found, a lot of scholars utilize the method for adaptive learning to guarantee the correctness of detecting device, thereby process the problem that testing result and tracking results are coordinated mutually, " Tracking-Learning-Detection " delivering on < < IEEE Transaction on Pattern Analysis and Machine Intelligence > > (the 34th phase 1409-1422 page in 2012) such as Zdenek etc., this article has utilized the framed structure of " following the tracks of-study-detect (TLD) ", utilize and follow the tracks of and detect the method processing tracking problem combining.Specifically, first, input one two field picture, after image grid, obtains a large amount of image block; Then, tracker filters out satisfactory image block, and detecting device is also demarcated all image block that may become in appearance target; Finally, the method that adopts tracking link and detection to combine can utilize the result of detecting device to reinitialize tracker when tracker failure, utilizes tracking results to expand training sample set simultaneously and trains online detecting device, improves the precision of detecting device.From the disclosed experimental result of this technology, TLD has good effect on long-time tracking.Still there are a lot of restrictions in the method but: 1) be only applicable to monotrack 2) if there is larger variation in the outward appearance of target, or occurred to block completely, the method effect is bad, and this is because can not correctly provide in this case the possible position of target at thread detector.

Chinese patent literature CN103176185A Shen Qing Publication day: 2013.06.26, a kind of method and system for detection of road barricade thing are disclosed, the first detection of obstacles model of this technology based on video shooting device, the second detection of obstacles model based on video shooting device and millimetre-wave radar, the 3rd detection of obstacles model based on three-dimensional laser radar and infrared pick-up device, and make described a plurality of model form complementary detection by the fuzzy neural network algorithm based on rough set, thereby the characteristic information of Real-time Obtaining road barricade thing.But this technical equipment cost is high, thereby can not do effective tracking to barrier, in conjunction with historical information, makes and follow testing result accurately.

Summary of the invention

The present invention is directed to prior art above shortcomings, a kind of video multi-target tracking based on cooperate reasoning is proposed, can solve under multiple target tracking in tracking technique for detecting the disposal route of track algorithm combination and the processing of multiple goal mutual relationship, utilize Copula to describe relation between multiple goal, not only solved to detect with the result of following the tracks of and merged problem, also from the overall situation, consider simultaneously, combine the relation between each target, drawn globally optimal solution.

The present invention is achieved by the following technical solutions, first the present invention by reading in a two field picture of video file and it being carried out to image grid processing, then adopt at thread detector and as the position candidate of the KLT track algorithm spotting of tracker, synthesis result after screening respectively, secondly the position candidate result drawing is quantized to scoring, finally utilize Copula target following situation is described and using the optimum solution based on Copula as target in the position of this frame, realize target is followed the tracks of.

The first frame when described image is video file first carries out initialization operation before image gridization is processed, and is specially: manually input needs the target number of following the tracks of, and then manually frame goes out target.

Described initialization operation refers to: initialization detecting device and tracker, by the method for random fern on-line study, upgrade detecting device, and the tracker based on KLT algorithm also can be selected unique point in target zone simultaneously, for the target following of next frame.

Described image gridization is processed, refer to: with moving window not of uniform size, scan whole two field picture and obtain the image block that diverse location varies in size, be used as candidate target, be specially: first according to the size of initialization target, geometric ratio draws the moving window of a series of different scale sizes, and the proportional range of change of scale is 1.2 ^-10～1.2 ¹⁰; Each moving window is successively according to order from top to bottom traversal entire image from left to right, and moving window displacement size is 0.1 of window size.Like this, just can obtain the image block that a lot of diverse locations vary in size.

Described position candidate obtains in the following manner:

First, computed image piece density variance, too small when image block density variance, be excluded, tracked target template image is obtained when the first frame initialization, and the concrete poor computing formula in image block side is:

wherein: p _irefer to the gray level image of i width image block, E () refers to and is averaging function, when

i width image block is just retained, wherein: l is preset parameter, the variance that represents template image.

Then, input picture block is extracted to two value tags, the sorter that utilizes the online training of random fern algorithm to obtain estimates that each passes through image block that density variance judges and the similarity of detected target, and similarity judgment formula is: P (c ₁| x)= wherein: c ₁represent training classification, while training, only have two kinds here, use c similar to detected target ₁represent, dissimilar with detected target, use c ₀represent; P _i(c ₁| x) represent i the posterior probability that fern obtains.

Finally, the posterior probability that all ferns are obtained averages, and obtains final posterior probability values, as similarity P (c ₁| x) >50%, input picture piece is similar to detected target, retains this image block.

Described demarcation refers to: when the first frame of image, do KLT algorithm initialization and process, do not follow the tracks of, each frame all utilizes KLT algorithm picks tracked target feature from previous frame target location afterwards, finds the characteristic area of answering in contrast in present frame; Then tracker determines whether retaining according to the unique point number of each image block inside of image block, if the unique point number in an image block areas has surpassed certain empirical value, this image block is retained with regard to being judged as candidate state so.

Described quantification scoring refers to: extract the Haar feature of candidate image piece, and by the authenticity of cascade Adaboost sorter evaluate candidate position, the evaluation quantizing:

wherein:

the meaning be detected target be parked in of cascade Adaboost sorter

layer,

represented detected rectangle frame

quantification scoring;

wherein: represented a Weak Classifier in cascade classifier, s (L) has represented a series of Weak Classifiers of L layer; Work as function F _ibe greater than certain empirical value, so just can pass through this one deck, otherwise can not pass through; Described Weak Classifier weight w _i,lto learn to draw by the learning method of AdaBoost.

Described Copula is comprised of the scoring of the spatial relation between each target and each target candidate position, by building this Copula model, asks the optimum solution of function, just can show that best position candidate is as the result of following the tracks of.

Described spatial relation refers to, in multiobject situation, also can think to change also not remarkable at the position relationship between each target between consecutive frame.Mutual alignment between previous frame target also can be used as the tracking results that reference Help improves this frame so.Based on this principle, that t moment, the position relationship probability between i target and N target

be described as:

$d(x_i^t,x_N^t)=e^{-{\mathrm{\&beta;}}_1{\left(\mathrm{\&Delta;v}\right)}^2-{\mathrm{\&beta;}}_2{\left(\mathrm{\&Delta;\&theta;}\right)}^2},$ Wherein: $\mathrm{\&Delta;\&nu;}=\mathrm{\&nu;}(x_i^t,x_N^t)-\mathrm{\&nu;}(x_i^{t-1},x_N^{t-1}),\mathrm{\&Delta;\&theta;}=\mathrm{\&theta;}(x_i^t,x_N^t)-$

function ν and function # have represented respectively distance and the angle in theorem in Euclid space.β ₁and β ₂played the effect of equilibrium distance and angle weight.Function

describe the spatial positional information between t moment target i and target N, comprised the variation of distance and the variation of angle.

Described Copula model refers to, the joint probability of all targets, $p(x_1^t,......,x_N^t)\&Proportional;\exp ({\mathrm{\&Sigma;}}_{i=1}^{N}g\left(x_i^t\right)+\mathrm{\&gamma;}{\mathrm{\&Sigma;}}_{i=1}^{N-1}d(x_i^t,x_N^t)),$ Wherein: γ is weight parameter, function

represented the t state space degree of confidence of target i constantly, function

describe the spatial positional information between t moment target i and target N, comprised the variation of distance and the variation of angle.

Described optimum solution utilizes standard belief propagation algorithm effectively to solve multiple target tracking problem, and the relation between each target is described with tree construction; When tracked target is the node in markov random file, a selected target is root node at random, and all the other are child node, to root node transmission of information, last object definition, are root node, other as the child node in tree construction; Constructor node is to the information transport function of root node transmission of information like this:

each leaf node carries the information to root node place, and root node is that the degree of confidence of tracked target N Nodes can be expressed as so:

when

while obtaining maximal value, it is the tracking results of this frame multiple target tracking.

Technique effect

Although the position candidate of target can be detected rapidly at thread detector, produce more error result, false alarm rate is higher.Secondly, at thread detector only to appearance change target susceptibility slowly.In the situation that training sample is few, for Rotation, non-rigid targets etc., the method is unsettled.With at thread detector, compare, offline inspection is more accurately but speed is slower.For the effect of these two kinds of detecting devices of balance, so the present invention uses online detection to carry out spotting position candidate, and utilize offline inspection to come, to position candidate scoring, to have taken into account the advantage of two kinds of technology.

The present invention not only balance offline inspection and detecting online, make detecting device have better robustness, can reach better testing result, be used for improving final tracking effect.And proposed to utilize Copula to describe target following situation, thereby integrated the structural information between multiple goal, simultaneously also effective integration the result of detecting device and tracker.Compare with general tracking, the present invention has accomplished to solve tracking problem from overall angle, can better solve the problems such as occlusion issue in multiple target tracking process and appearance change, and it is all had wide practical use in daily life with in producing.

Accompanying drawing explanation

The treatment scheme schematic diagram of Fig. 1 embodiment of the present invention.

Fig. 2 application example schematic diagram 1 of the present invention.

Fig. 3 application example schematic diagram 2 of the present invention.

Embodiment

Below embodiments of the invention are elaborated, the present embodiment is implemented take technical solution of the present invention under prerequisite, provided detailed embodiment and concrete operating process, but protection scope of the present invention is not limited to following embodiment.

Embodiment 1

The implementation environment of the present embodiment is, for removable or fixing camera, in indoor or outdoors collection, gets video, and available the present invention does to follow the tracks of and processes.Concrete using method is, first, the place frame followed the tracks of in needs monitoring good fixable or movably camera do the use of image data; Then, gather the video information input computer getting, with the present invention, analyze, the present invention can require input to need tracked target number at video flowing the first frame, manually with rectangle frame, goes out tracked target; Finally, meeting of the present invention is automatically accurately followed the tracks of and is marked and is selected target in video flowing.

As shown in Figure 1, the present embodiment comprises following operation steps: first read a two field picture of input video, if the first frame is done initialization setting; Afterwards, this two field picture rasterizing is input at thread detector and tracker and calibrates candidate target position after processing; Then, by candidate target Position input scorer, utilize off-line sorter to each position candidate, to do the scoring of a quantification; Finally, by the position candidate information quantizing and each object space structural information input Copula, by optimum solution, calculate the tracking results of each target of output.

As shown in Figure 1, the present embodiment first user is inputted video to be tracked, reads a two field picture of video, when video the first frame, carries out initialization process, by user, selects voluntarily target to be tracked, can be one or more.After initial target has been selected, the present invention follows the tracks of target automatically.

Next, this two field picture is carried out to rasterizing, obtain a lot of image blocks not of uniform size as preliminary position candidate.Again by rasterizing image difference Input Online detecting device and tracker.At thread detector, first get rid of the too small image block of density variance, i.e. the less image block of inclusion information amount, the feature of Program extraction residual image piece then, the detecting device by online training retains satisfactory image block as candidate target position; In tracker, first program chooses key point from the objective result of previous frame, then in this frame, find corresponding key point, meeting of the present invention is according to fixed empirical value in experiment, by analyzing the number of the key point being traced in each image block, decide and retain those image blocks as target candidate position.

Obtain behind target candidate position, this program can quantize scoring to each candidate image piece with off-line sorter, then in conjunction with the Spatial Relational Model between each target, input Copula is described the tracking situation of all targets.Program is using the optimum solution of Copula as last tracking results output.Obtain after the tracking results of present frame, program also can, to training at thread detector, guarantee the stability of next frame testing result.

After each frame of video image all travels through, the present embodiment finishes.

As shown in Figure 2, the present embodiment application example schematic diagram.In this example, the parameter of above mentioning in formula is set to: λ=10, γ=60, β ₁=0.1, β ₂=2, in scene, 4 targets are walked to the right from the left side of camera lens simultaneously, and target is not only leaned on very closely each other, can produce mutually and block, and the profile of 4 tracked targets is also rather similar, to following the tracks of, bring certain difficulty.This method is owing to having used structural inference function, consider that from overall angle thereby all target following results try to achieve optimum solution, reduced the impact of mutually blocking between target, frame#187 Green target is not subject to the impact of similar object around in the process of following the tracks of yet, and very accurately 4 targets is followed the tracks of simultaneously.From frame#220, can obviously find out, this method has good tracking effect under this scene.

As shown in Figure 3, be also application example schematic diagram of the present invention.In this example, the parameter of above mentioning in formula is set to: λ=10, γ=60, β ₁=0.1, β ₂=2, for the trace model of this method is had to direct cognition, structural object has been done to tracking and testing.Test video shown in Fig. 3 is recorded voluntarily, eye, nose and the corners of the mouth of tracked target behaviour face.People's face can be seen as a structural object, 5 parts of fixing, consists of, and centered by nose, surrounding has two eyes and the corners of the mouth.The algorithm that this method proposes can well be processed this occlusion issue, even if there be blocking of moment, trace model does not have lose objects yet herein.This is because structural reasoning joint probability function in this paper has utilized the information of the mutual relationship between each target, rather than only consider the probability of single target, when tracker and detecting device lose objects, still can infer the position that tracked target according to the mutual spatial relationship between target.

Claims (10)

1. the video multi-target tracking based on cooperate reasoning, it is characterized in that, by first reading in a two field picture of video file and it being carried out to image grid processing, then adopt at thread detector and as the position candidate of the KLT track algorithm spotting of tracker, synthesis result after screening respectively, secondly the position candidate result drawing is quantized to scoring, finally utilize Copula target following situation is described and using the optimum solution based on Copula as target in the position of this frame, realize target is followed the tracks of.

2. method according to claim 1, it is characterized in that, described initialization operation refers to: initialization detecting device and tracker, method by random fern on-line study is upgraded detecting device, tracker based on KLT algorithm also can be selected unique point in target zone simultaneously, for the target following of next frame.

3. method according to claim 1, it is characterized in that, described image gridization is processed, refer to: with moving window not of uniform size, scan whole two field picture and obtain the image block that diverse location varies in size, be used as candidate target, be specially: first according to the size of initialization target, geometric ratio draws the moving window of a series of different scale sizes, and the proportional range of change of scale is 1.2 ^-10～1.2 ¹⁰; Each moving window is successively according to order from top to bottom traversal entire image from left to right, and moving window displacement size is 0.1 of window size, obtains the image block that a lot of diverse locations vary in size.

4. method according to claim 1, is characterized in that, described position candidate obtains in the following manner:

First, computed image piece density variance, too small when image block density variance, be excluded, tracked target template image is obtained when the first frame initialization, and the concrete poor computing formula in image block side is: wherein: p _irefer to the gray level image of i width image block, E () refers to and is averaging function, when

i width image block is just retained, wherein: l is preset parameter,

the variance that represents template image;

Then, input picture block is extracted to two value tags, the sorter that utilizes the online training of random fern algorithm to obtain estimates that each passes through image block that density variance judges and the similarity of detected target, and similarity judgment formula is: Pc ₁| x)=

wherein: c ₁represent training classification, while training, only have two kinds here, use c similar to detected target ₁represent, dissimilar with detected target, use c ₀represent; P _i(c ₁| x) represent i the posterior probability that fern obtains;

Finally, the posterior probability that all ferns are obtained averages, and obtains final posterior probability values, as similarity P (c ₁| x) >50%, input picture piece is similar to detected target, retains this image block.

5. method according to claim 1, it is characterized in that, described demarcation refers to: when the first frame of image, do KLT algorithm initialization and process, do not follow the tracks of, each frame all utilizes KLT algorithm picks tracked target feature from previous frame target location afterwards, finds the characteristic area of answering in contrast in present frame; Then tracker determines whether retaining according to the unique point number of each image block inside of image block, if the unique point number in an image block areas has surpassed certain empirical value, this image block is retained with regard to being judged as candidate state so.

6. method according to claim 1, is characterized in that, described quantification scoring refers to: extract the Haar feature of candidate image piece, and by the authenticity of cascade Adaboost sorter evaluate candidate position, the evaluation quantizing:

wherein:

the meaning be detected target

be parked in of cascade Adaboost sorter

layer,

represented detected rectangle frame

quantification scoring;

wherein:

represented a Weak Classifier in cascade classifier, s (L) has represented a series of Weak Classifiers of L layer; Work as function F _ibe greater than certain empirical value, so

just can pass through this one deck, otherwise can not pass through; Described Weak Classifier weight w _i,lto learn to draw by the learning method of AdaBoost.

7. method according to claim 1, it is characterized in that, described Copula is comprised of the scoring of the spatial relation between each target and each target candidate position, by building this Copula model, ask the optimum solution of function, just can show that best position candidate is as the result of following the tracks of.

8. method according to claim 7, it is characterized in that, described spatial relation refers to, in multiobject situation, mutual alignment between previous frame target also can be used as the tracking results that reference Help improves this frame, therefore the position relationship probability between i target and N target at t constantly,

be described as:

$d(x_i^t,x_N^T)=e^{-{\mathrm{\&beta;}}_1{\left(\mathrm{\&Delta;v}\right)}^2-{\mathrm{\&beta;}}_2{\left(\mathrm{\&Delta;\&theta;}\right)}^2},$ Wherein: $\mathrm{\&Delta;\&nu;}=\mathrm{\&nu;}(x_i^t,x_N^t)-\mathrm{\&nu;}(x_i^{t-1},x_N^{t-1}),\mathrm{\&Delta;\&theta;}=\mathrm{\&theta;}(x_i^t,x_N^t)-$

function ν and function # have represented respectively distance and the angle in theorem in Euclid space; β ₁and β ₂played the effect of equilibrium distance and angle weight; Function

describe the spatial positional information between t moment target i and target N, comprised the variation of distance and the variation of angle.

9. method according to claim 7, is characterized in that, described Copula model refers to, the joint probability of all targets,

wherein: ε and γ are weight parameter, function

represented the t state space degree of confidence of target i constantly, function

describe the spatial positional information between t moment target i and target N, comprised the variation of distance and the variation of angle.

10. method according to claim 7, is characterized in that, described optimum solution utilizes standard belief propagation algorithm effectively to solve multiple target tracking problem, and the relation between each target is described with tree construction; When tracked target is the node in markov random file, a selected target is root node at random, and all the other are child node, to root node transmission of information, last object definition, are root node, other as the child node in tree construction; Constructor node is to the information transport function of root node transmission of information like this:

each leaf node carries the information to root node place, and root node is that the degree of confidence of tracked target N Nodes can be expressed as so: when

while obtaining maximal value, it is the tracking results of this frame multiple target tracking.

Images