CN101561928A - Multi-human body tracking method based on attribute relational graph appearance model - Google Patents

Abstract

Aiming at the problem of multi-human bodies tracking in fixed monitoring scene, the invention provides a multi-human body tracking method based on an attribute relational graph appearance model. The method comprises the following steps: firstly, setting up the attribute relational graph appearance model for a current frame human body detection region; secondly, computing the similarity between the attribute relational graph appearance model of tracking human body at the current frame and the attribute relational graph appearance model of tracking human body at the former frame; thirdly, ensuring the matching of interframe human body according to the similarity so as to ensure human body tracking condition and acquire motion track. The method can greatly improve the accuracy, the reliability and the real-time performance of the multi-human body tracking in the fixed monitoring scene.

Description

Multi-human body tracking method based on attribute relational graph appearance model

Technical field

What the present invention relates to is multi-human body tracking method in the fixed monitoring scene, specifically is a kind of multi-human body tracking method based on attribute relational graph appearance model.

Background technology

In higher public places of safety requirements such as bank, hotel, subways, the suspicious people who needs discovery in time to have abnormal behaviour.Adopt the fixing mode monitoring scene of video camera at present in these places mostly, therefore the computer vision monitoring need realize the accurate tracking to a plurality of human body compound movement situations under fixed scene, and this also is the basis that follow-up behavior is understood, unusual trace is analyzed the contour level visual processes.Because tracing process promptly is the different human body in the identification monitor video successive frame, and the human appearance feature is in most cases to distinguish the important evidence of different human body, therefore the tracking based on the human appearance model is subjected to extensive concern.At present, the method for following the tracks of human body based on display model mainly contains three kinds: 1) color histogram method: in the human appearance feature, color is an important clue, and traditional method of setting up display model is to adopt color histogram.But, owing to only obtaining the color statistical value, color histogram lost spatial information, cause mistake easily, the people who wears green upper garment, red trousers as the people and of the whole body red coloration upper garment, green trousers, the color histogram that their outward appearance is set up is almost completely consistent, can't only distinguish out two people according to its color histogram.2) method that combines with spatial information of color histogram: when calculating color histogram, add spatial information, as proposing body shape is divided into three parts, each several part is described its color characteristic with color histogram, and realize human body tracking with Condensation (bunching) algorithm, but, this method lack the partes corporis humani divided between the description of structural relation, follow the tracks of accuracy, reliability is low; 3) color correlogram method: the color correlogram calculates the probability of certain distance of being separated by between two pixels with co-occurrence matrix.Though the color correlogram can reflect different colours between spatial coherence, but since need to adopt three-dimensional array characterize different colours to and distance, make the method computation complexity very high, can't be used for monitoring in real time follows the tracks of, and lack the analysis to the target travel situation in the document, accuracy is followed the tracks of in influence.

Therefore, following the tracks of the subject matter that the method for human body exists based on display model has: how to set up the accuracy that display model is more accurately described human body, how to be improved many human body tracking.

Summary of the invention

Inventing technical matters to be solved is to propose a kind of multi-human body tracking method based on attribute relational graph appearance model, with feature combinations such as human appearance color, space structures, set up human appearance model more accurately and effectively, under the situation that monitoring scene is fixed, improve accuracy, the validity of many human body tracking.

For achieving the above object, technical scheme of the present invention is:

A kind of multi-human body tracking method based on attribute relational graph appearance model is characterized in that, may further comprise the steps:

1) obtains the image of the fixed scene that includes tracked human body continuously, attribute relational graph appearance model is set up in the human detection zone of current frame image: the human region that detects is divided into head, upper body and three body parts of lower limb; The corresponding described body part of each node of attribute relational graph appearance model, nodal community characterizes the feature of body part, is shown by following three scales: the mark of corresponding body part, color histogram, boundary rectangle frame center point coordinate; The limit of attribute relational graph appearance model is represented that by the straight-line segment of the central point that connects any two body parts of human body side attribute adopts the length and the angle of straight-line segment to represent;

2) calculate the attribute relational graph appearance model of present frame human body and the similarity that previous frame is followed the tracks of the attribute relational graph appearance model of human body;

3) follow the tracks of the present frame human body according to the similarity of calculating gained.

Described nodal community is defined as Δ (θ _i)={ ω _i, p _i, hist _i, wherein, ω _iThe mark of representing the body part of i tracked object: head, upper body, lower limb are labeled as " head ", " up " and " low " respectively; p _iThe center point coordinate of representing the body part boundary rectangle frame of i tracked human body; Hist _iBe the color characteristic of the body part of i tracked human body, hist _iBe expressed as with normalized HSV space one dimension color histogram: hist _i=9H+3S+V; Wherein, the value of H, S, V remarked pixel point three-component H, S, V in the HSV space;

Side attribute is defined as: r (θ _i, θ _j)={ r _l, r _g(i ≠ j), wherein r _lBe length attribute, represent two node θ _iAnd θ _jBetween length of straigh line, adopt the Euclidean distance between the central point in the nodal community to represent: r _l=| p _i-p _j|, p _i, p _jRepresent node θ respectively _iAnd θ _jAttribute in center point coordinate; r _gBe the angle attribute, represent this two node θ _iAnd θ _jBetween the straight-line segment angle, $r_g=\arcsin \frac{y_i-y_j}{r_l},$ Y wherein _i, y _jBe node θ _i, θ _jAttribute in central point y direction coordinate, r _gSpan be [0, π];

Described similarity is defined as follows:

$\mathrm{\τ}(G,\hat{G})=\underset{i}{\mathrm{\Σ}}{\mathrm{\λ}}_1{\mathrm{\δ}}_1(\mathrm{\Δ}\left({\mathrm{\θ}}_i\right),\mathrm{\Δ}\left({\widehat{\mathrm{\θ}}}_i\right))+\underset{i}{\mathrm{\Σ}}\underset{j}{\mathrm{\Σ}}{\mathrm{\λ}}_2{\mathrm{\δ}}_2(r({\mathrm{\θ}}_i,{\mathrm{\θ}}_j),\hat{r}\left({{\widehat{\mathrm{\θ}}}_i,\widehat{\mathrm{\θ}}}_j\right))$ (i≠j)

Wherein, λ ₁, λ ₂Be weight coefficient, and constraint condition is: λ ₁+ λ ₂=1, make similarity τ within interval [0,1], δ ₁(Δ (θ _i),

The expression G and

In the internodal similarity of corresponding similar body part, δ ₂(r (θ _i, θ _j),

Be G and

In connect the similarity on the limit of two similar body parts, G and

Be respectively the attribute relational graph appearance model of present frame human body and the attribute relational graph appearance model that previous frame is followed the tracks of human body;

Described step according to similarity tracking present frame human body is: set up the coupling matrix, list each surveyed area and previous frame and follow the tracks of the similarity of human body: the row of establishing the coupling matrix number is represented present frame human detection zone, row number represents previous frame tracking human body, with the matching degree τ of attribute relational graph appearance model _NmAs the matrix element value record, n represents the sequence number of present frame human body, and m represents the mark of former frame tracking human body, then according to matching threshold τ ₀Determine whether coupling, matching result is

By the situation of coupling matrix, determine many human bodies in the present frame tracking mode, obtain human body motion track, and upgrade its display model; Match condition reduces four kinds:

1) new human body occurs: τ ' during n is capable _NmBe 0: and sustained continuous three frames, thinking that then " new human body " occurs, human body n distributes new mark for human body newly occurring, sets up its attribute relational graph appearance model, begins to follow the tracks of this human body;

2) the normal tracking: n is capable and only has a τ ' _Nm=1 element: then human body n mates with tracking human body m, think " the normal tracking ", be human body n distribute labels m, write down its boundary rectangle frame center position, and with the display model of human body n as following the tracks of the attribute relational graph appearance model that human body m upgrades;

3) block tracking: n is capable a plurality of element τ ' to occur _Nm=1: then think " blocking tracking ", human body n is made up of a plurality of human bodies that block, and the human body that is blocked is τ ' _NmThe tracking human body of=1 element column correspondence;

4) human body disappears: m column element τ ' _NmBe 0: and this situation occurs at continuous three frames, then thinks " human body disappearance ", promptly follows the tracks of human body m and disappears, this human body of deletion from tracking sequence.

As improvement, under the situation of blocking, the position of adopting the Kalman filter prediction to block human body, thus definite attribute relational graph appearance model that respectively blocks human body is finished the tracking under the situation of blocking.

The present invention at first sets up attributed relational graph to the detected human body connected region of present frame and describes the human appearance feature, calculate the similarity of following the tracks of the attribute relational graph appearance model of human body with previous frame again, judge interframe human body match condition by similarity, follow the tracks of human body according to different match condition, thereby realize the many human body tracking in the successive frame.Particular content is as follows:

(1) detected human body is set up attribute relational graph appearance model

The present invention proposes to utilize attributed relational graph to set up the human appearance model.At first detected human region is divided into head, upper body, three parts of lower limb in the health ratio, the node of attribute relational graph appearance model is represented body part, nodal community is defined as mark, color histogram, the boundary rectangle frame center point coordinate of body part, the limit of attribute relational graph appearance model is expressed as the straight-line segment that connects any two body parts of human body, and side attribute adopts length, the viewpoint definition of straight-line segment.

(2) similarity of calculating human body attribute relational graph appearance model

In order to measure the similarity of many human appearance of interframe model, need computation attribute graph of a relation display model similarity.Because the attributed relational graph similarity comprises nodal community similarity, side attribute similarity, the present invention is according to the definition of nodal community, side attribute, the similarity of each attributive character in analysis, computing node attribute, the side attribute draws human body attribute relational graph appearance model similarity thereby derive respectively.

(3) follow the tracks of many human bodies according to different match condition

According to the similarity of interframe human body attribute relational graph appearance model, set up the coupling matrix, analyze the match condition that draws the interframe human body and comprise four kinds: new human body occurs, human body disappears, normally follows the tracks of, blocks tracking.Follow the tracks of human body according to different match condition respectively, the situation of " blocking tracking " in this way, owing to can't detect the position of blocking human body, adopt the prediction of Kalman wave filter to block the position of human body at present frame, thereby set up the attribute relational graph appearance model that blocks human body, realize blocking the many human body tracking under the situation.

Beneficial effect:

Compared with prior art, superiority of the present invention is embodied in:

1, proposes to set up the human appearance model with attributed relational graph, can not only characterize the color and the space characteristics of each body part of human body, and structural relation between each body part has been described, can describe the human appearance feature more accurately, thereby improve the accuracy of follow-up many human body tracking greatly;

2, according to the definition of human body attribute relational graph appearance model, the similarity of each attributive character in nodal community by analytic attribute graph of a relation display model, the side attribute, derivation calculates the similarity of attribute relational graph appearance model, and it is more easy, effective to make similarity calculate;

3, according to the attribute relational graph appearance model similarity of deriving and calculating, four kinds of match condition of many human bodies of having set up the coupling matrix analysis between successive frame, and realize many human body tracking under the different situations in view of the above, obtain movement locus, thereby can more effectively realize the tracking of many human bodies under the complex situations, and the matching algorithm complexity is low, has improved the real-time of tracker.

Description of drawings:

Fig. 1 the inventive method overview flow chart

Fig. 2 makes up the human body attribute relational graph appearance model.A) the boundary rectangle collimation mark is known the human detection zone; B) each human body is divided into three body parts, and sets up the human body attribute relational graph appearance model in view of the above.

Fig. 3 makes up and blocks the human body attribute relational graph appearance model.A) respectively block the estimation range of human body; B) block the human body attribute relational graph appearance model.

The tracking results of the indoor monitor video sequence of Fig. 4.A)-f) be respectively the 71st frame, 106 frames, 138 frames, 176 frames, 206 frames and 241 frame tracking results.

Fig. 5 tracking error figure.

Embodiment

Below in conjunction with specific embodiment the present invention is described in further detail.

Embodiment 1: the general flow chart of present embodiment such as Fig. 1.

1, detected human body is set up attribute relational graph appearance model

After adopting the background subtraction method to detect the target prospect connected region, annotate surveyed area (a)) as Fig. 2 with the boundary rectangle collimation mark.Next need the surveyed area selected characteristic and set up the human appearance model, with this model as following the tracks of according in successive video frames, following the tracks of human body.Therefore, accurately whether the Feature Selection of surveyed area and the model of foundation directly influence tracking results.The present invention combines color, the spatial structure characteristic of human appearance, proposes to set up the human appearance model with attributed relational graph.

Owing to only human body is followed the tracks of, then the zone is surveyed in the health check-up of leting others have a look at of the boundary rectangle frame table of prospect connected region.According to the ratio of human body, as Fig. 2 b) shown in, connected region in the rectangle frame is divided into three the corresponding expression of part heads, upper body, lower limb, as the node θ of attributed relational graph _i(be i ∈ head, up, low}); The limit is represented by the straight-line segment that connects any two body parts of human body.Then utilize attributed relational graph to set up the human appearance model: the G={ ∑ _Δ, ∑ _r, ∑ wherein _Δ={ Δ (θ _Head), Δ (θ _Up), Δ (θ _Low) expression nodal community set, ∑ _r={ r (θ _Head, θ _Up), r (θ _Up, θ _Low), r (θ _Head, θ _Low) set of expression side attribute, being defined as follows of nodal community and side attribute:

1) defined node attribute: Δ (θ _i)={ ω _i, p _i, hist _i}

● ω _iThe mark of expression human body part: head, upper body, lower limb are labeled as " head ", " up " and " low " respectively;

● p _iThe central point of expression body part boundary rectangle frame;

● hist _iThe color characteristic of expression body part adopts the color histogram of this body part to represent.

Choose HSV space, the rgb value of each pixel in this body part is converted into the HSV value with human visual system, and in order to reduce computation complexity, hist _iAdopt normalized HSV space one dimension color histogram:

hist _i＝9H+3S+V (1)

Wherein, the value of H, S, V remarked pixel point three-component H, S, V in the HSV space.

2) definition side attribute: r (θ _i, θ _j)={ r _l, r _g(i ≠ j)

The feature on side attribute reflection limit is used to characterize two internodal space structure relations.Because the limit is expressed as the straight-line segment between any two body parts of human body, then side attribute adopts two attributes of length, angle of this straight-line segment to represent:

● r _l: length attribute is represented two node θ _iAnd θ _jBetween length of straigh line, adopt the Euclidean distance between the central point in the nodal community to represent: r _l=| p _i-p _j| (p _i, p _jRepresent node θ respectively _iAnd θ _jAttribute in center point coordinate);

● r _g: these two node θ of angle attribute representation _iAnd θ _jBetween the straight-line segment angle:

$r_g=\arcsin \frac{y_i-y_j}{r_l},$ Y wherein _i, y _jBe node θ _i, θ _jAttribute in central point y direction coordinate, r then _gSpan be [0, π].

Fig. 2 b) be depicted as the attribute relational graph appearance model of human body, its node represented by circle, and the limit represented by the line segment of band arrow, wherein the zone of arrow points generation relation on attributes.The node of the corresponding attributed relational graph of the round dot among the figure in the human body, the limit of the corresponding attributed relational graph of straight line.As seen, attribute relational graph appearance model has vividly described the structure and the pattern feature of the body part of human body, and then all also are converted into calculating between the attribute relational graph appearance model accordingly at the calculating between the pattern.

2, calculate the similarity of human body attribute relational graph appearance model

Because following the tracks of many human bodies promptly is to discern different human body in the monitor video successive frame, after adopting attributed relational graph to set up display model to each human detection zone, many human body tracking need be mated the different human body display model in successive frame, therefore solving tracking problem is to obtain human body attributed relational graph similarity between successive frame.

Because attributed relational graph comprises a plurality of nodes, limit, then the similarity of attributed relational graph is calculated needs under the constraint of nodal community, side attribute, seek a kind of optimum corresponding relation between two attributed relational graphs, this is a combinatorial optimization problem, solution commonly used such as decision tree search, simulated annealing etc.And the different body parts of human body are represented on node, limit in the human body attributed relational graph that the present invention proposes, the feature and the structure of the different body parts of their property value reflection human body, therefore the similarity of human body attributed relational graph is calculated between the nodal community that only needs at the similar body part of correspondence, the side attribute and is carried out, rather than combinatorial optimization problem.Human body attribute relational graph appearance model below in conjunction with the present invention's definition provides human body attribute relational graph appearance model calculation of similarity degree:

If the attribute relational graph appearance model in a certain human detection zone is G in the present frame, the attribute relational graph appearance model of a certain tracking human body is in the former frame

Definition identical with G, its nodal community

And side attribute is expressed as respectively $\mathrm{\Δ}\left({\widehat{\mathrm{\θ}}}_i\right)=\{{\widehat{\mathrm{\ω}}}_i,b\hat{l}{\mathrm{ob}}_i,h\hat{i}{\mathrm{st}}_i\},$ $r({\widehat{\mathrm{\θ}}}_i,{\widehat{\mathrm{\θ}}}_j)=\{{\hat{r}}_l,{\hat{r}}_g\}.$ Because attributed relational graph comprises three nodes and three limits among the present invention, and the body part of each node, the corresponding human body in limit and mutual structural relation thereof, therefore similarity only need be carried out between the node of representing similar human body part and limit, and it is as follows then to define similarity:

$\mathrm{\τ}(G,\hat{G})=\underset{i}{\mathrm{\Σ}}{\mathrm{\λ}}_1{\mathrm{\δ}}_1(\mathrm{\Δ}\left({\mathrm{\θ}}_i\right),\mathrm{\Δ}\left({\widehat{\mathrm{\θ}}}_i\right))+\underset{i}{\mathrm{\Σ}}\underset{j}{\mathrm{\Σ}}{\mathrm{\λ}}_2{\mathrm{\δ}}_2(r({\mathrm{\θ}}_i,{\mathrm{\θ}}_j),\hat{r}({\widehat{\mathrm{\θ}}}_i,{\widehat{\mathrm{\θ}}}_j)),(i\≠j)---\left(2\right)$

Wherein, first representation attribute graph of a relation display model G and In the internodal nodal community matching degree of all corresponding similar body parts, all connect the matching degree of the side attribute between limits of similar body part second expression.λ ₁, λ ₂Be weight coefficient, and constraint condition is: ∑ λ _1,2=1, make similarity τ within interval [0,1], δ ₁(Δ (θ _i), The internodal similarity of representing corresponding similar body part, δ ₂(r (θ _i, θ _j), Similarity between the limit of two similar body parts of expression connection.

1) computing node attribute similarity δ ₁(Δ (θ _i),

In formula (2), at the similar body part of two human bodies, nodal community Δ (θ _i)={ ω _i, p _i, hist _iMiddle mark ω _iBe identical, as mate " head " of two human bodies, i.e. mark ω _iBe " head ", then can redefine the two nodal community Δ (θ that need to calculate similarity _i), Δ (θ _i)={ p _i, hist _i, $\mathrm{\Δ}\left({\widehat{\mathrm{\θ}}}_i\right)=\{{\hat{p}}_i,h\hat{i}{\mathrm{st}}_i\},$ Therefore the nodal community similarity is defined as the weighted sum of position vector similarity and color histogram similarity:

${\mathrm{\δ}}_1(\mathrm{\Δ}\left({\mathrm{\θ}}_i\right),\mathrm{\Δ}\left({\widehat{\mathrm{\θ}}}_i\right))={\mathrm{\λ}}_p{\mathrm{\δ}}_p(p_i,{\hat{p}}_i)+{\mathrm{\λ}}_h{\mathrm{\δ}}_h({\mathrm{hist}}_i,h\hat{i}{\mathrm{st}}_i)---\left(3\right)$

Weight λ wherein _p+ λ _h=1, make δ ₁Value is in 0-1.

1. calculating location similarity

At first calculate Mahalanobis (Ma Shi) distance according to target state:

$\mathrm{\σ}(p_i,{\hat{p}}_i)=(p_i-{\hat{p}}_i)\×(P+\hat{P})\×{(p_i-{\hat{p}}_i)}^T---\left(4\right)$

Wherein, P,

Be respectively position vector p _i,

Covariance matrix, specifically ask method to illustrate below.The present invention utilizes kalman (Kalman) estimator to ask the Mahalanobis distance delta.

In monitor video, time interval Δ t is very little between consecutive frame usually, and the motion of human body can be similar to thinks uniform motion, and this linear system state equation and measurement equation are:

x(t)＝Ax(t-1)+w(t) (5)

y(t)＝Cx(t)+v(t) (6)

In the formula (5), w is a system noise, the levels of precision of reflection linear system model, and satisfying average is zero Gaussian distribution, its covariance matrix is Q, establishes Q=0.01 * I (I is a unit matrix); V is an observation noise, is that average is that zero white noise sequence and w are uncorrelated mutually, and its covariance matrix is R, can obtain by the variance of asking certain picture element observed reading in the background images in a period of time.Simultaneously, definition status vector x (t)=[p _i, v _i] ^TWith observation vector y (t)=p _i, v wherein _iBe that the position is p _iThe speed of point, then obtain system state equation A and observation equation C according to the uniform motion kinetics equation:

$A=\left[\begin{array}{cc}1& \mathrm{\&Delta;<figure-callout\; id="0"\; label="t"\; filenames="A20091004353700211.png"\; state="\{\{state\}\}">t</figure-callout>}\\ 0& 1\end{array}\right]$ C＝[1 0] (7)

According to the linear system equation of kalman estimator and definition, obtain every in the formula (4) and be calculated as respectively:

p _i＝y(t) (8)

${\hat{p}}_i=\hat{y}\left(t\right|t-1)---\left(9\right)$

P＝R (10)

$\hat{P}=\mathrm{CP}{\left(t\right|t-1)C}^T---\left(11\right)$

Then obtain present frame observation vector p and preceding frame pursuit gain thereof

Mahalanobis distance be:

$\mathrm{\&sigma;}={(y\left(t\right)-\hat{y}\left(t\right|t-1))}^T{(\mathrm{CP}\left(t\right|t-1)C^T+R)}^{-1}(y\left(t\right)-\hat{y}\left(t\right|t-1))---\left(12\right)$

Wherein,

Be the observed quantity predicted value of preceding frame (t-1 frame) pursuit gain in present frame (t frame), P (t|t-1) is the prediction matrix of state vector covariance matrix, obtains by the kalman estimator:

$\hat{y}\left(t\right|t-1)=\mathrm{Cx}\left(t\right|t-1)=C\&times;A\&times;x(t-1)---\left(13\right)$

P(t|t-1)＝AP(t-1|t-1)A ^T+Q(t-1) (14)

Obviously, less than certain threshold value, can think then that on behalf of same feature, two positions promptly match each other as σ.Obeys card side according to σ and distribute, and degree of freedom is 1 in this linear system, looks into chi-square distribution table (establishing the true and false acceptance probability of establishing is 95%) that then threshold epsilon is 3.841.Thus, obtain location similarity according to mahalanobis distance measure σ:

${\mathrm{\&delta;}}_p(p_i,{\hat{p}}_i)=\left\{\begin{array}{c}\frac{\mathrm{\&epsiv;}-\mathrm{\&sigma;}}{\mathrm{\&epsiv;}},(\mathrm{\&sigma;}<\mathrm{\&epsiv;})\\ 0,(\mathrm{\&sigma;}\&GreaterEqual;\mathrm{\&epsiv;})\end{array}\right.---\left(15\right)$

2. calculate the color histogram similarity

Tolerance histogram similarity commonly used be Bhattacharyya (Batachelia) distance, then two color histogram hist _i,

Between similarity be expressed as:

${\mathrm{\&delta;}}_h({\mathrm{hist}}_i,h\hat{i}{\mathrm{st}}_i)=\underset{\mathrm{\&beta;}=1}{\overset{72}{\mathrm{\&Sigma;}}}\sqrt{{\mathrm{hist}}_i\left(\mathrm{\&beta;}\right)h\hat{i}{\mathrm{st}}_i\left(\mathrm{\&beta;}\right)}---\left(16\right)$

Hist in the formula _i,

Represent respectively attributed relational graph G,

Nodal community in color histogram, by formula

(1) calculate, β represents color histogram hist _i,

Figure place.

Formula (15), (16) substitution formula (3) are then obtained the nodal community similarity.

2) calculate side attribute similarity δ ₂(r (θ _i, θ _j),

Ask the similarity of side attribute, as Fig. 2 b) shown in, the characteristic similarity of two straight-line segments promptly asked.Owing to comprise length of straigh line, two attributes of angle in the definition of side attribute, so the side attribute similarity adopts the similarity of length of straigh line, angle to calculate:

${\mathrm{\&delta;}}_2(r({\mathrm{\&theta;}}_i,{\mathrm{\&theta;}}_j),\hat{r}({\widehat{\mathrm{\&theta;}}}_i,{\widehat{\mathrm{\&theta;}}}_j))={\mathrm{\&lambda;}}_l{\mathrm{\&delta;}}_l(r_l,{\hat{r}}_l)+{\mathrm{\&lambda;}}_g{\mathrm{\&delta;}}_g(r_g,{\hat{r}}_g)---\left(17\right)$

Wherein, δ _l, δ _gBe respectively length of straigh line, angle matching degree, weight satisfies λ _l+ λ _g=1.In the formula, r _l,

Be respectively G and

The length attribute value on middle limit, r _gWith

Be respectively G each and

The angle property value on middle limit.

Length of straigh line matching degree δ _lTolerance length of straigh line difference:

${\mathrm{\&delta;}}_l=1-\frac{|r_l-{\hat{r}}_l|}{r_l+{\hat{r}}_l}---\left(18\right)$

Straight-line segment angle coupling δ _gTolerance straight-line segment angle difference:

Formula (18), (19) are updated to formula (17), obtain the side attribute matching degree.

At last, the nodal community similarity that formula (3) is calculated, the side attribute similarity substitution formula (2) that formula (17) is calculated, obtain attribute relational graph appearance model G and

Similarity.

3, the tracking situation analysis of mating based on attributed relational graph

After the attributed relational graph in human detection zone and former frame are followed the tracks of the similarity of attributed relational graph of human body in obtaining present frame, the corresponding coupling of tracking human body of human body that can in view of the above present frame be detected and former frame is to finish the tracking to many human bodies in the monitoring scene.The present invention at first sets up the coupling matrix, lists the similarity that each surveyed area and previous frame are followed the tracks of human body: the row of establishing the coupling matrix number represent present frame human detection zone, and row number are represented previous frame tracking human body, with the matching degree τ of attribute relational graph appearance model _Nm(n represents the sequence number of present frame human body, and m represents the mark of former frame tracking human body) is as the matrix element value record, then according to matching threshold τ ₀Determine whether coupling:

Table 1 t frame coupling matrix

By the match condition of coupling matrix, analyze, determine many human bodies in the present frame tracking mode, obtain human body motion track, and upgrade its display model.Match condition can reduce four kinds, corresponding four kinds of tracking situations:

1) new human body occurs: τ ' during n is capable _NmBe 0: and sustained continuous three frames, thinking that then " new human body " occurs, human body n distributes new mark for human body newly occurring, sets up its attribute relational graph appearance model, begins to follow the tracks of this human body;

2) the normal tracking: n is capable and only has a τ ' _Nm=1 element: then human body n mates with tracking human body m, think " the normal tracking ", be human body n distribute labels m, write down its boundary rectangle frame center position, and with the display model of human body n as following the tracks of the attribute relational graph appearance model that human body m upgrades;

3) block tracking: n is capable a plurality of τ ' _Nm=1 element: then think " blocking tracking ", human body n is made up of a plurality of human bodies that block, and the human body that is blocked is τ ' _NmThe tracking human body of=1 element column correspondence is respectively followed the tracks of the detected value of human body at present frame owing to lose, can't be by coupling matrix identification different human body, and this tracking situation adopts aftermentioned " to block the tracking under the situation " and handles;

4) human body disappears: m column element τ ' _NmBe 0: and this situation occurs at continuous three frames, then thinks " human body disappearance ", promptly follows the tracks of human body m and disappears, this human body of deletion from tracking sequence.

● block the tracking under the situation

Under the situation of blocking, block the foreground area that human body is formed owing to detecting, respectively block position of human body and can't distinguish, the position that the present invention adopts the prediction of Kalman wave filter to block human body, thereby determine respectively to block the attribute relational graph appearance model of human body, finish the tracking under the situation of blocking, concrete steps are as follows:

1. block the human body label according to what the coupling matrix obtained, block the position of human body by the Kalman wave filter according to its position prediction at present frame at previous frame, and according to this predicted position and block the size of human body, determine that this blocks the boundary rectangle frame zone of human body at present frame at previous frame boundary rectangle frame.

The motion process of human body boundary rectangle frame central point is modeled as the uniform motion model, and adopts the prediction of Kalman wave filter to block the position of human body boundary rectangle frame in present frame.State vector is expressed as x (t)=[p _t, v _t] ^T, observation vector is expressed as y (t)=p _t, p wherein _t, v _tRepresent central point x direction coordinate, speed respectively, then obtain state-transition matrix and the observing matrix identical with formula (7), employing formula (13) obtains the measured value of Kalman prediction

As blocking human body at the central point x of present frame direction coordinate figure, central point y direction coordinate also in like manner can get, and the center point coordinate of record prediction is to obtain pursuit path.Then, the boundary rectangle frame zone of blocking human body can obtain according to the predicted position of central point and in the boundary rectangle frame size of previous frame.As Fig. 3 a) shown in, frame of broken lines is represented the estimation range.

2. the prospect outward appearance that will block in the human body boundary rectangle frame is modeled as the attribute relational graph appearance model that blocks human body, is used for mating with the human body of next frame, carries out Continuous Tracking.Adopt and the identical definition of human body attribute relational graph appearance model G, the prospect of each estimation range is modeled as the attribute relational graph appearance model that blocks human body, block human body attribute relational graph appearance model such as Fig. 3 b), its orbicular spot is represented node, dotted line is represented the limit.

The method that the present invention proposes is passed through the method (tracking, the color correlogram tracking of color histogram tracking, employing color and spatial information) that the human appearance model follows the tracks of and is compared with at present main, the human body attribute relational graph appearance model of setting up has not only been described color characteristic, the spatial information of human appearance, also characterized the body structure feature of human body, improved the accuracy of human appearance model greatly, and display model and matching algorithm that the present invention proposes are easy, effective, can realize the real-time follow-up of many human bodies.Figure 4 shows that the experimental result of tracking in many people compound movement monitor video that the present invention proposes.The computer configuration that experiment is adopted is four-processor, internal memory 1GB, the dominant frequency 3.00GHz of running quickly, and in the running environment of Matlab7.0, travelling speed reaches about 10-11 frame/second, the real-time performance of tracking height.The human body of 3 different outward appearances is walked from different directions, is met in a hall and separates in this video, and video sequence resolution is 250 * 370, and frame speed is per second 25 frames, has 355 frames, wherein has 120 frames to occur blocking.Fig. 4 is respectively the 71st frame, 106 frames, 138 frames, 176 frames, 206 frames and 241 frame tracking results, wherein c shown in a)-f)), d) in realized that many people block the tracking under the situation, e), f) show blocking each human body of back and still can correctly follow the tracks of.Fig. 5 has compared the tracking of traditional color histogram modeling human appearance and the tracking error of tracking in this video that the present invention proposes, tracking error is calculated by the Euclidean distance between human body boundary rectangle frame center point coordinate and its actual value in the tracking results, as seen the tracking error of the tracking of the present invention's proposition has improved the accuracy of many human body tracking, validity in the fixed monitoring scene greatly far below traditional color histogram tracking.

Claims (3)

1, a kind of multi-human body tracking method based on attribute relational graph appearance model is characterized in that, may further comprise the steps:

1) obtains the image of the fixed scene that includes tracked human body continuously, attribute relational graph appearance model is set up in the human detection zone of current frame image: the human region that detects is divided into head, upper body and three body parts of lower limb; The corresponding described body part of each node of attribute relational graph appearance model, nodal community characterizes the feature of body part, is shown by following three scales: the mark of corresponding body part, color histogram, boundary rectangle frame center point coordinate; The limit of attribute relational graph appearance model is represented that by the straight-line segment of the central point that connects any two body parts of human body side attribute adopts the length and the angle of straight-line segment to represent;

2) calculate the attribute relational graph appearance model of present frame human body and the similarity that previous frame is followed the tracks of the attribute relational graph appearance model of human body;

3) follow the tracks of the present frame human body according to the similarity of calculating gained.

2, the multi-human body tracking method based on attribute relational graph appearance model according to claim 1 is characterized in that, described nodal community is defined as Δ (θ _i)={ ω _i, p _i, hist _i, wherein, ω _iThe mark of representing the body part of i tracked object: head, upper body, lower limb are labeled as " head ", " up " and " low " respectively; p _iThe center point coordinate of representing the body part boundary rectangle frame of i tracked human body; Hist _iBe the color characteristic of the body part of i tracked human body, hist _iBe expressed as with normalized HSV space one dimension color histogram: hist _i=9H+3S+V; Wherein, the value of H, S, V remarked pixel point three-component H, S, V in the HSV space;

Side attribute is defined as: r (θ _i, θ _j)={ r _l, r _g(i ≠ j), wherein r _lBe length attribute, represent two node θ _iAnd θ _jBetween length of straigh line, adopt internodal Euclidean distance to represent: r _l=| p _i-p _j|, p _i, p _jRepresent node θ respectively _i, θ _jAttribute in center point coordinate; r _gBe the angle attribute, represent this two node θ _iAnd θ _jBetween the straight-line segment angle, $r_g=\arcsin \frac{y_i-y_j}{r_l},$ Y wherein _i, y _jBe node θ _i, θ _jAttribute in central point y direction coordinate, r _gSpan be [0, π];

Described similarity is defined as follows:

$\mathrm{\&tau;}(G,\hat{G})=\underset{i}{\mathrm{\&Sigma;}}{\mathrm{\&lambda;}}_1{\mathrm{\&delta;}}_1(\mathrm{\&Delta;}\left({\mathrm{\&theta;}}_i\right),\mathrm{\&Delta;}\left({\widehat{\mathrm{\&theta;}}}_i\right))+\underset{i}{\mathrm{\&Sigma;}}\underset{j}{\mathrm{\&Sigma;}}{\mathrm{\&lambda;}}_2{\mathrm{\&delta;}}_2(r({\mathrm{\&theta;}}_i,{\mathrm{\&theta;}}_j),\hat{r}({\widehat{\mathrm{\&theta;}}}_i,{\widehat{\mathrm{\&theta;}}}_j)),(i\&NotEqual;j)$

Wherein, λ ₁, λ ₂Be weight coefficient, and constraint condition is: λ ₁+ λ ₂=1, make similarity τ within interval [0,1], δ ₁(Δ (θ _i),

The expression G and

In the internodal similarity of corresponding similar body part, δ ₂(r (θ _i, θ _j),

Be G and

In connect the similarity on the limit of two similar body parts, G and

Be respectively the attribute relational graph appearance model of present frame human body and the attribute relational graph appearance model that previous frame is followed the tracks of human body;

Described step according to similarity tracking present frame human body is: set up the coupling matrix, list each surveyed area and previous frame and follow the tracks of the similarity of human body: the row of establishing the coupling matrix number is represented present frame human detection zone, row number represents previous frame tracking human body, with the matching degree τ of attribute relational graph appearance model _NmAs the matrix element value record, n represents the sequence number of present frame human body, and m represents the mark of former frame tracking human body, then according to matching threshold τ ₀Determine whether coupling, matching result is

By the match condition of coupling matrix, can determine many human bodies in the present frame tracking mode, obtain human body motion track, and upgrade its display model; Match condition reduces four kinds:

1) new human body occurs: τ ' during n is capable _NmBe 0: and sustained continuous three frames, thinking that then " new human body " occurs, human body n distributes new mark for human body newly occurring, sets up its attribute relational graph appearance model, begins to follow the tracks of this human body;

2) the normal tracking: n is capable and has only a τ ' _Nm=1 element: then human body n mates with tracking human body m, think " the normal tracking ", be human body n distribute labels m, write down its boundary rectangle frame center position, and with the attribute relational graph appearance model of the human body n attribute relational graph appearance model as the renewal of following the tracks of human body m;

3) block tracking: n is capable a plurality of τ ' to occur _Nm=1 element: then think " blocking tracking ", human body n is made up of a plurality of human bodies that block, and the human body that is blocked is τ ' _NmThe tracking human body of=1 element column correspondence;

4) human body disappears: m column element τ ' _NmBe 0: and this situation occurs at continuous three frames, then thinks " human body disappearance ", promptly follows the tracks of human body m and disappears, this human body of deletion from tracking sequence.

3, the multi-human body tracking method based on attribute relational graph appearance model according to claim 2, it is characterized in that, under the situation of blocking, the position of adopting the Kalman filter prediction to block human body, thereby determine respectively to block the attribute relational graph appearance model of human body, finish the tracking under the situation of blocking.

Images