CN104469547A - Video abstraction generation method based on arborescence moving target trajectory - Google Patents

Abstract

The invention discloses a video abstraction generation method based on an arborescence moving target trajectory. The video abstraction generation method comprises the following steps of (1) detecting and tracking a moving target by means of the Gaussian background modeling method, and obtaining a box of the moving trajectory of the moving target; (2) conducting clustering on the box of the trajectory of the moving target on the basis of whether adhesion of the box of the trajectory of the moving target exists or not, and constructing target trees; (3) describing the target trees; (4) ranking the target trees according to the descending order from large to small; (5) constructing an empty container in an initialized mode, wherein the empty container generates video abstraction; (6) ranking the first target tree in the container; (7) determining the initial point position of the follow-up target trees ranked into the container; (8) ranking the target trees into the container; (9) judging whether all the target trees are ranked or not; (10) inputting the video abstraction generated by the container into which all the target trees ranked. According to the video abstraction generation method, the continuity of the moving target in the video abstraction can be kept, and the video abstraction concentration efficiency is high.

Description

A kind of video abstraction generating method based on tree-shaped movement objective orbit

Technical field

The invention belongs to image identification technical field, relate to a kind of video abstraction generating method based on tree-shaped movement objective orbit.

Background technology

Video summarization technique effectively concentrates by the redundant eliminating in massive video data and by remaining effective information for people's fast browsing.By viewing video frequency abstract, people just not only can need not browse original massive video data but also can understand the general contents of massive video.Because video frequency abstract is used for carrying out fast browsing to institute's event in monitor video, also need the concern event to finding when browsing to position playback simultaneously, so, the way that video frequency abstract is general is all, after the moving target in original video is carried out detecting and tracking, each moving target is generated video frequency abstract as an event chain.There is fatal problem in this way, the quality of video frequency abstract effect, depends in original video, the detection of moving target and the quality of tracking effect.But, the factors such as whether complicated the accuracy of the detecting and tracking algorithm of moving target, be limited to video monitoring environment dramatically, and whether moving target is intensive.For this reason, also limit the application of video frequency abstract.

Summary of the invention

The object of this invention is to provide a kind of video abstraction generating method based on tree-shaped movement objective orbit, no longer require that each moving target can both carry out complete detection and tracking, but allow severally to stick together, intersect, multiple target such as to block, be described with the form of goal tree, solve prior art to follow the tracks of because of moving object detection and be limited to moving target dense degree, and monitoring environment complexity and the video frequency abstract that causes generates limited problem.

The technical solution adopted in the present invention is, a kind of video abstraction generating method based on tree-shaped movement objective orbit, implements according to following steps:

Step 1, from monitor video, extract moving target, obtain the box of moving target movement locus

Employing mixed Gaussian background modeling method extracts the moving target in monitor video, afterwards, be same order calibration method to the maximum according to overlapping area between consecutive frame and carry out motion target tracking, in each frame, follow the tracks of each motion target area obtained to represent with its minimum boundary rectangle

The minimum enclosed rectangle of each moving target, in a two field picture, is a boundary rectangle, claims this boundary rectangle to confine, and the region in this two field picture is an agglomerate; These agglomerates stack up on a timeline, just define a box, and the starting point of box is the frame finding certain moving target, and terminal is the former frame that moving target disappears in the monitoring ken,

Suppose, appear at the moving target in the video monitoring ken to each, all adopt a box to describe, then the description of each moving target is such as formula shown in (1):

$O_k({x_L}^{m_k},{y_L}^{m_k};{x_R}^{m_k},{y_R}^{m_k};{x_0}^{m_k},{y_0}^{m_k},N_k),---\left(1\right)$

Wherein, O _k() represents a kth moving target, k=1,2 ..., N _s, m _k=1,2 ..., N _k, N _sfor the sum of moving target detected, N _krepresent the frame number of a kth target Continuous;

represent that a kth target is at m _kthe coordinate in the boundary rectangle upper left corner in frame,

represent that a kth target is at m _kthe coordinate in the boundary rectangle lower right corner in frame,

represent that a kth target is at m _kcenter-of-mass coordinate in frame,

In this step, similar situation be described with the form of tree in moving target recognition process, be designated same mark, namely tree gives a continuous print movement locus, no longer requires to only include a moving target;

Step 2, moving target box cluster

To detected moving target k=1,2 ..., N _s, m _k=1,2 ..., N _kjudge in motion process, whether to there is adhesion between its target, the target box that there is adhesion is all classified as same class, namely be referred to as to belong to same goal tree, and in each frame, the region that each moving target is confined by its minimum enclosed rectangle is called as an agglomerate of tree, if when there is multiple moving target in a frame, then there is multiple agglomerate in this frame;

Step 3, goal tree to be described;

Step 4, N number of goal tree that step 3 cluster obtains to be sorted

This N number of goal tree being arranged according to the descending that length is descending, in order to represent convenient, still setting the goal tree after this N number of sequence as Tree _id, id=1,2 ..., N;

Step 5, design summarization generation container to its initialization;

Step 6, enter first aim tree in container;

Step 7, determine that goal tree enters the position of container;

Step 8, will determine that the row's for the treatment of goal tree of position enters in container;

Step 9, judge whether also to have the goal tree needing to enter in container

If no, namely id=N+1 then sorts complete exiting, and goes to step 10,

Otherwise, get next goal tree Tree _id, enter step 7;

Step 10, the element value of container to be rounded, then the video frequency abstract for generating.

The invention has the beneficial effects as follows, movement objective orbit data in video extract by the method in advance, then these tracks are re-started to the planning on time shaft, at the total length of movement objective orbit impact severity as far as possible compressed video summary in acceptable situation, finally these moving targets are regenerated one section of summarized radio according to the new route planned, specifically:

First, do not lose moving target information, to greatest extent all possible useful information of upper reservation.Secondly, by the planning again to movement objective orbit, can significantly be removed by the redundant segments without moving target on time shaft, make video frequency abstract short as much as possible, summarized radio browses nature, smoothness, just for people's fast browsing provides maximum facility.Finally, the summary of generation is still normal video playout speed and mode, and visual effect also remains the visual effect of original video completely.

Accompanying drawing explanation

Fig. 1 is that a moving target boundary rectangle is superposed the box schematic diagram obtained by the inventive method on a timeline;

Fig. 2 is the movement objective orbit relation schematic diagram of the inventive method video frequency abstract;

Fig. 3 is the Trace Formation relation schematic diagram of the inventive method video frequency abstract;

Fig. 4 is the moving target adherence Separation track schematic diagram in the inventive method;

Fig. 5 is the moving target center of mass motion relation schematic diagram in the inventive method;

Fig. 6 is the moving target model schematic of the tree in the inventive method;

Fig. 7 arranges the collision relation schematic diagram into goal tree in the container in the inventive method;

Fig. 8 is the agglomerate collision relation schematic diagram in the inventive method.

Embodiment

Below in conjunction with the drawings and specific embodiments, the present invention is described in detail.

Video abstraction generating method based on tree-shaped movement objective orbit of the present invention belongs to dynamic video summary, namely by extracting effective information and they are fused into one section of video skimming from original monitor video, the moving target monitored in the monitoring ken in this video skimming, is contained.

Video abstraction generating method based on tree-shaped movement objective orbit of the present invention, implement according to following steps:

Step 1, from monitor video, extract moving target, obtain the box of moving target movement locus

Employing mixed Gaussian background modeling method extracts the moving target in monitor video, afterwards, be same order calibration method to the maximum according to overlapping area between consecutive frame and carry out motion target tracking, in each frame, follow the tracks of each motion target area obtained and represent with its minimum boundary rectangle.

(note: the tracking of this moving target detecting method and moving target in relevant professional book, or all has introduction in correlative theses, no longer repeats herein.)

As shown in Figure 1, the minimum enclosed rectangle of each moving target, in a two field picture, is a boundary rectangle, claims this boundary rectangle to confine, and the region in this two field picture is an agglomerate; These external agglomerates stack up on a timeline, just define a box, and the starting point of box is the frame finding certain moving target, and terminal is the former frame that moving target disappears in the monitoring ken,

Suppose, appear at the moving target in the video monitoring ken to each, all adopt a box to describe, then the description of each moving target is such as formula shown in (1):

$O_k({x_L}^{m_k},{y_L}^{m_k};{x_R}^{m_k},{y_R}^{m_k};{x_0}^{m_k},{y_0}^{m_k},N_k),---\left(1\right)$

Wherein, O _k() represents a kth moving target, k=1,2 ..., N _s, m _k=1,2 ..., N _k, N _sfor the sum of moving target detected, N _krepresent the frame number of a kth target Continuous;

represent that a kth target is at m _kthe coordinate in the boundary rectangle upper left corner in frame,

represent that a kth target is at m _kthe coordinate in the boundary rectangle lower right corner in frame,

represent that a kth target is at m _kcenter-of-mass coordinate in frame, (note: the computational methods of barycenter are mentioned in relevant teaching material and paper, no longer repeat here.)

As shown in Figures 2 and 3, be the track of moving target at original video, and after the Trace Formation of step below, the target trajectory schematic diagram of the video frequency abstract obtained, this schematic diagram, when not hindering understanding, omits the information in y-axis direction.In fig. 2, suppose there are A, B, C, D tetra-targets, these four targets are separate, there is not adhesion each other, block, the situation such as intersection, therefore, after eliminating the redundancy between these four tracks, then obtain the video frequency abstract of Fig. 3 just than being easier to.

In fact, in the monitor video of reality, moving target often can exist block, the situation such as adhesion, these situations can bring impact to the extraction of moving target.Such as shown in Fig. 4, have moving target 1, moving target 2 do not have adhesion on the left of visual field, in motion target tracking process, they obtain correct mark.When these two target travels are to stage casing, visual field, together with target 1 adheres to target 2, at this moment they are identified as moving target 3; When these two target travels are to visual field when right section, two targets are separated again, and at this moment moving target is identified as target 4, target 5, to be reflected on center of mass motion just as shown in Figure 5.Like this, be roughly in two moving targets simultaneously occurred in original video on a timeline and be but identified as five moving targets.Very possible they can be planned for five different time periods in follow-up movement objective orbit again planning process, this has just cut off their continuity itself, the visual effect of the last summary of impact greatly.

In this step, in moving target recognition process, similar situation is described with the form of tree, be designated same mark, provide control result as shown in Figure 6, in other words, tree gives a continuous print movement locus, no longer requires to only include a moving target;

Step 2, moving target box cluster

To detected moving target k=1,2 ..., N _s, m _k=1,2 ..., N _kjudge in motion process, whether to there is adhesion between its target, as shown in Figure 4 and Figure 5, the target box that there is adhesion is all classified as same class, namely be referred to as to belong to same goal tree, and in each frame, the region that each moving target is confined by its minimum enclosed rectangle is called as an agglomerate of tree, if when there is multiple moving target in a frame, then there is multiple agglomerate in this frame;

Step 3, goal tree to be described

First, objective definition tree uniquely identified parameter set Tree _id, this model is such as formula shown in (2):

${\mathrm{Tree}}_{\mathrm{id}}-(t_{\mathrm{start}}^{\mathrm{id}},t_{\mathrm{end}}^{\mathrm{id}},\{{\mathrm{Block}}_t^{\mathrm{id}}\},t=\mathrm{1,2},...,{\mathrm{\Δt}}_{\mathrm{id}}),---\left(2\right)$

Wherein, id=1,2 ..., N, id are the numberings of goal tree,

N is total number of the goal tree that step 2 obtains;

the start frame sequence number of moving target in original video in this goal tree,

its end frame sequence number in original video,

just represent the length of this goal tree;

for agglomerate collection, $\left\{{\mathrm{Block}}_t^{\mathrm{id}}\right\}=\{{\mathrm{Block}}_t^{i_b},i_b=\mathrm{1,2},...,{n_t}^{\mathrm{id}}\},$ be the number of the agglomerate of goal tree in t frame, the number of the moving target namely detected by step 1 is that the region that all moving target boxes boundary rectangle in respective frame being classified as same goal tree obtained by step 2 is confined is formed,

$\left\{{\mathrm{Block}}_t^{\mathrm{id}}\right\}=\{{\mathrm{Block}}_t^{i_b},i_b=\mathrm{1,2},...,{n_t}^{\mathrm{id}}\}$ In each agglomerate ${\mathrm{Block}}_t^{i_b},i_b=\mathrm{1,2},...,n_t^{\mathrm{id}}$ Information is described as:

${\mathrm{Block}}_t^{i_b}=(s_t^{i_b},{\mathrm{Rect}}_t^{i_b},r_t^{i_b},\{{\mathrm{pixel}}_t^{i_b}\left\}\right),---\left(3\right)$

Wherein, t=1,2 ..., Δ t _id,

i-th of t frame in goal tree _bthe frame number of individual agglomerate in original video,

i-th _bindividual agglomerate is the upper left corner of boundary rectangle and the coordinate in the lower right corner in former video, represents the area coordinate of agglomerate in original video frame,

i-th _b1/2 of the length of long sides of the maximum boundary rectangle of individual agglomerate,

be i-th _bthe pixel value in individual agglomerate region;

Step 4, N number of goal tree that step 3 cluster obtains to be sorted

Considering that the length of goal tree is the principal element affecting final digest total length, therefore, first this N number of goal tree is arranged according to the descending that length is descending, in order to represent convenient, still setting the goal tree after this N number of sequence as Tree _id, id=1,2 ..., N;

Step 5, design summarization generation container to its initialization

So-called summarization generation container, the target complete tree being used to step 3 to generate generates the three-dimensional array of one section of video frequency abstract after merging, wherein bidimensional represents the size of video frequency abstract two field picture, identical with original video frame picture size, the one-dimensional representation time in addition, like this, the data mode that expression video is time dependent sequence of image frames is just formed

This three-dimensional array of initialization, is referred to as structure empty, sees formula (4):

$C={\left[C_{i,j,l}\right]}_{m\×n\×\mathrm{\Δ}{t}_C},---\left(4\right)$

Wherein, c _{i, j, l}=0, i=1,2 ..., m, j=1,2 ..., n, l=1,2 ..., Δ t _c, the size of array is m × n × Δ t _c, m is the line number of frame of video picture, and n is the columns of frame of video picture, Δ t _cfor the length of container,

During initialization, make Δ t _c=Δ t _max, Δ t _max=max{ Δ t _id| id=1,2 ..., N}, namely the length of empty elects the length of maximum target tree as, after step 4 sorts, Δ t _max=Δ t ₁,

Make id=1, enter in container by first aim tree, the position entered is t _start=1;

Step 6, enter first aim tree in container

The longest goal tree Tree of first will be come ₁put into container, be not that the element of 0 should be all agglomerates of this goal tree in container, then have formula (5):

$c_{x,y,l}={\mathrm{pixel}}_l^{i_b},---\left(5\right)$

Wherein, $x_L^{i_b}\≤x\≤x_R^{i_b},y_L^{i_b}\≤y\≤y_R^{i_b},i_b=\mathrm{1,2},...,n_t^{\mathrm{id}},l=\mathrm{1,2},...,{\mathrm{\Δt}}_C,$

Afterwards, make id=id+1, select next goal tree and go to step the position that 7 determine to enter container;

Step 7, determine that goal tree enters the position of container

7.1) the every frame agglomerate having arranged goal tree is asked in the row's for the treatment of goal tree and container to collide

When arranging into new goal tree, and the collision detection process of having arranged between goal tree is in a reservoir, if the part below a region in Fig. 7 is the complete goal tree that sorted in container, the goal tree above arrow is the row's for the treatment of goal tree; As shown in b region in Fig. 7, illustrate on the right side of container, the row's for the treatment of goal tree is up moved from the bottommost of container, namely from container scope t _start∈ [1, Δ t _c] in, by t _start=1 starts, and calculate collision, in Fig. 7, c region is that the goal tree meeting acceptance level collision status enters,

The agglomerate collision process arranged between goal tree in every frame in the row's for the treatment of goal tree and container is, as shown in Figure 8, mark " 1, 2, 3 " digital frame is set to the goal tree that entered in the container agglomerate a certain frame (being set to t frame), mark " A, B, C, D " alphabetical frame be the agglomerate of the row's for the treatment of goal tree at this frame, being the position of agglomerate in two field picture with the collision between the agglomerate arranging goal tree when entering has some or all of identical, namely assert and create collision, in fig. 8, agglomerate D does not collide, agglomerate B, C there occurs slight impact, agglomerate A then there occurs serious collision with digital frame 2, it is also slight that the situation of slight impact affects Visual Observations Observations effect because it is overlapping not serious, therefore the collision of permission is considered,

If the row's for the treatment of goal tree is at t _athe agglomerate collection of frame is arrange goal tree in a reservoir at t _athe agglomerate collection of frame is

Suppose the agglomerate concentrated ${\mathrm{Block}}_{t_a}^{i_a}=(s_{t_a}^{i_a},{\mathrm{Rect}}_{t_a}^{i_a},r_{t_a}^{i_a},\{{\mathrm{pixel}}_{t_a}^{i_a}\left\}\right),$ Its center-of-mass coordinate is $(x_0^{i_a},y_0^{i_a}),i_a-\mathrm{1,2},...,n_{t_a}^{\mathrm{id}};$ the agglomerate concentrated ${\mathrm{Block}}_{t_a}^{j_a}=(s_{t_a}^{j_a},{\mathrm{Rect}}_{t_a}^{j_a},r_{t_a}^{j_a},\{{\mathrm{pixel}}_{t_a}^{j_a}\left\}\right),$ Its center-of-mass coordinate is $(x_0^{j_a},y_0^{j_a}),j_a=\mathrm{1,2},...,n_{t_a}^C;$

The criterion of then whether colliding between two between agglomerate is such as formula (6):

${\mathrm{Col}}_{t_a}(j_a,j_a)=\left\{\begin{array}{cc}1& \mathrm{if}(\sqrt{{(x_0^{i_a}-x_0^{j_a})}^2+{(y_0^{i_a}-y_0^{j_a})}^2}\≤|r_t^{i_a}-r_t^{j_a}\left|\right)\\ 0& \mathrm{if}(\sqrt{{(x_0^{i_a}-x_0^{j_a})}^2+{(y_0^{i_a}-y_0^{j_a})}^2}>{|r}_t^{i_a}-r_t^{j_a}|)\end{array}\right.,---\left(6\right)$

Wherein, then at t _ain frame, the collision Colli (t arranging goal tree in the row's for the treatment of goal tree and container _a) whether criterion is such as formula (7):

$\mathrm{Colli}\left(t_a\right)=\left\{\begin{array}{ccc}1& \mathrm{if}& \underset{k_s=1}{\overset{n_{t_a}^{\mathrm{id}}}{\mathrm{\Σ}}}\underset{i_s=1}{\overset{n_{t_a}^C}{\mathrm{\Σ}}}{\mathrm{Col}}_{t_a}(i_a,j_a)\≥0\\ 0& \mathrm{if}& \underset{k_s=1}{\overset{n_{t_a}^{\mathrm{id}}}{\mathrm{\Σ}}}\underset{i_s=1}{\overset{n_{t_a}^C}{\mathrm{\Σ}}}{\mathrm{Col}}_{t_a}(i_a,j_a)=0\end{array}\right.;---\left(7\right)$

7.2) collision frame by frame of having arranged goal tree is asked in the row's for the treatment of goal tree and container

Because the length of the row's for the treatment of goal tree is Δ t _id, therefore, according to step 7.1) and calculate the collision of every frame, the position of arranging goal tree of container is taken as t _start, t _start+ 1 ..., t _start-1+ Δ t _id, with all frames of the row's for the treatment of goal tree, i.e. t _a=1,2 ..., Δ t _id, frame by frame according to step 7.1) and calculate the agglomerate collision Colli (t of every frame _a), t _a=1,2 ..., Δ t _id;

7.3) the overall collision rate of having arranged goal tree is asked in the row's for the treatment of goal tree and container

The row's for the treatment of goal tree starting point is at t _startthe overall collision rate of goal tree has been arranged on position with container calculating formula such as formula (8):

${\mathrm{\ρ}}_C^{t_{\mathrm{start}}}=\frac{\underset{t_a=1}{\overset{\mathrm{\Δ}{t}_{\mathrm{id}}}{\mathrm{\Σ}}}\mathrm{Colli}\left(t_a\right)}{\mathrm{\Δ}{t}_{\mathrm{id}}};---\left(8\right)$

7.4) position that the row's for the treatment of goal tree can be placed is judged

According to the collision rate that formula (8) calculates span be user needs the dense degree according to accepting, and the requirement to video frequency abstract length, arranges collision rate threshold value ρ _th, preferred empirical value is ρ _th=1/3,

If represent that the goal tree of row in the row's for the treatment of goal tree and container does not exist collision, if show that collision situation belongs to acceptable degree, at this moment, the row's for the treatment of goal tree is arranged into the t in container _startposition on, go to step 8 realizations and this goal tree entered in container;

If show that every frame has collision, at this moment greatly can affect visual effect, if show that collision situation belongs to unacceptable degree, need the calculating position of collision changing the row's for the treatment of goal tree, even t for this reason _start=t _start+ 1 (namely showing that a frame is moved in the position of the row's for the treatment of goal tree backward), go to step 7.1) again carry out the calculating of collision rate, until find satisfied position t _start;

Step 8, will determine that the row's for the treatment of goal tree of position enters in container

Will by this goal tree Tree _idaccording to the original position t that step 7 is determined _startput into container, first adjust the length of container, if t _start+ Δ t _id> Δ t _c, then Δ t is had _c=(Δ t _id+ t _start), otherwise container length remains unchanged, and the calculating of the element value in container is such as formula (9):

$c_{x_a,y_a,t_a}=\left\{\begin{array}{ccc}{\mathrm{pixel}}_{t_a}^{i_a}& \mathrm{if}& c_{x_a,y_a,t_a}=0\\ (c_{x_a,y_a,t_a}+{\mathrm{pixel}}_{t_a}^{i_a})/2& \mathrm{if}& c_{x_a,y_a,t_a}-{\mathrm{pixel}}_{t_a}^{i_a}\≠0\end{array}\right.,---\left(9\right)$

Wherein, $x_L^{i_a}\≤x_a\≤x_R^{i_a},y_L^{i_a}\≤y_a\≤y_R^{i_a},i_a=\mathrm{1,2},...,n_{t_a}^{\mathrm{id}},t_a=\mathrm{1,2},...,\mathrm{\Δ}{t}_{\mathrm{id}},$

represent the average of the nonzero value in the agglomerate pixel value and container asking the row's for the treatment of goal tree on relevant position,

After this goal tree drained, make id=id+1, namely represent that the sequence number of goal tree adds the implication of 1;

Step 9, judge whether also to have the goal tree needing to enter in container

If no, namely id=N+1 then sorts complete exiting, and goes to step 10,

Otherwise, get next goal tree Tree _id, enter step 7;

Step 10, the element value of container being rounded, then the video frequency abstract for generating, exporting.

Claims (6)

1. based on a video abstraction generating method for tree-shaped movement objective orbit, its feature is: implement according to following steps:

Step 1, from monitor video, extract moving target, obtain the box of moving target movement locus

Employing mixed Gaussian background modeling method extracts the moving target in monitor video, afterwards, be same order calibration method to the maximum according to overlapping area between consecutive frame and carry out motion target tracking, in each frame, follow the tracks of each motion target area obtained to represent with its minimum boundary rectangle

The minimum enclosed rectangle of each moving target, in a two field picture, is a boundary rectangle, claims this boundary rectangle to confine, and the region in this two field picture is an agglomerate; These external agglomerates stack up on a timeline, just define a box, and the starting point of box is the frame finding certain moving target, and terminal is the former frame that moving target disappears in the monitoring ken,

Suppose, appear at the moving target in the video monitoring ken to each, all adopt a box to describe, then the description of each moving target is such as formula shown in (1):

$O_k({x_L}^{m_k},{y_L}^{m_k};{x_R}^{m_k},{y_R}^{m_k};{x_0}^{m_k},{y_0}^{m_k},N_k)---\left(1\right)$

Wherein, O _k() represents a kth moving target, k=1,2 ..., N _s, m _k=1,2 ..., N _k, N _sfor the sum of moving target detected, N _krepresent the frame number of a kth target Continuous;

represent that a kth target is at m _kthe coordinate in the boundary rectangle upper left corner in frame,

represent that a kth target is at m _kthe coordinate in the boundary rectangle lower right corner in frame,

represent that a kth target is at m _kcenter-of-mass coordinate in frame,

In this step, similar situation be described with the form of tree in moving target recognition process, be designated same mark, namely tree gives a continuous print movement locus, no longer requires to only include a moving target;

Step 2, moving target box cluster

To detected moving target k=1,2 ..., N _s, m _k=1,2 ..., N _kjudge in motion process, whether to there is adhesion between its target, the target box that there is adhesion is all classified as same class, namely be referred to as to belong to same goal tree, and in each frame, the region that each moving target is confined by its minimum enclosed rectangle is called as an agglomerate of tree, if when there is multiple moving target in a frame, then there is multiple agglomerate in this frame;

Step 3, goal tree to be described;

Step 4, N number of goal tree that step 3 cluster obtains to be sorted

This N number of goal tree being arranged according to the descending that length is descending, in order to represent convenient, still setting the goal tree after this N number of sequence as Tree _id, id=1,2 ..., N;

Step 5, design summarization generation container to its initialization;

Step 6, enter first aim tree in container;

Step 7, determine that goal tree enters the position of container;

Step 8, will determine that the row's for the treatment of goal tree of position enters in container;

Step 9, judge whether also to have the goal tree needing to enter in container

If no, namely id=N+1 then sorts complete exiting, and goes to step 10,

Otherwise, get next goal tree Tree _id, enter step 7;

Step 10, the element value of container to be rounded, then the video frequency abstract for generating.

2. the video abstraction generating method based on tree-shaped movement objective orbit according to claim 1, its feature is: in described step 3, objective definition tree uniquely identified parameter set Tree _id, this model is such as formula shown in (2):

${\mathrm{Tree}}_{\mathrm{id}}=(t_{\mathrm{start}}^{\mathrm{id}},t_{\mathrm{end}}^{\mathrm{id}},\{{\mathrm{Block}}_t^{\mathrm{id}}\},t=\mathrm{1,2},...,{\mathrm{\Δt}}_{\mathrm{id}}),---\left(2\right)$

Wherein, id=1,2 ..., N, id are the numberings of goal tree,

N is total number of the goal tree that step 2 obtains;

the start frame sequence number of moving target in original video in this goal tree,

its end frame sequence number in original video,

just represent the length of this goal tree;

$\left\{{\mathrm{Block}}_t^{\mathrm{id}}\right\}$ For agglomerate collection, $\left\{{\mathrm{Block}}_t^{\mathrm{id}}\right\}=\{{\mathrm{Block}}_t^{i_b},i_b=\mathrm{1,2},...,{n_t}^{\mathrm{id}}\},n_t^{\mathrm{id}}$ Be the number of the agglomerate of goal tree in t frame, the number of the moving target namely detected by step 1 is that the region that all moving target boxes boundary rectangle in respective frame being classified as same goal tree obtained by step 2 is confined is formed,

$\left\{{\mathrm{Block}}_t^{\mathrm{id}}\right\}=\{{\mathrm{Block}}_t^{i_b},i_b=\mathrm{1,2},...,{n_t}^{\mathrm{id}}\}$ In each agglomerate ${\mathrm{Block}}_t^{i_b},i_b=\mathrm{1,2},...,n_t^{\mathrm{id}}$ Information is described as:

${\mathrm{Block}}_t^{i_b}=(s_t^{i_b},{\mathrm{Rect}}_t^{i_b},r_t^{i_b},\{{\mathrm{pixel}}_t^{i_b}\left\}\right),---\left(3\right)$

Wherein, t=1,2 ..., Δ t _id,

i-th of t frame in goal tree _bthe frame number of individual agglomerate in original video,

i-th _bindividual agglomerate is the upper left corner of boundary rectangle and the coordinate in the lower right corner in former video, represents the area coordinate of agglomerate in original video frame,

i-th _b1/2 of the length of long sides of the maximum boundary rectangle of individual agglomerate,

be i-th _bthe pixel value in individual agglomerate region.

3. the video abstraction generating method based on tree-shaped movement objective orbit according to claim 2, its feature is: in described step 5, so-called summarization generation container, the target complete tree being used to step 3 to generate generates the three-dimensional array of one section of video frequency abstract after merging, wherein bidimensional represents the size of video frequency abstract two field picture, identical with original video frame picture size, the one-dimensional representation time in addition, like this, the data mode that expression video is time dependent sequence of image frames is just formed

This three-dimensional array of initialization, is referred to as structure empty, sees formula (4):

$C={\left[c_{i,j,l}\right]}_{m\×n\×{\mathrm{\Δt}}_C}---\left(4\right)$

Wherein, c _{i, j, l}=0, i=1,2 ..., m, j=1,2 ..., n, l=1,2 ..., Δ t _c, the size of array is m × n × Δ t _c, m is the line number of frame of video picture, and n is the columns of frame of video picture, Δ t _cfor the length of container,

During initialization, make Δ t _c=Δ t _max, Δ t _max=max{ Δ t _id| id=1,2 ..., N}, namely the length of empty elects the length of maximum target tree as, after step 4 sorts, Δ t _max=Δ t ₁,

Make id=1, enter in container by first aim tree, the position entered is t _start=1.

4. the video abstraction generating method based on tree-shaped movement objective orbit according to claim 3, its feature is: in described step 6, come the longest goal tree Tree of first ₁put into container, be not that the element of 0 should be all agglomerates of this goal tree in container, then have formula (5):

$c_{x,y,l}={\mathrm{pixel}}_l^{i_b},---\left(5\right)$

Wherein, $x_L^{i_b}\≤x\≤x_R^{i_b},y_L^{i_b}\≤y\≤y_R^{i_b},i_b=\mathrm{1,2},...,n_t^{\mathrm{id}},$ l＝1,2,...,Δt _C，

Afterwards, make id=id+1, select next goal tree and go to step the position that 7 determine to enter container.

5. the video abstraction generating method based on tree-shaped movement objective orbit according to claim 4, its feature is: in described step 7, specifically comprise the following steps:

7.1) the every frame agglomerate having arranged goal tree is asked in the row's for the treatment of goal tree and container to collide

When arranging into new goal tree, and the collision detection process of having arranged between goal tree in a reservoir is up moved from the bottommost of container by the row's for the treatment of goal tree, namely from container scope t _start∈ [1, Δ t _c] in, by t _start=1 starts, and calculates collision, is the goal tree meeting acceptance level collision status and enters,

The agglomerate collision process arranged between goal tree in every frame in the row's for the treatment of goal tree and container is, being the position of agglomerate in two field picture with the collision between the agglomerate arranging goal tree when entering has some or all of identical, namely assert and create collision, it is also slight that the situation of slight impact affects Visual Observations Observations effect because it is overlapping not serious, therefore the collision of permission is considered

If the row's for the treatment of goal tree is at t _athe agglomerate collection of frame is arrange goal tree in a reservoir at t _athe agglomerate collection of frame is

Suppose $\left\{{\mathrm{Block}}_{t_a}^{\mathrm{id}}\right\}$ The agglomerate concentrated ${\mathrm{Block}}_{t_a}^{i_a}=(s_{t_a}^{i_a},{\mathrm{Rect}}_{t_a}^{i_a},r_{t_a}^{i_a},\{{\mathrm{pixel}}_{t_a}^{i_a}\left\}\right),$ Its center-of-mass coordinate is $(x_0^{i_a},y_0^{i_a}),i_a=\mathrm{1,2},...,n_{t_a}^{\mathrm{id}};\left\{{\mathrm{Block}}_{t_a}^C\right\}$ The agglomerate concentrated ${\mathrm{Block}}_{t_a}^{j_a}=(s_{t_a}^{j_a},{\mathrm{Rect}}_{t_a}^{j_a},r_{t_a}^{j_a},\{{\mathrm{pixel}}_{t_a}^{j_a}\left\}\right),$ Its center-of-mass coordinate is $(x_0^{j_a},y_0^{j_a}),j_a=\mathrm{1,2},...,n_{t_a}^C,$

The criterion of then whether colliding between two between agglomerate is such as formula (6):

${\mathrm{Col}}_{t_a}(i_a,j_a)=\left\{\begin{array}{cc}1& \mathrm{if}(\sqrt{{(x_0^{i_a}-x_0^{j_a})}^2+{(y_0^{i_a}-y_0^{j_a})}^2}\≤|{r_t}^{i_a}-{r_t}^{j_a}\left|\right)\\ 0& \mathrm{if}(\sqrt{{(x_0^{i_a}-x_0^{j_a})}^2+{(y_0^{i_a}-y_0^{j_a})}^2}>|{r_t}^{i_a}-{r_t}^{j_a}\left|\right)\end{array}\right.,---\left(6\right)$

Wherein, then at t _ain frame, the collision Colli (t arranging goal tree in the row's for the treatment of goal tree and container _a) whether criterion is such as formula (7):

$\mathrm{Colli}\left(t_a\right)=\left\{\begin{array}{ccc}1& \mathrm{if}& \underset{k_s=1}{\overset{n_{t_a}^{\mathrm{id}}}{\mathrm{\Σ}}}\underset{i_s=1}{\overset{n_{t_a}^C}{\mathrm{\Σ}}}{\mathrm{Col}}_{t_a}(i_a,j_a)\≥0\\ 0& \mathrm{if}& \underset{k_s=1}{\overset{n_{t_a}^{\mathrm{id}}}{\mathrm{\Σ}}}\underset{i_s=1}{\overset{n_{t_a}^C}{\mathrm{\Σ}}}{\mathrm{Col}}_{t_a}(i_a,j_a)=0\end{array}\right.;---\left(7\right)$

7.2) collision frame by frame of having arranged goal tree is asked in the row's for the treatment of goal tree and container

Because the length of the row's for the treatment of goal tree is Δ t _id, therefore, according to step 7.1) and calculate the collision of every frame, the position of arranging goal tree of container is taken as t _start, t _start+ 1 ..., t _start-1+ Δ t _id, with all frames of the row's for the treatment of goal tree, i.e. t _a=1,2 ..., Δ t _id, frame by frame according to step 7.1) and calculate the agglomerate collision Colli (t of every frame _a), t _a=1,2 ..., Δ t _id;

7.3) the overall collision rate of having arranged goal tree is asked in the row's for the treatment of goal tree and container

The row's for the treatment of goal tree starting point is at t _startthe overall collision rate of goal tree has been arranged on position with container calculating formula such as formula (8):

${\mathrm{\ρ}}_C^{t_{\mathrm{start}}}=\frac{\underset{t_a=1}{\overset{{\mathrm{\Δt}}_{\mathrm{id}}}{\mathrm{\Σ}}}\mathrm{Colli}\left(t_a\right)}{{\mathrm{\Δt}}_{\mathrm{id}}};---\left(8\right)$

7.4) position that the row's for the treatment of goal tree can be placed is judged

According to the collision rate that formula (8) calculates span be

User needs the dense degree according to accepting, and the requirement to video frequency abstract length, arranges collision rate threshold value ρ _th,

If represent that the goal tree of row in the row's for the treatment of goal tree and container does not exist collision, if show that collision situation belongs to acceptable degree, at this moment, the row's for the treatment of goal tree is arranged into the t in container _startposition on, go to step 8 realizations and this goal tree entered in container;

If show that every frame has collision, at this moment greatly can affect visual effect, if show that collision situation belongs to unacceptable degree, need the calculating position of collision changing the row's for the treatment of goal tree, even t for this reason _start=t _start+ 1, namely show that a frame is moved in the position of the row's for the treatment of goal tree backward, go to step 7.1) again carry out the calculating of collision rate, until find satisfied position t _start.

6. the video abstraction generating method based on tree-shaped movement objective orbit according to claim 5, its feature is: in described step 8, by this goal tree Tree _idaccording to the original position t that step 7 is determined _startput into container, first adjust the length of container, if t _start+ Δ t _id> Δ t _c, then have otherwise container length remains unchanged, the calculating of the element value in container is such as formula (9):

$c_{x_a,y_a,t_a}=\left\{\begin{array}{ccc}{\mathrm{pixel}}_{t_a}^{i_a}& \mathrm{if}& c_{x_a,y_a,t_a}=0\\ (c_{x_a,y_a,t_a}+{\mathrm{pixel}}_{t_a}^{i_a})/2& \mathrm{if}& c_{x_a,y_a,t_a-{\mathrm{pixel}}_{t_a}^{i_a}\≠0}\end{array}\right.,---\left(9\right)$

Wherein, $x_L^{i_a}\≤x_a\≤x_R^{i_a},y_L^{i_a}\≤y_a\≤y_R^{i_a},i_a=\mathrm{1,2},...,n_{t_a}^{\mathrm{id}},$ t _a＝1,2,...,Δt _id，

represent the average of the nonzero value in the agglomerate pixel value and container asking the row's for the treatment of goal tree on relevant position,

After this goal tree drained, make id=id+1, namely represent that the sequence number of goal tree adds the implication of 1.