CN103391484B - A kind of track optimum combination method for double-deck video frequency abstract - Google Patents

Abstract

For a track optimum combination method for double-deck video frequency abstract, it comprises the steps: (S1) adds goal set to be added for the track of all targets in the target trajectory arrangement set of summary basic paragraph, calculates its goal activities energy function Ea _mwith object time energy function Et _m; (S2) each correlation energy Ec treating select target is calculated _mand target gross energy EO _m, and select the target that wherein gross energy is the highest to hit the mark as the best; (S3) generate the best summary frame coupling hit the mark and separate set; (S4) described coupling solution time ENERGY E Ft is calculated _kand gross energy EF _k, the summary frame coupling solution selecting gross energy minimum is as optimum Match solution; (S5) described the best being hit the mark to move to from target array to be added adds in target array; (S6) repeat S2 to S5 until target array to be added is empty set, and obtain each target at the order merged in frame of making a summary.

Description

A kind of track optimum combination method for double-deck video frequency abstract

Technical field

The present invention relates to a kind of track optimum combination method for double-deck video frequency abstract.

Background technology

Based on the double-deck video frequency abstract of contents of object towards monitor video, be base unit with " object ", by to the structure of video and the analysis of content, extract from former video significant " object ", and they are generated video that is compact, that fully can show video semanteme content by object trajectory optimum combination concentrate, by the summarized radio of the video simmer down to dozens of minutes of tens hours.

M section target trajectory reconfigures in N frame video frequency abstract frame by the optimum combination problem of track exactly, the trajectory optimization combined method of present main flow is the trajectory optimization combinatorial problem relying on genetic algorithm to solve with the method for all solutions of traversal, but there is two problems in this two various methodologies: first, the ambiguity in definition of optimal solution, collide less between target trajectory after only considered combination, do not consider the dependency relation of target trajectory and video frequency abstract interframe, can exist and Density inhomogeneity problem be there will be for partial video summary result; The second, utilize the optimal solution method for solving of genetic algorithm to cause and be absorbed in locally optimal solution; 3rd, algorithm complex is too high, and it is relevant that method and the genetic operator of genetic algorithm travel through number of times, travels through all solution methods and have secondary computing could reach optimal solution substantially.

Summary of the invention

One is the object of the present invention is to provide to avoid locally optimal solution, and simply a kind of track optimum combination method for double-deck video frequency abstract of computational methods, it is characterized in that: it comprises the steps:

(S1) for summary basic paragraph V _jtarget trajectory arrangement set in the track TR of all targets _madd target array to be added, calculate the goal activities energy function Ea of described target _mwith object time energy function Et _m;

(S2) each correlation energy Ec treating select target is calculated _mand target gross energy Eo _m, and select the target that wherein gross energy is the highest to hit the mark as the best;

(S3) generate the best summary frame coupling hit the mark and separate set;

(S4) described coupling solution time ENERGY E Ft is calculated _kand gross energy EF _k, the summary frame coupling solution selecting gross energy minimum is as optimum Match solution;

(S5) described the best being hit the mark to move to from target array to be added adds in target array;

(S6) repeat S2 to S5 until target array to be added is empty set, and obtain each target at the order merged in frame of making a summary.

On the basis of technique scheme, described step S1 comprises:

(1a) initialization target to be added array is target trajectory all in summary basic paragraph, has added target array for empty;

(1b) the movable energy function Ea of each target is calculated _m

$E_{a_m}=\underset{i=F_s}{\overset{F_e}{\mathrm{\Σ}}}{w}_{i*}{h}_i$

Wherein, F _swith F _estart frame and the abort frame of m target, w _iwith h _ithis target i-th bounding rectangles;

(1c) the time energy function Et of each target is calculated _m

${\mathrm{Et}}_m=\frac{\mathrm{NV}\left(j\right)}{2*(F_s+F_e)}$

F _swith F _ebe start frame and the abort frame of m target, NV (j) makes a summary basic paragraph V _jtotalframes.

On the basis of technique scheme, described step S2 comprises:

(2a) calculate and eachly treat select target and to have added correlation energy in original video of target in target array and EC _m:

${\mathrm{Ec}}_m=\underset{k\≠m}{\mathrm{\Σ}}\mathrm{tc}(m,k)$

Tc (m, k) is the number of times that the track of target m and target k appears in same frame in original video;

(2b) each ENERGY E O treating select target is calculated _m

EO _m=Ea _m+Et _m+Ec _m

(2c) EO in target to be selected is selected _mmaximum target hits the mark as the best.

On the basis of technique scheme, described step S3 comprises: set every frame to make a summary the maximum target numbers that can hold of frame as NF _max, the maximum filling area of every frame is AF _max, it is AI that target intersects maximum area _max;

If (3a) first the best hits the mark, initialization digests match solution is the first frame, and frame length is the total frame length of this target;

(3b) if not first the best hits the mark, find first to meet and held target number and be less than and filling area is less than and follow-up N frame all meets the frame k initial to be selected of above-mentioned condition, wherein N is the totalframes hit the mark;

(3c) calculate from frame k to be selected in the corresponding frame of N continuous frame, every frame and the crossing ENERGY E I of all added targets of this frame _k;

${\mathrm{EI}}_k=\underset{i=1}{\overset{N}{\mathrm{\Σ}}}\underset{j}{\mathrm{\Σ}}{B}_j^i\∩{\mathrm{BO}}^i$

Wherein, the bounding rectangles of a jth added target of the i-th frame started from frame k to be selected, BO ⁱi-th bounding rectangles that the best hits the mark;

If (3d) this frame EI to be selected _kbe less than AI _maxthen this frame k to be selected becomes a coupling and to separate and the summary frame coupling joining this target is separated in set, otherwise finds next and satisfied held target number and be less than NF _maxand filling area is less than AF _max, and follow-up N frame all meets the frame k to be selected of above-mentioned condition;

(3e) step (3c) is repeated to (3d), until the crossing ENERGY E I of current frame to be selected _kbe 0.

On the basis of technique scheme, described step S4 comprises:

(4a) each summary frame coupling solution time ENERGY E Ft is calculated _k

${\mathrm{EFt}}_k=\frac{k}{F_s}$

Wherein k is this coupling solution is kth frame, F _sit is the initial frame number that current the best hits the mark;

(4b) calculate summary frame coupling and separate gross energy EF _k, wherein EF _k=EI _k+ EFt _k

(4c) minimum the separating as this best summary frame coupling hit the mark of frame coupling solution gross energy of wherein making a summary is selected.

Relative to prior art, present invention reduces optimal solution and ask for operation times in process, the problem avoiding locally optimal solution it also avoid track Density inhomogeneity situation.

Accompanying drawing explanation

Fig. 1 is a kind of track optimum combination method flow diagram for double-deck video frequency abstract of the present invention.

Embodiment

Below in conjunction with accompanying drawing, invention is described further.

Track is had combinatorial problem most by the present invention, be decomposed into asking for that the best hits the mark and ask for problem with each optimum solution hit the mark, finally obtain Trace Formation order, please refer to Fig. 1, shown specific implementation method of the present invention describes in further detail by reference to the accompanying drawings, and it comprises the following steps:

(1) the best hits the mark selection

A) target energy calculates

For summary basic paragraph V _jtarget trajectory arrangement set in the track TR of m target _mcalculate its target energy Ea _mwith Et _m

(1a1) the movable energy function Ea of each target is calculated _m

${\mathrm{Ea}}_m=\underset{i=F_s}{\overset{F_e}{\mathrm{\Σ}}}{w}_{i*}{h}_i$

Wherein, F _swith F _estart frame and the abort frame of m target, w _iwith h _ithis target i-th bounding rectangles.

(1a2) the time energy function Et of each target is calculated _m

${\mathrm{Et}}_m=\frac{\mathrm{NV}\left(j\right)}{2*(F_s+F_e)}$

F _swith F _ebe start frame and the abort frame of m target, NV (j) makes a summary basic paragraph V _jtotalframes.

B) the best hits the mark selection

(1b1) initialization is being just V at target array TN to be added _jin all target trajectory, added target array TA for empty.

(1b2) calculate and eachly treat correlation energy in original video of target in select target and TA and EC _m:

${\mathrm{Ec}}_m=\underset{k\≠m}{\mathrm{\Σ}}\mathrm{tc}(m,k)$

Tc (m, k) is the number of times that the track of target m and target k appears in same frame in original video.

(1b3) each ENERGY E O treating select target is calculated _m

EO _m=Ea _m+Et _m+Ec _m

(1b4) EO in target to be selected is selected _mmaximum target hits the mark as the best.

(2) target optimum Match solution generates

A) frame of making a summary generates coupling and separates

Maximum target numbers that every frame summary frame can hold are NF _max, the maximum filling area of every frame is AF _max, it is AI that target intersects maximum area _max.

(2a1) initialization summary separates frame number is that first the best hits the mark total frame length.

(2a2) first find first to meet and held target number and be less than NF _maxand filling area is less than AF _max, and frame k, N initial to be selected that follow-up N frame all meets above-mentioned condition are the totalframes hit the mark.

(2a3) this initial frame k to be selected is calculated in the corresponding frame of N continuous frame, every frame and the crossing area EI of all added targets of this frame _kif, EI _kbe less than AI _maxthen this frame k to be selected becomes a coupling solution, otherwise finds next frame solution to be selected.

${\mathrm{EI}}_k=\underset{i=1}{\overset{N}{\mathrm{\Σ}}}\underset{j}{\mathrm{\Σ}}{B}_j^i\∩{\mathrm{BO}}^i$

Wherein, the bounding rectangles of a kth added target of the kth frame started from frame k to be selected, BO ⁱi-th bounding rectangles that the best hits the mark.

B) energy balane of coupling solution

(2b1) summary frame coupling solution time ENERGY E Ft is calculated _k

${\mathrm{EFt}}_k=\frac{k}{F_s}$

Wherein k is this coupling solution is kth frame, F _sit is the initial frame number that current the best hits the mark.

(2b1) summary frame coupling solution time ENERGY E F is calculated _k

EF _k=EI _k+EFt _k

C) optimum Match solution generates

(2c1) continue to calculate each coupling solution of satisfying condition and corresponding energy thereof, until EI _kwhen being 0, mean that this coupling solution does not intersect rectangle, or the frame number k that coupling is separated reaches last frame in summary solution, then increase N frame blank summary frame at summary frame end, meet EI _kit is the condition of 0.

(2c2) in all coupling solutions, EF is found _kthe coupling solution that a minimum frame hits the mark as the best.

D) summary merges order generation

Obtaining after a best hits the mark and mate solution with it, this target is moved to from target array TN to be added adding add in target array TA, proceed the best to hit the mark selection, until TN is empty, finally obtain the order that each target merges in summary frame.

The present invention can be converted into by the track combination optimization problem of target the problem that event trace carries out translation combination on a timeline, namely the spatio-temporal redundancies information in original video is made full use of, the activity in summarized radio is made to seem very compact, allow different time sections event synchronously occur, thus reach the effect of compressed video time.

Track constantly translation combination is asked how to be filled into more fully in frame of video by target trajectory, is sought optimal solution exactly to the problem that individual target trajectory is filled in frame summary frame.

Claims (5)

1., for a track optimum combination method for double-deck video frequency abstract, it is characterized in that: it comprises the steps:

(S1) for summary basic paragraph V _jtarget trajectory arrangement set in the track TR of all targets _madd target array to be added, calculate the goal activities energy function Ea of described target _mwith object time energy function Et _m;

(S2) each correlation energy Ec treating select target is calculated _mand target gross energy Eo _m, and select the target that wherein gross energy is the highest to hit the mark as the best;

(S3) generate the best summary frame coupling hit the mark and separate set;

(S4) described coupling solution time ENERGY E Ft is calculated _kand gross energy EF _k, the summary frame coupling solution selecting gross energy minimum is as optimum Match solution;

(S5) described the best being hit the mark to move to from target array to be added adds in target array;

(S6) repeat S2 to S5 until target array to be added is empty set, and obtain each target at the order merged in frame of making a summary.

2. a kind of track optimum combination method for double-deck video frequency abstract as claimed in claim 1, it is characterized in that, described step S1 comprises:

(1a) initialization target to be added array is target trajectory all in summary basic paragraph, has added target array for empty;

(1b) the movable energy function Ea of each target is calculated _m

${\mathrm{Ea}}_m=\underset{i=F_s}{\overset{F_e}{\Σ}}{w}_{i*}{h}_i$

Wherein, F _swith F _estart frame and the abort frame of m target, w _iwith h _ithis target i-th bounding rectangles;

(1c) the time energy function Et of each target is calculated _m

${\mathrm{Et}}_m=\frac{NV\left(j\right)}{2*(F_s+F_e)}$

F _swith F _ebe start frame and the abort frame of m target, NV (j) is summary basic paragraph V _jtotalframes.

3. a kind of track optimum combination method for double-deck video frequency abstract as claimed in claim 1, it is characterized in that, described step S2 comprises:

(2a) calculate and eachly treat select target and to have added correlation energy in original video of target in target array and Ec _m:

${\mathrm{Ec}}_m=\underset{k\≠m}{\Σ}tc(m,k)$

Tc (m, k) is the number of times that the track of target m and target k appears in same frame in original video;

(2b) each target gross energy EO treating select target is calculated _m

EO _m＝Ea _m+Et _m+Ec _m

(2c) EO in target to be selected is selected _mmaximum target hits the mark as the best.

4. a kind of track optimum combination method for double-deck video frequency abstract as claimed in claim 1, it is characterized in that, described step S3 comprises: set every frame to make a summary maximum target numbers that frame can hold are as NF _max, the maximum filling area of every frame is AF _max, it is AI that target intersects maximum area _max;

If (3a) first the best hits the mark, initialization digests match solution is the first frame, and frame length is the total frame length of this target;

(3b) if not first the best hits the mark, find first to meet and held target number and be less than NF _maxand filling area is less than AF _max, and follow-up N frame all meets the frame k initial to be selected of above-mentioned condition, wherein N is the totalframes hit the mark;

(3c) calculate from frame k to be selected in the corresponding frame of N continuous frame, every frame and the crossing ENERGY E I of all added targets of this frame _k;

${\mathrm{EI}}_k=\underset{i=1}{\overset{N}{\Σ}}\underset{j}{\Σ}{B}_j^i\∩{\mathrm{BO}}^i$

Wherein, the bounding rectangles of a jth added target of the i-th frame started from frame k to be selected, i-th bounding rectangles that the best hits the mark;

If (3d) this frame EI to be selected _kbe less than AI _mAXthen this frame k to be selected becomes a coupling and to separate and the summary frame coupling joining this target is separated in set, otherwise finds next and satisfied held target number and be less than NF _maxand filling area is less than AF _max, and follow-up N frame all meets the frame k to be selected of above-mentioned condition;

(3e) step (3c) is repeated to (3d), until the crossing ENERGY E I of current frame to be selected _kbe 0.

5. a kind of track optimum combination method for double-deck video frequency abstract as claimed in claim 1, it is characterized in that, described step S4 comprises:

(4a) each summary frame coupling solution time ENERGY E Ft is calculated _k

${\mathrm{EFt}}_k=\frac{k}{F_s}$

Wherein k is this coupling solution is kth frame, F _sit is the initial frame number that current the best hits the mark;

(4b) calculate summary frame coupling and separate gross energy EF _k, wherein EF _k=EI _k+ EFt _k, its EE _kfor target intersects energy;

(4c) minimum the separating as this best summary frame coupling hit the mark of frame coupling solution gross energy of wherein making a summary is selected.