CN104683803A - Moving object detecting and tracking method applied to compressed domain - Google Patents

Abstract

The invention discloses a moving object detecting and tracking method applied to a compressed domain. The method is characterized in that filtering and segmentation are carried out on the basis of a moving vector extracted from a compressed code stream, so that a moving object can be tracked. Firstly a moving object convex shell segmentation scheme based on the boundary searching is disclosed according to the moving vector information in the compressed code stream, the eight-directional boundary searching is carried out on the filtered moving vector to obtain a moving area, the rule clustering is carried out on the scattered moving area to obtain multi-target segmentation, and the moving object tracking method based on the kalman filtering is provided according to a segmentation result. The complicated video analysis is directly substituted by the analysis on the code stream information when in decoding, so that a great amount of decoding time is saved, the time for analyzing the monitor video can be greatly reduced, the operation complexity can be effectively alleviated by virtue of the rule clustering method and the boundary searching method provided in the moving object segmentation method, and the real-time tracking feasibility can be improved.

Description

A kind of Moving Objects detection and tracking method being applied to compression domain

[technical field]

The present invention relates to coding and decoding video, moving object detection, monitor video analysis field, particularly based on the single goal moving object tracking method in H.264 compression domain field.

[background technology]

In recent years along with the broad development of digital video monitoring, the dynamics of monitoring is also raised greatly, due to the raising of digital video resolution, also more and more higher to the requirement of Computer Automatic Monitor, in the tracking of pixel domain, also slowly embodied defect.A large amount of monitor videos can be stored to rear end, greatly improves the requirement stored rear end, has also slowly drawn the compression to rear end video, reduced the resource bottleneck that large data cause.And what obtain in video after compression is the information of non-pixel domain, carry out the detection to target, need to carry out complete decompress(ion) to video, then the further research and application to video is being carried out, and decompression procedure causes the great wasting of resources and time waste, thus propose the scheme of in the compressed domain object video directly being carried out to video analysis.

The compression of monitor video the most frequently used at present adopts compressed format H.264/AVC, can reduce the memory capacity of original video in this form on the basis not reducing video quality as much as possible.Often there are motion vector and DCT residual error coefficient two main informations in general compressed bit stream, and what H.264/AVC adopt infra-frame prediction in compressed format is that spatial prediction obtains, its DCT residual error coefficient can not the colouring information of Correct original video, and remaining only motion vector can as Information Pull.

The object video detection and tracking scheme of compression domain generally comprises the filtering of motion vector, segmentation, follow the tracks of three part compositions, because the motion vector information in the compressed bit stream that compressed encoding obtains adopts the most similar color block in appointed area to mate in the next frame according to present frame, thus the alternate position spike obtained is motion vector, so motion vector not obtains is the movable information of the moving target of standard, so there is a series of such as illumination, pneumatic, the shake of camera and the noise motion vector caused, so need to carry out filtering operation to motion vector, obtain reliable and stable motion vector field.The detection of Moving Objects often needs the scanning carrying out overall pixel to calculate in pixel domain, classifies, thus obtain Moving Objects by calculating the moving targets of form to entirety such as its energy function.But along with the raising of resolution, the global search of pixel domain is limited to the real-time of video tracking, iterating calculates its energy function and carries out the method for cluster, makes iterations have growth as index, meet its requirement of real-time, very strict requirement is needed to hardware.And object tracking stage, need to carry out sample learning to this object, the feature that extraction can need is mated, and this type of scheme needs to consume a large amount of sample learning time, and often to the tracking of video, the target of its study is often limited in fixing shape, in the study of rigid motion, because position of its local study can't change, its sample is relatively stable, reasonable tracking effect can be obtained, and in motion for software target, the subregion of this target can be moved in opposed area, thus will change to the regional learning of absolute position, thus well can not extract the feature in this region, thus there is a large amount of study mistakes, cause following the tracks of unsuccessfully.The present invention is intended to for above-mentioned technical barrier provides a kind of brand-new solution.

[summary of the invention]

First extract motion vector information according to from H.264 compressed format code stream, utilize this information disclosure a kind of based on the Moving Objects fast detecting of compression domain and the scheme of tracking.Propose a kind of detection mode of brand-new moving target, in conjunction with the characteristic of this testing result and Kalman filtering, propose one tracking scheme quickly and easily.High complicated video object detects and transfers to decoding end by the present invention, by the method that decoding limit, limit is analyzed, set up the skeleton pattern of Moving Objects fast, thus obtain the positional information of moving target, then the positional information of Kalman prediction next frame is carried out according to the positional information of present frame, differentiated by error detection and detect whether mistake, thus improve filtering parameter, obtain the tracing positional of moving target.

In order to realize the present invention, the present invention goes to realize it by four major parts, obtains fast detecting tracking scheme of the present invention through the filtering of motion vector, adaptive boundary search, connected domain cluster and Kalman filtering.

1) based on the motion vector filtering of time-space domain:

According to claim 2, the motion vector of extraction has noise, needs to carry out preliminary treatment, first by reducing amount of calculation, adopting the absolute value of x and y both direction component of MV and the amplitude as MV, first carrying out single threshold filtering, see change type:

From time domain angle, motion MV has the consistency of direction and amplitude, respectively x and the y component of the MV of frame every in N continuous frame is carried out iteration respectively, obtain normalized accumulation MV, and carry out equalization according to its characteristic:

$N\left({\mathrm{MV}}^{t\→t_{\mathrm{ref}}}\right)=\frac{{\mathrm{MV}}^{t\→t_{\mathrm{ref}}}}{t-t_{\mathrm{ref}}}$

Then the metastable MV field obtained, it has locally coherence feature on spatial domain and time domain, its time-space domain method is exactly that the MV of the MV of current block and neighborhood block is carried out irrelevance calculating from amplitude and two, direction dimension respectively, can regard the MV that this MV is Moving Objects when being less than certain threshold value as:

${\mathrm{MVNCI}}_{\mathrm{amp}}^{i,j}=\left|\frac{{\mathrm{MV}}_{\mathrm{amp}}^{i,j}-\frac{1}{N}\underset{(i^{\′},j^{\′})\∈\mathrm{\θ}}{\mathrm{\Σ}}{\mathrm{MV}}_{\mathrm{amp}}^{i^{\′},j^{\′}}}{\frac{1}{N}\underset{(i^{\′},j^{\′})\∈\mathrm{\θ}}{\mathrm{\Σ}}{\mathrm{MV}}_{\mathrm{amp}}^{i^{\′},j^{\′}}}\right|$

${\mathrm{MVNCI}}_{\mathrm{dir}}^{i,j}=\frac{1}{N}\underset{(i^{\′},j^{\′})\∈\mathrm{\θ}}{\mathrm{\Σ}}|{\mathrm{MV}}_{\mathrm{dir}}^{i,j}-{\mathrm{MV}}_{\mathrm{dir}}^{i^{\′},j^{\′}}|$

The irrelevance obtained limits by the method for setting threshold, and especially for MV deflection, 0 and 360 belong to same direction, first MV being normalized to (0, π], can solution party to unification problem.

2) based on the adaptive boundary of 8 direction templates:

According to claim 3, the reliable and stable MV field obtained, obtains connected domain block by carrying out boundary search to moving region, thus can avoid the situation that the moving target regional area brought due to filtering lacks.First context of methods arranges a 3*3 and offsets template, be used for representing eight neighborhood search directions of current search block, wherein 0 represents current search block, 1 ~ 8 eight directions representing search respectively, each direction block stores the deviation post of this direction relative to current search block, reposition namely:

(x′ _cur，y′ _cur)＝(x _cur，y _cur)+(x _offset，y _offset)

Scanning definition first initiating searches block by zig-zag, guaranteeing that its block searched for is boundary block by adopting clockwise scanning direction order.

For making up the disappearance of motion vector and improving its splitting speed, propose a kind of partitioning algorithm of convex hull, the boundary block of above-mentioned boundary search is carried out asking for convex hull, area filling is carried out to the block in convex hull, thus form several connected region blocks.First by determining a boundary point starting point, then utilizing and sorting with the angle of its starting point, filtering segment boundary point, finally by the right-hand rule, direction is unified to boundary point and search for thus obtain final salient point.The convex hull be made up of convex hull point is just connected domain region, and this region is not also Moving Objects target, belongs to a part for Moving Objects target.

3) rule-based connected domain cluster:

According to claim 4, obtain each motion communication territory, then need to carry out cluster for each connected domain, by moving region as iteration unit, thus can greatly reduce its number of iterations here, improve the real-time of video tracking.Because the connected domain number of its iteration is few, the clustering from convergence can not be adopted, so propose based on motion vector magnitude, direction and connected domain distance two key elements as clustering rule.Again the moving region belonging to each class is solved convex hull to it again after cluster completes, thus obtain larger convex hull, this convex hull just can as Moving Objects.

The problem of motion target area can be amplified carelessly for the mask produced due to convex hull segmentation, a kind of method optimizing mask is proposed, by the mask information of front and back totally 5 frames, mask is redefined to present frame, can make, because accidentalia corrects convex hull enlarged area in time when certain frame convex hull amplifies, to make it more objective and meet real motion object.

4) based on the moving object tracking of Kalman filtering:

According to claim 5, after obtaining multiple Moving Objects convex hull, the convex hull maximum boundary of each Moving Objects is represented Moving Objects as rectangle frame border, manual demarcation is carried out according to the interested target of user, then calculate and calculate its registration with all Moving Objects rectangle frames, obtain maximum just as the Moving Objects needing to follow the tracks of, using the present frame position of the center of this Moving Objects as this target.

Kalman filtering is by the positional information of former frame, thus predict the positional information of next frame, and the positional information that can obtain with detection is as the parameter optimization of Kalman's equation, thus constantly upgrading Kalman filtering parameter, the next frame predicted value that filtering is obtained can closing to reality target location more.

[accompanying drawing explanation]

In conjunction with reference accompanying drawing and ensuing detailed description, the present invention will be easier to understand, the structure member that wherein same Reference numeral is corresponding same, wherein:

Fig. 1 is that the Moving Objects fast detecting of a kind of compression domain in the present invention and tracking realize block diagram;

Fig. 2 is to iteration accumulation schematic diagram after motion vector, and wherein (a) represents that the MV of Moving Objects MV field after backward iteration accumulation synthesizes field, and (b) represents the synthesis field of noise MV field after accumulation;

Fig. 3 is 8 direction neighborhood search templates;

Fig. 4 is 8 scanning direction way of search schematic diagrames, and wherein (a) is the front and back block scan order of 8 scanning directions, and (b) is the boundary block after 8 scanning directions;

Fig. 5 is convex hull filling process schematic diagram.

[embodiment]

For enabling above-mentioned purpose of the present invention, feature and advantage become apparent more, and below in conjunction with the drawings and specific embodiments, the present invention is further detailed explanation.

The object of the present invention is to provide a kind of Moving Objects in Video Sequences detecting and tracking framework of compression domain.Fig. 1 shows the system framework of the Moving Objects detecting and tracking method in the present invention, please refer to Fig. 1, described method is: step 001, decoding end is by extracting motion vector field, then be transported to filtering end and carry out filtering, first through backward iteration accumulation and time-space domain filtering, more reliable and more stable motion vector field is obtained.As shown in Figure 2, after backward iteration accumulation, the sports ground with movement tendency can amplify and correction of movement trend, and noise motion vector is disorderly and unsystematic due to it, thus reduces its movement tendency.

Step 002, carries out mask fabrication according to filtered motion vector, if wherein moving target is then 1, non-athletic target is 0.Fig. 2 illustrates the skew template in its 8 direction, and wherein 0 represents current search block, and 1 ~ 8 eight directions representing search respectively, each direction block stores the deviation post of this direction relative to current search block.Represent its way of search according to Fig. 3, first define initial search direction 2, zigzag and scan all pieces of traversal, run into the first moving mass, carry out clockwise neighborhood search; After searching next block, initial search direction is subtracted 2 in the current end direction of search, continue to search for clockwise; Do not find moving mass after 8 direction search, searching position is positioned to previous search block, and forwards the 2nd step continuation to; It is overlapping that lower direction search block or last search block and first search for block, and mark current region is a connected domain, and continues the boundary search of next connected domain.Then according to Fig. 4, ask salient point to boundary block, thus form convex hull, and fill convex hull inside, its moving region is thought in unification.

Step 003, according to the motion communication territory obtained, cluster is carried out to each connected domain, distance between two connected domains adopts all pieces in blocks all in each connected domain and another connected domain to ask distance, and adopt Hausdorff distance to define two object distance, make this distance more objectively can meet the distance of irregularly shaped object.The standard of cluster has two, amplitude and the direction of average motion vector respectively, connected domain distance two key elements in addition, in decision criteria, according to two rule settings distance threshold TDIS and motion vector threshold value TNCI, wherein TNCI can upgrade again according to after every merged block, is determined by interval threshold tNCI.μ ₁and μ ₂in different videos, choose different values, in the sequence that Moving Objects is slow and direction is single, the MV of connected region has larger judgement effect, μ on the irrelevance of direction ₂value need bigger than normal, generally get 0.8, and Moving Objects motion amplitude is comparatively large and direction is changeable when, motion vector is comparatively mixed and disorderly, and MV has larger judgement effect on amplitude irrelevance, μ ₁value need bigger than normal, generally get 0.6.

Step 004, according to the Moving Objects obtained, to the maximum rectangle frame region of Moving Objects as target, using the position of regional center as target.Here the target predicted position defining the i-th frame is tBB (i), the position detecting the destination object obtained is defined as dBB (i), define irrelevance e=tBB (i)-dBB (i), if irrelevance e is greater than the mean motion range difference of twice, so we just assert that detecting the position obtained makes a mistake, to predict tBB (i+1) by dBB (i-1), thus the real-time reaching moving target is followed the tracks of.

Above-mentioned explanation fully discloses the specific embodiment of the present invention.It is pointed out that the scope be familiar with person skilled in art and any change that the specific embodiment of the present invention is done all do not departed to claims of the present invention.Correspondingly, the scope of claim of the present invention is also not limited only to described embodiment.

Claims (5)

1., based on fast detecting and the tracking framework of compression domain Moving Objects, it is characterized in that, described method comprises:

Time-space domain filtering is carried out based on the motion vector information in H.264 code stream;

8 direction-adaptive boundary searches are carried out based on motion vector filtering result;

Carry out rule confidence obtain multiple target Moving Objects based on searching for the moving region that obtains;

Based on the interesting target tracking scheme of Kalman filtering, error detection.

2. the time-space domain filtering method based on motion vector information according to claim 1, according to the motion vector of the Unit 4 × 4 obtained from decoded bit stream, carries out carrying out irrelevance calculating from two aspects of spatial domain and time domain to motion vector.

Adopt the absolute value of x and y two-dimensional directional of MV and the amplitude as MV, utilize single threshold filtering, then the MV block of MV to 8 neighborhoods filtered is carried out to the calculating of irrelevance, utilize threshold value to obtain the filtered MV block in time-space domain.Its computing formula is:

$R(\stackrel{\→}{a},\stackrel{\→}{b})=\left\{\begin{array}{cc}0& \mathrm{if}\stackrel{\→}{a}=\stackrel{\→}{b}=\left(\mathrm{0,0}\right)\\ \frac{\left|\right|\stackrel{\→}{a}-\stackrel{\→}{b}\left|\right|}{\left|\right|\stackrel{\→}{a}\left|\right|+\left|\right|\stackrel{\→}{b}\left|\right|}& \mathrm{otherwise}\end{array}\right.$

3. according to claim 1ly carry out 8 direction-adaptive boundary searches based on motion vector filtering result, first Zig-zag scanning is carried out to present frame, finding boundary block, then according to 8 direction templates, guaranteeing that its block searched for is boundary block by adopting clockwise scanning direction order.So far form closed loop and form the border of moving region, then boundary block is solved formation convex hull, obtain complete connected domain and to fall apart block.

4. according to claim 1ly carry out rule confidence obtain multiple target Moving Objects method based on searching for the connected domain that obtains, first its average motion vector is calculated to each moving region obtained, then formulate two judgment criteria: one is the Hausdoff distance between different motion region, two is amplitude and direction irrelevances of different motion zone leveling motion vector.Its judgment criteria is as follows:

$p_{\mathrm{NCI}}={\mathrm{\μ}}_1{p}_{\mathrm{amp}}^{\left(i\right)\left(j\right)}+{\mathrm{\μ}}_2{p}_{\mathrm{dir}}^{\left(i\right)\left(j\right)}={\mathrm{\μ}}_1\frac{|{\mathrm{MV}}_{\mathrm{amp}}^{\left(i\right)}-{\mathrm{MV}}_{\mathrm{amp}}^{\left(j\right)}|}{{\mathrm{MV}}_{\mathrm{amp}}^{\left(j\right)}}+{\mathrm{\μ}}_2\frac{|{\mathrm{MV}}_{\mathrm{dir}}^{\left(i\right)}-{\mathrm{MV}}_{\mathrm{dir}}^{\left(j\right)}|}{{\mathrm{MV}}_{\mathrm{dir}}^{\left(j\right)}}$

Wherein with represent i-th piece of amplitude relative to jth block and direction irrelevance respectively, with represent i-th piece with the position coordinates of jth block.According to two rule settings distance threshold TDIS and motion vector threshold value TNCI, wherein TNCI can upgrade again according to after every merged block, is determined by interval threshold tNCI.

Then the connected domain boundary block that cluster obtains again solves and obtains convex hull, thus obtain the convex target of complete Moving Objects, then the problem of motion target area can be amplified carelessly for the mask produced due to convex hull segmentation, a kind of method optimizing mask is proposed, by the mask information of front and back totally 5 frames, mask is redefined to present frame, can make, because accidentalia corrects convex hull enlarged area in time when certain frame convex hull amplifies, to make it more objective and meet real motion object.

5. according to claim 1, based on the interesting target tracking scheme of Kalman filtering, error detection:

First according to the multiple target Moving Objects obtained, according to the convex hull size setting initialization tracking box of each destination object, carry out interesting target demarcating the initial target tracking box that obtains and split the target frame obtained and carry out ratio value calculating, determine the target needing to follow the tracks of.

Then the target's center position obtained is set to BB (x ₀, y ₀, len _x, len _y) for following the tracks of numerical value, utilize Kalman filtering to set up motion prediction model and optimized reconstruction model, predicted state can be set up in the position according to former frame:

${\hat{X}}_k=A{\hat{X}}_{k-1}+{\mathrm{BU}}_{k-1}$

$P_k^{-}=A{P}_{k-1}{A}_k^T+Q$

The predicted value obtained can in conjunction with present frame detect the value obtained be reentered into iterative in again correct Prediction Parameters:

${\hat{X}}_k={\hat{X}}_k^{-}+K_k(Z_k-H{\hat{X}}_k^{-})$

$K_k=P_k^{-}{H}^T{({\mathrm{HP}}_k^{-}{H}^T+R)}^{-1}$

$P_k=(I-K_{k}H)P_k^{-}$

Then by predicting that the position that the position obtained carries out obtaining with detector verifies, judge whether detector detects mistake, thus the positional information that output tracking device obtains.