CN101127122A - Content self-adaptive gradual-progression type sheltering analysis target tracking algorism - Google Patents

Abstract

The utility model relates to a gradual and self adaptive tracking algorithm for overlapped targets analysis, belonging to the technical field of computer vision and pattern analysis. In the target tracking process, the tracking performance often gets worsened; and even the target is lost due to the failure of detection. Most of the prior algorithms are only dependent on the gray value of the picture elements in the current interest regions of the picture, to determine whether occlusion exists, so the result is not ideal. Context information of the space time is used in the utility model to gradually analyze the occlusion conditions of the region of interest, and the utility model is combined with a reference target and a motion vector, to further improve the reliability of the analysis results. Therefore, compared to the prior algorithm, the algorithm of the utility model has the advantages of better distinguishing the occluder from target, and the effectiveness of the algorithm is proved by a plurality of experiment results of the live video streaming.

Description

A kind of content self-adaptive gradual-progression type sheltering analysis target tracking algorism

Technical field

The invention belongs to computer vision and pattern analysis technical field, be specifically related to a kind of content self-adaptive gradual-progression type sheltering analysis target tracking algorism.

Background technology

Target following has a wide range of applications in man-machine interaction, supervision automatically, video frequency searching, Traffic monitoring and automobile navigation.The task of target following is to determine how much states of target in each frame of video flowing, comprises position, size and orientation etc.Owing to do not limit the outward appearance of tracked target, and the outward appearance of target can change in tracing process, adds the interference of complicated prospect and background, and target tracking algorism is faced with lot of challenges, is one of research focus of computer vision field.

Target tracking algorism can be divided three classes, and a class is a tracking (point tracking) ^[1,2], second class is that nuclear is followed the tracks of (kerneltracking) ^[3～7,10], the 3rd class is that silhouette is followed the tracks of (silhouette tracking) ^[8,9]The target tracking algorism that the present invention proposes belongs to the nuclear track algorithm.This algorithm characterizes target with display model (that is template), and the geological information of target in each frame described with affine transformation parameter usually ^[10]

For the nuclear track algorithm, one of maximum challenge is exactly processing target this problem that is blocked how ^[3,5-7]Why this problem is difficult to solve is because target and shelter can be any outward appearances, and the time of blocking also can be arbitrarily.If track algorithm fails to detect shelter, then can cause declining to a great extent of tracking accuracy ^[5], even owing to the template thing that is blocked destroys and to cause track rejection ^[3]

Have in the existing document and much attempt by surveying the target tracking algorism that shelter solves occlusion issue.The target appearance that document [3] comes modeling to observe with the mixing of three distributions.The component that wherein is used for describing shelter is assumed that to satisfy evenly and distributes.Another kind method judges by the size that the observed reading that detects pixel departs from estimated value whether this pixel is blocked ^[5-7]When the statistical property of shelter met the hypothesis of above these algorithms just, they can obtain effect preferably.Yet in most of the cases, the assumed condition of these methods can not satisfy, because in the outdoor scene video flowing is followed the tracks of, it is often longer to block the duration, thereby does not satisfy evenly and distribute; In addition, shelter may be comparatively approaching with target on gray-scale value, even thereby certain pixel be blocked, its observed reading is still very near estimated value.Document [12] is by relatively failing to be blocked situation by the image block of motion compensation well and the kinetic characteristic analysis between the overall goals.This method is comparatively effective for surveying shelter, because it has utilized contextual information of time.But this method produces error accumulation and transmission through regular meeting, does not carry out error correction because it utilizes with reference to display model.

List of references

[1]C.Rasmussen，and?G.Hager.Probabilistic?data?association?methods?for?tracking?complexvisual?objects.IEEE?Trans.on?Pattern?Analysis?and?Machine?Intelligence，23(6)：560-576，2001.

[2]C.Hue，J.L.Cadre，P.Prez.Sequential?Monte?Carlo?methods?for?multiple?target?trackingand?data?fusion.IEEE?Trans.on?Signal?Processing，50(2)：309-325，2002.

[3]A.D.Jepson，D.J.Fleet，and?T.F.EI-Maraghi.Robust?online?appearance?model?for?visualtracking.IEEE.Trans.on?Pattern?Analysis?and?Machine?Intelligence，25(10)：1296-1311，2003.

[4]D.Comaniciu，V.Ramesh，and?P.Meer.Kernel-based?object?tracking，IEEE?Trans.onPattern?Analysis?and?Machine?Intelligenee，25(5)：564-577，2003

[5]S.K.Zhou，R.Chellappa，and?B.Moghaddam.Visual?tracking?and?recognition?usingappearance-adaptive?models?for?particle?filters.IEEE?Trans.on?Image?Processing，13(11)：1491-1506，2004.

[6]H.T.Nguyen，M.Worring，and?R.van?den?Boomgaard.Occlusion?robust?adaptive?templatetracking.Proc.IEEE?Int’l?Conf.Computer?Vision，1：678-683，2001.

[7]H.T.Nguyen，and?A.W.M.Smeulders.Fast?occluded?object?tracking?by?a?robust?appearancefilter.IEEE?Trans.on?Pattern?Analysis?and?Machine?Intelligence，26(8)：1099-1104，2004.

[8]Y.Chen，Y.Rui，and?T.Huang.Jpdaf?based?HMM?fbr?real-time?contour?tracking.Proc.IEEE?Conf.on?Computer?Vision?and?Pattern?Recognition，1：543-550，2001.

[9]A.Yilmaz，X.Li，and?M.Shan.Contour?based?object?tracking?with?occlusion?handling?invideo?acquired?using?mobile?cameras.IEEE?Trans.on?Pattern?Analysis?and?MachineIntelligence，26(11)：1531-1536，2004.

[10]S.Baker，and?I.Matthews.Lucas-Kanade?20years?on：a?unifying?framework.Int’l?JournalComputer?Vision，53(3)：221-255，2004.

[11]I.Matthews，T.Ishikawa，and?S.Baker.The?template?update?problem.IEEE?Trans.onPattern?Analysis?and?Machine?Intelligence，26(6)：810-815，2004.

[12]K.Hariharakrishnan，and?D.Schonfeld.Fast?object?tracking?using?adaptive?block?matching.IEEE?Trans.on?Multimedia，7(5)：853-859，2005.

[13]J.Pan，and?B.Hu.Robust?object?tracking?against?template?drift.IEEE?Int’l?Conf.on?ImageProc.2007，to?be?published.

Summary of the invention

The objective of the invention is to propose a kind of content self-adaptive gradual-progression type sheltering analysis (Content-Adaptive ProgressiveOcclusion Analysis, CAPOA) target tracking algorism.

Key of the present invention is how to utilize space-time contextual information and reference target and motion vector, and (Region of Interest, the situation of blocking ROI) is made effective judgement to area-of-interest adaptively.Specifically comprise following content:

At first utilize the space-time contextual information that the situation of blocking of area-of-interest is made initial analysis, further revise PRELIMINARY RESULTS by reference target and motion vector then, obtain final sheltering analysis result.

The from coarse to fine analysis progressively of area-of-interest blocked situation, and dynamically determine the analysis resolution of zones of different.

By setting up interference figure and estimate to utilize the space-time contextual information that the situation of blocking of area-of-interest is made initial analysis, and obtain the ratio of tentatively the blocking γ of each image block based on the reverse of image block.

The first frame target is carried out the increment type interpolation to reference target and filtering obtains.

The dynamic analysis resolution in certain zone of decision: if the ratio of tentatively the blocking γ of image block that should the place, zone is zero, and this image block is enough big, and analysis resolution that then should the zone is exactly the size of current image block; If the ratio of tentatively the blocking γ of image block that should the place, zone is zero, and this image block is big inadequately, this image block has passed through the checking of reference target and motion vector simultaneously, and analysis resolution that then should the zone is exactly the size of current image block; If the ratio of tentatively the blocking γ of image block that should the place, zone is non-vanishing, analysis resolution that then should the zone is taken as mxm..

Further revise the preliminary sheltering analysis result who utilizes the space-time contextual information to obtain by reference target: if the ratio of tentatively the blocking γ of certain image block is zero, and this image block is enough big, can determine that then this image block is not blocked; If the ratio of tentatively the blocking γ of certain image block is zero, and this image block is big inadequately, then relatively the reference target matching error of this image block and reverse compensating error, if the difference of the former with the latter is not more than threshold value t, can determine that then this image block is not blocked, otherwise need carry out the motion vector checking; If the ratio of tentatively the blocking γ of certain image block is greater than zero and reached highest resolution, then relatively the reference target matching error of this image block and reverse compensating error, if the difference of the former with the latter is greater than threshold value t, then the final sheltering analysis result of this image block is exactly the initial analysis result, otherwise need carry out the motion vector checking.

Further revise the preliminary sheltering analysis result who utilizes the space-time contextual information to obtain by motion vector: the image block for carrying out also can't determining after reference target is verified the situation of blocking, carry out the motion vector checking; If the ratio of tentatively the blocking γ of this image block is zero, then calculate the motion vector of this image block and the Euclidean distance between target motion vectors, if this distance is not more than threshold value σ ₁, can determine that then this image block is not blocked, otherwise see and reach highest resolution whether; If reach, determine that then this image block is blocked fully; If the ratio of tentatively the blocking γ of this image block greater than zero less than 1, then calculate the motion vector of this image block and the Euclidean distance between target motion vectors and shelter motion vector respectively, if the former is the big and latter, then the final sheltering analysis result of this image block is exactly the initial analysis result, otherwise determines that this image block is not blocked; If the ratio of tentatively the blocking unity gamma of this image block then calculates the motion vector of this image block and the Euclidean distance between the shelter motion vector, if this distance is not more than threshold value σ ₂, then the final sheltering analysis result of this image block is exactly the initial analysis result, otherwise determines that this image block is not blocked.

Determine that centre coordinate is positioned at ω _bThe threshold value t of image block:

$t\left({\mathrm{\ω}}_b\right)=\frac{3-2\mathrm{\γ}\left({\mathrm{\ω}}_b\right)}{N_{\mathrm{BT}}}\underset{x_t\∈{\mathrm{\Ω}}_{\mathrm{BT}}}{\mathrm{\Σ}}{\mathrm{\σ}}_E^2\left(x_t\right)$

Ω wherein _BTBe the template zone of this image block correspondence, N _BTBe the number of pixels in this zone, σ _EIt is the evaluated error of template zone pixel value.

Threshold value σ ₁Be 3 times of standard deviation of the motion vector of all image blocks that have been confirmed as the target area; Threshold value σ ₂Be 3 times of standard deviation of the motion vector of all image blocks that have been confirmed as the shelter zone.

Be described further below.

1, is with template matches and the template renewal of sheltering

In the algorithm that the present invention proposes, the template matches (masked template matching) that adopts band to shelter finds the optimum coordinates transformation parameter of present frame to determine the geometric position of target.In this process, shelter is sheltered fully, rather than only reduces the weights at place, shelter place as document [6] and [7].This is because CAPOA algorithm of the present invention will explicitly detect shelter.The CAPOA algorithm will be discussed in the next section.The template matches process that band is sheltered can be expressed as

$\hat{a}=\underset{a}{\arg \min }\frac{1}{\mathrm{sum}\left(M\right)}\underset{x\∈{\mathrm{\Ω}}_T}{\mathrm{\Σ}}{(\frac{I_n[\mathrm{\φ}(x;a)-\hat{T}\left(x\right)]}{{\mathrm{\σ}}_E\left(x\right)}\·M\left(x\right))}^2---\left(1\right)$

Wherein,  is the optimal estimation of coordinate conversion parameter, I _nRepresent the n two field picture,

The template that expression estimates, σ _EIt is the expectation value of the evaluated error in the filtering of Kalman's outward appearance ^[13], φ (x; A) be the arbitrary coordinate conversion of depending on parameter a (having reflected translation, convergent-divergent, rotation of target etc. usually), Ω _TRepresent all of template pixel, M shelters with template template of a size, and it is zero in the pixel place value that is blocked, and is 1 in other pixel place value.Sum (M) calculates the number of all template pixels that are not blocked.

The renewal of template can be expressed as follows:

$\hat{T}\left(x\right)\←\hat{T}\left(x\right)+G\left(x\right)\·\left\{I_n\right[\mathrm{\φ}(x;\hat{a})]-\hat{T}\left(x\right)\}\·M\left(x\right)---\left(2\right)$

Wherein, G (x) is the kalman gain in the outward appearance wave filter ^[13]The part that is blocked in the template is not updated.By new template more, track algorithm can be dealt with the dough deformation of target and gradually changing of illumination condition.

The key that improves tracking effect is how to obtain correct template is sheltered M.This need be by means of the following content self-adaptive gradual-progression type sheltering analysis algorithm that will discuss.

2, content self-adaptive gradual-progression type sheltering analysis (CAPOA) algorithm

The overall architecture of CAPOA algorithm is shown among Fig. 1.Wherein the functional block of grey is further described in Fig. 2.The CAPOA algorithm has four inputs: area-of-interest to be analyzed (ROI), reference target (T _Ref), former frame image (I _Prev) and the interference figure (U of former frame _Prev).The CAPOA algorithm has two outputs: the interference figure (U after the renewal _Out) and upgrade after template shelter (M _Out).Their concrete definition will provide hereinafter.

2.1 gradual scanning area-of-interest (ROI)

The algorithm that the present invention proposes detects shelter and is based on image block, rather than as in the document [3,5-7] like that based on individual other pixel.This is because spatial context has very large help for the judgement situation of blocking.One of information that this is relied on when also being human brain judgement shelter.For example, when block two people face parts, have only by analyzing different space structures just may detect the generation of blocking.

Weigh the result preferably in order to obtain one between reliability and resolution, the present invention adopts progressive method scanning area-of-interest (ROI).ROI is a just in time image-region of coverage goal.This zone is a shelter with interior non-target object, and the object beyond this zone is all thought background object.The task of CAPOA algorithm is to determine in the ROI which zone thing that is blocked occupies.In whole C APOA algorithm, ROI experiences repeatedly scanning process, and new each time scanning process all is reduced to the length of side of image block to be analyzed original half.Scanning process is only analyzed the still undetermined image-region of occlusion state each time.After the situation of blocking of whole ROI was all determined, gradual scanning stopped (see figure 1).If the length of side of area-of-interest is respectively D ₁And D ₂, then total scanning times is

N _s＝min{log ₂(min(D ₁，D ₂)/5)，3}(3)

Like this, the minimum length of side of image to be analyzed piece is 5 pixels, and total scanning times mostly is 3 times most.Occlusion state how to determine an image block is shown in Fig. 2, and will describe in detail at down several joints.

2.2 utilize the space-time context

Existing most of algorithm all depends on the current target appearance that observes and itself infers whether block generation, but effect is all undesirable.This is because shelter information itself is not included in the target appearance that observes.In nontarget area when all prior imformations of relevant shelter all are included in the first frame selected target in fact.By the evolutionary process of following the tracks of the nontarget area just can know in the nontarget area object whether " intrusion " target area.Therefore, whether be blocked, can do reverse to this image block and estimate in previous frame, whether be arranged in the nontarget area to observe this image block in order to judge an image block among the ROI.Like this, the situation of blocking of this image block just can be traced back to and be blocked the first known frame of situation priori.Any shelter that enters ROI from the nontarget area (be included in and just occur after the first frame initialization) can both so be detected.In this process, the space-time contextual information just is fully utilized.

In a frame figure, the situation of blocking of target represents that with " interference figure (outlier map) " value of this figure in the nontarget area is 1, and the value in the target area is zero.During the first frame initialization, the target area is exactly ROI, remaining image-region all is the nontarget area, and all potential shelters all are arranged in the nontarget area (this moment, they still were background object, had only entered ROI at them and had been only shelter later on) of interference figure.Before the CAPOA algorithm of operation present frame, the interference figure (U of former frame _Prev) be known.Operation CAPOA algorithm is in order to obtain the interference figure (U of present frame _Out).For each the image to be analyzed piece (being called " current image block ") described in 2.1 joints, at former frame image (I _Prev) in carry out estimation, the image block that finds it to mate most.Here claim this image block to be " reverse estimated image piece ".In theory, the situation of blocking of current image block can be directly from being positioned at the U of reverse estimated image piece _PrevPart is duplicated.But, if current image block is less or just from blocking behind, then its motion estimation result is no longer reliable.So, only according to estimation and U _PrevThe situation of blocking that obtains is judged often not accurate enough, thereby claims the interference figure that obtains in this stage to be " interim interference figure ", and this interference figure need further be revised.The interim interference figure part that is positioned at the current image block place can obtain by following formula:

$U^{\′}\left({\mathrm{\ω}}_b\right)=U_{\mathrm{prev}}\left({\widetilde{\mathrm{\ω}}}_b\right)---\left(4\right)$

Wherein, ω _bBe the centre coordinate of current image block, U ' represents interim interference figure,

Expression is corresponding to reverse estimated image piece position

The former frame interference figure part at place.

For being positioned at ω _bThe current image block at place, note γ (ω _b) be U ' (ω _b) in the shared ratio of number of pixels that is blocked, then current image block can be divided into three classes, as shown in Figure 2.Each class image block all has specific further checking procedure.Its core concept is: 1) if γ is (ω _b) being not equal to zero, then corresponding image-region is analyzed with the highest resolution in the scanning process the last time and is further checked, so that find details and detect the less target area that reappears out from the opposite side of shelter behind; 2) if γ is (ω _b) equal zero, to such an extent as to, just further check then only when the size of image block is too little can not guarantee very reliable estimation the time.The image block that needs further to check in current scanning process is called " uncertain image block ", and their occlusion state will be determined by merging more information.

2.3 utilize reference target to carry out the check first time

Reference target T _RefCan be regarded as through the later template of convergent-divergent.Increment type interpolation and filtering obtains by former reference target is carried out at the end of handling each frame for it.When first frame, T _RefBe initialized as selected target.Each frame is guaranteed T all the time later on _RefBig with realistic objective etc.This realizes by following steps: the back finds that the size of target changes to some extent if present frame disposes, then at first former reference target is carried out the convergent-divergent interpolation, make its size and current goal measure-alike, and then the reference target after the renewal convergent-divergent interpolation, make it comprise the appearance change of target at present frame.The value of some pixels can be expressed as follows in the reference target after the renewal:

T _ref(n+1)＝T′ _ref(n)+G·{ROI(n)-T′ _ref(n)}·(1-u _n)(5)

T wherein _Ref(n+1) be some pixel values that will be used for the reference target of n+1 frame, T ' _Ref(n) be value through the respective pixel in the reference target of the n frame after the convergent-divergent interpolation, G is the kalman gain of the corresponding position that obtains according to (2) formula, and ROI (n) is the pixel value of ROI corresponding position, u _nIt is the value of the resultant interference figure corresponding position of n frame.Because the variation of target size is very little between adjacent two frames, therefore each all directly to go out the method for reference target from the template interpolation much smaller for the error ratio of above-mentioned increment type interpolation, and comprise more details.Therefore, reference target is more suitable in benchmark as a comparison than template itself.

If a uncertain image block belongs to target, then it necessarily to reference target in appropriate section comparatively similar.Based on this, seek this uncertain image block in the reference target relevant position and optimum matching image block on every side, and computation of match errors, weigh with mean square deviation (MSE).This image block also has a matching error (obtaining) when doing the reverse estimation in former frame.For brevity, previous matching error is e ² _Ref, a back matching error is e ² _BwdIf the γ (ω of a uncertain image block _b) equal zero then and if only if its e ² _RefCompare e ² _BwdWhen little or bigger, this image block can be confirmed as not carrying on the back really and block, both U _Out(ω _b)=U ' (ω _b).Similarly, if the γ (ω of a uncertain image block _b) be not equal to zero, then and if only if its e ² _RefMuch larger than e ² _BwdThe time, U _Out(ω _b)=U ' (ω _b) set up.The criterion that Here it is with reference target when doing to check for the first time.Here, key issue is threshold value a: e how obtaining adaptively must using in the criterion at each image block ² _Ref-e ² _BwdThis threshold value is represented with t in Fig. 2.

Because the expectation value σ of evaluated error in (1) formula _EReflected that the observed reading of a pixel that belongs to target departs from the expectation size of respective pixel values in the template (or reference target), therefore allows threshold value t depend on the average σ of the template pixel relevant with this image block ² _EReasonably.In fact, can prove an e who belongs to the image block of target ² _Ref-e ² _BwdThe expectation value of (being t) is

$E\{e_{\mathrm{ref}}^2-e_{\mathrm{bwd}}^2\}=\frac{1}{N_{\mathrm{BT}}}\underset{x_t\∈{\mathrm{\Ω}}_{\mathrm{BT}}}{\mathrm{\Σ}}{\mathrm{\σ}}_E^2\left(x_t\right)---\left(6\right)$

Wherein, Ω _BTRepresent the template zone relevant, N with this image block _BTIt is the number of pixels in this zone.Be omitted in this proof.

With the result who utilizes the space-time context to obtain comprehensive after, at being positioned at ω _bThe threshold value t of uncertain image block be provided with as follows:

$t\left({\mathrm{\ω}}_b\right)=E\{e_{\mathrm{ref}}^2-e_{\mathrm{bwd}}^2\}\·[3-2\mathrm{\γ}\left({\mathrm{\ω}}_b\right)]=\frac{3-2\mathrm{\γ}\left({\mathrm{\ω}}_b\right)}{N_{\mathrm{BT}}}\underset{x_t\∈{\mathrm{\Ω}}_{\mathrm{BT}}}{\mathrm{\Σ}}{\mathrm{\σ}}_E^2\left(x_t\right)---\left(7\right)$

Less γ (ω _b) mean that this image block more likely belongs to target, therefore should establish threshold value more greatly.From (7) formula as seen, the threshold value t among Fig. 2 is a content-adaptive for the different images piece: higher evaluated error or lower blocking than regular meeting produce bigger t.

2.4 utilize motion vector to carry out the check second time

For the uncertain image block of the described criterion of a joint in check for the first time, not satisfying, need further check with motion vector and utilize the judgement of blocking that the space-time context obtains.Obviously, the image block that belongs to target (or shelter) has and the similar motion vector of target (or shelter).Based on this, the motion vector by more uncertain image block and the motion vector of target (or shelter) can help to determine the situation of blocking of this image block.

The concrete criterion of check is shown among Fig. 2 for the second time.Here, v _BlkThe motion vector of representing uncertain image block, it estimates to obtain by reverse; v _Tgt(v _Otl) represent the motion vector of target (or shelter), it obtains by calculating all motion vector mean values that have been confirmed as the pixel place image block of target (or shelter); σ _Tgt(σ _Otl) be the root-mean-square value of the motion vector of all pixel place image blocks that have been confirmed as target (or shelter) to the Euclidean distance of average motion vector.

The situation of blocking of whole all pixels of ROI is all determined back (be equivalent to the pixel value that is positioned at ROI in the interference figure and all determine the back), and the CAPOA algorithm has just been finished.Being positioned at the outer pixel value of ROI in the interference figure all is 1.Whenever being updated the back, interference figure (promptly obtains U _OutThe back), template is sheltered also and is upgraded according to following formula:

M _out(x)＝1-U _out{round[φ(x；)]} (8)

Round[wherein] representative operation that each element of vector is rounded.

According to foregoing, the concrete operations step of a kind of content self-adaptive gradual-progression type sheltering analysis target tracking algorism of the present invention is as follows:

1. in first frame selected target zone (being Initial R OI).

2. following initialization interference figure U: the target area value is 0, and all the other regional values are 1.

3. following initialization template

By initial coordinate conversion φ (x; a _s) sampling Initial R OI, promptly $\hat{T}\left(x\right)=I_n\left[\mathrm{\φ}(x;a_s)\right],$ Wherein

a _sInitial coordinate transformation parameter for target.

4. reference target T _RefBe initialized as Initial R OI.

5. template is sheltered M and is initialized as and big complete 1 matrixes such as template.

6. read in next frame.

With template by coordinate transform φ (x; A) be mapped to present frame.Obtain reflecting the coordinate conversion parameter vector  of the geological information of target in present frame, i.e. operation (1) formula by the image-region of seeking the present frame that mates most with prediction module.

8. the total degree of scanning process is determined according to (3) formula.

9. ROI being divided into series of rectangular image to be analyzed piece scans.The initial length of side of image to be analyzed piece is taken as half of the ROI length of side.

10. (its centre coordinate is positioned at ω to each image to be analyzed piece that comprises the pixel of also not determining occlusion state _b), in former frame, do reverse and estimate.

11. obtain the interim interference figure U ' of present frame according to (4) formula.

12. determine the occluded pixels ratio γ (ω of reverse estimated image piece according to the former frame interference figure _b).γ (ω _b) value that is specially the former frame interference figure of reverse estimated image piece position is the ratio that the number of 1 pixel accounts for the total number of pixel.

13. if γ (ω _b) equal zero and the image to be analyzed piece enough big, then be defined as the interim interference figure of the present frame of this image to be analyzed piece position, i.e. U at the interference figure of the present frame of this image to be analyzed piece position _Out(ω _b)=U ' (ω _b).Forwarded for the 26th step to.

14. if γ (ω _b) equal zero and the image to be analyzed piece big inadequately, then seek this image block in the reference target relevant position and optimum matching image block on every side, and computation of match errors, weigh with mean square deviation (MSE), and be designated as e ² _RefRemember that simultaneously it is e that this image block is made the matching error of reverse estimation in former frame ² _BwdIf the e of this image block ² _Ref-e ² _BwdBe not more than threshold value t, then U by the definition of (7) formula _Out(ω _b)=U ' (ω _b).Forwarded for the 26th step to.

15. if γ (ω _b) equal zero and the image to be analyzed piece is big inadequately and the e of this image block ² _Ref-e ² _BwdGreater than threshold value t, then calculate the motion vector v of target by the definition of (7) formula _TgtIt obtains by calculating all motion vector mean values that have been confirmed as the pixel place image block of target.Remember that simultaneously this image block estimates that by do reverse in former frame the motion vector obtain is v _BlkCalculate v _BlkWith v _TgtBetween Euclidean distance ‖ v _Blk-v _Tgt‖.Calculate the root-mean-square value of the motion vector of all pixel place image blocks that have been confirmed as target, be designated as σ to the Euclidean distance of average motion vector _TgtIf ‖ is v _Blk-v _Tgt‖ is not more than σ _Tgt3 times, U then _Out(ω _b)=U ' (ω _b).Forwarded for the 26th step to.

16. if γ (ω _b) equal zero and the image to be analyzed piece is big inadequately and the e of this image block ² _Ref-e ² _BwdGreater than threshold value t and ‖ v by the definition of (7) formula _Blk-v _Tgt‖ is greater than σ _Tgt3 times and be last scanning process, then this image block is defined as blocking entirely, i.e. U _Out(ω _b)=1, wherein 1 is complete 1 matrix.Forwarded for the 26th step to.

17. if γ (ω _b) equal zero and the image to be analyzed piece is big inadequately and the e of this image block ² _Ref-e ² _BwdGreater than threshold value t and ‖ v by the definition of (7) formula _Blk-v _Tgt‖ is greater than σ _Tgt3 times and be not last scanning process, then the occlusion state of this image block is still uncertain.Forwarded for the 26th step to.

18. if γ (ω _b) greater than zero less than 1 and be not last scanning process, then the occlusion state of this image block is uncertain.Forwarded for the 26th step to.

19. if γ (ω _b) greater than zero less than 1 and be last scanning process, then seek this image block in the reference target relevant position and optimum matching image block on every side, and computation of match errors, weigh with mean square deviation (MSE), and be designated as e ² _RefRemember that simultaneously it is e that this image block is made the matching error of reverse estimation in former frame ² _BwdIf e ² _Ref-e ² _BwdGreater than the threshold value t by the definition of (7) formula, then U _Out(ω _b)=U ' (ω _b).Forwarded for the 26th step to.

20. if γ (ω _b) greater than zero less than 1 and be last scanning process and e ² _Ref-e ² _BwdBe not more than threshold value t, then calculate the motion vector v of target by the definition of (7) formula _TgtIt obtains by calculating all motion vector mean values that have been confirmed as the pixel place image block of target.Calculate the motion vector v of shelter then _OtlIt obtains by calculating all motion vector mean values that have been confirmed as the pixel place image block of shelter.Remember that simultaneously this image block estimates that by do reverse in former frame the motion vector obtain is v _BlkCalculate v respectively _BlkWith v _TgtBetween Euclidean distance ‖ v _Blk-v _Tgt‖ and v _BlkWith v _OtlBetween Euclidean distance ‖ v _Blk-v _Otl‖.If ‖ is v _Blk-v _Otl‖ is not more than ‖ v _Blk-v _Tgt‖, then U _Out(ω _b)=U ' (ω _b).Forwarded for the 26th step to.

21. if γ (ω _b) greater than zero less than 1 and be last scanning process and e ² _Ref-e ² _BwdBe not more than threshold value t and ‖ v by the definition of (7) formula _Blk-v _Otl| greater than ‖ v _Blk-v _Tgt‖, then this image block is defined as blocking entirely, i.e. U _Out(ω _b)=0, wherein 0 is full null matrix.Forwarded for the 26th step to.

22. if γ (ω _b) equal 1 and be not last scanning process, then the occlusion state of this image block is uncertain.Forwarded for the 26th step to.

23. if γ (ω _b) equal 1 and be last scanning process, then seek this image block in the reference target relevant position and optimum matching image block on every side, and computation of match errors, weigh with mean square deviation (MSE), and be designated as e ² _RefRemember that simultaneously it is e that this image block is made the matching error of reverse estimation in former frame ² _BwdIf e ² _Ref-e ² _BwdGreater than the threshold value t by the definition of (7) formula, then U _Out(ω _b)=U ' (ω _b).Forwarded for the 26th step to.

24. if γ (ω _b) equal 1 and be last scanning process and e ² _Ref-e ² _BwdBe not more than threshold value t, then calculate the motion vector v of shelter by the definition of (7) formula _OtlIt obtains by calculating all motion vector mean values that have been confirmed as the pixel place image block of shelter.Remember that simultaneously this image block estimates that by do reverse in former frame the motion vector obtain is v _BlkCalculate v _BlkWith v _OtlBetween Euclidean distance ‖ v _Blk-v _Otl‖.Calculate the root-mean-square value of the motion vector of all pixel place image blocks that have been confirmed as shelter, be designated as σ to the Euclidean distance of average motion vector _OtlIf ‖ is v _Blk-v _Otl‖ is not more than σ _Otl3 times, U then _Out(ω _b)=U ' (ω _b).Forwarded for the 26th step to.

25. if γ (ω _b) equal 1 and be last scanning process and e ² _Ref-e ² _BwdBe not more than threshold value t and ‖ v by the definition of (7) formula _Blk-v _Otl‖ is greater than σ _Otl3 times, then this image block is defined as blocking entirely, i.e. U _Out(ω _b)=0, wherein 0 is full null matrix.

26. if, then forwarded for the 10th step to comprise the image to be analyzed piece of also not determining the pixel of occlusion state among the ROI in addition.

27. the length of side of image to be analyzed piece is reduced by half.

28., then forwarded for the 9th step to if the situation of blocking of whole ROI is also not definite fully.

Shelter M 29. obtain the template of present frame by (8) formula.

30. pass through more new template of (2) formula

31. upgrade reference target by (5) formula _Tref

32., then forwarded for the 6th step to, otherwise finish if video flowing has been untreated.

Description of drawings

Fig. 1: CAPOA algorithm overview flow chart.

Fig. 2: analyze current image block and block the process flow diagram of situation.

Fig. 3: different track algorithms are handled a performance of blocking for a long time relatively.First to fourth row has shown the tracking results of document [4], [7], [12] and algorithm of the present invention respectively.In each row, show the 1980th, 2013,2032 and 2052 frames from left to right respectively.In first row, tracking results is represented with an ellipse; In remaining each row, tracking results is with the rectangle frame of cross to represent in the middle of one.For second row and the fourth line, the situation of blocking (or template is sheltered) of current goal and the lower right corner that is presented at every width of cloth image when front template from left to right; For the third line, only show the situation of blocking of current goal in the relevant position.The pixel that is blocked is represented with black.

Embodiment

We have compared different track algorithms and have handled the performance of blocking on a large amount of outdoor scene video flowings.These outdoor scene video flowings comprise different types of target and the various scene of blocking, and in addition, the motion of camera is arbitrarily.We are divided into two types to the scene of blocking of 30 test video streams: short-term is blocked and is blocked for a long time.If a duration of blocking surpasses 25 frames, then be considered to block for a long time.We have compared the performance of algorithm of the present invention and some current latest algorithm ([4,7,12]).In experiment, we consider only to comprise the coordinate transform of translation and zooming parameter, because used template is dynamic.

Experimental result is shown in table 1.Can observe document [7] algorithm process short-term and block more effectively in experiment, because in blocking for a long time, shelter meeting " infiltration " goes in the display model to cause display model destroyed.The similar algorithm (for example [3,5,6]) that blocks the mechanism of detection of other all employings is all met this phenomenon inevitably.The performance of document [12] algorithm is much better, but owing to block situation analysis result's propagation of error and fail effectively to detect the target area that reappears from the shelter another side, the performance of this algorithm is still not ideal enough.Because document [4] algorithm (Mean Shift) does not have the mechanism that explicit processing is blocked, thereby its effect is the poorest.Algorithm of the present invention blocks in each class on nearly all video flowing of situation has all followed the tracks of target well.Can observe, algorithm of the present invention always can detect shelter effectively, and the template thing that seldom is blocked destroys.

For the sake of clarity, Fig. 3 has shown the tracing process of a more difficult video flowing.This video flowing comprises one and blocks for a long time, and one of them man (being target) is seriously blocked by two other pedestrian and surpassed 50 frames.Because this man is walking about, the outward appearance of target itself also constantly changes.Some part of shelter is very close with the gray scale of target.All track algorithms all carry out identical initialization and begin following the tracks of at the 1920th frame.Can see that document [4] algorithm fails to catch up with target, because when blocking generation, histogrammic separating capacity is not enough.Document [7] algorithm so comparatively can't distinguished shelter and target in the complex environment well.Document [12] algorithm is more effective than aforementioned algorithm for surveying shelter, but is used to check result based on motion analysis because it does not have a display model, and it blocks and has occurred a large amount of mistakes among the situation analysis result.Algorithm of the present invention has best sheltering analysis performance.It also has the ability of differentiating the background interference thing (see the template among each figure of fourth line shelter).What deserves to be mentioned is, the interference figure that algorithm of the present invention generates never is right-on (template of seeing fourth line the 3rd width of cloth figure is sheltered), but these mistakes are always corrected by twice checkout procedure of CAPOA algorithm rapidly, thereby seldom can go (template of seeing fourth line the 4th width of cloth figure is sheltered) by spread out.If twice checkout procedure in the CAPOA algorithm removed, algorithm then of the present invention can be run into and document [12] algorithm similar problem.

Claims (9)

1. content self-adaptive gradual-progression type sheltering analysis target tracking algorism, it is characterized in that at first utilizing the space-time contextual information that the situation of blocking of area-of-interest is made initial analysis, further revise PRELIMINARY RESULTS by reference target and motion vector then, obtain final sheltering analysis result.

2. content self-adaptive gradual-progression type sheltering analysis target tracking algorism according to claim 1 is characterized in that from coarse to fine the analysis progressively of area-of-interest blocked situation, and dynamically determines the analysis resolution of zones of different.

3. content self-adaptive gradual-progression type sheltering analysis target tracking algorism according to claim 1, it is characterized in that by setting up interference figure and estimate to utilize the space-time contextual information that the situation of blocking of area-of-interest is made initial analysis, and obtain the ratio of tentatively the blocking γ of each image block based on the reverse of image block.

4. content self-adaptive gradual-progression type sheltering analysis target tracking algorism according to claim 1 is characterized in that reference target carries out the increment type interpolation to the first frame target and filtering obtains.

5. according to claim 1,2,3, one of 4 described content self-adaptive gradual-progression type sheltering analysis target tracking algorisms, it is characterized in that dynamically determining the analysis resolution in certain zone: if the ratio of tentatively the blocking γ of image block that should the place, zone is zero, and this image block is enough big, and analysis resolution that then should the zone is exactly the size of current image block; If the ratio of tentatively the blocking γ of image block that should the place, zone is zero, and this image block is big inadequately, this image block has passed through the checking of reference target and motion vector simultaneously, and analysis resolution that then should the zone is exactly the size of current image block; If the ratio of tentatively the blocking γ of image block that should the place, zone is non-vanishing, analysis resolution that then should the zone is taken as mxm..

6. according to claim 1,3,4, one of 5 described content self-adaptive gradual-progression type sheltering analysis target tracking algorisms, it is characterized in that further revising the preliminary sheltering analysis result who utilizes the space-time contextual information to obtain: if the ratio of tentatively the blocking γ of certain image block is zero by reference target, and this image block is enough big, can determine that then this image block is not blocked; If the ratio of tentatively the blocking γ of certain image block is zero, and this image block is big inadequately, then relatively the reference target matching error of this image block and reverse compensating error, if the difference of the former with the latter is not more than threshold value t, can determine that then this image block is not blocked, otherwise need carry out the motion vector checking; If the ratio of tentatively the blocking γ of certain image block is greater than zero and reached highest resolution, then relatively the reference target matching error of this image block and reverse compensating error, if the difference of the former with the latter is greater than threshold value t, then the final sheltering analysis result of this image block is exactly the initial analysis result, otherwise need carry out the motion vector checking.

7. according to claim 1,3,5, one of 6 described content self-adaptive gradual-progression type sheltering analysis target tracking algorisms, it is characterized in that further revising the preliminary sheltering analysis result who utilizes the space-time contextual information to obtain by motion vector: the image block for carrying out also can't determining after reference target is verified the situation of blocking, carry out the motion vector checking; If the ratio of tentatively the blocking γ of this image block is zero, then calculate the motion vector of this image block and the Euclidean distance between target motion vectors, if this distance is not more than threshold value σ ₁, can determine that then this image block is not blocked, otherwise see and reach highest resolution whether; If reach, determine that then this image block is blocked fully; If the ratio of tentatively the blocking γ of this image block greater than zero less than 1, then calculate the motion vector of this image block and the Euclidean distance between target motion vectors and shelter motion vector respectively, if the former is the big and latter, then the final sheltering analysis result of this image block is exactly the initial analysis result, otherwise determines that this image block is not blocked; If the ratio of tentatively the blocking unity gamma of this image block then calculates the motion vector of this image block and the Euclidean distance between the shelter motion vector, if this distance is not more than threshold value σ ₂, then the final sheltering analysis result of this image block is exactly the initial analysis result, otherwise determines that this image block is not blocked.

8. content self-adaptive gradual-progression type sheltering analysis target tracking algorism according to claim 6 is characterized in that determining that centre coordinate is positioned at ω _bThe threshold value t of image block:

$t\left({\mathrm{\ω}}_b\right)=\frac{3-2\mathrm{\γ}\left({\mathrm{\ω}}_b\right)}{N_{\mathrm{BT}}}\underset{x_t\∈{\mathrm{\Ω}}_{\mathrm{BT}}}{\mathrm{\Σ}}{\mathrm{\σ}}_E^2\left(x_t\right)$

Ω wherein _BTBe the template zone of this image block correspondence, N _BTBe the number of pixels in this zone, σ _EIt is the evaluated error of template zone pixel value.

9. content self-adaptive gradual-progression type sheltering analysis target tracking algorism according to claim 7 is characterized in that threshold value σ ₁Be 3 times of standard deviation of the motion vector of all image blocks that have been confirmed as the target area; Threshold value σ ₂Be 3 times of standard deviation of the motion vector of all image blocks that have been confirmed as the shelter zone.

Images