CN103530894A - Video target tracking method based on multi-scale block sparse representation and system thereof - Google Patents

Abstract

The invention relates to a video target tracking method based on multi-scale block sparse representation and a system thereof. Compared with the prior art, the invention solves the defect that the conventional target tracking method cannot achieve a better effect. The method comprises the following steps: initializing a target feature template; tracking a target image; updating a target appearance feature template; checking whether a video is read or not. Through the adoption of the method, target tracking precision and efficiency in complex scenes are improved, and the application degree of a target tracking technology in various scenes is improved.

Description

A kind of video object method for tracing and system thereof based on multiple dimensioned rarefaction representation

Technical field

The present invention relates to intelligent video technology field, is a kind of video object method for tracing and system thereof based on multiple dimensioned rarefaction representation specifically.

Background technology

Target tracking is an important research direction in computer vision field, and in the application of many reality, as the understanding of video monitoring, video scene, interactive video processing etc., all plays vital effect.Present stage researchist has proposed multiple different tracing algorithm, and under different application scenarioss, has obtained certain success.But traditional image local block size based on local sparse representation model production method selects directly impact to follow the trail of effect, follow the trail of the objective is when living through longer video sequence, block, the shape of target appearance, the illumination of environment etc. all can change, therefore target following does not often reach good effect, and the technical merit of target following is still not high.How to develop a kind of in video frequency tracking process outward appearance represent that target tracking method that model is more effective, healthy and strong and system thereof have become and be badly in need of the technical matters that solves.

Summary of the invention

The defect that the object of the invention is not reach in order to solve method for tracking target in prior art better effects, provides a kind of video object method for tracing and system thereof based on multiple dimensioned rarefaction representation to solve the problems referred to above.

To achieve these goals, technical scheme of the present invention is as follows:

A video object method for tracing for multiple dimensioned rarefaction representation, comprises the following steps:

Initialization target signature template, to video object position initialization, constructs target appearance feature templates by video the first two field picture;

The image that follows the trail of the objective, by reading video next frame image configuration next frame external appearance characteristic, and and the first frame external appearance characteristic template between carry out similarity comparison, maximum similarity be that video object is in the position of next frame;

Upgrade target appearance feature templates, candidate's external appearance characteristic of similarity maximum in previous step is updated to target appearance feature templates, for the external appearance characteristic of next frame image, carry out similarity comparison;

Check whether video reads complete, if read completely, complete video object and follow the trail of, if do not read complete, the operation of the image of proceeding to follow the trail of the objective.

Described initialization target signature template comprises the following steps:

Initialization video the first frame target image position, searches out the position of target image in video the first two field picture;

According to video the first frame target image, carry out establishing target view data dictionary;

The first frame video object image is carried out to multiple dimensioned processing, by constructed destination image data dictionary, calculate the sparse coefficient of target image under multiple dimensioned, and this sparse coefficient is represented as target appearance feature templates.

The described image that follows the trail of the objective comprises the following steps:

Read video next frame image;

By particle filter mode, in next frame video image, select candidate target image, build multiple dimensioned blocks of data dictionary;

Successively candidate target image is carried out to multiple dimensioned and process and calculate corresponding sparse coefficient;

The sparse coefficient of calculated candidate target image and the similarity between target appearance feature templates, the candidate target image of maximum similarity is video object in the position of next frame.

Described renewal target appearance feature templates is defined as follows:

$T_{\mathrm{new}}^s=\mathrm{\ω}{T}_{\mathrm{first}}^s+(1-\mathrm{\ω})T_{\mathrm{temp}}^s$

Here ω, for upgrading weights, is set as 0.9 in this method. be illustrated in feature templates new under yardstick s.

Comprising the following steps of described establishing target view data dictionary:

The first two field picture I and corresponding target image in given video;

By get K image local piece in object region, obtain set D={d _i| i=1:K}, here d _ibe i localized target image block, destination image data dictionary is D.

Comprising the following steps of the multiple dimensioned blocks of data dictionary of described structure:

Next frame image I+1 and corresponding candidate target image in given video;

In candidate target image-region, get K image local piece, the size of each topography's piece is made as respectively 3x3,5x5, and 7x7,9x9,11x11 is totally 5 yardstick pieces;

Obtain set

here d _ibe i localized target image block, s is 5 yardstick piece parameters, and target image multi-Scale Data piece dictionary is D ^s.

The calculating of the external appearance characteristic template of described target comprises the following steps:

If

the topography's piece extracting under different scale from the first frame target image, by multiple dimensioned blocks of data dictionary, each topography's piece

have a corresponding sparse coefficient, its account form is as follows:

${\hat{a}}_i^s=\arg \min {\left|\right|{\mathrm{\&alpha;}}_i^s\left|\right|}_1\mathrm{subjectto}{\left|\right|p_i^s-D^s{\mathrm{\&alpha;}}_i^s\left|\right|}_2<\mathrm{\&epsiv;}$

for corresponding target topography piece

sparse coefficient;

Collect Local sparsity coefficient and be expressed as target appearance feature templates, being defined as follows:

$T^s={[{\hat{a}}_1^s,{\hat{a}}_2^s,...,{\hat{a}}_K^s]}^T.$

The sparse coefficient of described target image and the similarity between target appearance feature templates are calculated as follows:

If sim is (T _r, T _q) representing similarity between the first frame target image feature templates and next frame candidate target characteristics of image target, it is defined as follows:

$\mathrm{sim}(T_r,T_q)=\underset{s=1}{\overset{m}{\mathrm{\&Sigma;}}}{\mathrm{\&lambda;}}^s\mathrm{\&rho;}(T_r^s,T_q^s)$

Here λ ^srepresent similarity weights coefficient under different scale s, ρ is two Bhattacharyya distances between target image, and the less similarity of its distance is higher, is defined as follows:

$\mathrm{\&rho;}(T_r^s,T_q^s)=\underset{j=1}{\overset{K}{\mathrm{\&Sigma;}}}\sqrt{T_r^s\left(j\right)\&CenterDot;T_q^s\left(j\right)}.$

A system for the video object method for tracing of multiple dimensioned rarefaction representation, comprising:

Initialization load module, for setting the video initialized location parameter that will follow the trail of the objective, starts video frequency tracking system, real-time tracing video object;

Target image multi-Scale Data dictionary builds module, for initialization target image being carried out to multiple dimensioned dictionary, calculates and builds;

Target appearance feature templates computing module, for the calculating of the sparse coefficient of multiple dimensioned lower target image, and using sparse coefficient as clarification of objective template;

Candidate target image similarity module, for similarity between main calculating target image feature templates and next frame candidate target characteristics of image target;

Target signature template renewal module, for upgrading the first frame feature templates, to adapt to the variation of target outward appearance etc. in video;

Described initialization load module is connected to target image multi-Scale Data dictionary and builds module, target image multi-Scale Data dictionary builds module and is connected to target appearance feature templates computing module, target appearance feature templates computing module is connected with target signature template renewal module by candidate target image similarity module, the target signature template renewal module target image multi-Scale Data dictionary structure module that is linked back.

Beneficial effect

A kind of video object method for tracing and system thereof based on multiple dimensioned rarefaction representation of the present invention, has compared with prior art improved precision and the efficiency of the target tracking in complex scene, has promoted the level of application of target tracking technology in all kinds of scenes.Utilize video image theoretical at the rarefaction representation under multiple dimensioned, by Image Multiscale process, the series of steps such as multiple dimensioned dictionary structure, multiple dimensioned rarefaction representation, similarity calculating, target localization, target in automatic tracing video, according to the result of following the trail of, automatically upgrade target appearance characteristic model, to adapt to the target tracking task in complicated video scene.

Accompanying drawing explanation

Fig. 1 is video object method for tracing process flow diagram of the present invention

Fig. 2 is video object tracing system johning knot composition of the present invention

Embodiment

For making that architectural feature of the present invention and the effect reached are had a better understanding and awareness, in order to preferred embodiment and accompanying drawing, coordinate detailed explanation, be described as follows:

As shown in Figure 1, a kind of video object method for tracing based on multiple dimensioned rarefaction representation of the present invention, comprises the following steps:

The first step, initialization target signature template, to video object position initialization, constructs target appearance feature templates by video the first two field picture.Initialization target signature template is the video object position of finding in video the first two field picture, and carries out initialized process operation.Its initialization target signature template comprises the following steps:

(11) initialization video the first frame target image position, searches out the position of target image in video the first two field picture.In actual use, if desired personage's head portrait is carried out to target tracking, personage's head portrait is just target image so, determines the position of personage's head portrait, has namely determined target image position, and this target image is exactly required content of following the trail of below.

(12) according to video the first frame target image, carry out establishing target view data dictionary, because the image of the first note is definite, target image wherein is also determined, target image is still the scope of an image, at this, just need to carry out cutting apart of all size window to target image, this set of cutting apart is just called destination image data dictionary.

Carry out establishing target view data dictionary and also comprise the following steps, by following methods, built:

(121) the first two field picture I and corresponding target image in given video;

(121), by get K image local piece in object region, obtain set D={d _i| i=1:K}, here d _ibe i localized target image block, destination image data dictionary is D.

(13) the first frame video object image is carried out to multiple dimensioned processing, by constructed destination image data dictionary, calculate the sparse coefficient of target image under multiple dimensioned, and this sparse coefficient is represented as target appearance feature templates.By target image being carried out to multiple dimensioned processing, by the further refinement of target image, guaranteed the accuracy that target image is followed the trail of, combining target view data dictionary calculates sparse coefficient again, this sparse coefficient of pin is as the target appearance feature templates that is used for comparing, to guarantee that accurately investigation is to the image that follows the trail of the objective of next frame.The computing method of the external appearance characteristic template of target image are as follows:

(131) establish the topography's piece extracting under different scale from the first frame target image, by multiple dimensioned blocks of data dictionary, each topography's piece have a corresponding sparse coefficient, its account form is as follows:

${\hat{a}}_i^s=\arg \min {\left|\right|{\mathrm{\&alpha;}}_i^s\left|\right|}_1\mathrm{subjectto}{\left|\right|p_i^s-D^s{\mathrm{\&alpha;}}_i^s\left|\right|}_2<\mathrm{\&epsiv;}$

for corresponding target topography piece

sparse coefficient;

(132) collect Local sparsity coefficient and be expressed as target appearance feature templates, being defined as follows:

$T^s={[{\hat{a}}_1^s,{\hat{a}}_2^s,...,{\hat{a}}_K^s]}^T.$

So far, complete the overall process of initialization target signature template, realized location and template gatherer process to target image in image the first frame, for postorder image ratio is to doing basis.

Second step, the image that follows the trail of the objective, by reading video next frame image configuration next frame external appearance characteristic, and and the first frame external appearance characteristic template between carry out similarity comparison, maximum similarity be that video object is in the position of next frame.This process is that the image of next frame is carried out to determining of target image, thereby finds the target image position of required tracking in next frame image.Its image that follows the trail of the objective comprises the following steps:

(21) read video next frame image, from video next frame image, find target image position, thereby realize target is followed the trail of.

(22) by particle filter mode, in next frame video image, select candidate target image, build multiple dimensioned blocks of data dictionary.Next frame video image is equally also needed to carry out cutting apart of all size window, and its method that builds multiple dimensioned blocks of data dictionary is:

(221) next frame image I+1 and corresponding candidate target image in given video, determine the image that will cut apart.

(222) in candidate target image-region, get K image local piece, the size of each topography's piece is made as respectively 3x3,5x5, and 7x7,9x9,11x11 is totally 5 yardstick pieces, generally can have 300-500 image local piece.

(223) obtain set

here d _ibe i localized target image block, s is 5 yardstick piece parameters, and target image multi-Scale Data piece dictionary is D ^s.

(23) successively candidate target image is carried out to multiple dimensioned and process and calculate corresponding sparse coefficient, after getting target image multi-Scale Data piece dictionary, by being that candidate target image carries out multiple dimensioned processing to certain image, by the further refinement of candidate target image, then combining target view data dictionary calculates sparse coefficient.The computing method of sparse coefficient are identical with the method for compute sparse coefficient in the 13rd step, thereby calculate the corresponding sparse coefficient that multiple dimensioned degree is processed.

(24) the sparse coefficient of calculated candidate target image and the similarity between target appearance feature templates, the candidate target image of maximum similarity is video object in the position of next frame.The sparse coefficient of candidate target image and target appearance feature templates are contrasted, the quantity of the sparse coefficient of candidate target image is a plurality of, contrast with target appearance feature templates seriatim, thereby find the sparse coefficient of maximum similarity, thereby further determine that video object is in the position of next frame.Sparse coefficient and the similarity calculating method between target appearance feature templates of target image are as follows:

If sim is (T _r, T _q) representing similarity between the first frame target image feature templates and next frame candidate target characteristics of image target, it is defined as follows:

$\mathrm{sim}(T_r,T_q)=\underset{s=1}{\overset{m}{\mathrm{\&Sigma;}}}{\mathrm{\&lambda;}}^s\mathrm{\&rho;}(T_r^s,T_q^s)$

Here λ ^srepresent similarity weights coefficient under different scale s, ρ is two Bhattacharyya distances between target image, and the less similarity of its distance is higher, is defined as follows:

$\mathrm{\&rho;}(T_r^s,T_q^s)=\underset{j=1}{\overset{K}{\mathrm{\&Sigma;}}}\sqrt{T_r^s\left(j\right)\&CenterDot;T_q^s\left(j\right)}.$

The 3rd step, upgrades target appearance feature templates, and candidate's external appearance characteristic of similarity maximum in previous step is updated to target appearance feature templates, for the external appearance characteristic of next frame image, carries out similarity comparison.Owing to having completed the location tracking of target image in previous step, the target image in the image of present frame will be as target appearance feature templates so, for the work of comparing of next frame image.The method of upgrading target appearance feature templates is as follows:

$T_{\mathrm{new}}^s=\mathrm{\&omega;}{T}_{\mathrm{first}}^s+(1-\mathrm{\&omega;})T_{\mathrm{temp}}^s$

Here ω, for upgrading weights, is set as 0.9 in this method.

be illustrated in feature templates new under yardstick s.

The 4th step, checks whether video reads complete, if read completely, complete video object and follow the trail of, if do not read complete, the operation of the image of proceeding to follow the trail of the objective.Judge whether current video target tracking all completes, if current video finishes, complete all tracing processes; If current video does not finish, also have the picture of next frame to exist, continue to get back to second step, the operation of the image of proceeding to follow the trail of the objective, until video finishes, tracing process finishes.

As shown in Figure 2, the system of a kind of video object method for tracing based on multiple dimensioned rarefaction representation of the present invention, is characterized in that, comprise: initialization load module, be used for setting the video initialized location parameter that will follow the trail of the objective, start video frequency tracking system, real-time tracing video object.Target image multi-Scale Data dictionary builds module, for initialization target image being carried out to multiple dimensioned dictionary, calculates and builds.Target appearance feature templates computing module, for the calculating of the sparse coefficient of multiple dimensioned lower target image, and using sparse coefficient as clarification of objective template.Candidate target image similarity module, for similarity between main calculating target image feature templates and next frame candidate target characteristics of image target.Target signature template renewal module, for upgrading the first frame feature templates, to adapt to the variation of target outward appearance etc. in video.

Initialization load module is connected to target image multi-Scale Data dictionary and builds module, the data after initialization is passed to target image multi-Scale Data dictionary and build the structure that module is carried out multi-Scale Data dictionary.Target image multi-Scale Data dictionary builds module and is connected to target appearance feature templates computing module, carries out the calculating of target appearance feature templates.Target appearance feature templates computing module is connected with target signature template renewal module by candidate target image similarity module, the data of target appearance feature templates computing module reach candidate target image similarity module carry out similarity relatively after, after having determined the target image of following the trail of, reach again target signature template renewal module, the target image tracking is defined as to the target signature template of subsequent figures picture comparison, then the target signature template renewal module target image multi-Scale Data dictionary that is linked back builds module, by data reach target image multi-Scale Data dictionary build module a heavier new round carry out multi-Scale Data dictionary and build module, comparison process.

More than show and described ultimate principle of the present invention, principal character and advantage of the present invention.The technician of the industry should understand; the present invention is not restricted to the described embodiments; what in above-described embodiment and instructions, describe is principle of the present invention; the present invention also has various changes and modifications without departing from the spirit and scope of the present invention, and these changes and improvements all fall in claimed scope of the present invention.The protection domain that the present invention requires is defined by appending claims and equivalent thereof.

Claims (9)

1. the video object method for tracing based on multiple dimensioned rarefaction representation, is characterized in that, comprises the following steps:

11) initialization target signature template, to video object position initialization, constructs target appearance feature templates by video the first two field picture;

12) image that follows the trail of the objective, by reading video next frame image configuration next frame external appearance characteristic, and and the first frame external appearance characteristic template between carry out similarity comparison, maximum similarity be that video object is in the position of next frame;

13) upgrade target appearance feature templates, candidate's external appearance characteristic of similarity maximum in previous step is updated to target appearance feature templates, for the external appearance characteristic of next frame image, carry out similarity comparison;

14) check whether video reads complete, if read completely, complete video object and follow the trail of, if do not read complete, the operation of the image of proceeding to follow the trail of the objective.

2. a kind of video object method for tracing based on multiple dimensioned rarefaction representation according to claim 1, is characterized in that, described initialization target signature template comprises the following steps:

21) initialization video the first frame target image position, searches out the position of target image in video the first two field picture;

22) according to video the first frame target image, carry out establishing target view data dictionary;

23) the first frame video object image is carried out to multiple dimensioned processing, by constructed destination image data dictionary, calculate the sparse coefficient of target image under multiple dimensioned, and this sparse coefficient is represented as target appearance feature templates.

3. a kind of video object method for tracing based on multiple dimensioned rarefaction representation according to claim 1, is characterized in that, the described image that follows the trail of the objective comprises the following steps:

31) read video next frame image;

32) by particle filter mode, in next frame video image, select candidate target image, build multiple dimensioned blocks of data dictionary;

33) successively candidate target image is carried out to multiple dimensioned and process and calculate corresponding sparse coefficient;

34) the sparse coefficient of calculated candidate target image and the similarity between target appearance feature templates, the candidate target image of maximum similarity is that video object is in the position of next frame.

4. a kind of video object method for tracing based on multiple dimensioned rarefaction representation according to claim 1, is characterized in that, described renewal target appearance feature templates is defined as follows:

$T_{\mathrm{new}}^s=\mathrm{\&omega;}{T}_{\mathrm{first}}^s+(1-\mathrm{\&omega;})T_{\mathrm{temp}}^s$

Here ω, for upgrading weights, is set as 0.9 in this method.

be illustrated in feature templates new under yardstick s.

5. a kind of video object method for tracing based on multiple dimensioned rarefaction representation according to claim 2, is characterized in that, the comprising the following steps of described establishing target view data dictionary:

51) the first two field picture I and corresponding target image in given video;

52), by get K image local piece in object region, obtain set D={d _i| i=1:K}, here d _ibe i localized target image block, destination image data dictionary is D.

6. a kind of video object method for tracing based on multiple dimensioned rarefaction representation according to claim 3, is characterized in that, the comprising the following steps of the multiple dimensioned blocks of data dictionary of described structure:

61) next frame image I+1 and corresponding candidate target image in given video;

62) in candidate target image-region, get K image local piece, the size of each topography's piece is made as respectively 3x3,5x5, and 7x7,9x9,11x11 is totally 5 yardstick pieces;

63) obtain set

here d _ibe i localized target image block, s is 5 yardstick piece parameters, and target image multi-Scale Data piece dictionary is D ^s.

7. a kind of video object method for tracing based on multiple dimensioned rarefaction representation according to claim 2, is characterized in that, the calculating of the external appearance characteristic template of described target comprises the following steps:

71) establish the topography's piece extracting under different scale from the first frame target image, by multiple dimensioned blocks of data dictionary, each topography's piece

have a corresponding sparse coefficient, its account form is as follows:

${\hat{a}}_i^s=\arg \min {\left|\right|{\mathrm{\&alpha;}}_i^s\left|\right|}_1\mathrm{subjectto}{\left|\right|p_i^s-D^s{\mathrm{\&alpha;}}_i^s\left|\right|}_2<\mathrm{\&epsiv;}$

for corresponding target topography piece

sparse coefficient;

72) collect Local sparsity coefficient and be expressed as target appearance feature templates, being defined as follows:

$T^s={[{\hat{a}}_1^s,{\hat{a}}_2^s,...,{\hat{a}}_K^s]}^T.$

8. a kind of video object method for tracing based on multiple dimensioned rarefaction representation according to claim 3, is characterized in that, the sparse coefficient of described target image and the similarity between target appearance feature templates are calculated as follows:

If sim is (T _r, T _q) representing similarity between the first frame target image feature templates and next frame candidate target characteristics of image target, it is defined as follows:

$\mathrm{sim}(T_r,T_q)=\underset{s=1}{\overset{m}{\mathrm{\&Sigma;}}}{\mathrm{\&lambda;}}^s\mathrm{\&rho;}(T_r^s,T_q^s)$

Here λ ^srepresent similarity weights coefficient under different scale s, ρ is two Bhattacharyya distances between target image, and the less similarity of its distance is higher, is defined as follows:

$\mathrm{\&rho;}(T_r^s,T_q^s)=\underset{j=1}{\overset{K}{\mathrm{\&Sigma;}}}\sqrt{T_r^s\left(j\right)\&CenterDot;T_q^s\left(j\right)}.$

9. a system for the video object method for tracing based on multiple dimensioned rarefaction representation, is characterized in that, comprising:

Initialization load module, for setting the video initialized location parameter that will follow the trail of the objective, starts video frequency tracking system, real-time tracing video object;

Target image multi-Scale Data dictionary builds module, for initialization target image being carried out to multiple dimensioned dictionary, calculates and builds;

Target appearance feature templates computing module, for the calculating of the sparse coefficient of multiple dimensioned lower target image, and using sparse coefficient as clarification of objective template;

Candidate target image similarity module, for similarity between main calculating target image feature templates and next frame candidate target characteristics of image target;

Target signature template renewal module, for upgrading the first frame feature templates, to adapt to the variation of target outward appearance etc. in video;

Described initialization load module is connected to target image multi-Scale Data dictionary and builds module, target image multi-Scale Data dictionary builds module and is connected to target appearance feature templates computing module, target appearance feature templates computing module is connected with target signature template renewal module by candidate target image similarity module, the target signature template renewal module target image multi-Scale Data dictionary structure module that is linked back.

Images