CN1300746C - Video frequency motion target adaptive tracking method based on multicharacteristic information fusion - Google Patents

Abstract

The present invention relates to a self-adaptive tracking method for video motion objects based on multi-characteristic information fusion, which is used in the fields of computer vision, image processing and pattern recognition. The method comprises the following steps: firstly, extracting characteristic information about a target; then, using a fuzzy logic method to fuse various kinds of characteristic information, wherein the fused information is used for describing observation information of the target; finally, finding out a candidate target which has the most similar information with the observation information of a target template in the current image target candidate zone by adopting a genetic algorithm which has a heuristic search function. The present invention leads the reliability of tracking to be greatly improved by fusing the characteristic information in a self-adaptive mode and adopting the heuristic search method when searching for the target and can be widely used in various civil and military systems, such as video monitoring systems, video conference systems, robot vision navigation systems, manufactured product monitoring systems, military target tracking systems, etc. Thus, the present invention has extensive market prospect and high application value.

Description

Video frequency motion target adaptive tracking method based on the multicharacteristic information fusion

Technical field

What the present invention relates to is a kind of method that is used for the adaptive tracing of technical field of image processing, specifically is a kind of video frequency motion target adaptive tracking method that merges based on multicharacteristic information.

Background technology

Video frequency object tracking is a research focus of computer vision, Flame Image Process and area of pattern recognition.Accurately following the tracks of in real time of target in the video image all is with a wide range of applications on civil and military.There are some difficult points in video frequency object tracking always, as: clarification of objective generation significant change during the tracking, follow the tracks of during target block, various interference in the background image or the like.At these difficult points, Chinese scholars has proposed a variety of trackings, these methods comprise Kalman filter, Condensation, MeanShift and CamShift etc., but majority is to utilize the single characteristic information of target that target is followed the tracks of, often be difficult to the robust tracking of realization to target, this is because clarification of objective information can change owing to the displacement of target or the interference of background image during following the tracks of, if variation has taken place in a certain characteristic information of target, if still utilize this characteristic information tracking target, then can cause and follow the tracks of failure.At the problems such as poor robustness that video target tracking method under the single characteristic information exists, many scholars have proposed to utilize the various features information of target that target is followed the tracks of.The advantage of this method is can realize message complementary sense between various characteristic informations, if wherein a kind of characteristic information lost efficacy during following the tracks of, can utilize other characteristic informations to continue to keep tracking to target.The difficult point of this method is to realize the optimum fusion of various characteristic informations.At present the tracking that merges based on multicharacteristic information of great majority is when merging each characteristic information, each feature fixedly weights have been adopted, the deficiency of this fusion method is: if there is a characteristic information to change obviously during following the tracks of, and its weights remain unchanged, information after then merging can be unreliable, utilize this information trace target can cause tracking effect undesirable, even can follow the tracks of failure.

Find by prior art documents, D.Comaniciu etc. are at " IEEE Transactions onPattern Analysis and Machine Intelligence " (pp.564-577,2003) deliver " Kernel-based Object Tracking " (based on the target following of nuclear, pattern analysis and machine intelligence IEEE magazine) on.This article has proposed to utilize the method for color and edge feature information trace target, and the test findings in the literary composition illustrates that this method has tracking performance preferably.But the situation when this method is not considered color of object generation significant change and do not make full use of complementary information between color and edge feature, so this method usually can become inapplicable in actual applications owing to the dynamic change of scene.

Summary of the invention

The objective of the invention is to overcome deficiency of the prior art, a kind of video frequency motion target adaptive tracking method that merges based on multicharacteristic information is provided, make it improve tracking power moving target under the complex environment.

The present invention is achieved by the following technical solutions, the present invention at first extracts clarification of objective information, utilizing fuzzy logic method that various characteristic informations are carried out self-adaptation then merges, and the information after will merging is used to describe the observation information of target, carry out target search at last, adopt genetic algorithm in the present image object candidate area, to find out and the most similar candidate target of To Template observation information with heuristic search function based on genetic algorithm.

Below the inventive method is further described, particular content is as follows:

(1) extracts clarification of objective information

Clarification of objective is the CF feature in the present invention.The color characteristic of target mainly is by its color distribution reflection, when describing color distribution, has adopted the weighting color histogram of carrying space positional information, and this histogram has been considered the spatial positional information of pixel, has improved the robustness of describing color characteristic; Object Shape Feature Extraction when describing the target shape feature, has adopted oval template by image being carried out edge extracting and carrying out obtaining after the range conversion, promptly adopts the shape of target in the big or small oval approximate description image of difference.This mainly is because the head that tracked target is behaved, and generally speaking, people's contouring head shape can be with an oval approximate description.

(2) the characteristic information self-adaptation merges

When Fusion of Color and shape facility information, adopted the fuzzy logic method self-adaptation to regulate both weights.The weight that fuzzy logic is regulated in the next frame fusion information both according to the confidence level of present frame color of object and shape information realizes the adaptive weighted fusion of two kinds of characteristic informations.

(3) based on the target search of genetic algorithm

When target search, adopted genetic algorithm; Genetic algorithm is kind of a heuristic search, and its search efficiency is far superior to exhaustive search algorithm.Genetic algorithm continuous iteration optimizing in the current frame image object candidate area searches out and the most similar candidate target of To Template observation information.When utilizing the genetic algorithm ferret out, at first the center and the radius of candidate target region are encoded, utilize then and intersect and the new individuality of the continuous generation of mutation operator, utilize at last and select operator to find out the best individuality of fitness, the real number that this individuality decoding obtains is the radius size of the position and the region thereof of corresponding present frame target.

The present invention has adopted the observation information of the fusion information description target of CF; When Fusion of Color and shape information, adopted the method for self-adaptation fusion, improved the reliability of target observation information greatly; In this fusion method, fuzzy logic has realized the adaptive weighted fusion of CF information according to both weight in the online adjusting next frame of the confidence level of present frame color of object and the shape information observation information; In target search and during following the tracks of, adopt genetic algorithm with heuristic search function, improved search efficiency.The present invention is merged each characteristic information by self-adaptation and is adopted when the target search heuristic search to make the reliability of following the tracks of improve greatly, can be widely used in having vast market prospect and using value in all kinds of civilian and military systems such as video monitoring system, video conferencing system, robot visual guidance system, industrial products supervisory system, military target tracker.

Description of drawings

Fig. 1 is a disposal route The general frame of the present invention.

Wherein Fig. 1 (a) merges block diagram for the CF information weighting; Fig. 1 (b) is online adjusting weights block diagram; Fig. 1 (c) is a target search process block diagram.

Fig. 2 is the membership function figure of fuzzy logic system input, output variable among the present invention.

Fig. 3 is the tracking effect figure of target.

Wherein Fig. 3 (a) is for utilizing the tracking effect figure of colouring information separately; Fig. 3 (b) is for utilizing the tracking effect figure of shape information separately; The tracking effect figure that Fig. 3 (c) merges for non-self-adapting; Fig. 3 (d) is tracking effect figure of the present invention.

Fig. 4 is the change curve of each characteristic information weights during following the tracks of.

Embodiment

In order to understand technical scheme of the present invention better, embodiments of the present invention are further described below in conjunction with accompanying drawing.

The The general frame of a kind of video frequency motion target adaptive tracking method that merges based on multicharacteristic information that Fig. 1 proposes for the present invention.The concrete implementation detail of each several part is as follows:

The weighting fusion block diagram of Fig. 1 (a) CF information

1. colouring information

When describing color of object and distribute, adopted the color histogram method of carrying space positional information, this method has been considered the spatial positional information of pixel, therefore to the description of color distribution robust more.The center of known target zone if (elliptic region) be x=(x, y); Radius is h=(h _x, h _y), h _x, h _yBe respectively long axis of ellipse and minor axis, the position of target area interior pixel is x _i=(x ₁, y _i), i=1 ... n _h, then the color distribution of target can be described with following formula:

${\hat{p}}^{\left(u\right)}(x,h)=C_h\underset{i=1}{\overset{n_h}{\mathrm{\Σ}}}g\left({\left|\right|\frac{x-x_i}{h}\left|\right|}^2\right)\mathrm{\δ}[b\left(x_i\right)-u]...\left(1\right)$

Wherein δ is the Delta function, n _hBe total number of target area interior pixel, function b (x _i) be to be positioned at x _iThe map of pixel color level index on histogram at place, u is a color level index in the histogram, u=1 ..., L, L are the sum of color level on the histogram, C _hBe normaliztion constant, $\underset{u=1}{\overset{L}{\mathrm{\Σ}}}{\hat{p}}^{\left(u\right)}(x,h)=1$ Under the condition, can get $C_h=1/\underset{i=1}{\overset{n_h}{\mathrm{\Σ}}}g\left({\left|\right|\frac{x-x_i}{h}\left|\right|}^2\right),$ G () is the weights function:

Function g () gives different weights with the pixel on the diverse location in the target area, and with the near more pixel of target's center, the weights of giving are big more, and this is because the peripheral pixel of target may be blocked or belong to background pixel, is insecure relatively cause.

2. color similarity degree

When the evaluate color similarity, adopt the Bhattacharyya distance to weigh.The color distribution of hypothetical target template is $\hat{q}={\left\{{\hat{q}}^{\left(u\right)}\right\}}_{u=1...L},$ The color distribution of candidate target is $\hat{p}(x,h)={\left\{{\hat{p}}^{\left(u\right)}(x,h)\right\}}_{u=1...L},$ Then candidate target and the To Template similarity coefficient on color is:

$\mathrm{\ρ}[\hat{p}(x,h),\hat{q}]=\underset{u=1}{\overset{L}{\mathrm{\Σ}}}\sqrt{{\hat{p}}^{\left(u\right)}(x,h){\hat{q}}^{\left(u\right)}}...\left(3\right)$

This coefficient is the Bhattacharyya coefficient, and the Bhattacharyya distance is:

$d_c[\hat{p}(x,h),\hat{q}]=\sqrt{1-\mathrm{\ρ}[\hat{p}(x,h),\hat{q}]}...\left(4\right)$

After obtaining the Bhattacharyya distance of two color histograms, the color likelihood function that is defined as follows:

$p\left(z_c\right|x,h)=\frac{1}{\sqrt{2\mathrm{\π}}{\mathrm{\σ}}_c}\exp (-\frac{d_c^2[\hat{q},\hat{p}(x,h)]}{2{\mathrm{\σ}}_c^2})...\left(5\right)$

σ wherein _cBe Gauss's covariance, the color of the big more explanation candidate target of value in (5) formula is similar more to the color of To Template.

3. shape information

The head that tracked target is behaved among the present invention.Generally speaking, head part's contour shape is similar to elliptical shape, therefore can be with the shape of an oval approximate representation target, and promptly the shape template of the number of people is an ellipse.

4. shape similarity

When estimating the shape similarity, adopt the Chamfer distance to weigh.If the binary map of known target shape template is T, the binary map of current frame image is I, and the range image of present frame is DI.Then the distance of the Chamfer between candidate target and the To Template can be calculated by following formula,

$d_s[T(x,h),I]=\sqrt{\frac{1}{\left|T\right|}\underset{t\∈T}{\mathrm{\Σ}}\mathrm{DI}\left(t\right)}...\left(6\right)$

Wherein, T (x, h) center and the size of expression shape template in present frame is respectively x and h, | pixel value is the total number of pixel of " 1 " among the T| representative image T, t pixel that pixel value is " 1 " among the t presentation video T, when DI (t) represents on image T being put in range image DI, be arranged in the gray-scale value of t pixel value of image T for the DI under the location of pixels of " 1 ".After obtaining two kinds of Chamfer distances between the shape, the shape likelihood function that is defined as follows:

$p\left(z_s\right|x,h)=\frac{1}{\sqrt{2\mathrm{\π}}{\mathrm{\σ}}_s}\exp (-\frac{d_s^2[T(x,h),I]}{2{\mathrm{\σ}}_s^2})...\left(7\right)$

σ wherein _sBe Gauss's covariance, the value in (7) formula is big more, illustrates that the shape of candidate target is similar more to shape template.

5. merge

The observation information of target is described jointly by CF information, if the center of known n candidate target is x _n=(x ⁽ⁿ⁾, y ⁽ⁿ⁾) and the radius of region be $h_n=(h_x^{\left(n\right)},h_y^{\left(n\right)}),$ Then the observation likelihood function of this candidate target is defined as follows:

p (z|x _n，h _n)＝αp(z _c|x _n，h _n)+βp(z _s|x _n，h _n)，α+β＝1 (8)

P (z wherein _c| x _n, h _n) and p (z _s| x _n, h _n) being respectively the likelihood function of CF, 0≤α≤1 and 0≤β≤1 is respectively the weights of CF information.If (8) value in the formula is big more, then this candidate target is similar more on CF to To Template, and this candidate target is that the possibility of real goal is just big more.

The online adjusting weights of Fig. 1 (b) block diagram

1. CF Reliability of Information

If the center of known present frame target be x=(x, y) and the radius of region be h=(h _x, h _y), then the color confidence level of target is defined as:

$e_c=\frac{1}{\sqrt{2\mathrm{\π}}{\mathrm{\σ}}_c}\exp (-\frac{d_c^2[\hat{q},\hat{p}(x,h)}{2{\mathrm{\σ}}_c^2})...\left(9\right)$

The shape confidence level of target is defined as:

$e_s=\frac{1}{\sqrt{2\mathrm{\π}}{\mathrm{\σ}}_s}\exp (-\frac{d_s^2[T(x,h),I}{2{\mathrm{\σ}}_s^2})...\left(10\right)$

2. fuzzy logic

Most based on multicharacteristic information fusion goal tracking when merging each characteristic information; each characteristic information fixing weights have been adopted; promptly α and the β in (8) formula immobilizes during target following; this fusion method can become improper owing to the dynamic change of scene and the displacement of target in actual applications; usually can be owing to the unexpected variation of certain characteristic information; and cause tracking effect undesirable, even can follow the tracks of failure.The present invention adopts the fuzzy logic self-adaptation to regulate the weights of each characteristic information in order to overcome this shortcoming, has realized the adaptive weighted fusion of each characteristic information.

Fuzzy logic mainly is made up of four parts, that is: obfuscation, fuzzy rule base, fuzzy reasoning mechanism and ambiguity solution.The present invention has adopted single-point obfuscation, product reasoning and center of gravity ambiguity solution, and fuzzy rule has adopted the form in (11) formula.

R ^j：if e ₁ is A ₁ ^j and e ₂ is A ₂ ^j…and e _N is A _N ^j then u is B ^j (11)

R wherein ^jBe j bar fuzzy rule, j=1 ..., k is a number of fuzzy rules, e _iFor i input of fuzzy logic (i=1 ..., N), N is the sum of fuzzy logic input, u is the result of fuzzy reasoning, A _i ^jAnd B ^jBe the fuzzy language value of input and output variable, characterize by membership function.

Fuzzy logic be input as colouring information confidence level e _cWith shape information confidence level e _sFuzzy logic is output as colouring information weights α, and shape information weights β is calculated by (8) formula.With e _cAnd e _sFuzzy turning to { SR, S, M, B, BR}, fuzzy { ST, VS, SR, S, M, B, BR, VB, the BT} of turning to of α; Each fuzzy language variable implication is: ST (very little), VS (little), SR (less), S (small), M (medium), B (little big), BR (bigger), VB (greatly), BT (very big).The membership function of input and output variable and fuzzy reasoning table are respectively shown in Fig. 2 and table 1.

Table 1. fuzzy reasoning table

α		e c
							SR	S	M	B	BR
		e s	SR S M B BR	M S SR VS ST	B M S SR VS	BR B M S SR						VB BR B M S	BT VB BR B M

Target search process such as Fig. 1 (c)

The search of target is realized by genetic algorithm among the present invention, and genetic algorithm has the heuristic search function, is far superior to the exhaustive search method on search efficiency.

1. determine the search volume

The search volume is the initial individual spatial dimension that distributes of genetic algorithm.The radius of supposing the center of target in the former frame image and region thereof be respectively x=(x, y) and h=(h _x, h _y), then the search volume of genetic algorithm is in present frame: being [x-a, x+a] on x, is [y-b, y+b] on y, at h _xGo up and be [h _x-h _a, h _x+ h _a], at h _yGo up and be [h _y-h _b, h _y+ h _b], a wherein, b, h _aAnd h _bBe constant, the size of these parameters is relevant with the motion state of target, and when target travel was violent, then a and b got a little louder, when target sizes changes when violent, and h then _aAnd h _bGet a little louder, otherwise, point then all got.

2. produce initial individual

When utilizing genetic algorithm to carry out target search, it is initially individual at first will to produce some, these individualities produce by computer random in the search volume, each individuality is a string of binary characters, promptly individuality has been adopted the binary coding mode, wherein each individual corresponding a kind of possible separating, genetic algorithm constantly acts on these individualities by genetic manipulation, and selects good individuality according to the natural principle of " survival of the fittest ".

3. intersect and variation

Intersecting and making a variation is two important parameters of genetic algorithm, and genetic algorithm constantly produces new individuality by these two operators, and the excellent characteristic of parent individuality is entailed offspring individual.Among the present invention, single-point cross method and even variation method have been adopted; Crossing-over rate is p _c=0.8, aberration rate is p _m=0.01.

4. decoding

Because the individuality in the colony is a string of binary characters, can not be directly as the separating of reality, so these strings of binary characters must be converted to real number.Decoding is converted to real number with these strings of binary characters, corresponding respectively one group of possible the separating of these real numbers exactly.

5. the weighting fusion of CF information

Each individuality in the colony obtains one group of real number by decoding, the center of the corresponding candidate target of this group real number and the radius of its region.After obtaining the center and zone radius of candidate target, utilize (5) formula and (7) formula to calculate this candidate target and the similarity of To Template on CF, utilize (8) formula then they weighting fusion.The weighting fusion detailed process is shown in Fig. 1 (a).

6. fitness function calculates

The foundation of fitness function design is: similar more on CF to To Template when candidate target, the fitness function value of then giving this candidate target is big more.N individual fitness function is as follows in the definition colony:

f(x _n，h _n)＝p(z|x _n，h _n) (12)

X wherein _n=(x ⁽ⁿ⁾, y ⁽ⁿ⁾) and $h_n=(h_x^{\left(n\right)},h_y^{\left(n\right)})$ Be respectively n the pairing real number in individual decoding back, the center of their corresponding n candidate targets and the radius of its region, p (z|x _n, h _n) be the observation likelihood function of the candidate target of this individuality correspondence, in (8) formula, did definition.

7. end condition

Genetic algorithm search end condition is its iterations, and iterations is many more, and then search precision can be high more, they be that target tracking accuracy is high more, but iterations is many more, real-time is poor more, therefore in practice, can only between compromise and consider to select iterations.Adopted iteration 15 times.

8. export target position

After the genetic algorithm end condition satisfied, the best individuality of fitness was retained and decodes, and the real number that decoding obtains is the radius of center He its region of target in this frame.

9. calculate the CF Reliability of Information

Behind the radius that obtains present frame target location and its region, utilize (9) formula and (10) formula to calculate the confidence level of color of object and shape information.

10. the calculating of next frame weights

After the confidence level that obtains present frame color of object and shape information, utilize fuzzy logic to regulate the weight of CF information in fusion information in the next frame, concrete adjustment process is shown in Fig. 1 (b).

Fig. 3 is the tracking effect figure of target, and wherein Fig. 3 (a) is for utilizing the tracking effect figure of colouring information separately; Fig. 3 (b) is for utilizing the tracking effect figure of shape information separately; The tracking effect figure that Fig. 3 (c) merges for non-self-adapting; Fig. 3 (d) is tracking effect figure of the present invention.From figure the result as can be seen, the method among the present invention has improved the tracking power to moving target under the complex environment greatly.

Fig. 4 is the change curve of each characteristic information weights during following the tracks of.As can be seen from the figure, when colouring information or shape information became unreliable, the fusion method among the present invention can weaken their influences in fusion information by reducing its weights.

Claims (1)

1, a kind of video frequency motion target adaptive tracking method that merges based on multicharacteristic information is characterized in that, at first extracts the CF characteristic information of target; Under the fuzzy logic framework two kinds of characteristic informations are merged then, fuzzy logic is regulated both weights in fusion information according to the degree of reliability of two kinds of characteristic informations during following the tracks of, and realizes that the self-adaptation of two kinds of characteristic informations merges; Carry out target search at last based on genetic algorithm, designed the fitness function that merges based on the multicharacteristic information self-adaptation, genetic algorithm searches out in the present image object candidate area and the most similar candidate target of To Template observation information as search criterion with this fitness function;

Describedly under the fuzzy logic framework, two kinds of characteristic informations are merged, specific as follows: the CF information of at first extracting target, then two kinds of characteristic informations are carried out fusion treatment, when merging two kinds of characteristic informations, utilize the fuzzy logic method self-adaptation to regulate both weights, the weight that fuzzy logic is regulated in the next frame fusion information both according to the fiduciary level of color of object during following the tracks of and shape information realizes that the self-adaptation of two kinds of characteristic informations merges;

Described fusion is specially: the observation information of target is described jointly by CF information, if the centre bit stomach of known n candidate target is x _n=(x ⁽ⁿ⁾, y ⁽ⁿ⁾) and the radius of region be $h_n=(h_x^{\left(n\right)},h_y^{\left(n\right)}),$ Then the observation likelihood function of this candidate target is defined as follows:

p(z|x _n，h _n)＝αp(z _c|x _n，h _n)+βp(z _s|x _n，h _n)，α+β＝1

P (z wherein _c| x _n, h _n) and p (z _s| x _n, h _n) be respectively the likelihood function of CF, 0≤α≤1 and 0≤β≤1 is respectively the weights of CF information, if the value in the following formula is big more, then the place near the steps candidate target is similar more on CF to To Template, and this candidate target is that the possibility of real goal is just big more;

Described fuzzy logic has adopted single-point obfuscation, product reasoning and center of gravity ambiguity solution, and fuzzy rule has adopted the form in the following formula:

R ^j：if e ₁ is A ₁ ^j and e ₂ is A ₂ ^j…and e _N is A _N ^j then u is B ^j

R wherein ^jBe j bar fuzzy rule, j=1 ..., k is a number of fuzzy rules, e _iBe i input of fuzzy logic, i=1 ..., N, N are the sum of fuzzy logic input, u is the result of fuzzy reasoning, A _i ^jAnd B ^jBe the fuzzy language value of input and output variable, characterize by membership function; Fuzzy logic be input as colouring information confidence level e _cWith shape information confidence level e _sFuzzy logic is output as colouring information weights α, and shape information weights β is with e _cAnd e _sFuzzy turning to { SR, S, M, B, BR}, fuzzy { ST, VS, SR, S, M, B, BR, VB, the BT} of turning to of α; Each fuzzy language variable implication is: ST for very little, VS be little, SR be less, S be small, M be medium, B be little big, BR for big, VB for greatly, BT is for very big;

Described target search based on genetic algorithm, specific as follows:

When target search, adopted genetic algorithm with heuristic search function, designed the fitness function that merges based on the multicharacteristic information self-adaptation, continuous iteration optimizing in the current frame image object candidate area searches out and the most similar candidate target of To Template observation information genetic algorithm as search criterion with this fitness function, this candidate target is the target that will follow the tracks of, when utilizing the genetic algorithm ferret out, at first the center and the radius of candidate target region are encoded, utilize then and intersect and the new individuality of the continuous generation of mutation operator, and calculate each individual fitness function, utilize at last and select operator to find out the best individuality of fitness, the real number that this individuality decoding obtains is the radius size of the position and the region thereof of corresponding present frame target.

Images