CN101714256B - Omnibearing vision based method for identifying and positioning dynamic target - Google Patents

Abstract

The invention relates to an omnibearing vision based method for identifying and positioning a dynamic target, belonging to the technical field of dynamic image analysis. The method comprises the following steps of: 1, acquiring an omnibearing vision sequence image, and preprocessing the omnibearing vision sequence image to obtain a binary image separating a moving target and a background area; 2, searching a local area by an optical flow method, matching feature points between adjacent frames of the image, and detecting the moving target of an image sequence; 3, estimating the moving state of the moving target by a particle filtering algorithm, and predicting the parameter of the moving target in a subsequent frame so as to complete a tracking process. The invention can obviously reduce the calculating amount and enhance the accuracy by identifying and positioning the dynamic target by the method.

Description

Dynamic object identification and localization method based on omni-directional visual

Technical field

The invention belongs to the dynamic image analysis technical field, relate to a kind of Target Recognition and localization method based on omni-directional visual.

Background technology

The basic task of dynamic image analysis is to detect movable information, recognition and tracking fortune target from image sequence.It relates to Flame Image Process, graphical analysis, artificial intelligence and pattern-recognition, computer vision etc. and studies carefully the field; it is very active branch in Flame Image Process and the computer vision neighborhood; obtained widespread use in commercial production, the fields such as health, national defense construction for the treatment of, therefore the research to it has crucial sincere justice.

In order to discern moving target and to realize to its tracking, people adopt the method for optical flow field usually, from the image sequence that contains moving target of real-time collection, extract optical flow field, filter out the bigger motion target area of light stream and calculate the velocity of moving target, thereby realized the tracking of moving target.

The object detection method based on light stream in the past mainly is divided into two classes: (1) utilizes the differential optic flow technique promptly to utilize the fundamental equation of light stream, and additional certain constraint obtains fine and close optical flow field, extracts moving target again.The deficiency of the method is that calculated amount is bigger, and real-time is not strong.(2) use the characteristic light stream technology, seek unique point and mate in image, obtain the sparse optical flow field, the real-time of extracting this method of target is improved, but the quantity of information deficiency causes the omission of target easily.And aspect target following, way is in the past usually separated it, and after realize detecting, the feature of based target is followed the tracks of again, does the complexity that has just increased algorithm process like this, brings complicated processing procedure when the entering and withdraw from of target.

Summary of the invention

The objective of the invention is to above-mentioned deficiency at prior art, the present invention proposes a kind of under omni-directional visual the effective ways of maneuvering target recognition and tracking.Real-time and robustness that this method can improve identification and follow the tracks of make the mobile robot have the comprehensive function of continental embankment independent navigation and maneuvering target tracking.

The technical solution used in the present invention is as follows:

A kind of dynamic object identification and localization method based on omni-directional visual comprise the following steps:

Step 1: obtain the omni-directional visual sequence image, this sequence image is carried out pre-service, obtain a moving target and background and make a distinction bianry image;

Step 2: carry out local area search with optical flow method, carry out the coupling of unique point between the image consecutive frame, detect the moving target of image sequence;

Step 3: target state is estimated that the parameter of predicted motion target in subsequent frame finished tracing process by particle filter algorithm.

As preferred implementation, above-mentioned dynamic object identification and localization method based on omni-directional visual, step 2 is wherein carried out according to following method, if the moving image function f (x, y) be continuous function about variable x, y, during moment t, (x, the gray-scale value of y) locating are f to 1 a=on the image _t(x, y), when moment t+ Δ t, this this point moves to reposition, and its position on image becomes (x+ Δ x, y+ Δ y), and gray-scale value is designated as f _{T+ Δ t}(x+ Δ x, y+ Δ y), the purpose of coupling is exactly the corresponding point of seeking a, makes f _t(x, y)=f _{T+ Δ t}(x+ Δ x, y+ Δ y), and make an a=(x, y) in the neighborhood of the M * N that sets, least mean-square error MSE (Δ x, Δ y) minimum, can make MSE (Δ x, Δ y) minimum be Optimum Matching point opt=(Δ x, Δ y),

Make f=f _t(x, y)-f _{T+ Δ t}(x, y), $\▿f={[\frac{{\∂f}_{t+\mathrm{\Δt}}}{\∂\mathrm{\Δx}},\frac{{\∂f}_{t+\mathrm{\Δt}}}{\∂\mathrm{\Δy}}]}^T$ Be the gradient of pixel (Δ x, Δ y), then, $\frac{\mathrm{MN}}{2}\·{\left[\frac{\∂\mathrm{MSE}(\mathrm{\Δx},\mathrm{\Δy})}{\∂(\mathrm{\Δx},\mathrm{\Δy})}\right]}^T=\underset{m=1}{\overset{M}{\mathrm{\Σ}}}\underset{n=1}{\overset{N}{\mathrm{\Σ}}}{\▿f}^T\·(\mathrm{\Δx},\mathrm{\Δy})-\underset{m=1}{\overset{M}{\mathrm{\Σ}}}\underset{n=1}{\overset{N}{\mathrm{\Σ}}}f$

Order $U=\underset{m=1}{\overset{M}{\mathrm{\Σ}}}\underset{n=1}{\overset{N}{\mathrm{\Σ}}}{\▿f}^T,$ $V=\underset{m=1}{\overset{M}{\mathrm{\Σ}}}\underset{n=1}{\overset{N}{\mathrm{\Σ}}}f,$ Try to achieve Optimum Matching point opt=(Δ x, Δ y)=U ^-1V mates by seek unique point in image, detects the moving target of image sequence;

Step 3 is wherein carried out according to following method:

(1) according to the result in second step, initial target is positioned, and obtains the initial motion parameter of target:

P ^Init=(P ^Initx, P ^Init _y), establish each particle and represent a kind of possible motion state, getting population is N, the initial weight w of particle _i=1, then have N possible motion state parameters P ⁱ=(P ⁱ _X, P ⁱ _Y), (i ∈ 1 ... N).

(2) carry out particle resampling process, eliminate the less particle of weights, keep the bigger particle of weights;

(3) change the iterative process of particle filter algorithm over to: from each later frame of second frame, each particle is carried out system state to be shifted and systematic observation, calculate the weights of particle, and all particles are weighted estimated value with the export target state, finish tracing process;

Carry out state transitions according to following formula: to particle N ⁱ, P is arranged ⁱ _Xt=A ₁P ⁱ _Xt-1+ B ₁w ⁱ _T-1And P ⁱ _Xt=A ₂P ⁱ _Xt-1+ B ₂w ⁱ _T-1, wherein, A ₁, A ₂, B ₁, B ₂Be constant, A gets 1, and B is that particle is propagated radius, and w is the random number in [1,1];

Carry out systematic observation according to following method:

(1) after each particle state shifts, utilize new coordinate and, calculate a minimum average B configuration absolute difference function MAD ⁱ

(2) establishing probability density function is $p\left(z_k\right|x_k^i)=\exp \{-\frac{1}{{2\mathrm{\σ}}^2}{\mathrm{MAD}}^i\}$ Wherein, σ is a constant, and then the weights of each particle are: $w_k^i=w_{k-1}^{i}p\left(z_k\right|x_k^i);$

(3) weights to each particle carry out normalized: $w_k^i=w_k^i/\underset{i=1}{\overset{N}{\mathrm{\Σ}}}{w}_k^i;$

(4) further optimal estimation, the posterior probability of establishing the t moment is known, and then tracking parameter P is expressed as: $p_{\mathrm{Xt}}^{\mathrm{opt}}=\underset{i=1}{\overset{N}{\mathrm{\Σ}}}{w}^i{p}_X^i,$ $p_{\mathrm{Xt}}^{\mathrm{opt}}\mathrm{Yt}=\underset{i=1}{\overset{N}{\mathrm{\Σ}}}{w}^i{p}_X^i,$ Afterwards, can make t=t+1 again, return resampling then.

Substantive distinguishing features of the present invention is, at first the omni-directional visual image is carried out pre-service, seeking unique point with optical flow method in image then mates, obtain the sparse optical flow field, at last by the parameter of particle filter predicted motion target in subsequent frame, set up the coupling matrix between consecutive frame, analyze the coupling matrix and judge the moving target state, thus pursuit movement target effectively.Compare with existing method, the method that adopts the present invention to propose can reduce operand significantly and improve accuracy rate.

Description of drawings

Fig. 1 general flow chart that is used for the compound recognition and tracking device of light stream-particle of omni-directional visual environment of the present invention.

Embodiment

Referring to Fig. 1, dynamic object identification and localization method based on omni-directional visual of the present invention comprise the following steps:

Step 1: obtain the omni-directional visual sequence image, image is carried out pre-service, target and background is separated, prepare for follow-up optical flow field calculates.By gauss low frequency filter image is carried out smoothly in advance, carry out the gradient sharpening then, find the movement edge of image object,, carry out Threshold Segmentation in order to cut apart target object and background.At first directly select to determine a threshold value by histogram, take dynamically to adjust threshold value for sequence image, allow each gray values of pixel points of image and this threshold value compare then, if greater than this threshold value, just this gray values of pixel points is changed to 255 (expression backgrounds), otherwise this gray values of pixel points is changed to 0 (object), so just moving target and background has been made a distinction.Just become bianry image through the image of Threshold Segmentation, had only 0 and 255 two kind of gray-scale value.

Step 2: carry out local area search with optical flow method, carry out the coupling of unique point between the image consecutive frame.

For sequence image, the consecutive frame time interval is very little, and spatial point moves little in adjacent two two field pictures, and front and back frame object space correlativity is bigger.

If (x y) is continuous function about variable x, y to the moving image function f.If during moment t, (x, the gray-scale value of y) locating are f to 1 a=on the image _t(x, y), when moment t+ Δ t, this point moves to reposition, and its position on image becomes (x+ Δ x, y+ Δ y), and gray-scale value is designated as f _{T+ Δ t}(x+ Δ x, y+ Δ y), the purpose of coupling is exactly the corresponding point of seeking a, allows it and f _t(x, y) equate, promptly

f _t(x，y)＝f _t+Δt(x+Δx，y+Δy)(1)

And make an a=(x, y) in the neighborhood of the m * n that sets, least mean-square error MSE (Δ x, Δ y) minimum.

$\mathrm{MSE}(\mathrm{\Δx},\mathrm{\Δy})=\frac{1}{\mathrm{MN}}\underset{m=1}{\overset{M}{\mathrm{\Σ}}}\underset{n=1}{\overset{N}{\mathrm{\Σ}}}{[f_t(x,y)-f_{t+\mathrm{\Δt}}(x+\mathrm{\Δx},y+\mathrm{\Δy})]}^2---\left(2\right)$

Can make MSE (Δ x, Δ y) minimum be Optimum Matching point opt=(Δ x, Δ y).

Make that MSE (Δ x, Δ y) is zero to the first order derivative of (Δ x, Δ y):

$\frac{\∂\mathrm{MSE}(\mathrm{\Δx},\mathrm{\Δy})}{\∂(\mathrm{\Δx},\mathrm{\Δy})}|(\mathrm{\Δx},\mathrm{\Δy})=\mathrm{opt}=\left(\mathrm{0,0}\right)---\left(3\right)$

By (2), can get

$\frac{\∂\mathrm{MSE}(\mathrm{\Δx},\mathrm{\Δy})}{\∂(\mathrm{\Δx},\mathrm{\Δy})}=-\frac{2}{\mathrm{MN}}\underset{m=1}{\overset{M}{\mathrm{\Σ}}}\underset{n=1}{\overset{N}{\mathrm{\Σ}}}[f_t(x,y)-f_{t+\mathrm{\Δt}}(x+\mathrm{\Δx},y+\mathrm{\Δy})]\·\left(\frac{{\∂f}_{t+\mathrm{\Δt}}}{\∂\mathrm{\Δx}}\frac{{\∂f}_{t+\mathrm{\Δt}}}{\∂\mathrm{\Δy}}\right)---\left(4\right)$

Launch with Taylor's formula:

$\frac{\∂\mathrm{MSE}\left(\text{\Δx,\Δy}\right)}{\∂(\mathrm{\Δx},\mathrm{\Δy})}=-\frac{2}{\mathrm{MN}}\underset{m=1}{\overset{M}{\mathrm{\Σ}}}\underset{n=1}{\overset{N}{\mathrm{\Σ}}}[f_t(x,y)-f_{t+\mathrm{\Δt}}(x,y)-(\frac{{\∂f}_{t+\mathrm{\Δt}}}{\∂\mathrm{\Δx}},\frac{{\∂f}_{t+\mathrm{\Δt}}}{\∂\mathrm{\Δy}})\·(\mathrm{\Δx},\mathrm{\Δy})]\·(\frac{{\∂f}_{t+\mathrm{\Δt}}}{\∂\mathrm{\Δx}},\frac{{\∂f}_{t+\mathrm{\Δt}}}{\∂\mathrm{\Δy}})---\left(5\right)$

Make f=f _t(x, y)-f _{T+ Δ t}(x, y)

$\▿f={[\frac{{\∂f}_{t+\mathrm{\Δt}}}{\∂\mathrm{\Δx}},\frac{{\∂f}_{t+\mathrm{\Δt}}}{\∂\mathrm{\Δy}}]}^T$ Be the gradient of pixel (Δ x, Δ y),

(5) but abbreviation get $\frac{\∂\mathrm{MSE}(\mathrm{\Δx},\mathrm{\Δy})}{\∂(\mathrm{\Δx},\mathrm{\Δy})}=-\frac{2}{\mathrm{MN}}\underset{m=1}{\overset{M}{\mathrm{\Σ}}}\underset{n=1}{\overset{N}{\mathrm{\Σ}}}[f-{\▿f}^T\·(\mathrm{\Δx},\mathrm{\Δy})]\·{\▿f}^T---\left(6\right)$

Again because $\▿f\·{\▿f}^T=1$

But the following formula abbreviation is:

$\frac{\mathrm{MN}}{2}\·{\left[\frac{\∂\mathrm{MSE}(\mathrm{\Δx},\mathrm{\Δy})}{\∂(\mathrm{\Δx},\mathrm{\Δy})}\right]}^T=\underset{m=1}{\overset{M}{\mathrm{\Σ}}}\underset{n=1}{\overset{N}{\mathrm{\Σ}}}{\▿f}^T\·(\mathrm{\Δx},\mathrm{\Δy})-\underset{m=1}{\overset{M}{\mathrm{\Σ}}}\underset{n=1}{\overset{N}{\mathrm{\Σ}}}f---\left(7\right)$

Order $U=\underset{m=1}{\overset{M}{\mathrm{\Σ}}}\underset{n=1}{\overset{N}{\mathrm{\Σ}}}{\▿f}^T,$ $V=\underset{m=1}{\overset{M}{\mathrm{\Σ}}}\underset{n=1}{\overset{N}{\mathrm{\Σ}}}f$

Can get Optimum Matching point opt=(Δ x, Δ y)=U ^-1V

Mate by in image, seeking unique point, detect the moving target of image sequence.

Step 3: utilize the validity feature of target, by particle filter algorithm target state is estimated, the parameter of predicted motion target in subsequent frame finished tracing process.

At first carry out the particle initialization, the initial target piece is positioned, obtain particle w _kTemplate, as manual initialization, auto-initiation or the like obtains target w afterwards again _kOriginal state, i.e. the state P of its initial time of occurring ^Init=(P ^Init _x, P ^Init _y), getting population is N (each particle is represented a kind of possible motion state), establishes the initial weight w of particle _i=1, then have N possible motion state parameters P ⁱ=(P ⁱ _X, P ⁱ _Y)

(i ∈ 1 ... N), P wherein ⁱCan select p ^InitPoint in the certain limit on every side.

Capable then particle resampling process is eliminated the less particle of weights, keeps the bigger particle of weights.

At last, preset iterations, change the iterative process of particle filter algorithm over to.From each later frame of second frame, each particle carried out system state shifts and systematic observation, calculate the weights of particle, and all particles are weighted estimated value with the export target state.

State transitions: to particle N ⁱ, have

P ⁱ _Xt＝A ₁P ⁱ _Xt-1+B ₁w ⁱ _t-1(8)

P ⁱ _Xt＝A ₂P ⁱX _t-1+B ₂w ⁱ _t-1(9)

Wherein, A ₁, A ₂, B ₁, B ₂Be constant, general A gets 1, and B is that particle is propagated radius (in the system state transfer process, the scope that particle institute can propagate), and w is [1,1] interior random number.

Systematic observation: after each particle state shifted, MADi of the new coordinate Calculation of promptly available correspondence established probability density function and is $p\left(z_k\right|x_k^i)=\exp \{-\frac{1}{{2\mathrm{\σ}}^2}{\mathrm{MAD}}^i\}---\left(10\right)$

Wherein, σ is a constant, and MAD is a minimum average B configuration absolute difference function.

$\mathrm{MAD}(i,j)=\frac{1}{M\×N}\underset{m=1}{\overset{M}{\mathrm{\Σ}}}\underset{n=1}{\overset{N}{\mathrm{\Σ}}}|T(m,n)-F(m+i,n+j)|$

Then the weights of each particle are: $w_k^i=w_{k-1}^{i}p\left(z_k\right|x_k^i)---\left(11\right)$

Normalization: $w_k^i=w_k^i/\underset{i=1}{\overset{N}{\mathrm{\Σ}}}{w}_k^i---\left(12\right)$

Further optimal estimation, the posterior probability of establishing the t moment is known, and then tracking parameter P can be expressed as:

$p_{\mathrm{Xt}}^{\mathrm{opt}}=\underset{i=1}{\overset{N}{\mathrm{\Σ}}}{w}^i{p}_X^i,$ $p_{\mathrm{Xt}}^{\mathrm{opt}}\mathrm{Yt}=\underset{i=1}{\overset{N}{\mathrm{\Σ}}}{w}^i{p}_X^i,---\left(13\right)$

Afterwards, can make t=t+1 again, return resampling then.

Claims (3)

1. dynamic object identification and localization method based on an omni-directional visual comprise the following steps:

Step 1: obtain the omni-directional visual sequence image, this sequence image is carried out pre-service, obtain the bianry image that object and background are made a distinction;

Step 2: carry out local area search with optical flow method, carry out the coupling of unique point between the image consecutive frame, detect the moving target of image sequence;

Step 3: target state is estimated that the motion state parameters of predicted motion target in subsequent frame finished tracing process by particle filter algorithm.

2. dynamic object identification and localization method based on omni-directional visual according to claim 1, step 2 is wherein carried out according to following method, establishes moving image function f (x, y) be continuous function about variable x, y, during moment t, (x, the gray-scale value of y) locating are f to 1 a=on the image _t(x, y), when moment t+ Δ t, this point moves to reposition, and its position on image becomes (x+ Δ x, y+ Δ y), and gray-scale value is designated as f _{T+ Δ t}(x+ Δ x, y+ Δ y), the purpose of coupling is exactly the corresponding point of seeking a, makes f _t(x, y)=f _{T+ Δ t}(x+ Δ x, y+ Δ y), and make an a=(x, y) in the neighborhood of the M * N that sets, least mean-square error MSE (Δ x, Δ y) minimum, can make MSE (Δ x, Δ y) minimum be Optimum Matching point opt=(Δ x, Δ y),

Make f=f _t(x, y)-f _{T+ Δ t}(x, y),

Be the gradient of pixel (Δ x, Δ y), then, $\frac{\mathrm{MN}}{2}\·{\left[\frac{\∂\mathrm{MSE}(\mathrm{\Δx},\mathrm{\Δy})}{\∂(\mathrm{\Δx},\mathrm{\Δy})}\right]}^T=\underset{m=1}{\overset{M}{\mathrm{\Σ}}}\underset{n=1}{\overset{N}{\mathrm{\Σ}}}\▿f^T\·(\mathrm{\Δx},\mathrm{\Δy})-\underset{m=1}{\overset{M}{\mathrm{\Σ}}}\underset{n=1}{\overset{N}{\mathrm{\Σ}}}f$

Order

Try to achieve Optimum Matching point opt=(Δ x, Δ y)=U ^-1V mates by seek unique point in image, detects the moving target of image sequence.

3. dynamic object identification and localization method based on omni-directional visual according to claim 1, step 3 is wherein carried out according to following method:

(1) according to the result of step 2, initial target is positioned, and obtains the initial motion parameter of target:

P ^Init=(P ^Init _x, P ^Init _y), establish each particle and represent a kind of possible motion state, getting population is N, the initial weight w of particle _i=1, then have N possible motion state parameters P ⁱ=(P ⁱ _X, P ⁱ _Y), (i ∈ 1 ... N);

(2) carry out particle resampling process, eliminate the less particle of weights, keep the bigger particle of weights;

(3) change the iterative process of particle filter algorithm over to: from each later frame of second frame, each particle is carried out system state to be shifted and systematic observation, calculate the weights of particle, and all particles are weighted estimated value with the export target state, finish tracing process.

Images