CN106887010A - Ground moving target detection method based on high-rise scene information - Google Patents

Abstract

The invention discloses a kind of Ground moving target detection method based on high-rise scene information, there is the technical problem of false-alarm for solving existing multi-target detection method.Technical scheme is to extract preliminary testing result using frame difference method first；The light stream vector of each point is calculated again, the position in its next frame is judged after the target of present frame is superimposed with its light stream vector, realize the association to target, eliminate a part of false-alarm；Finally judge motor point and background dot using the high layer information fundamental matrix F of scene, eliminate substantial amounts of false-alarm.

Description

Ground moving target detection method based on high-rise scene information

Technical field

The present invention relates to a kind of multi-target detection method, more particularly to a kind of ground motion based on high-rise scene information Object detection method.

Background technology

Multi-target detection is a challenging task in computer vision field.The most base of traditional motion detection Realized in frame difference.Yet with the actual three-dimensional scenic of scene, frame difference has parallax, causes a large amount of false-alarms.Document " Goyal H.Frame Differencing with Simulink model for Moving Object Detection[J] .International Journal of Advanced Research in Computer Engineering& Technology, 2013,2 (1) " discloses a kind of multi-target detection method (frame differential method).The method is assumed to be carried on the back in scene Scape is level, and by difference, the part for being higher by ground level will detect that.In affine transformation not being accounted for due to the method Parallax caused by the three-dimensional structure of scene, therefore with substantial amounts of false-alarm, do not applied to for actual three-dimensional scenic and comprising few Amount noise.

The content of the invention

In order to overcome the shortcomings of that existing multi-target detection method has false-alarm, the present invention provides a kind of based on high-rise scene letter The Ground moving target detection method of breath.The method extracts preliminary testing result using frame difference method first；Each point is calculated again Light stream vector, judge the position in its next frame after the target of present frame is superimposed with its light stream vector, realize to target Association, eliminates a part of false-alarm；Finally judge motor point and background dot using the high layer information fundamental matrix F of scene, remove Substantial amounts of false-alarm.

The technical solution adopted for the present invention to solve the technical problems is：A kind of ground motion based on high-rise scene information Object detection method, is characterized in comprising the following steps：

Step one, frame are poor.

For the scene of different height, different image registration algorithms is used.For high-altitude shoot video sequence due to Its meet sparse optical flow three using Lucas-Kanade sparse optical flows it is assumed that therefore realize Image Feature Point Matching；For low The image that sky shoots is unsatisfactory for the assumed condition of light stream due to it, therefore uses sobel operator extraction image characteristic points.By sparse Light stream or sobel operators realize images match, finally estimate the affine transformation between two images using RANSAC, specific as follows：

In formula, C_pAnd C_nIt is the pixel coordinate of the characteristic point of former frame and next frame, C'_pAnd C'_nBe conversion after former frame and The pixel coordinate of next frame, A_k-1And A_k+1It is the affine transformation matrix of 2*3.The image difference of former frame and next frame affine transformation Difference present frame, obtains Preliminary detection result, as follows：

D_k=| | S_k-S'_k-1||∪||S_k-S'_k+1|| (2)

In formula, D_kRepresent image, S' after difference_k-1And S'_k+1It is the result after present frame and next frame affine transformation, S_kIt is Current frame image.Binaryzation finally is carried out to difference image, threshold value is set to 40.

Step 2, light stream association.

Light stream is estimated：Classical optical flow method is based primarily upon the hypothesis of brightness constancy, pixel small movements and Space Consistency. In continuous videos, it is assumed that the corresponding grey scale pixel value of object has not because motion changes：

I (x, y, t)=I (x+dx, y+dy, t+dt) (3)

In formula, x and y is transverse and longitudinal coordinate, and I is image intensity value.Above formula Taylor expansion has:

I_xdx+I_ydy+I_tDt=0 (4)

In formula, I_x I_y I_tThe gradient of correspondence direction is represented respectively.Be converted to vector form as follows：

In formula, u and v is respectively the light stream size on correspondence direction.Above formula is represented by：

Ad=b (6)

Solved using least square methodMinimum be worth to light stream vector d, it is as follows：

D=(A^TA)^-1A^Tb (7)

Prediction association：First, it is assumed that each point is (x in the coordinate of the frame of kth -1_k-1,y_k-1), the light stream according to step one is estimated Stratagem is omited, and obtains a light stream vector V, then target is as follows in the position prediction of next frame：

In formula,It is prediction coordinate, (V_x,V_y) it is light stream motion vector.

Secondly, all targets for being obtained to first frame difference, predict it in the position of next frame by light stream.To each target For, if it has certain destination matches in enough points and next frame, they are same target, and decision function is fixed Justice is as follows:

In formula,It is the target of quadratic difference detection, S^kIt is the state equation of point, next calculates the confidence of object matching Degree：

In formula, α is belonging to the sum of the point of target, and two target association probability are：

α_ρ=α/β (11)

In formula, β is institute's quantity summation a little in target, and the probability of two target associations of receiving is set to ε=0.8, if Two targetsWithInterrelated, then the incidence relation between them is defined as:

To each target, a relation integration A={ A is defined_m,...,A_n, A in formula_mRepresentTo each association For set, only when the number of the relation integration of target is more than given threshold, using it as candidate target.

Step 3, the motion detection based on high layer information.

Using the characteristic point of sobel operator extraction images, the matching to image characteristic point is completed according to beeline.X= (x, y) and x'=(x', y') are a pair of match points in image, be converted into it is single answer vector X=[x, y, 1] and X'=[x', y',1]^T, they meet：

X'^TFX=0 (13)

In formula, F is the fundamental matrix for 3*3.Obtained by solving system of linear equations substantially using 8 algorithms of normalization Matrix F.The characteristic point matched in actual calculating process will not strictly meet above formula, therefore, corrected using Sampson, lead to Exterior point in the correct amount judgement for calculate matching is crossed, Samposon confidence levels K is defined as follows：

K=X'^TFX/M (14)

In formula, (FX)₁=f₁₁x+f₁₂y+f₁₃, (x, y) is the coordinate of the pixel of X.Analogy determines (FX)₂,(F^TX')₁, (F^TX')₂, so that it is determined that the Sampson confidence levels of each point.

Outer dot matrixH and W are respectively image height and width, and it is defined as follows：

The interior exterior point ratio calculation of each candidate target is as follows：

In formula,It is a candidate target,Be candidate quantity summation a little, moving target decision function M It is defined as follows：

In formula, η is the probability threshold value of exterior point, only when the exterior point ratio of candidate is more than η, can just determine that it is one The target of motion.

The beneficial effects of the invention are as follows：The method extracts preliminary testing result using frame difference method first；Calculate each again The light stream vector of point, judges the position in its next frame after the target of present frame is superimposed with its light stream vector, realize to target Association, eliminate a part of false-alarm；Finally judge motor point and background dot using the high layer information fundamental matrix F of scene, go Except substantial amounts of false-alarm.

The present invention is elaborated with reference to specific embodiment.

Specific embodiment

Ground moving target detection method of the present invention based on high-rise scene information is comprised the following steps that：

1st, frame is poor.

For the scene of different height, different image registration algorithms is used.For high-altitude shoot video sequence due to Its meet sparse optical flow three using Lucas-Kanade sparse optical flows it is assumed that therefore realize Image Feature Point Matching；For low The image that sky shoots is unsatisfactory for the assumed condition of light stream due to it, therefore uses sobel operator extraction image characteristic points.By sparse Light stream or sobel operators realize images match, finally estimate the affine transformation between two images using RANSAC, specific as follows：

In formula, C_pAnd C_nIt is the pixel coordinate of the characteristic point of former frame and next frame, C'_pAnd C'_nIt is the pixel seat after conversion Mark, wherein A_k-1And A_k+1It is the affine transformation matrix of 2*3.The image difference difference of former frame and next frame affine transformation is current Frame, obtains Preliminary detection result, as follows：

D_k=| | S_k-S'_k-1||∪||S_k-S'_k+1|| (20)

In formula, D_kRepresent image, S' after difference_k-1And S'_k+1It is the result after present frame and next frame affine transformation, S_kIt is Current frame image.Binaryzation finally is carried out to difference image, threshold value is set to 40.

2nd, light stream association.

Light stream association is mainly two parts：Light stream estimates that prediction is associated.

1) light stream is estimated：Classical optical flow method is based primarily upon the hypothesis of brightness constancy, pixel small movements and Space Consistency. In continuous videos, it is assumed that the corresponding grey scale pixel value of object has not because motion changes：

I (x, y, t)=I (x+dx, y+dy, t+dt) (21)

In formula, x and y is transverse and longitudinal coordinate, and I is image intensity value.Above formula Taylor expansion has:

I_xdx+I_ydy+I_tDt=0 (22)

In formula, I_x I_y I_tThe gradient of correspondence direction is represented respectively.Be converted to vector form as follows：

In formula, u and v is respectively the light stream size on correspondence direction.Above formula is represented by：

Ad=b (24)

Solved using least square methodMinimum be worth to light stream vector d, it is as follows：

D=(A^TA)^-1A^Tb (25)

2) prediction association：First, it is assumed that each point is (x in the coordinate of the frame of kth -1_k-1,y_k-1), according to the light stream of step one Estimate strategy, obtain a light stream vector V, then target is as follows in the position prediction of next frame：

In formula,It is prediction coordinate, (V_x,V_y) it is light stream motion vector.

Secondly, all targets for being obtained to first frame difference, predict it in the position of next frame by light stream.To each target For, if it has certain destination matches in enough points and next frame, they are same target, and decision function is fixed Justice is as follows:

In formula,It is the target of quadratic difference detection, S^kIt is the state equation of point, next calculates the confidence of object matching Degree：

In formula, α is belonging to the sum of the point of target, and two target association probability are：

α_ρ=α/β (29)

In formula, β is institute's quantity summation a little in target, and the probability of two target associations of receiving is set to ε=0.8, if Two targetsWithInterrelated, then the incidence relation between them is defined as:

To each target, a relation integration A={ A is defined_m,...,A_n, A in formula_mRepresentTo each association For set, only when the number of the relation integration of target is more than given threshold, using it as candidate target.

3rd, the motion detection based on high layer information.

Using the characteristic point of sobel operator extraction images, the matching to image characteristic point is completed according to beeline.X= (x, y) and x'=(x', y') are a pair of match points in image, be converted into it is single answer vector X=[x, y, 1] and X'=[x', y',1]^T, they meet：

X'^TFX=0 (31)

In formula, F is the fundamental matrix for 3*3.Obtained by solving system of linear equations substantially using 8 algorithms of normalization Matrix F.The characteristic point matched in actual calculating process will not strictly meet above formula, therefore, corrected using Sampson, lead to Exterior point in the correct amount judgement for calculate matching is crossed, Samposon confidence levels K is defined as follows：

K=X'^TFX/M (32)

In formula, (FX)₁=f₁₁x+f₁₂y+f₁₃, (x, y) is the coordinate of the pixel of X.Analogy determines (FX)₂,(F^TX')₁, (F^TX')₂, so that it is determined that the Sampson confidence levels of each point,

Outer dot matrixH and W are respectively image height and width, and it is defined as follows：

The interior exterior point ratio calculation of each candidate target is as follows：

In formula,It is a candidate target,Be candidate quantity summation a little, moving target decision function M It is defined as follows：

In formula, η is the probability threshold value of exterior point, only when the exterior point ratio of candidate is more than η, can just determine that it is one The target of motion.

Claims (1)

1. a kind of Ground moving target detection method based on high-rise scene information, it is characterised in that comprise the following steps：

Step one, frame are poor；

For the scene of different height, different image registration algorithms is used；Video sequence for high-altitude shooting is full due to it Three of sufficient sparse optical flow using Lucas-Kanade sparse optical flows it is assumed that therefore realize Image Feature Point Matching；Clapped for low latitude The image taken the photograph is unsatisfactory for the assumed condition of light stream due to it, therefore uses sobel operator extraction image characteristic points；By sparse optical flow Or sobel operators realize images match, the affine transformation between two images is finally estimated using RANSAC, it is specific as follows：

$\left\{\begin{array}{c}{C}_p^{\′}=A_{k-1}*C_p\\ C_n^{\′}=A_{k+1}*C_n\end{array}\right.---\left(1\right)$

In formula, C_pAnd C_nIt is the pixel coordinate of the characteristic point of former frame and next frame, C'_pAnd C'_nIt is former frame and next after conversion The pixel coordinate of frame, A_k-1And A_k+1It is the affine transformation matrix of 2*3；The image difference difference of former frame and next frame affine transformation Present frame, obtains Preliminary detection result, as follows：

D_k=| | S_k-S'_k-1||∪||S_k-S'_k+1|| (2)

In formula, D_kRepresent image, S' after difference_k-1And S'_k+1It is the result after present frame and next frame affine transformation, S_kIt is current Two field picture；Binaryzation finally is carried out to difference image, threshold value is set to 40；

Step 2, light stream association；

Light stream is estimated：Classical optical flow method is based primarily upon the hypothesis of brightness constancy, pixel small movements and Space Consistency；Continuous In video, it is assumed that the corresponding grey scale pixel value of object has not because motion changes：

I (x, y, t)=I (x+dx, y+dy, t+dt) (3)

In formula, x and y is transverse and longitudinal coordinate, and I is image intensity value；Above formula Taylor expansion has:

I_xdx+I_ydy+I_tDt=0 (4)

In formula, I_xI_yI_tThe gradient of correspondence direction is represented respectively；Be converted to vector form as follows：

$\[\begin{array}{cc}{I}_x& I_y\end{array}\]\left[\begin{array}{c}u\\ v\end{array}\right]=-I_t---\left(5\right)$

In formula, u and v is respectively the light stream size on correspondence direction；Above formula is represented by：

Ad=b (6)

Solved using least square methodMinimum be worth to light stream vector d, it is as follows：

D=(A^TA)^-1A^Tb (7)

Prediction association：First, it is assumed that each point is (x in the coordinate of the frame of kth -1_k-1,y_k-1), plan is estimated in the light stream according to step one Slightly, a light stream vector V is obtained, then target is as follows in the position prediction of next frame：

$\left\{\begin{array}{c}{x}_k^e=x_{k-1}+V_x\\ y_k^e=y_{k-1}+V_y\end{array}\right.---\left(8\right)$

In formula,It is prediction coordinate, (V_x,V_y) it is light stream motion vector；

Secondly, all targets for being obtained to first frame difference, predict it in the position of next frame by light stream；To each target Speech, if it has certain destination matches in enough points and next frame, they are same target, decision function definition It is as follows:

$S^k(x,y)=\left\{\begin{array}{cc}1,& \begin{array}{cc}if& (x,y)\∈O_k^l\end{array};\\ 0,& others\end{array}\right.---\left(9\right)$

In formula,It is the target of quadratic difference detection, S^kIt is the state equation of point, next calculates the confidence level of object matching：

$\mathrm{\α}=\underset{(x,y)\∈O_k^l}{\Σ}{S}^k(x,y)---\left(10\right)$

In formula, α is belonging to the sum of the point of target, and two target association probability are：

α_ρ=α/β (11)

In formula, β is institute's quantity summation a little in target, and the probability of two target associations of receiving is set to ε=0.8, if two TargetWithInterrelated, then the incidence relation between them is defined as:

$A_{(O_{k-1}^l,O_k^l)}=\left\{\begin{array}{cc}1,& \begin{array}{cc}if& {\mathrm{\α}}_{\mathrm{\ρ}}>\mathrm{\ϵ}\end{array}\\ 0,& others\end{array}\right.---\left(12\right)$

To each target, a relation integration A={ A is defined_m,...,A_n, A in formula_mRepresentTo each relation integration For, only when the number of the relation integration of target is more than given threshold, using it as candidate target；

Step 3, the motion detection based on high layer information；

Using the characteristic point of sobel operator extraction images, the matching to image characteristic point is completed according to beeline；X=(x, y) It is a pair of match points in image with x'=(x', y'), is converted into and single answers vector X=[x, y, 1] and X'=[x', y', 1]^T, They meet：

X^'TFX=0 (13)

In formula, F is the fundamental matrix for 3*3；Using 8 algorithms of normalization fundamental matrix is obtained by solving system of linear equations F；The characteristic point matched in actual calculating process will not strictly meet above formula, therefore, corrected using Sampson, by meter The correct amount for calculating matching judges interior exterior point, and Samposon confidence levels K is defined as follows：

K=X'^TFX/M (14)

$M={\left(FX\right)}_1^2+{\left(FX\right)}_2^2+{\left(F^T{X}^{\′}\right)}_1^2+{\left(F^T{X}^{\′}\right)}_2^2---\left(15\right)$

In formula, (FX)₁=f₁₁x+f₁₂y+f₁₃, (x, y) is the coordinate of the pixel of X；Analogy determines (FX)₂,(F^TX')₁, (F^TX')₂, so that it is determined that the Sampson confidence levels of each point；Outer dot matrix Out_i,j,H and W points Not Wei image height and width, it is defined as follows：

${\mathrm{Out}}_{i,j}=\left\{\begin{array}{cc}1,& outliers\\ 0,& others\end{array}\right.---\left(16\right)$

The interior exterior point ratio calculation of each candidate target is as follows：

$\mathrm{\σ}=\underset{(i,j)\∈O_k^l}{\Σ}{\mathrm{Out}}_{i,j}/N_k^l---\left(17\right)$

In formula,It is a candidate target,Be candidate quantity summation a little, moving target decision function M definition It is as follows：

$M_{O_k^l}=\left\{\begin{array}{cc}1,& \mathrm{\σ}>\mathrm{\η}\\ 0,& others\end{array}\right.---\left(18\right)$

In formula, η is the probability threshold value of exterior point, only when the exterior point ratio of candidate is more than η, can just determine that it is a motion Target.