CN106534614A - Rapid movement compensation method of moving target detection under mobile camera - Google Patents

Abstract

The invention discloses a rapid movement compensation method of moving target detection under a mobile camera. The method comprises steps of firstly, carrying out coarse matching on adjacent images by use of a phase correlation method so as to obtain offset amounts of the adjacent images; carrying out primary compensation on the images; then, carrying out feature point matching on two images by use of a KLT tracking method and solving the homography between the images, thereby achieving secondary compensation for image background movement; and finally, using a movement history image MHI method of multiframe frame difference to carry out segmentation on a target based on region features so as to obtain a movement target. According to the invention, the background movement is compensated twice by use of the rapid movement compensation method, so accuracy of the moving target detection method under the mobile camera can be ensured.

Description

The quick motion compensation process of moving object detection under mobile camera

Technical field

The invention belongs to image detection and process field, the quick fortune of moving object detection under particularly a kind of mobile camera Dynamic compensation method.

Background technology

In Search tracking system, moving target quickly and accurately can be partitioned into from sequence image by the system of usually requiring that Come, to realize the detection of moving target and tracking, preliminary letter is provided for technologies such as follow-up track association, target identifications Breath.In video monitoring system, scene is typically static, and the detection of moving target quickly and easily can be realized, but It is very high to hardware platform requirements.For the feelings of the camera motions such as Vehicular automatic driving generally, hand-supported camera Condition, as camera motion situation not can determine that, it is therefore desirable to compensate to camera motion or demarcate.

Most basic moving target detecting method such as frame difference method, multiple frame cumulation calculus of finite differences, optical flow method, block motion analysis method etc., For the moving object detection under simple static background has extraordinary effect.But the moving object detection place of reality is often In outdoor, background is often affected by illumination variation and sleety weather etc., and the motion of target itself also results in which in two dimension The change of geometric shape on image, puts forward higher requirement to target detection under camera motion state, these factors All so that the detection of moving target becomes complex.

Traditional motion compensation process is to carry out feature point detection to adjacent two field pictures using the method for Feature Points Matching, so Matching double points are obtained by the method for Feature Points Matching afterwards, so as to obtain the transformation matrix between adjacent two frame, to be transported Dynamic compensation.This method amount of calculation is too big, is not suitable for the higher occasion of real-time, and is also easy to produce more mistake when matching It is overdue right so that estimation is not accurate enough, and then have impact on detectivity.

The content of the invention

It is an object of the invention to provide moving under a kind of mobile camera based on global motion compensation and motion history image The quick motion compensation process of target detection, can be effectively improved the not high problem of conventional motion compensation method real-time.

The technical solution for realizing the object of the invention is：

Under a kind of mobile camera, the quick motion compensation process of moving object detection, comprises the following steps：

Step 1, using phase correlation method, by ask for that phase place correlation function peak obtains two width adjacent images two Dimension side-play amount；

Two width adjacent images, according to the side-play amount of two width adjacent images, are carried out single compensation by step 2；

Step 3, carries out characteristic point using the KLT matching process of classical feature based to the adjacent image after single compensation Matching；

Step 4, the corresponding homography matrix of two width adjacent image of hierarchical solving, then carries out secondary benefit to image motion background Repay；

Step 5, to the adjacent image after second compensation using based on multiframe frame difference motion history image tracking carry out Segmentation obtains final motion detection target.

Compared with prior art, its remarkable result is the present invention：

(1) present invention is compensated to background motion at twice using FMC methods, it is ensured that moved under mobile camera The accuracy of object detection method；

(2) quick motion compensation process of the invention is based on global motion compensation and motion history image, i.e., regard ignoring In the case of difference, it is believed that homography matrix constraint is met between adjacent image, in two steps global motion is compensated, often The algorithm complex of secondary compensation institute foundation is little.

(3) present invention improves the processing speed and systematic function of moving target detecting method under mobile camera.

Description of the drawings

Fig. 1 is the quick motion compensation process flow chart of moving object detection under mobile camera of the invention.

+ 1 frame of adjacent kth frame and kth in video sequence in Fig. 2 (a), Fig. 2 (b) respectively embodiment of the present invention Image.

Fig. 3 is phase place correlation function image in embodiment.

Fig. 4 is image of the kth frame image after motion compensation twice in the embodiment of the present invention.

Fig. 5 is the binary image after motion history image segmentation in the embodiment of the present invention.

Fig. 6 is the movement destination image that detected according to Fig. 5 in the embodiment of the present invention.

Specific embodiment

With reference to Fig. 1, the quick motion compensation process of moving object detection under a kind of mobile camera of the present invention, step is such as Under：

Step 1：Using phase correlation method, by ask for that phase place correlation function peak obtains two width adjacent images two Dimension side-play amount；Specially：

The video sequence obtained by the motion cameras such as vehicle-mounted, handheld camera, generally there are larger change in displacement, phase place Correlation method can be effectively estimated to translation transformation；F (x, y) is two dimensional image signal, and its Fourier transform pairs is(u, v) is the translational movement of adjacent two interframe, according to the time domain specification of Fourier transformationCarry out estimating two dimension using the phase information in image cross-correlation power spectrum Translational movement；I_k(x, y) and I_k+1(x, y) is adjacent two field pictures, ignores the rotation transformation between adjacent two frame, therefore only There is translation transformation, the translational movement of two interframe is (u, v), its Fourier transformation is respectively F_k(ξ, η) and F_k+1(ξ, η), then The crosspower spectrum of two width images is

For F_kThe conjugation of (ξ, η)；

Then phase place correlation function is obtained by inverse Fourier transform is

Cps (ξ, η)=δ (x-u, y-v)

Phase place correlation function is a unit impulse function at translational movement (u, v) place, and it is not 0 at (u, v) place, and Other positions are all 0, and the side-play amount of two width images is obtained by asking for peak.

Two width adjacent images, according to the side-play amount of two width adjacent images, are carried out single compensation, the figure after compensation by step 2 There is the radiation conversion of lower degree as between, can accurately be estimated using LK light streams.

Step 3, carries out characteristic point using the KLT matching process of classical feature based to the adjacent image after single compensation Matching；Specially：

Image sequence after single compensation largely eliminates the motion of background, meets KLT methods and requires image The less condition of movement velocity, using KLT (Kanade-Lucas-Tomasi) matching process of classical feature based, leads to Cross a side-play amount to describe the change of the pixel in characteristic window W comprising feature texture information, for front piece image Any feature point X, on adjacent image, pixel meets J (X)=I (X-d), X=[x, y]^T, displacement D=[d_x,d_y]^T, I (X) and J (X) represent respectively before and after two field pictures, d_x、d_yRespectively adjacent image pixels point exists Displacement on abscissa, ordinate direction, weighs on two adjacent images matching degree between pixel with residual error κ：

ω (X) is weighting function；

After single compensation, displacement d very littles using Taylor expansion, are removed high-order term, residual error κ are made to d derivations Obtaining the minimizing condition of residual error is

Zd=e

Wherein

Z is 2 × 2 matrixes, and e is 2 × 1 matrixes.

Optimum Matching point range is worth to by asking for the minimum of residual error κ again.

Step 4：Two width of hierarchical solving is adopted on the basis of RANSAC (Random Sample Consensus) method The corresponding homography matrix of image, then carries out second compensation to image background motion；

Hierarchical solving method is a successively iterative process, and its flow process is as follows：

In step 4-1, initialization adjacent image, pixel is whole angle points；

Step 4-2, the homography matrix that whole angle points are calculated by RANSAC methods；

The homography matrix residual error of the whole angle points of step 4-3, traversal, is set to intra-office point if the first threshold less than setting, Point not in the know is set to otherwise；

If the number of step 4-4, the intra-office for filtering out point be more than 8, prove that homography matrix has solution, then repeat step 4-1～ Step 4-3 is till the residual error of all intra-office points is less than first threshold.

Step 5：To motion history image (MHI) method of the imagery exploitation after second compensation based on multiframe frame difference, profit With the correlation spatially of objective contour in consecutive image, in the same time consecutive image is not added by every two field picture is corresponding Weigh superposition and form motion history image, then segmentation is carried out to it and obtain final motion detection target.The core of MHI methods Being image is constantly updated to MHI after system timestamps conversion, propulsion over time, and present image is corresponding Profile always has the gray value of maximum, and impact of the past profile in current MHI will be less and less, works as past frame When exceeding certain threshold value with the interval of present frame, its impact will be cleared；Its step is as follows：

Step 5-1, adjacent differential computing is done to the image after second compensation, it is poor by D (t)=| I (t)-I (t ± △) | Partial image, wherein, minus sign represents previous frame image, and plus sige represents latter two field picture, and △ represents that difference is spaced, I (t) and I (t ± △) is the image after having been compensated for, and D (t) is adjacent image difference value；

The gray value H of previous frame image_F(x, y, t) is expressed as:

W is time stamp (attenuation), from present frame more away from image, its impacts will be less and less, typically can use D=255/L, L are MHI length, and t is the current kinetic image moment, and D (x, y, t) is adjacent for the current kinetic image moment Gradation of image difference value, T are the Second Threshold of setting；

Step 5-2, using medium filtering removing the noise caused because of matching error or background small movements, choose Previous frame image gray scale H_F(t) and a later frame gradation of image H_BT the smaller value in () is used as final movement destination image gray scale Value, so as to realize the detection to target image：

M (t)=min (medfilt (H_F(t),medfilt(H_B(t)))。

Fig. 2 (a), (b) be one embodiment of the invention video sequence in+1 frame of adjacent kth frame and kth image, 3 is phase place correlation function image, and in figure, the peak of pulse is the side-play amount between adjacent image.

Fig. 4 is image of the kth frame image after motion compensation twice, as can be seen that contrasting+1 width image of kth in figure, Background motion is eliminated.

Fig. 5 be motion history image segmentation after binary image, as can be seen from the figure target more intactly by Extract.

Fig. 6 is the moving target detected according to Fig. 5, it can be seen that according to the binary image of Fig. 5, institute Outlining the movement destination image for coming has good effect.

Claims (5)

1. under a kind of mobile camera moving object detection quick motion compensation process, it is characterised in that including following Step：

Step 1, using phase correlation method, by ask for that phase place correlation function peak obtains two width adjacent images two Dimension side-play amount；

Two width adjacent images, according to the side-play amount of two width adjacent images, are carried out single compensation by step 2；

Step 3, carries out characteristic point using the KLT matching process of classical feature based to the adjacent image after single compensation Matching；

Step 4, the corresponding homography matrix of two width adjacent image of hierarchical solving, then carries out secondary benefit to image motion background Repay；

Step 5, to the adjacent image after second compensation using based on multiframe frame difference motion history image tracking carry out Segmentation obtains final motion detection target.

2. under mobile camera according to claim 1 moving object detection quick motion compensation process, which is special Levy and be, phase correlation method described in step 1 is first to two width adjacent image I_k(x, y) and I_k+1(x, y) asks for cross-power Spectrum：

$cps(\mathrm{\ξ},\mathrm{\η})=\frac{F_{k+1}(\mathrm{\ξ},\mathrm{\η})F_k^{*}(\mathrm{\ξ},\mathrm{\η})}{\left|F_{k+1}(\mathrm{\ξ},\mathrm{\η})F_k^{*}(\mathrm{\ξ},\mathrm{\η})\right|}=e^{-j2\mathrm{\π}(\mathrm{\ξ}u+\mathrm{\η}v)}$

(u, v) is the side-play amount of adjacent two interframe, and the Fourier transformation of two width adjacent images is respectively F_k(ξ, η) and F_k+1(ξ, η),For F_kThe conjugation of (ξ, η)；

Then obtaining phase place correlation function by inverse Fourier transform is

Cps (ξ, η)=δ (x-u, y-v)；

Phase place correlation function is a unit impulse function at translational movement (u, v) place, and it is not 0 at (u, v) place, and Other positions are all 0, and the side-play amount of two width images is obtained by asking for peak.

3. under mobile camera according to claim 1 moving object detection quick motion compensation process, which is special Levy and be：Spy is carried out using the KLT matching process of classical feature based to the adjacent image after single compensation in step 3 Point matching is levied, specially：

Using the KLT matching process of classical feature based, described comprising feature texture information by a side-play amount Pixel change in characteristic window W, for any feature point X of front piece image, on adjacent image, pixel is full Sufficient J (X)=I (X-d), X=[x, y]^T, displacement d=[d_x,d_y]^T, I (X) and J (X) represent respectively before and after two frames Image, d_x、d_yRespectively displacement of the adjacent image pixels point on abscissa, ordinate direction, is weighed with residual error κ Matching degree between pixel on two adjacent images of amount：

$\mathrm{\κ}=\underset{W}{\Σ}\[J(X+\frac{d}{2})-I(X-\frac{d}{2}){\]}^2\mathrm{\ω}\left(X\right)$

ω (X) is weighting function.

After single compensation, displacement d very littles, residual error κ are caused to d derivationsObtain residual error minimum condition

Zd=e

Wherein

$Z=\underset{W}{\Σ}g\left(X\right)g^T\left(X\right)\mathrm{\ω}\left(X\right)$

$e=\underset{W}{\Σ}\[I\left(X\right)-J\left(X\right)\]g\left(X\right)\mathrm{\ω}\left(X\right)$

$g\left(X\right)={\[\frac{\∂}{\∂x}\left(\frac{I+J}{2}\right)\frac{\∂}{\∂y}\left(\frac{I+J}{2}\right)\]}^T$

Z is 2 × 2 matrixes, and e is 2 × 1 matrixes；

Optimum Matching point range is worth to finally by the minimum of residual error κ is asked for.

4. under a kind of mobile camera according to claim 1 moving object detection FMC methods, its feature exists In：Hierarchical solving method described in step 4 is the process of successively iteration, specially：

In step 4-1, initialization adjacent image, pixel is whole angle points；

Step 4-2, the homography matrix that whole angle points are calculated by RANSAC methods；

The homography matrix residual error of the whole angle points of step 4-3, traversal, is set to intra-office point if the first threshold less than setting, Point not in the know is set to otherwise；

If the number of step 4-4, the intra-office for filtering out point be more than 8, prove that homography matrix has solution, then repeat step 4-1～ Step 4-3 is till the residual error of all intra-office points is less than first threshold.

5. under a kind of mobile camera according to claim 1 moving object detection FMC methods, its feature exists In：Described in step 5 to the adjacent image after second compensation using based on multiframe frame difference motion history image tracking Carry out segmentation and obtain final motion detection target, comprise the following steps：

Step 5-1, adjacent differential computing is done to the image after second compensation, it is poor by D (t)=| I (t)-I (t ± △) | Partial image, wherein, minus sign represents previous frame image, and plus sige represents latter two field picture, and △ represents that difference is spaced, I (t) and I (t ± △) is the image after having been compensated for, and D (t) is adjacent image difference value；

The gray value H of previous frame image_F(x, y, t) is expressed as:

$H_F(x,y,t)=\left\{\begin{array}{ccc}\max (0,H_F(x,y,t-1)-w)& if& D(x,y,t)<T\\ 255& if& D(x,y,t)\&GreaterEqual;T\end{array}\right.$

W is time stamp, and t is the current kinetic image moment, and D (x, y, t) is current kinetic image moment adjacent image gray scale difference Score value, T are the Second Threshold of setting；

Step 5-2, using medium filtering removing the noise caused because of matching error or background small movements, choose Previous frame image gray scale H_F(t) and a later frame gradation of image H_BT the smaller value in () is used as final movement destination image gray scale Value, so as to realize the detection to target image：

M (t)=min (medfilt (H_F(t),medfilt(H_B(t)))。