CN104616320A - Method for detecting vehicle in low-altitude aerial video based on gradient inhibition and epipolar constraint - Google Patents

Abstract

The invention belongs to the processing field of digital images and especially relates to a method for detecting vehicle in a low-altitude aerial video based on gradient inhibition and epipolar constraint. The vehicle detection method comprises the following steps of: (S1) registering frame-to-frame images to obtain registration result images; (S2) detecting initial moving objects to obtain the approximate region of the vehicle and the objective region falsely detected in the image background; (S3) acquiring the accurate region of the vehicle to obtain detection result images including vehicle pixels and residual parallax pixels; (S4) intact vehicle segmentation: firstly performing morphological operation to the detection result images, filtering the parallax pixels to obtain vehicle pixels, and then segmenting the intact vehicle by a gray projection method. According to the parallax elimination method integrated with gradient inhibition and epipolar constraint, the impact of strong parallax to vehicle detection can be well eliminated, so that the vehicle in the low-altitude aerial video can be quickly and effectively detected.

Description

Take photo by plane and to suppress based on gradient in video and the vehicle checking method of epipolar-line constraint in low latitude

Technical field

The invention belongs to digital image processing field, particularly take photo by plane and to suppress based on gradient in video and the vehicle checking method of epipolar-line constraint in a kind of low latitude.

Background technology

Low latitude by strong parallax effects video of taking photo by plane is different from the feature of general video, is in particular in the following aspects:

First, low latitude video of taking photo by plane is video under a kind of imaging platform moves, i.e. dynamic background video, imaging platform motion in three dimensions causes projecting to the motion of static background pixel on the plane of delineation and the motion of moving target pixel, the motion in the picture of static background pixel is kinetic by imaging platform, and moving target motion is in the picture then to be moved by imaging platform and the Motion-Joint of target self causes.And the motion of background and the self-movement of target will cause the global motion of entire image pixel, adopt the moving target detecting method under quiet imaging platform to cause the flase drop of entire image, serious interference is caused to moving object detection.Second, the low latitude size of moving target in video of taking photo by plane is less, the distance taken photo by plane in low latitude is at least tens meters, in the image of imaging device shooting, target occupies less region, and general Moving Objects from Surveillance Video often occupies very large region, and the resolution of target is also larger, pedestrian in such as Indoor Video video, therefore, the target resolution taken photo by plane in video in low latitude is much smaller, even minimum to tens pixels.And the too small impact being very easily subject to noise of target, target may be treated as noise and eliminate, and therefore increases the detection difficulty of moving target further.3rd, take photo by plane in low latitude in video, tall and big object (trees, building and mountain range etc.) is there is in the background of target travel, the size of these static background objects be can not ignore relative to image-forming range, larger visual angle change is there is between frame and frame in video, very strong parallax can be produced, after inter frame image registration and initial motion target detection, the structural information of these objects can be erroneously detected into target, especially the marginal information of object, further disturbed motion target detection, causes the inaccurate of moving object detection.

The present invention mainly solves to eliminate and to take photo by plane the accurate test problems of vehicle in video by the low latitude of parallax effects, and existing parallax removing method is divided three classes substantially: the first kind is plane-disparity constraint method; Equations of The Second Kind is relative position constraint method; 3rd class is other method.

The first kind, plane-disparity constraint method is (the concrete list of references: Irani M proposed by M.Irani first, Anandan P.Parallax geometry of pairs of points for 3d sceneanalysis [M] //Computer Vision-ECCV'96.Springer Berlin Heidelberg, 1996:17-30.), its main thought be by set up point-to-point between Rigid Constraints detect motion pixel, general needs suppose constant reference planes in video between different frame, but this hypothesis may be false, along with the change in location between the continuous changing frame of shooting area and frame is also very large, fixing reference planes are obviously improper.

Equations of The Second Kind, relative position constraint method is that a kind of geometrical constraint that utilizes is to carry out the determination methods of pixel motion, it generally obtains the constraint condition between pixel from multiple image, (the concrete list of references: Luong Q T of the epipolar-line constraint method based on two width images that ratio Luong proposes, Faugeras OD.The fundamental matrix:Theory, algorithms, and stability analysis [J] .International Journal of Computer Vision, 1996,17 (1): 43-75.).Three line constraint (concrete lists of references: Hartley R between the adjacent three width images that Hartley proposes, Zisserman A.Multiple view geometry in computer vision [M] .Cambridge university press, 2003.).And Sun Hao propose a kind of based on four visual angles look more limit restraint detect self-movement target (concrete list of references: Sun Hao. motion imaging platform close shot video frequency motion target detection technique research [D]. the National University of Defense Technology, 2011.), the special circumstances that epipolar-line constraint cannot detect self-movement target can be overcome, reach good effect, but these class methods also increase along with its computation complexity of increase of amount of images, and epipolar-line constraint method is that this class methods medium velocity is the fastest thereupon.Above two class methods mainly utilize the geometric relationship between image to retrain each pixel in image.

3rd class, other method, the gradient proposed as V.Reily suppresses method (concrete list of references: Reilly V, Idrees H, Shah M.Detection and tracking of large number of targetsin wide area surveillance [M] //Computer Vision – ECCV 2010.Springer BerlinHeidelberg, 2010:186-199.), parallax pixel is utilized mainly to appear at this feature of background edge, thus the suppression of the little parallax pixel of visual angle change can be eliminated, there is algorithm simple, speed is fast, the advantages such as easy realization.The people such as Y.Qian propose a kind of motor pattern analytical approach based on 4D space, Optic flow information is utilized to project in 4D space, be partitioned into different motor patterns, carry out judgement (the concrete list of references: Yu Q of self-movement target, Medioni G.Motion pattern interpretationand detection for tracking moving vehicles in airborne video [C] //ComputerVision and Pattern Recognition, 2009.CVPR 2009.IEEE Conference on.IEEE, 2009:2671-2678.).

Summary of the invention

Technical matters to be solved by this invention is, not enough for prior art, provides low latitude to take photo by plane to suppress based on gradient in video and the vehicle checking method of epipolar-line constraint, improves low latitude and to take photo by plane the detection speed of vehicle in video, accuracy and better robustness.

For solving the problems of the technologies described above, the technical solution adopted in the present invention is: taking photo by plane and to suppress based on gradient in video and the vehicle checking method of epipolar-line constraint in a kind of low latitude, comprises the following steps:

(S1) inter frame image registration: choose the two field picture that comprises vehicle to be detected as with reference to image in video is taken photo by plane in low latitude, a rear two field picture of this frame is as image subject to registration, detect the SURF feature of two width images respectively, then two width characteristics of image are mated, calculate the affine transformation parameter of image registration, image subject to registration is carried out inverse transformation according to the parameter calculated, obtains registration result image, by two width image unifications in same coordinate system;

(S2) initial motion target detection: utilize frame difference method to process to the two width images (that is: reference picture and registration result image) after registration, obtains being become order target area by flase drop in the approximate region of vehicle and image background;

(S3) precise area of vehicle is obtained: first utilize gradient to suppress method to eliminate part parallax pixel, then the parallax pixel adopting epipolar-line constraint method elimination gradient to suppress method not eliminated, obtains the testing result image comprising vehicle pixel and residue parallax pixel;

(S4) be partitioned into complete vehicle: first morphological operation is carried out to the testing result image in described step (S3), filtering parallax pixel, the vehicle pixel obtained, then utilize Gray Projection method to be partitioned into complete vehicle.

Compared with prior art, the beneficial effect adopting the present invention to obtain is: what the present invention proposed utilizes the parallax removing method that gradient suppresses and epipolar-line constraint combines, can be good at eliminating the impact of strong parallax on vehicle detection, and then detect the vehicle taken photo by plane in video in low latitude sooner and effectively.First gradient suppresses method to be a kind of flase drop that can object edge be suppressed very well to produce, especially effect time less by parallax effects is ideal, but gradient suppresses method well can not eliminate the impact of strong parallax, therefore have employed epipolar-line constraint method, in image after suppressing gradient, the pixel of flase drop carries out attention judgement, eliminate residue parallax pixel, obtain accurate object pixel, finally utilize the dividing method of Gray Projection to split complete vehicle.

Figure of description

Fig. 1 is method flow diagram of the present invention;

Fig. 2 is reference picture and image subject to registration;

Fig. 3 is inter frame image registration process flow diagram;

Fig. 4 is initial motion object detection results image;

Fig. 5 obtains result images after carrying out gradient suppression;

The schematic diagram of Fig. 6 epipolar-line constraint;

The result images that Fig. 7 obtains after carrying out epipolar-line constraint;

Fig. 8 vehicles segmentation result images;

Embodiment

Below, the invention will be further described with embodiment by reference to the accompanying drawings.

As shown in Figure 1, be method flow diagram of the present invention, the invention provides a kind of low latitude and take photo by plane and to suppress based on gradient in video and the vehicle checking method of epipolar-line constraint, specifically comprise step as follows:

(S1) inter frame image registration: as shown in Figure 2, the frame comprising and detect vehicle is chosen in video is taken photo by plane in low latitude, wherein the 55th frame is as reference image, 56th frame is as image subject to registration, the size of image is 480 × 640 pixels, detect SURF (the Speeded Up RobustFeatures) feature of two width images respectively, the feature point number wherein detected in reference picture is 1021, the feature point number detected in image subject to registration is 1104, then two width characteristics of image are mated, obtaining match point number is 759, according to the feature point number of these couplings, calculate the affine transformation parameter of image registration, this parameter is $P=\left[\begin{array}{ccc}0.9979& -0.0157& 0\\ 0.0157& 0.9979& 0\\ 2.03446& 4.6905& 1\end{array}\right],$ Carry out image inverse transformation, by two width image unifications in same coordinate system, the flow process of inter frame image registration as shown in Figure 3.

The present invention selects six parameter transformation model, i.e. affine Transform Models common in this area, and it generally can represent with the matrix of three six parameters in rank, matrix last be classified as 0,0,1, as matrix P, other six parameters calculate according to the coordinate of the unique point of match point.

Coordinate according to matching characteristic point just can estimate this matrix P, then just coordinate (concrete list of references: Zeng Wenfeng can be carried out, Japanese plum mountain, Wang Jiangan. based on the translation in the image registration of affine Transform Model, Rotation and Zoom [J]. infrared and laser engineering, 2001,30 (1): 18-20.) conversion, namely each coordinate treating registering images all converts according to the matrix P calculated, finally realize the conversion of whole image, process of image registration that Here it is.

(S2) carry out initial motion target detection, initial motion target detection is carried out to two width imagery exploitation frame difference methods after registration, obtain being become order target area by flase drop in the approximate region of vehicle and background.The principle that realizes of frame difference method is that the result images after frame difference is generally bianry image by poor for the two width images being used for frame difference, sets a threshold value and judges work difference result, the judgement being greater than this threshold value is moving target, be set to 1, the judgement being less than threshold value is background, is set to 0.As shown in the formula:

D(x,y)＝|I _q(x,y)-I _p(x,y)| (1)

$R(x,y)=\left\{\begin{array}{cc}1& D(x,y)>T\\ 0& D(x,y)\≤T\end{array}\right.---\left(2\right)$

Wherein, I _q(x, y) represents the gray-scale value of the image of q two field picture under this coordinate, I _p(x, y) represents the gray-scale value of the image of p two field picture under this coordinate, and (x, y) represents image coordinate, and span is the size of image; D (x, y) is the result images after frame difference, image after the judgement that R (x, y) is threshold value; T is threshold value.In this example, get threshold value T=10, the experimental result image obtained two width imagery exploitation frame difference methods after registration as shown in Figure 4.

(S3) precise area of vehicle is obtained.Consider that strong parallax is taken photo by plane on low latitude the impact of vehicle detection in video, vehicle is there is in initial motion object detection results figure, i.e. object pixel, target vehicle as shown in Figure 2, and the background information of vehicle is become by flase drop, especially the edge of tall and big object in background, i.e. parallax pixel, the background objects trees namely in video.

(S31) gradient is first utilized to suppress method to eliminate part parallax pixel, gradient suppress principle be ask original image gradient, by testing result image and gradient image poor, set certain threshold value, draw up edge's part parallax pixel, whole process can be expressed from the next:

I _r＝|I-I _Tr|-▽I (3)

Wherein, I is reference picture, I _trfor the image after image registration transformation, | I-I _tr| be the two-value difference image of initial motion target detection, ▽ I is the gray level image of reference picture, I _rfor the result images after gradient suppresses.In the present embodiment, the 55th two field picture is reference picture; Image after image registration transformation is according to image registration flow process, calculates parameter, the image after converting the 56th two field picture.Gradient is utilized to suppress method to eliminate the experimental result of part parallax pixel as shown in Figure 5.

(S32) suppress method can not eliminate the situation of parallax pixel completely for gradient and adopt epipolar-line constraint method to eliminate residue parallax pixel, obtaining the testing result comprising most of object pixel and a small amount of parallax pixel.Epipolar-line constraint, be also called two view geometry constraints, it represents geometric relationship inherent between two width images, does not need the three-dimensional structure information knowing scene, only relevant with the internal and external parameter of video camera.Below, its principle is briefly described, the schematic diagram of epipolar-line constraint as shown in Figure 6,

Suppose it is a rest point in M three dimensions, m and m ' is M point respectively (is the imaging in the camera of same scene in picture plane, namely the image taken) subpoint on I and I ', l and l ' is the polar curve of two width images respectively, in Fig. 5, C is the photocentre of left camera, C ' is the photocentre of right camera, e is limit, e ' is antipodal points, can calculate fundamental matrix F by the unique point calculating two width images, and on the other hand, by fundamental matrix F, two polar curve l and l ' can be calculated:

l′＝F×m，l＝F ^T×m′ (4)

Generally, if three-dimensional M point, utilize epipolar-line constraint to pixel whether motion judges, namely M point may be static background point or moving target point, now define a distance D _epiLineit is judged, distance D _epiLinebe calculated as follows:

D _epiLine＝(|m·l|+|m′·l′|)2 (5)

Wherein, | ml| represents the distance of m to polar curve l, | m ' l ' | represent the distance of m ' to polar curve l '.

In theory, D can be used _epiLineto M, whether motion conditions judges, works as D _epiLine> 0, then judge that this point is motor point, otherwise be exactly static background point.But, in actual computation process, there will be error owing to considering, a threshold value D can be set _threshold, work as D _epiLine> D _thresholdtime, judge that M point is as motor point, otherwise be rest point.

Wherein, fundamental matrix F is exactly the geometric expression of epipolar-line constraint.Utilize 8 methods to calculate fundamental matrix in the present invention, its Computing Principle is as follows:

The definition of fundamental matrix is,

m ^'TFm＝0

Wherein, be any pair match point in two width images, fundamental matrix F is an order is the matrix of 2, works as match point when (at least 7 to) are abundant, just can be used for calculating fundamental matrix F, $F=\left[\begin{array}{ccc}{F}_{11}& F_{12}& F_{13}\\ F_{21}& F_{22}& F_{23}\\ F_{31}& F_{32}& F_{33}\end{array}\right],$ Note m _i=[u _i, v _i, 1] ^tand m _i'=[u _i', v _i', 1] ^tfor Corresponding matching point, each group match point provides a linear equation of the unknown element about F, its coefficient available point m _i=[u _i, v _i, 1] ^tand m _i'=[u _i', v _i', 1] ^tcoordinate represent, concerning any pair match point, by equation expansion as shown in the formula:

u' _iu _iF ₁₁+u' _iv _iF ₁₂+u' _iF ₁₃+v' _iu _iF ₂₁+v' _iv _iF ₂₂+v' _iF ₂₃+u _iF ₃₁+v _iF ₃₂+F ₃₃＝0

If there is n to such match point, then can obtain n equation, structure vector:

f＝[F ₁₁,F ₁₂,F ₁₃,F ₂₁,F ₂₂,F ₂₃,F ₃₁,F ₃₂,F ₃₃] ^T

According to the coordinate of n to match point, structural matrix:

$A=\left[\begin{array}{ccccccccc}{u^{\′}}_1{u}_1& {u^{\′}}_1{v}_1& {u^{\′}}_1& {v^{\′}}_1{u}_1& {v^{\′}}_1{v}_1& {v^{\′}}_1& u_1& v_1& 1\\ .& .& .& .& .& .& .& .& .\\ .& .& .& .& .& .& .& .& .\\ .& .& .& .& .& .& .& .& .\\ {u^{\′}}_n{u}_n& {u^{\′}}_n{v}_n& {u^{\′}}_n& {v^{\′}}_n{u}_n& {v^{\′}}_n{v}_n& {v^{\′}}_n& u_n& v_n& 1\end{array}\right]$

Thus have:

Af＝0

When n >=8, can linear solution vector f.In the solution procedure of reality, the constraint condition of 8 methods is || Af|| is minimum, and then estimates matrix F.

(concrete list of references: Chen Zezhi etc. the weighting normalization linear algorithm [J] that basis matrix is estimated. Journal of Software, 2001,12 (3): 420-426.)

In the present embodiment, get D _threshold=15, the fundamental matrix calculated: $F=\left[\begin{array}{ccc}7.5340e-07& 6.8771e-05& -0.0184\\ -6.8574e-05& 2.6318e-06& 0.1877\\ 0.0195& -0.1890& 0.9642\end{array}\right],$ Result after epipolar-line constraint as shown in Figure 7.

(S4) complete vehicle is partitioned into.Through step S3, comprise most object pixel and a small amount of parallax pixel in the result images obtained, by morphological operation process, be specially: first carry out etching operation, the structure that wherein structural element is is radius with 1, $\mathrm{se}=\left[\begin{array}{ccc}0& 1& 0\\ 1& 1& 1\\ 0& 1& 0\end{array}\right],$ After carry out expansive working, the structural element that structural element is is radius with 2, filtering parallax pixel, obtains accurate object pixel; " Gray Projection method " is finally utilized to split complete vehicle.Gray Projection ratio juris be by pixel each in bianry image to often arranging projection, then get the row of pixel value non-zero, suppose that the size of image is w × h pixel, then obtain to often arranging projection:

$\mathrm{Sum}\left(x\right)=\underset{y=1}{\overset{h}{\mathrm{\Σ}}}I(x,y),x=1...w---\left(6\right)$

Select the row of Sum (x) > 0, the i.e. row of moving target existence, set the size of a distance threshold to coordinate to classify, and give different class marks respectively, the pixel that label is identical is exactly same target, then all pixels of this label is got to the border of row.Further, search for the row of all non-zero pixels, obtain the border of each moving target, utilize rectangle to draw moving target region, the experimental result image obtained as shown in Figure 8.

Below be only the preferred embodiment of the present invention, protection scope of the present invention be not only confined to above-described embodiment, all technical schemes belonged under thinking of the present invention all belong to protection scope of the present invention.It should be pointed out that for those skilled in the art, some improvements and modifications without departing from the principles of the present invention, should be considered as protection scope of the present invention.

Claims (3)

1. take photo by plane and to suppress based on gradient in video and the vehicle checking method of epipolar-line constraint in low latitude, it is characterized in that, comprise the following steps:

(S1) inter frame image registration: choose the two field picture that comprises vehicle to be detected as with reference to image in video is taken photo by plane in low latitude, a rear two field picture of this frame is as image subject to registration, detect the SURF feature of two width images respectively, then two width characteristics of image are mated, calculate the affine transformation parameter of image registration, image subject to registration is carried out inverse transformation according to the parameter calculated, obtains registration result image;

(S2) initial motion target detection: process two width imagery exploitation frame difference methods after registration, obtains being become order target area by flase drop in the approximate region of vehicle and image background;

(S3) precise area of vehicle is obtained: first utilize gradient to suppress method to eliminate part parallax pixel, then the parallax pixel adopting epipolar-line constraint method elimination gradient to suppress method not eliminated, obtains the testing result image comprising vehicle pixel and residue parallax pixel;

(S4) be partitioned into complete vehicle: first morphological operation is carried out to the testing result image in described step (S3), filtering parallax pixel, the vehicle pixel obtained, then utilize Gray Projection method to be partitioned into complete vehicle.

2. take photo by plane and to suppress based on gradient in video and the vehicle checking method of epipolar-line constraint in low latitude as claimed in claim 1, it is characterized in that: the frame difference method in described step (S2) is by poor for the two width images being used for frame difference, set a threshold value to judge work difference result images, the judgement being greater than this threshold value is moving target and is set to 1, and the judgement being less than threshold value is background and is set to 0.

3. take photo by plane and to suppress based on gradient in video and the vehicle checking method of epipolar-line constraint in low latitude as claimed in claim 1, it is characterized in that: the morphological operation in described step (S4) for first to carry out etching operation, after carry out expansive working.