CN103955948A - Method for detecting space moving object in dynamic environment - Google Patents

Abstract

The invention provides a method for detecting a space moving object in a dynamic environment. Four front-frame and back-frame images are acquired through a binocular camera, reliable feature points in the four images are extracted, three-dimensional reconstruction is conducted on the feature points, scene flow of the images is calculated, then the feature points with similar motion are clustered, and the moving object in a scene is detected.

Description

Space movement target detection method under a kind of dynamic environment

Technical field

The invention belongs to Image processing and compute machine vision field, being specifically related to a kind of space rope that is mounted in is the method that the binocular solid camera in robot system detects space movement target.

Background technology

Space rope is that robot system is made up of " Sheng Xi robot+space tether+space platform ", there is the features such as safety, flexible, operating distance is far away, can be widely used in comprising maintainable technology on-orbit, annotate in-orbit, auxiliary become rail and in-orbit the auxiliary assembling in space station etc. in-orbit in service, become one of study hotspot of space manipulation technical field.This novel intelligent miniature robot, can independently approach target in space.In the process of approaching at it, utilize self-contained two CCD camera measure system to target detect in real time, Tracking and Measurment.Wherein first the step of most critical is, under dynamic background, moving target is carried out to automatic Detection and Extraction, and this relates to Detection for Moving Target.

Moving object detection is widely used in all many-sides such as robot navigation, weapon guidance, traffic flow monitoring, human motion analysis, video compress at present, relating to many research fields such as computer vision, pattern-recognition, statistics, image understanding, is that image is processed one of the focus of research field and difficult point.Have autokinetic movement because the kinetic characteristic of space Sheng Xi robot causes its video camera carrying, with target relative motion, problem is become and be more difficult to solve, this detects totally unfavorable by the moving target to follow-up automatically.

For moving object detection problem on motion platform, a kind of conventional method need to first be eliminated camera displacement, and the image of taking by different angles or the position of Same Scene carries out registration, then detects moving target.But this method need to first be eliminated camera displacement, make whole moving object detection algorithm more complicated, step is more, and the stability of whole algorithm and precision are also difficult to ensure.

This patent in Sheng Xi robot, carry video camera generation autokinesis time detect the problem of moving target, a kind of moving target detecting method based on sparse scene flows has been proposed, realize the automatic detection of moving target under dynamic background.

Summary of the invention

The object of the invention is to make up the deficiency of existing Detection for Moving Target, the detection method of the space movement target under a kind of dynamic environment is provided, the method has good robustness, and simple to operate, and computing velocity is fast, and testing result is accurate.

For achieving the above object, technical scheme of the present invention is:

A kind of space rope is robot motion's object detection method, two frame fourth officer images before and after obtaining by binocular camera, extract reliable unique point in fourth officer image, then this unique point is carried out to three-dimensional reconstruction, the scene flows of computed image, then, will there is the feature points clustering of similar movement, thereby detect the object moving in scene.

The method of extracting reliable unique point in image is: first detect unique point, then the unique point obtaining is screened, retain the unique point of coupling.

The method of described three-dimensional reconstruction is:

Suppose that the coordinate of matching characteristic point in binocular camera image is respectively: x=[u _l, v _l] ^t, x=[u _r, v _r] ^t, this unique point is mapped to the three-dimensional coordinate X=[X in world coordinate system, Y, Z] ^t∈ R ³for:

$X=\frac{(u_L-c_{u,L})\·b}{d},$ $Y=\frac{(v_L-c_{v,L})\·b}{d},$ $Z=\frac{b\·f}{d},$

Wherein, the baseline that b is stereoscopic camera, c _u,Land c _v,Lbe the principal point of left camera, f is focal length, and d represents parallax, d=|u _l-u _r|.

Suppose that the speed V that unique point is mapped to the three-dimensional coordinate point of world coordinate system is constant in the time period t of 5 frames, t=0.5s, described scene flows is according to following formula calculating: $f^{\′}\left(x\right)\≈\frac{137}{60}f\left(x\right)-5f(x-1)+5f(x-2)-\frac{10}{3}f(x-3)+\frac{5}{4}f(x-4)-\frac{1}{5}f(x-5),$

Wherein, f'(x) be scene flows, f (x), f (x-1), f (x-2), f (x-3), f (x-4), f (x-5) represents respectively matching characteristic point in the 5 two field pictures position in three-dimensional system of coordinate.

The method with the feature points clustering of similar movement is: connect as node uses Delaunay triangulation taking the matching characteristic point detecting, obtain Delaunay triangle set T, the difference of the scene flows on two summits to the each triangle sideline in the triangle set T obtaining is carried out threshold decision, obtains the cluster of the unique point with similar scene flows.

The method with the feature points clustering of similar movement is: poor to two node i on same sideline and their scene flows of j definition, relatively the scene flows of two nodes on same sideline is poor, if difference is no more than given threshold value, retains this sideline, otherwise remove this sideline.

Detect after the object moving in scene, to error-detecting to stationary body reject, to guarantee the reliability and stability of moving object detection, the concrete grammar of rejecting is: exceed reasonable range scale if detect the range scale in the region that obtains object, from testing result, reject, described range scale refers to the regional extent that object occupies in image, and described reasonable range scale is 120 pixel × 30 pixels.

Detect after the object moving in scene, the object detecting is carried out to association, to guarantee the reliability and stability of moving object detection, associated concrete grammar is: the speed of supposing the object detecting is constant in time interval Δ t=0.5s, the position detecting in previous moment according to object and speed, dope the predicted position of current time, consider again measuring error and predicated error in tolerance interval, around current predicted position, place a path thresholding, then in whether the detection position of current time judgment object drops on this predicted path thresholding, if dropped in predicted path thresholding, think that so this object detection is normal, otherwise, think object detection lose, the moving object detecting is at least all correctly detected at two continuous time step Δ t, and in the process detecting, lose once at the most, so just determine that it is the moving object of correct detection, otherwise think that it is error detection, should reject, finally remain, be the moving object in the scene finally detecting, described path thresholding is centered by predicted position, taking 1 meter of space spheric region as radius.

Compared with prior art, the present invention at least has following beneficial effect: the present invention has adopted the algorithm that sparse scene stream is cut apart to detect the moving target in dynamic scene, compared with conventional art, do not need first to detect the displacement of camera, overcome the complex steps of prior art, realize complexity, real-time is bad, the low deficiency that waits of degree of accuracy.The present invention utilizes the calculating of the sparse scene flows of reliable characteristic point, can significantly improve travelling speed, ensures real-time.Adopt Delaunay triangulation to cut apart scene flows, can identify efficiently the moving target in scene.In invention, the Moving Objects detecting is carried out to association, ensured accuracy and the reliability of algorithm.

Brief description of the drawings

Fig. 1 is algorithm main-process stream block diagram of the present invention.

Embodiment

The inventive method is mainly made up of parts such as three-dimensional reconstruction algorithm, sparse scene flow algorithm, scene flows clustering algorithm, object association algorithms.The method specifically comprises that step is as follows:

Step 1: the front and back two frame four width images that obtain binocular camera;

Step 2: the continuous image of this two frame, to carrying out feature point extraction, is retained to those corresponding reliable unique points in this four width image;

Step 3: utilize the parallax of left and right image, each unique point is carried out to three-dimensional reconstruction, obtain the coordinate of each unique point in 3D world coordinate system;

Step 4: detect the finite difference of 5 3D positions that obtain by calculating each unique point, the approximate scene flows that obtains in 5 time intervals;

Step 5: connect as node uses Delaunay triangulation taking the unique point detecting;

Step 6: judge scene flows poor of two nodes on same line, if difference is no more than given threshold value, retain this sideline, otherwise remove this sideline, so just the feature points clustering with similar scene flows together;

Step 7: the stationary body of removing error detection;

Step 8: the Moving Objects detecting is carried out to association, further guarantee the reliability and stability of testing result.

Below in conjunction with accompanying drawing, the present invention is described in detail.It should be pointed out that described embodiment is only intended to be convenient to the understanding of the present invention, and it is not played to any restriction effect.

As shown in Figure 1, the moving target detecting method that the embodiment of the present invention provides comprises:

Step 1: obtain left camera former frame image x _{l, k-1}, current frame image x _l,k, right camera former frame image x _{r, k-1}, current frame image x _r,k;

Step 2: the extraction of reliable characteristic point in stereo-picture, concrete steps are decomposed as follows:

(1): to image x _{l, k-1}, x _l,k, x _{r, k-1}, x _r,kcarry out respectively feature point detection, obtain the feature point set of four width images, be respectively P1={p11, p12 ..., p1n}, P2={p21, p22 ..., p2n}, P3={p31, p32 ..., p3n}, P4={p41, p42 ..., p4n};

(2): the unique point that step (1) is obtained is screened: first the unique point in feature point set P1 and P2 is mated, retain the unique point of coupling, again by the Feature Points Matching in the unique point and the P3 that remain, retain the unique point of coupling, by the Feature Points Matching in the unique point and the P4 that remain, retain the unique point of coupling again.Feature point set P1, P2, P3, those unique points that remain in P4 form new point set, are called matching characteristic point set Q1={q11, q12 ..., q1j}, Q2={q21, q22 ..., q2j}, Q3={q31, q32 ..., q3j}, Q4={q41, q42 ..., q4j};

The wherein detection of unique point, what utilize is the method proposing in list of references " A.Geiger; J.Ziegler; and C.Stiller.Stereoscan:Dense3d reconstruction in real-time.In IEEE Intelligent Vehicles Symposium; Baden-Baden; Germany, June2011 ", repeats no more herein.

Step 3: to the matching characteristic point pair set Q2 in two width images of left and right camera present frame, Q4 carries out three-dimensional reconstruction, obtains the coordinate of each unique point in 3D world coordinate system:

If the coordinate of matching characteristic point in left camera image is x=[u _l, v _l] ^t, the coordinate in right camera image is x=[u _r, v _r] ^t, this unique point is mapped to the three-dimensional coordinate X=[X in world coordinate system, Y, Z so] ^t∈ R ³for:

$X=\frac{(u_L-c_{u,L})\·b}{d}---\left(1\right)$

$Y=\frac{(v_L-c_{v,L})\·b}{d}---\left(2\right)$

$Z=\frac{b\·f}{d}---\left(3\right)$

Wherein b represents the baseline of stereoscopic camera, c _u,Land c _v,Lbe the principal point of left camera, f is focal length, and d represents parallax, d=|u _l-u _r|.

Step 4: calculate scene flows, suppose that the speed V that unique point is mapped to the three-dimensional coordinate point of world coordinate system is constant within the time period of 5 frames (t=0.5s), velocity can be regarded world point X as so _kthe first order derivative of position, wherein, k is k two field picture, k=1 ... 5:

$V=\frac{\mathrm{\Δ}{X}_k}{\mathrm{\Δt}}---\left(4\right)$

1/ Δ t=10Hz is constant for sampling rate, wherein, and the sampling time interval before and after Δ t represents between two two field pictures, Δ X _krepresent world point X _kcorresponding three-dimensional position in two two field pictures of front and back poor.

For the convenience of calculating, use the approximate differentiate of backward difference, formula is as follows:

$f^{\′}\left(x\right)\≈\frac{137}{60}f\left(x\right)-5f(x-1)+5f(x-2)-\frac{10}{3}f(x-3)+\frac{5}{4}f(x-4)-\frac{1}{5}f(x-5)---\left(5\right)$

Wherein, f'(x) be scene flows, f (x), f (x-1), f (x-2), f (x-3), f (x-4), f (x-5) represents respectively matching characteristic point in the 5 two field pictures position in three-dimensional system of coordinate.

Can be calculated the scene flows of scene by formula (4) and (5).

Step 5: connect as node uses Delaunay triangulation taking the matching characteristic point detecting, obtain Delaunay triangle set T.

Concrete grammar has detailed introduction in list of references " C.B.Barber; D.P.Dobkin; and H.Huhdanpaa.The quickhull algorithm for convex hulls.ACM Transactions on Mathematical Software; 22 (4): 469 – 483; 1996 ", repeats no more herein.

Step 6: the difference of the scene flows on two summits to the each triangle sideline in the triangle set T obtaining in upper step is carried out threshold decision, obtains the cluster of the unique point with similar scene flows, and concrete steps are as follows:

(1): the difference to two node i on same sideline and their scene flows of j definition is as follows:

$\mathrm{\Δ}(V_i,V_j)=\sqrt{{(V_i-V_j)}^T{{\mathrm{\Σ}}_{i,j}}^{-1}(V_i-V_j)}---\left(6\right)$

Wherein, V _i, V _jrepresent respectively the scene flows on two node i and j;

The covariance matrix Σ of scene flows is as follows:

Σ＝JSJ ^T (7)

Wherein S measures the diagonally opposing corner measurement noise matrix that noise is 0.5 pixel.

Matrix J is the Jacobian matrix of scene flows, for 3D world point X=[X, Y, Z] ^t, its Jacobian matrix J provides as follows:

$J=\frac{\mathrm{dV}}{\mathrm{dx}}=\left[\begin{array}{ccc}\frac{{\∂V}_X}{\∂u_{L,k}}& \frac{\∂V_X}{{\∂u}_{R,k}}& \frac{\∂V_X}{\∂v_k}\\ \frac{{\∂V}_Y}{\∂u_{L,k}}& \frac{{\∂V}_Y}{{\∂u}_{R,k}}& \frac{\∂V_Y}{\∂v_k}\\ \frac{{\∂V}_Z}{{\∂u}_{L,k}}& \frac{\∂V_Z}{{\∂u}_{R,k}}& \frac{\∂V_Z}{{\∂v}_k}\end{array}\right],i=0...5---\left(8\right)$

(2): the relatively difference Δ (V of the scene flows of two nodes on same sideline _i, V _j), if difference is no more than given threshold value (being made as 30 pixels in the present embodiment), retain this sideline, otherwise remove this sideline.So just the feature points clustering with similar scene flows together, gone out the object O ' moving in scene with regard to Preliminary detection ₁, O ' ₂..., O ' _n.

Step 7: the static object of removing error detection:

For the static region of error detection, the range of size in the region that its detection obtains has often exceeded the rational range of size of target, therefore can judge, range scale is exceeded to the region of reasonable range scale (being made as 120 pixel × 30 pixels in the present embodiment), from testing result, reject, retain object O ₁, O ₂..., O _m.Range scale herein refers to the regional extent that object occupies in image.

Step 8: to the moving object O detecting ₁, O ₂..., O _mcarry out association, further guarantee the reliability and stability of testing result, concrete steps are as follows:

(1): for the object O detecting _i, suppose that its speed is constant in time Δ t=0.5s;

(2): according to the position of its previous moment with and speed, dope current time position

(3): consider measuring error and predicated error in tolerance interval, in current time predicted position placing a path thresholding around (is set in the present embodiment with predicted position centered by, taking 1 meter of space spheric region as radius);

(4): at current time, detect object O _iposition be if judge that it has dropped on predicted position path thresholding in, think that so this object detection is normal; If detection position drop on predicted position path thresholding outside, think so object detection lose;

(5): the moving object detecting is at least all correctly detected at two continuous time step Δ t, and in the process detecting, lose once at the most, so just determine that it is the moving object of correct detection, otherwise think that it is error detection, should reject, finally remain, be the moving object in the scene finally detecting.

So far, just correctly stably detected the moving target in scene.

The above; it is only the embodiment in the present invention; but protection scope of the present invention is not limited to this; any people who is familiar with this technology is in the disclosed technical scope of the present invention; can understand conversion or the replacement expected; all should be encompassed in of the present invention comprise scope within, therefore, protection scope of the present invention should be as the criterion with the protection domain of claims.

Claims (8)

1. the space movement target detection method under a dynamic environment, it is characterized in that: two frame fourth officer images before and after obtaining by binocular camera, extract reliable unique point in fourth officer image, then this unique point is carried out to three-dimensional reconstruction, the scene flows of computed image, then, will there is the feature points clustering of similar movement, thereby detect the object moving in scene.

2. the space movement target detection method under a kind of dynamic environment according to claim 1, it is characterized in that: the method for extracting reliable unique point in image is: first detect unique point, then the unique point obtaining is screened, retain the unique point of coupling.

3. the space movement target detection method under a kind of dynamic environment according to claim 2, is characterized in that: the method for described three-dimensional reconstruction is: the coordinate of supposition matching characteristic point in binocular camera image is respectively: x=[u _l, v _l] ^t, x=[u _r, v _r] ^t, this unique point is mapped to the three-dimensional coordinate X=[X in world coordinate system, Y, Z] ^t∈ R ³for:

$X=\frac{(u_L-c_{u,L})\·b}{d},$ $Y=\frac{(v_L-c_{v,L})\·b}{d},$ $Z=\frac{b\·f}{d},$

Wherein, the baseline that b is stereoscopic camera, c _u,Land c _v,Lbe the principal point of left camera, f is focal length, and d represents parallax, d=|u _l-u _r|.

4. the space movement target detection method under a kind of dynamic environment according to claim 1, it is characterized in that: the speed V that supposition unique point is mapped to the three-dimensional coordinate point of world coordinate system is constant in the time period t of 5 frames, t=0.5s, described scene flows calculates according to following formula: $f^{\′}\left(x\right)\≈\frac{137}{60}f\left(x\right)-5f(x-1)+5f(x-2)-\frac{10}{3}f(x-3)+\frac{5}{4}f(x-4)-\frac{1}{5}f(x-5),$ Wherein, f'(x) be scene flows, f (x), f (x-1), f (x-2), f (x-3), f (x-4), f (x-5) represents respectively matching characteristic point in the 5 two field pictures position in three-dimensional system of coordinate.

5. the space movement target detection method under a kind of dynamic environment according to claim 4, it is characterized in that: the method with the feature points clustering of similar movement is: connect as node uses Delaunay triangulation taking the matching characteristic point detecting, obtain Delaunay triangle set T, the difference of the scene flows on two summits to the each triangle sideline in the triangle set T obtaining is carried out threshold decision, obtains the cluster of the unique point with similar scene flows.

6. according to the space movement target detection method under a kind of dynamic environment described in claim 4 or 5, it is characterized in that: the method with the feature points clustering of similar movement is: poor to two node i on same sideline and their scene flows of j definition, relatively the scene flows of two nodes on same sideline is poor, if difference is no more than given threshold value, retain this sideline, otherwise remove this sideline.

7. according to the space movement target detection method under a kind of dynamic environment described in any one in claim 1 to 5, it is characterized in that: detect after the object moving in scene, to error-detecting to stationary body reject, to guarantee the reliability and stability of moving object detection, the concrete grammar of rejecting is: exceed reasonable range scale if detect the range scale in the region that obtains object, from testing result, reject, described range scale refers to the regional extent that object occupies in image, and described reasonable range scale is 120 pixel × 30 pixels.

8. the space movement target detection method under a kind of dynamic environment according to claim 6, it is characterized in that: detect after the object moving in scene, the object detecting is carried out to association, to guarantee the reliability and stability of moving object detection, associated concrete grammar is: the speed of supposing the object detecting is constant in time interval Δ t=0.5s, the position detecting in previous moment according to object and speed, dope the predicted position of current time, consider again measuring error and predicated error in tolerance interval, around current predicted position, place a path thresholding, then in whether the detection position of current time judgment object drops on this predicted path thresholding, if dropped in predicted path thresholding, think that so this object detection is normal, otherwise, think object detection lose, the moving object detecting is at least all correctly detected at two continuous time step Δ t, and in the process detecting, lose once at the most, so just determine that it is the moving object of correct detection, otherwise think that it is error detection, should reject, finally remain, be the moving object in the scene finally detecting, wherein, described path thresholding is centered by predicted position, taking 1 meter of space spheric region as radius.