CN104647390A - Multi-camera combined initiative object tracking method for teleoperation of mechanical arm - Google Patents

Abstract

The invention relates to a multi-camera combined initiative object tracking method for teleoperation of a mechanical arm, which belongs to the field of the teleoperation of the mechanical arm. The method comprises the steps of installing a plurality of cameras in different angles on the basis of the teleoperation, calibrating each camera, realizing the combined initiative tracking by adopting the matching of particle filter and sift local characteristics, enabling a to-be-tracked target to always stay in the center of a view field, and monitoring each angle of the mechanical arm, so that the tracking failure caused by factors such as the shielding can be avoided, and the tracking robustness can be improved by adding the human-machine interaction. According to the method, in the teleoperation process, a new target is marked in a human-machine interaction way, so that the target always stays in an observation area of an operator, or the target is updated, so that a subsequent control strategy can be made by the operator.

Description

For the multiple-camera associating active tracing order calibration method of mechanical arm remote operating

Technical field

The present invention relates to a kind of multiple-camera for mechanical arm remote operating associating active tracing order calibration method, belong to mechanical arm remote operating field.

Background technology

For the remote operating of mechanical arm, vision system is a key technology in mechanical arm remote operating, and it provides real-time image information and the spatial pose information of target object for motion arm.State residing for motion arm, environment and working stage are also fed back to terrestrial operation person by vision system intuitively simultaneously.

In current teleoperation method, video camera provides image information, do not utilize the active tracing of image information realization to specific objective in the mechanical arm course of work, and camera all remains unchanged, can not active accommodation visual angle, do not consider that manipulator motion exceedes camera visual field and situation about cannot monitor.

In existing technical literature, patent of invention " control system and method thereof based on the Space teleoperation robot of Kinect ", publication number is CN201310193564.7, utilizes Kinect to realize three-dimensional environment modeling, and carries out Concordance to prediction environment.The shortcoming of the method is, only adopts a video camera to take mechanical arm region, and does not follow the tracks of realization of goal.In addition, the method camera keeps motionless, cannot guarantee that mechanical arm is all the time in visual field.

Summary of the invention

The object of the invention is to propose a kind of multiple-camera for mechanical arm remote operating associating active tracing order calibration method, on remote operating basis, for a set of multiple camera active vision system of mechanical arm remote operating design, with Real-time Collection, transmitting image, for operator provides visual observation foundation intuitively.

The associating of the multiple-camera for the mechanical arm remote operating active tracing order calibration method that the present invention proposes, comprises the following steps:

(1) multiple video camera is placed in the upper left of mechanical arm, upper right and front, video camera is demarcated, obtain the Intrinsic Matrix M of multiple cameras respectively ₁, outer parameter matrix M ₂, distortion factor and module and carriage transformation matrix T ₁₂, T ₂₃, T ₃₄, concrete steps are as follows:

(1-1) set the coordinate of spatial point P in camera coordinate system as P (x _c, y _c, z _c), make spatial point P along the ray cast by photocentre on a plane of delineation, image coordinate system is set up in the plane of delineation, obtaining the projection coordinate of this spatial point P in image coordinate system is P (x, y), by this projection coordinate, the coordinate be expressed as in computer picture coordinate system is P (u, v), obtained by pinhole imaging system principle, the coordinate (x of spatial point P in camera coordinates _c, y _c, z _c) with the coordinate conversion relation of projection coordinate P (x, y) be: $\left\{\begin{array}{c}x=f\frac{x_C}{z_C}\\ y=f\frac{y_C}{z_C}\end{array}\right.,$ Wherein f is focal length of camera;

(1-2) image-generating unit is set, the x-axis of this image-generating unit in above-mentioned image coordinate system and the physical size on y-axis direction are respectively dx and dy, coordinate (the u of any one pixel under above computer image coordinate system in image-generating unit, there is following coordinate conversion relation in the coordinate (x, y) v) and under above-mentioned image coordinate system: $\left\{\begin{array}{c}u=\frac{x}{\mathrm{dx}}+u_0\\ v=\frac{y}{\mathrm{dy}}+v_0\end{array}\right.,$ Wherein, O (u ₀, v ₀) for being arranged on video camera primary optical axis any point at the imager coordinate of computer picture coordinate system, this imager coordinate and principal point coordinate;

(1-3) according to the coordinate conversion relation of above-mentioned steps (1-1) and step (1-2), the Intrinsic Matrix M of video camera is obtained ₁for: $M_1=\left[\begin{array}{ccc}\frac{f}{\mathrm{dx}}& 0& u_0\\ 0& \frac{f}{\mathrm{dy}}& v_0\\ 0& 0& 1\end{array}\right],$ Wherein, for the normalization focal length of focal length of camera f on the u axle of camera image plane coordinate system, for the normalization focal length of focal length of camera f on the v axle of camera image plane coordinate system;

(1-4) set the coordinate of spatial point P in world coordinate system as (x _w, y _w, z _w), there is following relation in the coordinate of spatial point P in camera coordinate system and the coordinate in world coordinate system: $\left[\begin{array}{c}{x}_C\\ y_C\\ z_C\end{array}\right]=\left[\begin{array}{cc}R& t\end{array}\right]\left[\begin{array}{c}{x}_w\\ y_w\\ z_w\\ 1\end{array}\right],$ Wherein, R is the unit orthogonal matrix of 3 × 3, and t is the D translation vector between camera coordinate system and world coordinate system, and definition [R t] is wherein external parameters of cameras matrix M ₂;

(1-5) according to the camera intrinsic parameter M that above-mentioned steps (1-3) and step (1-4) obtain ₁with outer parameter M ₂, the Transformation Relation of Projection obtained between the coordinate of spatial point P in world coordinate system and the projection coordinate of spatial point P in computer picture coordinate system is: P=M ₁m ₂;

(1-6) set the radial distortion parameter of video camera as k ₁, k ₂and k ₃, make k ₁, k ₂and k ₃meet following equation group: $\left\{\begin{array}{c}x=x_u(1+k_1{r}^2+k_2{r}^4+k_3{r}^6)\\ y=y_u(1+k_1{r}^2+k_2{r}^4+k_3{r}^6)\end{array}\right.,$ Solve this equation group, obtain the radial distortion parameter k of video camera ₁, k ₂and k ₃, wherein, (x, y) is the home position coordinate of spatial point P in image coordinate system, (x _u, y _u) the ideal position coordinate that obtained by image-forming principle for spatial point P;

If the tangential distortion parameter of video camera is p ₁and p ₂, make p ₁and p ₂meet following equation group: $\left\{\begin{array}{c}x=x_u+2p_1{x}_u{y}_u+p_2(r^2+2x_u^2)\\ y=y_u+2p_2{x}_u{y}_u+p_1(r^2+2y_u^2)\end{array}\right.,$ Solve this equation group, obtain the tangential distortion parameter p of video camera ₁and p ₂;

(1-7) radial distortion parameter obtained according to above-mentioned steps (1-6) is k ₁, k ₂and k ₃with tangential distortion parameter p ₁and p ₂, there is following transformational relation between the home position coordinate of spatial point P and ideal position coordinate:

$\left\{\begin{array}{c}x=x_u(1+k_1{r}^2+k_2{r}^4+k_3{r}^6)+2p_1{x}_u{y}_u+p_2(r^2+2x_u^2)\\ y=y_u(1+k_1{r}^2+k_2{r}^4+k_3{r}^6)+2p_2{x}_u{y}_u+p_1(r^2+2y_u^2)\end{array}\right.;$

Following transformational relation is there is between the ideal position coordinate of spatial point P and home position coordinate:

$\left\{\begin{array}{c}{x}_u=\frac{x-2p_1\mathrm{xy}+p_2(r^2+2x^2)}{1+k_1{r}^2+k_2{r}^4+k_3{r}^6}\\ y_u=\frac{y-2p_2\mathrm{xy}+p_1(r^2+2x^2)}{1+k_1{r}^2+k_2{r}^4+k_3{r}^6}\end{array}\right.;$

(1-8) travel through each video camera in multiple video camera, repeat step (1-1)-step (1-7), complete the demarcation of video camera;

(1-9) set spin matrix between first video camera in multiple video camera and second video camera as R ₁₂, the translation matrix between first video camera and second video camera is Tran ₁₂, obtain the module and carriage transformation matrix T between first video camera and second video camera ₁₂: T ₁₂=[R ₁₂tran ₁₂];

(1-10) travel through often any two video cameras in multiple video camera, repeat step (1-9), obtain the module and carriage transformation matrix T of multiple video camera ₁₂, T ₂₃, T ₃₄;

(2) make to wait to capture target through the calibrated multiple cameras active tracing of above-mentioned steps (1), specifically comprise the following steps:

(2-1) upper left of mechanical arm, upper right and front is placed in multiple through above-mentioned steps (1) calibrated video camera, if wait to capture target's center S in video camera imaging plane P _don be projected as S', calculate the distance d between S' and the center C of video camera imaging plane, setpoint distance threshold value th, the d that adjusts the distance judges, if d > is th, then sends an adjustment instruction to video camera, the direction making camera lens carry out distance d is reduced is rotated, until d≤th, if d≤th, then camera lens is made to keep origin-location;

(2-2) adopt particle filter tracking algorithm, video camera associating active tracing is waited to capture target, and concrete steps are as follows:

(2-2-1) make any video camera in multiple cameras obtain to wait the sequence of video images capturing target, to t in sequence of video images ₀the image in moment carries out manual mark, mark out wherein wait capture target area;

(2-2-2) capture the center of target area for initial point with above-mentioned waiting, produce a particle collection , wherein m is the particle number of particle set, m=1 ..., M, each particle represents one and treats the region that crawl target may exist; If particle collection in the motion Normal Distribution of particle, particle collection in each particle independent propagation, obtain t particle collection and the t+1 moment particle collection ;

(2-2-3) t is established ₀moment particle collection reference histograms be reference histograms q ^*total L gray level, if t particle collection color histogram be q _t(x)=q _t(n; X), n=1,2...L, x be particle collection in particle, to t particle collection in each particle independent propagation after, the t+1 moment particle collection that obtains observe, obtain particle collection in the color histogram of each particle region and reference histograms, calculate particle collection in Pasteur distance D between the color histogram of each particle region and reference histograms: definition particle weights are ω, and make ω=D, the value of N is 200;

(2-2-4) carrying out posterior probability calculating to above-mentioned particle weights, there is probability expectation E (x in what obtain t+1 moment particle _t+1): wherein, ω _t+1each particle weights in t+1 moment;

(2-2-5) by above-mentioned desired value E (x _t+1) as waiting that capturing target exists probability optimal estimation in the t+1 moment, and by above-mentioned particle collection in the center that there is the region that probability optimal particle covers as wait capture center, target area;

(2-2-6) repeat above-mentioned steps (2-2-2) ~ step (2-2-5), obtain waiting to capture target there is probability optimal estimation and waiting to capture center, target area at moment t to subsequent time t+1;

(2-2-7) repeat step (2-1), what make camera lens aligning above-mentioned steps (2-2-6) treats crawl center, target area;

(2-2-8) local feature region waiting to capture target area of above-mentioned steps (2-2-7) is extracted;

(2-2-9) make other several video cameras in multiple cameras obtain to wait the sequence of video images capturing target, from multiple sequence of video images, extract local feature region respectively;

(2-2-10) that step (2-2-8) is extracted waits that the local feature region capturing target area mates with all local feature region that step (2-2-9) is extracted, and obtains the precise region waiting to capture target area of other several video cameras in multiple cameras;

(2-2-11) repeat above-mentioned steps (2-2-2) ~ step (2-2-10), the precise region waiting to capture target area every platform video camera being followed the tracks of respectively obtain, realize multiple-camera associating active tracing target.

The associating of the multiple-camera for the mechanical arm remote operating active tracing order calibration method that the present invention proposes, has the following advantages:

1, the associating of the multiple-camera for mechanical arm remote operating active tracking method of the present invention, adopts multiple video camera, realizes active tracing spy being captured to target.

2, the multiple video cameras in the present invention, are placed in different angles, and carry out associating active tracing, tracked target is in optical center all the time, monitor mechanical arm all angles, and avoid because blocking, tracking failure that the factor such as background clutter causes.

3, present invention employs particle filter tracking algorithm and sift Feature Points Matching, realize multiple video camera and combine tracking, improve the robustness of Camera location.

Accompanying drawing explanation

Fig. 1 is video camera imaging principle schematic in the inventive method.

Fig. 2 is camera control model schematic in the inventive method

Detailed description of the invention

The associating of the multiple-camera for the mechanical arm remote operating active tracing order calibration method that the present invention proposes, comprises the following steps:

(1) multiple video camera is placed in the upper left of mechanical arm, upper right and front, video camera is demarcated, obtain the Intrinsic Matrix M of multiple cameras respectively ₁, outer parameter matrix M ₂, distortion factor and module and carriage transformation matrix T ₁₂, T ₂₃, T ₃₄, concrete steps are as follows:

(1-1) as shown in Figure 1, if the coordinate of spatial point P in camera coordinate system is P (x _c, y _c, z _c), make spatial point P along the ray cast by photocentre on a plane of delineation, image coordinate system is set up in the plane of delineation, obtaining the projection coordinate of this spatial point P in image coordinate system is P (x, y), by this projection coordinate, the coordinate be expressed as in computer picture coordinate system is P (u, v), obtained by pinhole imaging system principle, the coordinate (x of spatial point P in camera coordinates _c, y _c, z _c) with the coordinate conversion relation of projection coordinate P (x, y) be: $\left\{\begin{array}{c}x=f\frac{x_C}{z_C}\\ y=f\frac{y_C}{z_C}\end{array}\right.,$ Wherein f is focal length of camera;

(1-2) image-generating unit is set, the x-axis of this image-generating unit in above-mentioned image coordinate system and the physical size on y-axis direction are respectively dx and dy, coordinate (the u of any one pixel under above computer image coordinate system in image-generating unit, there is following coordinate conversion relation in the coordinate (x, y) v) and under above-mentioned image coordinate system: $\left\{\begin{array}{c}u=\frac{x}{\mathrm{dx}}+u_0\\ v=\frac{y}{\mathrm{dy}}+v_0\end{array}\right.,$ Wherein, O (u ₀, v ₀) for being arranged on video camera primary optical axis any point at the imager coordinate of computer picture coordinate system, this imager coordinate and principal point coordinate;

(1-3) according to the coordinate conversion relation of above-mentioned steps (1-1) and step (1-2), the Intrinsic Matrix M of video camera is obtained ₁for: $M_1=\left[\begin{array}{ccc}\frac{f}{\mathrm{dx}}& 0& u_0\\ 0& \frac{f}{\mathrm{dy}}& v_0\\ 0& 0& 1\end{array}\right],$ Wherein, for the normalization focal length of focal length of camera f on the u axle of camera image plane coordinate system, for the normalization focal length of focal length of camera f on the v axle of camera image plane coordinate system;

(1-4) set the coordinate of spatial point P in world coordinate system as (x _w, y _w, z _w), there is following relation in the coordinate of spatial point P in camera coordinate system and the coordinate in world coordinate system: $\left[\begin{array}{c}{x}_C\\ y_C\\ z_C\end{array}\right]=\left[\begin{array}{cc}R& t\end{array}\right]\left[\begin{array}{c}{x}_w\\ y_w\\ z_w\\ 1\end{array}\right],$ Wherein, R is the unit orthogonal matrix of 3 × 3, and t is the D translation vector between camera coordinate system and world coordinate system, and definition [R t] is wherein external parameters of cameras matrix M ₂;

(1-5) according to the camera intrinsic parameter M that above-mentioned steps (1-3) and step (1-4) obtain ₁with outer parameter M ₂, the Transformation Relation of Projection obtained between the coordinate of spatial point P in world coordinate system and the projection coordinate of spatial point P in computer picture coordinate system is: P=M ₁m ₂;

(1-6) set the radial distortion parameter of video camera as k ₁, k ₂and k ₃, make k ₁, k ₂and k ₃meet following equation group: $\left\{\begin{array}{c}x=x_u(1+k_1{r}^2+k_2{r}^4+k_3{r}^6)\\ y=y_u(1+k_1{r}^2+k_2{r}^4+k_3{r}^6)\end{array}\right.,$ Solve this equation group, obtain the radial distortion parameter k of video camera ₁, k ₂and k ₃, wherein, (x, y) is the home position coordinate of spatial point P in image coordinate system, (x _u, y _u) the ideal position coordinate that obtained by image-forming principle for spatial point P;

If the tangential distortion parameter of video camera is p ₁and p ₂, make p ₁and p ₂meet following equation group: $\left\{\begin{array}{c}x=x_u+2p_1{x}_u{y}_u+p_2(r^2+2x_u^2)\\ y=y_u+2p_2{x}_u{y}_u+p_1(r^2+2y_u^2)\end{array}\right.,$ Solve this equation group, obtain the tangential distortion parameter p of video camera ₁and p ₂;

(1-7) radial distortion parameter obtained according to above-mentioned steps (1-6) is k ₁, k ₂and k ₃with tangential distortion parameter p ₁and p ₂, there is following transformational relation between the home position coordinate of spatial point P and ideal position coordinate:

$\left\{\begin{array}{c}x=x_u(1+k_1{r}^2+k_2{r}^4+k_3{r}^6)+2p_1{x}_u{y}_u+p_2(r^2+2x_u^2)\\ y=y_u(1+k_1{r}^2+k_2{r}^4+k_3{r}^6)+2p_2{x}_u{y}_u+p_1(r^2+2y_u^2)\end{array}\right.;$

Following transformational relation is there is between the ideal position coordinate of spatial point P and home position coordinate:

$\left\{\begin{array}{c}{x}_u=\frac{x-2p_1\mathrm{xy}+p_2(r^2+2x^2)}{1+k_1{r}^2+k_2{r}^4+k_3{r}^6}\\ y_u=\frac{y-2p_2\mathrm{xy}+p_1(r^2+2x^2)}{1+k_1{r}^2+k_2{r}^4+k_3{r}^6}\end{array}\right.;$

(1-8) travel through each video camera in multiple video camera, repeat step (1-1)-step (1-7), complete the demarcation of video camera;

(1-9) set spin matrix between first video camera in multiple video camera and second video camera as R ₁₂, the translation matrix between first video camera and second video camera is Tran ₁₂, obtain the module and carriage transformation matrix T between first video camera and second video camera ₁₂: T ₁₂=[R ₁₂tran ₁₂];

(1-10) travel through often any two video cameras in multiple video camera, repeat step (1-9), obtain the module and carriage transformation matrix T of multiple video camera ₁₂, T ₂₃, T ₃₄;

(2) make to wait to capture target through the calibrated multiple cameras active tracing of above-mentioned steps (1), specifically comprise the following steps:

(2-1) upper left of mechanical arm, upper right and front is placed in multiple through above-mentioned steps (1) calibrated video camera, if wait to capture target's center S in video camera imaging plane P _don be projected as S', as shown in Figure 2, calculate the distance d between S' and the center C of video camera imaging plane, setpoint distance threshold value th, the d that adjusts the distance judges, if d > is th, then send an adjustment instruction to video camera, the direction making camera lens carry out distance d is reduced is rotated, until d≤th, if d≤th, then camera lens is made to keep origin-location;

(2-2) adopt particle filter tracking algorithm, video camera associating active tracing is waited to capture target, and concrete steps are as follows:

(2-2-1) make any video camera in multiple cameras obtain to wait the sequence of video images capturing target, to t in sequence of video images ₀the image in moment carries out manual mark, mark out wherein wait capture target area;

(2-2-2) capture the center of target area for initial point with above-mentioned waiting, produce a particle collection , wherein m is the particle number of particle set, m=1 ..., M, each particle represents one and treats the region that crawl target may exist; If particle collection in the motion Normal Distribution of particle, particle collection in each particle independent propagation, obtain t particle collection and the t+1 moment particle collection ;

(2-2-3) t is established ₀moment particle collection reference histograms be reference histograms q ^*total L gray level, if t particle collection color histogram be q _t(x)=q _t(n; X), n=1,2...L, x be particle collection in particle, to t particle collection in each particle independent propagation after, the t+1 moment particle collection that obtains observe, obtain particle collection in the color histogram of each particle region and reference histograms, calculate particle collection in Pasteur distance D between the color histogram of each particle region and reference histograms: definition particle weights are ω, and make ω=D, the value of N is 200;

(2-2-4) carrying out posterior probability calculating to above-mentioned particle weights, there is probability expectation E (x in what obtain t+1 moment particle _t+1): wherein, ω _t+1each particle weights in t+1 moment;

(2-2-5) by above-mentioned desired value E (x _t+1) as waiting that capturing target exists probability optimal estimation in the t+1 moment, and by above-mentioned particle collection in the center that there is the region that probability optimal particle covers as wait capture center, target area;

(2-2-6) repeat above-mentioned steps (2-2-2) ~ step (2-2-5), obtain waiting to capture target there is probability optimal estimation and waiting to capture center, target area at moment t to subsequent time t+1;

(2-2-7) repeat step (2-1), what make camera lens aligning above-mentioned steps (2-2-6) treats crawl center, target area;

(2-2-8) local feature region waiting to capture target area of above-mentioned steps (2-2-7) is extracted;

(2-2-9) make other several video cameras in multiple cameras obtain to wait the sequence of video images capturing target, from multiple sequence of video images, extract local feature region (i.e. sift local feature region) respectively,

(2-2-10) that step (2-2-8) is extracted waits that the local feature region capturing target area mates with all local feature region that step (2-2-9) is extracted, and obtains the precise region waiting to capture target area of other several video cameras in multiple cameras;

(2-2-11) repeat above-mentioned steps (2-2-2) ~ step (2-2-10), the precise region waiting to capture target area every platform video camera being followed the tracks of respectively obtain, realize multiple-camera associating active tracing target.

Claims (1)

1., for a multiple-camera associating active tracing order calibration method for mechanical arm remote operating, it is characterized in that the method comprises the following steps:

(1) multiple video camera is placed in the upper left of mechanical arm, upper right and front, video camera is demarcated, obtain the Intrinsic Matrix M of multiple cameras respectively ₁, outer parameter matrix M ₂, distortion factor and module and carriage transformation matrix T ₁₂, T ₂₃, T ₃₄, concrete steps are as follows:

(1-1) set the coordinate of spatial point P in camera coordinate system as P (x _c, y _c, z _c), make spatial point P along the ray cast by photocentre on a plane of delineation, image coordinate system is set up in the plane of delineation, obtaining the projection coordinate of this spatial point P in image coordinate system is P (x, y), by this projection coordinate, the coordinate be expressed as in computer picture coordinate system is P (u, v), obtained by pinhole imaging system principle, the coordinate (x of spatial point P in camera coordinates _c, y _c, z _c) with the coordinate conversion relation of projection coordinate P (x, y) be: $\left\{\begin{array}{c}x=f\frac{x_C}{z_C}\\ y=f\frac{y_C}{z_C}\end{array}\right.,$ Wherein f is focal length of camera;

(1-2) image-generating unit is set, the x-axis of this image-generating unit in above-mentioned image coordinate system and the physical size on y-axis direction are respectively dx and dy, coordinate (the u of any one pixel under above computer image coordinate system in image-generating unit, there is following coordinate conversion relation in the coordinate (x, y) v) and under above-mentioned image coordinate system: $\left\{\begin{array}{c}u=\frac{x}{\mathrm{dx}}+u_0\\ v=\frac{y}{\mathrm{dy}}+v_0\end{array}\right.,$ Wherein, O (u ₀, v ₀) for being arranged on video camera primary optical axis any point at the imager coordinate of computer picture coordinate system, this imager coordinate and principal point coordinate;

(1-3) according to the coordinate conversion relation of above-mentioned steps (1-1) and step (1-2), the Intrinsic Matrix M of video camera is obtained ₁for: $M_1=\left[\begin{array}{ccc}\frac{f}{\mathrm{dx}}& 0& u_0\\ 0& \frac{f}{\mathrm{dy}}& v_0\\ 0& 0& 1\end{array}\right],$ Wherein, for the normalization focal length of focal length of camera f on the u axle of camera image plane coordinate system, for the normalization focal length of focal length of camera f on the v axle of camera image plane coordinate system;

(1-4) world coordinate system is set up, if the coordinate of spatial point P in world coordinate system is (x _w, y _w, z _w), there is following relation in the coordinate of spatial point P in camera coordinate system and the coordinate in world coordinate system: $\left[\begin{array}{c}{x}_C\\ y_C\\ z_C\end{array}\right]=\left[\begin{array}{cc}R& t\end{array}\right]\left[\begin{array}{c}{x}_W\\ y_W\\ z_W\\ 1\end{array}\right],$ Wherein, R is the unit orthogonal matrix of 3 × 3, and t is the D translation vector between camera coordinate system and world coordinate system, and definition [R t] is wherein external parameters of cameras matrix M ₂;

(1-5) according to the camera intrinsic parameter M that above-mentioned steps (1-3) and step (1-4) obtain ₁with outer parameter M ₂, the Transformation Relation of Projection obtained between the coordinate of spatial point P in world coordinate system and the projection coordinate of spatial point P in computer picture coordinate system is: P=M ₁m ₂;

(1-6) set the radial distortion parameter of video camera as k ₁, k ₂and k ₃, make k ₁, k ₂and k ₃meet following equation group: $\left\{\begin{array}{c}x=x_u(1+k_1{r}^2+k_2{r}^4+k_3{r}^6)\\ y=y_u(1+k_1{r}^2+k_2{r}^4+k_3{r}^6)\end{array}\right.,$ Solve this equation group, obtain the radial distortion parameter k of video camera ₁, k ₂and k ₃, wherein, (x, y) is the home position coordinate of spatial point P in image coordinate system, (x _u, y _u) the ideal position coordinate that obtained by image-forming principle for spatial point P;

If the tangential distortion parameter of video camera is p ₁and p ₂, make p ₁and p ₂meet following equation group: $\left\{\begin{array}{c}x=x_u+2p_1{x}_u{y}_u+p_2(r^2+2x_u^2)\\ y=y_u+2p_2{x}_u{y}_u+p_1(r^2+2y_u^2)\end{array}\right.,$ Solve this equation group, obtain the tangential distortion parameter p of video camera ₁and p ₂;

(1-7) radial distortion parameter obtained according to above-mentioned steps (1-6) is k ₁, k ₂and k ₃with tangential distortion parameter p ₁and p ₂, there is following transformational relation between the home position coordinate of spatial point P and ideal position coordinate:

$\left\{\begin{array}{c}x=x_u(1+k_1{r}^2+k_2{r}^4+k_3{r}^6)+2p_1{x}_u{y}_u+p_2(r^2+2x_u^2)\\ y=y_u(1+k_1{r}^2+k_2{r}^4+k_3{r}^6)+2p_2{x}_u{y}_u+p_1(r^2+2y_u^2)\end{array}\right.;$

Following transformational relation is there is between the ideal position coordinate of spatial point P and home position coordinate:

$\left\{\begin{array}{c}{x}_u=\frac{x-2p_1\mathrm{xy}+p_2(r^2+2x^2)}{1+k_1{r}^2+k_2{r}^4+k_3{r}^6}\\ y_u=\frac{y-2p_2\mathrm{x\; y}+p_1(r^2+{2x}^2)}{1+k_1{r}^2+k_2{r}^4+k_3{r}^6}\end{array}\right.;$

(1-8) travel through each video camera in multiple video camera, repeat step (1-1)-step (1-7), complete the demarcation of video camera;

(1-9) set spin matrix between first video camera in multiple video camera and second video camera as R ₁₂, the translation matrix between first video camera and second video camera is Tran ₁₂, obtain the module and carriage transformation matrix T between first video camera and second video camera ₁₂: T ₁₂=[R ₁₂tran ₁₂];

(1-10) travel through often any two video cameras in multiple video camera, repeat step (1-9), obtain the module and carriage transformation matrix T of multiple video camera ₁₂, T ₂₃, T ₃₄;

(2) make to wait to capture target through the calibrated multiple cameras active tracing of above-mentioned steps (1), specifically comprise the following steps:

(2-1) upper left of mechanical arm, upper right and front is placed in multiple through above-mentioned steps (1) calibrated video camera, if wait to capture target's center S in video camera imaging plane P _don be projected as S', calculate the distance d between S' and the center C of video camera imaging plane, setpoint distance threshold value th, the d that adjusts the distance judges, if d > is th, then sends an adjustment instruction to video camera, the direction making camera lens carry out distance d is reduced is rotated, until d≤th, if d≤th, then camera lens is made to keep origin-location;

(2-2) adopt particle filter tracking algorithm, video camera associating active tracing is waited to capture target, and concrete steps are as follows:

(2-2-1) make any video camera in multiple cameras obtain to wait the sequence of video images capturing target, to t in sequence of video images ₀the image in moment carries out manual mark, mark out wherein wait capture target area;

(2-2-2) wait that the center capturing target area is for initial point, produces a particle collection with above-mentioned wherein m is the particle number of particle set, m=1 ..., M, each particle represents one and treats the region that crawl target may exist; If particle collection in the motion Normal Distribution of particle, particle collection in each particle independent propagation, obtain the particle collection of t with the particle collection in t+1 moment

(2-2-3) t is established ₀moment particle collection reference histograms be reference histograms q ^*total L gray level, if t particle collection color histogram be q _t(x)=q _t(n; X), n=1,2...L, x are particle collection in particle, to t particle collection in each particle independent propagation after, the t+1 moment particle collection obtained observe, obtain particle collection in the color histogram of each particle region and reference histograms, calculate particle collection in Pasteur distance D between the color histogram of each particle region and reference histograms: definition particle weights are ω, and make ω=D, the value of N is 200;

(2-2-4) carrying out posterior probability calculating to above-mentioned particle weights, there is probability expectation E (x in what obtain t+1 moment particle _t+1): wherein, ω _t+1each particle weights in t+1 moment;

(2-2-5) by above-mentioned desired value E (x _t+1) as waiting that capturing target exists probability optimal estimation in the t+1 moment, and by above-mentioned particle collection in the center that there is the region that probability optimal particle covers as wait capture center, target area;

(2-2-6) repeat above-mentioned steps (2-2-2) ~ step (2-2-5), obtain waiting to capture target there is probability optimal estimation and waiting to capture center, target area at moment t to subsequent time t+1;

(2-2-7) repeat step (2-1), what make camera lens aligning above-mentioned steps (2-2-6) treats crawl center, target area;

(2-2-8) local feature region waiting to capture target area of above-mentioned steps (2-2-7) is extracted;

(2-2-9) make other several video cameras in multiple cameras obtain to wait the sequence of video images capturing target, from multiple sequence of video images, extract local feature region respectively;

(2-2-10) that step (2-2-8) is extracted waits that the local feature region capturing target area mates with all local feature region that step (2-2-9) is extracted, and obtains the precise region waiting to capture target area of other several video cameras in multiple cameras;

(2-2-11) repeat above-mentioned steps (2-2-2) ~ step (2-2-10), the precise region waiting to capture target area every platform video camera being followed the tracks of respectively obtain, realize multiple-camera associating active tracing target.