CN101320483A - Three-dimensional reconstruction method of rotating stereovision - Google Patents

Abstract

The present invention relates to a rotary solid vision three-dimension reconstructing method, comprising the steps as follows: firstly, a vidicon is demarcated manually to obtain a vidicon projection matrix when a rotary platform does not rotate; secondly, the vidicon projection matrix before the rotary platform rotates is decomposed to obtain a vidicon internal parameter matrix and an external parameter matrix while not rotating; thirdly, the vidicon projection matrix rotating for any angle is obtained in terms of rotation angle and the vidicon internal parameter matrix before rotating; fourthly, the space three-dimension information of an object is obtained by the information of an object picture which is shot from two angles or multiple angles and the vidicon projection matrix at the time. The present invention provides a rotary solid vision three-dimension reconstructing method which can greatly reduce demarcating times, improve calculation efficiency and reduce the cost.

Description

A kind of three-dimensional rebuilding method of rotating stereovision

Technical field

The present invention relates to three-dimensional rebuilding method, especially a kind of three-dimensional rebuilding method that is used for rotating stereovision based on computer vision.This method is mainly used in three-dimensional object surface reconstruct, has wide wide range of application, typically comprises workpiece sensing in the process industry, scene depth perception, reverse engineering, object dimensional scanning etc.

Background technology

Three-dimensional reconstruction is a vital task of computer vision.The task of three-dimensional reconstruction is that the two-dimensional image information from object obtains its three-dimensional spatial information.The application of three-dimensional reconstruction is very extensive, as medical domain, and public security field, meteorology, engineering or the like.The conventional method of three-dimensional reconstruction is to calculate the three-dimensional information of object in given space by the video camera projection matrix that reaches on two orientation from the object image information that different orientation obtained.But before object dimensional is rebuild, to obtain the projection matrix of video camera earlier, promptly video camera be demarcated.

If want object is finished the reconstruction of whole peripheral profile, must obtain the pictorial information of object by video camera in arbitrary orientation.According to traditional three-dimensional reconstruction algorithm, just must demarcate the video camera in each orientation, all be cumbersome and demarcate each time.The shortcoming of its existence: calibration cost height, very flexible.

Summary of the invention

Calibration cost height for the three-dimensional rebuilding method that overcomes the conventional stereo vision, the deficiency of very flexible the invention provides a kind of three-dimensional rebuilding method that can reduce the demarcation number of times greatly, improve counting yield, reduce cost and have the rotating stereovision of certain dirigibility.In the following method, suppose that camera intrinsic parameter does not change in whole process.

The technical solution adopted for the present invention to solve the technical problems is:

A kind of three-dimensional rebuilding method of rotating stereovision, described three-dimensional rebuilding method may further comprise the steps:

1), video camera is manual demarcates: an axle setting world coordinate system overlaps with turning axle, and a coordinate plane is parallel to the scaling board plane, and the scaling board plane parallel is in turning axle; Set the axle that overlaps with turning axle and be Y-axis, with the parallel plane coordinate plane of scaling board be XOY plane, obtain the world coordinates of centre point on the scaling board, mobile scaling board on the Z direction, obtain in the space not world coordinates, the world coordinates (X of setting space point at conplane a plurality of points _w, Y _w, Z _W), extract the image pixel coordinate figure obtain the center of circle on the scaling board (μ, v), there are following relation in the world coordinates of point and image coordinate:

$Z_c\left[\begin{array}{c}u\\ v\\ 1\end{array}\right]=M\·\left[\begin{array}{c}{X}_w\\ Y_w\\ Z_w\\ 1\end{array}\right]=\left[\begin{array}{cccc}{m}_{11}& m_{12}& m_{13}& m_{14}\\ m_{21}& m_{22}& m_{23}& m_{24}\\ m_{31}& m_{32}& m_{33}& m_{34}\end{array}\right]\·\left[\begin{array}{c}{X}_w\\ Y_w\\ Z_w\\ 1\end{array}\right]---\left(1\right)$

In the formula (1), M is the video camera projection matrix, is 3 * 4 matrixes; Z _cBe the coordinate components of point on Z direction under the camera coordinate system; Get following two equations by following formula:

$\left\{\begin{array}{c}{X}_w{m}_{11}+Y_w{m}_{12}+{Z_{w}m}_{13}+m_{14}-{\mathrm{uX}}_w{m}_{31}-u{Y}_w{m}_{32}-u{Z}_w{m}_{33}={\mathrm{um}}_{34}\\ X_w{m}_{21}+Y_w{m}_{22}+Z_w{m}_{23}+m_{24}-v{X}_w{m}_{31}-v{Y}_w{m}_{32}-{\mathrm{vZ}}_w{m}_{33}=v{m}_{34}\end{array}\right.---\left(2\right)$

Projection matrix M has 11 degree of freedom, does not calculate projection matrix M in conplane spatial point by six;

2), the video camera projection matrix before the rotation platform rotation is designated as M ¹, pass through M ¹The following formulate of relation of the world coordinates of the point that is reflected and image coordinate:

$Z_{c1}\left[\begin{array}{c}{u}_1\\ v_1\\ 1\end{array}\right]=M^1\·\left[\begin{array}{c}{X}_w\\ Y_w\\ Z_w\\ 1\end{array}\right]=\left[\begin{array}{cccc}{m}_{11}^1& m_{12}^1& m_{13}^1& m_{14}^1\\ m_{21}^1& m_{22}^1& m_{23}^1& m_{24}^1\\ m_{31}^1& m_{32}^1& m_{33}^1& m_{34}^1\end{array}\right]\·\left[\begin{array}{c}{X}_w\\ Y_w\\ Z_w\\ 1\end{array}\right]---\left(3\right)$

By to M ¹Take following formula to decompose:

Order: $\stackrel{\→}{m_1}={(m_{11}^1,m_{12}^1,m_{13}^1)}^T,$ $\stackrel{\→}{m_2}={(m_{21}^1,m_{22}^1{m}_{23}^1)}^T,$ $\stackrel{\→}{m_3}={(m_{31}^1,m_{32}^1{,m}_{33}^1)}^T.$

Then, $\stackrel{\&RightArrow;}{r_3}=m_{34}^1\stackrel{\&RightArrow;}{m_3},$ ${\mathrm{<figure-callout\; id="0"\; label="u"\; filenames="A20081006369200161.png,A20081006369200162.png"\; state="\{\{state\}\}">u</figure-callout>}}_0={\left(m_{34}^1\right)}^2{\stackrel{\&RightArrow;}{m_1}}^T\stackrel{\&RightArrow;}{m_3},$ ${\mathrm{<figure-callout\; id="0"\; label="v"\; filenames="A20081006369200161.png,A20081006369200162.png"\; state="\{\{state\}\}">v</figure-callout>}}_0={\left(m_{34}^1\right)}^2{\stackrel{\&RightArrow;}{m_2}}^T\stackrel{\&RightArrow;}{m_3},$ $a_x={\left(m_{34}^1\right)}^2|\stackrel{\&RightArrow;}{m_1}\&times;\stackrel{\&RightArrow;}{m_3}|,$ $a_y={\left(m_{34}^1\right)}^2|\stackrel{\&RightArrow;}{m_2}\&times;\stackrel{\&RightArrow;}{m_3}|,$ $\stackrel{\&RightArrow;}{r_1}=\frac{m_{34}^1}{a_x}(\stackrel{\&RightArrow;}{m_1}-u_0\stackrel{\&RightArrow;}{m_3}),$ $\stackrel{\&RightArrow;}{r_2}=\frac{m_{34}^1}{a_y}(\stackrel{\&RightArrow;}{m_2}-v_0\stackrel{\&RightArrow;}{m_3}),$ $t_z=m_{34}^1,$ $t_x=\frac{m_{34}^1}{a_x}(m_{14}^1-u_0),$ $t_y=\frac{m_{34}^1}{a_y}(m_{24}^1-v_0)$

$M^1=M_1\&CenterDot;M_2^1=\left[\begin{array}{cccc}{a}_x& 0& {\mathrm{<figure-callout\; id="0"\; label="u"\; filenames="A20081006369200161.png,A20081006369200162.png"\; state="\{\{state\}\}">u</figure-callout>}}_0& 0\\ 0& a_{\mathrm{<figure-callout\; id="0"\; label="y\; v"\; filenames="A20081006369200161.png,A20081006369200162.png"\; state="\{\{state\}\}">y</figure-callout>}}& v_{}{}_0& 0\\ 0& 0& 1& 0\end{array}\right]\&CenterDot;\left[\begin{array}{cc}{\stackrel{\&RightArrow;}{r_1}}^T& t_x\\ {\stackrel{\&RightArrow;}{r_2}}^T& t_y\\ {\stackrel{\&RightArrow;}{r_3}}^{\mathrm{<figure-callout\; id="0"\; label="T\; t\; z"\; filenames="A20081006369200161.png,A20081006369200162.png"\; state="\{\{state\}\}">T</figure-callout>}}& t_z{}_{}\\ {\stackrel{\&RightArrow;}{0}}^T& 1\end{array}\right]---\left(4\right)$

$=\left[\begin{array}{cccc}{m}_{11}^1& m_{12}^1& m_{13}^1& m_{14}^1\\ m_{21}^1& m_{22}^1& m_{23}^1& m_{24}^1\\ m_{31}^1& m_{32}^1& m_{33}^1& m_{34}^1\end{array}\right]$

M ₁Be camera intrinsic parameter matrix, M ₂ ¹External parameters of cameras matrix when not rotating;

3), the video camera projection matrix before the rotation platform rotation is:

$M^1=M_1\&CenterDot;M_2^1=\left[\begin{array}{cccc}{a}_x& 0& {\mathrm{<figure-callout\; id="0"\; label="u"\; filenames="A20081006369200161.png,A20081006369200162.png"\; state="\{\{state\}\}">u</figure-callout>}}_0& 0\\ 0& a_{\mathrm{<figure-callout\; id="0"\; label="y\; v"\; filenames="A20081006369200161.png,A20081006369200162.png"\; state="\{\{state\}\}">y</figure-callout>}}& v_{}{}_0& 0\\ 0& 0& 1& 0\end{array}\right]\&CenterDot;\left[\begin{array}{cc}{\stackrel{\&RightArrow;}{r_1}}^T& t_x\\ {\stackrel{\&RightArrow;}{r_2}}^T& t_y\\ {\stackrel{\&RightArrow;}{r_3}}^{\mathrm{<figure-callout\; id="0"\; label="T\; t\; z"\; filenames="A20081006369200161.png,A20081006369200162.png"\; state="\{\{state\}\}">T</figure-callout>}}& t_z{}_{}\\ {\stackrel{\&RightArrow;}{0}}^T& 1\end{array}\right]---\left(5\right)$

Behind the rotation platform rotation θ angle, the video camera projection matrix is

$M^2=\left[\begin{array}{cccc}{a}_x& 0& {\mathrm{<figure-callout\; id="0"\; label="u"\; filenames="A20081006369200161.png,A20081006369200162.png"\; state="\{\{state\}\}">u</figure-callout>}}_0& 0\\ 0& a_{\mathrm{<figure-callout\; id="0"\; label="y\; v"\; filenames="A20081006369200161.png,A20081006369200162.png"\; state="\{\{state\}\}">y</figure-callout>}}& v_{}{}_0& 0\\ 0& 0& 1& 0\end{array}\right]\&CenterDot;\left[\begin{array}{cc}{\stackrel{\&RightArrow;}{r_1}}^T& t_x\\ {\stackrel{\&RightArrow;}{r_2}}^T& t_y\\ {\stackrel{\&RightArrow;}{r_3}}^{\mathrm{<figure-callout\; id="0"\; label="T\; t\; z"\; filenames="A20081006369200161.png,A20081006369200162.png"\; state="\{\{state\}\}">T</figure-callout>}}& t_z{}_{}\\ {\stackrel{\&RightArrow;}{0}}^T& 1\end{array}\right]\&CenterDot;\left[\begin{array}{cccc}\cos \mathrm{\&theta;}& 0& -\sin \mathrm{\&theta;}& 0\\ 0& 1& 0& 0\\ \sin \mathrm{\&theta;}& 0& \cos \mathrm{\&theta;}& 0\\ 0& 0& 0& 1\end{array}\right]---\left(6\right)$

$=M^1\&CenterDot;\left[\begin{array}{cccc}\cos \mathrm{\&theta;}& 0& -\sin \mathrm{\&theta;}& 0\\ 0& 1& 0& 0\\ \sin \mathrm{\&theta;}& 0& \cos \mathrm{\&theta;}& 0\\ 0& 0& 0& 1\end{array}\right]$

4), the point in the space under world coordinate system coordinate and the relation of the pixel coordinates of image represent by following formula:

$Z_{c2}\left[\begin{array}{c}{u}_2\\ v_2\\ 1\end{array}\right]=M^1\&CenterDot;\left[\begin{array}{cccc}\cos \mathrm{\&theta;}& 0& -\sin \mathrm{\&theta;}& 0\\ 0& 1& 0& 0\\ \sin \mathrm{\&theta;}& 0& \cos \mathrm{\&theta;}& 0\\ 0& 0& 0& 1\end{array}\right]\left[\begin{array}{c}{X}_w\\ Y_w\\ Z_w\\ 1\end{array}\right]---\left(7\right)$

$=M^2\&CenterDot;\left[\begin{array}{c}{X}_w\\ Y_w\\ Z_w\\ 1\end{array}\right]$

In the following formula (7), θ is the unspecified angle that comprises 0 degree, when be 0 be rotation when spending before; Different two θ obtain about X by assignment _w, Y _w, Z _wFour equations, thereby obtain the some three dimensional space coordinate under world coordinate system, further obtain the three-dimensional coordinate of the point on the body outline, realize whole image Reconstruction.

Technical conceive of the present invention is: under rotating stereovision, just can finish by a video camera and object to be carried out 360 ° comprehensive shooting, and each camera coordinate system same turning axle in the space is rotated.Can be described by projection matrix.Below two kinds of experimental techniques all be used for calculating projection matrix, and then finish reconstruction.

Rotating stereovision is a special case of conventional stereo vision.It is under the situation of object certain root axle rotation in the space, with a fixed cameras object is carried out shooting around a week.Perhaps certain the root axle in the camera intrinsic space is rotated under the situation, to carry out the stereoscopic vision of taking in a week at its object within the vision.

With respect to traditional manual algorithm of demarcating, the present invention is many simply and easily, has shortened the time of calculating greatly, for the raising of later counting yield also lays the foundation.

Based on the reconstruction of rotating stereovision, just can obtain the video camera projection matrix that video camera is in arbitrary orientation as long as video camera demarcated object from the both direction, can reduce the demarcation number of times like this, reduce cost.

Beneficial effect of the present invention mainly shows: 1, demarcate number of times for once, shortened the time of calculating greatly;

2, reduced cost.

Description of drawings

Fig. 1 is the scaling board synoptic diagram that camera calibration is used.

Fig. 2 is a coordinate figure of rebuilding eight summits of carton that obtain.

Fig. 3 is the point diagram of the unique point after rebuilding.

Fig. 4 is the synoptic diagram after rebuilding.

Fig. 5 is used for the picture of the scaling board of calibrating camera

Embodiment

Below in conjunction with accompanying drawing the present invention is further described.

With reference to Fig. 1～Fig. 5, a kind of three-dimensional rebuilding method of rotating stereovision, described three-dimensional rebuilding method may further comprise the steps:

1), video camera is manual demarcates: an axle setting world coordinate system overlaps with turning axle, and a coordinate plane is parallel to scaling board plane (the scaling board plane parallel is in turning axle).Set, the axle that overlaps with turning axle is a Y-axis herein, with the parallel plane coordinate plane of scaling board be XOY plane.Scaling board such as appendix are shown in Figure 5, so just can obtain the world coordinates of centre point on the scaling board.Mobile scaling board on the Z direction just can obtain in the space not the world coordinates at conplane a plurality of points.World coordinates (the X of setting space point _w, Y _w, Z _W), extract the image pixel coordinate figure obtain the center of circle on the scaling board (μ, v), there are following relation in the world coordinates of point and image coordinate:

$Z_c\left[\begin{array}{c}u\\ v\\ 1\end{array}\right]=M\&CenterDot;\left[\begin{array}{c}{X}_w\\ Y_w\\ Z_w\\ 1\end{array}\right]=\left[\begin{array}{cccc}{m}_{11}& m_{12}& m_{13}& m_{14}\\ m_{21}& m_{22}& m_{23}& m_{24}\\ m_{31}& m_{32}& m_{33}& m_{34}\end{array}\right]\&CenterDot;\left[\begin{array}{c}{X}_w\\ Y_w\\ Z_w\\ 1\end{array}\right]---\left(1\right)$

M is the video camera projection matrix in the formula, is 3 * 4 matrixes.Z _cBe the coordinate components of point on Z direction under the camera coordinate system.Can get following two equations by following formula:

$\left\{\begin{array}{c}{X}_w{m}_{11}+Y_w{m}_{12}+{Z_{w}m}_{13}+m_{14}-{\mathrm{uX}}_w{m}_{31}-u{Y}_w{m}_{32}-u{Z}_w{m}_{33}={\mathrm{um}}_{34}\\ X_w{m}_{21}+Y_w{m}_{22}+Z_w{m}_{23}+m_{24}-v{X}_w{m}_{31}-v{Y}_w{m}_{32}-{\mathrm{vZ}}_w{m}_{33}=v{m}_{34}\end{array}\right.---\left(2\right)$

N point can obtain 2N equation.Projection matrix M has 11 degree of freedom, can not calculate projection matrix M in conplane spatial point by six.Owing to there is error, can calculate M (P57 in the reference literature 1 herein) with least square method in the reality by N＞6 point.

2), obtain camera intrinsic parameter matrix and the outer parameter matrix when not rotating.The top Metzler matrix that obtains is the video camera projection matrix before the rotation platform rotation.This matrix is designated as M ¹Then pass through M ¹The following formulate of relation of the world coordinates of the point that is reflected and image coordinate:

$Z_{c1}\left[\begin{array}{c}{u}_1\\ v_1\\ 1\end{array}\right]=M^1\&CenterDot;\left[\begin{array}{c}{X}_w\\ Y_w\\ Z_w\\ 1\end{array}\right]=\left[\begin{array}{cccc}{m}_{11}^1& m_{12}^1& m_{13}^1& m_{14}^1\\ m_{21}^1& m_{22}^1& m_{23}^1& m_{24}^1\\ m_{31}^1& m_{32}^1& m_{33}^1& m_{34}^1\end{array}\right]\&CenterDot;\left[\begin{array}{c}{X}_w\\ Y_w\\ Z_w\\ 1\end{array}\right]---\left(3\right)$

By to M ¹Take following formula to decompose.

Order: $\stackrel{\&RightArrow;}{m_1}={(m_{11}^1,m_{12}^1,m_{13}^1)}^T,$ $\stackrel{\&RightArrow;}{m_2}={(m_{21}^1,m_{22}^1,m_{23}^1)}^T,$ $\stackrel{\&RightArrow;}{m_3}={(m_{31}^1,m_{32}^1{,m}_{33}^1)}^T.$

Then, $\stackrel{\&RightArrow;}{r_3}=m_{34}^1\stackrel{\&RightArrow;}{m_3},$ $u_0={\left(m_{34}^1\right)}^2{\stackrel{\&RightArrow;}{m_1}}^T\stackrel{\&RightArrow;}{m_3},$ $v_0={\left(m_{34}^1\right)}^2{\stackrel{\&RightArrow;}{m_2}}^T\stackrel{\&RightArrow;}{m_3},$ $a_x={\left(m_{34}^1\right)}^2|\stackrel{\&RightArrow;}{m_1}\&times;\stackrel{\&RightArrow;}{m_3}|,$ $a_y={\left(m_{34}^1\right)}^2|\stackrel{\&RightArrow;}{m_2}\&times;\stackrel{\&RightArrow;}{m_3}|,$ $\stackrel{\&RightArrow;}{r_1}=\frac{m_{34}^1}{a_x}(\stackrel{\&RightArrow;}{m_1}-u_0\stackrel{\&RightArrow;}{m_3}),$ $\stackrel{\&RightArrow;}{r_2}=\frac{m_{34}^1}{a_y}(\stackrel{\&RightArrow;}{m_2}-v_0\stackrel{\&RightArrow;}{m_3}),$ $t_z=m_{34}^1,$ $t_x=\frac{m_{34}^1}{a_x}(m_{14}^1-u_0),$ $t_y=\frac{m_{34}^1}{a_y}(m_{24}^1-v_0)$

$M^1=M_1\&CenterDot;M_2^1=\left[\begin{array}{cccc}{a}_x& 0& u_0& 0\\ 0& a_{\mathrm{<figure-callout\; id="0"\; label="y\; v"\; filenames="A20081006369200161.png,A20081006369200162.png"\; state="\{\{state\}\}">y</figure-callout>}}& v_{}{}_0& 0\\ 0& 0& 1& 0\end{array}\right]\&CenterDot;\left[\begin{array}{cc}{\stackrel{\&RightArrow;}{r_1}}^T& t_x\\ {\stackrel{\&RightArrow;}{r_2}}^T& t_y\\ {\stackrel{\&RightArrow;}{r_3}}^T& t_{\mathrm{<figure-callout\; id="0"\; label="z"\; filenames="A20081006369200161.png,A20081006369200162.png"\; state="\{\{state\}\}">z</figure-callout>}}\\ {\stackrel{\&RightArrow;}{0}}^T& 1\end{array}\right]---\left(4\right)$

$=\left[\begin{array}{cccc}{m}_{11}^1& m_{12}^1& m_{13}^1& m_{14}^1\\ m_{21}^1& m_{22}^1& m_{23}^1& m_{24}^1\\ m_{31}^1& m_{32}^1& m_{33}^1& m_{34}^1\end{array}\right]$

M ¹Be camera intrinsic parameter matrix, M ₂ ¹External parameters of cameras matrix when not rotating

3), obtain and rotated video camera projection matrix at any angle.

Video camera projection matrix before the rotation platform rotation is:

$M^1=M_1\&CenterDot;M_2^1=\left[\begin{array}{cccc}{a}_x& 0& u_0& 0\\ 0& a_{\mathrm{<figure-callout\; id="0"\; label="y\; v"\; filenames="A20081006369200161.png,A20081006369200162.png"\; state="\{\{state\}\}">y</figure-callout>}}& v_{}{}_0& 0\\ 0& 0& 1& 0\end{array}\right]\&CenterDot;\left[\begin{array}{cc}{\stackrel{\&RightArrow;}{r_1}}^T& t_x\\ {\stackrel{\&RightArrow;}{r_2}}^T& t_y\\ {\stackrel{\&RightArrow;}{r_3}}^T& t_{\mathrm{<figure-callout\; id="0"\; label="z"\; filenames="A20081006369200161.png,A20081006369200162.png"\; state="\{\{state\}\}">z</figure-callout>}}\\ {\stackrel{\&RightArrow;}{0}}^T& 1\end{array}\right]---\left(5\right)$

Behind the rotation platform rotation θ angle, the video camera projection matrix is

$M^2=\left[\begin{array}{cccc}{a}_x& 0& u_0& 0\\ 0& a_{\mathrm{<figure-callout\; id="0"\; label="y\; v"\; filenames="A20081006369200161.png,A20081006369200162.png"\; state="\{\{state\}\}">y</figure-callout>}}& v_{}{}_0& 0\\ 0& 0& 1& 0\end{array}\right]\&CenterDot;\left[\begin{array}{cc}{\stackrel{\&RightArrow;}{r_1}}^T& t_x\\ {\stackrel{\&RightArrow;}{r_2}}^T& t_y\\ {\stackrel{\&RightArrow;}{r_3}}^{\mathrm{<figure-callout\; id="0"\; label="T\; t\; z"\; filenames="A20081006369200161.png,A20081006369200162.png"\; state="\{\{state\}\}">T</figure-callout>}}& t_z{}_{}\\ {\stackrel{\&RightArrow;}{0}}^T& 1\end{array}\right]\&CenterDot;\left[\begin{array}{cccc}\cos \mathrm{\&theta;}& 0& -\sin \mathrm{\&theta;}& 0\\ 0& 1& 0& 0\\ \sin \mathrm{\&theta;}& 0& \cos \mathrm{\&theta;}& 0\\ 0& 0& 0& 1\end{array}\right]---\left(6\right)$

$=M^1\&CenterDot;\left[\begin{array}{cccc}\cos \mathrm{\&theta;}& 0& -\sin \mathrm{\&theta;}& 0\\ 0& 1& 0& 0\\ \sin \mathrm{\&theta;}& 0& \cos \mathrm{\&theta;}& 0\\ 0& 0& 0& 1\end{array}\right]$

4), obtain the space three-dimensional information of object by the pictorial information of two angles or the captured object of a plurality of angles.Point in the space under world coordinate system coordinate and the relation of the pixel coordinates of image can represent by following formula:

$Z_{c2}\left[\begin{array}{c}{u}_2\\ v_2\\ 1\end{array}\right]=M^1\&CenterDot;\left[\begin{array}{cccc}\cos \mathrm{\&theta;}& 0& -\sin \mathrm{\&theta;}& 0\\ 0& 1& 0& 0\\ \sin \mathrm{\&theta;}& 0& \cos \mathrm{\&theta;}& 0\\ 0& 0& 0& 1\end{array}\right]\left[\begin{array}{c}{X}_w\\ Y_w\\ Z_w\\ 1\end{array}\right]---\left(7\right)$

$=M^2\&CenterDot;\left[\begin{array}{c}{X}_w\\ Y_w\\ Z_w\\ 1\end{array}\right]$

θ is the unspecified angle that comprises 0 degree, when be 0 be rotation when spending before.Just can obtain by being worth different two θ about X _w, Y _w, Z _wFour equations.Thereby obtain the some three dimensional space coordinate under world coordinate system, further obtain the three-dimensional coordinate of the point on the body outline, the three-dimensional information of whole object just can obtain.

The three-dimensional rebuilding method of present embodiment may further comprise the steps:

1), video camera is carried out manual demarcation;

2), obtain camera intrinsic parameter matrix and the outer parameter matrix when not rotating;

3), obtain and rotated video camera projection matrix at any angle;

4), obtain the space three-dimensional information of object by the pictorial information of two angles or the captured object of a plurality of angles.

The experimentation of present embodiment is:

The employed standard component of camera calibration is a scaling board, with reference to Fig. 1.The space element that is used to calculate the video camera projection matrix is the center of circle on the scaling board.By mobile platform, can obtain not in a plurality of spatial point in same plane.This experiment is set, and is the world coordinate system Y direction perpendicular to the direction of mobile platform, is directed downwards; The horizontal of mobile platform is the world coordinate system X-direction, and forward to the right; The vertical of mobile platform is the world coordinate system Z-direction, and forward is away from camera direction.Whole calibrating procedure is divided into that data are obtained and data processing.

(1) data are obtained

1. rotate rotation platform, guarantee the zero graduation alignment mark of rotation platform, can guarantee that like this scaling board plane is perpendicular to the Z axle.

2. scaling board is fixed on the objective table, guarantees that the scaling board bottom is concordant with the objective table bottom surface, guarantee that the scaling board left hand edge is concordant with the objective table left hand edge.So just can determine to demarcate on the scaling board world coordinates of the round heart.

3. by adjusting mobile platform, make scaling board within sweep of the eye at video camera.Vertical initial position setting of mobile platform is for after moving forward and backward 40mm, and scaling board is all in the camera coverage scope.And scaling board can just be taken the whole calibrating plate when video camera is nearest.

4. adjust focal length of camera, make mobile platform scaling board image when initial position the most clear.

5. each moves 10mm before and after initial position and mobile platform are on longitudinal direction to obtain scaling board, and 20mm is totally 5 width of cloth images.

6. mobile platform is returned to initial position, 5 ° of control rotation platform rotations, repeating step content 5..

(2) data processing

In the step, 10 width of cloth scaling board images have been obtained altogether in the above.5 width of cloth scaling board images before the rotation are classified as first group, are used to calculate the preceding projection matrix of rotation.5 width of cloth scaling board images after the rotation are classified as another group, are used to calculate the video camera projection matrix when having rotated 5 °.

To preceding set of diagrams picture, every width of cloth extracts 7 * 11 coordinates of demarcating the round heart; Five width of cloth images can obtain the image coordinate of 385 spatial point, and these 385 spatial point are not at grade.Obtain rotation video camera projection matrix M before by least square method ¹Five width of cloth images after the rotation are also carried out respective handling, can draw postrotational video camera projection matrix M ²

The video camera projection matrix that obtains is as follows:

Table 1 video camera projection matrix M ¹

Table 2 video camera projection matrix M ²

After obtaining projection matrix, do the feasibility of the reconstruction proof algorithm of a carton, prove that its error also is within allowed band.

Below be proof of algorithm and verify error

From two aspects algorithm is verified.The one, verify by the spatial point of known D coordinates value, promptly, see whether it equates with actual value by the three-dimensional coordinate of image information recovery point.The 2nd, verify by the simple square of known length, promptly recover the angle point of simple square earlier, calculate the distance of angle point then, compare with actual value.

For a situation that coordinate axis overlaps with turning axle of first kind of world coordinate system, we select for use is two centre points on the scaling board.Table 3 is experimental results:

Table 3

For coming verification algorithm by simple square.We have selected the rectangular parallelepiped carton for use; Used projection matrix is resulting during the front is demarcated.

The cylindrical object that supports carton below is in order to allow carton fully within camera coverage.Earlier material object is carried out the picture collection, carry out shooting, can obtain 72 width of cloth pictorial informations altogether about material object every 5 one weeks of degree.Our task is only rebuild eight summits of rectangular parallelepiped carton, and used initial alignment matrix is the video camera projection matrix of table 1 and table 2.According to M in the formula (7) ₂With M ₁Relation calculate the video camera projection matrix M of rotation after arbitrarily angled ₂To the carton summit that in two width of cloth figure, all occurs, calculate its three-dimensional coordinate.The coordinate on summit equals each mean value of rebuilding the three-dimensional point coordinate that obtains.

Table 4 is coordinate figures of rebuilding eight summits of carton that obtain.

name	The value of X(mm)	The value of Y(mm)	The value of Z(mm)
				Point1	-40.034	-87.892	-83.684
Point2	-39.179	-37.57	-84.615
				Point3	-41.726	-85.182	51.852
Point4	-41.215	-36.214	51.56
				Point5	149.64	-86.893	-78.562
Point6	149.84	-35.912	-78.383
				Point7	145.52	-87.671	56.894
Point8	145.88	-36.512	59.107

Table 4

Table 5 is the length on 12 limits calculating by eight points rebuilding above and the comparison and the relative error thereof of physical length.It can be seen from the table, its relative error proves the feasibility of algorithm within allowed band.

name	Actual value(mm)	Reconstructed value(mm)	Relative error
				DP12
	50	50.338	0.676％
				DP13	138	135.57	1.76％
DP15	190	189.74	0.137％
				DP42	138	136.2	1.30％
DP43	50	48.971	2.06％
				DP48	190	187.25	1.45％
DP62	190	189.13	0.458％
				DP65
	50	50.981	1.96％
				DP68	138	137.55	0.326％
DP73	190	187.33	1.41％
				DP75	138	135.52	1.80％
DP78	50	51.209	2.41％

Table 5

Fig. 2 is eight profile diagrams that the summit couples together with straight line that reconstruction is obtained.

One complex object is rebuild: verified the error range and algorithm feasibility of projection matrix by the front after, can be with a complex object with rebuilt.For example the clock and watch image is carried out three-dimensional reconstruction.

Before experiment, with correction fluid some unique points of mark on clock and watch, these promises can be described the profile of clock out earlier.

Idiographic flow is:

(1) marks some unique points (profile that can manifest clock) at the Zhong Shangyong correction fluid

(2) these points are carried out mark (conveniently coupling together with bundle of lines point)

(3) data are obtained (carrying out the picture that 360 degree obtain clock)

(4) rebuild particular point, as Fig. 3

(5) connect these points with straight line, show the profile of clock

Figure after the reconstruction such as accompanying drawing 4.

Claims (1)

1, a kind of three-dimensional rebuilding method of rotating stereovision is characterized in that: described three-dimensional rebuilding method may further comprise the steps:

1), video camera is manual demarcates: an axle setting world coordinate system overlaps with turning axle, and a coordinate plane is parallel to the scaling board plane, and the scaling board plane parallel is in turning axle; Set the axle that overlaps with turning axle and be Y-axis, with the parallel plane coordinate plane of scaling board be XOY plane, obtain the world coordinates of centre point on the scaling board, mobile scaling board on the Z direction, obtain in the space not world coordinates, the world coordinates (X of setting space point at conplane a plurality of points _w, Y _w, Z _W), extract the image pixel coordinate figure obtain the center of circle on the scaling board (μ, v), there are following relation in the world coordinates of point and image coordinate:

$Z_c\left[\begin{array}{c}u\\ v\\ 1\end{array}\right]=M\&CenterDot;\left[\begin{array}{c}{X}_w\\ Y_w\\ Z_w\\ 1\end{array}\right]=\left[\begin{array}{cccc}{m}_{11}& m_{12}& m_{13}& m_{14}\\ m_{21}& m_{22}& m_{23}& m_{24}\\ m_{31}& m_{32}& m_{33}& m_{34}\end{array}\right]\&CenterDot;\left[\begin{array}{c}{X}_w\\ Y_w\\ Z_w\\ 1\end{array}\right]---\left(1\right)$

In the formula (1), M is the video camera projection matrix, is 3 * 4 matrixes; Z _cBe the coordinate components of point on Z direction under the camera coordinate system; Get following two equations by following formula:

$\left\{\begin{array}{c}{X}_w{m}_{11}+Y_w{m}_{12}+Z_w{m}_{13}+m_{14}-u{X}_w{m}_{31}-u{Y}_w{m}_{32}-u{Z}_w{m}_{33}={\mathrm{um}}_{34}\\ X_w{m}_{21}+Y_w{m}_{22}+Z_w{m}_{23}+m_{24}-v{X}_w{m}_{31}-v{Y}_w{m}_{32}-v{Z}_w{m}_{33}={\mathrm{vm}}_{34}\end{array}\right.---\left(2\right)$

Projection matrix M has 11 degree of freedom, does not calculate projection matrix M in conplane spatial point by six;

2), the video camera projection matrix before the rotation platform rotation is designated as M ¹, pass through M ¹The following formulate of relation of the world coordinates of the point that is reflected and image coordinate:

$Z_{c1}\left[\begin{array}{c}{u}_1\\ v_1\\ 1\end{array}\right]=M^1\&CenterDot;\left[\begin{array}{c}{X}_w\\ Y_w\\ Z_w\\ 1\end{array}\right]=\left[\begin{array}{cccc}{m}_{11}^1& m_{12}^1& m_{13}^1& m_{14}^1\\ m_{21}^1& m_{22}^1& m_{23}^1& m_{24}^1\\ m_{31}^1& m_{32}^1& m_{33}^1& m_{34}^1\end{array}\right]\&CenterDot;\left[\begin{array}{c}{X}_w\\ Y_w\\ Z_w\\ 1\end{array}\right]---\left(3\right)$

By to M ¹Take following formula to decompose:

Order: $\stackrel{\&RightArrow;}{m_1}={(m_{11}^1,m_{12}^1,m_{13}^1)}^T,$ $\stackrel{\&RightArrow;}{m_2}={(m_{21}^1,m_{22}^1,m_{23}^1)}^T,$ $\stackrel{\&RightArrow;}{m_3}={(m_{31}^1,m_{32}^1,m_{33}^1)}^T.$

Then, ${\stackrel{\&RightArrow;}{r}}_3=m_{34}^1{\stackrel{\&RightArrow;}{m}}_{3,}$ $u_0={\left(m_{34}^1\right)}^2{\stackrel{\&RightArrow;}{m_1}}^T\stackrel{\&RightArrow;}{m_3},$ $v_0={\left(m_{34}^1\right)}^2{\stackrel{\&RightArrow;}{m_2}}^T\stackrel{\&RightArrow;}{m_3},$ $a_x={\left(m_{34}^1\right)}^2|\stackrel{\&RightArrow;}{m_1}\&times;\stackrel{\&RightArrow;}{m_3}|,$ $a_y={\left(m_{34}^1\right)}^2|\stackrel{\&RightArrow;}{m_2}\&times;\stackrel{\&RightArrow;}{m_3}|,$ $\stackrel{\&RightArrow;}{r_1}=\frac{m_{34}^1}{a_x}(\stackrel{\&RightArrow;}{m_1}-u_0\stackrel{\&RightArrow;}{m_3}),$ $\stackrel{\&RightArrow;}{r_2}=\frac{m_{34}^1}{a_y}(\stackrel{\&RightArrow;}{m_2}-v_0\stackrel{\&RightArrow;}{m_3}),$ $t_z=m_{34}^1,$ $t_x=\frac{m_{34}^1}{a_x}(m_{14}^1-u_0),$ $t_y=\frac{m_{34}^1}{a_y}(m_{24}^1-u_0),$

$M^1=M_1\&CenterDot;M_2^1=\left[\begin{array}{cccc}{a}_x& 0& u_0& 0\\ 0& a_y& v_0& 0\\ 0& 0& 1& 0\end{array}\right]\&CenterDot;\left[\begin{array}{cc}{\stackrel{\&RightArrow;}{r_1}}^T& t_x\\ {\stackrel{\&RightArrow;}{r_2}}^T& t_y\\ {\stackrel{\&RightArrow;}{r_3}}^T& t_z\\ {\stackrel{\&RightArrow;}{0}}^T& 1\end{array}\right]---\left(4\right)$

$=\left[\begin{array}{cccc}{m}_{11}^1& m_{12}^1& m_{13}^1& m_{14}^1\\ m_{21}^1& m_{22}^1& m_{23}^1& m_{24}^1\\ m_{31}^1& m_{32}^1& m_{33}^1& m_{34}^1\end{array}\right]$

M ₁Be camera intrinsic parameter matrix, M ₂ ¹External parameters of cameras matrix when not rotating;

3), the video camera projection matrix before the rotation platform rotation is:

$M^1=M_1\&CenterDot;M_2^1=\left[\begin{array}{cccc}{a}_x& 0& u_0& 0\\ 0& a_y& v_0& 0\\ 0& 0& 1& 0\end{array}\right]\&CenterDot;\left[\begin{array}{cc}{\stackrel{\&RightArrow;}{r_1}}^T& t_x\\ {\stackrel{\&RightArrow;}{r_2}}^T& t_y\\ {\stackrel{\&RightArrow;}{r_3}}^T& t_z\\ {\stackrel{\&RightArrow;}{0}}^T& 1\end{array}\right]---\left(5\right)$

Behind the rotation platform rotation θ angle, the video camera projection matrix is

$M^2=\left[\begin{array}{cccc}{a}_x& 0& u_0& 0\\ 0& a_y& v_0& 0\\ 0& 0& 1& 0\end{array}\right]\&CenterDot;\left[\begin{array}{cc}{\stackrel{\&RightArrow;}{r_1}}^T& t_x\\ {\stackrel{\&RightArrow;}{r_2}}^T& t_y\\ {\stackrel{\&RightArrow;}{r_3}}^T& t_z\\ {\stackrel{\&RightArrow;}{0}}^T& 1\end{array}\right]\&CenterDot;\left[\begin{array}{cccc}\cos \mathrm{\&theta;}& 0& -\sin \mathrm{\&theta;}& 0\\ 0& 1& 0& 0\\ \sin \mathrm{\&theta;}& 0& \cos \mathrm{\&theta;}& 0\\ 0& 0& 0& 1\end{array}\right]---\left(6\right)$

${=M}^1\&CenterDot;\left[\begin{array}{cccc}\cos \mathrm{\&theta;}& 0& -\sin \mathrm{\&theta;}& 0\\ 0& 1& 0& 0\\ \sin \mathrm{\&theta;}& 0& \cos \mathrm{\&theta;}& 0\\ 0& 0& 0& 1\end{array}\right]$

4), the point in the space under world coordinate system coordinate and the relation of the pixel coordinates of image represent by following formula:

$Z_{c2}\left[\begin{array}{c}{u}_2\\ v_2\\ 1\end{array}\right]=M^1\&CenterDot;\left[\begin{array}{cccc}\cos \mathrm{\&theta;}& 0& -\sin \mathrm{\&theta;}& 0\\ 0& 1& 0& 0\\ \sin \mathrm{\&theta;}& 0& \cos \mathrm{\&theta;}& 0\\ 0& 0& 0& 1\end{array}\right]\left[\begin{array}{c}{X}_w\\ Y_w\\ Z_w\\ 1\end{array}\right]---\left(7\right)$

$=M^2\&CenterDot;\left[\begin{array}{c}{X}_w\\ Y_w\\ Z_w\\ 1\end{array}\right]$

In the following formula (7), θ is the unspecified angle that comprises 0 degree, when be 0 be rotation when spending before; Different two θ obtain about X by assignment _w, Y _w, Z _wFour equations, thereby obtain the some three dimensional space coordinate under world coordinate system, further obtain the three-dimensional coordinate of the point on the body outline, realize whole image Reconstruction.

Images