CN102682467A - Plane- and straight-based three-dimensional reconstruction method - Google Patents

Abstract

The invention relates to a plane- and straight-based three-dimensional reconstruction method, which is used for linearly reconstructing affine and metric geometry of a scene according to images acquired by two common moving cameras by using any one plane and two pairs of mutually orthorhombic straight lines, which are not on the plane, in the scene. The plane- and straight-based three-dimensional reconstruction method comprises the following steps of: according to a relation between infinite homography and affine reconstruction and absolute conic image and metric reconstruction, obtaining the infinite homography between two images by using transformation between limited plane homography and infinite homography and further carrying out affine reconstruction; and obtaining an absolute conic image by using the transformation between the orthogonality of two lines and the absolute conic, and further carrying out metric reconstruction. In addition, five internal parameters of each camera can be linearly solved in the metric reconstruction process.

Description

Three-dimensional rebuilding method based on plane and straight line

Technical field

The invention belongs to the computer vision research field; It is a kind of three-dimensional rebuilding method based on plane and straight line; Utilize in the scene any plane and the mutually orthogonal parallel lines of two couple on this plane not, how much of the affine and tolerance of the re-construct that the image that obtains from two general camera motions is linear.

Background technology

The three-dimensional reconstruction technology is one of important topic of computer vision research.So-called three-dimensional reconstruction just is meant the spatial geometric shape that recovers three-dimensional body from the image of three-dimensional body; Document " Computer Vision:A Modern Approach " (David A.Forsyth; Jean Ponce Faugeras work; Forestry Yan, Wang Hong etc. translate. Electronic Industry Press, 2004) show that best tolerance reconstruction result also will differ a similarity transformation with real object.For the different visual task, need different reconstruction standards, concrete, several kinds of dissimilar reconstruct are for how much: projective reconstruction, confirm the intersection point on straight line and plane etc.; Affine reconstruction is calculated mid point and the centre of form of a point set and the collimation of two lines etc. at 2; The orthogonality of line line and line face and the ratio of any line segment etc. are confirmed in tolerance reconstruct.

Faugeras " Stratification of three-dimensional vision:projective; affine; and metric representations " (Journal Optical Society of America; 1995,12 (3)) at first propose three-dimensional reconstruction was resolved into for 3 steps: obtain projection according to the image corresponding point and rebuild, calculate video camera matrix under the projection meaning; Rebuild SPATIAL CALCULATION plane at infinity position in projection, obtain video camera matrix under the affine meaning; In affine space, further impose restriction to confirm the picture of absolute conic, rebuild thereby obtain tolerance.Can know that based on how much of video camera imagings and two dimensional image character two dimensional image essence is exactly the projective space image, the calculating affine geometry obtains the infinity list exactly and answers, and computation measure is exactly for how much a picture of confirming absolute conic.

Document " The impossibility of affine reconstruction from perspective image pairs obtained by a translating camera with varying parameters " (Hu ZhanYi; Wu FuChao, Proceedings of the 5 ^ThAsian Conference on Computer Vision, Melbourne, Australia, 2002,2:782-784) point out in that have no under the situation of scene information can not affine reconstruction, more can not tolerance reconstruct; Document " A Linear Method for Rank 2 Fundamental Matrix with Noncompulsory Constraint " (Shiming Wang; Juan Wang; Yue Zhao; IEEE Robotics and Biomimetics 2009) provided that to utilize the fundamental matrix order be that 2 constraint linearity solves fundamental matrix, obtained the projective reconstruction of object; Document " Linear Solving the Infinite Homography Matrix with Epipole " (Yue Zhao; Shimin Wang; Juan Wang; Hongqiang Ding, The International Conference on Computer modeling and simulation 2010.2010-2) utilize limit constraint solving infinite distance list to answer, obtain the affine reconstruction of object.Document " based on the affine reconstruction of plane and straight line " (Hu Zhanyi; Wang Guanghui, Wu Fuchao, Chinese journal of computers; 26 (6); 2003 (6) .722-728) provided under variable parameter model, contained two Zhang Ping's parallel planes in a sheet of planar and pair of parallel straight line or the scene if contain in the scene, then can be from two translation viewpoint lower linear scene is carried out affine reconstruction; Point out simultaneously if contain pair of parallel plane and pair of parallel straight line in the scene, then can be from the affine geometry of the linear re-construct of general motion viewpoint.

Summary of the invention

The present invention proposes a kind of under the camera model of preset parameter; Utilize in the scene any plane and the mutually orthogonal parallel lines of two couple on this plane not, the method that the affine and tolerance of the re-construct that the image that obtains from two general camera motions is linear is how much.This method only needs video camera to take the affine geometry that reconstructs scene that 2 width of cloth images just can be linear and tolerance from different orientation how much.

Technical solution of the present invention

1, a kind of three-dimensional rebuilding method based on plane and straight line; It is characterized in that: under fixing camera parameters; According to plane in the photographed scene and the mutually orthogonal parallel lines of two couple on this plane not, can be from how much of the affine and tolerance of the linear re-construct of the image that two general camera motions obtain.Concrete steps comprise: find the solution the plane about the homography matrix H between two video cameras, find the solution vanishing point coordinate on two groups of parallel lines, calculate the infinite distance and singly answer H _∞, find the solution absolute conic the picture ω.

(1) finds the solution the plane about the homography matrix H between two video cameras

If space plane π goes up and singly to answer H between two video cameras that the corresponding point

of any point in two width of cloth images are then induced by plane π; What satisfied

adopted here is the homogeneous coordinates of point; Each such equation can only provide two equation of constraint about H, and therefore the H that solves that the individual point of N (N>=4) can be linear on the π is arranged.

(2) find the solution vanishing point coordinate on two groups of parallel lines

If two groups of parallel lines

and

picture and in two width of cloth images utilizes the projective transformation invariance, then have:

$\left\{\begin{array}{c}{p}_1=l_1^{\left(1\right)}\×l_2^{\left(1\right)}\\ p_2=l_3^{\left(1\right)}\×l_4^{\left(1\right)}\end{array}\right.$ With $\left\{\begin{array}{c}{q}_1=l_1^{\left(2\right)}\×l_2^{\left(2\right)}\\ q_2=l_3^{\left(2\right)}\×l_4^{\left(2\right)}\end{array}\right..$

P wherein ₁, p ₂And q ₁, q ₂Be the intersection points of two groups of parallel lines in two width of cloth images.

(3) calculate the infinite distance and singly answer H _∞

If e=is (e ₁, e ₂, 1) ^TIt is the limit of second width of cloth image.According to how much of the utmost points; Corresponding has can obtain an equation about e for arbitrfary point in the scene, and utilizing not, individual corresponding

linearity of the M on the π of plane (M>=2) solves e '.

Under two viewpoints, the arbitrary plane list answers H and infinite distance singly to answer H _∞Between have following relation: H=H _∞+ ea ^T, a=(a wherein ₁, a ₂, a ₃) ^TBe 3 * 1 vectors, e is second limit under the viewpoint, and H is then arranged _∞=H-ea ^T, i.e. λ _ip _i=H _∞q _i=Hq _i-ea ^Tq _i(i=1,2).

Make Hq _i=(u, v, w) ^T, p _i=(up _i, vp _i) ^T, q _i=(uq _i, vq _i) ^T(i=1,2) then have:

$\left[\begin{array}{ccc}{\mathrm{up}}_i{\mathrm{uq}}_i-{\mathrm{uq}}_i{e}_1& {\mathrm{up}}_i{\mathrm{vq}}_i-{\mathrm{vq}}_i{e}_1& {\mathrm{up}}_i-e_1\\ {\mathrm{vp}}_i{\mathrm{uq}}_i-{\mathrm{uq}}_i{e}_2& {\mathrm{vp}}_i{\mathrm{vq}}_i-{\mathrm{vq}}_i{e}_2& {\mathrm{vp}}_i-e_2\end{array}\right]\left[\begin{array}{c}{a}_1\\ a_2\\ a_3\end{array}\right]=\left[\begin{array}{c}{\mathrm{up}}_{i}w-u\\ {\mathrm{vp}}_{i}w-v\end{array}\right].$

Utilize least square method to solve a=(a ₁, a ₂, a ₃) ^T, then obtain H _∞=H-ea ^T

Structure affine reconstruction video camera projection matrix P ⁽¹⁾=[I 0], P ⁽²⁾=[H _∞E].Utilize the triangle principle to find the solution the affine coordinates of each point.

(4) find the solution absolute conic the picture ω

Point transformation m '=H on plane at infinity _∞Under the m, absolute conic be transformed to ω '=H _∞ ^-Tω H _∞ ^-1, intrinsic parameter remains unchanged in the camera motion process, and ω=H is then arranged _∞ ^-Tω H _∞ ^-1Work as l ₁, l ₂During quadrature, have

Structure video camera matrix P=P ⁽¹⁾H ^-1, P '=P ⁽²⁾H ^-1, wherein

M satisfies MM ^T=(H _∞ ^Tω H _∞) ^-1

Advantage of the present invention is:

(1) the present invention mainly is applicable to and contains the plane and the condition of the parallel lines of two groups of quadratures on this plane not in the scene, and scene realizes easily.

(2) method of the present invention only utilize plane at infinity list under two viewpoints should and the arbitrary plane list should between linear relationship, linear calculate the infinite distance single should, simplified affine solution procedure.

(3) utilize the straight line of two groups of quadratures among the present invention, for the camera intrinsic parameter matrix provides more constraint, according to line conversion on the plane at infinity and quadrature constraint, carry out scene tolerance can linear 5 intrinsic parameters that solve video camera when rebuilding.

Description of drawings

Utilize the affine geometry and tolerance how much of plane and straight line re-construct among Fig. 1 the present invention, and the process flow diagram of finding the solution camera intrinsic parameter.

The rectangular parallelepiped rebuilding module synoptic diagram that Fig. 2 the present invention adopts.

Embodiment

2, be that the present invention is described in further detail below.A kind of three-dimensional rebuilding method based on plane and straight line has been proposed; It is characterized in that: under fixing camera parameters; According to plane in the photographed scene and the mutually orthogonal parallel lines of two couple on this plane not, can be from how much of the affine and tolerance of the linear re-construct of the image that two general camera motions obtain.Concrete steps comprise: find the solution the plane about the homography matrix H between two video cameras, find the solution vanishing point coordinate on two groups of parallel lines, calculate the infinite distance and singly answer H _∞, find the solution absolute conic the picture ω, find the solution the camera intrinsic parameter matrix K.

(1) finds the solution the plane about the homography matrix H between two video cameras

If space plane π goes up and singly to answer H between two video cameras that the corresponding point of any point in two width of cloth images are then induced by plane π; What satisfied

adopted here is the homogeneous coordinates of point; Each such equation can only provide two equation of constraint about H, and therefore the H that solves that the individual point of N (N>=4) can be linear on the π is arranged.

(2) find the solution vanishing point coordinate on two groups of parallel lines

If two groups of parallel lines

and

picture and

in two width of cloth images utilizes the projective transformation invariance, then have:

$\left\{\begin{array}{c}{p}_1=l_1^{\left(1\right)}\×l_2^{\left(1\right)}\\ p_2=l_3^{\left(1\right)}\×l_4^{\left(1\right)}\end{array}\right.$ With $\left\{\begin{array}{c}{q}_1=l_1^{\left(2\right)}\×l_2^{\left(2\right)}\\ q_2=l_3^{\left(2\right)}\×l_4^{\left(2\right)}\end{array}\right..$

P wherein ₁, p ₂And q ₁, q ₂Be the intersection points of two groups of parallel lines in two width of cloth images.

(3) calculate the infinite distance and singly answer H _∞

Under the fixed cameras parameter, establishing the camera intrinsic parameter matrix is K, and the rotation matrix between two viewpoints is R, and translation vector is T=(t ₁, t ₂, t ₃) ^T, then having according to the video camera imaging geometric model, H is singly answered in the infinite distance _∞=sKRK ^-1, any finite plane π singly answers H=H _∞+ ea ^T, wherein

Be the limit of second width of cloth image,

(n is the unit normal vector of plane π, and d plane π is to the distance at first video camera center).According to utmost point geometric relationship; Corresponding

has

can obtain an equation about e for arbitrfary point in the scene, and utilizing not, individual corresponding

linearity of the M on the π of plane (M>=2) solves e.

H=H under two viewpoints _∞+ ea ^T, H is then arranged _∞=H-ea ^T, i.e. λ _ip _i=H _∞q _i=Hq _i-ea ^Tq _i(i=1,2).Make Hq _i=(u, v, w) ^T, p _i=(up _i, vp _i) ^T, q _i=(uq _i, vq _i) ^T(i=1,2) then have:

$\left[\begin{array}{ccc}{\mathrm{up}}_i{\mathrm{uq}}_i-{\mathrm{uq}}_i{e}_1& {\mathrm{up}}_i{\mathrm{vq}}_i-{\mathrm{vq}}_i{e}_1& {\mathrm{up}}_i-e_1\\ {\mathrm{vp}}_i{\mathrm{uq}}_i-{\mathrm{uq}}_i{e}_2& {\mathrm{vp}}_i{\mathrm{vq}}_i-{\mathrm{vq}}_i{e}_2& {\mathrm{vp}}_i-e_2\end{array}\right]\left[\begin{array}{c}{a}_1\\ a_2\\ a_3\end{array}\right]=\left[\begin{array}{c}{\mathrm{up}}_{i}w-u\\ {\mathrm{vp}}_{i}w-v\end{array}\right].$

Utilize least square method to solve a=(a ₁, a ₂, a ₃) ^T, then obtain H _∞=H-ea ^T

Structure affine reconstruction video camera projection matrix P ⁽¹⁾=[I 0], P ⁽²⁾=[H _∞E].Utilize the triangle principle to find the solution the affine coordinates of each point.

(4) find the solution absolute conic the picture ω

Point transformation m '=H on plane at infinity _∞Under the m, absolute conic be transformed to ω '=H _∞ ^-Tω H _∞ ^-1, intrinsic parameter remains unchanged in the camera motion process, and ω=H is then arranged _∞ ^-Tω H _∞ ^-1

Structure video camera matrix P=P ⁽¹⁾H ^-1, P '=P ⁽²⁾H ^-1, wherein M satisfies MM ^T=(H _∞ ^Tω H _∞) ^-1

(5) find the solution the camera intrinsic parameter matrix K

To decomposing, and last element is carried out normalization handle and obtain the camera intrinsic parameter matrix K when ω carries out Cholesky.

Embodiment

The present invention proposes a kind of under the camera model of preset parameter; Utilize in the scene any plane and the mutually orthogonal parallel lines of two couple on this plane not; The affine and method of measuring how much from the linear re-construct of the image of two general camera motions acquisitions; Calculation process is as shown in Figure 1, and the experiment module structural representation that the present invention adopts is as shown in Figure 2.

With an instance embodiment of the present invention are made more detailed description below:

The experiment module that adopts based on the three-dimensional rebuilding method of plane and straight line is a square that any rib is long, and is as shown in Figure 2.A, B, C, D are four summits of one face, are the parallel lines of two groups of quadratures adopting among the present invention with AB, CD and AD, BC, are space plane with the square left surface.

Utilize method among the present invention to containing the scene of square module, the affine and tolerance how much of the re-construct that the image that obtains from two general camera motions is linear, concrete steps are following:

(1) extract minutiae

The image resolution ratio that adopts among the present invention is 1600 * 1300, and input picture is chosen 2 width of cloth clearly, and the image that is evenly distributed of unique point, utilizes function among the Opencv to extract the coordinate of image characteristic point.

(2) find the solution the plane about the homography matrix H between two video cameras

If space plane π goes up the corresponding point

of any point in two width of cloth images (wherein; Singly answer H between two video cameras that

then induced by plane π; What satisfied adopted here is the homogeneous coordinates of point; Each such equation can only provide two equation of constraint about H, and therefore the H that solves that the individual point of N (N>=4) can be linear on the π is arranged.

(3) find the solution vanishing point coordinate on two groups of parallel lines

If two groups of parallel lines

and

picture

and

in two width of cloth images utilizes the projective transformation invariance, then have:

$\left\{\begin{array}{c}{p}_1=l_1^{\left(1\right)}\×l_2^{\left(1\right)}\\ p_2=l_3^{\left(1\right)}\×l_4^{\left(1\right)}\end{array}\right.$ With $\left\{\begin{array}{c}{q}_1=l_1^{\left(2\right)}\×l_2^{\left(2\right)}\\ q_2=l_3^{\left(2\right)}\×l_4^{\left(2\right)}\end{array}\right..$

P wherein ₁, p ₂And q ₁, q ₂Be the intersection points of two groups of parallel lines in two width of cloth images.

Solve p ₁=(33753.24-6840.11 1) ^T, p ₂=(1491.18-2005.67 1) ^T,

q ₁＝(497.55?-6420.63?1) ^T，q ₂＝(3045.52?1119.89?1) ^T。

(4) calculate the infinite distance and singly answer H _∞

Under the fixed cameras parameter,, corresponding for arbitrfary point in the scene according to utmost point geometric relationship

Have

Can obtain a equation, utilize the individual correspondence of the M on the π of plane (M>=2) about e

Linearity solves e, e=(12749.36 6939.08 1) ^T

H=H under two viewpoints _∞+ ea ^T, H is then arranged _∞=H-ea ^T, i.e. λ _ip _i=H _∞q _i=Hq _i-ea ^Tq _i(i=1,2).Make Hq _i=(u, v, w) ^T, p _i=(up _i, vp _i) ^T, q _i=(uq _i, vq _i) ^T(i=1,2) then have:

$\left[\begin{array}{ccc}{\mathrm{up}}_i{\mathrm{uq}}_i-{\mathrm{uq}}_i{e}_1& {\mathrm{up}}_i{\mathrm{vq}}_i-{\mathrm{vq}}_i{e}_1& {\mathrm{up}}_i-e_1\\ {\mathrm{vp}}_i{\mathrm{uq}}_i-{\mathrm{uq}}_i{e}_2& {\mathrm{vp}}_i{\mathrm{vq}}_i-{\mathrm{vq}}_i{e}_2& {\mathrm{vp}}_i-e_2\end{array}\right]\left[\begin{array}{c}{a}_1\\ a_2\\ a_3\end{array}\right]=\left[\begin{array}{c}{\mathrm{up}}_{i}w-u\\ {\mathrm{vp}}_{i}w-v\end{array}\right].$

Utilize least square method solve a=(7.17e-009,8.87e-019,2.42e-004) ^T, then obtain

$H_{\∞}=H-{\mathrm{ea}}^T=\left[\begin{array}{ccc}-0.00027553614028& 0.00154678583604& -4.05873795796926\\ -0.00192817313245& -0.00076957465678& -1.45259484383115\\ 0.00000021753278& 0.00000011934166& -0.00257127872323\end{array}\right].$

Structure affine reconstruction video camera projection matrix P ⁽¹⁾=[I 0], P ⁽²⁾=[H _∞E].Utilize the triangle principle to find the solution the affine coordinates of each point.

(5) find the solution absolute conic the picture ω

Point transformation m '=H on plane at infinity _∞Under the m, absolute conic be transformed to ω '=H _∞ ^-Tω H _∞ ^-1, intrinsic parameter remains unchanged in the camera motion process, and ω=H is then arranged _∞ ^-Tω H _∞ ^-1, calculate $\mathrm{\ω}=\left[\begin{array}{ccc}0.00000019753492& -0.00000000001975& -0.00015801509458\\ -0.00000000001975& 0.00000019753492& -0.00012838189515\\ -0.00015801509458& -0.00012838189515& 0.99999997927462\end{array}\right].$

Structure video camera matrix P=P ⁽¹⁾H ^-1, P '=P ⁽²⁾H ^-1, wherein

M satisfies MM ^T=(H _∞ ^Tω H _∞) ^-1

(6) find the solution the camera intrinsic parameter matrix

To decomposing, and last element is carried out normalization handle and obtain the camera intrinsic parameter matrix when ω carries out Cholesky $K=\left[\begin{array}{ccc}2000.00000000008& 0.20000000007& 800.00000000005\\ 0& 1999.99999999998& 649.99999999992\\ 0& 0& 1\end{array}\right]$

Whole process is carried out according to process flow diagram shown in Figure 1, input picture successively, extract the characteristics of image point coordinate, find the solution the plane at the homography matrix H under two viewpoints, find the solution vanishing point coordinate on two groups of parallel lines, calculate the infinite distance and singly answer H _∞, construct affine video camera matrix, find the solution the picture ω of absolute conic, structure real camera matrix, find the solution the camera intrinsic parameter matrix K.

Claims (1)

1. three-dimensional rebuilding method based on plane and straight line; It is characterized in that: under fixing camera parameters; According to plane in the photographed scene and the mutually orthogonal parallel lines of two couple on this plane not, can be from how much of the affine and tolerance of the linear re-construct of the image that two general camera motions obtain.Concrete steps comprise: find the solution the plane about the homography matrix H between two video cameras, find the solution vanishing point coordinate on two groups of parallel lines, calculate the infinite distance and singly answer H _∞, find the solution absolute conic the picture ω.

(1) finds the solution the plane about the homography matrix H between two video cameras _∞

If space plane π goes up and singly to answer H between two video cameras that the corresponding point

of any point in two width of cloth images are then induced by plane π, satisfies

therefore have π to go up the H that solves that the individual point of N (N>=4) can be linear.

(2) find the solution vanishing point coordinate on two groups of parallel lines

If two groups of parallel lines

and

picture

and

in two width of cloth images utilizes the projective transformation invariance, then have:

$\left\{\begin{array}{c}{p}_1=l_1^{\left(1\right)}\×l_2^{\left(1\right)}\\ p_2=l_3^{\left(1\right)}\×l_4^{\left(1\right)}\end{array}\right.$ With $\left\{\begin{array}{c}{q}_1=l_1^{\left(2\right)}\×l_2^{\left(2\right)}\\ q_2=l_3^{\left(2\right)}\×l_4^{\left(2\right)}\end{array}\right..$

P wherein ₁, p ₂And q ₁, q ₂Be the intersection points of two groups of parallel lines in two width of cloth images.

(3) calculate the infinite distance and singly answer H _∞

If e=is (e ₁, e ₂, 1) ^TIt is the limit of second width of cloth image.According to how much of the utmost points; Corresponding

has profit to obtain an equation about e ' for arbitrfary point in the scene, and utilizing not, individual corresponding linearity of the M on the π of plane (M>=2) solves e '.

Under two viewpoints, the arbitrary plane list answers H and infinite distance singly to answer H _∞Between have following relation: H=H _∞+ ea ^T, a=(a wherein ₁, a ₂, a ₃) ^TBe 3 * 1 vectors, e is second limit under the viewpoint, and H is then arranged _∞=H-ea ^T, i.e. λ _ip _i=H _∞q _i=Hq _i-ea ^Tq _i(i=1,2).

Make Hq _i=(u, v, w) ^T, p _i=(up _i, vp _i) ^T, q _i=(uq _i, vq _i) ^T(i=1,2) then have:

$\left[\begin{array}{ccc}{\mathrm{up}}_i{\mathrm{uq}}_i-{\mathrm{uq}}_i{e}_1& {\mathrm{up}}_i{\mathrm{vq}}_i-{\mathrm{vq}}_i{e}_1& {\mathrm{up}}_i-e_1\\ {\mathrm{vp}}_i{\mathrm{uq}}_i-{\mathrm{uq}}_i{e}_2& {\mathrm{vp}}_i{\mathrm{vq}}_i-{\mathrm{vq}}_i{e}_2& {\mathrm{vp}}_i-e_2\end{array}\right]\left[\begin{array}{c}{a}_1\\ a_2\\ a_3\end{array}\right]=\left[\begin{array}{c}{\mathrm{up}}_{i}w-u\\ {\mathrm{vp}}_{i}w-v\end{array}\right].$

Utilize least square method to solve a=(a ₁, a ₂, a ₃) ^T, then obtain H _∞=H-ea ^T

(4) find the solution absolute conic the picture ω

Point transformation m '=H on plane at infinity _∞Under the m, absolute conic be transformed to ω '=H _∞ ^-Tω H _∞ ^-1, intrinsic parameter remains unchanged in the camera motion process, and ω=H is then arranged _∞ ^-Tω H _∞ ^-1Work as l ₁, l ₂During quadrature, have

Images