CN106980601B - High-precision basic matrix solving method based on trinocular polar line constraint - Google Patents

Abstract

The invention discloses a high-precision solving method of a basic matrix based on trinocular polar line constraint, belongs to the field of computer vision detection, and relates to a high-precision optimization solving method of the basic matrix by utilizing trinocular polar line constraint. The method utilizes three cameras of a common view field in space to shoot the same scene, and utilizes the coordinate deviation of the intersection point of polar lines in a third image corresponding to the characteristic points of the first two images and the actual characteristic point to carry out iterative minimization, thereby optimizing a basic matrix between the two images. And (3) firstly solving an initial basic matrix, then solving the intersection point of the two polar lines in the third image, and optimizing the basic matrix by using the minimum coordinate difference value between the intersection point coordinates of the two polar lines and the mark point of the third image as an optimization target. The method realizes the high-precision acquisition method of the basic matrix between the camera images, effectively reduces the problems in the subsequent camera calibration and feature point matching processes, and is beneficial to improving the precision of vision measurement.

Description

High-precision basic matrix solving method based on trinocular polar line constraint

Technical Field

The invention belongs to the field of computer vision detection, and relates to a method for performing high-precision optimization solving on a basic matrix by utilizing trinocular polar line constraint.

Background

The basis matrix describes algebraically the geometric relationship between two-dimensional images of the same three-dimensional scene taken at two different viewpoints. The basis matrix F includes not only the rotation and translation information between the two cameras in the physical space, but also the intrinsic parameter information between the two cameras. Therefore, correct solution of the basic matrix has a crucial influence on the subsequent calibration process of the camera and elimination of the mismatching features.

The current estimation algorithms of the basis matrix can be generally divided into three categories: linear methods, non-linear iterative methods and robust methods. The most common method is that the 8-point algorithm calculates the basis matrix linearly using a given plurality of (N ≧ 8) points, but the objective function of the 8-point algorithm in the minimization process has no physical significance, and when the image data is noisy, i.e., the corresponding points are inaccurate, the solution accuracy of the basis matrix F given by the 8-point algorithm is low. Chenzezhi et al, "a linear algorithm of a basis matrix for high-precision estimation", software science, 2002,13 (4): 840-845, considering that each matching point has different influence on solving the basis matrix, and introducing the residual difference and epipolar distance function as the weight factor to make the influence of the high-precision matching point on solving the basis matrix larger than that of the low-precision matching point. The method has the advantages of high solving speed and high robustness, but the actual pixel coordinate values of the matching points are changed in the weighting process, so that the precision is influenced.

Disclosure of Invention

The invention aims to solve the technical problem that under the conditions of inaccurate feature point extraction and imaging noise, the solution of a basic matrix is not accurate enough due to a common 8-point method, and discloses a high-precision solution method of the basic matrix based on trinocular polar line constraint.

The technical scheme adopted by the invention is a high-precision basic matrix solving method based on trinocular polar line constraint, the method adopts three cameras of a common view field in a space to shoot the same scene, and utilizes other characteristic mark points except 8 characteristic points used for solving an initial matrix in the space, so that the solving error of the basic matrix is reduced; through the introduced epipolar constraint, the coordinate deviation of the intersection point of the epipolar lines in the third image corresponding to the feature points of the first two images and the actual feature point is iteratively minimized, and the solved initial value of the basic matrix is optimized, so that the basic matrix between the two images is optimized; the method comprises the following specific steps:

step 1: initial basis matrix extraction

The basic matrix F describes the association relationship of points and lines between two images in an algebraic form. Setting the normalized coordinates of pixel points of the same characteristic mark point in the space in the shot left image, the shot middle image and the shot right image as p_l＝(x_l,y_l,1)^T、p_m＝(x_m,y_m,1)^T、p_r＝(x_r,y_r,1)^TThe basis matrix F satisfies the following equation:

p_l ^TFp_r＝0 (1)

let F be (F)_ij) Then the constraint equations for the fundamental matrix can be written in the form:

x_rx_lf₁₁+x_ry_lf₁₂+x_rf₁₃+y_rx_lf₂₁+y_ry_lf₂₂+y_rf₂₃+x_lf₃₁+y_lf₃₂+f₃₃＝0 (2)

remember f ═ f₁₁,f₁₂,f₁₃,f₂₁,f₂₂,f₂₃,f₃₁,f₃₂,f₃₃)^TAnd it is a 9-dimensional column vector consisting of three row vectors of F, then the above equation can be written in the form of vector inner product:

(x_rx_l,x_ry_l,x_r,y_rx_l,y_ry_l,y_r,x_l,y_l)f＝0 (3)

for the conventional 8-point method, given 8 corresponding points, a system of linear equations can be obtained:

in practical cases, the basis matrix cannot be determined directly by solving the linear system of equations directly, but the system of equations is solved under the constraint condition | | | f | | | | 1

Make singular value decomposition of A be A ═ UDV^TThe solution to problem (5) is then the last column vector of V, i.e., f ═ V₉With this, the basis matrix F can be constructed.

Step 2: finding the intersection of two polar lines in the third image

The projected point P on the image plane of the left camera is known from the shooting geometry_lCorresponding to the projected point P on the right camera_rNecessarily in the polar line l of the right image_rUpper and right polar line l_rThe equation satisfies:

l_r＝F₁p_l(6)

similarly, the projection point P on the image plane of the middle camera_mCorresponding to the projected point P on the right camera_rNecessarily in the polar line l of the right image_r' Upper, right polar line l_r' the equation satisfies:

l_r'＝F₂p_m(7)

two polar lines intersect at a point p_r', setting the pixel coordinate p of the intersection point_r'＝(u,v,1)^TI.e. by

l_r-l_r'|_(u,v)＝0 (8)

And step 3: optimizing a basis matrix

Theoretically the intersection point p of two polar lines_r' should be associated with the characteristic mark point p_rCoincidence is realized, but due to a series of problems of inaccurate coordinate extraction of characteristic points, imaging noise and the likeThe solution of the fundamental matrix F is biased, which in turn leads to p_r' and p_rCoordinate value deviation exists between the two points, and the basic matrix needs to be optimized at the moment.

All the feature points except the feature points used by the 8-point method are selected, and the corresponding polar line intersection point coordinates (u) on the right image are solved_i,v_i) Coordinates (u) of intersection of two polar lines_i,v_i) Correction to the right image is the actual feature point coordinates (x)_ir,y_ir) (ii) a An optimization model is established by iteratively minimizing the deviation between the coordinates of the intersection point of the two polar lines and the coordinates of the actual characteristic points, and the target optimization function is as follows:

solving the objective function by adopting an LM (linear regression) nonlinear optimization algorithm, wherein when the deviation of the pixel point is minimum, the corresponding parameter is the optimized basic matrix F₁'、F₂'。

The method has the advantages that the remaining characteristic mark points except 8 characteristic points used for solving the initial matrix in the space are effectively utilized, the solving error of the basic matrix is reduced, and the basic matrix can better meet the photographic geometric significance. The method realizes the high-precision acquisition method of the basic matrix between the camera images, effectively reduces the problems in the subsequent camera calibration and feature point matching processes, and is beneficial to improving the precision of vision measurement.

Drawings

Fig. 1 is a schematic view of a trinocular vision measuring system. In the figure, 1-is a left camera, 2-is a middle camera, 3-is a right camera, and 4-is a checkerboard calibration board.

FIG. 2 is a schematic diagram of a basic matrix for triathlon epipolar constraint optimization. 6. 7, 8-the left, middle and right images are respectively the images shot by the left, middle and right cameras, 4-a chessboard pattern calibration plate, 5-any feature point on the calibration plate, p₁，p₂，p₃Left, middle and right projection points of the feature point 5 in the three images, l₁₃Is a left projection point p₁By the basis matrix F₁₃Mapping to epipolar lines, l, in the right image 3₂₃Is a middle projection point p₂By the basis matrix F₂₃To epipolar lines in the right image 3; p is a radical of_cIs polar line l₁₃And l₂₃The intersection point of (a).

Detailed Description

The following detailed description of the embodiments of the invention refers to the accompanying drawings.

Fig. 1 is a schematic view of a three-eye vision measuring system, and the three cameras used in the present invention are of the types: SVcam-svs11002 with a resolution of 4008 × 2672; three cameras shoot the checkerboard calibration plate 4 in a common view field, the number of the checkerboard feature angular points used in the embodiment is 31, and a graphic workstation is used for processing 31 marker points shot by the three cameras. The optimization measurement method based on the trinocular polar line basic matrix comprises the following specific steps:

step 1: initial basis matrix extraction

Solving the basis matrix F between the left, middle and right images 6, 7, 8 in FIG. 2, respectively₁₃、F₂₃. And selecting any 8 characteristic points on the checkerboard, and finding out corresponding point pairs of the characteristic points in the three images. The following system of equations is solved using the pixel coordinates of the 8 sets of point pairs, and equations (1) through (5):

the singular value of A is then decomposed into A ═ UDV^TThe solution of equation (5) is then the last column vector of V, i.e., f ═ V₉With this, a basis matrix F between the left, middle and right images can be constructed₁₃、F₂₃。

Step 2: finding the intersection of two polar lines in the third image

The rest 23 characteristic point pairs except the characteristic point used by the 8-point method are passed through the initial value F of the basic matrix₁₃、F₂₃Solving epipolar lines of 23 feature points in the left and middle images in the right image by using equations (6) to (8), wherein the epipolar line equation is as follows:

obtaining pixel coordinates of 23 intersections of epipolar lines, each p_c1,p_c2,…p_c23。

And step 3: optimizing a basis matrix

Due to the fact that the solving precision of the initial basis matrix F is low, the polar line intersection point coordinate p is caused_cWith the actual right projection point p in the right image₃And the initial values of the basic matrixes need to be optimized at the moment. Intersection point p_cCoordinate (x)_ci,y_ci) With the actual right projected point p₃Coordinates (u)_3i,v_3i) The pixel deviation in the x direction and the y direction exists, an optimization model is established by iteratively minimizing the deviation between the intersection point coordinates of the two polar lines and the actual characteristic point coordinates, and the target optimization function is as follows:

solving the objective function by adopting an LM (linear regression) nonlinear optimization algorithm, wherein when the deviation of the pixel point is minimum, the corresponding parameter is the optimized basic matrix F₁₃'、F₂₃'。

Using the basis matrix F obtained after optimization₁₃'、F₂₃The intersection point of two polar lines on a right image is obtained by solving any 31 feature points P on the checkerboard, wherein the coordinate of the intersection point and the coordinate of an actual feature point are less than 5 pixel coordinates in the X direction and less than 1 pixel coordinate in the Y direction, and the method has a good application effect.

According to the method, the traditional 8-point basis matrix solving algorithm is improved, and the basis matrix is optimized by using the rest characteristic points, so that the problems that the solving precision of the initial basis matrix F is low and the objective function in the minimization process has no physical significance are solved, and the solving precision of the basis matrix is improved.

Claims (1)

1. The high-precision basic matrix solving method based on the trinocular polar line constraint is characterized in that three cameras of a common view field in a space are used for shooting the same scene, and other characteristic mark points except 8 characteristic points used for solving an initial matrix in the space are utilized, so that the solving error of the basic matrix is reduced; through the introduced epipolar constraint, the coordinate deviation of the intersection point of the epipolar lines in the third image corresponding to the feature points of the first two images and the actual feature point is iteratively minimized, and the solved initial value of the basic matrix is optimized, so that the basic matrix between the two images is optimized; the method comprises the following specific steps:

step 1: initial basis matrix extraction

The basic matrix F describes the incidence relation of points and lines between two images in an algebraic form; setting the normalized coordinates of pixel points of the same characteristic mark point in the space in the three shot left, middle and right images as p_l＝(x_l,y_l,1)^T、p_t＝(x_t,y_t,1)^T、p_r＝(x_r,y_r,1)^TThe basis matrix F satisfies the following equation:

p_l ^TFp_r＝0 (1)

let F be (F)_ij) Then the constraint equation for the fundamental matrix is written in the form:

x_rx_lf₁₁+x_ry_lf₁₂+x_rf₁₃+y_rx_lf₂₁+y_ry_lf₂₂+y_rf₂₃+x_lf₃₁+y_lf₃₂+f₃₃＝0 (2)

remember f ═ f₁₁,f₁₂,f₁₃,f₂₁,f₂₂,f₂₃,f₃₁,f₃₂,f₃₃)^TAnd it is a 9-dimensional column vector consisting of three row vectors of F, then the above equation is written in the form of vector inner product:

(x_rx_l,x_ry_l,x_r,y_rx_l,y_ry_l,y_r,x_l,y_l,1)f＝0 (3)

for the conventional 8-point method, given 8 corresponding points, a linear system of equations is obtained:

in practical cases, the basis matrix cannot be determined by directly solving the linear system of equations, but the system of equations is solved under the constraint of 1:

make singular value decomposition of A be A ═ UDV^TThe solution of equation (5) is then the last column vector of V, i.e., f ═ g₉Thereby constructing a basis matrix F;

step 2: finding the intersection of two polar lines in the third image

The projected point P on the left camera image plane can be known from the shooting geometry_lCorresponding to the projected point P on the right camera_rNecessarily located at the polar line l of the left camera corresponding to the right image_rUpper, left camera corresponds to polar line l of right image_rThe equation satisfies:

l_r＝F₁P_l(6)

similarly, the projection point P on the image plane of the camera_tCorresponding to the projected point P on the right camera_rNecessarily located in the polar line l of the corresponding right image of the camera_r' Upper, middle camera corresponds to polar line l of right image_r' the equation satisfies:

l_r'＝F₂P_t(7)

two polar lines intersect at a point p_r', setting the pixel coordinate p of the intersection point_r'＝(u,v,1)^TI.e. by

l_r-l_r'|_(u,v)＝0 (8)

And step 3: optimizing a basis matrix

Selecting a method other than the 8-point methodAll the feature points except the obtained feature points are solved for corresponding polar line intersection point coordinates (u) on a right image_i,v_i) Coordinates (u) of intersection of two polar lines_i,v_i) Corrected to the coordinates (x) of the actual feature points of the right image_ir,y_ir) (ii) a An optimization model is established by iteratively minimizing the deviation between the coordinates of the intersection point of the two polar lines and the coordinates of the actual characteristic points, and the target optimization function is as follows:

solving the objective function by adopting an LM (linear regression) nonlinear optimization algorithm, wherein when the deviation of the pixel point is minimum, the corresponding parameter is the optimized basic matrix F₁₃'、F₂₃'。

Images