CN110146032A - Calibration method of synthetic aperture camera based on light field distribution - Google Patents

Abstract

The invention belongs to Technology of Precision Measurement field, specially a kind of synthetic aperture camera calibration method based on optical field distribution.The method of the present invention step are as follows: in order to overcome the pupil Aberration Problem of pinhole camera model, imaging pupil is divided into multiple sub-light pupils, according to practical optical field distribution, imaging sensor is divided into the corresponding different imaging regions of sub-light pupil；Optimize in conjunction with synthetic aperture camera model combined calibrating, building one more meets the camera model of actual imaging process.The present invention can effectively eliminate pupil aberration caused by single pin-hole imaging is assumed, overcome the direction ambiguousness of monocular vision；It is significant for the measurement accuracy for improving photogrammetric technology.

Description

Calibration method of synthetic aperture camera based on light field distribution

technical field

The invention belongs to the technical field of optical engineering, and in particular relates to a method for calibrating a synthetic aperture camera based on light field distribution.

Background technique

In modern precision measurement, photogrammetry is a commonly used 3D shape measurement technique. Photogrammetry is mainly divided into two categories: surface 3D measurement techniques for diffuse reflection and surface 3D measurement techniques for specular reflection. The former is typically represented by triangulation, especially fringe projection technology is widely used in industrial precision inspection. Its 3D reconstruction methods mainly include stereo vision and phase-height mapping. For specularly reflective surfaces, phase measurement deflection is commonly used. Because of its simple measurement system, large dynamic range, and strong anti-interference ability, it can be used for the measurement of complex surfaces, and has attracted extensive attention in recent years. The principle is to generate regular stripes on the display, and the stripes will be deformed after being reflected by the measured surface. The deformation pattern is taken by a CCD camera, and the surface gradient distribution of the measured surface can be calculated by deriving from the geometric relationship, and then the surface height can be obtained by integral. .

In photogrammetry and camera calibration, Zhang Zhengyou calibration method is generally used. The camera is simplified to an ideal pinhole imaging model, and it is considered that the light of each pixel of the camera CCD passes through the same point (the principal point of the camera pupil). However, since the pupil of the camera lens has a certain size, this assumption will produce pupil aberration, and the calculated light position and direction will contain obvious deviations, so the single pinhole model cannot perfectly express the object-image relationship of the camera. Three-dimensional measurement accuracy is seriously affected. Therefore, a more convenient and high-precision camera calibration method is needed.

Contents of the invention

The purpose of the present invention is to provide a method that can effectively model the camera imaging process, so as to realize accurate pupil aberration correction.

The present invention proposes a synthetic aperture camera joint calibration optimization method, the specific steps are as follows:

(1) Image plane segmentation based on light field distribution; use digital micromirror array (DMD) to sample the imaging pupil, and combine compressed sensing technology to calculate the four-dimensional light field distribution of the imaging system. According to the distribution of the light field, the imaging chief ray passing through each sub-pupil is calculated, the image plane is divided into multiple sub-regions corresponding to the sub-pupils, and each sub-pupil is assumed to be a pinhole model.

(2) Initial estimation of camera parameters; use the LCD screen to generate a marker point pattern as a calibration board, and place it at different positions to take calibration images. Use Zhang Zhengyou's calibration method to estimate the internal parameter ins _c of the camera and the external parameter {R _w2c |T _w2c } of the calibration plate at each position. The imaging process of the calibration plate can be expressed as:

Among them, (X _w , Y _w , Z _w ) is the world coordinate of the mark point on the calibration board, (μ, v) is the pixel coordinate of the mark point on the CCD, and Z _c is the coordinate of the mark point in the camera coordinate system Z-axis coordinates, (f _x , f _y ) is the focal length of the camera, (μ ₀ , v ₀ ) is the offset of the optical axis in the image coordinates.

The conversion relationship between the pixel coordinates (μ _s , v _s ) of the LCD screen and the world coordinates (X _w , Y _w ) is as follows:

Among them, it is assumed that the plane where the calibration board is located is the XY plane, so Z _w is set to 0, (m, n) is the pixel size of the screen, and (d _x , d _y ) is the pixel offset of the world coordinate origin of the calibration board.

(3) Synthetic aperture camera model joint calibration optimization; in the proposed model, the internal parameters of each sub-pupil are inconsistent, and the external parameters from the calibration plate at each position to all sub-pupils are also inconsistent, so it is necessary to The internal and external parameters are optimized. The calibration board at each position selects q marker points in each image plane sub-area of the camera. The objective equation to be optimized is as follows:

Among them, x={{ins _c ,R _w2c ,T _w2c } _ij ,ins _s } is the variable to be optimized, ins _s ={m,n} is the parameter to be optimized of the LCD screen calibration board, and i is the number of the sub-pupil , j is the position number of the calibration plate, is the real imaging coordinate of the kth marker point of the calibration plate at the jth position corresponding to the ith sub-pupil, is the reprojection coordinate of the kth marker point of the calibration plate at the jth position corresponding to the ith sub-pupil; formula (3) is a nonlinear least squares problem, which can be solved by using the Levenberg-Marquardt algorithm. The internal and external parameters of the camera provided by the Zhang Zhengyou calibration method in step (2) can be used as the initial solution of the Levenberg-Marquardt optimization process.

(4) Unify each sub-pupil into the coordinate system of the central pupil according to the obtained external parameters {R _w2c |T _w2c } of each sub-pupil:

in, is the rotation matrix from the i-th pupil coordinate system to the central pupil coordinate system, is the translation matrix from the i-th pupil coordinate system to the central pupil coordinate system, and is the rotation matrix and translation matrix from the world coordinate system to the central pupil coordinate system, and is the rotation matrix and translation matrix from the same world coordinate system to the i-th pupil coordinate system.

Based on the light field distribution characteristics, the invention divides the imaging sensor into different imaging regions corresponding to the sub-pupils; combined with the joint calibration optimization of the synthetic aperture camera model, a camera model more in line with the actual imaging process is effectively constructed.

The invention can effectively eliminate the pupil aberration caused by the hypothesis of pinhole imaging, overcome the direction ambiguity of monocular vision, and accurately calculate the direction of imaging light. It is of great significance to improve the measurement accuracy of photogrammetry technology.

Description of drawings

Figure 1 is an illustration of the corresponding imaging area of a single sub-pupil (DMD).

Figure 2 is a diagram of the camera imaging model.

Figure 3 shows the reprojection error after optimized calibration.

Detailed ways

The present invention is further illustrated below by means of embodiments in conjunction with the accompanying drawings.

Embodiment 1: During the measurement process, a suitable imaging optical path is set up, and the DMD is used to perform 5×5 pupil sampling. The CCD pixel resolution used in the experiment is 3840×3840, so the dimension of the light field matrix L is 25×14745600 ((5×5)×(3840×3840)). Since the actual imaging process is not strictly small-hole imaging, the light of a single object point will only pass through part of the pupil and be imaged on a certain CCD pixel, which is the light field distribution. In other words, each sub-pupil determines part of the light to be imaged on the CCD on, as shown in Figure 1. According to the light field distribution, the CCD image plane is divided according to the energy distribution of the sub-pupils on the image plane, and each sub-pupil is regarded as a pinhole imaging model. The new camera imaging model is shown in Figure 2. The object point P1 is imaged at p1 of the CCD through the 0th sub-pupil, and the passing pupil coordinate is O _c0 , and the object point P2 is imaged at the p2 of the CCD through the i-th sub-pupil, and the passing pupil coordinate is O _ci . Aid Zhang Zhengyou’s calibration method to obtain the internal and external parameters of each sub-pupil, and then jointly optimize multiple aperture imaging models to obtain high-precision synthetic aperture camera parameters, and then use formula 4 to convert each pupil coordinate system to the coordinate system O _c0 Under X _c0 Y _c0 Z _c0 , construct a camera model that is more in line with the actual imaging process, and effectively correct the pupil aberration of the camera. Figure 3 is the reprojection error map obtained by using the optimized camera parameters, and the RMS of the reprojection error is 0.12 pixels.

Claims (1)

1. A synthetic aperture camera calibration method based on light field distribution is characterized by comprising the following specific steps:

(1) dividing an image surface based on light field distribution; sampling an imaging pupil by using a digital micromirror array (DMD), and calculating the light field distribution of an imaging system by combining a compressive sensing technology; then, calculating an imaging main ray passing through each sub-pupil, dividing an image plane into a plurality of sub-areas corresponding to the sub-pupils, and assuming each sub-pupil as a pinhole model;

(2) initial estimation of camera parameters; generating a landmark point map using an LCD screenThe case is used as a calibration plate and is placed at different positions to shoot calibration images; estimation of internal parameters of a camera using the Zhangyingyou scaling method_cAnd a calibration plate external parameter { R } at each position_w2c|T_w2c}; the imaging process of the calibration plate is expressed as:

wherein (X)_w,Y_w,Z_w) Is the world coordinate of the mark point on the calibration plate, (mu, v) is the pixel coordinate of the mark point on the CCD, Z_cIs the Z-axis coordinate of the mark point in the camera coordinate system, (f)_x,f_y) Is the camera focal length (μ)₀,v₀) Is the offset of the optical axis at the image coordinates;

pixel coordinate (μ) of LCD screen_s,v_s) To world coordinate (X)_w,Y_w) The conversion relationship of (1) is as follows:

wherein, the plane of the calibration plate is assumed to be XY plane, so Z_wSet to 0, (m, n) is the pixel size of the screen, (d)_x,d_y) Pixel offset for the origin of world coordinates of the calibration plate;

(3) the synthetic aperture camera model is calibrated and optimized in a combined mode; the internal parameters of each sub-pupil in the model are inconsistent, and the external parameters from the calibration plate at each position to all sub-pupils are inconsistent, so that the internal and external parameters of each sub-pupil are optimized; respectively selecting q mark points on the calibration plate at each position in the image surface sub-area corresponding to the camera; the objective equation to be optimized is as follows:

wherein x { { ins { (ins)_c,R_w2c,T_w2c}_ij,ins_sIs the variable to be optimized, ins_sThe { m, n } is the parameter to be optimized of the LCD screen calibration plate, i is the number of the sub-pupil, j is the position number of the calibration plate,is the real imaging coordinate of the kth index point of the calibration plate at the jth position corresponding to the ith sub-pupil,the reprojection coordinate of the kth mark point of the calibration plate at the jth position corresponding to the ith sub-pupil; equation (3) is a nonlinear least squares problem, which is solved using the Levenberg-Marquardt algorithm; taking the internal and external parameters of the camera provided by the Zhangyingyou scaling method in the step (2) as an initial solution of a Levenberg-Marquardt optimization process;

(4) according to the obtained external parameter { R of each sub-pupil_w2c|T_w2cUnify each sub-pupil to the coordinate system of the central pupil:

wherein,is a rotation matrix of the ith pupil coordinate system to the central pupil coordinate system,is a translation matrix of the ith pupil coordinate system to the central pupil coordinate system,andis a rotation matrix and a translation matrix from a world coordinate system to a central pupil coordinate system,andis a rotation matrix and a translation matrix from the same world coordinate system to the ith pupil coordinate system.