CN110146032B - Synthetic aperture camera calibration method based on light field distribution - Google Patents

Abstract

The invention belongs to the technical field of precision measurement, and particularly relates to a synthetic aperture camera calibration method based on light field distribution. The method comprises the following steps: in order to overcome the pupil aberration problem of a pinhole camera model, an imaging pupil is divided into a plurality of sub-pupils, and an imaging sensor is divided into different imaging areas corresponding to the sub-pupils according to actual light field distribution; and combining the synthetic aperture camera model with calibration optimization to construct a camera model which is more in line with the actual imaging process. The invention can effectively eliminate pupil aberration caused by single pinhole imaging hypothesis, and overcome direction ambiguity of monocular vision; the method has important significance for improving the measurement precision of the photogrammetry technology.

Description

Synthetic aperture camera calibration method based on light field distribution

Technical Field

The invention belongs to the technical field of optical engineering, and particularly relates to a synthetic aperture camera calibration method based on light field distribution.

Background

In modern precision measurements, photogrammetry is a common three-dimensional topography technique. Photogrammetry is mainly divided into two categories: a surface three-dimensional measurement technique for diffuse reflection and a surface three-dimensional measurement technique for specular reflection. The former is typified by triangulation, and especially the fringe projection technology is widely applied to industrial precision detection. The three-dimensional reconstruction method mainly comprises stereoscopic vision and phase-height mapping. Whereas for specularly reflecting surfaces, phase measurement deflection is commonly used. The measuring system is simple, has a large dynamic range and strong anti-interference capability, can be used for measuring complex curved surfaces, and has attracted much attention in recent years. The principle is that regular stripes are generated on a display, the stripes are deformed after being reflected by the measured surface, a CCD camera is used for shooting a deformation pattern, the surface gradient distribution of the measured surface shape can be calculated through the derivation of the geometric relation, and then the surface shape height is obtained through integration.

In photogrammetry and camera calibration, a zhangyingyou calibration method is generally used. The camera is simplified into an ideal pinhole imaging model, and the light of each pixel of the CCD of the camera passes through the same point (the main point of the pupil of the camera). However, since the pupil of the camera lens has a certain size, pupil aberration is generated on the assumption, and the calculated light position and direction have obvious deviation, so that a single pinhole model cannot perfectly express the object image relationship of the camera, and the three-dimensional measurement accuracy is seriously affected. Therefore, a more convenient high-precision camera calibration method is needed.

Disclosure of Invention

The invention aims to provide a method capable of effectively modeling a camera imaging process so as to realize accurate pupil aberration correction.

The invention provides a combined calibration optimization method for a synthetic aperture camera, which comprises the following specific steps:

(1) dividing an image surface based on light field distribution; and sampling the imaging pupil by using a digital micromirror array (DMD), and calculating the four-dimensional light field distribution of the imaging system by combining a compressed sensing technology. And calculating an imaging main ray passing through each sub-pupil according to the light field distribution, dividing the image plane into a plurality of sub-areas corresponding to the sub-pupils, and assuming each sub-pupil as a pinhole model.

(2) Initial estimation of camera parameters; the LCD screen is used to generate a pattern of dots as a calibration plate and placed at different positions to take calibration images. Estimation of internal parameters of a camera using the Zhangyingyou scaling method_cAnd a calibration plate external parameter { R } at each position_w2c|T_w2c}. The imaging process of the calibration plate can be expressed as:

wherein (X)_w,Y_w,Z_w) Is the world coordinate of the mark point on the calibration plate, (mu, v) is the pixel coordinate of the mark point on the CCD, Z_cIs the Z-axis coordinate of the mark point in the camera coordinate system, (f)_x,f_y) Is the camera focal length (μ)₀,v₀) Is the shift of the optical axis at the image coordinates.

Pixel coordinate (μ) of LCD screen_s,v_s) To world coordinate (X)_w,Y_w) The conversion relationship of (1) is as follows:

wherein, the plane of the calibration plate is assumed to be XY plane, so Z_wSet to 0, (m, n) is the pixel size of the screen, (d)_x,d_y) Is the pixel offset from the origin of the world coordinates of the calibration plate.

(3) The synthetic aperture camera model is calibrated and optimized in a combined mode; in the proposed model, the internal parameters of each sub-pupil are not consistent, and the external parameters of the calibration plate at each position to all sub-pupils are not consistent, so that the internal and external parameters of each sub-pupil need to be optimized. And the calibration plate at each position respectively selects q mark points in each image surface sub-area of the camera. The objective equation to be optimized is as follows:

wherein x { { ins { (ins)_c,R_w2c,T_w2c}_ij,ins_sIs the variable to be optimized, ins_sThe { m, n } is the parameter to be optimized of the LCD screen calibration plate, i is the number of the sub-pupil, j is the position number of the calibration plate,

is the real imaging coordinate of the kth index point of the calibration plate at the jth position corresponding to the ith sub-pupil,

the reprojection coordinate of the kth mark point of the calibration plate at the jth position corresponding to the ith sub-pupil; equation (3) is a non-linear least squares problem that can be solved using the Levenberg-Marquardt algorithm. The camera internal and external parameters provided by the Zhangyingyou scaling method in the step (2) can be used as an initial solution of a Levenberg-Marquardt optimization process.

(4) According to the obtained external parameter { R of each sub-pupil_w2c|T_w2cUnify each sub-pupil to the coordinate system of the central pupil:

wherein the content of the first and second substances,

is a rotation matrix of the ith pupil coordinate system to the central pupil coordinate system,

is a translation matrix of the ith pupil coordinate system to the central pupil coordinate system,

and

is a rotation matrix and a translation matrix from a world coordinate system to a central pupil coordinate system,

and

is a rotation matrix and a translation matrix from the same world coordinate system to the ith pupil coordinate system.

The method divides an imaging sensor into different imaging areas corresponding to sub-pupils based on the light field distribution characteristics; and then, a camera model which is more in line with the actual imaging process is effectively constructed by combining the combined calibration optimization of the synthetic aperture camera model.

The invention can effectively eliminate pupil aberration caused by pinhole imaging hypothesis, overcome direction ambiguity of monocular vision and accurately calculate the direction of imaging light. The method has important significance for improving the measurement precision of the photogrammetry technology.

Drawings

Fig. 1 is a diagram of a single sub-pupil (DMD) corresponding imaging area.

Fig. 2 is a camera imaging model illustration.

Fig. 3 shows the reprojection error after the optimization calibration.

Detailed Description

The invention is further illustrated by the following examples in conjunction with the accompanying drawings.

In the measuring process, a proper imaging light path is established, and pupil sampling of 5 × 5 is carried out by using a DMD (digital micromirror device) in an experiment, the resolution of a CCD pixel used in the experiment is 3840 × 3840, so the dimension of a light field matrix L is 25 × 14745600((5 × 5) × (3840 × 3840)). since the actual imaging process is not strict pinhole imaging, light rays of a single object point only pass through a part of pupil to be imaged on a certain CCD pixel, namely the light field distribution, in other words, partial light rays of each sub-pupil determining part are imaged on the CCD, as shown in FIG. 1. according to the light field distribution, a CCD image surface is divided according to the energy distribution of sub-pupils on the image surface, each sub-pupil is taken as a pinhole imaging model, a new camera imaging model is shown in FIG. 2. an object point P1 passes through a 0 sub-pupil to be imaged on a P1 of the CCD, and the coordinate of the passing_c0The object point P2 passes through the ith sub-pupil to be imaged at P2 of the CCD, and the passing pupil coordinate is O_ci. Obtaining internal and external parameters of each sub-pupil by an auxiliary Zhang-Yong calibration method, then carrying out combined optimization on a plurality of aperture imaging models to obtain high-precision synthetic aperture camera parameters, and unifying each pupil coordinate system to a coordinate system O by using a formula 4_c0X_c0Y_c0Z_c0Then, a camera model which is more in line with the actual imaging process is constructed, and pupil aberration of the camera is effectively corrected. Fig. 3 is a re-projection error map obtained using optimized camera parameters, with a 0.12 pixel RMS of re-projection error.

Claims (1)

1. A synthetic aperture camera calibration method based on light field distribution is characterized by comprising the following specific steps:

(1) dividing an image surface based on light field distribution; sampling an imaging pupil by using a digital micromirror array (DMD), and calculating the light field distribution of an imaging system by combining a compressive sensing technology; then, calculating an imaging main ray passing through each sub-pupil, dividing an image plane into a plurality of sub-areas corresponding to the sub-pupils, and assuming each sub-pupil as a pinhole model;

(2) initial estimation of camera parameters; using an LCD screen to generate a mark point pattern as a calibration plate, and placing the mark point pattern at different positions to shoot calibration images; estimation of internal parameters of a camera using the Zhangyingyou scaling method_cAnd a calibration plate external parameter { R } at each position_w2c|T_w2c}; the imaging process of the calibration plate is expressed as:

wherein (X)_w,Y_w,Z_w) Is the world coordinate of the mark point on the calibration plate, (mu, v) is the pixel coordinate of the mark point on the CCD, Z_cIs the Z-axis coordinate, f, of the marker point in the camera coordinate system_x、f_yIs the focal length, μ, of the camera₀、v₀Is the offset of the optical axis at the image coordinates;

pixel coordinate (μ) of LCD screen_s,v_s) To world coordinate (X)_w,Y_w) The conversion relationship of (1) is as follows:

wherein, the plane of the calibration plate is assumed to be XY plane, so Z_wSet to 0, m, n are the pixel sizes of the screen, d_x、d_yPixel offset for the origin of world coordinates of the calibration plate;

(3) the synthetic aperture camera model is calibrated and optimized in a combined mode; the internal parameters of each sub-pupil in the model are inconsistent, and the external parameters from the calibration plate at each position to all sub-pupils are inconsistent, so that the internal and external parameters of each sub-pupil are optimized; respectively selecting q mark points on the calibration plate at each position in the image surface sub-area corresponding to the camera; the objective equation to be optimized is as follows:

wherein x { { ins { (ins)_c,R_w2c,T_w2c}_ij,ins_sIs the variable to be optimized, ins_sThe { m, n } is the parameter to be optimized of the LCD screen calibration plate, i is the number of the sub-pupil, j is the position number of the calibration plate,

is the real imaging coordinate of the kth index point of the calibration plate at the jth position corresponding to the ith sub-pupil,

the reprojection coordinate of the kth mark point of the calibration plate at the jth position corresponding to the ith sub-pupil; equation (3) is a nonlinear least squares problem, which is solved using the Levenberg-Marquardt algorithm; taking the internal and external parameters of the camera provided by the Zhangyingyou scaling method in the step (2) as an initial solution of a Levenberg-Marquardt optimization process;

(4) according to the obtained external parameter { R of each sub-pupil_w2c|T_w2cUnify each sub-pupil to the coordinate system of the central pupil:

wherein the content of the first and second substances,

is a rotation matrix of the ith pupil coordinate system to the central pupil coordinate system,

is a translation matrix of the ith pupil coordinate system to the central pupil coordinate system,

and

from world to central pupil coordinate systemA rotation matrix and a translation matrix,

and

is a rotation matrix and a translation matrix from the same world coordinate system to the ith pupil coordinate system.

Images