CN110248179B - Camera pupil aberration correction method based on light field coding - Google Patents

Abstract

The invention belongs to the technical field of precision measurement, and particularly relates to a camera pupil aberration correction method based on light field coding. The method comprises the following steps: in order to overcome the pupil aberration of the pinhole camera, each imaging ray is determined, and the imaging light field is described by adopting a four-dimensional function (x, y, u, v), wherein (x, y) represents the coordinates of an image point, (u, v) represents the coordinates of the ray on the pupil, namely represents the direction of the ray; carrying out binarization pupil encoding by adopting a digital micromirror array; based on the idea of compressed sensing, a light field matrix is solved by utilizing total variation optimization; and calculating pupil coordinates (u, v) corresponding to each image point (x, y), and combining a camera calibration result to effectively correct the pupil aberration of the camera, thereby constructing a camera model conforming to the actual imaging process. The invention can effectively eliminate pupil aberration caused by pinhole imaging hypothesis, overcome direction ambiguity of monocular vision and accurately calculate the direction of imaging light; the method has important significance for improving the measurement precision of the photogrammetry technology.

Description

Camera pupil aberration correction method based on light field coding

Technical Field

The invention belongs to the technical field of precision measurement, and particularly relates to a camera pupil aberration correction method based on light field coding.

Background

In modern precision measurements, photogrammetry is a common three-dimensional topography technique. Photogrammetry is mainly divided into two categories: a surface three-dimensional measurement technique for diffuse reflection and a surface three-dimensional measurement technique for specular reflection. The former is typified by triangulation, and especially the fringe projection technology is widely applied to industrial precision detection. The three-dimensional reconstruction method mainly comprises stereoscopic vision and phase-height mapping. Whereas for specularly reflecting surfaces, phase measurement deflection is commonly used. The measuring system is simple, has a large dynamic range and strong anti-interference capability, can be used for measuring complex curved surfaces, and has attracted much attention in recent years. The principle is that regular stripes are generated on a display, the stripes are deformed after being reflected by the measured surface, a CCD camera is used for shooting a deformation pattern, the surface gradient distribution of the measured surface shape can be calculated through the derivation of the geometric relation, and then the surface shape height is obtained through integration.

In photogrammetry and camera calibration, the camera is generally reduced to an ideal pinhole imaging model, considering that the rays of each pixel of the camera CCD pass through the same point (the principal point of the camera pupil). However, since the pupil of the camera lens has a certain size, pupil aberration is generated on the assumption, and the calculated light position and direction have obvious deviation, so that the pinhole model cannot perfectly express the object image relationship of the camera, and the three-dimensional measurement accuracy is seriously affected. Therefore, a simpler and more efficient pupil aberration correction method with high precision is needed.

Disclosure of Invention

The invention aims to provide a method capable of effectively correcting the pupil aberration of a camera so as to realize the reconstruction of an imaging light field of the camera.

The invention provides a camera pupil aberration correction method based on a light field coding technology, which comprises the following specific steps:

(1) to overcome the pupil aberrations of a pinhole camera, each imaging ray is determined, and the imaging light field L is described using a four-dimensional function (X, Y, u, v):

（1）

wherein (X, Y) represents the coordinates of the ray at the image plane Z, (u, v) and the coordinates of the ray on the pupil (corresponding to a Z coordinate of 0), i.e. the ray direction;

(2) constructing a light field coding light path; the light field coding light path comprises a main mirror, 2 relay mirrors, a digital micromirror array (DMD) and a CCD, as shown in FIG. 1; the LCD screen is used as an object, object space light is focused on a virtual imaging surface through the main mirror, converged light is converted into parallel light through the first relay mirror and projected on the DMD, and the parallel light is coded and sampled by the DMD to realize turning of a light path; the second relay lens converges the coded parallel light on a CCD image surface to realize image acquisition; hadamard binarization encoding is carried out on the DMD, and an effective pupil encoding acquisition process is realized; in fig. 1, L1 is the object distance, L2= f1+ k1+ f2, where f1 is the focal length of the primary mirror, f2 is the focal length of the relay mirror, k1 is the diffraction distance, L3 is the distance between the relay mirrors, and L4 is the focal length of the relay mirror;

(3) the method comprises the steps of vectorizing pupil pixels (u, v) to be used as rows of a light field matrix L, vectorizing pixel coordinates (X, Y) to be used as columns of the light field matrix L, determining total intensity of all light rays transmitted from (u, v) to the (X, Y) pixels by taking the numerical value of each element in the light field matrix L as an expression (1), determining an observation matrix phi according to binary coding corresponding to a DMD on the basis of a compressive sensing principle, forming a sparse matrix psi by corresponding even-number wavelet bases, and defining the following objective functions by setting a light field matrix L = psi α as a coefficient of the light field matrix L under the wavelet base:

（2）

wherein β is the camera shooting image under different pupil codes when calibrating the camera,

representing vectors or matriceskThe norm of the number of the first-order-of-arrival,

expressing the total variation operation, wherein lambda is a weight coefficient and is determined by the noise level of the image;

(4) solving the optimization problem (2) by adopting an alternating direction multiplier method to obtain a light field matrix L, obtaining accurate light ray position and direction by utilizing the participation of the camera in external parameter calibration, calculating pupil coordinates (u, v) corresponding to each image point (X, Y), and combining a camera calibration result to effectively correct the pupil aberration of the camera, thereby constructing a camera model which is more in line with the actual imaging process.

The optimization problem (2) is solved by adopting an Alternating Direction Multiplier Method (ADMM), and is converted into an augmented Lagrange equation:

（3）

wherein D is a differential operator corresponding to the total variation,tis a Lagrange multiplier, and rho is a weight coefficient; to formula (3) in unknown amountsα,s,tAnd grouping alternate optimization solution.

The invention is based on the light field coding and compressed sensing principle, wherein the sparse matrix and the observation matrix are freely arranged, and the appropriate sparse matrix and the observation matrix can be designed according to the distribution characteristic of the actual light field matrix. Meanwhile, two items of the objective function are optimized, the similarity between the reconstructed light field and the acquired data and the definition of the reconstructed target are respectively measured, and the weight coefficient can be adjusted according to actual needs to obtain an accurate light field matrix.

The invention can effectively eliminate pupil aberration caused by pinhole imaging hypothesis, overcome direction ambiguity of monocular vision and accurately calculate the direction of imaging light. The method has important significance for improving the measurement precision of the photogrammetry technology.

Drawings

FIG. 1 is a diagram of a light field encoding light path constructed by the present invention.

Fig. 2 is a Hadamard matrix for DMD coding.

Fig. 3 is an imaging simulation diagram.

Fig. 4 shows three-dimensional measurement before and after pupil aberration correction. Wherein, (a) is a three-dimensional measurement result after pupil aberration correction, (b) is a three-dimensional measurement error after pupil aberration correction, and (c) is a three-dimensional measurement error before pupil aberration correction.

Detailed Description

The invention is further illustrated by the following examples in conjunction with the accompanying drawings.

In the embodiment 1, in the measurement process, a suitable imaging optical path is established, as shown in fig. 1, an object side light ray in the optical path is focused on a virtual imaging surface through a main mirror, and then a converged light ray is converted into a parallel light through a first relay mirror to be projected on a DMD, the DMD converges an optical path turn on a CCD through a second relay mirror to be imaged, a mirror image of the second relay mirror and the CCD is in a virtual frame, L1 is an object distance, L2= f1+ k1+ f2, wherein f1 is a main mirror focal length, f2 is a relay mirror focal length, k1 is a diffraction distance, L3 is a relay mirror distance, L4 is a relay mirror focal length, specific optical path parameters are shown in table 1, Hadamard encoding (i.e., an observation matrix Φ) is performed by the DMD, an encoded image is acquired, an encoding pattern is shown in fig. 2, pixel resolution used in an experiment is 3840 × 3840, so that a dimension of a matrix L is 14745600 × 27, an image, an observation matrix, a pupil is obtained by a pupil, a pupil correction method, a pupil is obtained by a three-dimensional imaging pupil measurement method, a pupil correction method is established, a pupil correction method is performed by a pupil correction method, a pupil correction method is performed by a three-dimensional pupil measurement method, a pupil correction method is performed by a pupil correction method, a three-dimensional imaging pupil measurement method, a pupil correction method is performed by a pupil correction method which is performed by a three-dimensional imaging method which is performed by a pupil correction method which is performed by a three-dimensional imaging method, a three-dimensional imaging method which is performed by a three-dimensional imaging method which is.

（3）

Wherein D is a differential operator corresponding to the total variation,tis a Lagrange multiplier, and rho is a weight coefficient; to formula (4) in unknown amountsα,s,tAnd grouping alternate optimization solution.

TABLE 1 Primary optical path parameters

。

TABLE 2 calculation of Process Key parameters

。

Claims (2)

1. A camera pupil aberration correction method based on light field coding is characterized by comprising the following specific steps:

(1) the imaging light field L is described by a four-dimensional function (X, Y, u, v), i.e. the coordinates of a ray at the cross-section z are represented as:

（1）

wherein (X, Y) represents the coordinates of the ray at the image plane Z, (u, v) represents the coordinates of the ray on the pupil, the corresponding Z-coordinate is 0, i.e. represents the ray direction;

(2) constructing a light field coding light path: the light field coding light path comprises a main mirror, 2 relay mirrors, a digital micromirror array (DMD) and a CCD; the LCD screen is used as an object, object space light is focused on a virtual imaging surface through the main mirror, converged light is converted into parallel light through the first relay mirror and projected on the DMD, and the parallel light is coded and sampled by the DMD to realize turning of a light path; the second relay lens converges the coded parallel light on a CCD image surface to realize image acquisition; hadamard binarization encoding is carried out on the DMD, and an effective pupil encoding acquisition process is realized;

(3) the method comprises the steps of vectorizing pupil pixels (u, v) to be used as rows of a light field matrix L, vectorizing pixel coordinates (X, Y) to be used as columns of the light field matrix L, determining total intensity of all light rays transmitted from (u, v) to the (X, Y) pixels by taking the numerical value of each element in the light field matrix L as an expression (1), determining an observation matrix phi according to binary coding corresponding to a DMD on the basis of a compressive sensing principle, forming a sparse matrix psi by corresponding even-number wavelet bases, and defining the following objective functions by taking the light field matrix L = psi α as a coefficient of the light field matrix L under the wavelet base:

（2）

β is different light when calibrating cameraCamera images under pupil coding, representing vectors or matriceskThe norm of the number of the first-order-of-arrival,

expressing the total variation operation, wherein lambda is a weight coefficient and is determined by the noise level of the image;

(4) solving the optimization problem (2) by adopting an alternating direction multiplier method to obtain a light field matrix L, obtaining accurate light ray position and direction by utilizing the participation of the camera in external parameter calibration, calculating pupil coordinates (u, v) corresponding to each image point (X, Y), and combining a camera calibration result to effectively correct the pupil aberration of the camera, thereby constructing a camera model which is more in line with the actual imaging process.

2. The method for correcting the pupil aberration of the camera based on the light field coding according to claim 1, wherein the optimization problem (2) is solved by adopting an alternating direction multiplier method, and is converted into an augmented lagrangian equation:

（3）

wherein D is a differential operator corresponding to the total variation,tis a Lagrange multiplier, and rho is a weight coefficient; to formula (3) in unknown amountsα,s,tAnd grouping alternate optimization solution.

Images