CN110310338B - Light field camera calibration method based on multi-center projection model - Google Patents
Light field camera calibration method based on multi-center projection model Download PDFInfo
- Publication number
- CN110310338B CN110310338B CN201910547317.XA CN201910547317A CN110310338B CN 110310338 B CN110310338 B CN 110310338B CN 201910547317 A CN201910547317 A CN 201910547317A CN 110310338 B CN110310338 B CN 110310338B
- Authority
- CN
- China
- Prior art keywords
- light field
- field camera
- coordinate system
- camera
- point
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Studio Devices (AREA)
Abstract
The invention provides a light field camera calibration method based on a multi-center projection model, which comprises the steps of shooting calibration plates under different postures by moving the calibration plates or a light field camera, obtaining light field data, determining a plurality of groups of corner points on the calibration plates and matched corner point ray sets, constructing linear constraints of light field coordinate system rays and three-dimensional space points of the light field camera, calculating internal parameters and external parameters under corresponding postures of the light field camera through linear initialization, establishing a cost function based on a reprojection error, and iteratively solving optimal solutions of the internal parameters, the external parameters and the radial distortion parameters of the light field camera to be calibrated. The light field camera essentially records light rays in a space, and because the light rays are parameterized by adopting double parallel plane absolute coordinates, the problem of inaccuracy in three-dimensional point reconstruction is solved, and the purpose of accurately and robustly calibrating parameters in the light field camera is achieved.
Description
Technical Field
The invention belongs to a light field camera calibration method, relates to the fields of computer vision, computational photography and optical engineering, and particularly relates to a light field camera calibration method.
Background
The rise of the optical field imaging theory is a great innovation in the field of computer photography, and breaks through the limitations of the traditional imaging technology. The light field camera reduces the loss of shooting information by recording the position and angle information of light rays in space, and obtains novel imaging effects of variable viewpoint, digital refocusing, depth of field expansion, adjustability and the like. However, the accuracy of camera parameter calibration limits the development of light field camera performance to some extent. The accurate calibration result has great significance for correcting the light field image distortion and improving the imaging quality, and further promotes the wide application of the light field camera in the fields of depth estimation, three-dimensional reconstruction, light field reconstruction, instant positioning, map construction (SLAM) and the like.
In 2013, Dansereau et al set forth a decoding method for light field data initially sampled by a light field camera, put forward a light field camera imaging model containing 12 intrinsic parameters, and design a cost function by using the distance from a three-dimensional point to light rays, thereby completing the calibration of the intrinsic parameters of the light field camera. However, the method relies on the traditional camera array internal reference calibration method to estimate the initial value of the light field camera, and the complexity is high. On the other hand, parameters of an imaging model of the method are redundant, and a dependency relationship exists between a viewpoint coordinate and an image coordinate, so that the problem of non-uniform sampling of light field data in a decoding process is caused. In 2017, Bok et al proposed a projection model of a light field camera with six parameters based on the physical structure of the light field camera, and estimated the internal parameters of the light field camera by using line characteristics as measured values. However, the low resolution of the microlens image limits the accuracy of the line features, thereby affecting the accuracy of the calibration.
Disclosure of Invention
In order to overcome the defects of the prior art, the invention provides a light field camera calibration method based on a multi-center projection model, the multi-center projection model of the light field camera can more accurately describe the projection relation between a three-dimensional space and a light field sub-aperture two-dimensional image, and a set of light field camera calibration method is designed based on the multi-center projection model, so that the internal parameters of the light field camera can be more flexibly and accurately determined.
The technical scheme adopted by the invention for solving the technical problem comprises the following steps:
s1, establishing light field camera double parallel plane relative coordinate parameterization formed by parallel viewpoint planes and image planes, and constructing a light field camera multi-projection center model with a projection center changing along with the viewpoint; light ray r passing through a spatial point on the camera coordinate system of the light field camera ═ (s, t, x, y) T And three-dimensional space point (X, Y, Z) T Can be constructed with a linear constraint that is,
constructing three-dimensional space points (X) under the light field coordinate system of the light field camera according to the linear constraint d ,Y d ,Z d ) T Three-dimensional space point (X, Y, Z) under the camera coordinate system of the light field camera T A three-dimensional internal reference matrix K between them,
wherein, λ is a scaling factor,
is an intrinsic parameter of the light field camera, (k) i ,k j ) Is the scaling of the s-axis and t-axis directions on the viewpoint plane, (k) u ,k v ) Is the scaling of the x-axis y-axis direction on the image plane; (u) 0 /k u ,v 0 /k v ) Characterizing a principal point offset of the sub-aperture image; the three-dimensional space point transforms the world coordinate system to a world coordinate system through a rotation matrix R and a translation vector t of the light field camera, and a three-dimensional projection matrix between the world coordinate system and the light field coordinate system of the light field camera is constructed on the basis;
s2, obtaining a plurality of calibration board light field data with different postures by moving the calibration board or the light field camera to be calibrated; extracting absolute coordinates of double parallel plane light rays under a light field coordinate system of the light field camera from the light field data of the calibration plate according to an angular point extraction algorithm, and establishing a matching relation between the angular point under a world coordinate system and the light rays under the light field coordinate system of the light field camera to be calibrated; linear constraint of the corner point of the world coordinate system and the light ray of the light field coordinate system of the light field camera to be calibrated is constructed through a three-dimensional projection matrix between the world coordinate system and the light field coordinate system,
wherein r is i The ith column vector representing the rotation matrix R, (X) w ,Y w ,0,1) T Is the world coordinate of the corner point, l ═ i, j,u,v) T coordinates of biplane parameterization of light rays in a light field coordinate system; solving a simplified three-dimensional projection matrix P of a world coordinate system and a light field coordinate system according to linear constraints s And then calculating a light field camera three-dimensional internal reference matrix according to the orthogonality and consistency of the rotation matrix R and Cholesky decomposition, and calculating a simplified three-dimensional projection matrix P of each light field s Linearly solving each attitude external parameter (R, t) of the light field camera to be calibrated by the three-dimensional internal reference matrix;
s3, processing the first and second order radial distortion of the lens,
wherein (x) c ,y c ) T Offset value of distortion of image plane relative to viewpoint plane (x, y) T Is a distortion point, (x) u ,y u ) T Is a non-distortion point, and is characterized in that,
the distortion coefficient comprises k d ＝(k 1 ,k 2 ,x c ,y c )；
Light field camera intrinsic parameters by minimizing reprojection errors of light field view sub-aperture images at various poses
Extrinsic parameters (R) of light field camera in different poses p ,t p ) And light field camera radial distortion parameter k d ＝(k 1 ,k 2 ,x c ,y c ) Carrying out nonlinear optimization and constructing a cost function
Wherein, X w,n Three-dimensional coordinates of the nth calibration plate corner point under the world coordinate system,
is X w,n In the p light field, the ith sub-fieldUndistorted coordinates of image points on the aperture image; by minimizing the undistorted coordinates of all image points
And its estimated value
And obtaining the optimal solution of the internal parameter, the external parameter and the radial distortion parameter of the light field camera to be calibrated.
The nonlinear optimization method adopts a Levenberg-Marquardt algorithm.
The beneficial effects of the invention are: the method comprises the steps of shooting calibration plates under different postures by moving the calibration plates or a light field camera to obtain light field data, determining a plurality of groups of corner point light sets on the calibration plates and matched corner point light sets, constructing linear constraints of light rays and three-dimensional space points of a light field coordinate system of the light field camera, calculating internal parameters of the light field camera and external parameters under corresponding postures by linear initialization, establishing a cost function based on reprojection errors, and iteratively solving the optimal solution of the internal parameters, the external parameters and the radial distortion parameters of the light field camera to be calibrated. The light field camera essentially records light rays in a space, and because the light rays are parameterized by adopting double parallel plane absolute coordinates, the problem of inaccuracy in three-dimensional point reconstruction is solved, and the purpose of accurately and robustly calibrating parameters in the light field camera is achieved.
Drawings
FIG. 1(a) is a schematic view of a camera coordinate system and a world coordinate system of a light field camera, wherein the camera coordinate system of the light field camera shows a dual parallel plane and dual parallel plane absolute coordinate parameterization of light rays; FIG. 1(b) is a schematic view of a multi-center projection model of a light field camera as a function of viewpoint derived from a conventional camera projection model under bi-parallel plane relative coordinate parameterized coordinates;
FIG. 2(a) is a schematic optical path diagram of a light field camera that can be applied to embodiments of the present invention; FIG. 2(b) is a schematic diagram illustrating a light field coordinate system definition of a light field camera corresponding to the light path design applied in FIG. 2(a) in an embodiment of the present invention;
FIG. 3 is a schematic diagram of transformation of two parallel planes under a light field coordinate system of a light field camera and two parallel planes under a camera coordinate system of the light field camera, wherein FIG. 3(a) is a schematic diagram of transformation and related intrinsic parameter definition of a light field coordinate system of the light field camera and a viewpoint plane of the camera coordinate system, wherein FIG. 3(b) is a schematic diagram of transformation and related intrinsic parameter definition of an image plane of the light field coordinate system of the light field camera and the camera coordinate system;
FIG. 4 is a schematic diagram of a point in space where two rays intersect in a camera coordinate system;
fig. 5 is a flowchart of light field camera calibration according to an embodiment of the present invention.
Detailed Description
The present invention will be further described with reference to the following drawings and examples, which include, but are not limited to, the following examples.
The light field camera multi-center projection model is provided based on double parallel plane parameterization of the light field camera under absolute coordinates, the problem that three-dimensional point reconstruction is inaccurate under relative coordinates in a traditional method is solved, and meanwhile the problems that modeling is incomplete in the light sampling process of the light field camera and an internal parameter calibration method is inaccurate in the prior art are solved.
The invention provides a calibration method based on a multi-center projection model of a light field camera, which is used for calculating internal parameters and external parameters of the light field camera. The main links comprise: the method comprises the steps of establishing a multi-center projection model of the light field camera under absolute coordinates, linear constraint between three-dimensional points and light field recording light lines, linear solving of internal and external parameters of the light field camera, and establishment of a distortion model and a cost function in nonlinear optimization. The method comprises the following steps:
s1, light field camera multi-center projection model.
Establishing the relative coordinate parameterization of the double parallel planes of the light field camera consisting of the parallel viewpoint plane (s-t plane) and the image plane (x-y plane), and constructing a light field camera multi-projection center model with the projection center changing along with the viewpoint according to a common camera projection model. Light ray r passing through a spatial point on the camera coordinate system of the light field camera ═ (s, t, x, y) T And three-dimensional space point (X, Y, Z) T Can be constructed with the linear constraints of (a),
constructing three-dimensional space points (X) under the light field coordinate system of the light field camera according to the linear constraints d ,Y d ,Z d ) T Three-dimensional space point (X, Y, Z) in the camera coordinate system of the light field camera T A three-dimensional internal reference matrix K between them,
wherein, λ is a scaling factor,
is an intrinsic parameter of the light field camera (k) i ,k j ) Is the scaling of the s-axis and t-axis directions on the viewpoint plane, (k) u ,k v ) Is a scaling of the x-axis y-axis direction in the image plane. In addition, (u) 0 /k u ,v 0 /k v ) The principal point shifts of the sub-aperture images are characterized. The three-dimensional space point transforms the world coordinate system to the world coordinate system through the rotation matrix R and the translation vector t of the light field camera, and a three-dimensional projection matrix between the world coordinate system and the light field coordinate system of the light field camera is constructed on the basis.
S2, linear initialization
Obtaining a plurality of calibration plate light field data with different postures by moving the calibration plate or the light field camera to be calibrated; extracting absolute coordinates of double parallel plane rays under a light field coordinate system of the light field camera from the light field data of the calibration plate according to an angle point extraction algorithm, and establishing a matching relation between an angle point under a world coordinate system and rays under the light field coordinate system of the light field camera to be calibrated; linear constraints of the corner points of the world coordinate system and the light rays of the light field coordinate system of the light field camera to be calibrated are constructed through a three-dimensional projection matrix between the world coordinate system and the light field coordinate system,
wherein r is i The ith column vector representing the rotation matrix R, (X) w ,Y w ,0,1) T Is the world coordinate of the corner point, l ═ i, j, u, v) T Are the coordinates of the biplane parameterization of the light rays in the light field coordinate system. Solving a simplified three-dimensional projection matrix P of a world coordinate system and a light field coordinate system according to linear constraints s And further calculating a light field camera three-dimensional internal reference matrix according to the orthogonality and consistency of the rotation matrix R and Cholesky decomposition, and calculating a simplified three-dimensional projection matrix P of each light field s And linearly solving each attitude external parameter (R, t) of the light field camera to be calibrated by the three-dimensional internal reference matrix.
S3, performing nonlinear optimization on all parameters, specifically as follows:
s3.1, processing the first-order and second-order radial distortion of the lens, wherein due to the unique structure of the light field camera, the influence of the offset between double parallel planes on the distortion is added on the basis of the radial distortion of the traditional camera in the process of designing the first-order and second-order radial distortion model of the lens.
Wherein (x) c ,y c ) T Is the offset value of the distortion of the image plane relative to the viewpoint plane (x, y) T Is a distortion point, (x) u ,y u ) T Is a non-distortion point, and is characterized in that,
the distortion coefficient comprises k d ＝(k 1 ,k 2 ,x c ,y c )。
Methods for linearly calculating internal and external parameters of the light field camera through linear constraints of the angular point under the world coordinate system and the light ray under the light field coordinate system of the light field camera to be calibrated are introduced in S3.2 and S2. In order to further introduce a light field camera mirror image distortion model and obtain an accurate solution of internal and external parameters of the light field camera, the sub-aperture of each viewpoint of the light field under each posture is minimizedReprojection error of image versus light field camera intrinsic parameters
External parameter (R) of light field camera under different postures p ,t p ) And light field camera radial distortion parameter k d ＝(k 1 ,k 2 ,x c ,y c ) Carrying out nonlinear optimization and constructing a cost function
Wherein, X w,n Three-dimensional coordinates of the nth calibration plate corner point under the world coordinate system,
is X w,n The undistorted coordinates of the image point on the ith sub-aperture image in the p-th optical field. By minimizing the undistorted coordinates of all image points
And its estimated value
And obtaining the optimal solution of the internal parameter, the external parameter and the radial distortion parameter of the light field camera to be calibrated. The nonlinear optimization method comprises a Levenberg-Marquardt algorithm, a Gauss-Newton algorithm and the like. Since the Levenberg-Marquardt algorithm is an optimization algorithm based on a gradient domain, combines the advantages of the gradient method and the Newton method, has strong convergence, and can obtain effective results through optimization, the preferred scheme recommends using the Levenberg-Marquardt algorithm, and the invention includes but is not limited to the nonlinear optimization methods.
The method for calibrating the parameters in the light field camera provided by the embodiment of the invention comprises the following steps
S1 light field camera multi-center projection model
S1.1, establishing TPP absolute coordinate parameterization of light field camera light
Hair brushThe method adopts TPP to parameterize the light rays collected by the light field camera, and comprises the following specific steps: defining a 101 viewpoint s-t plane and a 102 image x-y plane of the light field camera, as shown in fig. 1a, where the 101 viewpoint plane is a 0 plane in 104 camera coordinate system Z and the 102 image plane is a f plane in Z, the biplane distance is typically normalized to 1. The light rays of the light field camera can be parameterized with TPP to 106r ═ (s, t, x, y) T To construct a space point (X, Y, Z) T Mapping to absolute coordinates (x, y) of the image plane, as shown in figure 1b,
where λ ═ Z is the scaling factor. Fig. 2a schematically shows an optical path diagram of a light field camera applied to the present embodiment. 201 denotes the main lens of the light field camera, 202 the microlens array is placed 201 at one focal length of the main lens of the light field camera, 204 the light field camera sensor plane is placed 203 at one focal length of the microlens. On the other hand, the light rays recorded by the light field camera with the optical path design shown in fig. 2a, i.e. the parameters of the light ray bi-parallel planes in the light field coordinate system of the light field camera, are represented by l ═ i, j, u, v) T As shown in FIG. 2b, the ray may pass through a homogeneous decoding matrix
Converted to a normalized ray r at the physical scale,
in which, as shown in figure 3,
is an intrinsic parameter of the light field camera, (k) i ,k j ) Is the scaling of the s-axis and t-axis directions in the plane of the 101 viewpoint, (k) u ,k v ) Is the scaling of the x-axis y-axis direction on the image plane 102. In addition to this, (u) 0 /k u ,v 0 /k v ) The principal point shifts of the sub-aperture images are characterized. The decoding matrix D represents the transformation of the double parallel plane parameterized rays in the light field coordinate system of the light field camera to the double parallel plane parameterized rays in the light field coordinate system of the light field camera. The invention can be applied to a light field camera comprising but not limited to the light path design, and only the data recorded by the light field camera is decoded into the double parallel plane parametric coordinates in a specific mode.
S1.2, three-dimensional internal reference matrix between light field coordinate system and camera coordinate system
Given an arbitrary spatial point (X, Y, Z) under the camera coordinate system T As shown in fig. 1b, the ray r passing through the point is (s, t, x, y) T The linear relationship to this point can be constructed by trigonometry,
when at least two light rays r exist m ＝(s m ,t m ,x m ,y m ) T And r n ＝(s n ,t n ,x n ,y n ) T Intersect at a point in space, as shown in FIG. 4, a point in space (X, Y, Z) can be reconstructed T ，
Solving the linear equations set formula 3 three-dimensional space points (X, Y, Z) under the light field coordinate system of the reconstructable light field camera T ，
Substituting equation 2 into equations 4 and 5 allows for three-dimensional space in the camera coordinate systemPunctuation (X, Y, Z) T ，
Integrating the above formula, three-dimensional space point (X) under light field coordinate system d ,Y d ,Z d ) T Three-dimensional space point (X, Y, Z) in coordinate system of camera T A three-dimensional internal reference matrix between the two,
where λ is the scaling factor, equation 8 satisfies the assumption k u /k v ＝k i /k j 。
In general, as shown in FIG. 1a, a point X in a world coordinate system is given 103 w The transformation between the 103 world coordinate system and the 104 camera coordinate system is based on the rotation matrix R ∈ SO (3) and the translation vector
Can be defined as X ═ RX w +t。
S2, linear initialization
S2.1, moving the calibration board or shooting a plurality of calibration board light field data of different postures by the light field camera.
And S2.2, determining the angular points on the multiple groups of calibration plates and the corresponding light rays, and encoding the recorded data of the light field camera.
S2.3, linear constraint of light field camera projection matrix, and solving light field camera projection matrix
Given an arbitrary spatial point (X) in the world coordinate system w ,Y w ,Z w ) T The linear constraint between the space point and the corresponding ray recorded by the light field camera is,
wherein, assuming that N light rays are recorded by the light field camera at a spatial point, M represents a measurement matrix of 2 Nx 4, P is a light field camera projection matrix, r i The i-th column vector of the rotation matrix R is represented. Without loss of generality, it is assumed that the punctuation plate plane is in the world coordinate system Z w On the plane of 0, the upper left corner point is set as the origin of coordinate system, and a simplified light field camera projection matrix P is established between the corner point on the plane of the calibration plate and the corresponding light ray recorded by the light field camera s ，
MP s [X w Y w Z w 1] T ＝0 (10)
Wherein, P s 4 x 3 light field camera projection matrix, K is a 3 x 3 matrix of the three-dimensional projection matrix P at the upper left corner, and the light field camera projection matrix P is calculated by combining the formula 10 and the formula 11 s ，
Wherein the content of the first and second substances,
is a projection matrix P s 12 x 1 column vectors straightened out by row. To solve the internal parametric matrix of the light field camera, h is shown in equation 11 i ＝(h 1i h 2i h 3i ) T Representing the ith column of matrix H.
S2.4, solving internal parameter matrix of light field camera
From the orthogonality and consistency of the rotation matrix R, equation 12 can be derived,
wherein
Characterization with symmetric array B
It is possible to obtain,
using five-dimensional vector b ═ b 11 ,b 13 ,b 22 ,b 23 ,b 33 ) T The non-zero elements of the symmetric matrix B are represented by Vb equal to 0,
where V is a 2n × 5 matrix, at least two linear equations, such as equation 15, are required to calculate the vector b containing the scaling factor. Thus, a symmetric matrix B can be calculated, and then a light field camera projection matrix containing a scaling factor can be obtained by Cholesky decomposition
According to projection matrix
Partial intrinsic parameters (k) of light field camera u ,k v ,u 0 ,v 0 ) It is possible to obtain a solution of,
wherein the content of the first and second substances,
is a light field camera projection matrix
M rows and n columns of elements.
S2.5, solving the external parameters of each light field
From the solved intrinsic parameters of the light field camera, the rotation matrix R and translation vector t of the light field camera can be calculated from equation 11,
to calculate two other intrinsic parameters (k) of a light field camera i ,k j ) The light field internal and external parameters calculated by the equations 16 and 17 are substituted into the equations 9 and X ═ RX w In + t, the reaction is simplified and can be obtained,
using equation 18, the light field camera intrinsic parameter (k) can be calculated by superimposing the multiple pose light field data i ,k j )。
S3 nonlinear optimization
S3.1, processing radial distortion
Due to the unique construction of the light field camera, the light rays are inevitably distorted due to the optical characteristics of the lens and the processing errors of the micro-lenses. The invention adds the influence of the offset between the double parallel planes on the distortion in the process of designing the first-order and second-order radial distortion models of the lens,
wherein (x) c ,yc) T Is the offset value of the distortion of the image plane relative to the viewpoint plane (x, y) T Is a distortion point, (x) u ,y u ) T Is a non-distortion point, and is characterized in that,
the distortion coefficient comprises k d ＝(k 1 ,k 2 ,x c ,y c )。
S3.2, establishing a cost function
S2 introduces a linear solving method for the internal parameters and the external parameters of the light field camera, in order to further obtain an accurate solution of the parameters of the light field camera, the internal parameters, the external parameters and the distortion coefficients of the light field camera are subjected to nonlinear optimization by minimizing the reprojection error of each viewpoint sub-aperture image of each light field, a cost function is constructed,
wherein N is assumed to be present pose Each record marks the light field of the calibration plate, and each calibration plate has N point A corner point, each light field having a total of N view A viewpoint, X w,n Three-dimensional coordinates of the corner point of the nth calibration plate in the world coordinate system,
is X w,n And the image point on the ith sub-aperture image in the pth light field is undistorted. By minimizing all image points undistorted coordinates
And its estimated value
And obtaining the optimal solution of the internal parameter, the external parameter and the radial distortion parameter of the light field camera to be calibrated. The nonlinear optimization method comprises a Levenberg-Marquardt algorithm, a Gauss-Newton algorithm and the like. Since the Levenberg-Marquardt algorithm is an optimization algorithm based on gradient domains, combines the advantages of the gradient method and the Newton method, has strong convergence, and can obtain effective results through optimization, the preferred scheme recommends the use of the Levenberg-Marquardt algorithm, and the invention includes but is not limited to the nonlinear algorithmsAnd (5) an optimization method.
The foregoing description of the preferred embodiments of the present invention has been presented for purposes of illustration and description. It is not intended to be exhaustive or to limit the invention to the precise form disclosed. Obviously, many modifications and variations will be apparent to practitioners skilled in the art. The embodiments were chosen and described in order to best explain the principles of the invention and its practical applications, to thereby enable others skilled in the art to understand the invention for various embodiments and with various modifications as are suited to the particular use contemplated.
Claims (2)
1. A light field camera calibration method based on a multi-center projection model is characterized by comprising the following steps:
s1, establishing a light field camera double-parallel plane relative coordinate parameterization formed by a parallel viewpoint plane and an image plane, and constructing a light field camera multi-projection center model with a projection center changing along with a viewpoint; the ray r passing through a spatial point on the camera coordinate system of the light field camera is (s, t, m, n) T And three-dimensional space point (X, Y, Z) T Can be constructed with a linear constraint that is,
constructing three-dimensional space points (X) under the light field coordinate system of the light field camera according to the linear constraint d ,Y d ,Z d ) T Three-dimensional space point (X, Y, Z) under the camera coordinate system of the light field camera T A three-dimensional internal reference matrix K between them,
wherein, λ is a scaling factor,
is an intrinsic parameter of the light field camera, (k) i ,k j ) Is due to visionScaling of the directions of the s-axis and t-axis in the point plane, (k) u ,k v ) Is the scaling of the x-axis y-axis direction on the image plane; (u) 0 /k u ,v 0 /k v ) Characterizing a principal point offset of the sub-aperture image; the three-dimensional space point transforms the world coordinate system to a world coordinate system through a rotation matrix R and a translation vector t of the light field camera, and a three-dimensional projection matrix between the world coordinate system and the light field coordinate system of the light field camera is constructed on the basis;
s2, obtaining a plurality of calibration board light field data with different postures by moving the calibration board or the light field camera to be calibrated; extracting absolute coordinates of double parallel plane rays under a light field coordinate system of the light field camera from the light field data of the calibration plate according to an angular point extraction algorithm, and establishing a matching relation between an angular point under a world coordinate system and rays under the light field coordinate system of the light field camera to be calibrated; linear constraint of the corner point of the world coordinate system and the light ray of the light field coordinate system of the light field camera to be calibrated is constructed through a three-dimensional projection matrix between the world coordinate system and the light field coordinate system,
wherein r is 1 And r 2 Respectively representing the 1 st, 2 nd column vectors,
is the world coordinate of the corner point, l ═ i, j, u, v) T Coordinates that are biplane parameterizations of light rays in a light field coordinate system; solving a simplified three-dimensional projection matrix P of a world coordinate system and a light field coordinate system according to linear constraints s And further calculating a light field camera three-dimensional internal reference matrix according to the orthogonality and consistency of the rotation matrix R and Cholesky decomposition, and calculating a simplified three-dimensional projection matrix P of each light field s Linearly solving each attitude external parameter (R, t) of the light field camera to be calibrated by the three-dimensional internal reference matrix;
s3, processing the first and second order radial distortion of the lens,
wherein the content of the first and second substances,
is an offset value of distortion of an image plane with respect to a viewpoint plane,
is the point of distortion,
is a non-distortion point, and is characterized in that,
the distortion coefficient comprises k d ＝(k 1 ,k 2 ,x c ,y c )；
Light field camera intrinsic parameters by minimizing reprojection errors of light field view subaperture images at various poses
Extrinsic parameters (R) of light field camera in different poses p ,t p ) And light field camera radial distortion parameter k d ＝(k 1 ,k 2 ,x c ,y c ) Carrying out nonlinear optimization and constructing a cost function
Wherein, X w,n Three-dimensional coordinates of the corner point of the nth calibration plate in the world coordinate system,
is X w,n Undistorted coordinates of image points on the h sub-aperture image in the p light field; by minimizing undistorted coordinates of all image points
And its estimated value
And obtaining the optimal solution of the internal parameter, the external parameter and the radial distortion parameter of the light field camera to be calibrated.
2. The light field camera calibration method based on multi-center projection model according to claim 1, characterized in that: the nonlinear optimization method adopts a Levenberg-Marquardt algorithm.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910547317.XA CN110310338B (en) | 2019-06-24 | 2019-06-24 | Light field camera calibration method based on multi-center projection model |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910547317.XA CN110310338B (en) | 2019-06-24 | 2019-06-24 | Light field camera calibration method based on multi-center projection model |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110310338A CN110310338A (en) | 2019-10-08 |
| CN110310338B true CN110310338B (en) | 2022-09-06 |
Family
ID=68076339
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910547317.XA Active CN110310338B (en) | 2019-06-24 | 2019-06-24 | Light field camera calibration method based on multi-center projection model |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110310338B (en) |
Families Citing this family (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110766761B (en) * | 2019-10-21 | 2023-09-26 | 北京百度网讯科技有限公司 | Method, apparatus, device and storage medium for camera calibration |
| CN110930463B (en) * | 2019-12-06 | 2023-03-10 | 北京迈格威科技有限公司 | Method and device for calibrating internal reference of monitoring camera and electronic equipment |
| CN111080714B (en) * | 2019-12-13 | 2023-05-16 | 太原理工大学 | Parallel binocular camera calibration method based on three-dimensional reconstruction |
| CN113744346B (en) * | 2020-04-02 | 2023-06-23 | 深圳市瑞立视多媒体科技有限公司 | Method, device, equipment and storage medium for calibrating sweeping field and simultaneously calibrating field in large space environment |
| CN111739068B (en) * | 2020-05-06 | 2024-03-01 | 西安电子科技大学 | Light field camera relative pose estimation method |
| CN112581540B (en) * | 2020-12-21 | 2022-11-18 | 东南大学 | Camera calibration method based on human body posture estimation in large scene |
| CN113034612B (en) * | 2021-03-25 | 2023-07-04 | 奥比中光科技集团股份有限公司 | Calibration device, method and depth camera |
| CN113393505B (en) * | 2021-06-25 | 2023-11-03 | 浙江商汤科技开发有限公司 | Image registration method, visual positioning method, related device and equipment |
| CN113808216A (en) * | 2021-08-31 | 2021-12-17 | 上海商汤临港智能科技有限公司 | Camera calibration method and device, electronic equipment and storage medium |
| CN113804124A (en) * | 2021-09-16 | 2021-12-17 | 飞亚达精密科技股份有限公司 | Three-dimensional measurement method and system based on neural network model |
| CN114266836B (en) * | 2022-03-01 | 2022-05-13 | 中国科学院自动化研究所 | Active vision three-dimensional calibration method, system and equipment based on galvanometer camera |
| CN114782545A (en) * | 2022-03-18 | 2022-07-22 | 山东大学 | Light field camera calibration method for eliminating distortion of main lens |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP3144894A1 (en) * | 2015-09-17 | 2017-03-22 | Thomson Licensing | Method and system for calibrating an image acquisition device and corresponding computer program product |
| CN107091810A (en) * | 2017-03-21 | 2017-08-25 | 合肥工业大学 | A kind of rotary optical chromatographic imaging system and imaging method based on linear array detector |
| CN107993266A (en) * | 2017-12-05 | 2018-05-04 | 西北工业大学 | Light-field camera intrinsic parameter caliberating device and method |
| CN108776980A (en) * | 2018-05-14 | 2018-11-09 | 南京工程学院 | A kind of scaling method towards lenticule light-field camera |
| CN109754431A (en) * | 2018-12-20 | 2019-05-14 | 西北工业大学 | Light-field camera scaling method based on self-polar triangle |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150146032A1 (en) * | 2013-11-22 | 2015-05-28 | Vidinoti Sa | Light field processing method |
| CN103914874B (en) * | 2014-04-08 | 2017-02-01 | 中山大学 | Compact SFM three-dimensional reconstruction method without feature extraction |
-
2019
- 2019-06-24 CN CN201910547317.XA patent/CN110310338B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP3144894A1 (en) * | 2015-09-17 | 2017-03-22 | Thomson Licensing | Method and system for calibrating an image acquisition device and corresponding computer program product |
| CN107091810A (en) * | 2017-03-21 | 2017-08-25 | 合肥工业大学 | A kind of rotary optical chromatographic imaging system and imaging method based on linear array detector |
| CN107993266A (en) * | 2017-12-05 | 2018-05-04 | 西北工业大学 | Light-field camera intrinsic parameter caliberating device and method |
| CN108776980A (en) * | 2018-05-14 | 2018-11-09 | 南京工程学院 | A kind of scaling method towards lenticule light-field camera |
| CN109754431A (en) * | 2018-12-20 | 2019-05-14 | 西北工业大学 | Light-field camera scaling method based on self-polar triangle |
Non-Patent Citations (3)
| Title |
|---|
| A Generic Multi-Projection-Center Model and Calibration Method for Light Field Cameras;Q. Zhang 等;《IEEE Transactions on Pattern Analysis and Machine Intelligence》;20180809;第2539-2552页 * |
| Ray Space Features for Plenoptic Structure-from-Motion;Yingliang Zhang 等;《2017 IEEE International Conference on Computer Vision》;20171225;第4641-4649页 * |
| 结构光场光束直线跟踪的两步标定方法;贾小军 等;《光电子·激光》;20181231;第276-286页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110310338A (en) | 2019-10-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110310338B (en) | Light field camera calibration method based on multi-center projection model | |
| Tang et al. | A precision analysis of camera distortion models | |
| Wei et al. | Implicit and explicit camera calibration: Theory and experiments | |
| Coorg et al. | Spherical mosaics with quaternions and dense correlation | |
| Chatterjee et al. | Algorithms for coplanar camera calibration | |
| CN110969668A (en) | Stereoscopic calibration algorithm of long-focus binocular camera | |
| Thirthala et al. | Multi-view geometry of 1d radial cameras and its application to omnidirectional camera calibration | |
| Ramalingam et al. | A generic structure-from-motion framework | |
| Gao et al. | Accuracy evaluation of optical distortion calibration by digital image correlation | |
| TWI435250B (en) | Method for calibrating accuracy of optical touch monitor | |
| Larsson et al. | Revisiting radial distortion absolute pose | |
| CN111145269B (en) | Calibration method for external orientation elements of fisheye camera and single-line laser radar | |
| JP2014178265A (en) | Calibration device, method and program | |
| CN112465877B (en) | Kalman filtering visual tracking stabilization method based on motion state estimation | |
| CN101149836A (en) | Three-dimensional reconfiguration double pick-up camera calibration method | |
| Su et al. | Auto-calibration and real-time external parameter correction for stereo digital image correlation | |
| Eichhardt et al. | Affine correspondences between central cameras for rapid relative pose estimation | |
| CN113744340A (en) | Calibrating cameras with non-central camera models of axial viewpoint offset and computing point projections | |
| CN110298890B (en) | Light field camera calibration method based on Planck parameterization | |
| CN110310337B (en) | Multi-view light field imaging system full-parameter estimation method based on light field fundamental matrix | |
| Dornaika et al. | Robust Camera Calibration using 2D-to-3D feature correspondences | |
| Fusiello | Elements of geometric computer vision | |
| CN115526941B (en) | Calibration device and calibration method for telecentric camera | |
| Dornaika et al. | Stereo geometry from 3d ego-motion streams | |
| Tagoe et al. | Determination of the Interior Orientation Parameters of a Non-metric Digital Camera for Terrestrial Photogrammetric Applications |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |