CN113923445A - Light field camera calibration method and system under condition of shift axis imaging - Google Patents

Abstract

The invention provides a light field camera calibration method and a light field camera calibration system under a tilt-shift imaging condition, wherein the method comprises the following steps: shooting black background white point calibration plates at different depth of field positions to obtain a light field image of a white point at a known three-dimensional position; adjusting the aperture size of each light field camera to the minimum shooting white background to obtain an aperture center projection image; irradiating each light field camera by using parallel light, and shooting a white background to obtain a central image of the micro lens; obtaining the diameters and the center coordinates of the diffusion circles of the white points at different positions imaged in the light field camera according to calculation through the light field image of the white point at the three-dimensional position, the projected image of the center of the aperture and the central image of the micro lens; fitting a mapping function of the diameter and circle center coordinates of the dispersion circle and the space position of the three-dimensional body; and according to the imaging rule of the light field camera and the mapping function of the space position of the three-dimensional body, the light field camera calibration under the condition of shift imaging is realized. The invention can realize the calibration of the light field camera under the condition of tilt-shift imaging and meet the calibration requirement of a multi-light field camera system.

Description

Light field camera calibration method and system under condition of shift axis imaging

Technical Field

The invention relates to the technical field of three-dimensional topography measurement, in particular to a light field camera calibration method and a light field camera calibration system under a tilt-shift imaging condition.

Background

The three-dimensional topography measurement technology is a core technology in the field of machine vision and the field of measurement. Three-dimensional topography measurement refers to measuring three-dimensional information of the surface of an object. In recent years, the appearance of light field cameras provides a new solution for three-dimensional topography measurement. The biggest difference between the light field camera and the traditional two-dimensional camera is that a micro-lens array is added in front of a sensor of the light field camera, so that the propagation direction of light in the space can be recorded, and a special original light field image is formed. In the image processing process, whether the corresponding relation between the object point on the measured object and the pixel influenced by the object point in the space is accurately obtained or not influences the measurement precision to a great extent. In the prior art, the correspondence between an object point on a measured object and a pixel influenced by the object point is determined based on an ideal gaussian optical model.

However, in an actual measurement experiment, as shown in fig. 1, there is an error that cannot be ignored in calculation based on an ideal gaussian optical model due to distortion caused by a difference in shape and refractive index of an optical observation window from air, distortion caused by a main lens mounting error, and distortion caused by a difference in refractive index of a measurement medium from air. In addition, in actual measurement, a certain included angle is formed between the plane of the measured object and the installation plane of the camera, and especially for measuring objects with large bending degree, such as turbine blades, etc., a tilt lens is needed to make the focal plane coincide with the plane of the measured object.

At present, few people research the calibration of the light field camera using the shift lens, that is, the prior art does not disclose the calibration method of the light field camera under the condition of shift imaging.

Patent document CN111351446A (application number: CN202010024192.5) discloses a calibration method of an optical field camera for three-dimensional topography measurement, which calibrates calibration plates at different positions in space and corresponding optical field original images to obtain a corresponding relationship between an optical field parallax image and three-dimensional space information; shooting a plurality of defocusing soft light pure color calibration plates by using a light field camera to obtain a light field white image; calculating according to the white image of the light field camera to obtain a vignetting removing matrix; iterative calculation is carried out to obtain a light field camera microlens subpixel level central coordinate matrix; the method comprises the following steps that a light field camera shoots a plurality of dot calibration plates with known three-dimensional space positions and carries out vignetting removing treatment; the method can efficiently and accurately convert the light field parallax image into the three-dimensional space information without main lens distortion. But the invention is not accurate and efficient and is not suitable for shooting objects under the condition of tilt-shift imaging.

Disclosure of Invention

Aiming at the defects in the prior art, the invention aims to provide a light field camera calibration method and system under the condition of tilt-shift imaging.

The invention provides a light field camera calibration method under a shift imaging condition, which comprises the following steps:

step S1: shooting black background white point calibration plates at different depth of field positions to obtain a light field image of a white point at a known three-dimensional position;

step S2: adjusting the aperture size of each light field camera to the minimum shooting white background to obtain an aperture center projection image;

step S3: irradiating each light field camera by using parallel light, and shooting a white background to obtain a central image of the micro lens;

step S4: obtaining the diameters and the center coordinates of the diffusion circles of the white points at different positions imaged in the light field camera according to calculation through the light field image of the white point at the three-dimensional position, the projected image of the center of the aperture and the central image of the micro lens;

step S5: fitting a mapping function of the diameter and circle center coordinates of the dispersion circle and the space position of the three-dimensional body;

step S6: and according to the imaging rule of the light field camera and the mapping function of the space position of the three-dimensional body, the light field camera calibration under the condition of shift imaging is realized.

Preferably, each white spot on the calibration plate illuminates one or more pixels, an exudate is imaged under each microlens covered by a circle of confusion, and the central position of the ith exudate is recorded as p_c(i)I is an ordinal number, wherein white dots on the calibration plate are arranged at equal intervals;

the white background is a white background with uniform light intensity, each white spot center in the projected image of the center of the aperture is an image generated by projection of the center of the aperture under the microlens, and the projection position of the center of the aperture corresponding to the ith white spot center under the ith microlens is recorded as C_a(i)；

Each exudate center in the central image of the microlens is the center of the microlens, and the ith exudate center, namely the central position of the ith microlens is marked as C_l(i)。

Preferably, in step S4:

the method comprises the following specific steps:

step S4.1: based on the shot projected image of the center of the aperture and the shot image of the center of the micro-lens, calculating the projection coordinate A (u, v) of the center of the aperture on the plane of the micro-lens array and the distance A of the center of the aperture from the plane of the micro-lens array according to a formula I^d：

(C_a(i)-C_l(i))A^d-f_l(C_l(i)-A(u，v))＝0 (1)

Wherein, C_a(i)The center of the ith white spot is the projection position of the center of a diaphragm corresponding to the ith micro lens; c_l(i)Is the ith white spot center, namely the ith microlens center position; f. of_lIs one focal length of the microlens;

the condition that the first formula is satisfied is as follows:

the light field camera is formed by that the micro lens array is positioned in front of the imaging sensor, and the distance between the micro lens array and the imaging sensor is one focal length f of the micro lens_lThe focusing light field camera of (1);

b1, for the center of the aperture, n formula I can be obtained when n microlenses are used for imaging, n is the number of the microlenses, and when n is larger than or equal to 3, A (u, v) and A are obtained by solving an overdetermined equation system^d；

Step S4.2: based on the shot light field image of the black bottom white point calibration plate, the ith white point O of the three-dimensional space calibration plate is calculated according to a formula II_iProjection coordinate Q of (x, y, z) corresponding convergent imaging point on the plane of the micro-lens array_i(u, v) and a converging imaging point Q_iDistance Q from the plane of the microlens array_i ^d：

(p_c(i)-C_l(i))Q_i ^d-f_l(C_l(i)-Q_i(u，v))＝0 (2)

Wherein the central position of the ith white spot is p_c(i)；

The condition that the second formula is satisfied is as follows:

a2, the light field camera is a focusing light field camera with a micro lens array positioned in front of the imaging sensor by one time of focal length;

(b2) central position C of microlens_l(i)The corresponding microlens is completely covered by the dispersion circle;

for a point light source O_iIf n microlenses satisfy the condition b2, n formulas II are obtained, and when n is larger than or equal to 3, Q is obtained by solving an overdetermined equation set_i(u, v) and Q_i ^d；

Step S4.3: based on the shot light field image of the black bottom white point calibration plate, the ith white point O of the three-dimensional space calibration plate is calculated according to the formula III_i(x, y, z) corresponding circle center coordinates C of diffusion circle_df(i)(u，v)：

A^dQ_i(u，v)-C_df(i)(u，v)(A^d-Q_i ^d)＝0 (3)

Step S4.4: black background based on shootingCalculating the ith white point O of the three-dimensional space calibration plate according to the formula three and four_i(x, y, z) corresponding to the circle of confusion D_df(i)：

p_c(j)The central position of the jth white spot;

the condition that the formula four holds is as follows:

a3, the light field camera is a focusing light field camera with a micro lens array positioned in front of the imaging sensor by one time of focal length;

b3 center position of microlens C_l(i)The corresponding microlens is completely covered by the dispersion circle;

for a point light source O_iThere are n microlenses satisfying the condition b3, i.e., n white spots p_c(i)When n is more than or equal to 2, solving the maximum distance max | p between the white spots_c(i)-p_c(j)I, j is left to n to obtain D_df(i)。

Preferably, in step S5:

the method comprises the following specific steps:

step S5.1: establishing a mapping model to fit the ith white point O of the three-dimensional space calibration plate_i(x, y, z) and the coordinates C of the center of the circle corresponding to the circle of confusion_df(i)(u, v) mapping function relationship

The mapping model is as follows:

wherein, λ represents a mapping relation coefficient,

a mapping matrix is represented that is,

by calibrating all white points O on the board as point light sources_i(x, y, z) and corresponding circle center coordinates C of circle of confusion_df(i)(u, v) is obtained; u and v respectively represent coordinates of a rectangular coordinate system; x, y and z respectively represent coordinates of a space rectangular coordinate system;

step S5.2: polynomial fitting three-dimensional space calibration plate ith white point O_i(x, y, z) and a converging imaging point Q_iDistance Q from the plane of the microlens array_i ^dMapping functional relationships

The model is as follows:

Q_i ^d＝a₀+a₁C_df(i)(u)+a₂C_df(i)(v)+a₃z+a₄C_df(i)(u)C_df(i)(v)+a₅C_df(i)(u)z+a₆(z)²

+a₇C_df(i)(v)z+a₈(C_df(i)(v))²+a₉(C_df(i)(u))²+a₁₀(C_df(i)(u))²C_df(i)(v)

+a₁₁(C_df(i)(v))²C_df(i)(u)+a₁₂(C_df(i)(u))²z+a₁₃(C_df(i)(v))²z

+a₁₄(z)²C_df(i)(v)+a₁₅(z)²C_df(i)(u)+a₁₆C_df(i)(u)C_df(i)(v)z

+a₁₇(C_df(i)(u))³+a₁₈(C_df(i)(v))³+a₁₉(z)³

wherein Q is_i ^dThe distance from the convergent imaging point to the plane of the microlens array; a is_0-19Is a polynomial coefficient; c_df(i)(u) is the abscissa of the center of the circle of confusion, C_df(i)(v) Is a longitudinal coordinate of the center of a circle of a dispersion circle;

step S5.3: establishing three-dimensional spatial calibrationPlate ith white point O_i(x, y, z) and the diameter D of the circle of confusion_df(i)Mapping function relationship of

The model is as follows:

wherein Q is_i(u, v) is the projection coordinate of a convergent imaging point corresponding to the ith white point of the three-dimensional space calibration plate on the plane of the micro-lens array; c_df(i)(u, v) are coordinates of the center of the circle of dispersion; a. the^dThe distance from the center of the aperture to the plane of the microlens array; d_df(i)Is the diameter of a dispersion circle; a (u, v) is the projection coordinate of the aperture center on the microlens array plane; the coefficients α, β, γ pass through all white points O on the calibration plate as point sources_iC for (x, y, z)_df(i)(u，v)、Q_i ^dAnd diameter D of circle of dispersion_df(i)Fitting to obtain; a. the_mDenotes the main lens aperture diameter, A_m＝f_m(1-M)/f_#，f_mDenotes the main lens focal length, M denotes the magnification factor, f_#Represents the aperture value of the main lens, and theta represents the shift angle;

is a point on the circle of dispersion,

is another point on the dispersion circle and

and is symmetrical about the circle center of dispersion.

Preferably, in step S6:

according to the imaging rule and the mapping function of the light field camera, calculating to obtain a white point O at any three-dimensional space position_j(x, y, z) corresponding circle center coordinates C of diffusion circle_df(j)(X, Y) and circle of confusion diameter D_df(j)And a microlens center position C_l(j)After all the micro-lenses influenced by the white point of any three-dimensional space position are obtained through calculation, the white point O of any space position is found_j(x, y, z) all pixels affected;

calculating the boundary of the affected area under each microlens according to the following two formulas, and determining all pixels affected under each microlens according to the boundary;

the upper bound formula:

lower boundary formula:

the upper boundary of the region under the microlens which is affected, C_l(j)Is the j-th microlens center position, f_lIs one focal length of the microlens, C_l(j)(u) is the central abscissa of the ith microlens, Q_i(u) is the convergent imaging point abscissa, C_l(j)(v) Is the central ordinate, p, of the ith microlens_lIs the pixel size of the microlens, p_pFor sensor pixel size, Q_i(v) Is a vertical coordinate of a convergent imaging point; q_i ^dThe distance from the convergent imaging point to the plane of the microlens array;

the lower boundary of the affected zone boundary under the microlens.

According to the invention, the light field camera calibration system used under the condition of shift-axis imaging comprises:

module M1: shooting black background white point calibration plates at different depth of field positions to obtain a light field image of a white point at a known three-dimensional position;

module M2: adjusting the aperture size of each light field camera to the minimum shooting white background to obtain an aperture center projection image;

module M3: irradiating each light field camera by using parallel light, and shooting a white background to obtain a central image of the micro lens;

module M4: obtaining the diameters and the center coordinates of the diffusion circles of the white points at different positions imaged in the light field camera according to calculation through the light field image of the white point at the three-dimensional position, the projected image of the center of the aperture and the central image of the micro lens;

module M5: fitting a mapping function of the diameter and circle center coordinates of the dispersion circle and the space position of the three-dimensional body;

module M6: and according to the imaging rule of the light field camera and the mapping function of the space position of the three-dimensional body, the light field camera calibration under the condition of shift imaging is realized.

Preferably, each white spot on the calibration plate illuminates one or more pixels, an exudate is imaged under each microlens covered by a circle of confusion, and the central position of the ith exudate is recorded as p_c(i)I is an ordinal number, wherein white dots on the calibration plate are arranged at equal intervals;

the white background is a white background with uniform light intensity, each white spot center in the projected image of the center of the aperture is an image generated by projection of the center of the aperture under the microlens, and the projection position of the center of the aperture corresponding to the ith white spot center under the ith microlens is recorded as C_a(i)；

Each white spot center in the central image of the microlens is the center of each white spotThe center of the microlens, i.e. the center position of the ith microlens, is marked as C_l(i)。

Preferably, in module M4:

the method comprises the following specific steps:

module M4.1: based on the shot projected image of the center of the aperture and the shot image of the center of the micro-lens, calculating the projection coordinate A (u, v) of the center of the aperture on the plane of the micro-lens array and the distance A of the center of the aperture from the plane of the micro-lens array according to a formula I^d：

(C_a(i)-C_l(i))A^d-f_l(C_l(i)-A(u，v))＝0 (1)

Wherein, C_a(i)The center of the ith white spot is the projection position of the center of a diaphragm corresponding to the ith micro lens; c_l(i)Is the ith white spot center, namely the ith microlens center position; f. of_lIs one focal length of the microlens;

the condition that the first formula is satisfied is as follows:

a1, the light field camera is a microlens array in front of the imaging sensor and the distance between the microlens array and the imaging sensor is one focal length f of the microlens_lThe focusing light field camera of (1);

b1, for the center of the aperture, n formula I can be obtained when n microlenses are used for imaging, n is the number of the microlenses, and when n is larger than or equal to 3, A (u, v) and A are obtained by solving an overdetermined equation system^d；

Module M4.2: based on the shot light field image of the black bottom white point calibration plate, the ith white point O of the three-dimensional space calibration plate is calculated according to a formula II_iProjection coordinate Q of (x, y, z) corresponding convergent imaging point on the plane of the micro-lens array_i(u, v) and a converging imaging point Q_iDistance Q from the plane of the microlens array_i ^d：

(p_c(i)-C_l(i))Q_i ^d-f_l(C_l(i)-Q_i(u，v))＝0 (2)

Wherein the central position of the ith white spot is p_c(i)；

The condition that the second formula is satisfied is as follows:

a2, the light field camera is a focusing light field camera with a micro lens array positioned in front of the imaging sensor by one time of focal length;

(b2) central position C of microlens_l(i)The corresponding microlens is completely covered by the dispersion circle;

for a point light source O_iIf n microlenses satisfy the condition b2, n formulas II are obtained, and when n is larger than or equal to 3, Q is obtained by solving an overdetermined equation set_i(u, v) and Q_i ^d；

Module M4.3: based on the shot light field image of the black bottom white point calibration plate, the ith white point O of the three-dimensional space calibration plate is calculated according to the formula III_i(x, y, z) corresponding circle center coordinates C of diffusion circle_df(i)(u，v)：

A^dQ_i(u，v)-C_df(i)(u，v)(A^d-Q_i ^d)＝0 (3)

Module M4.4: calculating the ith white point O of the three-dimensional space calibration plate according to the formula three and four based on the shot light field image of the black background white point calibration plate_i(x, y, z) corresponding to the circle of confusion D_df(i)：

p_c(j)The central position of the jth white spot;

the condition that the formula four holds is as follows:

a3, the light field camera is a focusing light field camera with a micro lens array positioned in front of the imaging sensor by one time of focal length;

b3 center position of microlens C_l(i)The corresponding microlens is completely covered by the dispersion circle;

for a point light source O_iThere are n microlenses satisfying the condition b3, i.e., n white spots p_c(i)When n is more than or equal to 2, solving the maximum distance max | p between the white spots_c(i)-p_c(j)I, j is left to n to obtain D_df(i)。

Preferably, in module M5:

the method comprises the following specific steps:

module M5.1: establishing a mapping model to fit the ith white point O of the three-dimensional space calibration plate_i(x, y, z) and the coordinates C of the center of the circle corresponding to the circle of confusion_df(i)(u, v) mapping function relationship

The mapping model is as follows:

wherein, λ represents a mapping relation coefficient,

a mapping matrix is represented that is,

by calibrating all white points O on the board as point light sources_i(x, y, z) and corresponding circle center coordinates C of circle of confusion_df(i)(u, v) is obtained; u and v respectively represent coordinates of a rectangular coordinate system; x, y and z respectively represent coordinates of a space rectangular coordinate system;

module M5.2: polynomial fitting three-dimensional space calibration plate ith white point O_i(x, y, z) and a converging imaging point Q_iDistance Q from the plane of the microlens array_i ^dMapping functional relationships

The model is as follows:

Q_i ^d＝a₀+a₁C_df(i)(u)+a₂C_df(i)(v)+a₃z+a₄C_df(i)(u)C_df(i)(v)+a₅C_df(i)(u)z+a₆(z)²

+a₇C_df(i)(v)z+a₈(C_df(i)(v))²+a₉(C_df(i)(u))²+a₁₀(C_df(i)(u))²C_df(i)(v)

+a₁₁(C_df(i)(v))²C_df(i)(u)+a₁₂(C_df(i)(u))²z+a₁₃(C_df(i)(v))²z

+a₁₄(z)²C_df(i)(v)+a₁₅(z)²C_df(i)(u)+a₁₆C_df(i)(u)C_df(i)(v)z

+a₁₇(C_df(i)(u))³+a₁₈(C_df(i)(v))³+a₁₉(z)³

wherein Q is_i ^dThe distance from the convergent imaging point to the plane of the microlens array; a is_0-19Is a polynomial coefficient; c_df(i)(u) is the abscissa of the center of the circle of confusion, C_df(i)(v) Is a longitudinal coordinate of the center of a circle of a dispersion circle;

module M5.3: establishing the ith white point O of the three-dimensional space calibration plate_i(x, y, z) and the diameter D of the circle of confusion_df(i)Mapping function relationship of

The model is as follows:

wherein Q is_i(u, v) is the projection of a convergent imaging point corresponding to the ith white point of the three-dimensional space calibration plate on the plane of the micro-lens arrayShadow coordinates; c_df(i)(u, v) are coordinates of the center of the circle of dispersion; a. the^dThe distance from the center of the aperture to the plane of the microlens array; d_df(i)Is the diameter of a dispersion circle; a (u, v) is the projection coordinate of the aperture center on the microlens array plane; the coefficients α, β, γ pass through all white points O on the calibration plate as point sources_iC for (x, y, z)_df(i)(u，v)、Q_i ^dAnd diameter D of circle of dispersion_df(i)Fitting to obtain; a. the_mDenotes the main lens aperture diameter, A_m＝f_m(1-M)/f_#，f_mDenotes the main lens focal length, M denotes the magnification factor, f_#Represents the aperture value of the main lens, and theta represents the shift angle;

is a point on the circle of dispersion,

is another point on the dispersion circle and

and is symmetrical about the circle center of dispersion.

Preferably, in module M6:

according to the imaging rule and the mapping function of the light field camera, calculating to obtain a white point O at any three-dimensional space position_j(x, y, z) corresponding circle center coordinates C of diffusion circle_df(j)(X, Y) and circle of confusion diameter D_df(j)And a microlens center position C_l(j)After all the micro-lenses influenced by the white point of any three-dimensional space position are obtained through calculation, the white point O of any space position is found_j(x, y, z) all pixels affected;

calculating the boundary of the affected area under each microlens according to the following two formulas, and determining all pixels affected under each microlens according to the boundary;

the upper bound formula:

lower boundary formula:

the upper boundary of the region under the microlens which is affected, C_l(j)Is the j-th microlens center position, f_lIs one focal length of the microlens, C_l(j)(u) is the central abscissa of the ith microlens, Q_i(u) is the convergent imaging point abscissa, C_l(j)(v) Is the central ordinate, p, of the ith microlens_lIs the pixel size of the microlens, p_pFor sensor pixel size, Q_i(v) Is a vertical coordinate of a convergent imaging point; q_i ^dThe distance from the convergent imaging point to the plane of the microlens array;

the lower boundary of the affected zone boundary under the microlens.

Compared with the prior art, the invention has the following beneficial effects:

1. the method can accurately calculate all pixels illuminated by light rays emitted by a point light source in space under the condition of shift imaging, and the light field camera shoots the three-dimensional space calibration plate to observe the distortion caused by the refractive index and the shape of the window and the installation error of the main lens. Compared with the conventional calculation method, the method is more accurate and efficient and is more suitable for shooting the object under the condition of shift imaging.

2. The invention can provide a unified coordinate system for the multi-light field camera system, and meets the calibration requirement of the multi-light field camera system;

3. the invention can establish the corresponding relation between the space coordinate of an object point in the space and the pixel influenced by the object point.

Drawings

Other features, objects and advantages of the invention will become more apparent upon reading of the detailed description of non-limiting embodiments with reference to the following drawings:

FIG. 1 is a diagram of the optical distortion present in light field camera imaging that needs to be addressed under tilt-shift imaging conditions in accordance with the present invention;

FIG. 2 is a flow chart of an embodiment of the present invention;

FIG. 3 is a schematic view of the apparatus and calibration plate for calibration according to the present invention;

FIG. 4 is a light field image of a calibration plate white point of the present invention;

FIG. 5 is a projected image of the aperture center obtained by shooting a white background when the aperture size of the light field camera is adjusted to the minimum;

FIG. 6 shows the microlens center image obtained by the present invention when parallel light is applied to each light field camera with the main lens removed.

Detailed Description

The present invention will be described in detail with reference to specific examples. The following examples will assist those skilled in the art in further understanding the invention, but are not intended to limit the invention in any way. It should be noted that it would be obvious to those skilled in the art that various changes and modifications can be made without departing from the spirit of the invention. All falling within the scope of the present invention.

Example 1:

the invention provides a light field camera calibration method under a shift imaging condition, which comprises the following steps:

step S1: shooting black background white point calibration plates at different depth of field positions to obtain a light field image of a white point at a known three-dimensional position;

step S2: adjusting the aperture size of each light field camera to the minimum shooting white background to obtain an aperture center projection image;

step S3: irradiating each light field camera by using parallel light, and shooting a white background to obtain a central image of the micro lens;

step S4: obtaining the diameters and the center coordinates of the diffusion circles of the white points at different positions imaged in the light field camera according to calculation through the light field image of the white point at the three-dimensional position, the projected image of the center of the aperture and the central image of the micro lens;

step S5: fitting a mapping function of the diameter and circle center coordinates of the dispersion circle and the space position of the three-dimensional body;

step S6: and according to the imaging rule of the light field camera and the mapping function of the space position of the three-dimensional body, the light field camera calibration under the condition of shift imaging is realized.

Specifically, each white spot on the calibration plate can illuminate one or more pixels, an exudate is imaged under each microlens covered by a circle of confusion, and the central position of the ith exudate is recorded as p_c(i)I is an ordinal number, wherein white dots on the calibration plate are arranged at equal intervals;

the white background is a white background with uniform light intensity, each white spot center in the projected image of the center of the aperture is an image generated by projection of the center of the aperture under the microlens, and the projection position of the center of the aperture corresponding to the ith white spot center under the ith microlens is recorded as C_a(i)；

Each exudate center in the central image of the microlens is the center of the microlens, and the ith exudate center, namely the central position of the ith microlens is marked as C_l(i)。

Specifically, in step S4:

the method comprises the following specific steps:

step S4.1: based on the shot projected image of the center of the aperture and the shot image of the center of the micro-lens, calculating the projection coordinate A (u, v) of the center of the aperture on the plane of the micro-lens array and the distance A of the center of the aperture from the plane of the micro-lens array according to a formula I^d：

(C_a(i)-C_l(i))A^d-f_l(C_l(i)-A(u，v))＝0 (1)

Wherein, C_a(i)The center of the ith white spot is the projection position of the center of a diaphragm corresponding to the ith micro lens; c_l(i)Is the ith white spot center, namely the ith microlens center position; f. of_lIs one focal length of the microlens;

the condition that the first formula is satisfied is as follows:

a1 light field cameraThe micro lens array is arranged in front of the imaging sensor and the distance between the micro lens array and the imaging sensor is one focal length f of the micro lens_lThe focusing light field camera of (1);

b1, for the center of the aperture, n formula I can be obtained when n microlenses are used for imaging, n is the number of the microlenses, and when n is larger than or equal to 3, A (u, v) and A are obtained by solving an overdetermined equation system^d；

Step S4.2: based on the shot light field image of the black bottom white point calibration plate, the ith white point O of the three-dimensional space calibration plate is calculated according to a formula II_iProjection coordinate Q of (x, y, z) corresponding convergent imaging point on the plane of the micro-lens array_i(u, v) and a converging imaging point Q_iDistance Q from the plane of the microlens array_i ^d：

(p_c(i)-C_l(i))Q_i ^d-f_l(C_l(i)-Q_i(u，v))＝0 (2)

Wherein the central position of the ith white spot is p_c(i)；

The condition that the second formula is satisfied is as follows:

a2, the light field camera is a focusing light field camera with a micro lens array positioned in front of the imaging sensor by one time of focal length;

(b2) central position C of microlens_l(i)The corresponding microlens is completely covered by the dispersion circle;

for a point light source O_iIf n microlenses satisfy the condition b2, n formulas II are obtained, and when n is larger than or equal to 3, Q is obtained by solving an overdetermined equation set_i(u, v) and Q_i ^d；

Step S4.3: based on the shot light field image of the black bottom white point calibration plate, the ith white point O of the three-dimensional space calibration plate is calculated according to the formula III_i(x, y, z) corresponding circle center coordinates C of diffusion circle_df(i)(u，v)：

A^dQ_i(u，v)-C_df(i)(u，v)(A^d-Q_i ^d)＝0 (3)

Step S4.4: calculating the ith white point O of the three-dimensional space calibration plate according to the formula three and four based on the shot light field image of the black background white point calibration plate_i(x, y, z) corresponding to the circle of confusion D_df(i)：

p_c(j)The central position of the jth white spot;

the condition that the formula four holds is as follows:

a3, the light field camera is a focusing light field camera with a micro lens array positioned in front of the imaging sensor by one time of focal length;

b3 center position of microlens C_l(i)The corresponding microlens is completely covered by the dispersion circle;

for a point light source O_iThere are n microlenses satisfying the condition b3, i.e., n white spots p_c(i)When n is more than or equal to 2, solving the maximum distance max | p between the white spots_c(i)-p_c(j)I, j is left to n to obtain D_df(i)。

Specifically, in step S5:

the method comprises the following specific steps:

step S5.1: establishing a mapping model to fit the ith white point O of the three-dimensional space calibration plate_i(x, y, z) and the coordinates C of the center of the circle corresponding to the circle of confusion_df(i)(u, v) mapping function relationship

The mapping model is as follows:

wherein, λ represents a mapping relation coefficient,

a mapping matrix is represented that is,

by calibrating all white points O on the board as point light sources_i(x, y, z) and correspondingCircle center coordinate C of dispersion circle_df(i)(u, v) is obtained; u and v respectively represent coordinates of a rectangular coordinate system; x, y and z respectively represent coordinates of a space rectangular coordinate system;

step S5.2: polynomial fitting three-dimensional space calibration plate ith white point O_i(x, y, z) and a converging imaging point Q_iDistance Q from the plane of the microlens array_i ^dMapping functional relationships

The model is as follows:

Q_i ^d＝a₀+a₁C_df(i)(u)+a₂C_df(i)(v)+a₃z+a₄C_df(i)(u)C_df(i)(v)+a₅C_df(i)(u)z+a₆(z)²

+a₇C_df(i)(v)z+a₈(C_df(i)(v))²+a₉(C_df(i)(u))²+a₁₀(C_df(i)(u))²C_df(i)(v)

+a₁₁(C_df(i)(v))²C_df(i)(u)+a₁₂(C_df(i)(u))²z+a₁₃(C_df(i)(v))²z

+a₁₄(z)²C_df(i)(v)+a₁₅(z)²C_df(i)(u)+a₁₆C_df(i)(u)C_df(i)(v)z

+a₁₇(C_df(i)(u))³+a₁₈(C_df(i)(v))³+a₁₉(z)³

wherein Q is_i ^dThe distance from the convergent imaging point to the plane of the microlens array; a is_0-19Is a polynomial coefficient; c_df(i)(u) is the abscissa of the center of the circle of confusion, C_df(i)(v) Is a longitudinal coordinate of the center of a circle of a dispersion circle;

step S5.3: establishing the ith white point O of the three-dimensional space calibration plate_i(x, y, z) and the diameter D of the circle of confusion_df(i)Mapping function relationship of

The model is as follows:

wherein Q is_i(u, v) is the projection coordinate of a convergent imaging point corresponding to the ith white point of the three-dimensional space calibration plate on the plane of the micro-lens array; c_df(i)(u, v) are coordinates of the center of the circle of dispersion; a. the^dThe distance from the center of the aperture to the plane of the microlens array; d_df(i)Is the diameter of a dispersion circle; a (u, v) is the projection coordinate of the aperture center on the microlens array plane; the coefficients α, β, γ pass through all white points O on the calibration plate as point sources_iC for (x, y, z)_df(i)(u，v)、Q_i ^dAnd diameter D of circle of dispersion_df(i)Fitting to obtain; a. the_mDenotes the main lens aperture diameter, A_m＝f_m(1-M)/f_#，f_mDenotes the main lens focal length, M denotes the magnification factor, f_#Represents the aperture value of the main lens, and theta represents the shift angle;

is a point on the circle of dispersion,

is another point on the dispersion circle and

and is symmetrical about the circle center of dispersion.

Specifically, in step S6:

according to the imaging rule and the mapping function of the light field camera, calculating to obtain a white point O at any three-dimensional space position_j(x, y, z) corresponding circle center coordinates C of diffusion circle_df(j)(X, Y) and circle of confusion diameter D_df(j)And a microlens center position C_l(j)After all the micro-lenses influenced by the white point of any three-dimensional space position are obtained through calculation, the white point O of any space position is found_j(x, y, z) all pixels affected;

calculating the boundary of the affected area under each microlens according to the following two formulas, and determining all pixels affected under each microlens according to the boundary;

the upper bound formula:

lower boundary formula:

the upper boundary of the region under the microlens which is affected, C_l(j)Is the j-th microlens center position, f_lIs one focal length of the microlens, C_l(j)(u) is the central abscissa of the ith microlens, Q_i(u) is the convergent imaging point abscissa, C_l(j)(v) Is the central ordinate, p, of the ith microlens_lIs the pixel size of the microlens, p_pFor sensor pixel size, Q_i(v) Is a vertical coordinate of a convergent imaging point; q_i ^dThe distance from the convergent imaging point to the plane of the microlens array;

the lower boundary of the affected zone boundary under the microlens.

Example 2:

example 2 is a preferred example of example 1, and the present invention will be described in more detail.

The person skilled in the art may understand that the light field camera calibration method under the condition of tilt-shift imaging according to the present invention is a specific implementation of the light field camera calibration system under the condition of tilt-shift imaging, that is, the light field camera calibration system under the condition of tilt-shift imaging may be implemented by executing the flow of steps of the light field camera calibration method under the condition of tilt-shift imaging.

According to the invention, the light field camera calibration system used under the condition of shift-axis imaging comprises:

module M1: shooting black background white point calibration plates at different depth of field positions to obtain a light field image of a white point at a known three-dimensional position;

module M2: adjusting the aperture size of each light field camera to the minimum shooting white background to obtain an aperture center projection image;

module M3: irradiating each light field camera by using parallel light, and shooting a white background to obtain a central image of the micro lens;

module M4: obtaining the diameters and the center coordinates of the diffusion circles of the white points at different positions imaged in the light field camera according to calculation through the light field image of the white point at the three-dimensional position, the projected image of the center of the aperture and the central image of the micro lens;

module M5: fitting a mapping function of the diameter and circle center coordinates of the dispersion circle and the space position of the three-dimensional body;

module M6: and according to the imaging rule of the light field camera and the mapping function of the space position of the three-dimensional body, the light field camera calibration under the condition of shift imaging is realized.

Specifically, each white spot on the calibration plate illuminates one or more pixels, and a white spot is imaged under each microlens covered by a circle of confusion, and the ith white spot is recordedThe central position is p_c(i)I is an ordinal number, wherein white dots on the calibration plate are arranged at equal intervals;

the white background is a white background with uniform light intensity, each white spot center in the projected image of the center of the aperture is an image generated by projection of the center of the aperture under the microlens, and the projection position of the center of the aperture corresponding to the ith white spot center under the ith microlens is recorded as C_a(i)；

Each exudate center in the central image of the microlens is the center of the microlens, and the ith exudate center, namely the central position of the ith microlens is marked as C_l(i)。

Specifically, in module M4:

the method comprises the following specific steps:

module M4.1: based on the shot projected image of the center of the aperture and the shot image of the center of the micro-lens, calculating the projection coordinate A (u, v) of the center of the aperture on the plane of the micro-lens array and the distance A of the center of the aperture from the plane of the micro-lens array according to a formula I^d：

(C_a(i)-C_l(i))A^d-f_l(C_l(i)-A(u，v))＝0 (1)

Wherein, C_a(i)The center of the ith white spot is the projection position of the center of a diaphragm corresponding to the ith micro lens; c_l(i)Is the ith white spot center, namely the ith microlens center position; f. of_lIs one focal length of the microlens;

the condition that the first formula is satisfied is as follows:

a1, the light field camera is a microlens array in front of the imaging sensor and the distance between the microlens array and the imaging sensor is one focal length f of the microlens_lThe focusing light field camera of (1);

b1, for the center of the aperture, n formula I can be obtained when n microlenses are used for imaging, n is the number of the microlenses, and when n is larger than or equal to 3, A (u, v) and A are obtained by solving an overdetermined equation system^d；

Module M4.2: based on the shot light field image of the black bottom white point calibration plate, the ith white point O of the three-dimensional space calibration plate is calculated according to a formula II_i(x, y, z) corresponding to a converging imaging point in the plane of the microlens arrayProjection coordinate Q of_i(u, v) and a converging imaging point Q_iDistance Q from the plane of the microlens array_i ^d：

(p_c(i)-C_l(i))Q_i ^d-f_l(C_l(i)-Q_i(u，v))＝0 (2)

Wherein the central position of the ith white spot is p_c(i)；

The condition that the second formula is satisfied is as follows:

a2, the light field camera is a focusing light field camera with a micro lens array positioned in front of the imaging sensor by one time of focal length;

(b2) central position C of microlens_l(i)The corresponding microlens is completely covered by the dispersion circle;

for a point light source O_iIf n microlenses satisfy the condition b2, n formulas II are obtained, and when n is larger than or equal to 3, Q is obtained by solving an overdetermined equation set_i(u, v) and Q_i ^d；

Module M4.3: based on the shot light field image of the black bottom white point calibration plate, the ith white point O of the three-dimensional space calibration plate is calculated according to the formula III_iCircle center coordinates C of diffusion circle corresponding to (x, y, Z)_df(i)(u，v)：

A^dQ_i(u，v)-C_df(i)(u，v)(A^d-Q_i ^d)＝0 (3)

Module M4.4: calculating the ith white point O of the three-dimensional space calibration plate according to the formula three and four based on the shot light field image of the black background white point calibration plate_i(x, y, z) corresponding to the circle of confusion D_df(i)：

p_c(j)The central position of the jth white spot;

the condition that the formula four holds is as follows:

a3, the light field camera is a focusing light field camera with a micro lens array positioned in front of the imaging sensor by one time of focal length;

b3 center position of microlens C_l(i)The corresponding microlens is completely covered by the dispersion circle;

for a point light source O_iThere are n microlenses satisfying the condition b3, i.e., n white spots p_c(i)When n is more than or equal to 2, solving the maximum distance max | p between the white spots_c(i)-p_c(j)I, j is left to n to obtain D_df(i)。

Specifically, in module M5:

the method comprises the following specific steps:

module M5.1: establishing a mapping model to fit the ith white point O of the three-dimensional space calibration plate_i(x, y, z) and the coordinates C of the center of the circle corresponding to the circle of confusion_df(i)(u, v) mapping function relationship

The mapping model is as follows:

wherein, λ represents a mapping relation coefficient,

a mapping matrix is represented that is,

by calibrating all white points O on the board as point light sources_i(x, y, z) and corresponding circle center coordinates C of circle of confusion_df(i)(u, v) is obtained; u and v respectively represent coordinates of a rectangular coordinate system; x, y and z respectively represent coordinates of a space rectangular coordinate system;

module M5.2: polynomial fitting three-dimensional space calibration plate ith white point O_i(x, y, z) and a converging imaging point Q_iDistance Q from the plane of the microlens array_i ^dMapping functional relationships

The model is as follows:

Q_i ^d＝a₀+a₁C_df(i)(u)+a₂C_df(i)(v)+a₃z+a₄C_df(i)(u)C_df(i)(v)+a₅C_df(i)(u)z+a₆(z)²

+a₇C_df(i)(v)z+a₈(C_df(i)(v))²+a₉(C_df(i)(u))²+a₁₀(C_df(i)(u))²C_df(i)(v)

+a₁₁(C_df(i)(v))²C_df(i)(u)+a₁₂(C_df(i)(u))²z+a₁₃(C_df(i)(v))²z

+a₁₄(z)²C_df(i)(v)+a₁₅(z)²C_df(i)(u)+a₁₆C_df(i)(u)C_df(i)(v)z

+a₁₇(C_df(i)(u))³+a₁₈(C_df(i)(v))³+a₁₉(z)³

wherein Q is_i ^dThe distance from the convergent imaging point to the plane of the microlens array; a is_0-19Is a polynomial coefficient; c_df(i)(u) is the abscissa of the center of the circle of confusion, C_df(i)(v) Is a longitudinal coordinate of the center of a circle of a dispersion circle;

module M5.3: establishing the ith white point O of the three-dimensional space calibration plate_i(x, y, z) and the diameter D of the circle of confusion_df(i)Mapping function relationship of

The model is as follows:

wherein Q is_i(u, v) is the projection coordinate of a convergent imaging point corresponding to the ith white point of the three-dimensional space calibration plate on the plane of the micro-lens array; c_df(i)(u, v) are coordinates of the center of the circle of dispersion; a. the^dThe distance from the center of the aperture to the plane of the microlens array; d_df(i)Is the diameter of a dispersion circle; a (u, v) is the projection coordinate of the aperture center on the microlens array plane; the coefficients α, β, γ pass through all white points O on the calibration plate as point sources_i(x, y, z) corresponds to C_df(i)(u，v)、Q_i ^dAnd diameter D of circle of dispersion_df(i)Fitting to obtain; a. the_mDenotes the main lens aperture diameter, A_m＝f_m(1-M)/f_#，f_mDenotes the main lens focal length, M denotes the magnification factor, f_#Represents the aperture value of the main lens, and theta represents the shift angle;

is a point on the circle of dispersion,

is another point on the dispersion circle and

and is symmetrical about the circle center of dispersion.

Specifically, in module M6:

according to the imaging rule and the mapping function of the light field camera, calculating to obtain a white point O at any three-dimensional space position_j(x, y, z) corresponding circle center coordinates C of circle in diffusion_df(j)(X, Y) and circle of confusion diameter D_df(j)And a microlens center position C_l(j)And calculating the influence of the white point at any three-dimensional space positionFinding white point O at any position in space after the micro-lens exists_j(x, y, z) all pixels affected;

calculating the boundary of the affected area under each microlens according to the following two formulas, and determining all pixels affected under each microlens according to the boundary;

the upper bound formula:

lower boundary formula:

the upper boundary of the region under the microlens which is affected, C_l(j)At the center of the jth microlens, fl is one focal length of the microlens, C_l(j)(u) is the central abscissa of the ith microlens, Qi (u) is the abscissa of the convergent imaging point, C_l(j)(v) Is the central ordinate, p, of the ith microlens_lIs the pixel size of the microlens, p_pFor sensor pixel size, Q_i(v) Is a vertical coordinate of a convergent imaging point; q_i ^dThe distance from the convergent imaging point to the plane of the microlens array;

the lower boundary of the affected zone boundary under the microlens.

Example 3:

example 3 is a preferred example of example 1, and the present invention will be described in more detail.

According to the light field camera calibration method used under the condition of shift-axis imaging, provided by the invention, the black matrix white point calibration plate is moved to different depth-of-field positions, and the following steps are executed at the different depth-of-field positions:

step 1: shooting a black-background white-point calibration plate by a single or a plurality of light field cameras to obtain light field images of white points at a plurality of known three-dimensional positions;

step 2: adjusting the aperture size of each light field camera to be minimum, and shooting a white background to obtain an aperture center projection image;

and step 3: removing lenses of the light field cameras, irradiating the light field cameras with parallel light, and shooting a white background to obtain a central image of the micro lens;

and 4, step 4: calculating to obtain the diameters and the center coordinates of diffusion circles of white points at different positions imaged in the light field camera according to the imaging rule of the light field camera by using the light field image of the white point at the known three-dimensional position, the projected image of the center of the aperture and the central image of the micro lens;

and 5: fitting a mapping function of the diameter and circle center coordinates of the dispersion circle and the space position of the three-dimensional body;

specifically, the light field camera calibration method for use under the condition of tilt-shift imaging further includes:

step 6: and finding all pixels influenced by white points at any positions in space according to the imaging rule of the light field camera and the mapping function, namely realizing the calibration of the light field camera under the condition of shift imaging.

Specifically, each white spot on the calibration plate in step 1 illuminates one or more pixels, an white spot is imaged under each microlens covered by a circle of confusion, and the central position of the ith white spot is recorded as p_c(i)(ii) a Wherein the white dots on the calibration plate are arranged at equal intervals.

Specifically, the white background in step 2 is a white background with uniform light intensity, each exudate center in the projected image at the center of the aperture is an image projected by the aperture center under the microlens, and the ith exudate center is recorded as the projection position of the aperture center corresponding to the ith microlens, and is C_a(i)。

Specifically, the center of each exudate in the central image of the microlens in step 3 is the center of the microlens, and the position of the ith exudate center, that is, the center of the ith microlens is recorded as C_l(i)。

Specifically, step 4 specifically includes:

step 4.1: based on the photographed projected image of the center of the aperture and the image of the center of the microlens, the projection coordinates A (u, v) of the center of the aperture on the plane of the microlens array and the distance A of the center of the aperture from the plane of the microlens array are calculated according to the following formula^d：

(C_a(i)-C_l(i))A^d-f_l(C_l(i)-A(u，v))＝0 (1)

Wherein the condition that equation (1) holds is: (a) the light field camera is characterized in that the micro-lens array is positioned in front of the imaging sensor, and the distance between the micro-lens array and the imaging sensor is one focal length f of the micro-lens_lThe focusing light field camera of (1); (b) for the aperture center, n equations (1) are obtained by imaging n microlenses in the step 2, and when n is larger than or equal to 3, A (u, v) and A are obtained by solving an overdetermined equation system^d。

Step 4.2: based on the shot light field image of the black background white point calibration plate, the ith white point O of the three-dimensional space calibration plate is calculated according to the following formula_iProjection coordinate Q of (x, y, z) corresponding convergent imaging point on the plane of the micro-lens array_i(u, v) and a converging imaging point Q_iDistance Q from the plane of the microlens array_i ^d：

(p_c(i)-C_l(i))Q_i ^d-f_l(C_l(i)-Q_i(u，v))＝0 (2)

The condition that equation (2) is satisfied is as follows: (a) the light field camera is a focusing light field camera with a micro lens array positioned in front of the imaging sensor by one focal length, and (b) the central position C of the micro lens_l(i)The corresponding microlens is completely covered by the dispersion circle; for a point light source O_iIn the step 1, n microlenses satisfying the condition (b) have n equations (2), and when n is larger than or equal to 3, Q is obtained by solving an overdetermined equation set_i(u, v) and Q_i ^d。

Step 4.3: based on the shot light field image of the black background white point calibration plate, the ith white point O of the three-dimensional space calibration plate is calculated according to the following formula_i(x, y, z) corresponding circle center coordinates C of diffusion circle_df(i)(u，v)：

A^dQ_i(u，v)-C_df(i)(u，v)(A^d-Q_i ^d)＝0 (3)

Step 4.4: calculating the ith white point O of the three-dimensional space calibration plate according to the formulas (3) and (4)) based on the shot light field image of the black background white point calibration plate_i(x, y, z) corresponding to the circle of confusion D_df(i)：

The condition that equation (4) is satisfied is as follows: (a) the light field camera is a focusing light field camera with a micro lens array positioned in front of the imaging sensor by one focal length, and (b) the central position C of the micro lens_l(i)The corresponding microlens is completely covered by the dispersion circle; for a point light source O_iIn step 1, n microlenses satisfy the condition (b), i.e., n white spots p_c(i)When n is more than or equal to 2, solving the maximum distance max | p between the white spots_c(i)-p_c(j)L, i, j belongs to n to obtain D_df(i)。

Specifically, step 5 includes the steps of:

step 5.1: establishing a mapping model to fit the ith white point O of the three-dimensional space calibration plate_i(x, y, z) and the coordinates C of the center of the circle corresponding to the circle of confusion_df(i)(u, v) mapping function relationship

The mapping model is as follows:

wherein, λ represents a mapping relation coefficient,

a mapping matrix is represented that is,

by means of a calibration plateAll white points O as point light sources_i(x, y, z) and corresponding circle center coordinates C of circle of confusion_df(i)(u, v) is obtained; u and v respectively represent coordinates of a rectangular coordinate system; x, y and z respectively represent coordinates of a space rectangular coordinate system;

step 5.2: polynomial fitting three-dimensional space calibration plate ith white point O_i(x, y, z) and a converging imaging point Q_iDistance Q from the plane of the microlens array_i ^dMapping functional relationships

The model is as follows:

Q_i ^d＝a₀+a₁C_df(i)(u)+a₂C_df(i)(v)+a₃z+a₄C_df(i)(u)C_df(i)(v)+a₅C_df(i)(u)z+a₆(z)²

+a₇C_df(i)(v)z+a₈(C_df(i)(v))²+a₉(C_df(i)(u))²+a₁₀(C_df(i)(u))²C_df(i)(v)

+a₁₁(C_df(i)(v))²C_df(i)(u)+a₁₂(C_df(i)(u))²z+a₁₃(C_df(i)(v))²z

+a₁₄(z)²C_df(i)(v)+a₁₅(z)²C_df(i)(u)+a₁₆C_df(i)(u)C_df(i)(v)z

+a₁₇(C_df(i)(u))³+a₁₈(C_df(i)(v))³+a₁₉(z)³

wherein, a_0-19Is a polynomial coefficient.

Step 5.3: establishing the ith white point O of the three-dimensional space calibration plate_i(x, y, z) and the diameter D of the circle of confusion_df(i)Mapping function relationship of

The model is as follows:

wherein the coefficients α, β, γ pass through all white points O on the calibration plate as point sources_iC for (x, y, z)_df(i)(u，v)、Q_i ^dAnd diameter D of circle of dispersion_df(i)Fitting to obtain; a. the_mDenotes the main lens aperture diameter, A_m＝f_m(1-M)/f_#，f_mDenotes the main lens focal length, M denotes the magnification factor, f_#Denotes the main lens aperture value and θ denotes the shift angle.

Specifically, the step 6 specifically includes:

calculating to obtain a white point O at any three-dimensional space position according to the imaging rule of the light field camera and the mapping function_j(x, y, z) corresponding circle center coordinates C of diffusion circle_df(j)(X, Y) and circle of confusion diameter D_df(j)And a microlens center position C_l(j)After all the micro-lenses influenced by the white point of any three-dimensional space position are obtained through calculation, the white point O of any space position is found_j(x, y, z) all pixels affected.

Specifically, the boundary of the affected area under each microlens is calculated according to the following two equations, all the affected pixels under each microlens are determined according to the boundary,

upper bound equation:

lower boundary equation:

example 4:

example 4 is a preferred example of example 1, and the present invention will be described in more detail.

According to the light field camera calibration method used under the condition of shift-axis imaging, the light field camera shoots the three-dimensional space calibration plate to eliminate optical distortion caused by the fact that the shape and the refractive index of an optical observation window are different from those of air, optical distortion caused by installation errors of a main lens and optical distortion caused by the fact that the refractive index of a measuring medium is different from that of air, the calculation precision of the corresponding relation between the space coordinate of an object point in space and pixels influenced by the space coordinate is improved, and the three-dimensional measurement quality is further improved.

According to the light field camera calibration method used under the condition of shift-axis imaging, provided by the invention, the black matrix white point calibration plate is moved to different depth-of-field positions, and the following steps are executed at the different depth-of-field positions:

step 1: shooting a black-background white-point calibration plate by a single or a plurality of light field cameras to obtain light field images of white points at a plurality of known three-dimensional positions; specifically, after the light field camera is used to complete the light field image acquisition of the three-dimensional flow field tracer particles, the focusing ring of the main lens is kept unchanged, as shown in fig. 3, the calibration plate is moved by using the high-precision manual/electric displacement table, and the light field camera shoots an image of the calibration plate at the position every time the calibration plate is moved by a small step, as shown in fig. 4. Each white spot on the calibration plate can illuminate one or more pixels, an exudate is imaged under each microlens covered by a diffusion circle, and the central position of the ith exudate is recorded as p_c(i)(ii) a Wherein the white dots on the calibration plate are arranged at equal intervals.

Step 2: each light is emittedAdjusting the aperture size of the field camera to the minimum, shooting a white background to obtain a projected image of the center of the ring, specifically, as shown in fig. 5, the white background in step 2 is a white background with uniform light intensity, each white spot center in the projected image of the center of the ring is an image generated by projection of the center of the ring under a microlens, and the ith white spot center is recorded as the projected position of the center of the ring corresponding to the ith microlens and is C_a(i)。

And step 3: using a collimator to irradiate a light field camera with the main lens removed, and shooting to obtain a central image of the microlens, as shown in fig. 6, each white spot center in the central image of the microlens is the center of the microlens, and the ith white spot center is recorded as the center position of the microlens_l(i)。

And 4, step 4: calculating to obtain the diameters and the center coordinates of diffusion circles of white points at different positions imaged in the light field camera according to the imaging rule of the light field camera by using the light field image of the white point at the known three-dimensional position, the projected image of the center of the aperture and the central image of the micro lens; specifically, step 4 includes the steps of:

step 4.1: based on the photographed projected image of the center of the aperture and the image of the center of the microlens, the projection coordinates A (u, v) of the center of the aperture on the plane of the microlens array and the distance A of the center of the aperture from the plane of the microlens array are calculated according to the following formula^d：

(C_a(i)-C_l(i))A^d-f_l(C_l(i)-A(u，v))＝0 (1)

Wherein the condition that equation (1) holds is: (a) the light field camera is characterized in that the micro-lens array is positioned in front of the imaging sensor, and the distance between the micro-lens array and the imaging sensor is one focal length f of the micro-lens_lThe focusing light field camera of (1); (b) for the aperture center, n equations (1) are obtained by imaging n microlenses in the step 2, and when n is larger than or equal to 3, A (u, v) and A are obtained by solving an overdetermined equation system^d。

Step 4.2: based on the shot light field image of the black background white point calibration plate, the ith white point O of the three-dimensional space calibration plate is calculated according to the following formula_iProjection coordinate Q of (x, y, z) corresponding convergent imaging point on the plane of the micro-lens array_i(u, v) and a converging imaging point Q_iDistance Q from the plane of the microlens array_i ^d：

(p_c(i)-C_l(i))Q_i ^d-f_l(C_l(i)-Q_i(u，v))＝0 (2)

The condition that equation (2) is satisfied is as follows: (a) the light field camera is a focusing light field camera with a micro lens array positioned in front of the imaging sensor by one focal length, and (b) the central position C of the micro lens_l(i)The corresponding microlens is completely covered by the dispersion circle; for a point light source O_iIn the step 1, n microlenses satisfying the condition (b) have n equations (2), and when n is larger than or equal to 3, Q is obtained by solving an overdetermined equation set_i(u, v) and Q_i ^d。

Step 4.3: based on the shot light field image of the black background white point calibration plate, the ith white point O of the three-dimensional space calibration plate is calculated according to the following formula_i(x, y, z) corresponding circle center coordinates C of diffusion circle_df(i)(u，v)：

A^dQ_i(u，v)-C_df(i)(u，v)(A^d-Q_i ^d)＝0 (3)

Step 4.4: based on the shot light field image of the black background white point calibration plate, the ith white point O of the three-dimensional space calibration plate is calculated according to the formulas (3) and (4)_i(x, y, z) corresponding to the circle of confusion D_df(i)：

The condition that equation (4) is satisfied is as follows: (a) the light field camera is a focusing light field camera with a micro lens array positioned in front of the imaging sensor by one focal length, and (b) the central position C of the micro lens_l(i)The corresponding microlens is completely covered by the dispersion circle; for a point light source O_iIn step 1, n microlenses satisfy the condition (b), i.e., n white spots p_c(i)When n is more than or equal to 2, solving the maximum distance max | p between the white spots_c(i)-p_c(j)L, i, j belongs to n to obtain D_df(i)。

Specifically, step 5 includes the steps of:

step 5.1: establishing a mapping model to fit the ith white point O of the three-dimensional space calibration plate_i(x, y, z) and the coordinates C of the center of the circle corresponding to the circle of confusion_df(i)(u, v) mapping function relationship

The mapping model is as follows:

wherein, λ represents a mapping relation coefficient,

a mapping matrix is represented that is,

by calibrating all white points O on the board as point light sources_i(x, y, z) and corresponding circle center coordinates C of circle of confusion_df(i)(u, v) is obtained; u and v respectively represent coordinates of a rectangular coordinate system; x, y and z respectively represent coordinates of a space rectangular coordinate system;

step 5.2: polynomial fitting three-dimensional space calibration plate ith white point O_i(x, y, z) and a converging imaging point Q_iDistance Q from the plane of the microlens array_i ^dMapping functional relationships

The model is as follows:

Q_i ^d＝a₀+a₁C_df(i)(u)+a₂C_df(i)(v)+a₃z+a₄C_df(i)(u)C_df(i)(v)+a₅C_df(i)(u)z+a₆(z)²

+a₇C_df(i)(v)z+a₈(C_df(i)(v))²+a₉(C_df(i)(u))²+a₁₀(C_df(i)(u))²C_df(i)(v)

+a₁₁(C_df(i)(v))²C_df(i)(u)+a₁₂(C_df(i)(u))²z+a₁₃(C_df(i)(v))²z

+a₁₄(z)²C_df(i)(v)+a₁₅(z)²C_df(i)(u)+a₁₆C_df(i)(u)C_df(i)(v)z

+a₁₇(C_df(i)(u))³+a₁₈(C_df(i)(v))³+a₁₉(z)³

wherein, a_0-19Is a polynomial coefficient.

Step 5.3: establishing the ith white point O of the three-dimensional space calibration plate_i(x, y, z) and the diameter D of the circle of confusion_df(i)Mapping function relationship of

The model is as follows:

wherein the coefficients α, β, γ pass through all white points O on the calibration plate as point sources_iC for (x, y, z)_df(i)(u，v)、Q_i ^dAnd diameter D of circle of dispersion_df(i)Fitting to obtain; a. the_mDenotes the main lens aperture diameter, A_m＝f_m(1-M)/f_#，f_mDenotes the main lens focal length, M denotes the magnification factor, f_#Denotes the main lens aperture value and θ denotes the shift angle.

Specifically, the step 6 specifically includes:

calculating to obtain a white point O at any three-dimensional space position according to the imaging rule of the light field camera and the mapping function_j(x, y, z) corresponding circle center coordinates C of diffusion circle_df(j)(X, Y) and circle of confusion diameter D_df(j)And a microlens center position C_l(j)After all the micro-lenses influenced by the white point of any three-dimensional space position are obtained through calculation, the white point O of any space position is found_j(x, y, z) all pixels affected.

Specifically, the boundary of the affected area under each microlens is calculated according to the following two equations, all the affected pixels under each microlens are determined according to the boundary,

upper bound equation:

lower boundary equation:

the two equations are one-dimensional case boundaries, and the two-dimensional case boundaries are expanded to two-dimensional cases according to the equations.

Those skilled in the art will appreciate that, in addition to implementing the systems, apparatus, and various modules thereof provided by the present invention in purely computer readable program code, the same procedures can be implemented entirely by logically programming method steps such that the systems, apparatus, and various modules thereof are provided in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers, embedded microcontrollers and the like. Therefore, the system, the device and the modules thereof provided by the present invention can be considered as a hardware component, and the modules included in the system, the device and the modules thereof for implementing various programs can also be considered as structures in the hardware component; modules for performing various functions may also be considered to be both software programs for performing the methods and structures within hardware components.

The foregoing description of specific embodiments of the present invention has been presented. It is to be understood that the present invention is not limited to the specific embodiments described above, and that various changes or modifications may be made by one skilled in the art within the scope of the appended claims without departing from the spirit of the invention. The embodiments and features of the embodiments of the present application may be combined with each other arbitrarily without conflict.

Claims (10)

1. A light field camera calibration method for use under tilt-shift imaging conditions, comprising:

step S1: shooting black background white point calibration plates at different depth of field positions to obtain a light field image of a white point at a known three-dimensional position;

step S2: adjusting the aperture size of each light field camera to the minimum shooting white background to obtain an aperture center projection image;

step S3: irradiating each light field camera by using parallel light, and shooting a white background to obtain a central image of the micro lens;

step S4: obtaining the diameters and the center coordinates of the diffusion circles of the white points at different positions imaged in the light field camera according to calculation through the light field image of the white point at the three-dimensional position, the projected image of the center of the aperture and the central image of the micro lens;

step S5: fitting a mapping function of the diameter and circle center coordinates of the dispersion circle and the space position of the three-dimensional body;

step S6: and according to the imaging rule of the light field camera and the mapping function of the space position of the three-dimensional body, the light field camera calibration under the condition of shift imaging is realized.

2. The light field camera calibration method for shift imaging conditions according to claim 1, characterized in that:

each white spot on the calibration plate illuminates one or more pixels, each microlens covered by a circle of confusion forms a white spot under it,the central position of the ith white spot is recorded as p_c(i)I is an ordinal number, wherein white dots on the calibration plate are arranged at equal intervals;

the white background is a white background with uniform light intensity, each white spot center in the projected image of the center of the aperture is an image generated by projection of the center of the aperture under the microlens, and the projection position of the center of the aperture corresponding to the ith white spot center under the ith microlens is recorded as C_a(i)；

Each exudate center in the central image of the microlens is the center of the microlens, and the ith exudate center, namely the central position of the ith microlens is marked as C_l(i)。

3. The light field camera calibration method for shift imaging conditions according to claim 1, characterized in that in step S4:

the method comprises the following specific steps:

step S4.1: based on the shot projected image of the center of the aperture and the shot image of the center of the micro-lens, calculating the projection coordinate A (u, v) of the center of the aperture on the plane of the micro-lens array and the distance A of the center of the aperture from the plane of the micro-lens array according to a formula I^d：

(C_a(i)-C_l(i))A^d-f_l(C_l(i)-A(u，v))＝0 (1)

Wherein, C_a(i)The center of the ith white spot is the projection position of the center of a diaphragm corresponding to the ith micro lens; c_l(i)Is the ith white spot center, namely the ith microlens center position; f. of_lIs one focal length of the microlens;

the condition that the first formula is satisfied is as follows:

a1, the light field camera is a microlens array in front of the imaging sensor and the distance between the microlens array and the imaging sensor is one focal length f of the microlens_lThe focusing light field camera of (1);

b1, for the center of the aperture, n formula I can be obtained when n microlenses are used for imaging, n is the number of the microlenses, and when n is larger than or equal to 3, A (u, v) and A are obtained by solving an overdetermined equation system^d；

Step S4.2: light based on photographed black background white dot calibration plateField image, calculating the ith white point O of the three-dimensional space calibration plate according to formula II_iProjection coordinate Q of (x, y, z) corresponding convergent imaging point on the plane of the micro-lens array_i(u, v) and a converging imaging point Q_iDistance Q from the plane of the microlens array_i ^d：

(p_c(i)-C_l(i))Q_i ^d-f_l(C_l(i)-Q_i(u，v))＝0 (2)

Wherein the central position of the ith white spot is p_c(i)；

The condition that the second formula is satisfied is as follows:

a2, the light field camera is a focusing light field camera with a micro lens array positioned in front of the imaging sensor by one time of focal length;

(b2) central position C of microlens_l(i)The corresponding microlens is completely covered by the dispersion circle;

for a point light source O_iIf n microlenses satisfy the condition b2, n formulas II are obtained, and when n is larger than or equal to 3, Q is obtained by solving an overdetermined equation set_i(u, v) and Q_i ^d；

Step S4.3: based on the shot light field image of the black bottom white point calibration plate, the ith white point O of the three-dimensional space calibration plate is calculated according to the formula III_i(x, y, z) corresponding circle center coordinates C of diffusion circle_df(i)(u，v)：

A^dQ_i(u，v)-C_df(i)(u，v)(A^d-Q_i ^d)＝0 (3)

Step S4.4: calculating the ith white point O of the three-dimensional space calibration plate according to the formula three and four based on the shot light field image of the black background white point calibration plate_i(x, y, z) corresponding to the circle of confusion D_df(i)：

p_c(j)The central position of the jth white spot;

the condition that the formula four holds is as follows:

a3, the light field camera is a focusing light field camera with a micro lens array positioned in front of the imaging sensor by one time of focal length;

b3 center position of microlens C_l(i)The corresponding microlens is completely covered by the dispersion circle;

for a point light source O_iThere are n microlenses satisfying the condition b3, i.e., n white spots p_c(i)When n is more than or equal to 2, solving the maximum distance max | p between the white spots_c(i)-p_c(j)I, j is left to n to obtain D_df(i)。

4. The light field camera calibration method for shift imaging conditions according to claim 1, characterized in that in step S5:

the method comprises the following specific steps:

step S5.1: establishing a mapping model to fit the ith white point O of the three-dimensional space calibration plate_i(x, y, z) and the coordinates C of the center of the circle corresponding to the circle of confusion_df(i)(u, v) mapping function relationship

The mapping model is as follows:

wherein, λ represents a mapping relation coefficient,

a mapping matrix is represented that is,

by calibrating all white points O on the board as point light sources_i(x, y, z) and corresponding circle center coordinates C of circle of confusion_df(i)(u, v) is obtained; u and v respectively represent coordinates of a rectangular coordinate system; x, y and z respectively represent coordinates of a space rectangular coordinate system;

step S5.2: polynomial fitting three-dimensional space calibration platei white points O_i(x, y, z) and a converging imaging point Q_iDistance Q from the plane of the microlens array_i ^dMapping functional relationships

The model is as follows:

Q_i ^d＝a₀+a₁C_df(i)(u)+a₂C_df(i)(v)+a₃z+a₄C_df(i)(u)C_df(i)(v)+a₅C_df(i)(u)z+a₆(z)²+a₇C_df(i)(v)z+a₈(C_df(i)(v))²+a₉(C_df(i)(u))²+a₁₀(C_df(i)(u))²C_df(i)(v)+a₁₁(C_df(i)(v))²C_df(i)(u)+a₁₂(C_df(i)(u))²z+a₁₃(C_df(i)(v))²z+a₁₄(z)²C_df(i)(v)+a₁₅(z)²C_df(i)(u)+a₁₆C_df(i)(u)C_df(i)(v)z+a₁₇(C_df(i)(u))³+a₁₈(C_df(i)(v))³+a₁₉(z)³

wherein Q is_i ^dThe distance from the convergent imaging point to the plane of the microlens array; a is_0-19Is a polynomial coefficient; c_df(i)(u) is the abscissa of the center of the circle of confusion, C_df(i)(v) Is a longitudinal coordinate of the center of a circle of a dispersion circle;

step S5.3: establishing the ith white point O of the three-dimensional space calibration plate_i(x, y, z) and the diameter D of the circle of confusion_df(i)Mapping function relationship of

The model is as follows:

wherein Q is_i(u, v) is the projection coordinate of a convergent imaging point corresponding to the ith white point of the three-dimensional space calibration plate on the plane of the micro-lens array; c_df(i)(u, v) are coordinates of the center of the circle of dispersion; a. the^dThe distance from the center of the aperture to the plane of the microlens array; d_df(i)Is the diameter of a dispersion circle; a (u, v) is the projection coordinate of the aperture center on the microlens array plane; the coefficients α, β, γ pass through all white points O on the calibration plate as point sources_iC for (x, y, z)_df(i)(u，v)、Q_i ^dAnd diameter D of circle of dispersion_df(i)Fitting to obtain; a. the_mDenotes the main lens aperture diameter, A_m＝f_m(1-M)/f_#，f_mDenotes the main lens focal length, M denotes the magnification factor, f_#Represents the aperture value of the main lens, and theta represents the shift angle;

is a point on the circle of dispersion,

is another point on the dispersion circle and

and is symmetrical about the circle center of dispersion.

5. The light field camera calibration method for shift imaging conditions according to claim 1, characterized in that in step S6:

according to the imaging rule and the mapping function of the light field camera, calculating to obtain a white point O at any three-dimensional space position_j(x, y, z) corresponding circle center coordinates C of diffusion circle_df(j)(X, Y) and circle of confusion diameter D_df(j)And a microlens center position C_l(j)After all the micro-lenses influenced by the white point of any three-dimensional space position are obtained through calculation, the white point O of any space position is found_j(x, y, z) all pixels affected;

calculating the boundary of the affected area under each microlens according to the following two formulas, and determining all pixels affected under each microlens according to the boundary;

the upper bound formula:

lower boundary formula:

the upper boundary of the region under the microlens which is affected, C_l(j)Is the j-th microlens center position, f_lIs one focal length of the microlens, C_l(j)(u) is the central abscissa of the ith microlens, Q_i(u) is the convergent imaging point abscissa, C_l(j)(v) Is the central ordinate, p, of the ith microlens_lIs the pixel size of the microlens, p_pFor sensor pixel size, Q_i(v) Is a vertical coordinate of a convergent imaging point; q_i ^dThe distance from the convergent imaging point to the plane of the microlens array;

the lower boundary of the affected zone boundary under the microlens.

6. A light field camera calibration system for use under tilt-shift imaging conditions, comprising:

module M1: shooting black background white point calibration plates at different depth of field positions to obtain a light field image of a white point at a known three-dimensional position;

module M2: adjusting the aperture size of each light field camera to the minimum shooting white background to obtain an aperture center projection image;

module M3: irradiating each light field camera by using parallel light, and shooting a white background to obtain a central image of the micro lens;

module M4: obtaining the diameters and the center coordinates of the diffusion circles of the white points at different positions imaged in the light field camera according to calculation through the light field image of the white point at the three-dimensional position, the projected image of the center of the aperture and the central image of the micro lens;

module M5: fitting a mapping function of the diameter and circle center coordinates of the dispersion circle and the space position of the three-dimensional body;

module M6: and according to the imaging rule of the light field camera and the mapping function of the space position of the three-dimensional body, the light field camera calibration under the condition of shift imaging is realized.

7. The light field camera calibration system for use under tilt-shift imaging conditions of claim 6, wherein:

each white spot on the calibration plate can illuminate one or more pixels, an exudate is imaged under each microlens covered by a diffusion circle, and the central position of the ith exudate is recorded as p_c(i)I is an ordinal number, wherein white dots on the calibration plate are arranged at equal intervals;

the white background is a white background with uniform light intensity, each white spot center in the projected image of the center of the aperture is an image generated by projection of the center of the aperture under the microlens, and the projection position of the center of the aperture corresponding to the ith white spot center under the ith microlens is recorded as C_a(i)；

In the micro-lensThe center of each exudate in the heart image is the center of the microlens, and the position of the ith exudate center, namely the center of the ith microlens, is recorded as C_l(i)。

8. The light field camera calibration system for tilt-shift imaging conditions according to claim 6 wherein in module M4:

the method comprises the following specific steps:

module M4.1: based on the shot projected image of the center of the aperture and the shot image of the center of the micro-lens, calculating the projection coordinate A (u, v) of the center of the aperture on the plane of the micro-lens array and the distance A of the center of the aperture from the plane of the micro-lens array according to a formula I^d：

(C_a(i)-C_l(i))A^d-f_l(C_l(i)-A(u，v))＝0 (1)

Wherein, C_a(i)The center of the ith white spot is the projection position of the center of a diaphragm corresponding to the ith micro lens; c_l(i)Is the ith white spot center, namely the ith microlens center position; f. of_lIs one focal length of the microlens;

the condition that the first formula is satisfied is as follows:

a1, the light field camera is a microlens array in front of the imaging sensor and the distance between the microlens array and the imaging sensor is one focal length f of the microlens_lThe focusing light field camera of (1);

b1, for the center of the aperture, n formula I can be obtained when n microlenses are used for imaging, n is the number of the microlenses, and when n is larger than or equal to 3, A (u, v) and A are obtained by solving an overdetermined equation system^d；

Module M4.2: based on the shot light field image of the black bottom white point calibration plate, the ith white point O of the three-dimensional space calibration plate is calculated according to a formula II_iProjection coordinate Q of (x, y, z) corresponding convergent imaging point on the plane of the micro-lens array_i(u, v) and a converging imaging point Q_iDistance Q from the plane of the microlens array_i ^d：

(p_c(i)-C_l(i))Q_i ^d-f_l(C_l(i)-Q_i(u，v))＝0 (2)

Wherein, the ithThe central position of each white spot is p_c(i)；

The condition that the second formula is satisfied is as follows:

a2, the light field camera is a focusing light field camera with a micro lens array positioned in front of the imaging sensor by one time of focal length;

(b2) central position C of microlens_l(i)The corresponding microlens is completely covered by the dispersion circle;

for a point light source O_iIf n microlenses satisfy the condition b2, n formulas II are obtained, and when n is larger than or equal to 3, Q is obtained by solving an overdetermined equation set_i(u, v) and Q_i ^d；

Module M4.3: based on the shot light field image of the black bottom white point calibration plate, the ith white point O of the three-dimensional space calibration plate is calculated according to the formula III_i(x, y, z) corresponding circle center coordinates C of diffusion circle_df(i)(u，v)：

A^dQ_i(u，v)-C_df(i)(u，v)(A^d-Q_i ^d)＝0 (3)

Module M4.4: calculating the ith white point O of the three-dimensional space calibration plate according to the formula three and four based on the shot light field image of the black background white point calibration plate_i(x, y, z) corresponding to the circle of confusion D_df(i)：

p_c(j)The central position of the jth white spot;

the condition that the formula four holds is as follows:

a3, the light field camera is a focusing light field camera with a micro lens array positioned in front of the imaging sensor by one time of focal length;

b3 center position of microlens C_l(i)The corresponding microlens is completely covered by the dispersion circle;

for a point light source O_iThere are n microlenses satisfying the condition b3, i.e., n white spots p_c(i)When n is more than or equal to 2, solving the maximum distance max | p between the white spots_c(i)-p_c(j)|，i，j∈n，Find D_df(i)。

9. The light field camera calibration system for tilt-shift imaging conditions according to claim 6 wherein in module M5:

the method comprises the following specific steps:

module M5.1: establishing a mapping model to fit the ith white point O of the three-dimensional space calibration plate_i(x, y, z) and the coordinates C of the center of the circle corresponding to the circle of confusion_df(i)(u, v) mapping function relationship

The mapping model is as follows:

wherein, λ represents a mapping relation coefficient,

a mapping matrix is represented that is,

by calibrating all white points O on the board as point light sources_i(x, y, z) and corresponding circle center coordinates C of circle of confusion_df(i)(u, v) is obtained; u and v respectively represent coordinates of a rectangular coordinate system; x, y and z respectively represent coordinates of a space rectangular coordinate system;

module M5.2: polynomial fitting three-dimensional space calibration plate ith white point O_i(x, y, z) and a converging imaging point Q_iDistance Q from the plane of the microlens array_i ^dMapping functional relationships

The model is as follows:

Q_i ^d＝a₀+a₁C_df(i)(u)+a₂C_df(i)(v)+a₃z+a₄C_df(i)(u)C_df(i)(v)+a₅C_df(i)(u)z+a₆(z)²+a₇C_df(i)(v)z+a₈(C_df(i)(v))²+a₉(C_df(i)(u))²+a₁₀(C_df(i)(u))²C_df(i)(v)+a₁₁(C_df(i)(v))²C_df(i)(u)+a₁₂(C_df(i)(u))²z+a₁₃(C_df(i)(v))²z+a₁₄(z)²C_df(i)(v)+a₁₅(z)²C_df(i)(u)+a₁₆C_df(i)(u)C_df(i)(v)z+a₁₇(C_df(i)(u))³+a₁₈(C_df(i)(v))³+a₁₉(z)³

wherein Q is_i ^dThe distance from the convergent imaging point to the plane of the microlens array; a is_0-19Is a polynomial coefficient; c_df(i)(u) is the abscissa of the center of the circle of confusion, C_df(i)(v) Is a longitudinal coordinate of the center of a circle of a dispersion circle;

module M5.3: establishing the ith white point O of the three-dimensional space calibration plate_i(x, y, z) and the diameter D of the circle of confusion_df(i)Mapping function relationship of

The model is as follows:

wherein Q is_i(u, v) is the projection coordinate of a convergent imaging point corresponding to the ith white point of the three-dimensional space calibration plate on the plane of the micro-lens array; c_df(i)(u, v) are coordinates of the center of the circle of dispersion; a. the^dThe distance from the center of the aperture to the plane of the microlens array; d_df(i)Is the diameter of a dispersion circle; a (u, v) is the projection coordinate of the aperture center on the microlens array plane; the coefficients α, β, γ pass through all white points O on the calibration plate as point sources_iC for (x, y, z)_df(i)(u，v)、Q_i ^dAnd diameter D of circle of dispersion_df(i)Fitting to obtain; a. the_mDenotes the main lens aperture diameter, A_m＝f_m(1-M)/f_#，f_mDenotes the main lens focal length, M denotes the magnification factor, f_#Represents the aperture value of the main lens, and theta represents the shift angle;

is a point on the circle of dispersion,

is another point on the dispersion circle and

and is symmetrical about the circle center of dispersion.

10. The light field camera calibration system for tilt-shift imaging conditions according to claim 6 wherein in module M6:

according to the imaging rule and the mapping function of the light field camera, calculating to obtain a white point O at any three-dimensional space position_j(x, y, z) corresponding circle center coordinates C of diffusion circle_df(j)(X, Y) and circle of confusion diameter D_df(j)And a microlens center position C_l(j)After all the micro-lenses influenced by white points at any three-dimensional space position are obtained through calculation, any position in space is foundWhite point O_j(x, y, z) all pixels affected;

calculating the boundary of the affected area under each microlens according to the following two formulas, and determining all pixels affected under each microlens according to the boundary;

the upper bound formula:

lower boundary formula:

the upper boundary of the region under the microlens which is affected, C_l(j)Is the j-th microlens center position, f_lIs one focal length of the microlens, C_l(j)(u) is the central abscissa of the ith microlens, Q_i(u) is the convergent imaging point abscissa, C_l(j)(v) Is the central ordinate, p, of the ith microlens_lIs the pixel size of the microlens, p_pFor sensor pixel size, Q_i(v) Is a vertical coordinate of a convergent imaging point; q_i ^dThe distance from the convergent imaging point to the plane of the microlens array;

the lower boundary of the affected zone boundary under the microlens.

Images