CN105654484B - Light-field camera external parameters calibration device and method - Google Patents
Light-field camera external parameters calibration device and method Download PDFInfo
- Publication number
- CN105654484B CN105654484B CN201511023954.5A CN201511023954A CN105654484B CN 105654484 B CN105654484 B CN 105654484B CN 201511023954 A CN201511023954 A CN 201511023954A CN 105654484 B CN105654484 B CN 105654484B
- Authority
- CN
- China
- Prior art keywords
- light
- color
- field camera
- angle
- lenticule
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Landscapes
- Studio Devices (AREA)
Abstract
The present invention provides a kind of light-field camera external parameters calibration device and method, color-coding plate is placed in front of light-field camera main lens, and color-coding plate is fixed on angle turntable；Light-field camera is then placed on multiple positions and acquires scene by the color-coding plate that light-field camera shooting different angle is placed in this, as prior information, completes external parameters calibration in conjunction with prior information.The present invention can demarcate the outer parameter of multiple biplanes of any attitude, not depend on fixed mode camera motion platform, external parameters calibration is more accurate, is not influenced by the error as caused by camera motion platform inertia.
Description
Technical field
The present invention relates to computer vision, calculate camera shooting and optical engineering field, and in particular to one kind is compiled based on color
The light-field camera external parameters calibration device and method of code plate, the rotation and translation that can be demarcated between multiple biplane light fields are closed
System.
Background technique
Optical field imaging theory is the research hotspot for calculating camera shooting field, and foreign study mechanism devises a variety of each tools accordingly
The haplopia optical field imaging system of characteristic, wherein the light-field camera based on microlens array becomes the optical field acquisition equipment of mainstream.But
Be limited to image-forming principle and sensor preparation process, the completeness that haplopia optical field imaging system samples target light field signal compared with
Difference, lead to it there are field angles limited, the depth of field is insufficient, position and angular resolution are low, depth recovery precision is poor, to blocking not
The disadvantages of robust.Therefore, what light field computational theory can solve effectively to haplopia light field faces from single view continuation to multiple views asks
Topic.
The matter of utmost importance that multiple views light field research faces is the outer ginseng calibration of light-field camera.Birklbauer etc. introduces one
Light-field camera four-dimensional movement model completes more in the case where fixed rotation stepping and focussing plane depth according to the model
The calibration of a light field and frequency domain fusion structure, but the case where it is not suitable for light-field camera translational motion.Xue etc. is simplifying light field
The plane transformation relationship between multiple light-field cameras, the item known to light-field camera motion conditions have been derived on the basis of camera model
More view light fields that high quality is realized under part are spliced and are merged, but cannot achieve light field phase when light-field camera motion conditions are unknown
Plane transformation between machine.Therefore, when unknown light-field camera rotation angle peace moves distance, existing algorithm cannot obtain light field
The outer ginseng of camera.
Summary of the invention
For overcome the deficiencies in the prior art, the present invention provide a kind of external parameters calibration device based on color-coding plate and
Algorithm can calculate the external parameter of light-field camera in the case where not depending on fixed light-field camera motion platform.
The technical solution adopted by the present invention to solve the technical problems is: a kind of outer ginseng caliberating device, including a color
Encoding board and an angle turntable；Color-coding plate is placed in front of light-field camera main lens within 2m, and color is compiled
Code plate is fixed on angle turntable；The color-coding plate is made of color coding sheet and microlens array；Color coding
At least 2*3 demarcating circle is printed on paper, demarcating circle color takes the two of RGB channel respectively in horizontal and vertical both direction
It plants, the uniform sampling within the scope of 0-255 along horizontal and vertical direction of the print point in demarcating circle, adjacent demarcating circle is in same direction
On the Color Channel that is taken it is different；Microlens array is placed in color coding sheet close to light-field camera direction side, compiles color
Each demarcating circle is located at a lenticule dead astern on code paper, and demarcates circular diameter and be less than lenslet diameter.
The minimum spacing of adjacent print point in the demarcating circleWherein, f' indicates color coding sheet
With the distance of lenticule optical center on color-coding plate, d indicates lenslet diameter in light-field camera, and M indicates angular samples number, i.e., empty
Between in the number that is recorded on the image sensor of same object point, k=min (α), if a lenticule rear imaging radius is r,
The distance at lenticule unit center to the gap center of contiguous microlens is a, then scale factor meets α r >=a.
The distance f' of lenticule optical center is corresponding with round spot single on sensor on the color coding sheet and color-coding plate
Print point number npRelationship it is as follows:
Wherein, y0Indicate coordinate of the print point in Y-axis, Q1、Q3It is beaten for two in demarcating circle single on color coding sheet
Print point, Q1The light of sending passes through main lens lower edge, Q3The light of sending passes through main lens top edge, drIndicate color coding
Lenslet diameter on plate, TdIndicate plane and light-field camera main lens light where microlens array optical center of lens in color-coding plate
The distance of the heart, D indicate the diameter of light-field camera main lens clear aperature, α0Indicate lenticule center and light field on color-coding plate
The line at camera main lens center and the angle of camera optical axis, α1、α3Respectively indicate Q1、Q3Through its front, lenticule is issued
The angle of light and camera optical axis.
The single round spot corresponds to print point number on horizontal and vertical no more than 10.
The minimum centers-distance of demarcating circle on the color-coding plateWherein, g indicates light
Interplanar spacing where lenticule optical center, z indicate lenticule battle array in light-field camera in field camera internal sensor and microlens array
Plane and virtual plane spacing where lenticule optical center in column, F indicate interplanar spacing where virtual plane and main lens, and D is indicated
Light-field camera main lens clear aperture diameter, the top edge light and the subscript adjacent with top demarcating circle that top demarcating circle issues
The lower edge light that fixed circle issues is met at a bit, TdIndicate plane and light where microlens array optical center of lens in color-coding plate
The distance of field camera main lens optical center.
The present invention also proposes a kind of light-field camera method for calibrating external parameters, comprising the following steps:
Step 1, color-coding plate is placed in angle turntable, corresponding print point is recorded in each rotation angle and is issued
Light information, rotate the light that all print points of multiple angle recordings issue；Angle turntable stepping is encoded less than or equal to color
The angle that single print point can be resolved on paperWherein, FoV is lenticule field angle,d'
To demarcate circular diameter, f' indicates the distance of lenticule optical center on color coding sheet and color-coding plate, nrTo demarcate circular diameter model
Enclose interior printable points；
Step 2, coordinate system where plate is XYZ color-coded, and wherein plane where microlens array is XOY, origin O
In microlens array plane towards light-field camera photographed scene side, longitudinal is X-axis, is laterally Y-axis, and color-coding plate place is flat
The normal vector in face is Z axis；Shoot color-coding plate, the angle that record demarcating circle emits beam, comprising the following steps:
(1) placing color-coding plate makes its initial position place plane perpendicular to camera optical axis；
(2) face XOY of color-coding plate is made positively and negatively respectively to rotate fixed stepping, rotation around X-axis using stepper motor
It is recorded the light that all print points are issued through lenticule in demarcating circle after repeatedly；
(3) scaling board normal direction is obtained relative to the rotation angle ξ ∈ (- 90 °, 90 °) of its initial position according to color-coding plate
Angle η=ξ of amount and camera optical axis；
(4) round spot center on sensor is extracted, is decoded according to internal reference, lenticule centre coordinate where calculating it passes through light
Line is propagated and refraction matrix, calculates the angle β of projection and camera optical axis of the light-field camera main lens extraneous light on the face YOZ,
Wherein A is refraction matrix, TtFor propogator matrix, F is the main focal length of lens, g indicate light-field camera internal sensor and
Interplanar spacing where lenticule optical center in microlens array, z are indicated in light-field camera in microlens array where lenticule optical center
Plane and virtual plane spacing, P0For round spot centre coordinate, P on CCD1Lenticule centre coordinate where indicating round spot, P2Indicate solution
The coordinate of light and main lens plane point of intersection that code goes out；
(5) light is calculated in the projection in the face YOZ and angle γ=η+β of Z axis；
(6) color-coding plate is rotated by 90 ° about the z axis, repeats step (2)~(5), calculate projection and Z of the light in the face XOZ
The angle of axis；
Step 3, in the image of each step shooting, the color that round spot is imaged in each demarcating circle on a sensor is fitted to
One color value, the colouring information to emit beam as print points different in demarcating circle；All directions on color-coding plate are sent out
The prior information of the color of light and angle corresponding relationship as light out；
Step 4, shooting has the scene of color-coding plate, according to the color value that round spot color on sensor is fitted, searches
The matched light of its in prior information projects on the face XOZ and the face YOZ and the angle of Z axis, corresponds in front of demarcating circle in conjunction with round spot
The coordinate at lenticule center establishes the emergent ray equation under XYZ coordinate system；
Step 5, incident ray equation is established, the position at round spot center on sensor record image is calculated, is looked into conjunction with internal reference
Lenticule centre coordinate where looking for it obtains light double flat areal coordinate according to two o'clock coordinate, by propagating, reflecting matrix, calculates
The equation of light biplane uvst outside main lens, and convert it to the X where camera1Y1Z1Under coordinate system；
Step 6, according to emergent ray equation and the matching of incident ray equation to establishing equation；
It is X with coordinate system where light-field camera1Y1Z1, wherein plane where sensor is X1O1Y1, origin O1Positioned at sensing
The device plane upper left corner (is observed) towards light-field camera photographed scene, and longitudinal is X1Axis is laterally Y1Axis, optical axis direction Z1；Assuming that
Coordinate system X1Y1Z1X-axis angle with coordinate system XYZ is ω, and Y-axis angle is, Z axis angle is κ, XYZ coordinate system change of origin
To X1Y1Z1The displacement of coordinate origin is (Tx,Ty,Tz), then the spin matrix between two coordinate systems
Translation matrix
Any straight line is expressed as l (x, y, z, r in XYZ coordinate systemx,ry,rz), (x, y, z) indicates any one on l
Coordinate of the point at XYZ, [rx,ry,rz]TIndicate the direction vector of l；Straight line l is transformed into X1Y1Z1Coordinate representation under coordinate system
For l1(x',y',z',rx',ry',rz'), (x', y', z') indicates point (x, y, z) in X1Y1Z1Under coordinate, [rx',ry',rz']T
Indicate l in X1Y1Z1Under direction vector；There is two o'clock A on l0(xa,ya,za) and B0(xb,yb,zb), l1On have two o'clock A1(xa',
ya',za') and B1(xb',yb',zb'), then have
l1Linear equation be
l0And l1Direction vector be respectively
Then transformation relation [the r between rectilinear direction vectorx',ry',rz']T=R-1[rx,ry,rz]T；
Find out l0On a bit (x, y, z) be transformed into l1The formula of upper (x', y', z')
[x',y',z']T=λ R-1[rx,ry,rz]T+R-1([x,y,z]T-T)；
Then straight line and X in XYZ coordinate system1Y1Z1The transformational relation of upper straight line is
Step 7, solve system of equation, to seek rotation, 6 dimension parameter of translationω、κ、T0、T1、T2And biplane
Uvst origin and rectangular coordinate system X1O1Y12 dimension parameters of origin relative displacement.
The beneficial effects of the present invention are: being compiled for the shortcomings that calibration lacks flexibility is joined outside current biplane by color
The difference of color value is imaged in round spot in different directions on code plate, and it is corresponding on encoding board to establish round spot imaging color value on CCD
Round spot emits beam the corresponding relationship in direction, obtains emergent ray equation in coordinates.It is decoded by internal reference, obtains entering outside main lens
Penetrate light equation in coordinates.The transfer equation of outgoing and incident ray pair is established, light-field camera is solved and color-coding plate place is sat
Mark the rotation and translation parameter of system.
The present invention can demarcate the outer parameter of multiple biplanes of any attitude, and it is flat not depend on fixed mode camera motion
Platform.In addition, external parameters calibration is more accurate due to the light information for having demarcated different angle using color-coding plate, not by by
Error caused by camera motion platform inertia influences.
Detailed description of the invention
Fig. 1 is light-field camera schematic diagram of internal structure, wherein (a) indicates that imaging sensor and microlens array are located at master
After the imaging plane of camera lens, (b) indicate imaging sensor and microlens array be located at main lens and main lens imaging plane it
Between；
Fig. 2 is color-coding plate structural schematic diagram；
Fig. 3 is print point distance computation schematic diagram；
Fig. 4 is color-coding plate placement schematic；
Fig. 5 is calibration circle center away from calculating schematic diagram；
Fig. 6 is that single print point can be resolved angle calculation schematic diagram；
Fig. 7 is that light angle calculates schematic diagram；
Fig. 8 is coordinate system transition diagram.
Specific embodiment
Present invention will be further explained below with reference to the attached drawings and examples, and the present invention includes but are not limited to following implementations
Example.
The outer ginseng caliberating device that the present invention uses includes a color-coding plate and an angle turntable.Color is encoded
Plate is placed in front of light-field camera main lens within 2m, and color-coding plate is fixed on angle turntable.Light-field camera point
For both of which: after being located at the imaging plane of main lens one is imaging sensor and microlens array, another kind is image
Sensor and microlens array are between main lens and main lens imaging plane.The present invention is suitable for both the above mode
With.Color-coding plate is made of color coding sheet and microlens array.Printing is more than or equal to 2*3 calibration on color coding sheet
Circle, demarcating circle color take two kinds of RGB channel, the print point edge in demarcating circle respectively in horizontal and vertical both direction
Horizontal and vertical direction uniform sampling within the scope of 0-255, to guarantee that adjacent demarcating circle imaging is distinguishable, adjacent demarcating circle is same
The Color Channel taken on one direction is different.Microlens array is placed in front of color coding sheet (close to light-field camera direction),
So that each demarcating circle on color coding sheet is located at a lenticule dead astern, and demarcates circular diameter and be less than lenslet diameter.
Assuming that coordinate system where light-field camera is X1Y1Z1, wherein plane where sensor is X1O1Y1, origin O1Positioned at biography
The sensor plane upper left corner (is observed) towards light-field camera photographed scene, and longitudinal is X1Axis is laterally Y1Axis, optical axis direction Z1；Face
Coordinate system where color encoding board is XYZ, and wherein plane where microlens array is XOY, and it is left that origin O is located at microlens array plane
Upper angle (observing towards light-field camera photographed scene), it is laterally Y-axis that longitudinal, which is X-axis, the normal vector of plane where color-coding plate
For Z axis.
The major parameter of apparatus of the present invention includes: the adjacent print point spacing on color-coding plate in single demarcating circle, face
In the spacing of plane where color encoding board and light-field camera main lens and color-coding plate the center of demarcating circle away from.Above three
A determination method for parameter is as follows:
Firstly, calculating the adjacent print point minimum spacing in single demarcating circle.Make the different printings in same demarcating circle
Point is imaged onto two different round spots on CCD, need to meet the following conditions:
Wherein, δ indicates adjacent print point minimum spacing, and d indicates lenslet diameter in light-field camera, and L indicates adjacent print
Point on main lens virtual plane imaging point spacing (virtual plane refer in one focal length of light-field camera inner distance main lens and
Perpendicular to the plane of optical axis), g indicates sensor and microlens array spacing, and z indicates microlens array and virtual plane spacing, F
Interplanar spacing where indicating virtual plane and main lens, f' indicate lenticule optical center on color coding sheet and color-coding plate away from
From M indicates angular samples number (angular samples number refers to the number that same object point is recorded on the image sensor in space).k
It is related with the arrangement of lenticule, interval, lenticule rear imaging region size.If a lenticule rear imaging radius is r, micro-
The distance at lens unit center to the gap center of contiguous microlens is a, defines scale factor, then α need to meet α r >=a, this
K=min (α) is chosen in invention.The adjacent print point minimum spacing in single demarcating circle can be found out by formula (1-1):
It is and single on sensor next, the spacing of plane where calculating color-coding plate and light-field camera main lens
The corresponding print point number n of round spotpRelationship such as formula (1-3).
Wherein, f' indicates the distance of lenticule optical center on color coding sheet and color-coding plate, y0Indicate print point in Y-axis
On coordinate, Q1、Q3For two print points in demarcating circle single on color coding sheet, Q1The light of sending passes through under main lens
Edge, Q3The light of sending passes through main lens top edge, drIndicate lenslet diameter on color-coding plate, TdIndicate color coding
The distance of plane and light-field camera main lens optical center where microlens array optical center of lens, D indicate light-field camera main lens in plate
The diameter of clear aperature, α0Indicate the line and camera light at lenticule center and light-field camera main lens center on color-coding plate
The angle of axis, α1、α3Respectively indicate Q1、Q3Through the angle of its front the lenticule light issued and camera optical axis, δ indicates phase
Adjacent print point minimum spacing.By adjusting npSize setting color-coding plate and main lens between distance, for guarantee calculate essence
Degree, single round spot correspond to print point number on horizontal and vertical no more than 10.
Finally, single demarcating circle is imaged as multiple round spots, phase on the sensor inside light-field camera on color-coding plate
Region is imaged on a sensor and is not overlapped for adjacent demarcating circle, and the center of demarcating circle is away from least meeting following item on color-coding plate
Part:
M+n=g+z+F
Wherein, g indicates interplanar spacing, z table where lenticule optical center in light-field camera internal sensor and microlens array
Show that plane and virtual plane spacing where lenticule optical center, x indicate single demarcating circle in light in microlens array in light-field camera
Round spot diameter is imaged on the camera sensor of field, F indicates interplanar spacing where virtual plane and main lens, and D indicates light-field camera master
Lens clear aperture diameter, what the top edge light and the lower demarcating circle adjacent with top demarcating circle that top demarcating circle issues issued
Lower edge light is met at a bit, and m indicates this point at a distance from sensor plane, and n indicates that two adjacent demarcating circles are sent out respectively up and down
Plane and main lens interplanar spacing where the intersection point of top edge light and lower edge light out, y indicate adjacent demarcating circle in light
Field camera virtual plane imaging point spacing, TdIndicate plane and light field phase where microlens array optical center of lens in color-coding plate
The distance of owner's optical center of lens.Adjacent demarcating circle minimum centers-distance is calculated by formula (1-4):
In addition, the present invention also proposes a kind of light-field camera method for calibrating external parameters, including the following steps:
Step 1, angle turntable stepping required when calibration prior information is determined.On a sensor due to a demarcating circle
Imaging point corresponding printing points it is less, therefore color-coding plate is placed in angle turntable, in each rotation angle
The light information that corresponding print point is issued is recorded, the light issued by rotating all print points of multiple angle recordings.Wherein,
Angle turntable stepping is less than or equal to the angle that single print point can be resolved on color coding sheet, can match with scaling board.
The angle, θ that single print point can be resolved on color coding sheet is calculated according to formula (1-6) and (1-7), so that it is determined that angle rotates
Platform stepping.
Wherein, FoV is lenticule field angle, and θ is the angle that single print point can be resolved on color coding sheet, and d' is mark
Determine circular diameter, f' indicates the distance of lenticule optical center on color coding sheet and color-coding plate, nrFor in demarcating circle diameter range
Printable points.
Step 2, color-coding plate, the angle that record demarcating circle emits beam are shot.The present invention is being sensed according to demarcating circle
Imaging position on device calculates the light angle that corresponding print point issues.Wherein, light angle includes light in the face YOZ, the face XOZ
Projection and Z axis angle, therefore calculate light first in the projection in the face YOZ and the angle of Z axis, then by color-coding plate around Z
Axis, which is rotated by 90 °, calculates light in the projection in the face XOZ and the angle of Z axis.Since the calculating process of both the above angle is identical,
Only illustrate light in the projection in the face YOZ and the angle calcu-lation process of Z axis.
Placing color-coding plate makes its initial position place plane perpendicular to camera optical axis, compiles color using stepper motor
The face XOY of code plate positively and negatively respectively rotates fixed stepping around X-axis, penetrates all print points in demarcating circle after rotating repeatedly
The light that lenticule issues is recorded.In rotation steps, the light being recorded is walked in the projection in the face YOZ and the angle calcu-lation of Z axis
It is rapid as follows:
(1) known color encoding board obtains scaling board normal direction relative to the rotation angle ξ ∈ (- 90 °, 90 °) of its initial position
Angle η=ξ of amount and camera optical axis.
(2) round spot center on sensor is extracted, is decoded according to internal reference, lenticule centre coordinate where calculating it passes through light
Line is propagated and refraction matrix, calculates the angle β of projection and camera optical axis of the light-field camera main lens extraneous light on the face YOZ.
Wherein A is refraction matrix, TtFor propogator matrix, F is the main focal length of lens, g indicate light-field camera internal sensor and
Interplanar spacing where lenticule optical center in microlens array, z are indicated in light-field camera in microlens array where lenticule optical center
Plane and virtual plane spacing, P0For round spot centre coordinate, P on CCD1Lenticule centre coordinate where indicating round spot, P2Indicate solution
The coordinate of light and main lens plane point of intersection that code goes out.
(3) angle γ=η+β of the light direction of propagation and color-coding plate normal direction is calculated.
Step 3, in the image of each step shooting, the color that round spot is imaged in each demarcating circle on a sensor is fitted to
One color value, the colouring information to emit beam as print points different in demarcating circle.In conjunction with step 2 angle calculation as a result,
The color and angle corresponding relationship that all directions on color-coding plate are emitted beam are as the prior information of light.
Step 4, it is known that prior information, shooting have the scene of color-coding plate.It is fitted according to round spot color on sensor
Color value, search its matched light in prior information and project on the face XOZ and the face YOZ and the angle of Z axis, in conjunction with round spot pair
The coordinate for answering lenticule center in front of demarcating circle, establishes the emergent ray equation under XYZ coordinate system.
Step 5, incident ray equation is established.The position for calculating round spot center on sensor record image, is looked into conjunction with internal reference
Lenticule centre coordinate where looking for it obtains light double flat areal coordinate according to two o'clock coordinate, by propagating, reflecting matrix, calculates
The equation of light biplane uvst outside main lens, and convert it to the X where camera1Y1Z1Under coordinate system.
Step 6, emergent ray and the incident ray matching obtained according to step 4 and step 5 is to establishing equation.Due to coordinate
It is XYZ and X1Y1Z1Between exist rotation, translation transformation, it is assumed that two coordinate system X-axis angles are ω, and Y-axis angle isZ axis folder
Angle is κ, XYZ coordinate system change of origin to X1Y1Z1The displacement of coordinate origin is (Tx,Ty,Tz), therefore between two coordinate systems
Spin matrix be
Translation matrix is
Any straight line is expressed as l (x, y, z, r in XYZ coordinate systemx,ry,rz), (x, y, z) indicates any one on l
Coordinate of the point at XYZ, [rx,ry,rz]TIndicate the direction vector of l.Straight line l is transformed into X1Y1Z1Coordinate representation under coordinate system
For l1(x',y',z',rx',ry',rz'), (x', y', z') indicates point (x, y, z) in X1Y1Z1Under coordinate, [rx',ry',rz']T
Indicate l in X1Y1Z1Under direction vector.Assuming that there is two o'clock A on l0(xa,ya,za) and B0(xb,yb,zb), l1On have two o'clock A1
(xa',ya',zaWith ' and) B1(xb',yb',zb').Then have
l1Linear equation be
l0And l1Direction vector be respectively
[rx,ry,rz]T=(xa-xa,ya-yb,za-zb)
[rx',ry',rz']T=(xa'-xa',ya'-yb',za'-zb') (1-14)
Transformation relation so between rectilinear direction vector
[rx',ry',rz']T=R-1[rx,ry,rz]T (1-15)
Convolution (1-12) (1-13) and (1-15) as a result, can find out l0On a bit (x, y, z) be transformed into l1Upper (x', y',
Z' formula)
[x',y',z']T=λ R-1[rx,ry,rz]T+R-1([x,y,z]T-T) (1-16)
Therefore straight line and X in XYZ coordinate system1Y1Z1The transformational relation of upper straight line is
Equation group is established thus according to transformational relation of the light matching between.
Step 7, solve system of equation, to seek rotation, 6 dimension parameter of translationω、κ、T0、T1、T2And biplane
Uvst origin and rectangular coordinate system X1O1Y12 dimension parameters of origin relative displacement.
The present invention is transformed using self-control light-field camera by the industrial interface camera of a VieworksVH-11MG.Figure
1 (a) expression imaging sensor and microlens array be located at after the imaging plane of main lens, Fig. 1 (b) indicate imaging sensor with
Microlens array is between main lens and main lens imaging plane.The light-field camera structure that the present invention uses such as Fig. 1 (a) institute
Show, light passes sequentially through main lens, lenticule after entering light-field camera, eventually arrives at sensor.101 focal length F=of main lens
50mm, clear aperture diameter D=5.609~6.723mm.103 arranged in regular hexagon shape of camera internal lenticule, at this timeSingle lenslet diameter d=0.3mm, focal length f=2.726mm.104 Pixel size of sensor is 9 μm * 9 μm, always
Pixel number is 2672*4008.Angular samples number M=4, microlens array range sensor plane g=3.316mm, lenticule battle array
Column and 102 distance z=15.317mm of virtual plane.
The color-coding plate structure that the present invention uses is as follows:
Microlens array: as shown in Fig. 2, hexagon is arranged, 201 focal length f of lenticuler=3mm, lenslet diameter dr=
2.286mm, microlens array are close to color coding sheet 301 and are placed, and f'=f is mader。
Color coding sheet: as shown in Fig. 2, total 2*3 demarcating circle, each demarcating circle is located at lenticule dead astern.Demarcating circle
As shown in figure 3, being calculated according to formula (1-1) (1-2), the present invention is beaten the index path of interior print point distance computation using 600dpi's
Print machine prints demarcating circle, and color value is that triple channel combination of two forms in demarcating circle, and color value takes 5~250, it is horizontal and vertical on
Consecutive points color value difference 5.Calibration circular diameter is slightly less than lenslet diameter, takes calibration circular diameter d'=2.117mm, calculates adjacent
Demarcate circle center away from index path as shown in figure 5, determining adjacent calibration circle center away from S=according to formula (1-4) (1-5)
13.716mm。
The distance between color-coding plate and light-field camera main lens calculate index path as shown in figure 4, and according to formula (1-
3) it calculates.The present invention is located at color-coding plate at 350~500mm of main lens, and is placed on angle turntable,
Angle turntable is controlled by computer to be rotated, and initial position is parallel with plane where camera main lens.It is single on color coding sheet
The angle calculation that a print point can be resolved by formula (1-6) (1-7) as shown in fig. 6, calculated, so that it is determined that angle turntable
Stepping.The present invention is by angle turntable 0.1 ° of stepping in the clockwise direction, and 20 ° of corotating, in stepping gap, control camera is taken pictures；
Angle turntable rotates counterclockwise, and stepping and total rotation angle are identical as clockwise direction, controls phase in stepping gap
Machine is taken pictures.Next color-coding plate is rotated by 90 ° around X-axis, it is identical before operating and rotating.
The image recorded when being rotated according to color-coding plate calculates demarcating circle and emits beam in the projection in the face XOZ and the face YOZ
Respectively with encoding board where plane normal angle, as shown in Figure 7.Angle turntable rotates ξ angle, obtains scaling board normal vector
With angle η=ξ of camera optical axis, β is calculated by formula (1-9), to calculate light angle γ=η+β.
In the scene image of shooting, the emergent ray equation under XYZ coordinate system is established using prior information, utilizes internal reference
Establish X1Y1Z1Incident ray equation under coordinate system.As shown in figure 8, establishing equation group according to formula (1-17), two are solved
Transition matrix R and T between coordinate system, that is, the outer ginseng of light-field camera.
Claims (6)
1. a kind of light-field camera external parameters calibration device, including a color-coding plate and an angle turntable, feature exist
In: color-coding plate is placed in front of light-field camera main lens within 2m, and color-coding plate is fixed on angle turntable
On；The color-coding plate is made of color coding sheet and microlens array；At least 2*3 calibration is printed on color coding sheet
Circle, demarcating circle color take two kinds of RGB channel, the print point edge in demarcating circle respectively in horizontal and vertical both direction
Horizontal and vertical direction uniform sampling within the scope of 0-255, the Color Channel that adjacent demarcating circle is taken in the same direction are different；
Microlens array is placed in color coding sheet close to light-field camera direction side, and each demarcating circle on color coding sheet is made to be located at one
A lenticule dead astern, and demarcate circular diameter and be less than lenslet diameter.
2. light-field camera external parameters calibration device according to claim 1, it is characterised in that: adjacent in the demarcating circle to beat
Print the minimum spacing of pointWherein, f' indicates lenticule optical center on color coding sheet and color-coding plate
Distance, d indicate lenslet diameter in light-field camera, and F indicates the main lens focal length of light-field camera, and M indicates angular samples number, i.e., empty
Between in the number that is recorded on the image sensor of same object point, k=min (α), if a lenticule rear imaging radius is r,
The distance at lenticule unit center to the gap center of contiguous microlens is a, then scale factor meets α r >=a.
3. light-field camera external parameters calibration device according to claim 1, it is characterised in that: the color coding sheet and face
The distance f' of lenticule optical center print point number n corresponding with round spot single on sensor on color encoding boardpRelationship it is as follows:
Wherein, y0Indicate coordinate of the print point in Y-axis, Q1、Q3For two printings in demarcating circle single on color coding sheet
Point, Q1The light of sending passes through main lens lower edge, Q3The light of sending passes through main lens top edge, drIndicate color-coding plate
Upper lenslet diameter, TdIndicate plane and light-field camera main lens optical center where microlens array optical center of lens in color-coding plate
Distance, D indicate light-field camera main lens clear aperature diameter, α0Indicate lenticule center and light field phase on color-coding plate
The line of owner lens centre and the angle of camera optical axis, α1、α3Respectively indicate Q1、Q3The light that lenticule issues through its front
The angle of line and camera optical axis；δ indicates the minimum spacing of adjacent print point in demarcating circle.
4. light-field camera external parameters calibration device according to claim 3, it is characterised in that: the single round spot correspondence is beaten
Print point number is on horizontal and vertical no more than 10.
5. light-field camera external parameters calibration device according to claim 1, it is characterised in that: the color-coding plate subscript
Surely round minimum centers-distanceWherein, g indicates light-field camera internal sensor and lenticule battle array
Interplanar spacing where lenticule optical center in column, z indicate in light-field camera plane and void where lenticule optical center in microlens array
Quasi-plane spacing, F indicate interplanar spacing where virtual plane and main lens, and D indicates light-field camera main lens clear aperture diameter,
The lower edge light that the top edge light and the lower demarcating circle adjacent with top demarcating circle that top demarcating circle issues issue meets at one
Point, TdIndicate the distance of plane and light-field camera main lens optical center where microlens array optical center of lens in color-coding plate；Institute
Virtual plane is stated to refer in one focal length of light-field camera inner distance main lens and perpendicular to the plane of optical axis.
6. a kind of light-field camera external parameters calibration method using claim 1 described device, it is characterised in that including following steps
It is rapid:
Step 1, color-coding plate is placed in angle turntable, the light that corresponding print point is issued is recorded in each rotation angle
Line information rotates the light that all print points of multiple angle recordings issue；Angle turntable stepping is less than or equal on color coding sheet
The angle that single print point can be resolvedWherein, FoV is lenticule field angle,d'
To demarcate circular diameter, f' indicates the distance of lenticule optical center on color coding sheet and color-coding plate, nrTo demarcate circular diameter model
Enclose interior printable points；
Step 2, coordinate system where plate is XYZ color-coded, and wherein plane where microlens array is XOY, and origin O is located at micro-
Lens array plane is towards light-field camera photographed scene side, and it is laterally Y-axis that longitudinal, which is X-axis, plane where color-coding plate
Normal vector is Z axis；Shoot color-coding plate, the angle that record demarcating circle emits beam, comprising the following steps:
(1) placing color-coding plate makes its initial position place plane perpendicular to camera optical axis；
(2) face XOY of color-coding plate is made positively and negatively respectively to rotate fixed stepping around X-axis using stepper motor, rotation is multiple
Later it is recorded the light that all print points are issued through lenticule in demarcating circle；
(3) according to color-coding plate relative to the rotation angle ξ ∈ (- 90 °, 90 °) of its initial position, obtain scaling board normal vector and
Angle η=ξ of camera optical axis；
(4) round spot center on sensor is extracted, is decoded according to internal reference, lenticule centre coordinate where calculating it is passed by light
Matrix is broadcast and is reflected, the angle β of projection and camera optical axis of the light-field camera main lens extraneous light on the face YOZ is calculated,
Wherein A is refraction matrix, TtFor propogator matrix, F is the main focal length of lens, and g indicates light-field camera internal sensor and lenticule
Interplanar spacing where lenticule optical center in array, z indicate in light-field camera in microlens array plane where lenticule optical center and
Virtual plane spacing, P0For round spot centre coordinate, P on CCD1Lenticule centre coordinate where indicating round spot, P2What expression decoded
The coordinate of light and main lens plane point of intersection；The virtual plane refer in one focal length of light-field camera inner distance main lens and
Perpendicular to the plane of optical axis；
(5) light is calculated in the projection in the face YOZ and angle γ=η+β of Z axis；
(6) color-coding plate is rotated by 90 ° about the z axis, repeats step (2)~(5), calculate projection and Z axis of the light in the face XOZ
Angle；
Step 3, in the image of each step shooting, the color that round spot is imaged in each demarcating circle on a sensor is fitted to one
Color value, the colouring information to emit beam as print points different in demarcating circle；All directions on color-coding plate are issued into light
Prior information of the color and angle corresponding relationship of line as light；
Step 4, shooting has the scene of color-coding plate, according to the color value that round spot color on sensor is fitted, searches priori
The matched light of its in information projects on the face XOZ and the face YOZ and the angle of Z axis, corresponds in conjunction with round spot micro- in front of demarcating circle
The coordinate at mirror center establishes the emergent ray equation under XYZ coordinate system；
Step 5, incident ray equation is established, the position at round spot center on sensor record image is calculated, searches it in conjunction with internal reference
Place lenticule centre coordinate obtains light double flat areal coordinate according to two o'clock coordinate, by propagating, reflecting matrix, calculates light
The equation of biplane uvst outside main lens, and convert it to the X where camera1Y1Z1Under coordinate system；
Step 6, according to emergent ray equation and the matching of incident ray equation to establishing equation；
It is X with coordinate system where light-field camera1Y1Z1, wherein plane where sensor is X1O1Y1, origin O1Positioned at sensor plane
The upper left corner is observed towards light-field camera photographed scene, longitudinal is X1Axis is laterally Y1Axis, optical axis direction Z1；False coordinate
It is X1Y1Z1X-axis angle with coordinate system XYZ is ω, and Y-axis angle isZ axis angle is κ, and XYZ coordinate system change of origin arrives
X1Y1Z1The displacement of coordinate origin is (Tx,Ty,Tz), then the spin matrix between two coordinate systems
Translation matrix
Any straight line is expressed as l (x, y, z, r in XYZ coordinate systemx,ry,rz), (x, y, z) indicates that any point exists on l
Coordinate under XYZ, [rx,ry,rz]TIndicate the direction vector of l；Straight line l is transformed into X1Y1Z1Coordinate representation under coordinate system is l1
(x',y',z',rx',ry',rz'), (x', y', z') indicates point (x, y, z) in X1Y1Z1Under coordinate, [rx',ry',rz']TTable
Show l in X1Y1Z1Under direction vector；There is two o'clock A on l0(xa,ya,za) and B0(xb,yb,zb), l1On have two o'clock A1(xa',ya',
za') and B1(xb',yb',zb'), then have
l1Linear equation be
l0And l1Direction vector be respectively
Then transformation relation [the r between rectilinear direction vectorx',ry',rz']T=R-1[rx,ry,rz]T；
Find out l0On a bit (x, y, z) be transformed into l1The formula of upper (x', y', z')
[x',y',z']T=λ R-1[rx,ry,rz]T+R-1([x,y,z]T-T)；
Then straight line and X in XYZ coordinate system1Y1Z1The transformational relation of upper straight line is
Step 7, solve system of equation, to seek rotation, 6 dimension parameter of translationω、κ、Tx、Ty、TzAnd biplane uvst is former
Point and rectangular coordinate system X1O1Y12 dimension parameters of origin relative displacement.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201511023954.5A CN105654484B (en) | 2015-12-30 | 2015-12-30 | Light-field camera external parameters calibration device and method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201511023954.5A CN105654484B (en) | 2015-12-30 | 2015-12-30 | Light-field camera external parameters calibration device and method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN105654484A CN105654484A (en) | 2016-06-08 |
| CN105654484B true CN105654484B (en) | 2019-01-18 |
Family
ID=56490018
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201511023954.5A Active CN105654484B (en) | 2015-12-30 | 2015-12-30 | Light-field camera external parameters calibration device and method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105654484B (en) |
Families Citing this family (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107492127B (en) * | 2017-09-18 | 2021-05-11 | 丁志宇 | Light field camera parameter calibration method and device, storage medium and computer equipment |
| CN107993266A (en) * | 2017-12-05 | 2018-05-04 | 西北工业大学 | Light-field camera intrinsic parameter caliberating device and method |
| CN109840922B (en) * | 2018-01-31 | 2021-03-02 | 中国科学院计算技术研究所 | Depth acquisition method and system based on binocular light field camera |
| CN108931205B (en) * | 2018-07-25 | 2024-04-09 | 杭州非白三维科技有限公司 | Three-dimensional scanning system and scanning method |
| CN109285194B (en) * | 2018-09-29 | 2021-06-29 | 人加智能机器人技术（北京）有限公司 | Camera calibration board and camera calibration data acquisition method |
| CN109785394A (en) * | 2018-12-31 | 2019-05-21 | 深圳积木易搭科技技术有限公司 | A kind of camera and turntable combined calibrating method, device and storage medium |
| CN111750850B (en) * | 2019-03-27 | 2021-12-14 | 杭州海康威视数字技术股份有限公司 | Angle information acquisition method, device and system |
| CN110322514B (en) * | 2019-06-24 | 2022-05-03 | 西北工业大学 | Light field camera parameter estimation method based on multi-center projection model |
| CN110310337B (en) * | 2019-06-24 | 2022-09-06 | 西北工业大学 | Multi-view light field imaging system full-parameter estimation method based on light field fundamental matrix |
| CN110365965B (en) * | 2019-06-26 | 2020-11-06 | 北京邮电大学 | Three-dimensional light field image generation method and device |
| CN111179353B (en) * | 2019-12-17 | 2023-05-26 | 清华大学深圳国际研究生院 | Micro lens array calibration method and system for light field camera |
| CN111968185B (en) * | 2020-09-01 | 2024-02-02 | 深圳辰视智能科技有限公司 | Calibration plate, nine-point calibration object grabbing method and system based on coding definition |
| CN113345032B (en) * | 2021-07-07 | 2023-09-15 | 北京易航远智科技有限公司 | Initialization map building method and system based on wide-angle camera large distortion map |
| CN113327211B (en) * | 2021-07-28 | 2021-11-02 | 清华大学 | Correction method and device for large-field-of-view high-resolution light field microscope system |
| CN113838145B (en) * | 2021-09-24 | 2024-04-30 | 重庆长安汽车股份有限公司 | Automatic calibration method for external parameters of vehicle-mounted camera |
| CN114782545A (en) * | 2022-03-18 | 2022-07-22 | 山东大学 | Light field camera calibration method for eliminating distortion of main lens |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1490765A (en) * | 2002-10-18 | 2004-04-21 | �Ϻ���ͨ��ѧ | Method for calibrating camera and colour reference thereby |
| US20050270375A1 (en) * | 2004-05-24 | 2005-12-08 | Pierre Poulin | Camera calibrating apparatus and method |
| CN101673397A (en) * | 2009-09-30 | 2010-03-17 | 青岛大学 | Digital camera nonlinear calibration method based on LCDs |
| CN102663732A (en) * | 2012-03-14 | 2012-09-12 | 中国科学院光电研究院 | Relative radiometric calibration method for light field camera |
-
2015
- 2015-12-30 CN CN201511023954.5A patent/CN105654484B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1490765A (en) * | 2002-10-18 | 2004-04-21 | �Ϻ���ͨ��ѧ | Method for calibrating camera and colour reference thereby |
| US20050270375A1 (en) * | 2004-05-24 | 2005-12-08 | Pierre Poulin | Camera calibrating apparatus and method |
| CN101673397A (en) * | 2009-09-30 | 2010-03-17 | 青岛大学 | Digital camera nonlinear calibration method based on LCDs |
| CN102663732A (en) * | 2012-03-14 | 2012-09-12 | 中国科学院光电研究院 | Relative radiometric calibration method for light field camera |
Non-Patent Citations (2)
| Title |
|---|
| 基于随机遮挡孔径的光场图像混叠检测方法;肖照林 等;《自动化学报》;20140228;第40卷(第2期);第247-254页 |
| 微透镜阵列式光场成像模型及其标定方法;张旭 等;《光学学报》;20141231;第34卷(第12期);第1-13页 |
Also Published As
| Publication number | Publication date |
|---|---|
| CN105654484A (en) | 2016-06-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105654484B (en) | Light-field camera external parameters calibration device and method | |
| CN110044300B (en) | Amphibious three-dimensional vision detection device and detection method based on laser | |
| CN105744163B (en) | A kind of video camera and image capture method based on depth information tracking focusing | |
| CN109242908A (en) | Scaling method for underwater two CCD camera measure system | |
| CN105940674B (en) | System and method for the baseline for adjusting the imaging system with micro lens arrays | |
| CN107121109A (en) | A kind of structure light parameter calibration device and method based on preceding plated film level crossing | |
| US10715711B2 (en) | Adaptive three-dimensional imaging system and methods and uses thereof | |
| CN109859272A (en) | A kind of auto-focusing binocular camera scaling method and device | |
| CN105115560B (en) | A kind of non-contact measurement method of cabin volume of compartment | |
| CN103761732A (en) | Three-dimensional imaging device with visible light and thermal infrared integrated and calibrating method thereof | |
| CN103940369A (en) | Quick morphology vision measuring method in multi-laser synergic scanning mode | |
| CN109712232B (en) | Object surface contour three-dimensional imaging method based on light field | |
| CN101858741A (en) | Zoom ranging method based on single camera | |
| CN101858743B (en) | Distance measuring device and method based on large-field shooting and image processing | |
| CN110132226A (en) | The distance and azimuth angle measurement system and method for a kind of unmanned plane line walking | |
| CN109325981A (en) | Based on the microlens array type optical field camera geometrical parameter calibration method for focusing picture point | |
| CN114157852B (en) | Virtual camera array three-dimensional imaging method and system based on rotating double prisms | |
| CN105004324A (en) | Monocular vision sensor with triangulation ranging function | |
| CN104976968A (en) | Three-dimensional geometrical measurement method and three-dimensional geometrical measurement system based on LED tag tracking | |
| CN206546159U (en) | Microscopic three-dimensional measurement apparatus and system | |
| CN108550171A (en) | The line-scan digital camera scaling method containing Eight Diagrams coding information based on Cross ration invariability | |
| CN112907647B (en) | Three-dimensional space size measurement method based on fixed monocular camera | |
| CN108205799B (en) | Image splicing method and device | |
| CN206563549U (en) | A kind of big visual field rapid dimensional measurement system | |
| CN106791335B (en) | A kind of compact big visual field optical field acquisition system and its analysis optimization method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |