CN109325981A - Based on the microlens array type optical field camera geometrical parameter calibration method for focusing picture point - Google Patents

Based on the microlens array type optical field camera geometrical parameter calibration method for focusing picture point Download PDF

Info

Publication number
CN109325981A
CN109325981A CN201811070094.4A CN201811070094A CN109325981A CN 109325981 A CN109325981 A CN 109325981A CN 201811070094 A CN201811070094 A CN 201811070094A CN 109325981 A CN109325981 A CN 109325981A
Authority
CN
China
Prior art keywords
formula
picture point
matrix
microlens array
array type
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201811070094.4A
Other languages
Chinese (zh)
Other versions
CN109325981B (en
Inventor
邱钧
刘畅
王媛
亢新凯
李健
史立根
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Information Science and Technology University
Original Assignee
Beijing Information Science and Technology University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Information Science and Technology University filed Critical Beijing Information Science and Technology University
Priority to CN201811070094.4A priority Critical patent/CN109325981B/en
Publication of CN109325981A publication Critical patent/CN109325981A/en
Application granted granted Critical
Publication of CN109325981B publication Critical patent/CN109325981B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10052—Images from lightfield camera

Abstract

The invention discloses a kind of based on the microlens array type optical field camera geometrical parameter calibration method for focusing picture point, method includes the following steps: S1, according to the focal imaging index path of microlens array type optical field camera, obtains object point and focus mapping relations of the picture point about main lens；S2 obtains focusing picture point and mapping relations of the detector picture point about lenticule according to the focal imaging index path of microlens array type optical field camera；S3 solves the coordinate for focusing picture point according to the detector picture point that detection obtains；S4 solves camera internal parameter matrix and external parameter square in peg model according to the coordinate of the S3 focusing picture point obtained；S5, the camera internal parameter matrix and external parameter matrix obtained by S4 demarcate the geometric parameter of microlens array type optical field camera.By using method provided by the invention, the geometrical parameter calibration of microlens array type optical field camera is carried out, can calibrate and realize that calculating imaging provides reliable parameter for subsequent optical field data.

Description

Based on the microlens array type optical field camera geometrical parameter calibration method for focusing picture point
Technical field
The present invention relates to computer vision and digital image processing techniques fields more particularly to a kind of based on focusing picture point Microlens array type optical field camera geometrical parameter calibration method.
Background technique
Optical field imaging technology can record space and the angle information of light simultaneously, can break through the office of conventional lenses imaging Limit.It can realize that digital refocusing, depth of field extension, scene depth calculates and the calculating such as scene three-dimensional reconstruction are imaged using light field data Technology is widely used in computer vision and calculates imaging field.
The different sample modes of light field form the different hardware system for obtaining light field data, such as microlens array type optical field Camera, structure camera array type light-field camera and mask-type light-field camera etc..Wherein, microlens array type optical field camera by Microlens array is added in imaging system, by the light beam splitting of main lens, continuous light field discretization completion light field data is adopted Collection.There is microlens array type optical field camera simple hardware configuration, portable devices and single exposure can obtain light field data etc. Advantage is the light filed acquisition device of current mainstream.The Typical Representative of microlens array type optical field camera is the designs such as Ng Ren Light-field camera (Plenoptic1.0).The imaging device is focal plane one microlens array of placement in traditional camera, and Image detector is placed at one times of focal length of lenticule.Georgiew etc. proposes focus type light-field camera (Plenoptic 2.0), imaging detector reduces the sampling of radiation direction dimension not on the focal plane of microlens array, with lower direction point Resolution exchanges relatively higher spatial resolution for, effectively increases the imaging resolution of refocusing image.
The geometrical parameter calibration of optical field imaging system is light field data calibration and the important prerequisite for realizing calculating imaging technique. The calibration process of the geometric parameter of camera is divided by the positive process model building of object point to detector picture point, and, it is anti-by Corner Detection Drill the process for solving camera geometric parameter.Positive process model building establishes picture point two-dimensional coordinate on object point three-dimensional coordinate and imaging surface Mapping relations, this mapping relations are described by camera geometric parameter.And the process of camera geometric parameter is solved, it is to utilize to detect Imaging surface on angular coordinate and object point coordinate, camera geometric parameter is solved based on positive process model building inverting, it is corresponding non-linear to ask Solution problem.That is, the scaling method of the geometric parameter of existing camera merges geometric parameter in a model, detection Angle point out, then the geometric parameter by angle point calculating microlens array type optical field camera.The calibration side of the geometric parameter of this camera The Model coupling degree of method is high, calculates partially complicated.
Summary of the invention
The purpose of the present invention is to provide a kind of based on the microlens array type optical field camera geometric parameter mark for focusing picture point Method is determined to solve the geometric parameter of camera.
To achieve the above object, the present invention provides a kind of based on the microlens array type optical field camera geometric parameters for focusing picture point Number scaling method, the microlens array type optical field camera geometrical parameter calibration method based on focusing picture point includes following step Rapid: S1 obtains object point and focuses picture point about main lens according to the focal imaging index path of microlens array type optical field camera Mapping relations；S2 obtains focusing picture point and detector picture point according to the focal imaging index path of microlens array type optical field camera Mapping relations about lenticule；S3 solves the coordinate for focusing picture point according to the detector picture point that detection obtains；S4, according to S3 The coordinate of the focusing picture point of acquisition solves camera internal parameter matrix and external parameter matrix in peg model；And S5, lead to The camera internal parameter matrix and external parameter matrix of S4 acquisition are crossed, the geometric parameter of microlens array type optical field camera is demarcated, The geometric parameter includes at a distance from the spacing and main lens to lenticule of microlens array and detector plane.
Further, " mapping relations of object point and focusing picture point about main lens " in S1 are expressed as formula (1), are set: Object point is expressed as M, and coordinate is (xw,yw,zw)；It focuses picture point and is expressed as m ', coordinate is (u ', v '):
Sm '=A [R t] M, (1)
In formula (1), s is scale factor；M=[xw,yw,zw,1]T；M '=[u ', v ', 1]T；[R t] indicates microlens array The external parameter matrix of type light-field camera, A indicate the inner parameter matrix of microlens array type optical field camera；
[R t [formula (3) and formula (2) being expressed as:
In formula (2) and formula (3), q1、q2、q3Respectively indicate three seats in camera coordinates system and world coordinate system conversion process The rotation angle of parameter, tx、ty、tzIndicate the distance that the origin of world coordinate system is translated along three reference axis；
The formula (4) that A is expressed as:
In formula (4), dx × dy is the Pixel Dimensions of detector, u0、v0For seat of the main lens centre under image coordinate system Mark, θ are the out of plumb inclination angle of two reference axis；
Wherein: camera coordinates system is using the center of main lens as origin O, xc、ycAxis is parallel to detector plane, zcAxis is vertical In detector plane；World coordinate system is to demarcate the center of object plane as origin Ow, xw、ywAxis is parallel to calibration object plane, zwAxis Perpendicular to calibration object plane；Image coordinate system is using detector plane center as origin Oc, u, v axis is parallel to xc、ycAxis.
Further, " the focusing picture point and mapping relations of the detector picture point about lenticule " in S2 is formula (8), if Fixed: detector picture point includes picture point m1With picture point m2, picture point m1It is (u in the coordinate of image coordinate system1,v1), it is corresponding micro- The center of mirror is (p in the coordinate of image coordinate system1,q1)；Picture point m2It is (u in the coordinate of image coordinate system2,v2), it is corresponding Lenticule center is (p in the coordinate of image coordinate system2,q2)；
Sm '=sT-1m1=A [R t] M (8)
In formula (8): m1It is expressed as picture point m1Coordinate vector；T is picture point m ' and picture point m1Between transition matrix；
m1It is expressed as m1=[u1,v1,1]T；
The formula (6) and formula (7) that T is expressed as:
δ=b/a=(u1-u2)/(p1-p2)-1 (7)。
Further, the detector picture point that S3 is specifically obtained according to detection, and being focused into using microlens array type As the geometrical relationship in index path, the coordinate for focusing picture point is solved；
" the detector picture point that detection obtains " is picture point m1、m2；
" geometrical relationship in the focal imaging index path of microlens array type " is expressed as formula (9):
Formula (10) then can be obtained according to formula (8):
Firstly, by picture point m1、m2Image coordinate system coordinate and corresponding lenticule center image coordinate system seat It marks substitution formula (7), δ value is calculated；
Then, the δ value being calculated is substituted into formula (9), solves and focuses picture point m ' coordinate (u ', v ').
Further, S4 is specifically included: S41, if peg model plane z in world coordinate systemw=0, it substitutes into formula (1), 3 × 3 homography matrix H can be obtained with corresponding focusing picture point m ' coordinate according to object point M, the maximum for being solution matrix H is seemingly So estimate, the focusing picture point in iterative image seeks lsqnonlin；S42, according to r1With r2Constraint condition, will The r obtained by matrix H1With r2Expression formula substitute into, then in equation that symmetrical matrix B and its calculating formula are brought into, finally To 2 × 6 matrix Vs, the homography matrix h of three width images is substituted intoiCan solution matrix b, and then the unknown ginseng in solution matrix A Number；S43 has solved matrix A, and matrix A is substituted into r1With r2Expression formula can solve r1、r2, and associate(d) matrix H can be solved Unknown parameter in matrix [R t] out；S44 demarcates n width image, and the focusing picture point of each image has m, to above-mentioned The parameter iteration optimization that method for solving is found out, seeks lsqnonlin, is carried out with the parameter found out to above-mentioned steps Nonlinear optimization.
Further, S41 is specifically included:
If peg model plane z in world coordinate systemw=0, (1) formula is substituted into, then there are formula (11):
3 × 3 homography matrix H can be obtained with corresponding focusing picture point m ' coordinate according to object point M, form is as follows:
H=[h1 h2 h3]=λ A [r1 r2 t] (12)
In formula (12), hi=[hi1,hi2,hi3]T, λ is any scalar；
For the maximal possibility estimation of solution matrix H, the focusing picture point in iterative image seeks lsqnonlin:
In formula (13), m 'iThe actual coordinate vector of picture point is focused for i-th in same piece image,According to Formula (11), formula (12) are by object point MiIt is calculated.
Further, S42 is specifically included:
By the property of spin matrix it is found that r in matrix R1With r2Meet following two constraint condition:
From formula (12)
r1=λ A-1h1,r2=λ A-1h2 (15)
In formula (15), λ=1/ | | A-1h1| |=1/ | | A-1h2| |, formula (15) substitution formula (14) can be obtained:
It enables
B is symmetrical matrix, can be calculated formula (18):
In formula (18):
B=[B11,B12,B22,B13,B23,B33]T (19)
vij=[hi1hj1,hi1hj2+hi2hj1,hi2hj2,hi3hj1+hi1hj3,hi3hj2+hi2hj3,hi3hj3]T (20)
Formula (17) and formula (18), which are substituted into formula (16), can obtain formula (21):
There are formula (22):
In formula (22), V is 2 × 6 matrixes, contains 6 unknown parameters, substitutes into the homography matrix h of three width imagesiIt can solve Matrix b, and then the unknown parameter in solution matrix A:
γ=- B12α2β/λ (26)
u0=γ v0/β-B13α2/λ (27)
So that it is determined that the inner parameter matrix A of microlens array type optical field camera.
Further, S43 is specifically included:
S42, which is solved matrix A substitution formula (15), can solve r1、r2, specific calculation expression is formula (29):
r3=r1×r2 (29)
Formula (30) can be obtained by formula (12):
T=λ A-1h3 (30)。
Further, S44 is specifically included:
N width image is demarcated, the focusing picture point of each image has m, the parameter that above-mentioned method for solving is found out into Row iteration optimization, seeks lsqnonlin:
In formula (31), m 'ijFor the focusing picpointed coordinate of i-th j-th of angle point of width image,According to Formula (8) by the i-th width image j-th of object point coordinate vector MjAnd Ai, Ri, tiIt is calculated, the initial value of A and [R t] are by S41 It is calculated to S43.
Further, S5 is specifically included:
The diameter d of main lens pupil diameter D and lenticule image meets following formula:
D/L=d/b (32)
Formula (33) can be obtained by formula (7):
A=b/ δ (33)
Formula (32) and formula (33) are substituted into α=(L-a)/(dx) in formula (4), formula (34) and formula (35) can be acquired:
L=a+ α dx (34)
B=α δ ddx/ (δ D-d) (35)
In formula (34) and formula (35), parameter alpha, δ become known parameters.
Model is divided into two conjugate relations by method provided by the invention, detects angle point, by angle point according to the second conjugation Relationship obtains focusing picture point, then the geometric parameter of microlens array type optical field camera, this method are calculated by the second conjugate relation Model coupling degree is low, calculates easy.
Detailed description of the invention
Fig. 1 is provided by the invention based on the microlens array type optical field camera geometrical parameter calibration method for focusing picture point Flow chart；
Fig. 2 is the focal imaging index path of microlens array type provided by the invention；
Fig. 3 is image coordinate system in the present invention, correspondence diagram between camera coordinates system and world coordinate system.
Specific embodiment
In the accompanying drawings, same or similar element is indicated using same or similar label or there is same or like function Element.The embodiment of the present invention is described in detail with reference to the accompanying drawing.
As shown in Figure 1, provided in this embodiment based on the microlens array type optical field camera geometric parameter mark for focusing picture point Determine method the following steps are included:
S1 obtains object point and focuses picture point about master according to the focal imaging index path of microlens array type optical field camera The mapping relations of lens, that is, " the first conjugate relation " mentioned below.
S2 obtains focusing picture point and detector picture point according to the focal imaging index path of microlens array type optical field camera About the mapping relations of lenticule, that is, " the second conjugate relation " mentioned below.
S3 solves the seat for focusing picture point using inverse conversion matrix or geometrical relationship according to the detector picture point that detection obtains Mark.
S4 solves the camera internal in peg model using iterative algorithm and joins according to the coordinate of the S3 focusing picture point obtained Matrix number and external parameter matrix." camera internal parameter matrix " in text refers to the inside of microlens array type optical field camera Parameter, " camera external parameter matrix " refer to the external parameter of microlens array type optical field camera.
S5, the camera internal parameter matrix and external parameter matrix obtained by S4 demarcate microlens array type optical field phase The geometric parameter of machine, the spacing b and main lens to lenticule distance L of the geometric parameter microlens array and detector plane.
Just five steps of the invention are described in detail separately below.
In one embodiment, in S1, microlens array type optical field camera before the detector in traditional camera by putting Microlens array is set, the field information of scene is obtained.
The index path of microlens array type optical field camera is as shown in Figure 2.As shown in Fig. 2, microlens array is by multiple arrays The structure of the lenticule composition of formula arrangement, such as the partial microlenses in microlens array shown in Figure 2, i.e. the first lenticule L1, the second lenticule l2, third lenticule l3, the 4th lenticule l4.
In Fig. 2, sequence from left to right is successively are as follows: the straight dashed line P1 at the place picture point m ', which is illustrated that, to be focused into where picture point Plane (being hereafter referred to as " being focused into image planes ").Focus picture point m1With focusing picture point m2The number line P2 at place is illustrated that Plane (being hereafter referred to as " detector face ") where detector.First lenticule l1, the second lenticule l2, third lenticule Plane where l3, the 4th lenticule l4 while the number line P3 at place illustrate that microlens array (is hereafter referred to as " micro- Lens array face ").Number line P3 where main lens lm illustrates that main lens plane.Straight dashed line P4 signal where object point M Be plane (being hereafter referred to as " object plane ") where object point.
Wherein: the spacing of microlens array surface and detector face is b, it may also be said to be: microlens array and detector are flat The spacing b in face, that is, one of the geometric parameter to be calibrated of the invention referred in S5 above.Main lens plane is to lenticule battle array The spacing in column face is L, it may also be said to be: the distance L of main lens to lenticule, that is, the present invention referred in S5 above is wait mark The two of fixed geometric parameter.
The distance for being focused into image planes to microlens array surface is a, and the focal length of lenticule is f, from lens imaging formula 1/a+1/b=1/f.The case where b < f is corresponded in Fig. 2, a < 0 at this time, the picture point that object point is formed through main lens is in microlens array The other side is the virtual image.
The distance for being focused into image planes to main lens plane is bL, the distance of object plane to main lens plane is aL.A is diaphragm, The pupil diameter of main lens is D, and the diameter of lenticule image is d.
In S1, for the microlens array arrangement mode of above-mentioned microlens array type, the identical lenticule battle array of focal length is considered Column provide the direct problem modeling that picture point is generated by object point.Direct problem modeling includes establishing object point and focusing picture point about main lens Mapping relations, and focus picture point and mapping relations of the detector picture point about lenticule.
Microlens array type focal imaging process shown in Fig. 2, the process is there are two conjugate relations, i.e. object point M and poly- Burnt picture point m ' is about two focusing picture point m on main lens planar conjugate, picture point m ' and detector1,m2About microlens array surface Conjugation, respectively corresponds " the first conjugate relation " and " the second conjugate relation " mentioned above.
" mapping relations of object point and focusing picture point about main lens " in S1 specifically:
For object point M and mapping relations of the picture point m ' about main lens are focused, are described by the first conjugate relation.Object point M warp Main lens forms the process of picture point m ', can be converted by the coordinate between world coordinate system, camera coordinates system and image coordinate system Relationship describes.
As shown in figure 3, setting: three-dimensional camera coordinate system is using the center of main lens as origin O, xc、ycAxis is parallel to detector Plane, zcAxis is perpendicular to detector plane.Two dimensional image coordinate system is using detector plane center as origin Oc, u, v axis is parallel to xc、ycAxis.Three-dimensional world coordinate system is to demarcate the center of object plane as origin Ow, xw、ywAxis is parallel to calibration object plane, zwAxis hangs down Directly in calibration object plane.The unit of three-dimensional camera coordinate system is pixel, and the unit of two dimensional image coordinate system is mm, three-dimensional world The unit of coordinate system is mm.The coordinate of object point M is (xw,yw,zw), the coordinate of picture point m ' is (u ', v '), and u ' corresponds to show in Fig. 2 The u ' to anticipate out.
Object point M is transformed into the process of the picture point m ' under two dimensional image coordinate system, is represented by formula (1):
Sm '=A [R t] M (1)
In formula (1):
S is scale factor；
M is represented by M=[xw,yw,zw,1]T；
M ' is represented by m '=[u ', v ', 1]T；
The external parameter matrix of [R t] expression microlens array type optical field camera；
The inner parameter matrix of A expression microlens array type optical field camera.
The external parameter matrix [R t] of camera describes the transformational relation between world coordinate system and camera coordinates system, should Transformational relation is represented by formula (2) and formula (3):
In formula (2) and formula (3), q1、q2、q3Indicate three reference axis in camera coordinates system and world coordinate system conversion process Rotation angle；tx、ty、tzIndicate origin OwRespectively along the distance of three reference axis translation.
The inner parameter matrix A of camera is represented by formula (4), formula (4) describe camera coordinates system and image coordinate system it Between transformational relation:
In formula (4): dx × dy is the Pixel Dimensions of detector；u0、v0For seat of the center under image coordinate system of main lens Mark, θ are the out of plumb inclination angle of camera coordinates system and world coordinate system.
In one embodiment, in S2, setting: picture point m1It is (u in the coordinate of image coordinate system1,v1), it is corresponding micro- The center of lens is (p in the coordinate of image coordinate system1,q1), u1、p1Correspond to the u illustrated in Fig. 21、p1；Picture point m2Scheming As the coordinate of coordinate system is (u2,v2), corresponding lenticule center is (p in the coordinate of image coordinate system2,q2), u2、p2It is corresponding For the u illustrated in Fig. 22、p2.It can be obtained by geometrical relationship of the lenticule in Fig. 3 under pin-hole model, main lens, which focuses, to be formed Picture point m ' and picture point m on detector1Coordinate transformation relation be formula (5):
m1=Tm ' (5)
In formula (5):
m1=[u1,v1,1]TFor picture point m1Coordinate vector；
T is picture point m ' and picture point m1Between transition matrix, be specifically represented by following formula (6) and formula (7):
δ=b/a=(u1-u2)/(p1-p2)-1 (7)
Formula (1) is substituted into formula (5), can obtain and focus picture point m on object point M and detector1Coordinate transformation relation be formula (8):
Sm '=sT-1m1=A [R t] M (8)
In one embodiment, it in S3, is grouped into first with the picture point detected on the detector face, it will be same The corresponding detector picture point of object point is as one group, and every group of detector picture point is according to T-1Or geometrical relationship obtains focusing picpointed coordinate.
According to the geometrical relationship in the focal imaging index path of microlens array type can formula (9):
I.e. in formula (8)
Two picture points are chosen in the detector picture point of the same object point of correspondence, by picpointed coordinate and the corresponding lenticule of picture point Centre coordinate substitutes into formula (1) and calculates δ value, and then solves the coordinate (u ', v ') of the focusing picture point in formula (9).
In one embodiment, in S4, in order to solve the microlens array type optical field camera internal parameter matrix in formula (8) A and external parameter matrix [R t], the focusing picpointed coordinate obtained using S3 are used for reference Zhang Zhengyou calibration method and use iterative algorithm It solves, its specific method includes:
S41, if peg model plane z in world coordinate systemw=0, it substitutes into formula (1).According to object point M and corresponding focusing 3 × 3 homography matrix H can be obtained in picture point m ' coordinate, are the maximal possibility estimation of solution matrix H, poly- in iterative image Burnt picture point seeks lsqnonlin.
S42, according to r1With r2Constraint condition, the r that will be obtained by matrix H1With r2Expression formula substitute into, then by symmetrical square In the equation that battle array B and its calculating formula are brought into, 2 × 6 matrix Vs are finally obtained, substitute into the homography matrix of three width images hiCan solution matrix b, and then the unknown parameter in solution matrix A.
S43 has solved matrix A, is substituted into r1With r2Expression formula can solve r1、r2, and associate(d) matrix H can be asked Solve the unknown parameter in matrix [R t].
S44 demarcates n width image, and the focusing picture point of each image has m, the ginseng found out to above-mentioned method for solving Number iteration optimization, seeks lsqnonlin, carries out nonlinear optimization with the parameter found out to above-mentioned steps.
In one embodiment, " substituting into formula (1) " in S41 is specifically such as formula (11):
" 3 × 3 homography matrix H " in S41 is specifically such as formula (12):
H=[h1 h2 h3]=λ A [r1 r2 t] (12)
" the focusing picture point in iterative image seeks lsqnonlin " in S41 is specifically such as formula (13):
M ' in formula (13)iThe actual coordinate vector of picture point is focused for i-th in same piece image,It can With according to formula (11), formula (12) by object point MiIt is calculated.
In one embodiment, " the r in S421With r2Constraint condition " specifically such as formula (14):
" the r that will be obtained by H in S421With r2Expression formula substitute into " specifically such as formula (15):
r1=λ A-1h1,r2=λ A-1h2 (15)
In formula (15), λ=1/ | | A-1h1| |=1/ | | A-1h2||。
After formula (15) are substituted into formula (14), formula (16) can be obtained:
" in the equation for bringing symmetrical matrix B and its calculating formula into " in S42 is specifically such as formula (17):
It enables
B is symmetrical matrix, can be calculated formula (18):
In formula (18):
B=[B11,B12,B22,B13,B23,B33]T (19)
vij=[hi1hj1,hi1hj2+hi2hj1,hi2hj2,hi3hj1+hi1hj3,hi3hj2+hi2hj3,hi3hj3]T (20)
" obtaining 2 × 6 matrix Vs " in S42 is specifically that formula (17) and formula (18) are substituted into formula (16), obtains formula (21):
There are formula (22):
In formula (22), V is 2 × 6 matrixes, contains 6 unknown parameters.
" unknown parameter in solution matrix A " in S42 specifically includes:
V is substituted into the homography matrix h of three width imagesiCan solution matrix b, and then obtain:
γ=- B12α2β/λ (26)
u0=γ v0/β-B13α2/λ (27)
So that it is determined that the inner parameter matrix A of microlens array type optical field camera.
In one embodiment, " r is substituted into S431With r2Expression formula can solve r1、r2, and can be solved in conjunction with H Unknown parameter in matrix [R t] " specifically includes:
Matrix A substitution formula (15) can be solved into r1、r2:
r3=r1×r2 (29)
Formula (30) can be obtained by formula (12):
T=λ A-1h3 (30)
In one embodiment, " the non-linear least square problem " in S44 is specifically such as formula (31):
In formula (31):
m′ijFor the focusing picpointed coordinate of i-th j-th of angle point of width image；
According to formula (8) by j-th of object point coordinate vector M of the i-th width imagejAnd Ai, Ri, tiMeter It obtains；
The initial value of A and [R t] are calculated by S41 to S43.
In one embodiment, in S5, after the inside and outside parameter matrix for acquiring microlens array type optical field camera, due to parameter α, δ become known parameters, then the spacing b and main lens face for solving microlens array surface and detector face arrive microlens array surface Distance L is specific as follows:
By similar triangles relationship in Fig. 2 it is found that the diameter of main lens pupil diameter and lenticule image meets following formula (32):
D/L=d/b (32)
Wherein, D is main lens pupil diameter, and d is the diameter of lenticule image.Formula (33) can be obtained by formula (7):
A=b/ δ (33)
Formula (32) and formula (33) are substituted into α=(L-a)/(dx) in formula (4), formula (34) and formula (35) can be acquired:
L=a+ α dx (34)
B=α δ ddx/ (δ D-d) (35)
Finally it is noted that the above embodiments are merely illustrative of the technical solutions of the present invention, rather than its limitations.This The those of ordinary skill in field is it is understood that be possible to modify the technical solutions described in the foregoing embodiments or right Part of technical characteristic is equivalently replaced；These are modified or replaceed, and it does not separate the essence of the corresponding technical solution originally Invent the spirit and scope of each embodiment technical solution.

Claims (10)

1. a kind of based on the microlens array type optical field camera geometrical parameter calibration method for focusing picture point, which is characterized in that including Following steps:
S1 obtains object point and focuses picture point about main lens according to the focal imaging index path of microlens array type optical field camera Mapping relations；
S2, according to the focal imaging index path of microlens array type optical field camera, obtain focusing picture point and detector picture point about The mapping relations of lenticule；
S3 solves the coordinate for focusing picture point according to the detector picture point that detection obtains；
S4 solves camera internal parameter matrix and external parameter in peg model according to the coordinate of the S3 focusing picture point obtained Matrix；And
S5, the camera internal parameter matrix and external parameter matrix obtained by S4, calibration microlens array type optical field camera Geometric parameter, the geometric parameter include at a distance from the spacing and main lens to lenticule of microlens array and detector plane.
2. the microlens array type optical field camera geometrical parameter calibration method as described in claim 1 based on focusing picture point, It is characterized in that, " mapping relations of object point and focusing picture point about main lens " in S1 are expressed as formula (1), and set: object point indicates For M, coordinate is (xw,yw,zw)；It focuses picture point and is expressed as m ', coordinate is (u ', v '):
Sm '=A [R t] M, (1)
In formula (1), s is scale factor；M=[xw,yw,zw,1]T；M '=[u ', v ', 1]T；[R t] indicates microlens array type optical The external parameter matrix of field camera, A indicate the inner parameter matrix of microlens array type optical field camera；
The formula (3) and formula (2) that [R t] is expressed as:
In formula (2) and formula (3), q1、q2、q3Respectively indicate three reference axis in camera coordinates system and world coordinate system conversion process Rotation angle, tx、ty、tzIndicate the distance that the origin of world coordinate system is translated along three reference axis；
The formula (4) that A is expressed as:
In formula (4), dx × dy is the Pixel Dimensions of detector, u0、v0For coordinate of the main lens centre under image coordinate system, θ is The out of plumb inclination angle of two reference axis；
Wherein: camera coordinates system is using the center of main lens as origin O, xc、ycAxis is parallel to detector plane, zcAxis is perpendicular to spy Survey device plane；World coordinate system is to demarcate the center of object plane as origin Ow, xw、ywAxis is parallel to calibration object plane, zwAxis is vertical In calibration object plane；Image coordinate system is using detector plane center as origin Oc, u, v axis is parallel to xc、ycAxis.
3. the microlens array type optical field camera geometrical parameter calibration method as claimed in claim 2 based on focusing picture point, It is characterized in that, " the focusing picture point and mapping relations of the detector picture point about lenticule " in S2 is formula (8), setting: detector Picture point includes picture point m1With picture point m2, picture point m1It is (u in the coordinate of image coordinate system1,v1), the center of corresponding lenticule exists The coordinate of image coordinate system is (p1,q1)；Picture point m2It is (u in the coordinate of image coordinate system2,v2), corresponding lenticule center It is (p in the coordinate of image coordinate system2,q2)；
Sm '=sT-1m1=A [R t] M (8)
In formula (8): m1It is expressed as picture point m1Coordinate vector；T is picture point m ' and picture point m1Between transition matrix；
m1It is expressed as m1=[u1,v1,1]T；
The formula (6) and formula (7) that T is expressed as:
δ=b/a=(u1-u2)/(p1-p2)-1 (7)。
4. the microlens array type optical field camera geometrical parameter calibration method as claimed in claim 3 based on focusing picture point, It is characterized in that, S3 is specifically the detector picture point obtained according to detection, and utilizes the focal imaging index path of microlens array type In geometrical relationship, solve focus picture point coordinate；
" the detector picture point that detection obtains " is picture point m1、m2；
" geometrical relationship in the focal imaging index path of microlens array type " is expressed as formula (9):
Formula (10) then can be obtained according to formula (8):
Firstly, by picture point m1、m2Image coordinate system coordinate and corresponding lenticule center image coordinate system coordinate generation Enter formula (7), δ value is calculated；
Then, the δ value being calculated is substituted into formula (9), solves and focuses picture point m ' coordinate (u ', v ').
5. the microlens array type optical field camera geometrical parameter calibration method as described in claim 1 based on focusing picture point, It is characterized in that, S4 is specifically included:
S41, if peg model plane z in world coordinate systemw=0, it substitutes into formula (1), according to object point M and corresponding focusing picture point 3 × 3 homography matrix H can be obtained in m ' coordinate, are the maximal possibility estimation of solution matrix H, the focusing picture in iterative image Point seeks lsqnonlin；
S42, according to r1With r2Constraint condition, the r that will be obtained by matrix H1With r2Expression formula substitute into, then by symmetrical matrix B and In the equation that its calculating formula is brought into, 2 × 6 matrix Vs are finally obtained, substitute into the homography matrix h of three width imagesiIt can ask Dematrix b, and then the unknown parameter in solution matrix A；
S43 has solved matrix A, and matrix A is substituted into r1With r2Expression formula can solve r1、r2, and associate(d) matrix H can be solved Unknown parameter in matrix [R t] out；
S44 demarcates n width image, and the focusing picture point of each image has m, and the parameter found out to above-mentioned method for solving changes Generation optimization, seeks lsqnonlin, carries out nonlinear optimization with the parameter found out to above-mentioned steps.
6. the microlens array type optical field camera geometrical parameter calibration method as claimed in claim 5 based on focusing picture point, It is characterized in that, S41 is specifically included:
If peg model plane z in world coordinate systemw=0, (1) formula is substituted into, then there are formula (11):
3 × 3 homography matrix H can be obtained with corresponding focusing picture point m ' coordinate according to object point M, form is as follows:
H=[h1 h2 h3]=λ A [r1 r2 t] (12)
In formula (12), hi=[hi1,hi2,hi3]T, λ is any scalar；
For the maximal possibility estimation of solution matrix H, the focusing picture point in iterative image seeks lsqnonlin:
In formula (13), m 'iThe actual coordinate vector of picture point is focused for i-th in same piece image,According to formula (11), formula (12) is by object point MiIt is calculated.
7. the microlens array type optical field camera geometrical parameter calibration method as claimed in claim 6 based on focusing picture point, It is characterized in that, S42 is specifically included:
By the property of spin matrix it is found that r in matrix R1With r2Meet following two constraint condition:
From formula (12)
r1=λ A-1h1,r2=λ A-1h2 (15)
In formula (15), λ=1/ | | A-1H1| |=1/ | | A-1h2| |, formula (15) substitution formula (14) can be obtained:
It enables
B is symmetrical matrix, can be calculated formula (18):
In formula (18),
B=[B11,B12,B22,B13,B23,B33]T (19)
vij=[hi1hj1,hi1hj2+hi2hj1,hi2hj2,hi3hj1+hi1hj3,hi3hj2+hi2hj3,hi3hj3]T (20)
Formula (17) and formula (18), which are substituted into formula (16), can obtain formula (21):
There are formula (22):
In formula (22), V is 2 × 6 matrixes, contains 6 unknown parameters, substitutes into the homography matrix h of three width imagesiIt can solution matrix B, and then the unknown parameter in solution matrix A:
γ=- B12α2β/λ (26)
u0=γ v0/β-B13α2/λ (27)
So that it is determined that the inner parameter matrix A of microlens array type optical field camera.
8. the microlens array type optical field camera geometrical parameter calibration method as claimed in claim 7 based on focusing picture point, It is characterized in that, S43 is specifically included:
S42, which is solved matrix A substitution formula (15), can solve r1、r2, specific calculation expression is formula (29):
r3=r1×r2 (29)
Formula (30) can be obtained by formula (12):
T=λ A-1h3 (30)。
9. the microlens array type optical field camera geometrical parameter calibration method as claimed in claim 8 based on focusing picture point, It is characterized in that, S44 is specifically included:
N width image is demarcated, the focusing picture point of each image there are m, and the parameter found out to above-mentioned method for solving changes Generation optimization, seeks lsqnonlin:
In formula (31), m 'ijFor the focusing picpointed coordinate of i-th j-th of angle point of width image,According to formula (8) By j-th of object point coordinate vector M of the i-th width imagejAnd Ai, Ri, tiIt is calculated, the initial value of A and [R t] are by S41 to S43 It is calculated.
10. the microlens array type optical field camera geometrical parameter calibration method as described in claim 1 based on focusing picture point, It is characterized in that, S5 is specifically included:
The diameter d of main lens pupil diameter D and lenticule image meets following formula:
D/L=d/b (32)
Formula (33) can be obtained by formula (7):
A=b/ δ (33)
Formula (32) and formula (33) are substituted into α=(L-a)/(dx) in formula (4), formula (34) and formula (35) can be acquired:
L=a+ α dx (34)
B=α δ ddx/ (δ D-d) (35)
In formula (34) and formula (35), parameter alpha, δ become known parameters.
CN201811070094.4A 2018-09-13 2018-09-13 Geometric parameter calibration method for micro-lens array type optical field camera based on focusing image points Active CN109325981B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201811070094.4A CN109325981B (en) 2018-09-13 2018-09-13 Geometric parameter calibration method for micro-lens array type optical field camera based on focusing image points

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201811070094.4A CN109325981B (en) 2018-09-13 2018-09-13 Geometric parameter calibration method for micro-lens array type optical field camera based on focusing image points

Publications (2)

Publication Number Publication Date
CN109325981A true CN109325981A (en) 2019-02-12
CN109325981B CN109325981B (en) 2020-10-02

Family

ID=65265312

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201811070094.4A Active CN109325981B (en) 2018-09-13 2018-09-13 Geometric parameter calibration method for micro-lens array type optical field camera based on focusing image points

Country Status (1)

Country Link
CN (1) CN109325981B (en)

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110009693A (en) * 2019-04-01 2019-07-12 清华大学深圳研究生院 A kind of Fast Blind scaling method of light-field camera
CN111710001A (en) * 2020-05-26 2020-09-25 东南大学 Object image mapping relation calibration method and device under multi-medium condition
CN111862237A (en) * 2020-07-22 2020-10-30 复旦大学 Camera calibration method for optical surface shape measurement and device for implementing method
CN112927296A (en) * 2021-02-03 2021-06-08 上海橙捷健康科技有限公司 Method and system for calibrating and calibrating spatial relative position
CN113115017A (en) * 2021-03-05 2021-07-13 上海炬佑智能科技有限公司 3D imaging module parameter inspection method and 3D imaging device
CN115150607A (en) * 2022-06-21 2022-10-04 北京理工大学 Focusing type plenoptic camera parameter design method based on multi-focal-length micro lens array
CN116152357A (en) * 2023-04-04 2023-05-23 国科天成科技股份有限公司 Parameter calibration system and method for infinity focusing camera

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104764588A (en) * 2015-03-31 2015-07-08 中国科学院西安光学精密机械研究所 Single-pulse laser dynamic focal spot position measuring device and method
CN105488810A (en) * 2016-01-20 2016-04-13 东南大学 Focused light field camera internal and external parameter calibration method
GB2540922A (en) * 2015-01-25 2017-02-08 Vicente Blasco Claret Jorge Aberration corrected full resolution virtual image real world
CN107610182A (en) * 2017-09-22 2018-01-19 哈尔滨工业大学 A kind of scaling method at light-field camera microlens array center
CN108426585A (en) * 2018-03-12 2018-08-21 哈尔滨工业大学 A kind of geometric calibration method of light-field camera

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
GB2540922A (en) * 2015-01-25 2017-02-08 Vicente Blasco Claret Jorge Aberration corrected full resolution virtual image real world
CN104764588A (en) * 2015-03-31 2015-07-08 中国科学院西安光学精密机械研究所 Single-pulse laser dynamic focal spot position measuring device and method
CN105488810A (en) * 2016-01-20 2016-04-13 东南大学 Focused light field camera internal and external parameter calibration method
CN107610182A (en) * 2017-09-22 2018-01-19 哈尔滨工业大学 A kind of scaling method at light-field camera microlens array center
CN108426585A (en) * 2018-03-12 2018-08-21 哈尔滨工业大学 A kind of geometric calibration method of light-field camera

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
PIOTR SULIGA ,TOMASZ WRONA: "Microlens array calibration method for a light field camera", 《2018 19TH INTERNATIONAL CARPATHIAN CONTROL CONFERENCE (ICCC)》 *
张春萍，王庆: "光场相机成像模型及参数标定方法综述", 《中国激光》 *

Cited By (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110009693A (en) * 2019-04-01 2019-07-12 清华大学深圳研究生院 A kind of Fast Blind scaling method of light-field camera
CN110009693B (en) * 2019-04-01 2020-12-11 清华大学深圳研究生院 Rapid blind calibration method of light field camera
CN111710001A (en) * 2020-05-26 2020-09-25 东南大学 Object image mapping relation calibration method and device under multi-medium condition
CN111710001B (en) * 2020-05-26 2023-04-07 东南大学 Object image mapping relation calibration method and device under multi-medium condition
CN111862237A (en) * 2020-07-22 2020-10-30 复旦大学 Camera calibration method for optical surface shape measurement and device for implementing method
CN111862237B (en) * 2020-07-22 2024-01-05 复旦大学 Camera calibration method for optical surface shape measurement and device for realizing method
CN112927296A (en) * 2021-02-03 2021-06-08 上海橙捷健康科技有限公司 Method and system for calibrating and calibrating spatial relative position
CN113115017A (en) * 2021-03-05 2021-07-13 上海炬佑智能科技有限公司 3D imaging module parameter inspection method and 3D imaging device
CN113115017B (en) * 2021-03-05 2022-03-18 上海炬佑智能科技有限公司 3D imaging module parameter inspection method and 3D imaging device
CN115150607A (en) * 2022-06-21 2022-10-04 北京理工大学 Focusing type plenoptic camera parameter design method based on multi-focal-length micro lens array
CN116152357A (en) * 2023-04-04 2023-05-23 国科天成科技股份有限公司 Parameter calibration system and method for infinity focusing camera

Also Published As

Publication number Publication date
CN109325981B (en) 2020-10-02

Similar Documents

Publication Publication Date Title
CN109325981A (en) Based on the microlens array type optical field camera geometrical parameter calibration method for focusing picture point
CN105488810B (en) A kind of focusing light-field camera inside and outside parameter scaling method
EP3516625B1 (en) A device and method for obtaining distance information from views
TWI555379B (en) An image calibrating, composing and depth rebuilding method of a panoramic fish-eye camera and a system thereof
CN105678742B (en) A kind of underwater camera scaling method
US10217293B2 (en) Depth camera-based human-body model acquisition method and network virtual fitting system
CN108765328B (en) High-precision multi-feature plane template and distortion optimization and calibration method thereof
ES2402229T3 (en) Image fusion method and device
CN107025670A (en) A kind of telecentricity camera calibration method
CN110044300A (en) Amphibious 3D vision detection device and detection method based on laser
CN109903227A (en) Full-view image joining method based on camera geometry site
CN109859272A (en) A kind of auto-focusing binocular camera scaling method and device
CN110146030A (en) Side slope surface DEFORMATION MONITORING SYSTEM and method based on gridiron pattern notation
CN109712232B (en) Object surface contour three-dimensional imaging method based on light field
CN109855603A (en) A kind of focusing measurement method and terminal
CN110874854A (en) Large-distortion wide-angle camera binocular photogrammetry method based on small baseline condition
CN108279677A (en) Track machine people&#39;s detection method based on binocular vision sensor
CN113971691A (en) Underwater three-dimensional reconstruction method based on multi-view binocular structured light
CN108917633A (en) Fuel assembly deformation detection system based on underwater binocular vision
CN112164119B (en) Calibration method for multi-camera system placed in surrounding mode and suitable for narrow space
CN105241450A (en) Sky polarization mode detection method and system based on four-quadrant polaroid
CN110298890A (en) A kind of light-field camera scaling method based on Planck parametrization
CN110956668A (en) Focusing stack imaging system preset position calibration method based on focusing measure
CN109345595A (en) A kind of stereo visual sensor calibration method based on ball lens
CN110310337A (en) A kind of more view optical field imaging system population parameter estimation methods based on light field fundamental matrix

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant