CN109345595A - A kind of stereo visual sensor calibration method based on ball lens - Google Patents
A kind of stereo visual sensor calibration method based on ball lens Download PDFInfo
- Publication number
- CN109345595A CN109345595A CN201811071566.8A CN201811071566A CN109345595A CN 109345595 A CN109345595 A CN 109345595A CN 201811071566 A CN201811071566 A CN 201811071566A CN 109345595 A CN109345595 A CN 109345595A
- Authority
- CN
- China
- Prior art keywords
- lens
- ball lens
- camera
- ball
- structural parameters
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
- G06T7/85—Stereo camera calibration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
- G06T2207/10021—Stereoscopic video; Stereoscopic image sequence
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Length Measuring Devices By Optical Means (AREA)
Abstract
The invention discloses a kind of stereo visual sensor calibration methods based on ball lens, comprising: puts gridiron pattern target repeatedly in place, the target imaging that shooting passes through more pieces of lens refractions；Extract the X-comers of each lens refraction in image；The calibration of structural parameters initial value is carried out using analytic method；The optimal solution of structural parameters is obtained by nonlinear optimization.The present invention is big compared to perspective projection viewing field of camera, and a camera is used only, three-dimensional reconstruction can be realized, the large view field measurement being suitble under cramped conditions.
Description
Technical field
The present invention relates to transducer calibration technologies and vision measurement technology, and in particular to a kind of solid based on ball lens
Vision sensor calibration method.
Background technique
Stereo vision sensor based on catadioptric element have it is compact-sized, be easy to build with the advantages such as low in cost,
And synchronous error is not present, cause the extensive concern of scholar in recent years.Stereo vision sensor based on ball lens is by one
Platform video camera combines two at most ball lens compositions, has larger field, is suitble to complete in narrow space in big visual field
Measurement.
The calibration of stereo vision sensor based on ball lens includes the calibration of camera and structure ginseng of video camera
Number calibration two parts.There are many document about Calibration of camera intrinsic parameters, therefore discuss structural parameters calibration method.It closes
In the model structure parameter scaling method mainly include following two.One is the virtual camera method of the propositions such as Kah, by that will be
System is considered as more " virtual cameras ", demarcates to the structural parameters between " virtual camera ", to realize the mark to the model
It is fixed；Another kind is " analytic method ", i.e., by establishing the analytic modell analytical model of refractive light paths, the analytic solutions of structural parameters is obtained, thus complete
At the calibration of structural parameters.Agrawal et al. is made that outstanding contributions in this field, by establishing in ball lens meridian plane
Coordinate, using incident ray, object point and optical axis are coplanar and different lens are that refraction is carried out to same target as constraint, complete
About optical axis direction, length and world coordinate system to the calibration of camera coordinate system spin matrix and translation vector.
But existing main scaling method based on projected array stereo vision sensor or between lens and camera
The relative position requirement put is more stringent, such as requires camera parallel with lens, is difficult to realize in actual operation；Or it needs
Manual intervention, such as hand labeled ball lens contour, calibration process are complicated；Or it is precision deficiency, it is especially saturating in ball
Since aberration is larger, depth of field deficiency causes image quality difference to cause feature point extraction inaccurate for mirror edge, leads to stated accuracy
It is poor, it completes to measure using calibration result.
Summary of the invention
In view of this, the main purpose of the present invention is to provide a kind of stereo visual sensor calibrations based on ball lens
Method can be realized the high-precision calibrating of the stereo vision sensor, and complete to accurately measure on this basis.
In order to achieve the above objectives, the technical solution adopted by the present invention is that:
A kind of stereo visual sensor calibration method based on ball lens, this method comprises:
A, the video camera in the stereo vision sensor based on ball is demarcated, by gridiron pattern target in suitable position
It sets and puts repeatedly, at least once, two lens be imaged can to it, the stereo vision sensor shooting based on ball lens passes through
The target imaging of more pieces of lens refractions；Extract the X-comers of each lens refraction in image；
B, the calibration of structural parameters initial value is carried out using analytic method；
C, the optimal solution of structural parameters is obtained by nonlinear optimization.
Steps are as follows for the realization for the target imaging that shooting is reflected by more pieces of lens in step a:
(11) ball lens array is made of two ball lens；
(12) the distance between ball lens and the distance between ball lens array and video camera are adjusted, make two it is saturating
Mirror is all imaged in field range, and it is clear to focus.
Steps are as follows for the realization for the X-comers that each lens reflect in extraction image in step a:
(21) by multiple dimensioned Robust Algorithm of Image Corner Extraction, angle point initial value is obtained；
(22) by pattern distortion antidote, the image characteristic point coordinate of no camera lens distortion is obtained.
Using the realization of the calibration of analytic method progress structural parameters, steps are as follows in step b:
(31) structural parameters demarcated include: the side that each ball lens centre of sphere and camera perspective projection center constitute axis
To Ai(i=1,2), each ball lens centre of sphere and camera perspective projection center distance di(i=1,2), world coordinate system arrives
The spin matrix R and translation vector t of camera coordinate system；
(32) according to constraint and analytic equation solve A respectively between meridian plane constraint and lens altogetheri, [R t] and di。
The optimal solution for obtaining structural parameters in step c by nonlinear optimization, using trust region reflective
Method carries out nonlinear optimization as objective function away from error minimum using re-projection error minimum and reconstruction point, solves Ai, [R
T], diOptimal solution.
The advantages of the present invention over the prior art are that:
(1) present invention obtains accurate target characteristic point coordinate by uncertainty method, with re-projection error and rebuilds
Point, collectively as objective function, carries out nonlinear optimization using multiple target postures away from error, obtains vertical based on ball lens
Body vision transducer calibration method, precision is high, and re-projection error reaches 0.1 Pixel-level.The present invention makes the stereo vision sensor
The stated accuracy of model reaches the degree that can be used for rebuilding and measure, and has practical value it more.
(2) the invention proposes the stereo vision sensor reconstruction model based on ball lens, which is applied non-
In linear optimization, keep calibration result accurate in actual measurement.The method is suitable for the sensing of the stereoscopic vision based on ball lens
Device high-precision calibrating and measurement.
Detailed description of the invention
Fig. 1 is that the present invention is based on the stereo visual sensor calibration method flow diagrams of ball lens；
Fig. 2 is that the stereoscopic vision based on ball lens senses peg model；
Fig. 3 is coplanar constraint present on meridian plane in refracting process；
Fig. 4 is the stereo vision sensor measurement model based on ball lens.
Specific embodiment
The basic idea of the invention is that: image characteristic point coordinate is obtained using uncertainty method；It is obtained using analytic method
The initial value of structural parameters；Using re-projection error and reconstruction error as objective function, realized using plurality of pictures saturating based on ball
The calibration of the stereo vision sensor of mirror；It is intersected in a bit according to each lens emergent ray of same point, realizes the stereoscopic vision
The reconstruction and measurement of sensor.
It is below with reference to the stereo vision sensor based on ball array that a video camera and two ball lens form
Example, invention is further described in detail.
As shown in Figure 1, the present invention is based on the stereo visual sensor calibration methods of ball lens, it mainly include following step
It is rapid:
Step 11: the measurement process of the stereo vision sensor based on ball lens.
Here, the measurement process of the stereo vision sensor is explained.An object point P is through two ball lens in space
On the image plane at p after refraction1, p2Two picture points.After extracting image characteristic point, is removed and distorted using distortion formula, it can be with
Obtain orthoscopic image characteristic point coordinate p1=[u1, v1]T, p2=[u2, v2]T, according to video camera perspective relation:
Its coordinate p under camera coordinate system can be calculated separatelyc,1=[xc,1,yc,1,zc,1]T, pc,2=[xc,2,
yc,2,zc,2]T, according to Agrawal et al. in " Single Image Calibration of Multi-Axial Imaging
The refractive light paths analytic modell analytical model mentioned in a System " text can calculate separately eye point and emergent ray on each lens
Direction.
Step 12: building the stereo vision sensor system based on ball lens.
Here, two ball lens and video camera are formed into the stereo vision sensor based on ball lens, so that camera shooting
Machine can completely take two ball lens, and it is clear to focus.Camera lens are focused to ball lens, are led to
It crosses ball lens and can observe object and be clearly imaged.
Step 13: the video camera in the stereo vision sensor based on ball lens is demarcated.
Here, the inner parameter for solving video camera, specific method for solving are demarcated to the video camera of visual sensor
In article " A flexible new technique for camera calibration [R] .Microsoft of Zhang Zhengyou
It is had a detailed description in Corporation, NSR-TR-98-71,1998 ".
Step 14: a gridiron pattern target being put at least 1 time before stereo vision sensor, makes two lens can be right
It is imaged.Stereo vision sensor based on ball lens shoots the target image in two lens.
Here, Fig. 2 is the stereo vision sensor schematic diagram based on ball lens.Wherein PijIndicate i-th of posture of target
J-th point, pm,1ijIndicate the 1st ball lens to target characteristic point imaging, pm,2ijIndicate the 2nd ball lens to target
Mark characteristic point imaging, p1ijAnd p2ijIt respectively indicates on the plane of delineation through the 1st and the 2nd ball lens refraction imaging point.
World coordinate system O-XY, world coordinate system and camera coordinate system o are established in target facec-xcyczcBetween spin matrix and
Translation vector is respectively R and t.From camera perspective projection center, to the axis direction Unit Vector of the 1st ball lens centre of sphere
Amount is A1, the axis direction unit vector to the 2nd ball lens centre of sphere is A2.1st ball lens are arrived with the 2nd ball lens
The distance at camera perspective projection center is referred to as the wheelbase d of two axis1And d2。
Step 15: extracting X-comers coordinate in two lens.
Here, in order to overcome the problems, such as that rims of the lens point image quality is lower, point methods is mentioned using multiple dimensioned, use m
The Harris angle point grid template of different scale parameter, extracts to obtain m coordinate, obtains coordinate set Q=to a characteristic point P
{p1,p2,…pm, take its average valueAs the accurate coordinates value for calibration.
Step 16: demarcating to obtain the initial value of parameter using analytic method using revised coordinate.
Here, basic skills is according to Agrawal in article " Single Image Calibration of Multi-
The method proposed in Axial Imaging System ", when for using multiple postures to demarcate, makes certain improvement,
Specifically includes the following steps:
Step 161: if being demarcated using a posture, as shown in figure 3, for each lens, on its meridian plane
There are coplanar constraints: (RP (j)+t)T(Ai× v (i, j))=0, wherein for any placement position, if P (j) is target
J-th of characteristic point, R and t are respectively the spin matrix peace from the world coordinate system being connected on target to camera coordinate system
Move vector, AiFor i-th of ball lens, v (i, j) refers to light of j-th of characteristic point after i-th of lens reflects.For convenience
It calculates, in meridian plane using camera perspective projection center COP as origin, optical axis is that a reference axis establishes coordinate system, according to light
Road is reversible, and COP is considered as to the sending point of all light, then v (i, j) is the incidence reflected to j-th of refraction point through i-th of lens
Light.The corresponding meridian plane of different characteristic point intersects at optical axis, can demarcate to obtain the direction of axis using at least eight characteristic point；
When being demarcated using multiple postures, axis direction demarcates to obtain initial value using any posture.
Step 162: a picture can be only used using the light combination coplanar constraint from different lens and complete outside
The calibration of parameter；When being demarcated using multiple postures, the external ginseng of each posture relative camera coordinate system is demarcated respectively
Number.
Step 163: after obtaining above-mentioned parameter, characteristic point being transformed under camera coordinate system from world coordinate system, then
It projects on meridian plane, the available one 12 rank equations about wheelbase d, solve equation removal imaginary root and utilizes geometry about
The available last solution of beam.When using multiple postures, each posture calculates separately wheelbase, takes its median as the first of wheelbase
Value.
Step 17: obtaining the optimal solution of stereo vision sensor structural parameters by nonlinear optimization, complete calibration.
Here, gridiron pattern target is put repeatedly, using maximum likelihood criterion to stereoscopic vision sensor structure parameter into
Row optimization, obtains optimal solution of the structural parameters under maximum-likelihood criterion.
Using re-projection error minimum and reconstruction point away from error minimum as objective function, using trust region
Reflective method carries out nonlinear optimization.
If j-th of characteristic point is after first ball lens and second ball lens refraction in image in i-th of posture
Planar imaging point is p1ijAnd p2ij, re-projection formula is shown in " the Analytical Forward Projection For such as Agrawal
Axial Non-Central Dioptric And Catadioptric Cameras ", if the object point P under camera coordinate systemij
Two re-projection points be respectivelyWithM picture is used altogether, and every N number of characteristic point of picture, then re-projection error can be with
It indicates are as follows:
Wherein, Dist (A, B) indicates the distance between A, B two o'clock.
Such as stereo vision sensor measurement model based on ball lens as shown in figure 4, for any dimensional target point P,
The refraction point of nearly object point side is respectively m2lAnd m2r, emergent ray is respectively v2lAnd v2r, then the intersection point of two emergent lights is that object point exists
Coordinate in space.It is located at m under camera coordinate system2l=[mlx,mly,mlz]T, m2r=[mrx,mry,mrz]T, v2l=[vlx,vly,
vlz]T, v2r=[vrx,vry,vrz]T, then have:
k1、k2It is scale factor,It is the reconstruction object point coordinate under video camera system.Formula (3) can rewrite are as follows:
k1·v2l-k2·v2r=m2r-m2l(4)
Expand into matrix form:
It enablesFormula (5) is readily modified as Ak=b.Take the least square of k
Solve k=(ATA)-1(ATB) as the optimal solution of kThen respectively with the reconstruction point coordinate of the calculating of two lens
Are as follows:Take the two average value as reconstruction coordinate
For j-th of characteristic point P of i-th of target posture in calibration processij, rebuilding coordinate isNonlinear optimization
When, take put between the adjacent target punctuate of transverse direction and longitudinal direction away from and difference of the actual point away from D it is minimum, and in this, as reconstruction error target
Function.If N number of characteristic point of each posture is divided into m row n column, i-th of target posture r row c column characteristic point is Pi,rcThen should
Partial objective function can indicate are as follows:
frec(a)=frec1(a)+frec2(a)(8)
Catalogue scalar functions are as follows:
F (a)=min (frep(a)+frec(a))(9)
Wherein a is parameter to be optimized, a=(c1,c2,r1,r2,...rM,t1,t2,…,tM), using nonlinear optimization method,
Optimal solution of a under maximum likelihood criterion can be solved.c1=A1×d1, c2=A2×d2, two ball lens are described to taking the photograph
The structural parameters demarcated needed for the direction vector at camera perspective projection center, as stereo vision sensor system.
Claims (5)
1. a kind of stereo visual sensor calibration method based on ball lens, it is characterised in that: this method comprises:
A, the video camera in the stereo vision sensor based on ball is demarcated, gridiron pattern target is put in place
It puts repeatedly, at least once, two lens be imaged can to it, the stereo vision sensor shooting based on ball lens passes through more pieces
The target imaging of lens refraction；Extract the X-comers of each lens refraction in image；
B, the calibration of structural parameters initial value is carried out using analytic method；
C, the optimal solution of structural parameters is obtained by nonlinear optimization.
2. a kind of stereo visual sensor calibration method based on ball lens according to claim 1, it is characterised in that:
Steps are as follows for the realization for the target imaging that shooting is reflected by more pieces of lens in step a:
(11) ball lens array is made of two ball lens；
(12) the distance between ball lens and the distance between ball lens array and video camera are adjusted, keeps two lens complete
Portion is imaged in field range, and it is clear to focus.
3. a kind of stereo visual sensor calibration method based on ball lens according to claim 1, it is characterised in that:
Steps are as follows for the realization for the X-comers that each lens reflect in extraction image in step a:
(21) by multiple dimensioned Robust Algorithm of Image Corner Extraction, angle point initial value is obtained；
(22) by pattern distortion antidote, the image characteristic point coordinate of no camera lens distortion is obtained.
4. a kind of stereo visual sensor calibration method based on ball lens according to claim 1, it is characterised in that:
Using the realization of the calibration of analytic method progress structural parameters, steps are as follows in step b:
(31) structural parameters demarcated include: the direction A that each ball lens centre of sphere and camera perspective projection center constitute axisi
(i=1,2), each ball lens centre of sphere and camera perspective projection center distance di(i=1,2), world coordinate system is to taking the photograph
The spin matrix R and translation vector t of camera coordinate system；
(32) according to constraint and analytic equation solve A respectively between meridian plane constraint and lens altogetheri, [R t] and di。
5. a kind of stereo visual sensor calibration method based on ball lens according to claim 1, it is characterised in that:
The optimal solution for obtaining structural parameters in step c by nonlinear optimization, using Trust region reflective method, with
Re-projection error minimum and reconstruction point carry out nonlinear optimization as objective function away from error minimum, solve Ai, [R t], diMost
Excellent solution.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811071566.8A CN109345595B (en) | 2018-09-14 | 2018-09-14 | Stereoscopic vision sensor calibration method based on spherical lens |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811071566.8A CN109345595B (en) | 2018-09-14 | 2018-09-14 | Stereoscopic vision sensor calibration method based on spherical lens |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109345595A true CN109345595A (en) | 2019-02-15 |
| CN109345595B CN109345595B (en) | 2022-02-11 |
Family
ID=65305108
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811071566.8A Active CN109345595B (en) | 2018-09-14 | 2018-09-14 | Stereoscopic vision sensor calibration method based on spherical lens |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109345595B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111482963A (en) * | 2020-04-08 | 2020-08-04 | 江西理工大学 | Calibration method of robot |
| CN117351091A (en) * | 2023-09-14 | 2024-01-05 | 成都飞机工业（集团）有限责任公司 | Camera array calibration device and use method thereof |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101118648A (en) * | 2007-05-22 | 2008-02-06 | 南京大学 | Road conditions video camera marking method under traffic monitoring surroundings |
| CN101526338A (en) * | 2009-04-15 | 2009-09-09 | 北京信息科技大学 | Field calibration method of structural parameter of line structure light vision sensing system |
| CN106127745A (en) * | 2016-06-17 | 2016-11-16 | 凌云光技术集团有限责任公司 | The combined calibrating method of structure light 3 D visual system and line-scan digital camera and device |
-
2018
- 2018-09-14 CN CN201811071566.8A patent/CN109345595B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101118648A (en) * | 2007-05-22 | 2008-02-06 | 南京大学 | Road conditions video camera marking method under traffic monitoring surroundings |
| CN101526338A (en) * | 2009-04-15 | 2009-09-09 | 北京信息科技大学 | Field calibration method of structural parameter of line structure light vision sensing system |
| CN106127745A (en) * | 2016-06-17 | 2016-11-16 | 凌云光技术集团有限责任公司 | The combined calibrating method of structure light 3 D visual system and line-scan digital camera and device |
Non-Patent Citations (5)
| Title |
|---|
| AMIT AGRAWAL 等: "Single image calibration of multi-axial imaging systems", 《26TH IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION》 * |
| ZHENXING WANG 等: "A two-step calibration method of a large FOV binocular stereovision sensor for onsite measurement", 《MEASUREMENT》 * |
| 刘震 等: "立体视觉传感器的一种灵活靶标新标定方法", 《光学学报》 * |
| 陈京 等: "多视点标定图像的交替迭代度量重建方法", 《北京交通大学学报》 * |
| 黄邦奎 等: "多传感器线结构光视觉测量系统全局校准", 《光电子·激光》 * |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111482963A (en) * | 2020-04-08 | 2020-08-04 | 江西理工大学 | Calibration method of robot |
| CN111482963B (en) * | 2020-04-08 | 2022-11-25 | 江西理工大学 | Calibration method of robot |
| CN117351091A (en) * | 2023-09-14 | 2024-01-05 | 成都飞机工业（集团）有限责任公司 | Camera array calibration device and use method thereof |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109345595B (en) | 2022-02-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106127745B (en) | The combined calibrating method and device of structure light 3 D vision system and line-scan digital camera | |
| CN110044300B (en) | Amphibious three-dimensional vision detection device and detection method based on laser | |
| CN102509261B (en) | Distortion correction method for fisheye lens | |
| CN107025670A (en) | A kind of telecentricity camera calibration method | |
| JP4147059B2 (en) | Calibration data measuring device, measuring method and measuring program, computer-readable recording medium, and image data processing device | |
| CN101577002B (en) | Calibration method of fish-eye lens imaging system applied to target detection | |
| CN105698699A (en) | A binocular visual sense measurement method based on time rotating shaft constraint | |
| CN111583411A (en) | Three-dimensional model building method based on oblique photography | |
| CN109615663A (en) | Panoramic video bearing calibration and terminal | |
| CN105759435B (en) | Overall view ring belt stereo imaging system and its scaling method and imaging method | |
| CN109325981B (en) | Geometric parameter calibration method for micro-lens array type optical field camera based on focusing image points | |
| CN112985293B (en) | Binocular vision measurement system and measurement method for single-camera double-spherical mirror image | |
| CN105258710A (en) | High-precision camera principal point calibration method | |
| CN102243432A (en) | Panoramic three-dimensional photographing device | |
| CN102903101B (en) | Method for carrying out water-surface data acquisition and reconstruction by using multiple cameras | |
| CN110006634B (en) | Viewing field angle measuring method, viewing field angle measuring device, display method and display equipment | |
| CN109855603A (en) | A kind of focusing measurement method and terminal | |
| CN112489109B (en) | Three-dimensional imaging system method and device and three-dimensional imaging system | |
| CA3233222A1 (en) | Method, apparatus and device for photogrammetry, and storage medium | |
| CN108010125A (en) | True scale three-dimensional reconstruction system and method based on line-structured light and image information | |
| CN109345595A (en) | A kind of stereo visual sensor calibration method based on ball lens | |
| CN116147534A (en) | Multi-view three-dimensional laser scanning system based on mirror surface assistance and complex surface panoramic measurement method | |
| CN109186942A (en) | The test parallelism detection method, apparatus and readable storage medium storing program for executing of structure light video camera head | |
| CN115359127A (en) | Polarization camera array calibration method suitable for multilayer medium environment | |
| JP2006220603A (en) | Imaging apparatus |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |