CN108344360B - Laser scanning type global calibration device and method for vision measurement system - Google Patents
Laser scanning type global calibration device and method for vision measurement system Download PDFInfo
- Publication number
- CN108344360B CN108344360B CN201711126176.1A CN201711126176A CN108344360B CN 108344360 B CN108344360 B CN 108344360B CN 201711126176 A CN201711126176 A CN 201711126176A CN 108344360 B CN108344360 B CN 108344360B
- Authority
- CN
- China
- Prior art keywords
- camera
- coordinate system
- laser
- plane
- calibrated
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 72
- 238000005259 measurement Methods 0.000 title claims abstract description 32
- 239000013598 vector Substances 0.000 claims abstract description 103
- 239000011159 matrix material Substances 0.000 claims abstract description 60
- 238000013519 translation Methods 0.000 claims abstract description 53
- 230000000007 visual effect Effects 0.000 claims description 46
- 238000005457 optimization Methods 0.000 claims description 24
- 230000003287 optical effect Effects 0.000 claims description 17
- 238000006243 chemical reaction Methods 0.000 claims description 12
- 230000009466 transformation Effects 0.000 claims description 4
- 238000010586 diagram Methods 0.000 description 4
- 238000012546 transfer Methods 0.000 description 2
- 238000006073 displacement reaction Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 238000011089 mechanical engineering Methods 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G01B11/00—Measuring arrangements characterised by the use of optical techniques
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Length Measuring Devices By Optical Means (AREA)
Abstract
The invention discloses a laser scanning type global calibration device and a laser scanning type global calibration method for a vision measurement system, wherein the laser scanning type global calibration device comprises the following steps: the two line lasers are fixedly arranged on a multi-degree-of-freedom rotary table, the rotary table is controlled to rotate, a laser plane is projected into the field of view of the camera to be calibrated, a laser plane equation under the coordinate system of the camera is calibrated by means of a plane characteristic point target, the rotary table is rotated for multiple times to enable the laser plane to change the position in the field of view of the camera, and the laser plane is continuously calibrated. And rotating the rotary table to project the laser plane to the other camera field to be calibrated to calibrate the laser plane. Through the rotation data output by the turntable in real time and the laser plane equations of the calibrated laser plane under the camera coordinate system and the turntable coordinate system respectively, the position parameters between two cameras to be calibrated, namely a rotation matrix and a translation vector, are solved by taking the laser plane as a medium. The method can be suitable for global calibration of a large-visual-field multi-sensor vision measurement system, and is wide in application range and high in calibration precision.
Description
Technical Field
The invention relates to the technical field of vision measurement, in particular to a laser scanning type global calibration method and device of a vision measurement system.
Background
The three-dimensional vision measurement system has the advantages of non-contact, high speed, high measurement precision and the like, and is widely applied to the fields of industry and the like. In order to complete the large-space and large-size vision measurement task, the measurement space range of a single vision sensor is limited, and a plurality of vision sensors are required to work cooperatively to form a large-range multi-sensor distributed vision measurement system. In the multi-sensor distributed vision measurement system, the measurement data of each sensor is based on the relative measurement of local module sensors, and there is no common field of view, in order to unify the measurement data of each vision sensor, it is necessary to unify the measurement data of each vision sensor into a global world coordinate system, that is, to determine the position and direction, i.e., the rotation vector and the translation vector, of each vision sensor coordinate system relative to the global world coordinate system, i.e., to perform global calibration on all vision sensors. The position and direction relation is difficult to know through sensor installation, and must be obtained through global calibration in a measurement field through a specific means, and the global calibration precision determines the overall test precision of the multi-sensor vision measurement system.
At present, the common global calibration method of the multi-sensor vision measurement system without the common view field includes: the calibration method is based on direct measurement of control points by a three-dimensional measuring device. Lu in reference 1 and loming in reference 2 propose to establish a spatial three-dimensional coordinate measuring system by two theodolites, directly measure the three-dimensional coordinates of control points on the light plane, and realize the global calibration of the multi-sensor vision measuring system. Kitahara in [ reference 3] uses a laser tracker to perform global calibration. The three-dimensional measuring equipment is suitable for large view field due to high precision, is widely applied to the visual measurement of large-size components, but has low assembly efficiency and consumes labor cost. The other is a method based on a plane mirror, Kumar in [ Ref.4 ] and P.L Bebraly in [ Ref.5 ] which use plane mirrors to allow multiple vision sensors without a common field of view to view the same target, thereby achieving global calibration. This method is only suitable for global calibration between cameras at close range without common field of view, due to the influence of the view angle of the plane mirror and the depth of field of the camera. Thirdly, based on a self-calibration method, Esquirel in [ reference 6] and Roman in [ reference 7] let the vision sensor observe a target object with a specific structure in the field of view to carry out global calibration on the multi-vision sensor without a public field of view. The method based on self-calibration is difficult to obtain scene information meeting requirements in an industrial measurement field, and the global calibration precision is difficult to meet the requirements of visual measurement. And fourthly, based on a high-precision target method with characteristic points, Liu [ reference 8] realizes the global calibration of two visual sensors by using the fixed constraint relation of a biplane target. The method can only carry out global calibration on the multi-vision sensor in a calibration mode, and because the two plane characteristic point targets are fixedly connected, the plane targets are easy to be integrally adjusted and limited by the depth of field of the two cameras and can only swing at a small angle, so that images with unclear corner points are easy to generate, and the provided spatial position is restrained to be close to each other, thereby reducing the precision of the global calibration. Liu reference 9 performs global calibration using oblique laser line projection on two non-pinned planar feature point targets. The method has good flexibility, can calibrate the visual sensors at various field angles, but is not easy to operate on site when the overall calibration of the multi-visual sensor with complex layout is carried out.
The present invention references are as follows:
[1]R.S.Lu,Y.F.Li.A global calibration method for large-scale multi-sensor visual measurement system,Sensors and Actuators A:Physical.2004,116(3):383-393.
[2] the research and application of the multi-sensor machine vision measurement system, doctor academic thesis, Tianjin university 1996.
[3]Kitahara I,Saito H,Akimichi S,Onno T,Ohta Y,Kanade T.Large-scalevirtualized reality.IEEE computer vision and pattern recognition(CVPR).technical sketches,2001.
[4]R.K.Kumar,A.Ilie,J.Frahmet,et al.Simple calibration of non-overlapping cameras using a planar mirror:Application to vision-basedrobotics.In Proc.of CVPR.USA,2008.
[5]P.Lebraly,C.Deymier,et al.Flexible extrinsic calibration of non-overlapping cameras using a planar mirror:Application to vision-basedRobotics.IEEE.International Conference on Intelligent Robots and Systems,Taipei,Taiwan,2010.
[6]S.Esquivel,F.Woelk,R.Koch.Calibration of a multi-camera rig fromnon-overlapping views.Lecture Notes in Computer Science.2007,4713:82-91.
[7]P.Roman,B.Horst.Localization and trajectory reconstruction insurveillance cameras with nonoverlapping views.IEEE Transactions on PatternAnalysis and Machine Intelligence.2010,32(4):709-721.
[8]Z.Liu,G.J.Zhang,Z.Z.Wei,et al.A golbal calibration method formultiple vision sensors based on multiple targets.Measurement Science andTechnology,2011,22(12):125102.
[9]Q.Z.Liu,et al.Global calibration method of multi-sensor visionsystem using skew laser lines.Chinese Journal of Mechanical Engineering.2012,25(2):405-410.
Disclosure of Invention
In view of this, the present invention provides a laser scanning global calibration method and device for a vision measurement system, which are suitable for global calibration of a multi-vision sensor measurement system in a super-large working space and have a wide application range.
In order to achieve the purpose, the technical scheme of the invention is realized as follows:
the invention provides a laser scanning type global calibration device of a vision measurement system, which comprises an auxiliary camera, two line lasers, a multi-degree-of-freedom turntable and a plane characteristic point target; wherein:
the auxiliary camera is used for calibrating the laser scanning system, shooting clear light bar images projected onto the planar characteristic point target by the line lasers fixed on the multi-degree-of-freedom turntable, and resolving a plane equation of a laser plane projected by the two line lasers under an auxiliary camera coordinate system according to the images;
the line laser is fixedly arranged on the multi-degree-of-freedom rotary table, forms a laser scanning system together, can scan along with the rotary table in a rotating mode and projects a laser plane onto a planar characteristic point target;
the multi-degree-of-freedom turntable is used for bearing the two line lasers, forms a laser scanning system together with the two line lasers, and can output rotation state data in real time;
and the planar characteristic point target is used for forming light bars by projecting the line laser on the planar characteristic point target, and the characteristic points provide homography constraint for calculating a light plane equation under a camera coordinate system.
In the above scheme, the line lasers are fixedly mounted on the multi-degree-of-freedom turntable to form a laser scanning system together, and laser planes projected by the two line lasers are not coincident or parallel. The multi-degree-of-freedom rotary table can be two degrees of freedom or three degrees of freedom. The characteristic point pattern of the planar characteristic point target can be a checkerboard pattern, a grid pattern or a circular lattice pattern. The planar characteristic point target, the auxiliary camera, the two line lasers and the multi-degree-of-freedom rotary table jointly form a laser scanning type global calibration device.
The invention also provides a laser scanning type global calibration method of the vision measurement system, which comprises the following steps:
a. the calibration of the laser scanning system is characterized in that two line lasers are fixedly installed on a multi-degree-of-freedom rotary table to form the laser scanning system, a plane characteristic point target is used for solving an optical plane equation of the two laser planes under an auxiliary camera, a rotation matrix and a translation vector between an auxiliary camera coordinate system and a rotary table coordinate system are solved through a hand-eye calibration method, and after a Levenberg-Marquardt nonlinear method is used for optimizing the rotation matrix and the translation vector, the equation of the two laser planes under the auxiliary camera coordinate system is converted into the rotary table coordinate system.
b. The overall calibration of the multi-vision sensor is specifically that a laser scanning system projects two laser planes onto a planar characteristic point target in a visual field of a camera to be calibrated, an optical plane equation of the two laser planes under a coordinate system of the camera to be calibrated is solved, a rotary table is rotated to project the two laser planes onto a planar characteristic point target in the visual field of another auxiliary camera to be calibrated, an optical plane equation of the two laser planes under the coordinate system of the camera is solved, a conversion relation between the coordinate systems of the two cameras and the rotary table is established, a rotation matrix of the two cameras to be calibrated is solved by using a new vector of which a normal vector of the two laser planes is vertical to the normal vector, the rotary table is rotated at least twice in the visual field of the camera by the laser planes, translation vectors of the two cameras to be calibrated are solved, and after the rotation matrix and the translation vectors are optimized by using a Levenberg-Marquardt nonlinear method, and obtaining the optimized estimation of the rotation matrix and the translation vector between the two cameras.
The method comprises the following steps that a, two line lasers are fixedly installed on a multi-degree-of-freedom rotary table to form a laser scanning system, a plane characteristic point target is used for solving an optical plane equation of two laser planes under an auxiliary camera, a rotation matrix and a translation vector between an auxiliary camera coordinate system and a rotary table coordinate system are solved through a hand-eye calibration method, after a Levenberg-Marquardt nonlinear method is used for optimizing the rotation matrix and the translation vector, the equation of the two laser planes under the auxiliary camera coordinate system is converted into the rotary table coordinate system, and the method is implemented as follows:
(1) the two line lasers are installed and fixed on the multi-degree-of-freedom rotary table to form a scanning system, laser planes projected by the two line lasers are not coincident and not parallel, and are projected on a plane characteristic point target, so that the light bar brightness is uniform and is not dispersed, and the plane characteristic point target does not generate strong mirror reflection and diffuse reflection;
(2) the method comprises the steps of placing a planar feature point target in a clear visual field range of an auxiliary camera, shooting light strip images projected on the planar feature point target by two line lasers through the auxiliary camera, moving the planar feature point target in a small distance in the depth of field direction of the auxiliary camera, and calculating a laser plane equation of a laser plane in a coordinate system of the auxiliary camera through a structured light plane calibration method. Rotating the multi-degree-of-freedom rotary table for multiple times in a small amplitude manner to ensure that two laser planes displace in the visual field of the auxiliary camera, and continuously calibrating a light plane equation of the laser planes in the coordinate system of the auxiliary camera through the plane characteristic point target;
(3) forming a new vector by using normal vectors of two laser plane equations and a vertical normal vector and a rotation state parameter output by a multi-freedom-degree turntable in real time, establishing an AX (X-XB) matrix equation, and solving a position relation between an auxiliary camera coordinate system and a turntable coordinate system, namely a rotation matrix and a translation vector between the auxiliary camera coordinate system and the turntable coordinate system, through a plurality of groups of light plane equations of the two laser planes solved under the auxiliary camera coordinate system;
and (4) establishing an optimization objective function by taking the rotation matrix and the translation vector between the auxiliary camera coordinate system and the turntable coordinate system in the step (3) as initial values, and obtaining the optimization estimation of the rotation matrix and the translation vector by using a Levenberg-Marquardt nonlinear optimization method. And converting the laser plane equation of the two line laser planes in the camera coordinate system into the turntable coordinate system through the optimized result.
B, the laser scanning system projects two laser planes onto a planar characteristic point target in the visual field of the camera to be calibrated, an optical plane equation of the two laser planes under the coordinate system of the camera to be calibrated is solved, the rotary table is rotated to project the two laser planes onto a planar characteristic point target in the visual field of the other auxiliary camera to be calibrated, an optical plane equation of the two laser planes under the coordinate system of the camera is solved, a conversion relation between the coordinate systems of the two cameras and the coordinate system of the rotary table is established, a rotation matrix of the two cameras to be calibrated is solved by using a new vector of which the normal vector of the two laser planes is vertical to the normal vector, the rotary table is rotated at least twice in the visual field of the camera by the laser planes, translation vectors of the two cameras to be calibrated are solved, the rotation matrix and the translation vectors are optimized by using a Levenberg-Marquardt nonlinear method, and then the optimization estimation of the rotation matrix and the translation vectors between, the method comprises the following implementation steps:
(1) a laser scanning system consisting of two line lasers and a multi-degree-of-freedom rotary table is placed between two cameras to be calibrated without a common view field. Placing a plane characteristic point target in a clear visual field range of one camera to be calibrated, rotating a turntable to enable two laser planes to project on the plane characteristic point target, shooting a clear light bar image with characteristic points by the camera, moving the plane characteristic point target back and forth in the clear visual field range of the camera to enable the two laser planes to be always positioned in the visual field of the camera and to be imaged clearly, and solving a laser plane equation under a coordinate system of the camera by a structured light plane calibration method;
(2) placing a plane characteristic point target in front of another camera to be calibrated, rotating a rotary table to enable two laser planes to project on the plane characteristic point target, shooting a clear light bar image with characteristic points by the camera, moving the plane characteristic point target back and forth within a clear visual field range of the camera to enable the two laser planes to be always positioned in the visual field of the camera and to be imaged clearly, and solving a laser plane equation under a coordinate system of the camera by a structured light plane calibration method;
(3) establishing a conversion relation from the two camera coordinate systems to be calibrated to the rotary table coordinate system through a plane equation of the two laser planes under the two camera coordinate systems to be calibrated respectively and a plane equation of the two laser planes under the rotary table coordinate system obtained through solution;
(4) the spatial normal vectors of the two laser plane equations can be known through the plane equations of the two laser planes respectively under the coordinate systems of the two cameras to be calibrated, new vectors perpendicular to the two normal vectors are obtained through cross multiplication of the two normal vectors, and the rotation matrix between the two cameras to be calibrated is solved by utilizing the relationship of the new vectors between the coordinate systems of the two cameras to be calibrated and the coordinate system of the rotary table;
(5) rotating the multi-degree-of-freedom rotary table to enable the two laser planes to scan at least twice in the visual fields of the two cameras to be calibrated respectively, and calculating a translation vector between the two cameras to be calibrated according to the conversion relation between the coordinate systems of the two cameras to be calibrated and the coordinate system of the rotary table;
(6) and (5) taking the solved rotation matrix and translation vector in the steps (3) and (5) as initial values, establishing an optimization objective function, and obtaining the optimization estimation of the rotation matrix and the translation vector by using a Levenberg-Marquardt nonlinear optimization method.
Compared with the prior art, the invention has the advantages that:
the invention provides a laser scanning type global calibration method and device of a vision measurement system, which are used for realizing the global calibration of large-visual-field multi-vision sensing. Two line lasers are fixedly arranged on a multi-freedom-degree rotary table, the rotary table can rotate in three degrees of freedom in space, and laser planes projected by the lasers can be scanned in space in a three-dimensional mode. And controlling the rotary table to rotate, projecting the laser plane into the visual field of one of the cameras to be calibrated, and calibrating the light plane equation of the laser plane in the camera coordinate system by means of the plane characteristic point target. And rotating the turntable for multiple times in a small amplitude to change the position of the structured light in the visual field of the camera, and continuously calibrating the laser plane. The turret is then rotated so that the laser plane is projected into the field of view of another camera to be calibrated. Through the rotation data output by the turntable in real time and the laser plane equations of the calibrated laser plane under the camera coordinate system and the turntable coordinate system respectively, the position parameters between two cameras to be calibrated, namely a rotation matrix and a translation vector, are solved by taking the laser plane as a medium. The method can be suitable for global calibration of a large-visual-field multi-sensor vision measurement system, and is wide in application range and high in global calibration precision.
Drawings
FIG. 1 is a schematic diagram of laser scanning global calibration;
FIG. 2 is a schematic view of laser plane calibration;
fig. 3 is a schematic diagram of laser scanning system calibration.
Detailed Description
The invention is further described with reference to the following figures and detailed description.
The laser scanning global calibration principle is shown in fig. 1, and there is no common field of view between two sets of cameras to be calibrated. A laser scanning system consisting of a laser and a rotary table is placed between two cameras to be calibrated, and the laser scanning system projects a laser plane onto a target in the visual field of the cameras and clearly images.
The invention mainly comprises the following steps:
step 1: and (5) calibrating the laser scanning system. The two line lasers are fixedly installed on a multi-degree-of-freedom rotary table to form a laser scanning system, a plane characteristic point target is used for solving an optical plane equation of the two laser planes under an auxiliary camera, a rotation matrix and a translation vector between an auxiliary camera coordinate system and the rotary table coordinate system are solved through a hand-eye calibration method, and after the rotation matrix and the translation vector are optimized through a Levenberg-Marquardt nonlinear method, the equation of the two laser planes under the auxiliary camera coordinate system is converted into the rotary table coordinate system.
Step 11: the two line lasers are installed and fixed on the multi-degree-of-freedom rotary table to form a scanning system, laser planes projected by the two line lasers are not coincident and not parallel, and are projected on a plane characteristic point target, so that the light bar brightness is uniform and is not dispersed, and the plane characteristic point target does not generate strong mirror reflection and diffuse reflection;
step 12: the method comprises the steps of placing a plane characteristic point target in a clear visual field range of an auxiliary camera, shooting light strip images projected on the plane characteristic point target by two line lasers through the auxiliary camera, utilizing the small-distance movement of the plane characteristic point target in the depth of field direction of the auxiliary camera, and calculating a laser plane equation of a laser plane in a coordinate system of the auxiliary camera through a structured light plane calibration method, wherein the schematic diagram of the laser plane calibration method is shown in fig. 2. And the multi-degree-of-freedom rotary table is rotated for multiple times in a small amplitude, so that the light plane equation of the laser plane in the auxiliary camera coordinate system is continuously calibrated through the plane characteristic point targets in the visual field of the two laser plane displacement auxiliary cameras.
Step 121: the two cameras to be calibrated and the auxiliary camera are respectively calibrated to solve the internal parameters K of the cameras, namely:
here αx＝f/dx,αy＝f/dy，αx、αyScale factors, or effective focal length, d, being the u-axis and v-axis respectivelyx、dyPixel spacing u in horizontal and vertical directions, respectively0、v0Is the optical center and gamma is the non-normal factor of the u and v axes.
For the actual pixel coordinates, (u, v) for the ideal pixel coordinates, k1，k2Is a radial distortion parameter. The specific solving method is in the article "A flexible new technique for camera calibration [ R ] of Zhang Zhengyou]Microsoft corporation, NSR-TR-98-71,1998 ", is described in detail.
Step 122: and (5) solving a laser plane equation.
Setting a camera coordinate system as O-XYZ, an internal reference matrix of the camera as K, and a plane equation of a light plane pi under the camera coordinate system as follows:
aX+bY+cZ+d＝0 (2)
the specific solving method is described in detail in the Zhou-kuan article "Complete simulation of structured lighting vision sensor through plan target of unknown orientations, Image and vision computing.2005" and the military article "Universal method for structured lighting vision sensor on the spot, Mechanical Engineering, 2009".
Step 13: the method comprises the steps of forming a new vector by using normal vectors of two laser plane equations and a vertical normal vector, and rotating state parameters output by a multi-freedom-degree turntable in real time, establishing an AX (X-XB) matrix equation, and solving the position relation between an auxiliary camera coordinate system and a turntable coordinate system, namely a rotating matrix and a translation vector between the auxiliary camera coordinate system and the turntable coordinate system, through a plurality of groups of light plane equations of the two laser planes solved under the auxiliary camera coordinate system.
Step 131: the calibration schematic diagram of the laser scanning system is shown in fig. 2, and two line lasers are fixed on a high-precision turntable to form the scanning system. And rotating the turntable to project the laser plane onto the plane target, and shooting the plane target and the projected light bar image by the auxiliary camera.
The light plane before and after each rotation of the turntable is recorded as
Where the subscript i denotes the laser number (i ═ 1, 2). N represents a unit normal vector of the plane, and the homogeneous coordinate of the plane pi is recorded as
Where d is the 4 th dimension of vector Π. The convention uses subscripts c and p to denote variables relating to the camera coordinate system and the turret coordinate system, respectively.
In the camera coordinate system, the planes before and after rotation are as follows:
wherein,
Rc、Tcis the rotation matrix and translation vector by which the turntable rotates so that the plane changes in the camera coordinate system.
Under the turntable coordinate system, the planes before and after rotation are as follows:
wherein,
Rpis a rotation matrix of the rotation of the turntable under the turntable coordinate system.
Before and after the rotary table rotates, the two laser planes from the camera coordinate system to the rotary table coordinate system have the following relations:
wherein,
Rpc、Tpcis the rotation matrix and translation vector from the camera coordinate system to the turret coordinate system.
Step 132: a rotation matrix between the camera coordinate system and the turret coordinate system is calculated.
From equation (3), we can obtain the relationship between the normal vectors of the planes under the camera coordinate system:
n′ci＝Rcnci(6)
since there are two light planes (i ═ 1,2) and the two laser planes are neither coincident nor parallel, their normal vectors are not parallel in any state. The two normal vectors are cross-multiplied to obtain a new vector that is perpendicular to both vectors. So RcThe solution is as follows:
Rc＝[n′c1n′c2n′c3][nc1nc2nc3]-1(7) wherein,
the following relationship exists between the rotation matrixes before and after the rotation of the turntable according to the formula (3), the formula (4) and the formula (5):
RpRpc＝RpcRc(8)
wherein the rotation matrix RcFrom equation (6), the rotation matrix R can be solvedpCan be obtained by the turntable system outputting data. Equation (8) is a typical AX ═ XB type equation, and a plurality of equations such as equation (8) can be obtained by rotating the turntable a plurality of times, and R is obtained by simultaneous solutionpc. The specific solution is described in "Calibration of the weighted cylindrical semiconductors by Solving the Equation of the Equation AX XB, IEEETrans.
Step 133: a translation vector between the camera coordinate system and the turret coordinate system is calculated.
Expanding equation (5) yields:
from the formula (3), d can be obtainedci＝d′ciIn conjunction with equation (9) we can obtain information about TpcThe equation of (c):
[(Rpcnci)T-(Rpcn′ci)T]Tpc＝dci-d′ci(10)
where i is 1,2, one rotation of the turntable may provide 2 constraint equations. T ispcHaving 3 degrees of freedom, RpcHaving solved so that the turret rotates at least twice, the translation vector R between the two coordinate systems can be solved from equation (10)pc。
Step 14: and establishing an optimization objective function by taking a rotation matrix and a translation vector between an auxiliary camera coordinate system and a turntable coordinate system as initial values, and obtaining the optimization estimation of the rotation matrix and the translation vector by using a Levenberg-Marquardt nonlinear optimization method. And converting the laser plane equation of the two line laser planes in the camera coordinate system into the turntable coordinate system through the optimized result.
Step 141: optimizing R to solve abovepcAnd TpcAs an initial valueOptimizing line parameters, and establishing an optimized objective function:
wherein x ═ { R ═ Rpc,TpcI denotes the number of the laser, j, k denotes the number of the turntable rotation, n denotes the number of turntable rotations, nciIs the expression of the laser plane in the camera coordinate system,
is a transformation matrix of the turntable,
is a transformation matrix of the camera coordinate system and the turntable coordinate system. In order to ensure the orthogonality of the rotation vectors, the rotation vector R in the variable x in formula (11)pcExpressed in its gaussian vector form. The LM optimization method is adopted to carry out optimization solution on the formula (11), and R is finally obtainedpc,TpcTo estimate the optimum.
Step 2: global calibration of a multi-vision sensor. The laser scanning system projects two laser planes onto a plane characteristic point target in the visual field of a camera to be calibrated, an optical plane equation of the two laser planes under a coordinate system of the camera to be calibrated is solved, a rotary table is rotated to enable the two laser planes to project onto a plane characteristic point target in the visual field of another auxiliary camera to be calibrated, an optical plane equation of the two laser planes under the coordinate system of the camera is solved, a conversion relation between the coordinate systems of the two cameras and the coordinate system of the rotary table is established, a rotation matrix of the two cameras to be calibrated is solved by using a new vector of which a normal vector of the two laser planes is perpendicular to the normal vector of the two laser planes, the rotary table is rotated at least twice in the visual field of the camera, translation vectors of the two cameras to be calibrated are solved, and after the rotation matrix and the translation vectors are optimized by using a Levenberg-Marquardt nonlinear method, the optimized estimation of the rotation matrix and the.
The global calibration is illustrated with two cameras. As shown in fig. 3, the scanning system consisting of the laser and the turntable is placed between the two cameras to be calibrated. Projecting a light plane into the visual field of a left camera by rotation, then performing light plane solution by means of a plane target, then rotating a rotary table, projecting the light plane into the visual field of a right camera, and performing light plane solution by means of the plane target; the position relation between the two cameras, namely a rotation matrix and a translation vector, is calculated by taking the light plane as an intermediate.
Step 21: a laser scanning system consisting of two line lasers and a multi-degree-of-freedom rotary table is placed between two cameras to be calibrated without a common view field. The method comprises the steps of placing a plane characteristic point target in a clear view range of one camera to be calibrated, rotating a rotary table to enable two laser planes to project on the plane characteristic point target, shooting a clear light bar image with characteristic points by the camera, moving the plane characteristic point target back and forth in the clear view range of the camera to enable the two laser planes to be always located in the view range of the camera and to be imaged clearly, and solving a laser plane equation under a coordinate system of the camera by a structured light plane calibration method.
Step 22: the method comprises the steps of placing a plane characteristic point target in front of another camera to be calibrated, rotating a rotary table to enable two laser planes to project on the plane characteristic point target, shooting a clear light bar image with characteristic points by the camera, moving the plane characteristic point target back and forth within a clear visual field range of the camera to enable the two laser planes to be always located within the visual field of the camera and to be imaged clearly, and solving a laser plane equation under a coordinate system of the camera by a structured light plane calibration method.
Step 23: and establishing a conversion relation from the two camera coordinate systems to be calibrated to the turntable coordinate system respectively through a plane equation of the two laser planes under the two camera coordinate systems to be calibrated respectively and a plane equation of the two laser planes under the turntable coordinate system obtained by solving.
We denote by subscripts cl and cr the variables associated with the a and B camera coordinate systems, respectively. Plane pi in the field of view of the camera A under the coordinate system of the turntableplrAnd a plane pi positioned in the visual field of the B camera after rotationpriThe relationship between them is as follows:
Πpri＝HprlΠpli(12)
wherein
Under the coordinate system of the turntable, a plane pipliII on the planepriThe transfer matrix of (2).
Plane pili、ΠriFrom the a camera coordinate system and the B camera coordinate system to the turret coordinate system, respectively, the following transformation relationships exist:
wherein
And
the transfer matrices of the planes in the field of view of the a, B cameras from the respective camera coordinate systems to the turret coordinate system, respectively.
Step 24: the spatial normal vectors of the two laser plane equations can be known through the plane equations of the two laser planes respectively under the coordinate systems of the two cameras to be calibrated, new vectors perpendicular to the two normal vectors are obtained through cross multiplication of the two normal vectors, and the rotation matrix between the two cameras to be calibrated is solved by utilizing the relationship of the new vectors between the coordinate systems of the two cameras to be calibrated and the coordinate system of the rotary table.
From equation (13), the relationship between the expression of the plane normal vector in the two camera coordinate systems and the two turntable coordinate systems is:
where i is 1,2, this allows the solution of RpclAnd Rpcl. Then the rotation matrix between the two cameras can be easily solved:
step 25: and rotating the multi-freedom-degree rotary table to enable the two laser planes to scan at least twice in the visual fields of the two cameras to be calibrated respectively, and calculating a translation vector between the two cameras to be calibrated according to the conversion relation between the coordinate systems of the two cameras to be calibrated and the coordinate system of the rotary table.
By working up the expansion equation (13), the equation associated with the translation vector can be obtained:
the laser plane scans for at least two times in the visual field of the A camera and the B camera respectively, so that the translation vectors T from the coordinate systems of the two cameras to the coordinate system of the turntable can be solvedpclAnd Tpcr。
The homographic conversion relationship from the B camera coordinate system to the A camera coordinate system is as follows:
and because of
Therefore:
step 26: and (5) taking the solved rotation matrix and translation vector in the steps (3) and (5) as initial values, establishing an optimization objective function, and obtaining the optimization estimation of the rotation matrix and the translation vector by using a Levenberg-Marquardt nonlinear optimization method.
R solved abovelrAnd TlrPerforming parameter optimization as an initial value, and establishing an optimization objective function:
wherein x ═ { R ═ Rlr,TlrI denotes the number of the laser, j denotes the number of the turn table rotation, n denotes the number of turn table rotations,/li,ΠriIs the expression of the laser plane in two camera coordinate systems,
is the conversion parameter from the B camera coordinate system to the A camera coordinate system.
Also, the rotation parameter R in the argument of equation (19)lrExpressing by the Rodrigues vector form, optimizing the formula by an LM optimization method to finally obtain Rlr,TlrAnd optimizing and estimating.
Examples
Based on the laser scanning type global calibration device, the present invention is further described in detail by taking two cameras to be calibrated, an auxiliary camera, a turntable, two line lasers and a planar target as an example in combination with a specific embodiment.
The model of one auxiliary camera and two cameras to be calibrated is Allied Vision Technologies, the auxiliary camera is matched with a Schneifer optical lens with the focal length of 23mm, and the image resolution is 1600 multiplied by 1200 pixels. Two cameras to be calibrated are matched with a Schneifer optical lens with a focal length of 17mm, the image resolution is 1600 multiplied by 1200 pixels, the visual field of the visual sensor is about 400mm multiplied by 300mm, and the measurement distance is 1000 mm. The laser is a single-line red laser with the wavelength of 635nm and the power of 50 mw. And the processing precision of the plane target is 0.02 mm.
The internal parameters of the auxiliary camera obtained according to the method described in step 121 are shown in table 1.
TABLE 1 Global System Camera Advantage
According to the method in step 122, the turntable is rotated 10 times, the auxiliary camera captures the light bar target image for calibrating two laser planes after each rotation, and the rotation parameters of the turntable are recorded at the same time, as shown in table 2.
TABLE 2 turntable rotation parameters
According to the method in the step 13, laser plane equations in all rotation states are calibrated, and then the steps are adopted, wherein the rotation matrix and the translation vector from the camera coordinate system to the turntable coordinate system are solved as follows:
according to the method in step 14, the laser light plane in the monocular camera coordinate system is converted into the turntable coordinate system, and in combination with the rotation state of the turntable, the light plane equation of the laser light plane in the turntable coordinate system in the zero position state of the turntable can be obtained:
according to the method in the step 2, the turntable is rotated to enable the laser planes to be projected to 5 different positions in the viewing fields of the A, B two cameras respectively, the rotation parameters of the turntable are recorded, the light planes are calibrated for the different positions, light plane equations of each group of light planes under the coordinate system of the camera A and the coordinate system of the camera B can be obtained, then the rotation and translation parameters between the two cameras A, B are solved, and the results are as follows:
the art related to the present invention is not described in detail.
Claims (3)
1. A laser scanning global calibration method for a vision measurement system, the method comprising:
a. the calibration of the laser scanning system is specifically that two line lasers are fixedly installed on a multi-degree-of-freedom rotary table to form the laser scanning system, a plane characteristic point target is used for solving an optical plane equation of the two laser planes under an auxiliary camera, a rotation matrix and a translation vector between an auxiliary camera coordinate system and a rotary table coordinate system are solved through a hand-eye calibration method, and after a Levenberg-Marquardt nonlinear method is used for optimizing the rotation matrix and the translation vector, the equation of the two laser planes under the auxiliary camera coordinate system is converted into the rotary table coordinate system;
b. the overall calibration of the multi-vision sensor is specifically that a laser scanning system projects two laser planes onto a planar characteristic point target in a visual field of a camera to be calibrated, an optical plane equation of the two laser planes under a coordinate system of the camera to be calibrated is solved, a rotary table is rotated to project the two laser planes onto a planar characteristic point target in the visual field of another auxiliary camera to be calibrated, an optical plane equation of the two laser planes under the coordinate system of the camera is solved, a conversion relation between the coordinate systems of the two cameras and the rotary table is established, a rotation matrix of the two cameras to be calibrated is solved by using a new vector of which a normal vector of the two laser planes is vertical to the normal vector, the rotary table is rotated at least twice in the visual field of the camera by the laser planes, translation vectors of the two cameras to be calibrated are solved, and after the rotation matrix and the translation vectors are optimized by using a Levenberg-Marquardt nonlinear method, and obtaining the optimized estimation of the rotation matrix and the translation vector between the two cameras.
2. The laser scanning global calibration method of the vision measurement system as claimed in claim 1, wherein in the step a, two line lasers are fixedly installed on a multi-degree-of-freedom turntable, the light plane equations of two laser planes under the auxiliary camera are solved by using the planar feature point targets, the rotation matrix and the translation vector between the coordinate system of the auxiliary camera and the coordinate system of the turntable are solved by the hand-eye calibration method, and the equations of the two laser planes under the coordinate system of the auxiliary camera are converted into the coordinate system of the turntable after optimization, and the implementation steps are as follows:
(a1) the two line lasers are installed and fixed on the multi-degree-of-freedom rotary table to form a scanning system, laser planes projected by the two line lasers are not coincident and not parallel, and are projected on a plane characteristic point target, so that the light bar brightness is uniform and is not dispersed, and the plane characteristic point target does not generate strong mirror reflection and diffuse reflection;
(a2) placing a planar feature point target in a clear visual field range of an auxiliary camera, shooting light strip images projected on the planar feature point target by two line lasers through the auxiliary camera, moving the planar feature point target in a small distance in the depth of field direction of the auxiliary camera, and calculating a laser plane equation of a laser plane in a coordinate system of the auxiliary camera through a structured light plane calibration method; rotating the multi-degree-of-freedom rotary table for multiple times in a small amplitude manner to ensure that two laser planes displace in the visual field of the auxiliary camera, and continuously calibrating a light plane equation of the laser planes in the coordinate system of the auxiliary camera through the plane characteristic point target;
(a3) forming a new vector by using normal vectors of two laser plane equations and a vertical normal vector and a rotation state parameter output by a multi-freedom-degree turntable in real time, establishing an AX (X-XB) matrix equation, and solving a position relation between an auxiliary camera coordinate system and a turntable coordinate system, namely a rotation matrix and a translation vector between the auxiliary camera coordinate system and the turntable coordinate system, through a plurality of groups of light plane equations of the two laser planes solved under the auxiliary camera coordinate system;
(a4) establishing an optimization objective function by taking a rotation matrix and a translation vector between the auxiliary camera coordinate system and the turntable coordinate system in the step (a3) as initial values, and obtaining the optimization estimation of the rotation matrix and the translation vector by using a Levenberg-Marquardt nonlinear optimization method; and converting the laser plane equation of the two line laser planes in the camera coordinate system into the turntable coordinate system through the optimized result.
3. The laser scanning global calibration method of vision measuring system as claimed in claim 1, wherein in step b, the laser scanning system projects two laser planes onto the planar feature point target in the field of view of the camera to be calibrated, solves the optical plane equations of the two laser planes under the coordinate system of the camera to be calibrated, rotates the turntable to project the two laser planes onto the planar feature point target in the field of view of the other auxiliary camera to be calibrated, solves the optical plane equations of the two laser planes under the coordinate system of the camera, establishes the transformation relationship between the coordinate systems of the two cameras and the coordinate system of the turntable, solves the rotation matrix of the two cameras to be calibrated by using the new vector perpendicular to the normal vector of the two laser planes, rotates the turntable at least twice in the field of view of the cameras respectively, and solves the translation vectors of the two cameras to be calibrated, optimizing a rotation matrix and a translation vector by using a Levenberg-Marquardt nonlinear method to obtain the optimized estimation of the rotation matrix and the translation vector between the two cameras; the method comprises the following implementation steps:
(b1) placing a laser scanning system consisting of two line lasers and a multi-degree-of-freedom rotary table between two cameras to be calibrated without a common view field; placing a plane characteristic point target in a clear visual field range of one camera to be calibrated, rotating a turntable to enable two laser planes to project on the plane characteristic point target, shooting a clear light bar image with characteristic points by the camera, moving the plane characteristic point target back and forth in the clear visual field range of the camera to enable the two laser planes to be always positioned in the visual field of the camera and to be imaged clearly, and solving a laser plane equation under a coordinate system of the camera by a structured light plane calibration method;
(b2) placing a plane characteristic point target in front of another camera to be calibrated, rotating a rotary table to enable two laser planes to project on the plane characteristic point target, shooting a clear light bar image with characteristic points by the camera, moving the plane characteristic point target back and forth within a clear visual field range of the camera to enable the two laser planes to be always positioned in the visual field of the camera and to be imaged clearly, and solving a laser plane equation under a coordinate system of the camera by a structured light plane calibration method;
(b3) establishing a conversion relation from the two camera coordinate systems to be calibrated to the rotary table coordinate system through a plane equation of the two laser planes under the two camera coordinate systems to be calibrated respectively and a plane equation of the two laser planes under the rotary table coordinate system obtained through solution;
(b4) the spatial normal vectors of the two laser plane equations can be known through the plane equations of the two laser planes respectively under the coordinate systems of the two cameras to be calibrated, new vectors perpendicular to the two normal vectors are obtained through cross multiplication of the two normal vectors, and the rotation matrix between the two cameras to be calibrated is solved by utilizing the relationship of the new vectors between the coordinate systems of the two cameras to be calibrated and the coordinate system of the rotary table;
(b5) rotating the multi-degree-of-freedom rotary table to enable the two laser planes to scan at least twice in the visual fields of the two cameras to be calibrated respectively, and calculating a translation vector between the two cameras to be calibrated according to the conversion relation between the coordinate systems of the two cameras to be calibrated and the coordinate system of the rotary table;
(b6) and (c) establishing an optimization objective function by taking the solved rotation matrix and translation vector in the step (b3) and the step (b5) as initial values, and obtaining the optimized estimation of the rotation matrix and the translation vector by using a Levenberg-Marquardt nonlinear optimization method.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711126176.1A CN108344360B (en) | 2017-11-15 | 2017-11-15 | Laser scanning type global calibration device and method for vision measurement system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711126176.1A CN108344360B (en) | 2017-11-15 | 2017-11-15 | Laser scanning type global calibration device and method for vision measurement system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108344360A CN108344360A (en) | 2018-07-31 |
| CN108344360B true CN108344360B (en) | 2020-03-31 |
Family
ID=62962467
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711126176.1A Active CN108344360B (en) | 2017-11-15 | 2017-11-15 | Laser scanning type global calibration device and method for vision measurement system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108344360B (en) |
Families Citing this family (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109297436B (en) * | 2018-11-30 | 2021-11-23 | 北京伟景智能科技有限公司 | Binocular line laser stereo measurement reference calibration method |
| CN109559355B (en) * | 2018-12-04 | 2021-08-10 | 北京航空航天大学 | Multi-camera global calibration device and method without public view field based on camera set |
| CN109934830B (en) * | 2019-03-18 | 2020-10-30 | 杭州电子科技大学 | Real-time object rotation angle measuring device and method based on image calculation |
| CN110285831B (en) * | 2019-07-05 | 2020-12-29 | 浙江大学城市学院 | Calibration method for grid structure light projector |
| CN112304214B (en) * | 2019-08-01 | 2022-10-11 | 上海飞机制造有限公司 | Tool detection method and tool detection system based on photogrammetry |
| CN110370286B (en) * | 2019-08-13 | 2022-04-12 | 西北工业大学 | Method for identifying rigid body space position of dead axle motion based on industrial robot and monocular camera |
| CN110440692B (en) * | 2019-08-27 | 2020-10-20 | 大连理工大学 | Combined measurement and calibration method of laser tracker and structured light 3D scanner |
| CN111174698B (en) * | 2019-12-31 | 2024-08-23 | 吉林大学 | Automobile detection co-view-field-free camera global calibration system and method based on surface light field |
| CN111664807B (en) * | 2020-04-24 | 2022-08-12 | 河北津西钢铁集团股份有限公司 | Rotary structure optical method pipeline detection system |
| CN112379440A (en) * | 2020-11-02 | 2021-02-19 | 中国石油天然气集团有限公司 | Method and device for identifying solid filling characteristics of fracture and hole body |
| CN112578327B (en) * | 2020-12-01 | 2023-09-12 | 深圳市通用测试系统有限公司 | Calibration method, device and storage medium of spherical scanning test system |
| CN113177987B (en) * | 2021-04-30 | 2022-03-25 | 北京航空航天大学 | Outfield global calibration method and system of visual tracking measurement system |
| CN113280790B (en) * | 2021-05-07 | 2023-08-15 | 王亚超 | Building mapping device based on laser positioning |
| CN113771096A (en) * | 2021-11-09 | 2021-12-10 | 北京壹点灵动科技有限公司 | Method and device for processing pose information of mechanical arm |
| CN114279362A (en) * | 2021-12-28 | 2022-04-05 | 中国航天空气动力技术研究院 | Dynamic shape measuring device and method for heat-proof structure |
| CN114440792A (en) * | 2022-01-11 | 2022-05-06 | 重庆固高科技长江研究院有限公司 | Closed layout structure of multi-line laser sensing, scanning splicing and gluing scanning method |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20080096056A (en) * | 2007-04-26 | 2008-10-30 | 주식회사 엔케이스틸 | Inspection apparatus and method for dectecting surface defects of metal plate using vision system and laser depth profiler |
| CN101551918A (en) * | 2009-04-28 | 2009-10-07 | 浙江大学 | Acquisition method of large scene based on line laser |
| CN105698699A (en) * | 2016-01-26 | 2016-06-22 | 大连理工大学 | A binocular visual sense measurement method based on time rotating shaft constraint |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150042758A1 (en) * | 2013-08-09 | 2015-02-12 | Makerbot Industries, Llc | Laser scanning systems and methods |
-
2017
- 2017-11-15 CN CN201711126176.1A patent/CN108344360B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20080096056A (en) * | 2007-04-26 | 2008-10-30 | 주식회사 엔케이스틸 | Inspection apparatus and method for dectecting surface defects of metal plate using vision system and laser depth profiler |
| CN101551918A (en) * | 2009-04-28 | 2009-10-07 | 浙江大学 | Acquisition method of large scene based on line laser |
| CN105698699A (en) * | 2016-01-26 | 2016-06-22 | 大连理工大学 | A binocular visual sense measurement method based on time rotating shaft constraint |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108344360A (en) | 2018-07-31 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108344360B (en) | Laser scanning type global calibration device and method for vision measurement system | |
| CN109559355B (en) | Multi-camera global calibration device and method without public view field based on camera set | |
| CN111325801B (en) | Combined calibration method for laser radar and camera | |
| CN111801198B (en) | Hand-eye calibration method, system and computer storage medium | |
| CN104835117B (en) | Spherical panorama generation method based on overlapping mode | |
| CN104616292B (en) | Monocular vision measuring method based on global homography matrix | |
| Cai et al. | An effective method for camera calibration in defocus scene with circular gratings | |
| Song et al. | Survey on camera calibration technique | |
| CN101975588B (en) | Global calibration method and device of rigid rod of multisensor vision measurement system | |
| CN111811395B (en) | Monocular vision-based dynamic plane pose measurement method | |
| CN106408556B (en) | A kind of small items measuring system scaling method based on general imaging model | |
| CN110728715A (en) | Camera angle self-adaptive adjusting method of intelligent inspection robot | |
| Liu et al. | Novel calibration method for non-overlapping multiple vision sensors based on 1D target | |
| CN104729422B (en) | Method for calibrating a laser measuring device and system therefor | |
| CN109297436B (en) | Binocular line laser stereo measurement reference calibration method | |
| Liu et al. | External parameter calibration of widely distributed vision sensors with non-overlapping fields of view | |
| Xu et al. | An omnidirectional 3D sensor with line laser scanning | |
| CN111415391A (en) | Multi-view camera external orientation parameter calibration method adopting inter-shooting method | |
| CN104697463B (en) | The blanking feature constraint scaling method and device of a kind of binocular vision sensor | |
| CN111811433B (en) | Structured light system calibration method and device based on red and blue orthogonal stripes and application | |
| CN113205603A (en) | Three-dimensional point cloud splicing reconstruction method based on rotating platform | |
| CN113330487A (en) | Parameter calibration method and device | |
| CN111854636A (en) | Multi-camera array three-dimensional detection system and method | |
| CN113724337A (en) | Camera dynamic external parameter calibration method and device without depending on holder angle | |
| Sergiyenko et al. | Multi-view 3D data fusion and patching to reduce Shannon entropy in Robotic Vision |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |