Classifications

• • G—PHYSICS
  • G01—MEASURING; TESTING
  • G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
  • G01B11/00—Measuring arrangements characterised by the use of optical techniques
  • G01B11/002—Measuring arrangements characterised by the use of optical techniques for measuring two or more coordinates
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T7/00—Image analysis
  • G06T7/70—Determining position or orientation of objects or cameras
  • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T7/00—Image analysis
  • G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T2207/00—Indexing scheme for image analysis or image enhancement
  • G06T2207/30—Subject of image; Context of image processing
  • G06T2207/30244—Camera pose

Definitions

• the invention relates to a method for calibrating a measuring system based on at least one camera for determining the position of an object in a three-dimensional reference coordinate system, in which the outer and inner camera parameters are calibrated in different steps and the position of the camera is determined by an external measuring means.
• manipulators for example robots
• Optical measuring systems are often used for this purpose, which record images of the objects in the processing space of the manipulators and evaluate them by means of image processing in order to determine their orientation in the space from features of the recorded objects.
• a prerequisite for the functioning of such optical measuring systems is a calibration of the optical recording systems or cameras in a geometric camera model, which is the basis of the evaluation of the images.
• both so-called “internal” camera parameters which relate to the lens or lens properties of the camera and the relative arrangement of the lens and image sensor, for example CCD or CMOS sensor, as well as so-called “external” camera parameters to determine which geometric position, position and orientation of the camera in space.
• a second set of calibration methods attempts to exploit physical and geometric constraints to divide the parameters of the camera model into separate groups and determine them in separate, sequential steps. As a result of this reduction in the parameters to be determined in one step, the computational outlay is considerably reduced in comparison with the iterative search in the complete parameter space, although an equally high level of accuracy can be achieved.
• Such a method is described, for example, in the article Roger Y. Tsai, "A Versatile Camera Calibration Technique for High-Accuracy 3D Machine Vision Metrology Using Off-the-Shelf TV Cameras and Lenses", Vol. RA-3, No. 4, August 1987, in which geometric conditions for the movement of the camera are used in order to be able to determine camera parameters separately with simpler equation systems.
• a further disadvantage is that the accuracy required for measurements with pixel or subpixel accuracy for high distances from the camera to the object to be measured can not be achieved.
• the tool of, for example, 400 mm with measuring marks at the outer points and the determination of a measuring mark with the accuracy of 0.3 mm can be calculated from the beam set that at a distance of the camera to the measured object of 2000 mm, an error of 3 mm is formed, which is often insufficient for applications.
• the underlying this consideration structure is shown in Fig. 3.
• the object of the invention is therefore to provide a calibration method that is easy to handle with high accuracy and can be optimally inserted into automated production processes.
• the method proceeds in at least three method steps, wherein in a first method step the internal camera parameters of a specific camera are determined and permanently assigned to this camera, in a second method step the position of the camera mounted in the measuring system is determined and in a third method step the orientation the camera is determined in the three-dimensional reference coordinate system by evaluating colorbildem.
• the division into the aforementioned method steps is chosen such that the method steps can be easily implemented in a simple manner and with common technology in the industrial environment.
• the internal parameters of a camera type are determined separately and permanently assigned to this camera type. Because the determination of these internal camera parameters are independent of the later installation location of the camera, it is sufficient to determine these parameters once for each camera type, for example in a special measurement laboratory and to presume it known for further calibration methods with the same camera type.
• the position of the camera mounted in the measuring system is determined in particular with external measuring means. This measurement is completely independent of the determination of the internal camera parameters and therefore independent of any errors that have occurred.
• the inner camera parameters relating to the imaging properties of the camera as well as the spatial position of the camera in the world or reference coordinate system have therefore been determined in mutually independent measurements in completely independent measuring methods. In this case, it is possible to determine the absolute position of the camera in the reference coordinate system by means of simple standard measuring systems, for example laser tracers, which are often permanently installed without any problems for the purpose of measuring on robot-guided systems.
• the determination of the orientation of the camera in the room which causes a considerable measurement effort in comparison to the position determination when using external measuring means, is determined according to the invention by evaluating camera images using the camera parameters determined in the first two method steps.
• the image evaluation to be provided for this purpose is comparatively simple since only two measuring points known in space are to be evaluated in the camera image of the one camera. In contrast to calibration methods, in which all external and internal camera parameters are obtained from the pictures themselves, this means one significant performance increase without a loss of accuracy associated therewith, because the image evaluation in the third process step, the externally determined position data of the camera and the internal camera parameters are related to each other and used for the common calibration.
• a measuring system according to the invention is to be frequently set up at already existing workstations, it is important that the setup and calibration of the measuring system can take place in a time-optimized manner so that long downtimes, for example, of an assembly or production line do not occur.
• the first method step for determining the internal camera parameters takes place before assembly of the camera in the measuring system, for example under laboratory conditions using a measuring method known per se, in particular by introducing a highly accurate calibration body into a camera image.
• a calibration performed for a camera type is universally valid for this type of camera, so the first step of the procedure need not be repeated every time a measurement system with the same camera type is started. As a result, the profitability of the method according to the invention is further increased.
• the position of the camera can be easily determined by measuring a single excellent camera point in the form of a measuring mark, which is in known relation to the camera. This has the advantage that with the external measuring means for determining the camera position, only a single measuring point must be targeted, recorded and evaluated, whereby the determination of the position in space with common measuring means and little time is possible.
• an excellent feature in the camera body for example.
• a corner, edge or other particular punctiform camera point as Serve the trade mark.
• a lid with the measuring mark can be easily inserted or screwed into the objective receptacle of the camera and measured with an external measuring means, for example a laser tracker or a theodolite. Since the construction of the camera between lens shots and image sensor is known, a simple reference to the inner camera parameters determined in the first method step, which relate in particular to the lens-lens properties and the geometry of the lens-image sensor arrangement, can be produced. Possible position errors are compensated by the orientation calculated from the image.
• Position means the definition of the coordinates X, Y and Z of the world coordinate system.
• the orientation of the camera, i. in particular the orientation of the lens is not yet known after the second process step.
• the orientation of the camera can be determined in a further, in particular the third, method step by detecting two or more features in a camera image, the position of the features in the world or reference coordinate system being known.
• the three orientations RX k, Ry ⁇ ⁇ , Rz k set up mathematical at least two features, and imaged via the camera whose coordinates are in the image matrix in the form of two-dimensional image coordinate b and Y b requires X.
• the evaluation of these features in the camera image can take place automatically or graphically interactively by means of an image evaluation system installed in a computer. critical in that the positions of the features detected and evaluated in the camera image are known in the reference coordinate system.
• the features can be part of the object whose position is to be measured.
• a calibration body is a body specially made for the purpose of camera and system calibration, which is to be introduced into the field of view of a camera.
• the calibration body In order to achieve the required measurement accuracy in the determination of all camera parameters by means of the calibration body, the calibration body must be made at least a factor of 10 more accurate than the desired measurement accuracy. This is not necessary in the present three-stage calibration of the measuring system, since only the orientation of the camera in space is to be determined by means of (measurement) features recorded in the image. It has been found that for this purpose features can be used on the machining object itself. These are advantageously excellent form elements of the object, for example, holes, edges or the like., Which should be handled and measured anyway at the workplace.
• the position of the imaged in the camera features can be determined by measuring by means of an external measuring means or from the knowledge of the feature position in object coordinates and subsequent measurement of the object position in the reference coordinate system with external means.
• the feature position in object coordinates can be determined, for example, from known design data (CAD), a previous measurement of the feature positions by measuring the object, for example, in a measuring house or the like.
• CAD design data
• an assumption or definition of the current object position during system commissioning can also be used as a reference system.
• the positions of the object coordinates are not known from the design data and are determined by external measuring means.
• a robot installed in the system may be used to approach two positions in the camera image and to measure a feature on the robot via the image processing system or, in particular, to interact graphically interactively. Since the robot operates generally metrically calibrated, the coordinates of the positions of the two robot features in the reference coordinate system are known and can be used for the third method step.
• This procedure is particularly advantageous for the recalibration of cameras, for example in case of camera failure or camera replacement, since recalibration can be fully automatic.
• the method steps one and two need not be carried out in this case, since the determined parameters can be assumed to be constant.
• the third process step can be carried out fully automatically by a robot, possibly even with automatic commissioning, so that also the recalibration is particularly simple.
• FIG. 1 shows schematically the measurement setup for the method according to the invention for calibrating a measuring system based on at least one camera
• Fig. 2 is a camera of the measuring system
• Fig. 3 shows schematically an arrangement for error estimation.
• FIG. 1 schematically shows a measuring system 1 for determining the position of an object 2 designed as a body in a three-dimensional reference coordinate system 3, which forms the world coordinate system.
• the object 2 is detected with a calibrated optical camera 4 and determines the position of the object 2 based on certain object features.
• the camera 4 Before it is possible to use the measuring system 1 to determine the position and orientation of the object 2 in the coordinate system 3, the camera 4 must be calibrated when the measuring system 1 is put into operation.
• Calibration means the production of a computational relationship between the image coordinates X b , Y b of an image coordinate system 5, the
• Camera 4 and a metric reference or world coordinate system 3 which generally describes the three-dimensional space with the coordinates X, Y and Z.
• the method for establishing this relationship is the modeling of the physical relationship between the two coordinate systems 3, 5.
• all physical facts are modeled and para- metric, which determine the image of any spatial point on the image sensor of a particular digital camera 4.
• the external camera parameters are the position 6 and 7, the orientation of the cameras 4 in the reference coordinate system 3, which may be detected as a 6-dimensional vector (X k, Y k, Z k, k RX, Rijk, R ⁇ k).
• the internal camera parameters are, for example, the image width f, the distortion coefficients kappa Ki and K 2 , the dimensions of the image sensor elements d x and d y , the tiltings ⁇ x and ⁇ y of the image sensor plane with respect to the optical axis and the passage point S x and s y optical axis through the image sensor.
• the outer and inner parameters are calibrated in various steps, which are explained in more detail below.
• the internal camera parameters of the camera 4 are determined and permanently assigned to this camera 4.
• the camera 4 and the objective 8 of the camera 4 shown in FIG. 2 are constructed in a separate calibration space, for example during the production and pre-assembly of the measuring system 1.
• the objective 8 is adjusted to the conditions during subsequent use of the measuring system 1 and the camera 4 is put into operation.
• the image of a calibration body of known geometry (not shown in detail in the drawing) is recorded and evaluated in a manner known per se for carrying out the camera calibration. In this way, the internal parameters of the camera 4 used for the measuring system 1 are determined with the corresponding objective 8.
• the internal camera parameters 4 are determined for a particular camera 4 with a specific lens 8 in predetermined shooting situations.
• the internal camera parameters that can be determined in the laboratory before mounting the measuring system 1 apply to all cameras 4 of the same type as well as to other cameras and lenses with sufficient structural equality to the calibrated camera system comprising camera 4 and objective 8.
• the invention is not limited to the implementation of the first method step in the manner described above. Equivalently, the first method step can also be carried out after a fixed mounting of the camera 4 to be installed with objective 8 at the place of use, i. in a so-called onsite calibration, done.
• a calibration body is positioned near the place of use and the internal camera calibration is performed. The position of the calibration body in the reference coordinate system need not be known. There is therefore no external measuring means required for this. The calibration body only has to be detected and imaged by the camera 4.
• the calibration body is removed.
• this calibration of the internal camera parameters requires more time in the construction of the measuring system 1, in qualitative terms the result is basically the same.
• the first and second method step can be exchanged.
• the position 6 of the camera 4 is determined in a second method step, ie the coordinates X k , Y k , ZR of the camera in the reference frame are determined. coordinate system 3 determined.
• the position of this measuring mark 9 in the reference coordinate system 3 is determined exactly.
• the second method step it is also possible to remove the objective 8 from the permanently mounted camera 4 and to introduce a measuring mark 9 into the standardized and highly accurate objective recording of the camera 4.
• the position of this measuring mark 9 is determined by an external measurement, whereby due to the known and constant relationship between the measuring mark 9 and the image sensor in the camera 4 or other camera reference features, the camera position can be deduced.
• the orientation 7 of the camera ie the orientation of the objective 8
• camera images are evaluated in a third method step in order to determine the orientation 7 of the camera 4 in the three-dimensional reference coordinate system 3.
• Di ⁇ calculation of the orientation 7 of the camera 4 in space or generally external camera parameters is based on the solution of equations of projective mapping. This approach is called Lochbulmodell.
• the basis of the hole camera model is the boundary condition valid for all spatial points, that the visual ray of each spatial point (X, Y, Z) must pass through the projection center, which is formed by the hole of the pinhole camera.
• the point of impact of the line of sight on the image sensor determines the image coordinates X b and Yb.
• two equations can then be set up that follow the set of rays:
• f is the image distance
• X k, Y k, and Z k the coordinates of the space point in the camera coordinate system.
• XR is the coordinates of the spatial point in the camera reference coordinate system in vector representation
• X is the coordinates of the spatial point in the world reference system in vector representation
• T k is a transformation matrix of size 4x4 (representations in homogeneous coordinates) from the camera reference system to the world reference system.
• the transformation matrix berechet clearly from the six parameters Xk, Yk, Zk, RX k, Rijk, RZ k of the outer camera parameters.
• Methods are the translational parameters of the camera position X k , YR, ZR, known.
• Ry k> Rz So k not be determined. Therefore, at least two points in space are required to k the three orientations, k Rx, Ry from the resulting four equations (1) and (2), Rz k calculated to.
• a measurement object 2 for example in the form of a body as shown in FIG. 1, is positioned at the measurement location and two or more measurement features 10, which are visible in the camera image, are used to determine the camera orientation 7.
• the position of the measurement features 10 is known in the reference coordinate system 3.
• the following alternatives are conceivable.
• the position of the measurement features 10 can be determined in an object coordinate system 11 with the coordinates X 0 , Yo, Zo (in vector representation Xo).
• Measurement features 10 in the reference coordinate system 3 are then calculated from the relationship
• To is a transformation matrix of size 4X4 from the reference coordinate system 3 to the object coordinate system 11.
• these can also be determined by measuring the object 2, for example, in a measurement house. Since these positions of the measurement features 10 of the object 2 in the object coordinate Tensystem 11 are often known anyway, it is often easier, instead of an immediate measurement of all measurement features 10 with external measuring means directly in the reference coordinate system 3, only the position of the object 2 once in the reference coordinate system 3 to measure and as shown on the position of the measurement features 10 back ,
• special measuring marks can also be attached to any measuring object whose position is determined either by an external measurement directly at the application location or by a previous measurement an object coordinate system 11 and measuring the position of the object 2 can be done at the application.
• a separate calibration body can also be used at the application location, whose position in the reference coordinate system 3 is determined, from which the measurement features 10 in the reference coordinate system 3 can be derived. This can be done either due to a known geometry of the calibration or by direct measurement of the position of the measurement features.
• the camera 4 used in the measuring system 1 can be used to determine the inner and outer camera parameters with comparatively little effort and to set up the measuring system 1 in a time-optimized manner.
• This calibration in particular the method steps two and three, can also be carried out during operation without interruption of production for a recalibration of the measuring system 1.

Landscapes

• Engineering & Computer Science (AREA)
• Physics & Mathematics (AREA)
• General Physics & Mathematics (AREA)
• Computer Vision & Pattern Recognition (AREA)
• Theoretical Computer Science (AREA)
• Length Measuring Devices By Optical Means (AREA)

Priority Applications (3)

Applications Claiming Priority (2)

Publications (1)

Family

ID=36121493

Family Applications (1)

Country Status (6)

Cited By (2)

Families Citing this family (54)

Citations (4)

Family Cites Families (5)

• 2005
  • 2005-02-17 DE DE102005007536A patent/DE102005007536A1/de not_active Withdrawn
• 2006
  • 2006-02-08 ES ES06706728T patent/ES2321552T3/es not_active Expired - Lifetime
  • 2006-02-08 AT AT06706728T patent/ATE419508T1/de not_active IP Right Cessation
  • 2006-02-08 WO PCT/EP2006/001084 patent/WO2006087123A1/de not_active Ceased
  • 2006-02-08 US US11/816,246 patent/US8520067B2/en not_active Expired - Fee Related
  • 2006-02-08 DE DE502006002493T patent/DE502006002493D1/de not_active Expired - Lifetime
  • 2006-02-08 EP EP06706728A patent/EP1848961B1/de not_active Expired - Lifetime

Patent Citations (4)

Also Published As

Similar Documents

Legal Events