DE19536297A1 - Geometric calibration of optical 3-D sensors for three=dimensional measurement of objects - Google Patents

Abstract

The method involves strip projection and triangulation using at least one camera (10), pref. a video camera, an image sequence digitising and storage device, an illumination projector (11), an illumination device and an image control and processing computer. A calibration device (12) is placed at different positions inside and at the edge of the working vol. to be calibrated and illuminated so that regions of its signal marks have maximum grey values or colour modulation in the digitised and stored images without saturation. The camera is calibrated using a standard photogrammetric method. A projector with a light modulator projects image elements onto the calibration device. The projector is calibrated by observing it as an inverse camera. The measured and stored parameters are recovered for use with the operational sensor system to correct the beam geometry in the camera and projector.

Description

The invention relates to a method for geometric Ka calibration of optical 3D sensors for three-dimensional Measurement of objects relative to a reference coordinate system based on the principles of strip projection and triangulation, using at least one Ka mera, a device for digitization and spoke Image sequences of the camera, at least one for the camera mera fixed lighting projector, the time after each other light structures from at least one-dimensional Stripes generated within a working volume, min at least one  Lighting device for uniform lighting of reflective and / or scattering signal marks of a Ka calibration device and a computer for control and Processing the images and a device for through conduct of the procedure.

Methods for camera calibration are known. Here who the points recorded with a camera, their coordinates are known, i.e. the x, y and z coordinates in the World known and the images are evaluated. The potash bration of the camera is then based on the fact that the image dinates in accordance with the world coordinates be brought. The calibration of a projector is in the Difference more difficult because the projector certain can not see where it shines. The projector sends a certain stripe with the number x off, but there is no way with the projector find out where the strip ends up in the world, this requires additional auxiliary devices, namely more Cameras in the system. Observed with a calibrated and oriented these projector strips into camera the world and knows the stripes that the projector internally (virtual) has sent, you can see the connection between the external (visual) stripe of the projector and the projector's internal (virtual) strip len. However, you don't know where you are along an external strip. If the strip x is sent, so you can find in the world with the camera this stripe x again, but you don't know which x- y-position in the projector where the light was generated, then appears at x-y-z in the world. That means  tet that not every pixel of the camera a spatial Coordinate value can be assigned.

There are two-dimensional optical measurement methods with a Ma trix camera and a projector for uncoded or ko dated strips known, in which the three-dimensional Coordinates of the surface points from the two image coordinates Dinaten of the camera image and the on the respective image ordinate detected strip number can be calculated (Reinhard W. Malz: Coded light structures for 3-D measurement technology and inspection, thesis, University of Stuttgart 1992, series: Reports from the Institute for Technical Op University of Stuttgart). Procedures are also included known to several projectors.

Furthermore, there are photogrammetric optical measuring methods with several matrix cameras or a matrix camera in several known positions where the three-dimensional Coordinates of clearly determinable points in space from several Other views of the same can be determined numerically. For Generation of clearly definable points in the object space the natural object characteristics, for example regular or stochastic gray value or color distributions on the object, as well as artificial features like glued on or projected brands or regular or stochasti gray value or color distributions.

Also known are photogrammetric methods, the spatial position of one camera relative to several can determine visible reference points, their abs lute coordinates are known (spatial backward cut). Likewise, photogrammetric methods are  knows the several spatial positions of a camera relative can determine each other, provided that the multiple layers captured images have several common reference points contain, whose coordinates need not be known in advance sen (bundle adjustment). In this procedure, too In addition to the positions of the camera, the coordinates of the Points visible in several images are calculated (point mes solution).

Furthermore, there are photogrammetric calibration procedures Knows to determine the beam geometry of cameras. This is done by multiple observation of a temporal invariant calibration body using numerical methods the internal parameters of the camera in terms of focal length, Distortions etc. determined. Photogrammetric potash Brierverfahren achieve high precision and require no precision positioning devices. With an Ob jektfeld from z. B. 250 mm × 350 mm and a camera resolution of 500 × 700 pixels are lateral errors in the object field of <10 µm (1σ) achievable.

Furthermore there is an active optical triangulation ver drive to determine the beam geometry of projectors for surface measurement, in which the The projector must be able to project line crosses ren, the location of which by a photogrammetric camera ver band is measured (Th. Strutz: A precise active opti triangulation method for surface measurement, Thesis, Technical University Braunschweig, 1993). This Calibration therefore requires a special projector the line crosses are able to project.  

Numerical calibration methods are also known, which the through central perspectives of camera and projector as well Image space distorted by optical aberrations [ξ (x, y)] using polynomials on the 3D object space (x, y, z) ad aptieren (B. Breuckmann et al: Precision calibration of topometric sensors, LASER 95, Munich, 19.- 06/23/1995). A white plate is placed on top of several black circular rings are applied, several times exactly in Normal direction shifted and in accordance with the Sensor principle with different stripe patterns shines. The detected inside the circular rings Measured values are used for calibration. Disadvantageous is that this calibration is a highly accurate and well-defined ned mechanical positioning required because the Kali must be moved exactly in the normal direction. Furthermore, the correction calculation is based on the number long polynomial coefficients. The calibration ver driving is not for use in conjunction with photo grammatical procedures (navigation, bundle triangulation) suitable, which on compact and physically based Camera and projector models are supported.

The invention has for its object a method of the type mentioned and a device for To carry out the procedure with which to a measurement basically every pixel of the camera spatial coordinate value in a reference coordinate system stem can be assigned, the coordinate value be is reproducible.

The purpose of the invention is, for example, manufacturing machine such as pressing or milling or machine tools  to control the means of the invention so that the optical Recording a specimen as a highly accurate workpiece Image of the specimen can be made or any manipulations on the virtu made of visual material and again as a workpiece can be spent.

According to the invention, the object is that a) the calibration device inside and on the edge of the calibrating working volume one after the other in different positions relative to the camera and lighting project brought and the calibration device so be is lit that the areas of the signal marks of the potash briereinrichtung in the digitized and stored Images have the highest possible gray value or color modulation without oversteer, b) calibration of the Ka meras using a standard photogrammetric method follows to find the parameters of the inner and outer Orientation, c) the projector has a light modulator points, the externally visible pattern from any Stripes or pixels as picture elements on the calibration device projected, d) to calibrate the projector this is viewed as an inverse camera by the exterior a real or fictional x-y project pattern of defined width but indefinite length in Inside the projector is assigned, and for determination the parameters of the inner and outer orientation of the Be lighting projector multiple observations (image sequences) are used, in each of which geometric together relationships between the x-y projection patterns (internal pro ejector strip coordinates) and the corresponding ones externally visible patterns are produced, e) the found  which and saved calibration parameters during use of the 3D sensor system for measuring objects from one Algorithm can be reused, on the one hand the Beam geometries within the camera and projector corrected, on the other hand based on the principle of the tri angulation the x, y, z coordinates of the object in the object space calculated.

The advantage of the method according to the invention is in that with the same principle every pixel recording camera a spatial coordinate value in egg can be assigned to a reference coordinate system that is reproducible. The 3D sensor is used for this order from at least one camera, such as a video camera, Still Video Camera, Scanning Camera, and a facility for digitizing and storing sequences of images from the Ka mera, at least one fixed to the camera Lighting projector, the light sequentially in time structures of at least one-dimensional strips inside generated half a work volume, at least one Be lighting device for uniform illumination of reflective and / or scattering signal marks of a Ka calibration device and from a computer for control and Processing the pictures.

The calibration body (world) is for the purpose of calibration equipped with a black and white striped pattern, which is approximately orthogonal to the stripe pattern ended of the projector. This will strip the Pro periodically interrupted and approximately arise square bright spots of light. These spots of light can from a camera locally, i.e. in two or three dimensions  ions are detected and give a statement about the Location of the stripe in the reference coordinate system (world). If there are two cameras to calibrate a projector, which form a stereo pair, these cameras look at the Calibration plate with the white squares. This stereo can pair these square white spots of Ka Determine the vibratory plate in the room.

It is also possible to do without the second camera and the location of the white squares on the potash to accomplish the hotplate with a single camera. This is possible if with a camera the The location of the calibration plate in the room is determined using a so-called forward cut. The prerequisite for this is that the camera, so to speak, the geometry of the calibration plate knows and due to the perspective distortion Calculated the position of the calibration plate relative to the camera, to then determine in a second step where the white squares lie in the plane of the calibration plate, so that they are determined three-dimensionally. To the projector calibration is initially a two-camera kali system burns, d. H. it gets the inside of a camera itself Calibration done and then the relative assignment to each other, so that a complete absolute calibration tere stereo pair is available. This is suitable Absolutely measure points in space that came from both ras can be seen. Thus become white projector squares generated on the calibration plate, so you can also from can be measured in space. This will be different Positions of the calibration plate done, so the Track the beam path of the projector strips in space and through the projection center in the camera through the pro  extrapolate ejector pupil, with which the entire geometry of the projector.

The relative position and orientation of camera and pro ejector or cameras and projectors can tem porous, d. H. for the duration of calibration and measurement or permanently, for example by a mechanical rigid Connection fixed.

The calibration can be used for self-calibration of the camera direction advantageously at least four calibrated and one Numerous other signal marks or reference marks that must be in the camera's field of view. Dar can initially be solved with the help of direct solutions (literature Beyer) approximations for the elements of the outer orientie tion of the camera relative to the reference coordinate system (World coordinate system) can be calculated. Starting from the exact external orientation of the active-optical system in relation to the object-related Coordinate frames can be calculated.

Advantageously, for approximate automa table orientation of the calibration device relative to Camera at least three of the calibrated and / or the lot number of further signal marks must be encoded, provided that Coordinates of the latter are known at least approximately are. With the use of coded signal marks and direct Solutions for determining the approximations can do that Procedures are carried out fully automatically. You can the calculations are both indicators of accuracy speed, d. H. Standard deviations of the orientation elements,  as well as indicators of the robustness of the process output.

If there are several cameras, they can be on the same or under different focusing for different distances have the same or different just like them Angle of view can have the same as the cameras or have different spectral sensitivity can. Likewise, the cameras can be in the same or in look at different spatial directions and at the same time or take pictures at different times.

The method can produce signal marks with a high Contrast can be measured with very high accuracy and at least between 1/30 and 1/100 pixels.

If you want to calibrate systems that only consist of a Ka mera and a project exist, is additional information mation required, it must then be required that the Calibration plate is exactly described, d. that is, this is completely flat and that the black stripes that cut the projector strip, also precisely defined are nated. If this is the case, one Ka each picture shows the position of the calibration plate in the room are correct, which in turn means the position of the white squares on the calibration plate. With two-camera potash the white squares on the potash brierplatte seen from both cameras, more information The plate is not required. At the Einkameraka librierverfahren must loka to the calibration plate in the room to be able to also measure the retro marks or signal marks on the calibration plate. On the  The basis of this information is an affine transformation carried out, from which the position of the calibration plate results. This is observed in the affine transformation tete image of the retro marks or signal marks with a stored image of the calibration plate compared that Disk is then so ge through a numerical operation rotates until the image matches the saved image true, which then turns all six degrees of freedom of rotation translation in space are known.

Preferably, one with a camera and a projector active 3D measuring system calibrated and such a measuring system be built up, for which purpose a calibration second camera is provided with the first camera forms a stereo pair as long as the camera is calibrated. After the calibration has been carried out, the second one provided Camera are removed again, after which the calibrated Pro injector together with the first remaining camera active measuring system. This approach with two cameras for calibration can of course also for systems are used, which from the start and permanently two Ka exhibit meras. In this case, the strip could be ejected the projector with both one and the other another camera, so that, so to speak, two kopeck pelte measuring systems are available that once with a Ka mera and the projector and the other time with the other Camera and the same projector work. It is the same possible that both cameras, as when calibrating, too form a stereo pair when measuring and the projector as additional information for marking the object ver is applied.  

It is essential that the calibration plate be in its shape is known, the arrangements of the signal marks in advance sen and that the calibration plate over coded marks has an automatic orientation of the calibration plate in the room allows; at least four grid points on the calibration plate then serve to orient the same relative to the camera. It is advantageous if the four calibration points are coded to make their order can be identified automatically.

A projector is advantageously used which is only a one projected periodic grayscale or color grid projected, that can be converted into a phase image (lateral Phase shift method), the for calibration he required projector strips synthetically from the live image derived and the black-white-black-over of the synthetic stripes each place more constant Display the phase in the phase image.

The algorithm for recognizing the calibration parameters also corrects those provided in the projector model and deviating from common error models for cameras Distortion errors, for example due to a cylindrical Projection optics caused errors; not at first directly observable location along the projector strip for the purpose of correcting the distortion using an iterative Invoice determined.

If the calibration device is a flat calibration plate, it can have at least four calibrated signal marks point, and the black- white stripes of known geometry synthetic he  be created by using a calculated strip grid Help with the currently determined orientation parameters in the Image plane of the camera transformed and with the image of the Calibration plate is logically linked.

A plane is therefore advantageous as a calibration device Calibration plate, the at least four calibrated signal brands and a black and white striped grille Has geometry.

The calibration device can also be a non-plane Be calibration plate, the at least four calibrated Si markers and a black and white striped grille Has geometry, with at least two cameras for one with which the current topology of the Ka Libriereinrichtung is measurable at any time.

If the calibration device is a flat calibration plate which has at least four calibrated signal marks points, the black- white stripes of known geometry synthetic he be created by using a calculated strip grid Help with the currently determined orientation parameters in the Image plane of the camera transformed and with the image of the Calibration plate is logically linked.

Further advantages of the invention consist in particular in a cost reduction by dispensing with mechanical pre precision positioning devices such as coordinate measuring machines, Precision measuring robot, or additional precision measuring system systems, such as theodolite, interferometers, which otherwise the Measure length of workpiece or sensor.  

Brief description of the drawing, in which:

Fig. 1 shows a schematic representation of a camera projector unit as a sensor that is directed to a calibration plate,

Fig. 2 shows a possible embodiment of a calibration device in the form of a planar calibration plate,

Fig. 3 is a schematic illustration of the shift of the calibration plate in the working volume,

Fig. 4 an illuminated calibration with retroreflective marks,

Fig. 5 shows a typical illumination field of a projector, as applied in the process,

Fig. 6 is a calibration plate under illumination with a periodic projector stripe pattern.

Fig. 1 shows a compact embodiment of a 3D sensor consisting of a camera 10 and a projector 11, where referred to in the rays of the camera 10 with xy-coordinates and the one-dimensional Streifenkoordinate the projector 11 with z. Furthermore, a calibration plate 12 with a plurality of signal marks 13 is shown in FIG. 1, which is in the field of view of camera 10 and projector 11 .

Fig. 2 shows a possible embodiment of a calibration device as a flat plate 21 with the four corners A, B, C, D and with three different structures: First, coded marks 22 with a clearly identifiable label, which are arranged in the middle of the calibration plate 21 square are. Further uncoded marks 23 , furthermore parallel black and white stripe patterns 24 , which in use are then orthogonal to the projector stripes and cut out sections of the projector stripes piece by piece.

Fig. 3 shows schematically how a calibration plate 30 having its four corners A, B, C, D is moved in the working volume 31, wherein the working volume 31 is illustrated by a wireframe with the points P1 to P12 in correspondence of the corners A, B, C, D which correspond to points P1 to P12; these points are stored in a table, Table 1.

Table 1

FIG. 4 describes the calibration plate 41 of FIG. 2 and illumination with retroreflective marks 42 , only these being visible with a recognizable brightness.

Fig. 5 shows the typical lighting field of the projector tor, in which several projector strips are combined to form a strip block, for. B. in the strip block 51 the project strips 1 to 8 are combined and switched to white, in the strip block 52 the projector strips 9 to 16 are combined and switched black etc.

Fig. 6 shows the calibration plate of Fig. 2 under illumination with a periodic projector stripe pattern at an angle of 90 degrees; With this type of illumination, the projector light strips are subdivided into small square areas 61 , which are accessible for measurement of a photogrammetric detection and measurement by means of a camera and image correlation, specifically according to FIG. 3.

The invention is particularly applicable to a Camera-projector unit or a two-camera project Gate unit without the need for expensive precision calibration devices to calibrate to such a ka librated one-camera projector sensor or two-camera Projector sensor in principle for a subsequent measurement Every pixel of the camera has a spatial coordina ten value in a reference coordinate system in a clear and assign it in a reproducible way. Her position machines, such as presses or milling machines or tools machines can be controlled so that the optical  Recording a specimen as a highly accurate workpiece Image of the specimen can be made or any manipulations on the virtu made of visual material and again as a workpiece can be spent.

Claims (9)

1. Method for the geometric calibration of optical 3D sensors for three-dimensional measurement of objects relative to a reference coordinate system on the principles of stripe projection and triangulation, using at least one camera ( 10 ), preferably video camera, a device for digitizing and storing of image sequences of the camera, at least one lighting projector ( 11 ) fixed to the camera, which generates light structures from at least one-dimensional stripes within a working volume one after the other, at least one lighting device for uniformly illuminating reflecting and / or scattering signal marks of a calibration device ( 12 ) and a computer for controlling and processing the images, characterized in that

• a) the calibration device within and on the edge of the work volume to be calibrated successively brought into different positions relative to the camera and lighting projector and the calibration device is illuminated in such a way that the areas of the signal marks of the calibration device in the digitized and stored images have the highest possible gray value or Have color modulation without overdrive,
• b) the calibration of the cameras is carried out using a standard photogram metric method to find the parameters of the inner and outer orientation,
• c) the projector has a light modulator which projects externally visible patterns from any strips or pixels as image elements onto the calibration device,
• d) to calibrate the projector it is viewed as an inverse camera by assigning a real or fictional xy projection pattern of defined width but indefinite length inside the projector to the outer picture elements, and to determine the parameters of the inner and outer orientation of the Lighting projector several observations (image sequences) are used, in each of which geometric connections between the xy projection patterns (internal projector strip coordinates) and the corresponding externally visible patterns are made,
• e) the calibration parameters found and stored when using the 3D sensor system for measuring objects are reused by an algorithm which on the one hand corrects the beam geometries within the camera and the projector, and on the other hand the x based on the principle of triangulation, y, z coordinates of the object are calculated in the object space.

2. The method according to claim 1, characterized in that the camera using a calibration device with little at least four calibrated and a large number of other Si is self-calibrated. 3. The method according to claim 2, characterized in that for approximate automatic orientation of the potash briereinrichtung relative to the camera at least three of the calibrated and / or a variety of others Signal marks are encoded, provided the coordinates of the the latter are at least approximately known. 4. The method according to claim 1, characterized in that a projector is used that has only a single peri projected grayscale or color grid that projects into one Phase image can be converted (lateral phase shift procedure), whereby those required for calibration Projector strips synthetically stripped from the phase image be tet and the black-white-black transitions of the syn Theoretical stripes have places of constant phase in the pha represent the picture. 5. The method according to claim 1, characterized in that the algorithm for recognizing the calibration parameters the provided in the projector model and of usual mis deviating distortion errors for cameras,  for example by a cylindrical projection optics ver caused errors, also corrected and that the initially not directly observable location along the project for the purpose of correcting the distortion an iterative calculation is determined. 6. The method according to claim 1 using an ebe NEN calibration plate as a calibration device, thereby ge indicates that the calibration plate is at least four ka has librated signal marks, and that for calibration required black and white strip grids of known Geometry is generated synthetically by a calculated Strip grid with the help of the currently determined Orientie transformation parameters transformed into the image plane of the camera and logically ver with the image of the calibration plate is knotted. 7. Device for performing the method according to An saying 1, characterized in that the calibration direction is a flat calibration plate, at least four calibrated signal marks and one black and white Has striped grids of known geometry. 8. Device for performing the method according to An saying 1, characterized in that the calibration direction is a non-level calibration plate, which is little at least four calibrated signal marks and one black and white Stripe grid of known geometry, with little At least two cameras are used with which the ak current topology of the calibration device at any time is measurable.   9. Device for performing the method according to An claim 6, characterized in that the calibration direction is a flat calibration plate, at least has four calibrated signal marks, the Ka calibration required black and white strip grids known geometry is synthetically generated by a be calculated strip grid using the currently determined Orientation parameters in the image plane of the camera trans is formed and logical with the image of the calibration plate is linked.