DE102005047481A1 - Optical sensor e.g. camera, calibrating method for use in e.g. augmented reality system, involves determining transformation of optical sensors relative to tracking object by combination of two tracking-systems - Google Patents

Abstract

The method involves determining transformation of optical sensors (100) e.g. cameras, relative to a tracking object (300) by combination of two tracking-systems, where the object is located within a measuring chamber. Tracking-points (200) firmly lie in a coordination system (500) of the tracking systems. The coordination system and a global coordination system (400) stay in a fixed relationship. The optical sensors are attached at an optical tracking system. An independent claim is also included for a system for calibrating an optical sensor in a measuring chamber.

Description

The The present invention describes a system and a method for Calibration of an optical sensor in a surveying room, in particular for use in an augmented reality system, in the optical Metrology, image processing, media technology and / or in motion capture.

Out the prior art are so-called augmented reality systems known. These allow the overlay of computer generated, virtual information with visual impressions of the real environment. To this end, the visual impressions of the real world, preferably with semipermeable data glasses worn on the head, with virtual Information mixed. The insertion of the virtual information or objects can be context-dependent, i. adapted and derived be executed by the respective considered real environment. When Information can in principle Any type of data such as texts, illustrations etc. can be used.

Documented applications of technology envisage use in the production, service and development of complex products. Also, the use of technology from the production of aircraft is known. After the publication DE 198 32 974 A1 and DE 101 28 015 A1 is the use of augmented reality technologies in production environments known.

In addition are known from the prior art position detection systems, which the position and / or orientation of objects in a survey room determine. These so-called tracking systems allow, for example the acquisition of up to six degrees of freedom of an object. To the The use of systems with different physical principles of action. common are so-called optical tracking systems by different Method of computer-aided image processing the position of objects in the survey room and / or the position of the camera (or the image sensor of the camera) over the Determine detection of the objects.

In addition are from the prior art statistical methods of optimization from readings (Nash, J.C. "The Singular-Value Decomposition and Its Use to Solve Least-Squares Problems. "Ch. 3 in Compact Numerical Methods for Computers: Linear Algebra and Function Minimization, 2nd ed. Bristol, England: Adam Hilger, pp. 30-48, 1990).

adversely in the known method for the optical determination of the position is the restricted one Work area as well as the strong dependency of the quality of the results of environmental influences (e.g., brightness). Thus, the combination of different is advantageous Tracking method, for example, to compensate for such environmental influences.

Known Methods of combined tracking require a complex initialization procedure by an experienced user (e.g., by selecting defined Points with the mouse) to the different tracking systems to each other to relate to. Next affect the manual intervention very unfavorable on the accuracy of initialization and need one very high time expenditure.

moreover No methods are known which additionally acceleration and or states of motion can initialize.

Of the Invention is based on the object, the process of calibration an optical sensor with respect to significantly simplify and enhance an external tracking system accelerate and thus a better and more accurate overlay to reach.

The previously derived and derived from the prior art task is inventively a method according to claim 1 solved, in which the calibration of an optical sensor by the combination achieved with a second tracking system that is beneficial according to claim 9 applies in one or more of the areas mentioned therein. Furthermore, the invention relates to a system for calibrating an optical Sensors in a survey room with an implemented method according to the invention, as specified in claim 10.

Especially The invention relates to a method for calibrating an optical Sensor with a tracking system with respect to a tracking object with an external, additional tracking system, where the transformation of the optical sensor relative to that in the surveying space Tracking item about the combination at least two tracking systems is determined.

Further advantageous embodiments of the invention are specified in the subclaims.

In the following an embodiment of the invention with reference to the illustrated in the drawing 1 and 2 explained in more detail.

1 shows an advantageous system structure for calibrating an optical sensor in a surveying space.

2 shows a schematic representation of the optimization of the method according to the invention

1 shows an advantageous system structure for calibrating an optical sensor in a surveying space. A tracking item 300 is in a fixed relationship to the origin of the image receptor 100-1 of the optical sensor 100 , For example, the tracking item 300 realized in the form of another sensor, such as in the form of an ultrasonic sensor, or in the form of a body, the latter, for example, from another camera of the second tracking system 300-1 (whose function will be described later) can be recorded. The tracking item 300 is for example on the optical sensor 100 mounted and so in later use of the system always in a fixed relation to the origin of the image sensor 100-1 of the optical sensor 100 , The tracking item 300 is used in particular for later use of the system different tracking methods (with respect to the sensor 100 on the one hand, and in relation to the tracking item 300 on the other hand). When calibrating the system, it is the task in an initialization of this fixed reference of the tracking object 300 to the origin of the image receptor 100-1 of the optical sensor 100 to determine.

The optical sensor 100 , Advantageously, a camera, is doing to an optical tracking system 100-2 connected. This determines, using known image processing algorithms, the position (x _c , y _c , z _c ) and / or rotation (red x _c , red y _c , red z _c ) of the imager 100-1 compared to the known tracking points P ₁ , P ₂ , P ₃ , P ₄ , which act as optical markers, from the spatially unspecified world coordinate system 400 that in the tracking system 100-2 is known. This corresponds to the coordinate transformation (transformation for short) v _opt .

In addition, by a second, external tracking system 300-1 the position (x _t , y _t , z _t ) and rotation (red x _t , red y _t , red z _t ) of the tracking object 300 in the coordinate system 500 the second tracking system 300-1 certainly. This corresponds to the transformation v _geg . To determine the values, a wide variety of active principles (eg mechanical, acoustic, opto-electronic, interferometric principles of action) can be used by means of a corresponding sensor. For example, the tracking system 300-1 Also an optical tracking system, which is the tracking item 300 recorded by a camera and detected in position and / or rotation. With the transformation v _geg is thus the position and rotation of the tracking object 300 in the survey room with respect to the coordinate system 500 this second tracking system 300-1 known.

The world coordinate system 400 and the coordinate system 500 of the tracking system 300-1 are in a committed relationship that is known or unknown. The known tracking points (P ₁ , P ₂ , P ₃ , P ₄ ) are fixed in both coordinate systems.

To calibrate the entire system, the transformation must v _{res of} the tracking object 300 to the origin of the coordinate system of the image sensor 100-1 be determined. It is therefore necessary to the tracking object 300 with respect to the optical tracking points P ₁ , P ₂ , P ₃ , P ₄ . This corresponds to the transformation v _tra .

For this The procedure is as follows:

1. Determining the position and / or rotation of the optical tracking system 100-2 known points P ₁ , P ₂ , P ₃ , P ₄ with the second tracking system 300-1 in terms of the coordinate system 500 the second tracking system 300-1 , This corresponds to the transformation v _bas .

For this purpose, for example, another sensor with a tracking system in the coordinate system 500 used the individual points 200 (P ₁ , P ₂ , P ₃ , P ₄ ) anfährt, the tracking system of this further sensor, the individual points 200 in terms of position and / or rotation in relation to the coordinate system 500 puts. The second tracking system 300-1 in this way obtains individual transformations v _bas1 to v _bas4 , which are processed for transformation v _bas .

In another case, it is also advantageous to use the coordinate system 500 in the origin of the points 200 For example, the origin of the coordinate system 500 coincident with point P ₁ . This would result in a transformation v _bas , which assumes the value zero.

Depending on the tracking system, at least three tracking points (P ₁ , P ₂ , P ₃ ) are required to determine position and rotation.

2. Calculation of the transformation v _tra between the tracking object ( 300 ) and the optical tracking points (P ₁ , P ₂ , P ₃ , P ₄ ) from the transformations v _bas and v _geg . Both transformations are in the coordinate system 500 certainly.

3. Calculation of the Transformation v _{res of} the Tracking Item ( 300 ) to the image recorder ( 100-1 ) from the two transformations v _tra and v _opt . Both transformations are in terms of points 200 certainly.

In addition to steps 1-3, the following steps can be carried out in an advantageous embodiment of the system, in particular with movement of the arrangement of sensor 100 and tracking item 300 in the survey room:

4. Save the transformation v _res ,

5th Re-running steps 1-4,

6. Generate an optimized transformation v _resopt by applying mathematical optimization methods, advantageously the methods Downhill-Simplex or Levenberg-Marquardt.

In a further embodiment, the following steps can be carried out in an advantageous embodiment of the system:
Determining the transformations v _opt , v _tra , as described above, with respect to a first position of the image sensor 100-1 of the sensor 100 and the tracking item 300 , Thereafter, the arrangement of sensor 100 and tracking item 300 Moved in the survey room, the movement in each case for the sensor 100 and the tracking item 300 is recorded. For example, in each case 400 values for v _opt and v _{tra are} obtained in this way. However, these individual values are not always related to each other, but the difference, that is, the variation or deviation, of the individual values over time is determined with respect to v _opt and v _tra , respectively. From the respective total _deviation , the optimized transformation can be _calculated , optionally again using the methods Downhill-Simplex or Levenberg-Marquardt.

The Levenberg-Marquardt optimization procedure is described in: Hartley, R. and Zisserman, A .: "A Multiple View Geometry in Computer Vision ", Cambridge University Press 2003, while the Downhill simplex optimization method a linear optimization algorithm developed by Nelder and Mead in: "A Simplex Method for Function Minimization ", Computer Journal, 7: 308-313, 1965th

2 shows a schematic representation of the procedure described above. In each case different values are obtained for v _opt and v _tra (over v _geg or v _tra is equal to v _geg , if v _{bas is} zero, see above), here v _{opt_1} to v _{opt_3} and v _{geg_1} to v _{geg_3} for three different arrangement positions of the optical sensor 100 and the tracking item 300 , From the deviation A1, A2 or B1 and B2 of the individual values over time in each case with respect to v _opt and v _tra (v _geg ), the optimized transformation v _resopt can be calculated by the equation A · v ResOpt = v ResOpt · B is resolved after v _resopt .

In In this context, A1 ... An or B1 ... Bn are specific expressions of A or B in the listed Equation. If measurements A and B were perfect, then a single pair of measurements would be made A1 and B1 are enough to solve the equation. But the measurements are always inaccurate, one needs redundant data (A1 ... An or B1 ... Bn), to minimize the error in the result (in the sense of the smallest Squares).

Advantages and applications of the system and method are:
The advantages of the method described for calibration lie in the simple and user-friendly handling. There is no need to know anything about the mathematical background (transformations) of the various tracking systems and the calibration is possible through a simple user interface.

In addition, this is no longer necessary Measuring the image sensor with respect to the tracking object, with usual Measuring methods, in particular due to manufacturing tolerances of Optical sensor, with a sufficient accuracy only very hardly possible is. The subsequent Surveying additionally requires usually the disassembly of the sensor, which is also omitted.

Further are no complicated interventions by the user in the system necessary and all calculations are done automatically. This increases the accuracy the calibration.

By The fast and accurate calibration of the system can be done before especially in mobile environments, e.g. in the area of service and maintenance or in production, applications of augmented reality technology be realized quickly and efficiently.

100

optical sensor

100-1

imager

100-2

optical Tracking system

200

Known Points in the environment

300

Tracking the subject

300-1

Tracking system

400

World coordinate system

500

Coordinate system of the tracking system ( 300-1 )

v _{opt_1} to v _{opt_3}

transformation

v _{geg_1} to v _{geg_3}

transformation

v _res

transformation

A1, A2

deviation

B1, B2

deviation

Claims (10)

Method for calibrating an optical sensor in a surveying room, in which the transformation of the optical sensor ( 100 ) relative to a tracking object located in the survey space ( 300 ) is determined by the combination of at least two tracking systems. Method according to claim 1, characterized in that known tracking points ( 200 ) are fixed in coordinate systems of at least two tracking systems. Method according to claim 1 or 2, characterized in that four known tracking points ( 200 ) are fixed in coordinate systems of at least two tracking systems. Method according to one of claims 1 to 3, characterized in that the transformation of the optical sensor ( 100 ) relative to a tracking object ( 300 ) is determined in the following steps: a) determination of the position and rotation of tracking points known for an optical tracking system ( 200 ) with at least one additional tracking system with respect to a coordinate system of the additional tracking system; b) calculating a first transformation (v _opt ) between an image recorder ( 100-1 ) of the optical sensor ( 100 ) and the known tracking points ( 200 ); c) calculation of a second transformation (v _tra ) between the tracking object ( 300 ) and the tracking points fixed in at least two tracking coordinate systems ( 200 ); d) calculation of at least one third transformation (v _res ) of the tracking object ( 300 ) to the image recorder ( 100-1 ) of the optical sensor ( 100 ) From both the first and second transformations (v _tra and v _opt). A method according to claim 4, characterized in that in step d) the determination of the third transformation (v _res ) by the statistical evaluation of several measurements, in particular by singular value decomposition, is optimized. A method according to claim 4, characterized in that in step d) the determination of the third transformation (v _res ) by the use of multiple tracking systems and subsequent statistical evaluation of the measurements, in particular by singular value decomposition, is optimized. Method according to one of claims 4 to 6, characterized in that in step d) the determination of the third transformation (v _res ) by the statistical evaluation of several measurements relative to each other, in particular by using the Downhill Simplex or Levenberg-Marquardt method, is optimized. Method according to one of claims 4 to 7, characterized in that the following steps are carried out: - determination of the first and second transformations (v _opt , v _tra ) with respect to a first position of the sensor ( 100 ) and the tracking object ( 300 ), - movement of the sensor ( 100 ) and the tracking item ( 300 ) in the survey room, wherein the movement in each case for the image sensor ( 100-1 ) and the tracking object ( 300 ) is recorded on the basis of individual values (v _{opt_1} to v _{opt_3} , v _{geg_1} to v _{geg_3} ) for the first and second transformation (v _opt and v _tra ), - determination of a deviation (A1, A2, B1, B2) of the values over the Time in each case with respect to the first transformation (v _opt ) and the second transformation (v _tra ), - Determination of an optimized third transformation (v _resopt ) from the respective deviation (A1, A2, B1, B2), in particular using the method Downhill Simplex or Levenberg-Marquardt. Use of a method according to one of the preceding claims in one or more of the following areas: in an augmented state Reality System, in optical metrology, in image processing, in media technology and / or motion detection (Motion Capturing). System for calibrating an optical sensor in a survey room with an implemented method one of the claims 1 to 6.