WO2006094637A1

WO2006094637A1 - Method for comparing a real object with a digital pattern

Info

Publication number: WO2006094637A1
Application number: PCT/EP2006/001632
Authority: WO
Inventors: Thomas Wurdig
Original assignee: Daimlerchrysler Ag
Priority date: 2005-03-05
Filing date: 2006-02-23
Publication date: 2006-09-14
Also published as: DE102005010225A1

Abstract

The invention relates to a method and computer program product for producing a comparison between a physical object and a predefined three-dimensional computer-accessible construction pattern (3). An object image sequence (2) is predefined. A viewing direction and a viewing position are determined for each image of the sequence (2) with respect to the construction pattern (3). The depiction of the object viewed from the viewing direction and position is produced and inserted into the image. The image sequence (9) comprising the inserted image depictions delivers a comparison.

Description

Method for comparing a real object with a digital model

The invention relates to a method of a computer program product and a data processing system for generating a pictorial comparison between a physical object and a predetermined three-dimensional computer-available design model of the object.

The invention has for its object to provide a method and apparatus for comparing a physical object with a given design model of the object.

The object is achieved by a method having the features of claim 1, a computer program product having the features of claim 14 and a data processing system having the features of claim 16. Advantageous embodiments are specified in the subclaims. The method and the computer program product create a comparison between the physical item and the given three-dimensional computer-available design model of the item. Provided is a sequence of pictures of the object. For each image of the sequence, the viewing direction and viewing position relative to the design model are determined. A representation of the object that the object from the Viewing direction and the viewing position of the image is generated and inserted into the image. The sequence of images with inserted representations provides the comparison.

In the following an embodiment of the invention will be described in more detail with reference to the accompanying figures. Showing:

Fig. 1. The interaction of different software programs in carrying out the method according to the invention;

FIG. 2 shows an example of the connection of reference points and reference points; FIG.

Fig. 3. the first part of a flow chart which explains the embodiment of the method;

4 shows the second part of the flowchart of FIG. 3,

Fig. 5. the Lochkameramodell;

6 shows the projection laws of the hole camera model;

FIG. 7 shows a coordinate transformation in the image plane; FIG.

8 shows a coordinate transformation between two three-dimensional coordinate systems;

9 shows the epipolar geometry.

The embodiment refers to a factory floor as the physical object. This factory building was or will be built using a given computer-accessible three-dimensional design model. This design model acts as a geometric specification of the factory floor. In the same way, the process can also be used for other buildings, eg. B. Buildings for exhibiting and demonstrating objects, for museums, office buildings, railway stations or manufacturing plants.

By applying the method, it should be ascertained whether the factory building was actually built or will be like this the design model specifies whether the SoIl geometry of the factory floor coincides with the actual geometry. Deviations between nominal and actual geometry can, for. For example, due to coordination and communication errors between the companies involved in the construction or due to errors in the production of architectural drawings from the design model.

The procedure can be applied after completion of the factory floor. But it can also be applied after completion of selected stages of construction during construction, z. B. after the shell has been completed and before the components of the interior are installed. The method can also be used to determine subsequent structural changes to the factory floor, which were not tracked in the design model.

As mentioned above, the process can be both for a finished factory hall for the factory hall in different stages of construction, z. B. after completion of the shell, apply. If the procedure for a finished factory hall is applied, a design model is given, the factory hall including the interior, z. B. together with the doors and supply lines for the manufacturing equipment, specified. If, on the other hand, the factory building is examined after completion of a construction phase, then a reduced design model is specified. This reduced design model geometrically specifies only those components of the interior that already exist after completion of the corresponding construction phase. Should z. As the shell are examined, a reduced design model is used, which specifies only the components of the shell geometrically, but not the components of the interior. The reduced design model is preferably generated from the complete design model by using ones Components are removed, which are not available after completion of the respective construction phase.

Fig. 1 shows the interaction of different software programs and databases in carrying out the method according to the invention. The sequence 1 taken by a recording device 40 is predefined from images of the factory floor, in the image format of the recording device 40. A first transformer 10 has read access to this image sequence 1 and generates therefrom an image sequence 2 of images from a data processing system processable data format.

Provided is a computer-available three-dimensional design model 3 of the factory floor. A second transformer 11 has read access to this design model 3 and generates a design model 4 in a data format that can handle both a mapping program 16 and a path generation program 13. Both the mapping program 16 and the path generation program 13 have read access to the image sequence 2 as well as to the design model 4. The mapping program 16 generates a computer-available mapping 7 between reference points in the images and reference points of the design model 3. The path generation program 13 additionally has read access to the mapping 7 and generates a computer-accessible path 5 through the design model 3 by determining its viewing direction and viewing position relative to the design model 4 for each image of the sequence 1.

A third transformer 12 also has read access to the given design model 3 and generates a design model 6 in a data format that a rendering generator 14 can process. The presentation generator 14 has read access to the design model 6 as well as to the path 5. It generates a sequence 7 of Representations showing the factory hall respectively from the viewing direction and viewing position, which the path 5 defines.

A merge program 15 has read access to the image sequence 2 as well as to the sequence 7 of representations. It inserts in each image of the sequence 2 the corresponding representation of the sequence 7. As a result, a comparison 8 is generated in the form of a sequence.

Fig. 3 and Fig. 4 show a flow chart which explains the embodiment of the method.

In the design model 3 distinctive reference points are selected (step S12) and given the method. Preferably, such reference points are selected that correspond to points that can be easily identified automatically in the images of the factory floor. If the factory hall has rectangular pillars, the upper and lower corner points of these parallelepiped pillars are preferred reference points. Each pillar thus provides eight reference points: four points are in the corners of the lower boundary surface of the buttress, four points in the corners of the upper boundary surface. Corner points can be easily recognized automatically in the pictures.

Every reference point. of the design model 3 corresponds to a reference point of the real factory hall. In the exemplary embodiment, the vertices of buttresses are thus reference points. How many corner points of abutments occur in an image depends on the position and viewing direction of the recorder 40 at the moment of recording.

In particular, when the pillars are not cuboid, the reference points corresponding reference points in the real factory hall are optically marked, z. B. with stickers that stand out visually clearly from the environment. Each reference point, in the exemplary embodiment so each pillar, the design model 3 receives a unique in the entire design model 3 identifier, eg. B. similar to the 64 fields of a chessboard. Furthermore, each reference point of a buttress receives a relative to the support pillar unique identifier, eg. As an integer between 1 and 8. Thus, each reference point of the design model 3 is assigned a unique identifier.

Preferably, the marking of a reference point of the real factory hall contains the identifier of the reference point corresponding to the reference point, for. Example, the identifier of the buttress and the number of the vertex.

A sequence 1 of images of the factory floor is generated. Preferably, 25 frames per second are recorded. Each image shows the factory floor from a particular viewing direction and from a particular viewing position. Preferably, each image shows multiple vertices of multiple buttresses. Sequence 1 includes pictures showing the water and electricity supply to the production facilities in the factory floor. Furthermore, the sequence 1 images, the manufacturing cells, robots, conveyor belts u. in the hall. If the method is applied to the shell of a factory building, part of the sequence 1 images showing those places where installed water and power lines later and manufacturing ^cells' are so positioned.

In one embodiment, the images of sequence 1 are generated with a high definition film or video camera. For example, each image has a resolution of 1920 x 1080 pixels, ie 1920 pixels per line and 1080 pixels per column. Preferably, the resolution is chosen such that the ratio of pixels per line to pixels per column is 16: 9. It is also possible, the images with a standard digital camera or a usual video camera, even with a device that is used for private users.

Preferably, the recording device 40, so the camera mounted during recording on a tripod or a lift, so that the images are generated without blurring and the recording device 40 is rotatably mounted about a vertical axis of rotation. Preferably, the camera can be moved along this vertical axis of rotation, so it is mounted vertically adjustable. This makes it possible to bypass items in the factory floor, if these items obscure the unobstructed view of that area of the factory floor that the pictures are to show. For example, transport containers, rack storage or manufacturing facilities can affect this clear view.

Before capturing the first image, the recorder 40 is preferably measured in the factory floor. This calibration improves the evaluation of the images described below. When measuring the current position and the current direction of the recording device 40 are determined. Preferably, the position and the viewing direction with respect to a given coordinate system, for. The first image is taken from this line of sight and from this position, and it is clear from which viewing direction and from what viewing position this first image shows the factory floor. The

Viewing direction and the viewing position of the first image are given to the method.

As described above, a design model 3 is specified which geometrically specifies the factory floor. Furthermore, a three-dimensional coordinate system and a scale for the design model 3 are given. Each reference point is uniquely identified by its position in this coordinate system. Viewing directions are called vectors in this Coordinate system specified. The origin of this coordinate system is linked with a point of the reference coordinate system for the real factory hall, the coordinate axes with coordinate axes of this reference coordinate system. The predetermined scale indicates the size relationship between the design model 3 and the real factory floor. Through the links and the scale, each point of the factory floor corresponds exactly to one point of the design model 3 and vice versa.

The sequence 1 of images is transferred into a sequence 2 of images in a form that is automatically evaluable by a data processing system. For example, one file is created for each image in a graphic data format. ^Examples' for such data formats are JPEG, GIF or TIF. The sequence of files is stored in a data store to which the data processing system performing the method has read access.

Among the pictures of the sequence 2, some pictures are selected (step S2 with the result 29). How many shots are selected depends on the speeds at which the viewing direction and viewing position of the recorder 40 are changed between shots. The larger these speeds are, the greater the proportion of selected images in the total number of images. The first picture belongs to the selected pictures. For example, every hundredth image is selected. In particular, if the position and viewing direction of the recording device 40 is varied uniformly during recording, it is also possible that a fixed period of time, for. B. ten seconds, and the images are selected so that between the recording times for two consecutive selected images is the time.

Or any picture taken in a certain period of time will be selected. For example, 25 Frames per second and takes the period of a length of 10 seconds, so 250 consecutive images are selected.

The reference points in each selected image are determined (step S13). Preferably, in each selected image, each vertex of each buttress is provided with an identifier. For each selected image, it is determined which corner points of which pillars occur in the image. As a result, the reference points in each selected image are determined and provided with the identifier. An alternative embodiment provides that, when a plurality of successive images of the sequence are selected, each vertex of each buttress is provided with an identifier in the first image of the selected sequence.

The design model 3 is transformed into a data format that can be processed by the display generator 14 as well as by a mapping program 16 described below (step S5). As a result, a processable design model 6 is generated. This data format is preferably "Virtual Reality Modeling Language" (VRML), for example VRML is described in U. Debacher: "VRML Introduction", 2003, available at http: // www. , queried on 16. 9. 2004, is described. Instead of VRML can be z. B. also use STEP.

An association of these corner points in the images to the corresponding reference points of the design model 6 is established. A reference point of the factory floor and a reference point of the design model correspond to each other when the position of the reference point in the reference coordinate system corresponds to the position of the reference point in the coordinate system of the design model.

In order to establish the assignment of reference points to reference points, a comparison 20 is preferably generated. This step S3 is preferred by the assignment Program 16 performed. The comparison 20 shows on the one hand the selected image of the factory hall with the registered identifications of the supporting pillars, on the other hand a representation of the factory hall which was produced with the aid of the design model 6 and which shows the factory hall from a specific viewing direction and from a specific viewing position. This viewing direction and viewing position of the representation are changeable. One reference point in the image and one reference point of the design model 6 are marked one after the other. It is determined that these two points correspond to each other. Thereby, an association 21 reference points and reference points in each selected image is generated in step S4.

Preferably, after selecting images of the sequence 2 (step S2) and after establishing the assignment (step S4), it is checked whether each predetermined reference point of the design model 3 has been taken into account. For this purpose, it is checked whether each reference point corresponds to a reference point that occurs in at least one selected image. For these tests, the mappings produced. automatically searched. In this way, it is also determined whether a buttress is present only in the design model 3, but not in the real factory hall. If this is the case, a corresponding message is output.

Through the step just described, associations are made between the reference points and the reference points in the selected images. Next, it is determined which points of the non-selected images correspond to the reference points (step S7). In each unselected image, the reference points, that is, the vertices of abutments, are automatically determined (step S6). The corner points can be automatically identified as intersections of pillars with floor or ceiling of the factory floor. This will be a lot 22 of Reference points in the non-selected image of the sequence 2 determined.

It is thus clear at which positions in the respective image reference points occur, but not at which reference points these reference points correspond. The position of a reference point in the image is preferably indicated in a two-dimensional coordinate system, e.g. By specifying the pixel (number of the row and number of the column in the digitized image).

Next, the position of each reference point of the image is compared with the positions of the reference points in the last or the first subsequent selected or previously examined image of the sequence 2. In the last selected image it is known which reference point corresponds to this reference point. The position comparison determines which reference point of the unselected image is the same as the reference point of the selected image.

For example, the closest reference point in the last selected image is determined and used as the same reference point. Or the position of the reference point relative to the neighboring object is taken into account both in the unselected and in the selected image.

Fig. 2 shows an example of the combination of reference points and reference points. In the sequence, first the image BI and then the image B-2 were generated. BI was selected, B-2 not. Through the assignment, it is known that the reference point BP in BI corresponds to the reference point RP in the design model. It has been determined that BP-I and BP-2 are cornerstones of buttresses. Therefore, BP-I and BP-2 are reference points that occur in B-2. Image analysis reveals that BP-1 in B-2 is the front left corner point of the lower boundary surface of the buttress and that BP-2 is the front right corner point. Further, it is determined that BP in BI is the front left corner point of the lower boundary surface. This will become automatically concluded that BP in BI is the same reference point as BP-2 in B-2. Thus, BP and BP-2 both correspond to the reference point RP of the design model.

This method is preferably carried out forward, each starting from a selected image. Let N be the indent of a selected image in the sequence. First, the image is examined for the selected image, ie the image with the index N + 1. Then the image with index N + 2 is examined and so on.

It is possible that this method for a non-selected image does not provide a clear assignment of a reference point to a reference point. In this case, the image is subsequently selected and the association of the reference point with a reference point is established by means of the comparison as described above.

As a result, step S7 provides for each non-selected image a computer-available mapping 23 between reference points and reference points in the non-selected image.

Subsequently, for each image of the sequence 2, its viewing direction and viewing position are determined relative to the design model. This step S8 is performed by the path generation program 13. Previously, the given design model 3 is transformed into a design model 4 in a data format that is processable by the path generation program 13 (step S9).

The path generation program 13 imports the design model 4 and the set 27 of the reference points. For each image of the sequence 2, the path generation program 13 uses the positions of the reference points in the image and the positions of the corresponding reference points 27. Each reference point is characterized by its position in the three-dimensional coordinate system of the design model 3. Each reference point has a determined position in the respective image of the sequence 2. The viewing direction and viewing position of the image relative to the design model 3 is determined from this information (result 24). This determination is performed by the path generation program 13. The determination provides a point (the viewing position) and a vector (the viewing direction) in the coordinate system of the design model 3. Methods for determining the viewing direction and viewing position of an image will be described in more detail below.

The sequence of viewing directions and viewing positions of the images provides a path 5 of the recorder 40 through the design model 3.

After completion of this step S8 is determined for each image of the sequence:

what reference points occur in the image (results 28 and 22), which positions in the image have these reference points and which reference points of the design model 3 corresponds to which reference point of the image (results 21 and 23).

Next, for each frame of the sequence, a representation of the factory floor is created showing the factory floor from the same viewing direction and viewing position as the picture. This perspective view is generated using the given three-dimensional design model. It shows how the factory floor should look from the viewing direction and the viewing position, while the captured image shows what it actually looks like.

A presentation generation program 14 generates a sequence 8 of perspective views of the factory floor. For this purpose, the design model 4 is preferably imported in the above-mentioned data format "Virtual Reality Modeling Language" (VRML) .This display generation program 14 uses the design model 4 and generates the sequence 8 of perspective representations. For this purpose, it uses the previously determined viewing directions and viewing positions of the images, which is stored in the form of a computer-accessible path 5.

The sequence of the viewing positions and viewing directions is preferably stored in the form of a script in a standardized data format. The presentation generation program 14 reads this script and executes it automatically. The execution of this script reflects the work of a designer. This designer uses a data processing system with input devices, a screen and a CAD program to create perspective views of the factory floor using the design model 3 and display it on the screen device. With the input devices he specifies the viewing direction and the viewing position of the representations and changes them. Running the script replaces this user's preference.

The sequence 8 of representations and the sequence of images are mixed together to produce the target-actual comparison. In one embodiment, the predetermined sequence 1 is used directly for this, in another embodiment the image sequence 2 in the processable form. In this case, the corresponding representation is inserted into each image of the sequence 1. Preferably, the geometric objects of the representations are characterized as being different from the objects shown in the images. For example, the objects of the representations receive a specific coloration. Mixing together provides a comparison 9.

The following explains how to determine the viewing direction and the viewing position of an image. Methods for analyzing computer-accessible images are, for. B. off R. Rome & M. Kolesnik: "3D Scene Reconstruction from Image Data", 1998, available at http://www.is.fraunhofer.de/projects/Makro/reports/macro-report VI-1.pdf, from

R. & A. Hartles Zisserman: "Multiple View Geometry in Computer Vision", Cambridge University Press, Cambridge, 4 ^th ed 2002, as well as from.

0. Faugeras & Q. -T. Luong: "Three-Dimensional Computer Vision," known by Press, 2001.

First, the so-called camera model of the central projection is displayed by means of a pinhole camera. Based on the simple projection laws in Lochkameramodell the general case of

Projection laws of a camera image for the relationship between an object and a picture shown. It also explains how to make the relationship between two images and the imaged object using epipolar geometry. This representation describes the stereo analysis of two images, which are solved by means of a so-called fundamental matrix with the help of corresponding points of both images.

Fig. 5 illustrates the Lochkameramodell, in a simplified form vividly illustrates the structure of a camera. A pinhole camera can be understood as a cube, on whose one side (on the focal plane FO) there is a small hole, the optical camera center C. The opposite side is a semitransparent side, the image plane R, on which the recorded object appears upside down. G is the height of the object. Perpendicular to the focal plane through the camera center is the optical axis Ac. The distance between the mutually parallel focal and image planes is referred to as focal length or camera constant f. The projection laws illustrated by FIG. 6 describe the mapping from a three-dimensional object point M in the world coordinate system to a two-dimensional image point m in the image coordinate system. For simplification, the origin of the coordinate system is placed in the camera center C, with the z-axis in the direction of the optical axis. An object point M with the coordinates (x, y, z) is projected into the image plane R onto the image point m with the coordinates (u, v). In this case, the pixel coordinates (u, v) refer to the coordinate system with the origin c, which represents the point of intersection between the optical axis Ac and the image plane R. The origin c is the intersection of the optical axis c with the image plane R.

From this description the following equations are derived, which result with the help of the similarity triangles or with the aid of the laws of radiation: f_ u V z X y

With the aid of homogeneous coordinates, the central projection can be represented as a linear image and written in matrix form. The homogeneous coordinates (X ₅ Y, Z, T) ^{T of} the object point M in the three-dimensional space SR ³ with x = X / T, y = Y / T, z = Z / T, T ≠ 0 and the coordinates (U , V, S) ^{τ of} the pixel m in the two-dimensional space SR ² with u = U / S, v = V / S, S ≠ O. In matrix form, therefore, the figure can be represented as follows:

However, in this projection from the three-dimensional to the two-dimensional space, the depth information is lost. The pixel m corresponds to an infinite number of object points, the are on the projection beam through m and through the camera center C. Therefore, this image is not unique when reversing a pixel into an object point of the world coordinate system.

The projection law in the pinhole camera model represents a simplification of the general projection laws. As a rule, images of the sequence 2 are used in a digitized computer-accessible form, which necessitates both an extension and a specification of the pinhole camera model. The image is stored in a matrix of pixels. In the event that these cells are not square, a computational balance must be made so that the image does not appear compressed or stretched in one direction. In addition, it is advantageous if the origin of the coordinates is not in the center of the image, but in a corner of the image. These changes in a pixel coordinate system are possible with a coordinate transformation with the following transformation matrix H.

H

k _u and k _v represent the scaling factors in order to adjust the cell size, and U ₀₅ V ₀ the shift into the

Picture corner.

Fig. 7 illustrates this coordinate transformation. Here, C is the optical camera center, e_u_old and ev old the units on the axes of the old one Coordinate system, e_u_new and e_v_new the units on the axes of the new coordinate system and u_0 and v_0 the coordinates of C in the new coordinate system. It is new _, ev new k_u = and k_v =. e_u_alt e_v_old

If the two transformation matrices H and P are multiplied together, the result is a combination A = HP of the two transformations, which expresses a mapping of homogeneous three-dimensional world coordinates into homogeneous two-dimensional pixel coordinates:

The parameters fk _u , fk _v , u _Q , v _ϋ represent the internal structure of the

Recording device 40, z. As a camera, and are therefore referred to as intrinsic parameters of the recording device 40. In a few exceptional cases, there is an additional skew parameter, which is zero for most recording devices.

-fk _t u _f

A = 0

0 0 1 0

Since the recording device 40 is at a different position in the world coordinate system (see Fig. 8) and also has a different orientation in each case, it becomes necessary to move this from one position and viewing direction u to the next by using a suitably changed world coordinate system to compensate.

The initial situation is established, in which the camera center without twisting at the origin of the World coordinate system is located. This transformation consists of translation and rotation and can be represented by the homogeneous transformation matrix E. Since the camera center is transformed from three-dimensional space into three-dimensional space, we speak of a so-called collineation, which can be calculated using the following equation:

e

R ^ (1 <= i, j <= 3) are the individual elements of the rotation matrix and t _k the individual elements of the transformation matrix. These parameters describe the position and orientation of the camera to the world coordinate system and are therefore referred to as extrinsic parameters. This transformation matrix is also used to map two cameras onto a common coordinate system.

The complete projection equation is described by the multiplication of the two transformation matrices A and E. The result is the projection matrix PM = AE with a "

i = 1,2,3:

PM If the same recorder 40 is used in different recordings, the internal (intrinsic) parameters need only be calculated once. The external (extrinsic) parameters must be calculated several times when the position and / or the viewing direction change.

For the projection matrix PM another notation is introduced in parallel. The matrix A is broken down into a 3x3 calibration matrix KM and into a zero vector 0. This yields the following expression A = KM * [I | 0] (with the unit matrix I). The matrix E is in

RM tv shorthand with the 3x3 rotation matrix RM

0 ¹ and the translation vector tv shown. This results in the following calculation rule:

RM tv

PM = KM * [I | 0] * = KM * [RM | tv] or KM * RM * [l | tv]

0 ¹ 1

In the case of a correspondence of 3D world coordinates and pixels (x <r »X), the projection matrix PM is calculated. In this case, for the twelve matrix coefficients with six linear, independent equation pairs, a solution can be found which, however, is often too inaccurate due to inaccurate image coordinates x. For this reason, a larger number of correspondences are chosen, resulting in an overdetermined system of equations. This can be calculated using the usual numerical methods, such as the Householder method (QR decomposition) and the singular value decomposition. Thereafter, a decomposition into the inner and outer parameters takes place with the same numerical methods, cf. z. B. Faugeras, supra.

Preferably, points in the three-dimensional world coordinate system and pixels are associated with each other by means of the reference points and reference points.

If the case occurs, the image of an associated three-dimensional coordinates of the world coordinate system not given and the camera parameters are unknown, a geometric relationship of two images to the imaged object with corresponding pixels can also be calculated using the epipolar geometry.

Fig. 9 illustrates the epipolar geometry by way of example. In Fig. 9, M is a point in the three-dimensional world coordinate system whose coordinates are not known. Given the image shown on the left in Fig. 9 with the image plane Rl, in which the point M occurs at the point ml. We are looking for the pixel m2 in the image plane R2. Cl and C2 are the two optical camera centers in Fig. 9, R1 and R2 are the two image planes for C1 and C2, respectively, and ml and m2 are the pixels of M in R1 and R2, respectively. El is the image of C2 in Rl, E2 is the image of Cl in R2. El and ml define the epipolar straight line epl, E2 and m2 the epipolar straight line ep2. The two vectors C ₁ C ₂ and C ₁ M span the so-called epipolar plane, which intersects the two image planes R 1 and R 2 in the epipolar lines epl and ep 2. This means that the associated pixel m2 is thus located on the intersection line of the right projection plane.

Preferably, projective geometry methods are used for the calculations. This will be described below using the example of a two-dimensional space 9t ² . A straight line in the projective plane SR ² is described by means of the implicit form of the line equation ax + by + c = 0 as a homogeneous vector l ^τ = [abc] ^τ . The following relationships are evaluated:

As a result, the following laws apply:

A point p ^τ = (u, v, w) ^τ lies on the straight line 1, or a straight line 1 passes through the point p if l ^τ p = p ^τ l = au + bv + cw = 0.

A straight line 1 through two points p and g is calculated using the cross-product 1-p ^y .q. The intersection of two lines / and m is also calculated using the cross product: p = lxm

Three points p, q, and r are collinear if their determinant is zero, that is, if: det [p, q, r] = 0

The projective geometry helps to determine the fundamental matrix FM described below and thus to determine the intrinsic and extrinsic parameters, since it simplifies the calculations by their properties.

As described above and illustrated in FIG. 9, the epipolar geometry is used to determine the second pixel m2 when the first pixel ml as well as the epipolar straight line ep2 are known or already calculated. Only the inner (intrinsic) and outer (extrinsic) parameters of both cameras are used. To use these parameters, the projection matrices PM1 and PM2 are used to calculate a so-called fundamental matrix FM with three rows and three columns. It is calculated so that: ep2 = FM * ml. Similarly, the inverse applies: epl = FM ^τ * m2. FM is invariant under a spatial projective transformation. Since m2 is located on the epipolar straight line ep2, the following epipolar condition is evaluated: mζ * ep ₂

* FM * m ₁ = 0. With this condition, the correspondence between two pixels ml and m2 is calculated.

Four steps are described below to calculate and apply the fundamental matrix FM. First, the first projection matrix orthogonal pML is aligned to the world coordinate system, ie, PM = ₁ KM * [I \ 0]. The second projection matrix PM2 with PM ₂ = KM ₂ * [RM | tv], where RM is a rotation matrix and tv is a translation vector, is determined using the fundamental matrix FM. For ep2, the following calculation rule arises: ep ₂ = (KM ₂ * tv) x (K ₂ * RM * KM ₁ ^"1 * m,), Here, KMl and KM2 are the two calibration matrices using the relationship

(M * a) x (M * b) = c * M- ^τ * (axb) with a scaling factor sf into the form ep ₂ = KMf * tvx (RK ^'1 * sf * m _ι ) = KM; ^T * [t] _x * RM * KM ₁ ^'1 * sf * w.

Since homogenous coordinates are used for the pixel ml, the scaling factor sf = 1 can be set directly, resulting in a total of: ep ₂ = KMf * [tv] _x * RM * KM ₁ ^-1 Hi ₁ = FM * Hi ₁ and for the fundamental matrix FM: FM = KM ^"T * [tv] _x * RM * KM ₁ ^{" 1}

Here, [tv] _x denotes the matrix notation of a cross product, i. H . [tv] _x = tvx v with

0 - V ₃ V ₂

[tv] _x = 0 - V ₁

- V ₂ V ₁ 0

The fundamental matrix FM has two epipoles El and E2. The following applies to this: FM * E1 = 0 and FM ^T * E2 = 0. The reason is that for any pixel ml: E ₂ * ep ₂ = E ₂ * FM * m ₁ = 0, since the respective epipole located on each epipolar line.

With the aid of the fundamental matrix FM and the epipolar geometry, a three-dimensional reconstruction and the position and viewing direction of the recording device 40 are calculated for each image. The procedure just described is used in particular in step S8.

First, the fundamental matrix FM is calculated from two corresponding images. With the aid of the fundamental matrix FM, the two projection matrices PM1 and PM2 are subsequently processed in accordance with the calculation rules PM ₁ = KM * [I \ 0] and PM ₂ = KM ₂ * [RM \ tv] calculated. As a result, the corresponding 3D point X is calculated by the so-called back projection with the two corresponding pixels ml and m2. In this case, geometrically considered, the backprojecting beams are cut together. Here ml and m2 are given, and X is according to the

Calculation rule X = calculated.

The fundamental matrix FM is z. B. by means of the normalized 8-point algorithm, the z. From P. Hartley, supra, or by the minimum 7-point algorithm. In order to calculate the path 5, at least seven correspondences (pixels) between different images of the sequence must have been calculated.

To determine the projection matrices PM and the extrinsic parameters, a so-called essential matrix EM is calculated. Preferably, a numerical method, e.g. The singular value decomposition. The intrinsic parameters of the recording device 40 must be known. If these are not known, a self-calibration procedure is used, e.g. For example, the method of Kruppa equations. This method is based on the idea that the five intrinsic parameters are defined by the projection of the absolute conic to the ideal plane. Knowing at least five control points or assignments between the 3D world coordinates and 2D image coordinates is directly reconstructed.

The three-dimensional point X is calculated by calculating X as the intersection of the two backprojecting lines. The linear equation system PMl * X = ml and PM2 * X = m2 is solved. In the practical implementation arise through the not exactly determinable pixel coordinates difficulties. Consequently, geometrically speaking, the shortest distance of the two projection straight lines must be determined become. X is calculated as the midpoint of the connection line.

List of used reference signs and symbols

BP, BP-I, references BP-2

C picture center

C, Cl, C2 Optical Camera Centers

El replica of C2 in Rl

E2 image of Cl in R2 epl, ep2 epipolar line f focal length

F focal plane

FM Fundamental Matrix g Item height

I unit matrix

KM, KMl, KM2 calibration matrices

M point to be imaged in the three-dimensional world coordinate system

M, ml, m2 Pixels of M in the image planes R, Rl and R2, respectively

PM, PMl, PM2 Projection matrices

R, Rl, R2 image planes

RM rotation matrix

RP reference point

Sl transformation of the image sequence 1 into the processable image sequence 2

S2 Select pictures in the picture order 2

S3 generating the juxtaposition of the design model 6 and the selected image

S4 Assigning reference points of the selected

Claims

claims

A method for generating a comparison between a physical object and a predetermined three-dimensional computer-available design model (3) of the object, wherein a data processing system is used for generating, a sequence (2) of images of the object is given in a form processable by the data processing system, and and the method comprising the steps automatically performed by the data processing system, that for each image of the sequence (2)

Viewing direction and viewing position relative to the design model (3) is determined, for each image of the sequence (2) using the design model (3) a representation (25) of the object is generated showing the object from the viewing direction and the viewing position of the image, and for each image of the sequence (2) the respective generated representation (25) is inserted into the image in such a way that the viewing direction and the viewing position of the representation coincide with the viewing direction and the viewing position of the image.

2. The method according to claim 1, characterized in that the viewing direction and the viewing position of the first image is specified and for each successive image of the sequence (2) whose viewing direction and viewing position relative to the design model (3) is determined, for which the viewing direction and the viewing position at least one previous picture of the sequence (2).

3. The method according to claim 1 or claim 2, characterized in that

Points of the design model (3) are selected and given as reference points,

the positions of these reference points are determined in the design model (3), for each image of the sequence (2) it is determined which reference points in the image correspond to which reference point of the design model (3) and at which positions in the image which of these reference points occur, and for each subsequent image of the sequence (2) the viewing direction and the viewing position of the image relative to the design model (3) are determined using the positions of the reference points in the image and the positions of the reference points of the design model (3) corresponding to these reference points.

4. The method according to claim 3, characterized in that

- Pictures of the sequence (2) are selected for each selected image of the sequence (2) one

Assignment of each reference point occurring in the image to the reference point of the reference point corresponding to the reference point

For each non-selected image of the sequence (2), the determination of which reference points in the image which reference point of the image

Design model (3) and to which

Positions in the image which of these reference points occur using the established mapping and the

Positions of the reference points in the selected

Pictures is performed.

5. The method according to claim 4, characterized in that the selection of images of the sequence (2) is performed such that for each reference point at least one image is selected in which the reference point corresponding to this reference point occurs.

A method according to any one of claims 3 to 5, characterized in that as reference points a class of points of the design model (3) are selected which correspond to a class of reference points of the object which are automatically identifiable in the images of the object.

A method according to any one of claims 1 to 6, characterized in that the sequence (2) of images of the object is given such that each image of the sequence (2) the object from a different viewing direction and / or viewing position than the preceding image shows.

8. The method according to any one of claims 1 to 7, characterized in that for each image of the sequence (2) a perspective view of the object is generated.

9. The method according to any one of claims 1 to 8, characterized in that a sequence (1) of images of the object in an image format of a recording device (40) for recording the images is given and by transforming the images into images in one of the data processing system processable data format, the processable sequence (2) is specified.

10. Use of the method according to one of claims 1 to 9, for generating a comparison between a building and a construction model of the building.

11. Use of the method according to one of claims 1 to

9, for making a comparison between an interior of a building and a design model of the interior of the building.

12. Computer program product that can be loaded into the internal memory of a computer and

Software sections with which a method according to any one of claims 1 to 9 is executable when the product is running on a computer.

A computer program product stored on a computer readable medium and having computer readable program means for causing the computer to execute a method according to any one of claims 1 to 9.

A computer program product that stores an information forwarding interface to a first data store having a physical object design model (3) and an information forwarding interface to a second data store, wherein a sequence (2) of images of the article is stored in a form processable by the computer program product for each image of the sequence (2) determining its viewing direction and viewing position relative to the design model (3) using, for each image of the sequence, a representation (25) of the article showing the object from the viewing direction and the viewing position of the image, and for each image of the sequence, inserting the generated image (25) into the image such that the viewing direction and the viewing position of the image Representation with the viewing direction and the viewing position of the image match.

15. Computer program product according to claim 14, characterized in that in the second data store a computer-accessible

Description of the viewing direction and the

Viewing position of the first image are stored and the computer program product is designed so that it in the determination of the viewing direction and

Viewing position of an image of the sequence used the viewing direction and viewing position of at least one previous image.

16. A data processing system having an information forwarding interface to a first data store having a physical object design model (3) and an information forwarding interface to a second data store storing a sequence (2) of images of the article in a form processable by the computer program product. and for automatically performing the following steps: for each image of the sequence (2) determining its viewing direction and viewing position relative to the design model (3) using, for each image of the sequence, a representation (25) of the article comprising shows the object from the viewing direction and the viewing position of the image, and for each image of the sequence, inserting the generated representation (25) into the image such that the viewing direction and the viewing position of the representation match the viewing direction and the viewing position of the image.