GB2481459A

GB2481459A - Capturing A Surface Structure Of An Object Surface

Info

Publication number: GB2481459A
Application number: GB1010818.1A
Authority: GB
Inventors: Andre Stork; Martin Ritz; Manuel Scholz; Daniel Danch
Original assignee: Fraunhofer Gesellschaft zur Forderung der Angewandten Forschung eV
Current assignee: Fraunhofer Gesellschaft zur Forderung der Angewandten Forschung eV
Priority date: 2010-06-25
Filing date: 2010-06-25
Publication date: 2011-12-28
Anticipated expiration: 2030-06-25
Also published as: DE102011078052A1; GB2481459B; DE102011078052B4; GB201010818D0

Abstract

The invention relates to a method of capturing a surface structure of a surface of an object (1,fig.1) held by holder 13 wherein a pattern of electromagnetic waves is projected from a pattern source 2,4 (pattern projector 2, shiftable lens 4) onto the surface of the object (1), so that a projected pattern is incident on the surface of the object (1). A first reflected image, which is a result of a reflection of the projected pattern by the surface, is received by a camera 5 and is recorded, wherein the first image is recorded as a digital image comprising pixels. The projected pattern is shifted in a direction lateral to a projection direction into which the electromagnetic waves propagate, so that same parts of the projected pattern are incident at different locations at the surface of the object (1) compared to the projected pattern before shifting the pattern, wherein the pattern is shifted by moving at least one part of the pattern source 2, 4(this forms step a (c)). A second reflected image, which is a result of a reflection of the projected pattern by the surface, is received by a camera 5 and is recorded, wherein the second image is recorded as a digital image comprising pixels (this forms a step (d)). Steps c) and d) are repeated, so that further digital images are recorded which correspond to different shift positions of the projected pattern. Then, for a plurality of pixels of the first image, an allocation of the respective pixel to a coordinate of a corresponding point of the pattern of electromagnetic waves in a coordinate system of the pattern is determined by evaluating the first image, the second image and the further images. Then using the allocation and using geometric relations of the position and orientation of the pattern source (2.4) and of the camera (5), the surface structure is determined.

Description

Capturing a surface structure of an object surface The present invention relates to a method of and an arrangement for capturing a surface structure of a surface of an object. In particular, the information which is captured shall include information about the depth profile of the surface structure. As a result, the gathered information is either two-dimensional (including the depth direction and one lateral direction) or three-dimensional (including the depth direction and two lateral directions). In many cases, points on the surface of the object are determined, wherein the points form a three-dimensional cloud of point characterising the structure of the surface.

Several methods of capturing such a surface structure have been proposed. For example, the publication "Optical 3D-surface reconstruction by a multi-period phase shift method" by E. Lilienbium and B. Michaelis, Journal of Computers, Volume 2, No. 2, April 2007, pages 73 to 83 discloses the basic principles of some of the methods. Light patterns or patterns of other electromagnetic waves are projected onto the surface of the object instead of "pattern" the expressions "structure" or "fringe pattern" are used to describe that the illumination of the surface varies with the location and that the illumination profile of the incident electromagnetic waves is voluntarily chosen and pre-determined. Therefore, the information about the incident illumination profile can be used to determine the depth profile of the surface structure. The reflected illumination profile is received by at least one camera, wherein the viewing angle of the camera differs from the angle of the incident projected pattern. Consequently, the picture(s) which is/are taken by the camera(s) show(s) a modified illumination profile and the modification compared to the incident projected pattern depends on the depth profile.

From the above-mentioned publication, it is also known that the projected pattern can be * shifted in a direction lateral to the direction of wave propagation and several images of the resulting shifted pattern can be taken. If the projected pattern is periodic, such as a regular sequence of parallel dark and bright stripes, a Fourier analysis can be performed S.....

* of the several images and a phase can be obtained which contains the information which position of the projected pattern corresponds to a particular point in the images taken by the camera or cameras. The phase can be determined for each point in the images and, by using geometric relations of the position and orientation of the source of the projected pattern of the object and of the camera, the surface structure can be determined. One approach of using the geometric relations is the well-known so-called triangulation method.

The phase shifting mentioned above is performed by controlling the source of light in a corresponding manner. Typically, the source of light is a matrix of light emitting diodes.

For shifting a pattern of dark and bright parallel stripes, other columns or lines of the LED matrix are illuminated than before. However, the resolution in depth direction of the object surface which can be obtained by this approach of phase shifting or pattern shifting is limited by the distance and width of the neighbouring columns or tines of the illumination matrix. Using interpolation when the several images are evaluated which correspond to the different shift positions, a resolution of about the sum of the width and the distance of the LEDs may be achieved.

It is an object of the present invention to provide a method and an arrangement of the kind described above which allow for an increased resolution in depth direction.

It is a basic idea of the present invention to shift the projected pattern not or not only by controlling the light source to produce a shifted pattern. Rather, at least one optical element (such as a lens or a reflector) is controlled to transmit and/or reflect the pattern of electromagnetic waves according to a desired shift position onto the surface of the object.

Alternatively or in addition, the source of the electromagnetic waves (e.g. the matrix of LED5) may be shifted so that the projected pattern or electromagnetic waves is incident on the surface of the object at the desired shift position. More generally speaking, the pattern shift is achieved by mechanical movement of at least one part of the pattern source instead of controlling the generation of the electromagnetic waves so that a shifted : pattern is generated. "Mechanical movement" means and that a mechanical part of the * pattern source is moved, It does not mean that the movement is mechanically actuated.

Mechanical actuation is possible, but other types of actuations are preferred, as will be described below. "Movement of a part" does not include so-called "digital shifting" of the pattern by just controlling the electromagnetic wave generator, e.g. the light source.

****** * Consequently, the arrangement comprises a shifting device for shifting the projected pattern in a direction lateral to a projection direction into which the electromagnetic wave pattern propagates, wherein the shifting device is mechanically or otherwise coupled to at least one part of the pattern source which generates and projects the pattern of electromagnetic waves.

Moving at least one part of the pattern source in order to produce a shift of the projected pattern has the advantage that the shift position of the projected pattern can be adjusted more precisely and different shift positions can be achieved which have smaller distances to each other.

In particular, a method of capturing a surface structure of a surface of an object is proposed, wherein a) a pattern of electromagnetic waves is projected from a pattern source onto the surface of the object, so that a projected pattern is incident on the surface of the object, b) a first reflected image, which is a result of a reflection of the projected pattern by the surface, is received by a camera and is recorded, wherein the first image is recorded as a digital image comprising pixels, c) the projected pattern is shifted in a direction lateral to a projection direction into which the electromagnetic waves propagate, so that same parts of the projected pattern are incident at different locations at the surface of the object compared to the projected pattern before shifting the pattern, wherein the pattern is shifted by moving at least one part of the pattern source, d) a second reflected image, which is a result of a reflection of the projected pattern by the surface, is received by a camera and is recorded, wherein the second image is recorded as a digital image comprising pixels, e) steps c) and d) are repeated, so that further digital images are recorded which correspond to different shift positions of the projected pattern, f) for a plurality of pixels of the first image, an allocation of the respective pixel to a coordinate of a corresponding point of the pattern of electromagnetic waves in a coordinate system of the pattern is determined by evaluating the first image, the second image and the further images, g) using the allocation and using geometric relations of the position and orientation of the pattern source and of the camera, the surface structure is determined.

* Furthermore an arrangement of capturing a surface structure of a surface of an object is proposed, the arrangement comprising: a) an electromagnetic wave pattern projector comprising a pattern source for projecting the pattern of electromagnetic waves onto the surface of the object, so that a projected pattern is incident on the surface of the object, b) at least one camera for receiving and recording reflected images1 which are a result of a reflection of the projected pattern by the surface, wherein the camera is adapted to produce digital images comprising pixels, c) a shifting device for shifting the projected pattern in a direction lateral to a projection direction into which the electromagnetic wave pattern propagates, so that same parts of the projected pattern are incident at different locations at the surface of the object compared to the projected pattern before shifting the pattern, wherein the shifting device is coupled to at least one part of the pattern source to move the part and thereby shift the pattern, d) an evaluation device for evaluating reflected images, which correspond to different shift positions of the projected pattern, wherein the evaluation device is adapted to determine, for a plurality of pixels of the recorded images, an allocation of the respective pixel to a coordinate of a corresponding point of the pattern in a coordinate system of the pattern by evaluating the reflected images and is adapted to determine, using the allocation and using geometric relations of the position and orientation of the pattern source and of the camera, the surface structure of the object.

In particular, the pattern may be a periodic pattern, wherein the illumination effected by the pattern varies in one direction only, wherein this direction is preferably the lateral direction in which the projected pattern is shifted. If a matrix of controllable elements is used to produce the pattern, it is preferred that the periodic pattern comprises alternating dark and bright lines, wherein each bright line is produced by a line of the controllable matrix elements. Instead of a line, a column may be used. For example, if a standard *..* matrix of light emitting diodes is used to produce the pattern, every second column or line of the matrix is illuminated and the lines or columns in between the illuminated lines or columns are dark. Consequently, the period length of this pattern is equal to the width of two columns or two lines of the matrix elements, plus the distances between the lines or * columns of the matrix elements. * * .

Periodic patterns have the advantage that they can be shifted in fine steps over a period * length and that all possible illuminations of the object surface can be achieved by that.

Preferably, the pattern source comprises a source of the electromagnetic waves which is operated in the same manner for different shift positions of the projected pattern. In other words, the shift of the projected pattern is preferably generated only by moving the at least one part of the pattern source. This has the advantage that the same intensity profile of the electromagnetic waves generated for the different shift positions. In particular, the source of electromagnetic waves is adapted to produce the desired pattern of electromagnetic waves and the shifting device only shift and/or redirects the generated pattern in order to achieve the desired shift position.

If the source of electromagnetic waves produces the pattern of electromagnetic waves in the same manner for the different shift positions, it is preferred that the source of electromagnetic waves comprises a matrix of controllable elements for producing the pattern. For example, the matrix may be a matrix of light generating elements, such as LEDs, or may be a matrix of elements the wave transmission of which can be controlled, such as a matrix of liquid crystals.

According to a preferred embodiment of the shifting device, the at least one movable part of the pattern source is moved using electric and/or magnetic forces. Compared to mechanical forces smoother operation comprising less discontinuities of the movement can be achieved. In contrast applying mechanical forces would require bearings or other friction-subjected elements. Preferably, the electric and/or magnetic forces are directly applied to a holder of the optical element which is to be moved. "Directly" applying the forces means that the forces are applied to the holder, including any part or element which is fixed to the holder. It does not mean applying forces to parts or elements which are moveably connected to the holder, such as via a hinge. S...

The holder can be moved using a motor (preferably a linear motor), wherein the holder or * : * a part which is fixed to be holder is part of the motor.

In contrast to standard pattern projectors, which allow for movement of a lens in order to adjust the direction of the wave propagation, the shifting device of the present invention preferably comprises a control input to receive a control signal. A controller which is *. : connected to the control input generates the control signal and, thereby, controls the movement of the movable part.

Furthermore, it is preferred that the at least one movable part of the pattern source is held by a holder and that the holder is supported by a holder support, wherein the holder is movable relative to the holder support in order to allow a shift of the holder and the optical element, wherein the holder and the holder support are connected to each other by an elastically deformable material which allows for the relative movement. For the same reasons as described above, the effects of friction are reduced or eliminated completely and, therefore, very fine distances between neighbouring shift positions of the projected pattern can be achieved.

Preferably, the holder and the holder support are regions of an integral element, wherein at least one further region of the element which connects the holder with the holder support has a reduced thickness compared to the holder and the holder support in order to allow for elastic deformation of the further region. In particular, the integral element can be made of metal, copper or stainless steel.

Shifting the pattern of electromagnetic waves by moving at least one part of the pattern source, wherein the pattern source comprises at least one optical element for projecting the pattern onto the surface of the object, raises a problem which does not exist with conventional digital shifting. The shift of the pattern causes a modification of the illumination profile even if the generated pattern is not modified. The reason is that the generated pattern is transmitted and/or redirected by the at least one optical element.

Since optical elements have irregularities which cause distortion and aberration, in particular chromatic aberration, the step of allocating the pixels of the recorded images to a coordinate of a corresponding point of the generated pattern of electromagnetic waves becomes imprecise, unless additional steps are taken. One possible additional step is the calibration of the arrangement at least consisting of the pattern source and the camera for each shift position of the projected pattern. "Calibration" means that the geometric relations of these components of the arrangement, including the distortion and aberration produced by the optical elements are determined. However, performing a calibration procedure for each shift position increases the effort for preparing the capture of the surface structure significantly. If, for example, 200 shift positions shall be used, 200 calibrations need to be performed. On the other hand if these calibrations are not * . : performed, the resu'ts of very fine pattern shifting steps become imprecise, provided that the conventional evaluation procedures are applied. These conventional evaluation procedures include a Fourier transform or a wavelet transform.

In order to solve this problem it is proposed to fit a parameterised model to the recorded image values of the recorded images for each image pixel which is to be evaluated. For example, the image value may be a grey scale value or brightness. The image value is the result of the reflection of the projected pattern by the surface of the object and each specific pixe' corresponds to a surface point of the object. The fit may be performed using an appropriate optimisation algorithm.

Using a parameterised model has the advantage that the distortions and aberrations of optical element(s) are eliminated or nearly eliminated by the fit. The model is designed to model an expected type of behaviour of the image values as a function of the shift position of the projected pattern. In case of the projected pattern being a periodic pattern, the model preferably models one period or less than one period of the pattern, wherein the modelled range corresponds to the range of shift positions for which images are recorded by the at least one camera. In case that the optical element which is moved by the shifting device is an element which is transmitted by the electromagnetic waves of the projected pattern (the optical element being, for example, a focussing length), sharp contrasts of the components (e.g. lines or columns) of the projected pattern are diminished, in particular due to scattering. Another effect which diminishes the sharp contrasts is the variation of the distortion and aberration with the wave length of the electromagnetic waves. If, for example, white light is used, the spectral components of the light are differently redirected and scattered by the optical element. Furthermore, the different spectral components may be reflected by the surface of the object in different manner. These and further effects that diminish sharp contrasts of the pattern can be taken into account when the model is designed. However, in case of a periodic pattern and a corresponding model which comprises, for example, the periodicity and the phase as parameters, an exact modelling Sd of these effects or of the results of these effects is not necessary. Despite diminished contrasts, the periodicity can still be determined and, as well, the phase of the respective pixel. For example, in case of a sine or cosine function as the model, maximum amplitude, periodicity and phase are the parameters which define the model, and therefore the fit.

The maximum amplitude can be eliminated by normalising the recorded image values. For example, the maximum of the recorded values may be set to +1 and the minimum of the * )* : recorded value is for a respective pixel may be set to -1. * f

In case of the periodic pattern and the corresponding model, the phase which is identified by fitting the model to the recorded image values includes the information about the allocation of the respective pixel to the coordinate of the corresponding point of the projected pattern. This coordinate is referred to the coordinate system of the projecting pattern which differs from the coordinate system of the recorded image as viewed from the camera. Since the surface structure may have a depth profile (i.e. is not flat) the projected pattern at the different shift positions are used to identify the allocation of the pixel to the corresponding point of the projected pattern. If the projected pattern is periodic with respect to one direction lateral to the direction of wave propagation, the coordinate refers to a coordinate axis which extends in this lateral direction.

The invention also includes an evaluation device (in particular a computer using a corresponding computer program) which device is adapted to perform the evaluation of the images, in particular by fitting the model to image values, as described above and/or below.

In case of a small periodicity (i.e. a high frequency) of the projected pattern, shifting the pattern within one period and recording the respective images does not unambiguously yield in the information which point of the pattern or line of the pattern corresponds to the respective pixel of the recorded images. For example, if the phase of the parameterised model is identified by fitting the model to the recorded image values, the phase does not include the information to which period the corresponding point of the pattern belongs.

In order to overcome this problem, it is proposed to determine a range of the coordinate of the corresponding point of the pattern as an approximation. This range may be the range of one period of the periodic pattern. The range is identified in a manner which allows for unambiguous determination from the identified parameter or parameters which point of the pattern of electromagnetic waves correspond to the respective pixel (step f of the above listed steps of the method of capturing a surface structure). The above-mentioned * S..

.* *.. publication of E. Lilienbium et al. discloses several possibilities how the range of the * : coordinate can be determined. One method is the use of approximate values of the object surface. Another method is the so-called Gray code technique. Still a further method comprises the calculation of absolute phase measurements. Lilienbium also mentions a method called spatial phase unwrapping. According to Lilienblum, a more general method is the projection of several phase shift sequences which differ in their local period. There are two approaches of this more general method, the hierarchical and the number-theoretical approach. The publication of Lilienblum describes a specific embodiment of the number-theoretical approach. With respect to the present invention, it is preferred that additional patterns of electromagnetic waves are projected onto the surface of the object, wherein these additional patterns differ from the pattern which is shifted for recording respective images which correspond in each case to one shift position. For the additional patterns additional images are recorded in order to determine the range of the coordinate of the corresponding point as mentioned above. Preferably, the specific embodiment of the number-theoretical approach according to LUienbium is then performed.

Examples of the present invention will now be described with reference to the attached drawings. The figures of the drawings show: Fig. 1 schematically electromagnetic waves which are projected onto a surface of an object, which are reflected by the surface and which are received by a camera, Fig. 2 an arrangement similar to Fig. 1, wherein the Fig. shows a projector which generates the pattern of electromagnetic waves and a camera which receives the reflected electromagnetic waves, Fig. 3 schematically a pattern generator and a focussing lens which focuses the generated pattern onto the surface of an object, wherein the lens is in a neutral position, Fig. 4 the arrangement shown in Fig. 3, wherein the lens is in a shifted position, so that the pattern which is focused on the surface of the object is shifted compared to the neutral position of Fig. 3, Fig. 5 a frame construction for holding a camera, a projector, a shiftable length and a turnable table for fixing an object which is to be illuminated by a pattern of electromagnetic waves from the projector, so that the reflected waves can be received and recorded by the camera, Fig. 5a a top view on the arrangement shown in Fig. 5, Fig. 6 an exploded view of the shiftable length, a lens holder, which is movable relative to a holder support, a magnetic element, which is to be attached to the lens or to * : * the lens holder, and an electromagnet for generating a magnetic field in order to shift the length, Fig. 7 the lens holder and lens support of Fig. 6, wherein the magnetic element is connected to the lens which is held by the lens holder and wherein the lens holder is in a neutral position relative to the holder support, Fig. 8 the arrangement shown in Fig. 7, wherein the lens holder, the lens and the *.*...

* * magnetic element are shifted to the left, Fig. 9a measurements of the image value of camera pixel as a function of the shift position of the shifting device, wherein the measurement results for different pixels are shown, Fig. 9b the measurement values for a specific pixel, Fig. 9c the measurement values of Fig. 9b after normalisation of the image values, Fig. 9d the measurement values of Fig. 9c after applying a filter, Fig. 10 measurement values similarly to Fig. 9a to 9d for one pixel, wherein a fitted model function is also shown, and Fig. 11 a flow chart showing steps of obtaining and evaluating measurement values of a camera pixel as a function of the pattern shift of a pattern which is incident on the surface of an object The present invention can be applied in particular to digitizing object surfaces in three dimensions, e.g. for long-term preservation of their structure. In particular the structure of cultural heritage objects should be preserved for future generations. The present invention allows for capturing surface structures at high resolutions, in particular in the fine meso-scale, so that e.g. marble structure of sculptures or structures found on very small and movable objects like coins, can be captured at appropriate resolution. As described above, mechanical phase shifting instead of digital shifting allows for a higher resolution.

The mechanical phase shifting allows for smaller shifting steps (i.e. the resolution of the shift position is increased) and, therefore, artefacts, such as artificial lines in the captured structure, and effects of noise due to sharp contrasts in the projected pattern which can be observed with conventional digital shifting, can be avoided As will be described with reference to the attached figures, standard components, such as *.S.

a standard digital camera and a standard light projector can be used. Therefore, the costs * : * for the arrangement are comparatively low. For example, the projector may be a DLP (digital light processing) projector using an array of LEDs for light generation. By controlling the array of LEDs, the light pattern to be projected can be generated. *** *

Fig. 1 shows the basic principle of projecting a pattern of electromagnetic waves, in particular light, onto the surface of an object and taking a picture of the reflected *.*�** * * electromagnetic waves. The origin 0 of the waves is shown in the lower left of the Figure.

Usually, the generated waves diverge within a cone-shaped area, due to the lenses having a circular aperture which are typically used. A pattern P is generated, which is, for example, a sequence of dark and bright columns CO. Each of the columns corresponds to a plane in which the waves of the column propagate in the direction of the surface of the object 1. For one of the columns CO, the corresponding sectional area of the cone-shaped illuminated area and the plane of the column is indicated by L in Fig. 1.

The camera views the surface of the object I from a different viewing angle compared to the projector. In Fig. 1, the detector matrix 3 of the camera is shown. One point on the surface of the object 1 is referenced with TA. A straight line ST starting at surface point TA and intersecting the detector matrix 3 indicates the viewing direction of the camera.

Straight line ST intersects the matrix 3 at an intersecting point.

The surface of the object 1 comprises several wave-like structures. The image of one of these wave-like structures is shown at the detector matrix 3.

In order to evaluate the three-dimensional surface structure of the object 1, the corresponding plane of the corresponding pattern column can be identified, wherein the plane comprises the point on the surface of the object 1 which corresponds to a specific pixel of the camera detector or of the corresponding image taken by the camera detector 3. If the allocation between the pixel and the plane has been established, the three-dimensional location of the point on the surface of the object I can be calculated using additional geometric information, in particular the viewing angles of the projector and the camera and the distance of the projector and the camera. If this evaluation process is performed for many pixels and corresponding surface points of the object 1, the surface structure as a three-dimensional point cloud is obtained. *..

Fig. 2 shows a top view on an arrangement comprising a project 2, a camera 5 and an * intersecting point TA where an incident wave from the projector 2 is reflected in the direction of straight line ST towards the detector 3 of the camera 5. The incident ray is located within the cone-shaped area L in which electromagnetic waves propagate from * the projector 2 towards the object. Each of the rays, which propagates within cone-shaped area L, corresponds to one of the planes mentioned in connection with Fig. 1. In other * words: Each of the rays belongs to one of the columns of the pattern of electromagnetic ***S.* * * waves shown in Fig. 1. Consequently, each of the planes can be unambiguously described by a coordinate, for example having a value in the range between 0 and I, along a coordinate axis A which extends laterally to the direction of propagation, or, in other words, which extends in the direction in which the pattern of electromagnetic waves * is periodic. This direction is as well the direction in which the pattern can be shifted, as will be described in more detail later.

In the embodiment shown in Fig. 2, the projector 2 comprises a matrix 8 of light generating elements (for example LEDs). A lens or a combination of lenses and/or other optical elements (not shown in Fig. 2) within the projector directs the generated pattern towards the object within the propagation area L. An additional lens 4 is located within the propagation area L and is transmitted by the generated pattern of electromagnetic waves.

This lens 4 is shifted in the lateral direction mentioned above, for example in a direction which is parallel or in line with the coordinate axis A. The camera may comprise a matrix 3 of photoelectric elements as shown in the outlines of the camera 5.

Fig. 3 and 4 show the principle arrangement of a pattern generator 2, for example the projector shown in Fig. 1 or Fig. 2, wherein the pattern generator generates a pattern of electromagnetic waves to be incident on the surface of an object 1. In addition to the pattern generator, an additional optical element, in particular a lens 4, is provided which is shifted in order to shift the generated pattern in a direction lateral to the directions of propagation. This concept allows using standard pattern generators, such as DLP-projectors. However, it is also possible that the additional optical element is integrated in the same housing as the pattern generator. For example, a standard DLP-projector can be modified to also include in its housing a lens which is shiftable in order to shift the generated pattern. It would also be possible, to modify a standard pattern generator in such a manner that one of the optical elements or the only optical element of the generator is shiftable in order to produce the pattern shift. *4.. * * ****

Coming back to Fig. 3, different rays LO, LM, LU along which the electromagnetic wave generated by the pattern generator 2 propagate, are transmitted through optical element 4 and are directed, in particularfocused, on the surface of the object 1. In the specific embodiment shown in Fig. 3 and Fig. 4, the optical element is a focussing lens 4. For * * ** example, the distance between the lens 4 and the surface of the object 1, which is * indicated by reference sign d, is approximately equal to the focal length of the lens 4. If es....

* * the surface structure of the object us not planar, at least small deviations between the focal length and the distance d appear.

Fig. 3 shows the arrangement in a neutral position of the lens 4. The central ray LM is not redirected by the lens 4. In Fig. 4, lens 4 is shown in a shifted position, so that the central ray LM is redirected to another point on the surface of object 1. The shifted distance on the surface of the object 1 is denoted by PS. However, it is preferred to measure the shift in a coordinate system which is linked to the optical element instead of the object.

As shown in Fig. 5 and Fig. 5a a camera 5, a pattern projector 2, a lens holder 14 for holding a shiftable lens 4 and a holder 13 for attaching the object may be fixed to the same supporting structure 11, which is for example a frame-like structure comprising elongated structure elements which are attached to each other. The lens holder 14 can be fixed to the structure 11 and/or to the projector 2 via a holder support (not shown in Fig. 5a). The supporting structure 11 may be covered by a coating, for example black textile material which blocks the intrusion of external light and which avoids undesired reflection of light which is emitted from the pattern generator 2 to the coating or to the elements of the support structure 11.

Not limited to the embodiment shown in Fig. 5, the holder 13 for holding the object may be rotatable around a rotating axis which preferably extends in a direction towards a point in between the lens 4 and the camera 5. A motor 16 for generating the rotation of the holder 13 may be provided. Therefore, the object can be rotated and the method of the present invention can be performed at different rotational positions of the object. In this manner, problems caused by specular reflection and self shading of the object can be overcome.

However, in many cases, object rotation is not necessary. Therefore, the rotating device of Fig. 5 can be omitted. *S*S

** .. The relative positions and orientations of the generator 2, the lens 4, the camera 5 and the object or object holder 13 may be as principally shown in Fig. I to 4. This means that the different viewing angles of the pattern generator, which performs together with the at least : one shiftable optical element the pattern source, preferably intersect at an acute angle.

S

Fig. 6 shows a preferred embodiment of a shifting device which is adapted to shift an * optical element, in particular a focussing lens. The main components of the shifting device * * and the lens 4 are shown in an exploded view. In addition, the structure of the holder 23 and of a holder support 21 for supporting the holder 23 is shown in Fig. 7 and 8. The basic idea is that the holder has a cut-out for including the optical element to be shift, wherein the holder is connected to the holder support via thin parts of an integral element, so that the thin part can transfer the forces which are required for the holder support to support the holder, and, in addition, the thin parts or regions can elastically deform to allow for a shift of the optical element. The details of a specific embodiment will be described with reference to Fig. 7 and 8.

Fig. 6 also shows an embodiment of a preferred way of generating the shift of the optical element. A magnetic element 25, 26 which comprises in this case an electromagnet 26 and a electromagnet holder 25, is connected to the optical element holder 23 or to the optical element. Because of the exploded view, the connection cannot be seen from Fig. 6. The electric connections of the electromagnet 26 are not shown in the figures.

Furthermore, a magnet 27 (for example a static neodym magnet) is provided which produces the magnetic field which causes the magnetic element 25, 26 to shift. According to the preferred embodiment, the magnet is realized by a combination of two parts (for example two permanent magnets or one electromagnet and one permanent magnet) wherein the magnetic element 25, 26 is located in between the two parts of the electromagnet and can be moved within the space between the two parts. This construction having two parts, which are spaced, allows for highly homogeneous magnetic fields and high magnetic flux densities. Therefore, the corresponding shift forces can be generated and the shift position can be reproduced later with high precision.

This also significantly increases the resolution of the shifting steps and, therefore increases the depth resolution of the captured surface structure which can be achieved.

A controller driving the shifting device may be integrated in a computer system which also controls the projector and/or the evaluation of the images obtained by the camera. The shifting device may serve two goals. First, the optical element (in particular the focusing lens) itself may improve (shorten) the projector's depth of focus and, therefore, allows for a smaller distance between projector and object. This increases the resolution in depth * direction. Second, the projected image can be moved along one axis at sub-period * *** accuracy over a range of one period length of the projected pattern. With the precise shifting of the pattern, high frequency patterns (patterns having small period lengths) can * be used for the process of obtaining reflected images, which amplifies luminance changes measured for small surface structures. In addition, smaller pattern wavelengths increase the system's robustness against undesired scattered and reflected waves which do not directly propagate from the projector to the object and/or are not reflected from the object * to the camera.

It has been described above that the magnetic element which is connected to the lens or lens holder comprises an electromagnet and that the fixed, non-shiftable part of the shifting device comprises a permanent magnet. However, in contrast to the embodiment shown in Fig. 6, 7 and 8 the magnetic element 25, 26 may comprise a permanent magnet 26 instead of an electromagnet, and the fixed part 27 of the shifting device may comprise an electromagnet or two electromagnets. In particular, the two parts of non-shiftable element of the shifting device may be electromagnets.

The linear motor which is realized by the shiftable magnetic element and by the non-shiftable parts of the shifting device may be constructed and controlled in the same or similar manner as a so-called voice coil motor of a standard hard disc drive for storing digital data. Instead of the reading head or writing head of the hard disc drive, the lens holder is moved by the linear motor.

Preferably, the electric voltage which is applied to the electromagnet of the shifting device and, in addition, the electric current through the electromagnet are controlled in a closed loop arrangement, i.e. they are regulated. A closed loop control ensures that particular shift positions are reached at high position and can be reproduced. Although the voltage and current may be controlled, the shift is typically not proportional to the current, since the mechanical forces between the holder and the holder support are not proportional to the shift in most cases. It is preferred to calibrate the shifting device so that the required electric current through the electromagnet for each desired shift position is known. For example, the control can be realized using a computer program and the program is adapted to take the results of the calibration into account.

If the relation between the electric current through the electromagnet of the shifting device and the shift position is known, the electric current can directly be used as a measure of * ** the shift position, and thereby as a measure for the pattern shift. In this case, there is no need for a measurement of the coordinate of the pattern shift or the coordinate of the * holder of the optical element.

Fig. 7 shows the holder 23 which has essentially a rectangular outline. However, the holder 23 is connected to two interconnecting elements 36a, 36b via a thin region 34a, 34b and the interconnecting elements 36a, 36b, which extend in parallel to opposing surfaces of the holder 23, are connected to the holder support 21 via in each case one thin region 33a, 33b. The holder support 21, the interconnecting elements 36a, 36b and the holder 23 are regions of an integral part, which may be made, for example, of brass or another metal. The thin regions may be produces by drilling holes 31a, 31b, 32a, 32b into the integral part so that a part of the wall of the hole forms the thin region 33, 34.

At the bottom side of the holder 23 in Fig. 7 and 8, the magnetic element 25, 26 is connected to the holder 23. Small circles which include a cross and which are denoted by reference sign MA indicate the flux lines of the magnetic field which interacts between shiftable magnetic element arid the non-shiftable parts of the shifting device. As a result of the magnetic flux lines, which are perpendicular to the image plane of Fig. 7 and 8, the magnetic element is forced to the left (as indicated by an arrow) in the example of Fig. 8.

The result of the force is a shift of the magnetic element and the holder 23 which is connected to the magnetic element. The shift is possible, since the thin regions 33, 34 are elastically deformed. The holder support 21, which is fixed to the pattern generator (for example using fixing holes 30a, 30b, 30c) remains in place, whereas the interconnecting elements 36a, 36b are moved to the position shown in Fig. 8.

Fig. 9a shows picture values of four different pixels positions of the detector matrix of the camera. Each pixel of the detector matrix corresponds to one pixel of the images which are produced by the camera. For example, the detector element in the third column and the fourth row of the detector matrix produces the image pixels at (again) the third column and the fourth row. The measurement values or picture values shown in Fig. 9a, 9b, 9c and 9d are indicated by crosses or rectangles. The lines which interconnect the neighbouring measurement values only serve to increase visibility. These lines are not the result of fitting a model into the measurement values. However, a sinusoidal behaviour of the measurement values as functions of the shift position is shown for all four pixels, although deviations from the precise sinusoidal behaviour appear.

a.....

* The horizontal axes in Fig. 9a -9d cover a range of shift positions which corresponds to one period of the periodic pattern which is incident on the surface of the object. The scale of the range is from -2000 to 2000 corresponding to about 4000 steps of shifting the optical element and thereby the pattern. However, only about 40 measurement points are shown for each pixel.

Fig. 9b shows measurement values similarly to the measurement values of Fig. 9a for one pixel. The apparent deviation from the precise sinusoidal behaviour occurs for all pixels at the same shift position as can be seen in the region of Fig. 9a which is marked by a narrow vertical marking. Since the four pixels correspond to surface points of the object which are located at positions that are illuminated or not illuminated by different parts of the incident pattern, the properties of the camera are likely the reason for the effect. To correct for this effect, each image was normalized to the average luminance of the first image in the sequence of images which were taken for the sequence of shift positions.

The result of the normalization is shown in Fig. 9c. Compared to the behaviour in Fig. 9b, an improved sinusoidal behaviour of the measurement is shown.

Furthermore, luminance profiles of each pixel are filtered with a box filter having a size of five pixels (i.e. corresponding to a rectangular area of five times five pixels). The result of the filtering is shown in Fig. 9d. A nearly perfect sinusoidal behaviour can be observed.

Fig. 10 shows the result of fitting a model function (a sine function) in the measured values for one pixel, for example the measurement values of the curve in Fig. 9a which has the largest measurement values at shift position zero. The result of the fit is the phase of the sine function at a predetermined shift position, for example shift position zero. From this parameter (the fitted phase) the allocation of the pixel to the coordinate of the projected pattern at the predetermined shift position can be determined. Using the allocation and additional geometric information, the three-dimensional coordinates of the surface point on the surface of the object which corresponds to the pixel can be calculated.

Fig. ii shows a flowchart illustrating a preferred embodiment of the method of capturing the surface structure of an object. In step Si a calibration of the arrangement comprising the pattern generator, the at least one shiftable optical element and the camera is performed, for example using a calibration object instead of a real measurement object.

For example, a calibration object having a planar surface can be used for calibration. The calibration is performed for one shift position only, for example for shift position zero, which is the so-called neutral shift position of the shifting device. As a result of the calibration, the geometric quantities are determined which are necessary in order to calculate the three-dimensional position of a point on the surface of a real measurement object. In addition, the calibration results in the information which is needed to compensate for optical distortions and aberrations of the optical components of the projector, in particular the at least one shiftable optical element and the at least one camera.

In the following step S2, a pattern of electromagnetic waves is projected onto the surface of the object, wherein the shifting device is in the neutral shift position. Alternatively, the first projected pattern may be projected while the shifting device is at another predetermined shift position.

In the following step S3, the shifting device is moved to another shift position. Next step is again the recording of an image by the camera (or by plural cameras), step S2.

Steps S2 and S3 are repeated until sufficient image have been recorded by the camera(s), wherein each image corresponds to a different shift position. Then, the method continues with step S4 in which a model is fitted to the image values for each pixel.

The result of the fit is used in step S5 for determining the allocation between the pixel and the coordinate of the pattern which has illuminated or not illuminated the corresponding surface point of the object which corresponds to the pixel.

Using the result of the allocation of step S5, the three-dimensional location of the point on the surface of the object is determined iii step S6, for example using a triangulation and in particular using the results of the calibration of step Si.

* ***** * I The process of capturing images using projected patterns incident on the object is preferably performed in two stages, one for capturing phase-shifted images for three different wavelengths according to the method of Lilienblum et al. (see above), and one for capturing lens-shifted images, for example as described before with reference to Fig. * Ii. The first stage (which may be performed before and/or after steps Si to S4 of the * 1 method according to Fig. ii) assigns surface points to a certain period of the periodic pattern and resolves the uniqueness problem of the second stage. The result may be a globally unique but coarse phase value w of the periodic pattern. Using the approximation of the first stage, steps S5 to S6 of the second stage can be performed, i.e. the allocation * of the pixels to a coordinate of the pattern and the actual reconstruction of all surface points seen by the camera. Optionally, a surface mesh is constructed between the * samples of the reconstructed point cloud. As each 3D point correlates with the 2D image point it was reconstructed from, and as all 2D image points lie in a regular grid, the connectivity information applies to the 3D point cloud as well, and triangulation can be done in a straightforward way by inserting two triangles between any rectangular neighbourhood of four points.

Since a periodic function or model is fitted to the image values of each pixel of the camera it is sufficient to calibrate the arrangement for only one or only a few different shift positions of the shifting device.

Determining the phase of the periodic pattern as a parameter of the model requires the model to model the periodicity of the pattern and represent the captured data well. In order to facilitate the model fitting process, the measurement values are normalized, preferably to values between -1 and 1, as described above. Therefore, there is no parameter in the model needed which models the maximum amplitude of the periodic pattern.

The following model function may be used for the fit. It comprises the parameters frequency f (the reciprocal of the period length of the pattern) and phase 4. The quantity b(x) is the image value dependent on the shift position x: b(x) = a sin(ço + 2*) S.....

* Since the amplitude a is normalised, it can be omitted. However, alternatively, the starting value of the amplitude can be set to 1, but the amplitude can be fitted as a third parameter. The parameters are determined by optimization using the Levenberg-Marquardt-Algorithm for nonlinear least squares problems. Appropriate starting values are used for a robust optimization. For the phase, the value is left variable. The starting value for the amplitude is set to a = 1 due to the normalization, the frequency may be set empirically. The new frequency resulting from the fitting process is then used as a starting value of a further optimization.

The relative and more accurate phase determined by pattern shifting is now combined with an absolute and coarser phase resulting from classical phase shifting (see for example the above-mentioned embodiment where the method of Lilienbium was performed as a first stage of the method). The phase can be transformed to the range [O..11. The value of the phase comprises the information about the corresponding coordinate of the periodic pattern and can be interpreted as pointing to a range of the coordinate of the projected pattern, in particular to a range having the size of one pixel column of the pattern generating matrix of the pattern generator. The phase can be interpreted as pointing to a local position within the wave length X and has the same range as the phase o, but is in this case pointing to a location between the bounds of the wave length Awhich is identified by the coarser phase,. The following formula combines the two phases: w i "i a-,% co-ço+ +ç�, co,çoE[O...1J wherein d denotes the globally unique phase. To combine the two phases and benefit both from global uniqueness and high accuracy, the phase, is first transformed to point to a multiple of the lens-shifting wave length ?. by relating it to the projection width and dividing by the wave length ?. The non-integral remainder is then removed by subtracting now that the two phases are expressed using the same unit, and the result is rounded to the next integral multiple of the wave length. Finally, the accurate phase 4 is added, and the coordinate of the projected pattern is obtained by multiplying with the lens-shifting wave length. Subtraction of the phases using a common unit rather than more intuitively just rounding the course phase is necessary since especially at boundaries of wavelengths, image noise can lead to erroneous determination of the coordinate of the pattern. Since, as a result, the absolute coordinate for each camera pixel is known, the 3D coordinates of the corresponding surface point can be identified according to the basic principles of structured light reconstruction, for example as explained above with reference to Fig. 1. 0 *

Claims

Patent C'aims 1. A method of capturing a surface structure of a surface of an object (1) wherein a) a pattern of electromagnetic waves is projected from a pattern source (2, 4) onto the surface of the object (1), so that a projected pattern is incident on the surface of the object (1), b) a first reflected image, which is a result of a reflection of the projected pattern by the surface, is received by a camera (5) and is recorded, wherein the first image is recorded as a digital image comprising pixels, c) the projected pattern is shifted in a direction lateral to a projection direction into which the electromagnetic waves propagate, so that same parts of the projected pattern are incident at different locations at the surface of the object (1) compared to the projected pattern before shifting the pattern, wherein the pattern is shifted by moving at least one part of the pattern source (2, 4), d) a second reflected image, which is a result of a reflection of the projected pattern by the surface, is received by a camera (5) and is recorded, wherein the second image is recorded as a digital image comprising pixels, e) steps c) and d) are repeated, so that further digital images are recorded which correspond to different shift positions of the projected pattern, f) for a plurality of pixels of the first image, an allocation of the respective pixel to a coordinate of a corresponding point of the pattern of electromagnetic waves in a coordinate system of the pattern is determined by evaluating the first image, the second image and the further images, g) using the allocation and using geometric relations of the position and orientation of the pattern source (2, 4) and of the camera (5), the surface structure is determined. * *
2. The method of the preceding claim, wherein the pattern source (2,4) comprises an optical element, which reflects and/or transmits the pattern of electromagnetic waves to the surface of the object (1), and wherein the optical element is shifted and/or redirected in order to shift the projected pattern in step c). 4***
3. The method of one of the preceding claims, wherein in step f) for each of the pixels, a *:* parameterized model is fitted to the recorded image values of the first, second and further images, which image values are a function of the shift position, so that at least one fitted parameter is obtained for each of the pixels in step f), and wherein the fitted parameter is used to determine the allocation of the pixel to the coordinate of the corresponding point of the pattern of electromagnetic waves.
4. The method of the preceding claim, wherein the projected pattern is a pattern which is periodic with respect to the lateral direction into which the projected pattern is shifted.
5. The method of the preceding claim, wherein a range of the coordinate of the corresponding point of the pattern of electromagnetic waves is determined as an approximation and the range is used in step f) for the determination of the allocation of the pixel to the coordinate of the corresponding point.
6. An arrangement of capturing a surface structure of a surface of an object (1), the arrangement comprising: a) an electromagnetic wave pattern projector comprising a pattern source (2, 4) for projecting the pattern of electromagnetic waves onto the surface of the object (1), so that a projected pattern is incident on the surface of the object (1), b) at least one camera (5) for receiving and recording reflected images, which are a result of a reflection of the projected pattern by the surface, wherein the camera (5) is adapted to produce digital images comprising pixels, c) a shifting device (25, 26, 27) for shifting the projected pattern in a direction lateral to a projection direction into which the electromagnetic wave pattern propagates, so that same parts of the projected pattern are incident at different locations at the surface of the object (1) compared to the projected pattern before shifting the pattern, wherein the shifting device (25, 26, 27) is coupled to at least one part of the pattern source (2, 4) to move the part and thereby shift the pattern, d) an evaluation device for evaluating reflected images, which correspond to different p..' shift positions of the projected pattern, wherein the evaluation device is adapted to determine, for a plurality of pixels of the recorded images, an allocation of the respective pixel to a coordinate of a corresponding point of the pattern in a coordinate system of the pattern by evaluating the reflected images and is adapted to determine, using the allocation and using geometric relations of the : position and orientation of the pattern source (2, 4) and of the camera (5), the surface e:..s structure of the object (1).
7. The arrangement of the preceding claim, wherein the pattern source (2, 4) comprises an optical element (4), which is adapted to reflect and/or transmit the electromagnetic wave pattern to the surface of the object (1), wherein the shifting device (25, 26, 27) is adapted to shift and/or redirect the optical element (4) in order to shift the projected pattern in the lateral direction 8. The arrangement of the preceding claim, wherein the shifting device (25, 26, 27) comprises a holder (23) for holding the optical element (4) and comprises a holder support (21), wherein the holder (23) is moveable relative to the holder support (21) in order to allow a shift of the holder (23) and the optical element (4), wherein the holder (23) and the holder support (21) are connected to each other by an elastically deformable material which allows for the relative movement.9. The arrangement of the preceding claim, wherein the holder (23) and the holder support (21) are regions of an integral element (14) and wherein at least one further region (33, 34) of the element (14) which connects the holder (23) with the holder support (21) has a reduced thickness compared to the holder (23) and the holder support (21) in order to allow for elastic deformation of the further region.10. The arrangement of one of the preceding claims, the evaluation device is adapted to fit, for each of the plurality of pixels of the recorded images, a parameterized model to the recorded image values of the recorded images, which image values are a function of the shift position, so that at least one fitted parameter is obtained for each of the pixels, and wherein the fitted parameter is used by the evaluation device to determine the allocation of the pixel to the coordinate of the corresponding point of the pattern of electromagnetic waves.11. The arrangement of the preceding claim, wherein the projected pattern is a pattern S.. S which is periodic with respect to the lateral direction into which the projected pattern is shifted.12. The arrangement of the preceding claim, wherein the evaluation device is adapted to * determine a range of the coordinate of the corresponding point of the pattern of electromagnetic waves as an approximation and to use the range for the : determination of the allocation of the pixel to the coordinate of the corresponding point.S S....S SAMENDMENTS TO CLAIMS HAVE BEEN FILED AS FOLLOWED1. A method of capturing a surface structure of a surface of an object (1) wherein a) a pattern of electromagnetic waves is projected from a pattern source (2, 4) onto the surface of the object (1), so that a projected pattern is incident on the surface of the object (1), b) a first reflected image, which is a result of a reflection of the projected pattern by the surface, is received by a camera (5) and is recorded, wherein the first image is recorded as a digital image comprising pixels, c) the projected pattern is shifted in a direction lateral to a projection direction into which the electromagnetic waves propagate, so that same parts of the projected pattern are incident at different locations at the surface of the object (1) compared to the projected pattern before shifting the pattern, wherein the pattern is shifted by moving at least one part of the pattern source (2, 4), d) a second reflected image, which is a result of a reflection of the projected pattern by the surface, is received by a camera (5) and is recorded, wherein the second * image is recorded as a digital image comprising pixels, e) steps C) and d) are repeated, so that further digital images are recorded which * correspond to different shift positions of the projected pattern, f) for a plurality of pixels of the first image, an allocation of the respective pixel to a coordinate of a corresponding point of the pattern of electromagnetic waves in a * * coordinate system of the pattern is determined by evaluating the first image, the second image and the further images, g) using the allocation and using geometric relations of the position and orientation of the pattern source (2, 4) and of the camera (5), the surface structure is determined, wherein in step f) for each of the pixels, a parameterized model is fitted to the recorded image values of the first, second and further images, which image values are a function of the shift position, so that at least one fitted parameter is obtained for each of the pixels in step f), and wherein the fitted parameter is used to determine the allocation of the pixel to the coordinate of the corresponding point of the pattern of electromagnetic waves.2. The method of the preceding claim, wherein the pattern source (2, 4) comprises an optical element, which reflects and/or transmits the pattern of electromagnetic waves to the surface of the object (1), and wherein the optical element is shifted and/or redirected in order to shift the projected pattern in step c). $ 253. The method of the preceding claim, wherein the projected pattern is a pattern which is periodic with respect to the lateral direction into which the projected pattern is shifted.4. The method of the preceding claim, wherein a range of the coordinate of the corresponding point of the pattern of electromagnetic waves is determined as an approximation and the range is used in step f) for the determination of the allocation of the pixel to the coordinate of the corresponding point.5. An arrangement of capturing a surface structure of a surface of an object (1), the arrangement comprising: a) an electromagnetic wave pattern projector comprising a pattern source (2, 4) for projecting the pattern of electromagnetic waves onto the surface of the object (1), so that a projected pattern is incident on the surface of the object (1), b) at least one camera (5) for receiving and recording reflected images, which are a result of a reflection of the projected pattern by the surface, wherein the camera (5) is adapted to produce digital images comprising pixels, c) a shifting device (25, 26, 27) for shifting the projected pattern in a direction lateral to a projection direction into which the electromagnetic wave pattern propagates, so that same parts of the projected pattern are incident at different locations at the surface of the object (1) compared to the projected pattern before shifting the pattern, wherein the shifting device (25, 26, 27) is coupled to at least one part of the pattern source (2, 4)to move the part and thereby shift the pattern, d) an evaluation device for evaluating reflected images, which correspond to different shift positions of the projected pattern, wherein the evaluation device is adapted to determine, for a plurality of pixels of the recorded images, an allocation of the respective pixel to a coordinate of a corresponding point of the pattern in a coordinate system of the pattern by evaluating the reflected images and is adapted to determine, using the allocation and using geometric relations of the position and orientation of the pattern source (2, 4) and of the camera (5), the surface structure of the object (1) and wherein the evaluation device is adapted to fit, for each of the plurality of pixels of the recorded images, a parameterized model to the recorded image values of the recorded images, which image values are a function of the shift position, so that at least one fitted parameter is obtained for each of the pixels, and wherein the fitted parameter is used by the evaluation device to determine the allocation of the pixel to the coordinate of the corresponding point of the pattern of electromagnetic waves. p6. The arrangement of the preceding claim, wherein the pattern source (2, 4) comprises an optical element (4), which is adapted to reflect and/or transmit the electromagnetic wave pattern to the surface of the object (1), wherein the shifting device (25, 26, 27) is adapted to shift and/or redirect the optical element (4) in order to shift the projected pattern in the lateral direction 7. The arrangement of the preceding claim, wherein the shifting device (25, 26, 27) comprises a holder (23) for holding the optical element (4) and comprises a holder support (21), wherein the holder (23) is moveable relative to the holder support (21) in order to allow a shift of the holder (23) and the optical element (4), wherein the holder (23) and the holder support (21) are connected to each other by an elastically deformable material which allows for the relative movement.
8. The arrangement of the preceding claim, wherein the holder (23) and the holder support (21) are regions of an integral element (14) and wherein at least one further region (33, 34) of the element (14) which connects the holder (23) with the holder support (21) has a reduced thickness compared to the holder (23) and the holder support (21) in order to allow for elastic deformation of the further region.
9. The arrangement of one of claims 5 to 8, wherein the projected pattern is a pattern which is periodic with respect to the lateral direction into which the projected pattern is shifted. .
10. The arrangement of the preceding claim, wherein the evaluation device is adapted to determine a range of the coordinate of the corresponding point of the pattern of electromagnetic waves as an approximation and to use the range for the determination of the allocation of the pixel to the coordinate of the corresponding point.*.:r: INTELLECTUAL . ... PROPERTY OFFICE 27.Application No: GB1010818.1 Examiner: Mr Mike Walker Claims searched: ALL Date of search: 27 August 2010 Patents Act 1977: Search Report under Section 17 Documents considered to be relevant: Category Relevant Identity of document and passage or figure of particular relevance to claims X 1,2,6,7 US2009/0225333 Al (BENDALL) for example, see paragraphs 7 & 8; claims 1 to 4 X 1,2,6,7 W02008/1 16917 Al (S.O.I. TEC) see abstract; p.7,1.14 to p.10,l.2 X 1,2,6,7 W02008/052092 A2 (D4D TECHNOLOGIES) see abstract; p.6,11. 1-30 X 1,2,6,7 W02004/046645 A2 (SOLVISION) p.8,1.7 to p.9,1.30 X 1,2,6,7 EP1884740A2 (MITSUBISHI) see abstract; paras 13 to 19; 32 to 39 X 1,2,6,7 U52002/0072874 Al (MICHAELIS) whole document X 1,2,6,7 WO01/06210A1 (SOLVISION) see abstract X 1,2,6,7 US6373963 Bl (DEMERS) whole document X 1,6 US2004/0105580A1 (HAGER) see abstract; p.2, para 22 to p.3, para 30 Categories: X Document indicating lack of novelty or inventive A Document indicating technological background and/or state step of the art.Y Document indicating lack of inventive step if P Document published on or after the declared priority date but combined with one or more other documents of before the filing date of this invention.same category.& Member of the same patent family E Patent document published on or after. hut with priority date earlier than, the filing date of this application.Intellectual Property Office is an operating name of the Patent Office www.ipo.gov.uk t::r: INTELLECTUAL . ...* PROPERTY OFFICEField of Search:Search of GB, EP. WO & US patent documents classified in the following areas of the UKCX Worldwide search of patent documents classified in the following areas of the IPC GO1B The following online and other databases have been used in the preparation of this search report EPODOC, WPI International Classification: Subclass Subgroup Valid From None Intellectual Property Office is an operating name of the Patent Office www.ipo.gov.uk