EP2852814A1

EP2852814A1 - Colour coding for 3d measurement, more particularly for transparent scattering surfaces

Info

Publication number: EP2852814A1
Application number: EP13720338.6A
Authority: EP
Inventors: Anton Schick
Original assignee: Siemens AG
Current assignee: Siemens AG
Priority date: 2012-07-25
Filing date: 2013-04-26
Publication date: 2015-04-01
Also published as: JP6005278B2; US20150176983A1; JP2015522826A; WO2014016001A1; KR20150034289A; CN104583714B; CN104583714A; US9404741B2; KR101651174B1

Abstract

A device and a method for determining three-dimensional surface coordinates of an object by means of optical colour triangulation are proposed, wherein all lines (3) of a colour fringe pattern (1) in each case have a width (BR) set in such a way that, in a recorded image (7) of the line (3), all contrast maxima (CMax), of all spectral components of a line (3) are equal to a minimum contrast value (CMin). Proceeding from an invariable smallest width of a pattern line (3a) with a spectral component of highest contrast, further lines (3) can be correspondingly widened. The invention is particularly advantageously suitable for a 3D measurement for biological tissue.

Description

description

Color coding for 3D measurement, especially for transparent scattering surfaces

Field of the present invention are devices for determining three-dimensional surface coordinates of an object by means of optical color triangulation and corresponding methods.

The measurement of the third dimension of a surface is becoming increasingly important in many areas of medical technology. For example, in minimally invasive surgery, the lack of size and distance estimation must be replaced by direct vision through measuring techniques. In addition, surface data obtained, for example, within the abdomen during surgery, can be matched with data collected by other diagnostic methods, such as magnetic resonance, computerized tomography, or ultrasound techniques, to organs or diseased tissue better to recognize or locate. Likewise, changes due to a "new" position of the patient during an operation or due to periodic changes in position caused, for example, by respiration, should be taken into account. Conventionally, there are many 3D measurement methods, such as phase-coded active triangulation or laser scanning, which are in principle suitable for the application described. However, these methods are tailored to the measurement of non-transparent surfaces, as they often occur in industrial metrology. However, organic tissues have a much more complex interaction with light, are partially transparent to waves and have a light scattering capacity in the volume, which changes the structure of a projected pattern significantly and its recognition in a camera image to SD data reconstruction in active triangulation. sword. This creates gaps in the 3D surface or the measurement uncertainty can rise sharply.

Conventionally, in the field of dental medicine, monochromatic light is used for phase triangulation and a white color is sprayed to prevent the penetration of light into the enamel. This is an unpleasant additional process step for the patient, which impairs the acceptance of the method.

A method more suitable for medical 3D applications is disclosed in WO 01/48438. This disclosure suggests that the design of a two-dimensional color pattern consisting of colored pattern elements to provide a particularly compact and therefore interference-proof color pattern for coding. The aim is to determine a displacement position for a pattern element in the image recording of the two-dimensional color pattern projected onto an object. By subsequent triangulation with a known position of the projector in a camera, the three-dimensional data of an object point can be calculated.

Originally, Color Coded Triangulation (CCT) was also developed for medical applications and has significant advantages in the measurement of semi-transparent diffuse scattering media. Applications may be a three-dimensional measurement of the human face for biometric use in the cosmetic industry, a three-dimensional earcuff scan to produce optimally adapted hearing aids using this data, or a direct scan of the surface of the ear canal with a specially designed CCT. Scanner. The advantage of this measurement method is that it has many advantages of 3D measurement by means of active triangulation and is also very fast and comparatively robust. Fast means that it is able to measure in real time since only one image capture is needed to reconstruct the three-dimensional datasets. Robust means that by using the color coding of the projected pattern, a relatively good data reconstruction is possible even for biological surfaces, since it searches for color transitions or color edges during decoding and dispenses with the purely intensity-based data reconstruction.

In color triangulation CCT, color stripe patterns with identical stripe widths for all colors have hitherto been selected. This is a useful approach for objects that have no or very little depth of penetration into the object medium and the value of the modulation transfer function is almost the same for all colors (wavelengths of light). For biological objects, the modulation transfer function (MTF) breaks down with the degree of volume scattering, especially at high spatial frequencies. The volume spread tends to increase with wavelength. In fact, the effect also depends on the layer structure of, for example, the human skin. For the design of conventional color samples, corresponding variations are not considered.

In the dental field, the surface of teeth is scanned for the adaptation of precisely fitting crowns, etc. Here, too, it has been shown that the volume scattering makes it difficult to register valid measuring points.

It is an object of the present invention to provide an apparatus and a method for three-dimensional measurement or 3D measurement of transparent, in particular partially transparent, and scattering, in particular diffusely scattering, surfaces with effectively reduced contrast reduction and effectively increased measurement accuracy compared to conventional solutions , In particular, for this purpose, for example in the case of biological tissue, wavelength-dependent penetrating power into the surface-forming materials and resulting volume scattering should be taken into account. The object is achieved by a device according to the main claim and a method according to the independent claim.

According to a first aspect, an apparatus for determining three-dimensional surface coordinates of an object by means of optical color triangulation is claimed, comprising a projector device for illuminating the object with a set color stripe pattern, wherein the color stripe pattern extends along an axis and perpendicular to each with one to neighboring lines there is a different selection of spectral components of the projected light; a detecting means arranged in known relative position to the projector means for taking an image of the object onto which the color stripe pattern has been projected once; a computer device for calculating the three-dimensional surface coordinates by detecting the selection of the spectral components of a respective line and detecting a respective transition between two adjoining lines, wherein for all lines, by means of the projector device, the width of a respective projected line corresponding to the volume scattering effects of the spectral components selected for the line is set in the recorded image of the line all contrast maxima of all spectral components of this line are equal to a contrast minimum value.

According to a second aspect, a method for determining three-dimensional surface coordinates of an object by means of optical color triangulation is claimed with the following steps. Illumination of the object with a set color stripe pattern carried out by means of a projector device, wherein the color stripe pattern extends along an axis and consists of perpendicular lines with a different selection of spectral portions of the projected light relative to adjacent lines; by means of a detection device arranged in a known relative position to the projector device, picking up an image of the object onto which the color strip pattern was projected once; calculating the three-dimensional surface coordinates by means of computer means by recognizing the selected spectral components of a respective line and detecting a respective transition of two adjacent lines and adjusting for all lines, by means of the projector means the width of a respective projected line corresponding to the volume scattering effects of the ones selected for the line Spectral components such that in the recorded image of the line all contrast maxima of all spectral components of this line are equal to a contrast minimum value. It has been advantageously recognized that by means of the design of the width of the color stripes or lines according to the degree of volume scattering, this can be compensated. In this way one achieves the same for all colors or spectral components used

Contrast value. In this case, it is advantageous that the number of measured values which can be recognized as valid increases for biological objects. Hole areas in the 3D image are avoided. This applies to pixels along a critical line that is only weakly detectable.

Further embodiments are claimed in conjunction with the subclaims. According to an advantageous embodiment, for lines having in each case at least two selected spectral components, the width of the respective line can be adjusted correspondingly to the volume scattering effects of the spectral components selected for the line in such a way that regions of maximum gradient of the contrast profiles of these spectral components coincide in the recorded image of the line.

According to a further advantageous embodiment, the contrast minimum value may be the value of the contrast transfer function of a pattern line with a fixed minimum width and a single spectral component consisting of a short-wave color which produces a relatively smallest volume scattering effect. According to a further advantageous embodiment, starting from the invariable smallest width of the pattern line, the widths of the other lines may have remained unchanged or enlarged starting from this smallest width.

According to a further advantageous embodiment, the color causing a relatively small volume scattering effect can be blue.

According to a further advantageous embodiment, at least one spectral component can correspond to a single color. According to a further advantageous embodiment, the

Projector device produce the selected spectral components by mixing the individual colors red, green, blue.

According to a further advantageous embodiment, the detection device can have a red-green-blue filter.

According to a further advantageous embodiment, the invariable smallest width can be at least -mm.

The present invention will be described in more detail by means of exemplary embodiments in conjunction with the figures. Show it :

Figure 1 shows an embodiment of a conventional color stripe pattern;

Figure 2 modulation transfer functions different

Systems; Figure 3a shows another embodiment of a conventional color stripe pattern; FIG. 3b shows an exemplary embodiment of a color stripe pattern according to the invention;

FIG. 4 shows modulation transfer functions of various individual colors;

FIG. 5a shows contrast profiles of a conventional line consisting of mixed colors; FIG. 5b shows an illustration (color-dependent broadening) of FIG

Line in the eye after scattering on a volume with color-dependent scattering power;

FIG. 5c contrast profiles of the invention set

Line.

1 shows an embodiment of a conventional color stripe pattern 1. The prerequisite for triangulation is that a pattern projected onto a surface is only deformed by the surface shape, since the information about the three-dimensional shape lies only in this deformation and not in addition due to penetration is changed in materials in its structure and its contrast. This causes increased measurement uncertainty, missing pixels and increased susceptibility to external light. Figure 1 shows a conventionally used color-coded pattern for the CCT. The color stripe pattern 1 extends along an axis x. The color pattern 1 consists of lines 3 perpendicular to this axis x with a selection of spectral components of the projected light which is different in each case to adjacent lines. Three-dimensional surface coordinates can be determined by detecting the selection of the spectral components of a respective line and detecting a respective transition 5 of two adjacent lines 3. Traditionally, color stripe patterns with identical fringe patches have been chosen for all colors. If one were to project such a color stripe pattern 1, each with identical line widths, onto a surface, which diffuses the color neutral in all spatial directions, then the contrast would be equally good for all wavelengths and colors in an image taken with a detector or camera, the color transitions 5 could be easily detected. However, color-selective different penetration of light into a body as well as different scattering ability change the pattern to be imaged in sharpness and contrast. Figure 2 shows these influences. On the left side of Figure 2 is shown in the arrow direction, as a line pattern is detected on an object in the captured image. A stripe pattern 1 is transformed into a corresponding image 7. Reference numeral 9 shows the corresponding signal modulation by the object. Reference numeral 11 shows the resulting modulation due to the contrast losses in the captured image 7. Technically, this means that an original modulation due to the modulation transfer function (MTF), for example, of biological tissue, is degraded. FIG. 2 shows, on the right side, two modulation transfer functions which, once for a good system 15 and for a degraded system 17, represent contrast profiles as a function of the spatial frequency. The spatial frequency is defined by the number of line pairs per mm. A modulation transfer function MTF can also be named as a contrast transfer function.

FIG. 2 shows as functions on the right the modulation transfer function 15 of a good system and on the left a modulation transfer function 17 which is degraded as a result of volume dispersion compared to the modulation transfer function 15.

The following figures explain the idea of the present invention on the basis of exemplary embodiments. For the development of new color code patterns for optical triangulation, especially on biological surfaces, a precise knowledge of the optical parameters and a description of the light propagation in the tissue is necessary. The optical parameters of Tissues are wavelength dependent and include an absorption coefficient, a scattering coefficient, an angular distribution of scattering and a refractive index. The angular distribution of the scattering is characterized for example by a g-factor and a phase function. However, in a rough approximation, it can be said that in homogeneous diffusely scattering media, long-wave light penetrates more strongly into the material and causes a larger volume scattering effect. This means that when an infinitesimally small spot of light is projected onto the object, the non-directly elastically reflected photons penetrate the medium, undergo a variety of photon scattering processes, which can be both elastic and inelastic, and reach the surface at other locations. There is a broadening of the visible light spot, and more so the longer the wavelength of the light. Decisive in the SD measurement (three-dimensional detection) by means of color triangulation, however, is primarily the recognition of the color or line and the detection of the color transition or the respective color edge. This is partly prevented by the volume scattering effect, as the contrast decreases from ((Max - Min) / (Max + Min)) and the signal-to-noise ratio decreases accordingly. Figure 3a shows another embodiment of a conventional color stripe pattern. There are alternately red lines R and blue lines B arranged one behind the other. Each of the lines 3 each has a uniform width Br which is the same for all lines 3. Below are shown contrast curves for each of the lines 3 after scattering on a volume spreader. These contrast profiles can be detected by means of a detection device of the device according to the invention. Each of these contrast gradients C shows an increase in contrast up to a contrast maximum CMax and a subsequent fall in a respective contrast curve. The contrast curves are axisymmetric. FIG. 3 a clearly shows that a volume scattering effect for the color blue B is smaller than the volume scattering effect of the color red Correspondingly, the contrast curve 19 of the color blue B has a greater contrast maximum CMax than the contrast curve 19 of the color red R. FIG. 3 now shows a procedure according to the present invention. The width of the projected lines 3 is adapted to the respective volume scattering effect of the respective color. According to FIG. 3 b, the width of a line 3 with red color R is increased in such a way that the contrast maximum CMax of the contrast course 19 of the red line 3 is increased. This widening can be carried out until the contrast maximum CMax of the red line corresponds to the contrast maximum CMax of the blue line. The width of each blue line remains unchanged.

FIG. 4 shows a representation of the modulation transfer functions of lines of the color blue B and the color red R. Starting from the color blue, a specific contrast value is achieved for this width assigned to the color blue B. This is now set as contrast minimum value CMin. For a red line to also cause the same contrast minimum CMin in the captured image, the width must be made larger for each red line to get the same contrast as the blue stripe. According to this embodiment of a color pattern 1 according to the invention, the widths of the red lines 3 have been doubled. The legal value axis of the coordinate system according to FIG. 4 designates the number of line pairs per mm. Line pairs are labeled with the letter P.

Figures 5a to 5c show a solution according to the invention for the case that a respective color of a line 3 is generated by means of a projector device as a mixed color. The image acquisition at the CCT takes place for example by means of single-chip cameras or three-chip cameras. That is, a color pattern is formed only by mixing the colors red, green and blue (RGB mixture) through the RGB filter of the detector or camera the color pattern again unmixed. In this regard, an edge overlap does not interfere with the detection of the color transition, because, in principle, a color edge can be uniquely determined for each color. FIGS. 5a to 5c now show a second exemplary embodiment of a color pattern according to the invention. The color pattern has here only one illumination spot, which can also be referred to as line 3. The composition of the spectral components of this line is generated here by means of mixed colors. FIG. 5a shows the contrast profile in the recorded image after the volume scattering, wherein the illumination spot or the line 3 was produced by mixing the color red R and blue B. With such mixed colors, the volume scattering effect can spatially separate the participating RGB components in the biological medium. This means that edge transitions each have a chromatic aberration (formation of a dyeing area).

FIG. 5b shows how the very small violet spot 3, which according to FIG. 5a was produced by mixing red and blue and projected onto an object to be measured, is recognized by an eye. The corresponding observation with the eye according to FIG. 5b shows that the eye changes the violet spot 3 in its hue and produces a red color fringe on the left and on the right. When using a camera with an RGB filter as a detector, the mixed color is violet-segregated and would be detected as a blue and a dark red spot, with the red spot having a larger diameter. FIG. 5c now shows an adaptation according to the invention of the original line according to FIG. 5a. The width of the strip or line 3 produced with mixed colors is chosen such that the modulation transfer functions MTF for both colors - which are red and blue - is almost the same, exceeds a minimum value and additionally the areas of maximum slope the contrast curves 19 in the recorded image coincide. The areas of maximum slope are marked with SMax. By means of the inventive solutions, a three-dimensional measurement is effectively improved by means of CCT.

A device and a method for determining three-dimensional surface coordinates of an object by means of optical color triangulation are proposed, wherein all lines of a color stripe pattern each have a width adjusted such that in a recorded image of the line all contrast maxima CMax of all spectral components of a line 3 equal one Contrast minimum value CMin are. Starting from an invariable smallest width of a pattern line 3a with a spectral component of greatest contrast, further lines 3 can be correspondingly widened. The invention is particularly advantageous for a 3D measurement of biological tissue, which may be transparent and scattering.

Claims

claims

1. Device for determining three-dimensional surface coordinates of an object by means of optical color triangulation, with

- A projector device for illuminating the object with a set color stripe pattern (1), wherein the color stripe pattern (1) along an axis (x) and extending from this perpendicular lines (3) with a respective to adjacent lines different selection of spectral components of the projected light;

- A detection device arranged in a known relative position to the projector device for taking a picture (7) of the object onto which the color stripe pattern (1) was projected once;

- A computer device for calculating the three-dimensional surface coordinates by detecting the selection of the spectral components of a respective line (3) and detection of a respective transition (5) of two adjacent lines (3), characterized in that

for all lines (3), by means of the projector device the width (Br) of a respective projected line (3) is set such that in the recorded image (7) of the line (3) all contrast maxima (Cmax) of all spectral components of this line (3) equal to a minimum contrast value (Cmin).

2. Apparatus according to claim 1, characterized in that for lines (3) each having at least two selected spectral components, by means of the projector device, the width (Br) of the respective line (3) is set such that in the recorded image (7) of the line (3) areas of maximum slope of the contrast curves (19) of these spectral components coincide.

3. Device according to claim 1 or 2, characterized in that the contrast minimum value (Cmin) is the value of the contrast transfer function (15) of a pattern line (3a) with a fixed minimum width and a single Spectral component consisting of a short-wave color causing a relatively small volume scattering effect.

4. The device according to claim 3, characterized in that, starting from the invariable, smallest width of the pattern line (3 a), the widths (Br) of the other lines (3) have remained unchanged or enlarged starting from this smallest width.

5. Apparatus according to claim 3 or 4, characterized in that the effect of a relatively small volume scattering effect color is blue.

6. Device according to one of the preceding claims, character- ized in that at least one spectral component corresponds to a single color.

7. Device according to one of the preceding claims, characterized in that the projector device generates the selected spectral components by mixing the individual colors red-green-blue.

8. Device according to one of the preceding claims, characterized in that the detection device comprises a red-green-blue filter.

9. Device according to one of the preceding claims 3 to 8, characterized in that the invariable smallest width is at least 1/12 mm in the object area.

10. A method for determining three-dimensional surface coordinates of an object by means of optical color triangulation, comprising the steps

illuminating the object with a set color stripe pattern (1) carried out by means of a projector device, wherein the color stripe pattern (1) extends along an axis (x) and out of these perpendicular lines (3) each of adjacent lines has different choices of spectral components of the projected light;

by means of a detection device arranged in a known relative position to the projector device, picking up an image (7) of the object onto which the color stripe pattern (1) was projected once;

- Calculating the three-dimensional surface coordinates by means of a computer means by detecting the selected spectral components of a respective line (3) and detecting a respective transition (5) of two adjacent lines (3), characterized by

Adjusting for all lines (3), by means of the projector device, the width (Br) of a respective projected line (3) such that in the recorded image (7) of the line (3) all the contrast maxima (Cmax) of all spectral components of this line (3) are equal a minimum contrast value (Cmin).

11. The method according to claim 1, characterized in that for lines (3) each having at least two selected spectral components, by means of the projector device, the width (Br) of the respective line (3) is set such that in the recorded image (7) of the line (3) areas of maximum slope of the contrast profiles of these spectral components coincide.

12. The method according to claim 10 or 11, characterized in that the contrast minimum value (Cmin) of the value of the contrast transfer function (15) of a pattern line (3a) having a fixed minimum width and a single spectral component consisting of a relatively small volume scattering effect causing, short-wave color is.

13. Method according to claim 12, characterized in that starting from the invariable, smallest width of the pattern line (3a), the widths (Br) of the other lines (3) remain unchanged or enlarged starting from this smallest width.

14. The method according to claim 12 or 13, characterized in that the effect of a relatively small volume scattering effect color is blue.

15. The method according to any one of the preceding claims, characterized in that at least one spectral component corresponds to a single color.

16. The method according to any one of the preceding claims, characterized in that the projector device generates the selected spectral components by mixing the individual colors red-green-blue.

17. The method according to any one of the preceding claims, characterized in that the detection device comprises a red-green-blue filter.

18. The method according to any one of the preceding claims 12 to 17, characterized in that the invariable minimum width is at least 1/12 mm.