DE102021121121A1 - Device for determining geometric properties of a surface using polarized light - Google Patents

Abstract

Arrangement for determining geometric properties of a surface, characterized by at least one illumination (L0) with polarized light directed onto the surface and by two polarization cameras (C1, C2) in a stereo arrangement. To produce a point correspondence, either the non-polarized Components are evaluated or in a special arrangement of several illuminations (L0, L1, L2) either only the polarized or separately the polarized and non-polarized components. This enables the determination of the geometric properties of a surface by means of stereo analysis without movement, without complex pattern projection devices , with lighting units that are easy to implement and with freely available, internally unchanged cameras (polarization cameras). With the same arrangement, mixed dielectric and metallic surfaces can be evaluated, as well as diffuse and shiny scattering surfaces, without having to know how these material properties are distributed on the surface.

Description

The invention relates to devices for determining geometric properties of a surface. "Geometry" is broadly defined here and extends from the fine structure, to the coarse structure such as the waviness or bending of the surface of a component, to the geometry of an object or a scene with several objects, see 1 : The surface 1 can be curved and have more or less pronounced differences in height, including abrupt ones (1a). In addition to the surface geometry (elevation image with height in relation to a reference plane), structural features such as holes (1b), different roughness (1c), pores (1d), scratches (1e), etc. can be determined depending on the task Surface geometry is then to be produced; it can also be a task to relate albedo-related features or color features to surface geometry. The determination of the position of various objects in a scene on the basis of a height image created according to the methods and arrangements presented here is also part of the subject matter of the present invention.

Applications can be found in industry, agriculture and medicine, among others.

In the case of stereo systems, the basic task is to determine corresponding points. The search for corresponding points can be simplified by restricting the search to epipolar lines. A correspondence hypothesis P1/P2 is the assumption that a point P1 on the surface captured in camera C1 corresponds to a point P2 captured in camera C2. With geometrically calibrated cameras, the 3D coordinates of the associated surface point can be calculated for a correspondence hypothesis. In the search, features at a point P1 of camera C1 are compared with features of a point P2 in camera C2. The features are derived from data provided by the camera at pixels P1 in camera C1 and P2 in camera C2 (P1/P2) or there in small image sections. In the course of the correspondence determination, quality values are determined for correspondence hypotheses P1/P2, which result from the comparison of the characteristics of P1 with those of P2. A good choice of feature is characterized by the fact that the quality values are low with a high probability of being false hypotheses and high with a high probability of being high with correct hypotheses. The overall task boils down to the optimization of an overall quality while complying with the necessary geometric boundary conditions.

If, instead of individual scalar features, an entire feature vector of individual features that are as independent as possible is used, the correspondence determination is thereby simplified and the result is usually improved, ie the features are more significant. Basic methods for finding point correspondences on epipolar lines are described in detail in the literature. The arrangements and methods proposed in the invention serve to obtain features or feature vectors that are as significant as possible.

If a surface is illuminated with unpolarized light, the reflected light will be polarized to a certain extent, depending on the nature of the surface and the refractive index of the material. In principle, the surface normal at this point can be deduced from the polarization at a surface point. (“shape-from polarization”). The disadvantage: The polarization effects are i.a. weak, the calculated surface normal data are therefore imprecise or noisy, and they are ambiguous. See the literature cited further below.

According to [Zhu + Smith], a polarization camera [Sony] and a conventional RGB camera are used in a stereo arrangement. It is the combination of the results of conventional stereo evaluation with those of a shape-from-polarization evaluation using a polarization camera. The lighting is unpolarized. In the first step, both cameras are used as conventional cameras and, based on classic stereo processing, a rough depth map is first created with the two cameras. In the second step, the polarization camera alone is used to calculate the surface normals using classic shape-from-polarization methods. With the help of the coarse depth map, the ambiguities and uncertainties of the surface normal are largely eliminated and a denser depth map is in turn created from this. A disadvantage of the approach is that - in contrast to shape-from polarization - the conventional stereo evaluation does not work with surfaces with little structure, so that in such cases the combination brings no improvement over shape-from polarization.

In [Zhao et al.], results from RGB stereo images are combined with shape-from-polarization results into a consistent result. The illumination is unpolarized in this case. A variety of views are used here. Experimental results were obtained by rotating an object in front of an RGB polarization camera in 31 positions in a cloudy day. In addition to the disadvantage of the aforementioned approach ([Zhu + Smith]), the technical effort is added here due to the large number of recordings in different views.

In [Conon et al.], three RGB miniature cameras placed close to each other are used to avoid disturbing light reflections in endoscopy. A ring light with a polarization filter is placed around the cameras. A polarization filter is mounted in front of the cameras. Practical investigations into the relative orientation of the polarization filters show that the probability that a light reflection disappears when the picture is taken is highest when the camera polarization filter is crossed over against the ring light polarization filter. The disadvantage here is that the disruptive reflections in the case of curved objects (frequent case) cannot all be eliminated (the direction of polarization changes gradually and differently depending on the inclination at an object point) and that no reliable 3D evaluation can succeed in the case of objects with a homogeneous surface .

In [Huang et al.] an active, polarization-coded structured illumination is proposed, similar to the classical approaches with encoded illumination (specifically: Gray code), only with the difference that here not the brightness, but the polarization is used for encoding . The method works with a single conventional camera and a calibrated pattern projector (calibration required because of "ray plane triangulation"). The pattern projector uses the known polarizing properties of LCDs. A polarization filter is located in front of a conventional camera so that the camera can distinguish for each pixel whether the polarization is rotated at that point or not. The resulting binary images are evaluated using classic methods. Disadvantages are first of all the equipment required for the projector and the large number of images to be projected. However, the main disadvantage is that polarized light, which is scattered on diffuse surfaces, partially or completely loses its polarization and therefore often no evaluation can take place at the relevant points. And: If a polarized light beam hits a surface with polarization at an angle to the surface, the polarization angle is rotated, which worsens the effect of the polarization filter in front of the camera and thus makes the binary image generation less reliable.

The study [Cui et al.] is about determining the relative pose of an object when taking pictures in two different poses. The task thus deviates from the present subject matter of the invention. Two 3D point correspondences and the associated surface normals are sufficient for the relative pose. In order to determine the surface normal, a "shape from polarization" approach (see above) - with the corresponding disadvantages - is used, the lighting is not polarized, an RGB camera (Canon EOS 7D) with a pre-mounted polarization filter is used. The procedure described only works for diffuse surfaces. The existence of polarization cameras is mentioned in the introduction, but they are not used in the method and their use is not recommended, the methods described in detail cannot be transferred to polarization cameras. The task of [Cui et al.], which deviates from the purpose of the present invention, can also be achieved if only a few object locations can be reliably evaluated; this is not sufficient for the present invention.

The object of the invention is to avoid at least one of the disadvantages mentioned above.

The object is achieved according to the invention by methods and devices with the features of the appended independent patent claims. Preferred configurations and developments of the invention result from the appended dependent patent claims and the following description with associated drawings.

In contrast to the prior art, the invention presented here uses polarized illumination, in a special arrangement depending on the claim, and a camera stereo arrangement with two polarization cameras.

In a polarization camera, there are several polarization filters in the beam path between the optics and the CCD sensor, which are assigned to individual pixels and are aligned in different directions with respect to the assigned pixels. For small image regions, both the direction and the amount of the resulting polarized component can be calculated locally from the intensity contributions of each of several pixels.

A polarization image sensor chip from Sony is currently on the market; Here, the polarization in 4 main directions is recorded with polarization filters on a 2x2 pixel arrangement. The chip is intended to eliminate highlights in photos, see [Sony].

A color variant is also offered, whereby 2x2 of such 2x2 pixel arrangements are used in a Bayer filter arrangement. The following description relates only to the black and white variant, the generalization of the methods and arrangements presented here with additional color evaluation for colored objects follows directly, is addressed further below and is implied in the claims.

With polarization cameras, the main direction of polarization and the amount of polari calculated component for each 2x2 pixel array [JAI].

According to the inventor's first experiments, the methods described work with linear and, in principle, also with circularly polarized illumination. The idea that linearly polarized lighting is involved makes it easier to understand the description. However, the description and claims refer to linearly or circularly polarized lighting, so we are simply talking about polarized lighting.

When incident light is reflected off a surface, part of the incident light is usually specularly reflected and another part of the incident light is diffusely reflected. Highlights in photos are created when the incident light is largely reflected. The occurrence of highlights depends on the direction, i.e. on the angle at which the light strikes and on the angle at which the polarization camera is aimed at the surface. The intensity of specular highlights reaches a maximum when the polarization camera is aimed at the surface at the glancing angle.

In the case of surface reflections, one can roughly distinguish between two components: diffuse and specular reflection. In order to keep the explanation of the process principles simple, a binary distinction is made below between diffusely reflecting and specularly reflecting surfaces, in short matt or smooth surfaces. This property varies on the surface i.a. locally. In practice, of course, intermediate stages and locally continuous transitions are also possible.

In terms of material properties, a distinction is made between dielectric and conductive materials. In the applications in question here, conductive materials are usually metals (hence “metallic” for short). This property can also vary locally.

The invention is described in more detail using exemplary embodiments.

1st embodiment

For claim 1 see 2 : To determine the geometric properties of a surface 1, it is illuminated with at least one illumination L0 with polarized light, and the non-polarized component is evaluated. As an example of implementation, L0 is drawn schematically as an unpolarized point light source q0, equipped with a polarizing filter P0. Instead of the point light source, it is also possible to use one or more linearly or flatly extended light sources. The image is recorded with two polarization cameras C1 and C2 in a stereo arrangement. In principle, L0 can be positioned anywhere, so it does not have to be in the middle between C1 and C2, as shown.

From the data provided by the polarization cameras, the direction of polarization and the polarized component can be calculated for each 2x2 pixel arrangement. By subtracting the polarized component in the 2x2 pixel arrangement, one arrives - separately for each camera, because the degrees of polarization are i.a. different - to the non-polarized component resulting from diffuse scattering. For a correspondence hypothesis, the luminance (it determines the impression of brightness) of pixels in the polarization cameras is compared with one another. In the case of polarized lighting, the behavior of the luminance depends on whether there is specular or diffuse reflection. With a specular reflection, the light remains polarized. In the case of diffuse reflection, the luminance depends on the direction of illumination, but not on the viewing direction. In the case of a glossy, in the extreme case specular, reflection, the luminance depends on how well the condition "angle of incidence is equal to angle of reflection" is fulfilled, i.e. effectively dependent on both angles.

Using polarized lighting in order to subsequently remove the polarized component from the evaluation in the cameras seems absurd at first glance.

For ease of explanation, it is assumed that the surface is metallic and in places clearly smooth or clearly dull. Depending on the lighting geometry, annoying highlights can occur on smooth areas; The fatal thing is that such highlights appear on different surface areas for the two cameras, which significantly disturbs the stereo evaluation. With unpolarized lighting, you cannot remove the shiny spots even with polarization cameras, because the unpolarized light remains unpolarized after reflection at the smooth spots. Only in the case of dielectric materials with angles of incidence close to Brewster's angle does polarization occur, which can be eliminated using a polarization camera.

Polarized light, however, is reflected polarized at bright spots and unpolarized at dull spots. If you only use the unpolarized part in the camera, then there will be dark areas on smooth surface areas. A smooth spot appears dark in any camera, either because the polarized light at the glancing angle is reflected past the camera, or if it hits a camera, arrives polarized at the camera where the pola ized part is filtered out. The dark areas therefore appear on the same surface areas in both cameras and are therefore suitable for a stereo evaluation, in complete contrast to the situation with unpolarized lighting.

These considerations apply regardless of the geometric arrangement and design of the lighting and regardless of the direction of polarization on the polarizing film (more generally: regardless of the design of a polarization pattern on a large-area lighting unit), because a ray hitting a camera in a specular light, viewed backwards, can only come from a single spatial direction and thus only come from a smaller area of the lighting device, which rules out the possibility of different directions of polarization overlapping in the camera, which in principle could approximate unpolarized light.

A clear, binary distinction "matt vs. smooth" is mostly unrealistic and was made to facilitate the explanation. In practice there are intermediate stages, from the explanation it follows that in the cameras the remaining matte component is the same in both cameras. Since in principle only diffuse reflections appear in the images, the brightness (or color values) for a once selected lighting for the corresponding points of the two cameras are the same, which enormously simplifies the finding of correspondence.

The considerations also apply to dielectric surfaces, only there is a weakening of the polarized component in the direction of the plane of incidence at smooth points, which does not change anything about the remaining and evaluated unpolarized component.

The plane of incidence is spanned by the incident beam and the perpendicular to a surface element.

In this way, one image is obtained for each of the two cameras, both of which are free of polarized light and thus of disturbing highlights and can be processed using known methods of stereo image analysis.

Interreflections: The evaluation of the diffuse component in an arrangement according to the first exemplary embodiment (claim 1) also has the advantage, at least at matt points, that the images of a surface element in C1 and C2 are also the same if there are interreflections on the surface, because the Illumination geometry only needs to be the same for both images.

The arrangement advantageously has an evaluation unit, the evaluation unit being adapted to calculate an unpolarized light component from corresponding images of the two polarization cameras and to determine corresponding points on the surface using the unpolarized light component (claim 9).

2nd embodiment

Regarding claim 2: For unstructured, for example only slightly corrugated surfaces, it is favorable to modulate the lighting randomly or quasi-randomly. In the 2nd embodiment after 3 this is done simply by interposing a structuring element 2, the transparency and/or color and/or cross section of which varies roughly in the direction of the connecting line between C1 and C2, so that a variation in intensity and/or color is caused in the direction of the epipolar lines. 3 shows an example of a pane with varying transparency; the lines drawn there do not need to be reproduced sharply. A polarized light pattern projector (using polarizing filters or LCD polarization) can also be used.

A random or quasi-random modulation of the illumination, ie a locally irregular modulation, has the advantage over a regular, ie periodic modulation that an unambiguous assignment between the incident light of the illumination and the unstructured surface is made possible. This greatly simplifies the search for corresponding points.

One can advantageously use only one structuring element, only one polarization filter, but several (in 3 laterally) offset, advantageously differently colored point light sources generate several different modulation patterns. Instead of point light sources, it is also possible to use linearly extended light sources, which are oriented at least approximately in the direction of the drawn lines. The same applies here: Values that happen to be the same for a false correspondence hypothesis for one direction are very likely unequal for the other, significantly different direction. This applies not only because in the case of false hypotheses (P1 and P2 are different points on the surface) there is a high probability of different levels of illumination, but also because P1 and P2 then generally have different surface inclinations and the diffusely reflected component is from the Illumination angle dependent on surface element. The same applies here: It doesn't work without deducting the polarizing component, because polarized components are strong depends on the viewing direction and is therefore dependent on the camera. With multiple illuminations, an increasingly long feature vector with different feature values for false hypotheses generally arises between camera 1 and camera 2 for finding correspondence, which both simplifies finding correspondence and makes it more reliable.

Several images with different modulation patterns, as described above, make the evaluation more reliable and enable a more reliable evaluation even with homogeneous and slightly curved surfaces.

3rd embodiment

According to claim 3, several lights according to the previous exemplary embodiments (claim 1 or 2) are directed onto the surface from different directions. Several different lights with different lighting directions can be switched on one after the other, each time with newly calculated brightness (or color values) after calculating the polarized component. The considerations made in the second exemplary embodiment (claim 2) for illumination from different directions apply here accordingly. Advantageously, the directions are chosen so that they differ greatly from one another. Values that happen to be the same for a wrong correspondence hypothesis for one direction are then with high probability unequal for the other, clearly different direction. In addition, this makes it possible to better illuminate parts of the surface with very different inclinations.

For each of these illuminations, the polarized component is calculated for both cameras and the unpolarized component is used. Separately for each of these illuminations, the unpolarized, diffuse component is in principle independent of the viewing direction and is therefore in principle the same for both cameras.

Several images illuminated from different directions, as described above, make the evaluation more reliable and facilitate the evaluation even with homogeneous and slightly curved surfaces.

If the illumination occurs - even if only slightly - in different directions at different times (with one recording each by C1 and C2), the feature vectors are lengthened and the evaluation is thereby improved.

In a particularly advantageous embodiment, in the arrangement 1 several laterally offset point light sources or linearly extended light sources of different colors are realized with a single image recording (not shown) and color polarization cameras are used. The benefit is as above, with the added benefit of saving time in image acquisition.

It is advantageous if the arrangement is adapted with a number of lights from different directions, to switch the number of lights from different directions one after the other (claim 10).

4th embodiment

Regarding claim 4: The at least one illumination L0 here comprises at least two illuminations L1 and L2 with polarized light, which are arranged in such a way that when rays from L1 are reflected on a surface point P at the glancing angle to the camera C2, rays from L2 are also reflected on at least approximately the same surface point P in the grazing angle to the camera C1.

Here, in contrast to the 1st to 3rd embodiments (claims 1-3), the polarized component of the light is used and the image of the camera C1 is recorded with illumination L2 and the image of the camera C2 is recorded with illumination L1.

Due to the gloss condition in point P being fulfilled in both images, point P has the same brightness or luminance in both images. As a result, the deficiency explained in connection with the 1st exemplary embodiment, namely that shiny spots appear in the two images on different surface parts of the object in each case, does not apply.

In DE 10 2019 105 358 A1 corresponding arrangements are described that meet this condition, but without polarized light and without polarization cameras.

Such an arrangement shows 4 with lights L1 and L2 positioned laterally next to the lenses of the cameras C1 and C2, comprising light sources q1 and q2, respectively, and polarizing filters P1 and P2, respectively.

In this case, the at least one illumination L0 can include further illuminations that do not have to meet the glossiness condition described above at point P for the two illuminations L1 and L2.

A corresponding arrangement (see also the 8th exemplary embodiment) combines the two illuminations L1 and L2 4 (Evaluation of the gloss components) with the lighting L0 off 1 (Evaluation of diffuse components), cf. 9 .

The illuminations L0, L1 and L2 are all polarized. The illumination L0, used here as an additional third illumination, is arranged according to the first exemplary embodiment, the first illumination L1 and second illumination L2 according to the fourth exemplary embodiment. In the case of illumination with L0 and/or L1 and/or L2, the unpolarized component arriving in the polarization cameras C1 and C2 is evaluated. In the case of illumination with the second illumination L2, the polarized component arriving in the first polarization camera C1 is evaluated. Correspondingly, in the case of illumination with the first illumination L1, the polarized component arriving in the second polarization camera C2 is evaluated.

5th embodiment

5 shows a particularly advantageous arrangement of L1 and L2 as a ring light around the lenses of C1 and C2. (Claim 5).

The advantage of such an arrangement and this procedure is that the otherwise disturbing gloss components, which appear at different points in the cameras with conventional lighting, appear here at least approximately at the same surface point and can thus be used advantageously for stereo evaluation. The otherwise big problems with highlights are thus turned into an advantage.

In contrast to the previous exemplary embodiments (claims 1 to 3, and exemplary embodiment 8 according to claim 8 advantageously additionally thereto, see there), ONLY the polarized component in the 2×2 pixel arrangement is used here.

In the case of metallic surfaces, the degree of polarization on the surface element is 3 ( 6 ) depends only on the smoothness of the surface (ideally reflecting metal surface: 100%) and is the same in both directions, from L1 to C2 and from L2 to C1 (hereinafter referred to as “both directions”).

In the case of non-metallic surfaces, the degree of polarization on the surface element 3 is also dependent on the angle of incidence: the component perpendicular to the plane of incidence is basically retained and is the same for both directions. The component parallel to the plane of incidence depends on the angle of incidence and is also the same for both directions (angle of incidence = angle of reflection, identical material) and is zero or almost zero at Brewster's angle.

If one selects the polarization angle of the polarization films p1, p2 for both cameras in such a way that in a special arrangement, for example in the symmetrical arrangement 6 , the polarization angle is parallel to the plane of incidence in both directions, then with the same arrangement of cameras and illumination, but shifted and tilted surface element 3, 7 , (so that specularity condition is preserved) the polarization direction of L1 as viewed by camera C2 is rotated by an angle and the amount is changed by a factor; of L2 as seen by camera C1, the polarization direction is rotated by the same angle, just in the opposite direction, and the amount is changed by the same factor. In such an arrangement, the degree of polarization is therefore also the same in both directions. This also applies when looking after as a starting point 8th takes the polarization directions of p1 and p2 set perpendicular to the respective plane of incidence as the starting point. With other constellations of polarization directions, the situation becomes more complicated.

The equality considerations hold regardless of the dielectric constant of the material in question.

When setting up the system, the directions of polarization of the illuminations should be coordinated in such a way that for a preferably central, symmetrical arrangement of illuminations relative to the surface element in both directions, the illumination polarization angles on the surface element are either both parallel or both perpendicular to the respective plane of incidence; then the above symmetry considerations apply.

And another feature can be evaluated here: the direction of polarization. If a correspondence hypothesis is correct, then the same rotation of the polarization direction takes place in both cameras, only, as described above, when viewed from the camera's viewing direction, with the opposite sign.

With metallic surfaces it is not necessary for the evaluation of the polarization direction, the illumination polarization directions in the 4 or. 5 to be adjusted during assembly. It is sufficient to record the mean value of the two polarization orientations at C1 and C2 for a single point reflecting in the specular (normal polarization orientation) in a calibration process. In a current measurement, if the correspondence is correct, the mean value of the two polarization orientations ideally does not deviate from the mean value of the two normal polarization orientations. The absolute difference between the two mean values is therefore a measure of the quality of a correspondence.

The following applies to metallic and non-metallic surfaces: if the correspondence is correct, the polarization directions in C1 and C2 are rotated with the opposite sign by the same angle relative to the normal situation. The measured mean absolute difference in the twists is a characteristic of a measure of the quality of a correspondence.

Claim 5 relates to an arrangement according to the 4th exemplary embodiment (claim 4), the illumination L1 being arranged in a ring around C1 and the illumination L2 being arranged in a ring around C2, cf. 5 .

6th embodiment

Claim 6 relates to an arrangement according to the 4th exemplary embodiment (claim 4), in which the illumination L1 is actually or virtually in the beam path of C1 and L2 is actually or virtually in the beam path of C2 (no figure).

Such an arrangement can be reflected into the viewing beam path by means of a semi-transparent mirror. This is complex, but has the advantage that the polarizing points and components are always exactly the same, even with very heavily structured surfaces.

7th embodiment

Regarding claim 7: It is advantageous because it reduces the effort if the first illumination L1 and/or the second illumination L2 is also used as at least one illumination L0 according to claim 1, for which there are no mandatory positioning specifications. If the first illumination L1 and the second illumination L2 are used, in addition to the reduction in complexity, there is also the advantage, described above, of illumination from significantly different illumination directions.

• C1 mit L1 und C2 mit L1 und/oder
• C1 mit L2 und C2 mit L2.

The following is then evaluated to evaluate the diffuse components:

• C1 with L1 and C2 with L1 and/or
• C1 with L2 and C2 with L2.

The in the first and second polarization cameras C1; The unpolarized portion arriving at C2 is evaluated in the case of illumination with the first illumination L1 and/or the second illumination L2.

• C1 mit L2 und C2 mit L1

The following is then evaluated for the evaluation of the gloss components:

• C1 with L2 and C2 with L1

The polarized portion arriving in the first polarization camera C1 is evaluated when illuminated with the second illumination L2. Correspondingly, the polarized portion arriving in the second polarization camera C2 is evaluated when illuminated with the first illumination L1.

8th embodiment

Claim 8 relates to an arrangement which, in addition to the first illumination L1 and the second illumination L2, includes a third illumination with polarized light. The third illumination is arranged such that the rays of the third illumination are reflected on the surface point P out of grazing angle to the first polarization camera C1 and out of grazing angle to the second polarization camera C2.

Such an arrangement has the advantage that one can evaluate both the unpolarized components and the polarized components as described in relation to the above claims without having to switch lighting, i.e. with a single image recording, which is particularly advantageous in the case of fast-moving objects is. The condition specified in claim 8 for the third illumination can be achieved with certain prior knowledge of the surface structure. For example, with a ripple in a known direction, the third illumination can be oriented transversely thereto, so that rays reflected in the grazing angle are not seen in the polarization cameras C1; arrive C2. The more small-area the lighting units of the third lighting are designed, the easier it is to achieve the stated condition.

General advantages:

According to the invention, the geometric properties of a surface can be determined by means of stereo analysis without movement, without complex pattern projection devices, with lighting units that are easy to implement and with cameras (polarization cameras) that are freely available on the market and are internally unchanged. With the same arrangement, it is possible to evaluate mixed dielectric and metallic surfaces, as well as diffuse and shiny scattering surfaces, without knowing how these material properties are distributed on the surface.

In the case of non-conductive surfaces, it is not necessary to know the dielectric constants of the materials.

The procedure works for surfaces with only a small amount of diffuse reflection Arrangement according to Claim 1, in which the disturbing glossy spots which can appear at different locations on the surface for the two cameras are eliminated. With unpolarized lighting, on the other hand, such shiny spots would not be so easy to filter out.

For such surfaces, the arrangement according to exemplary embodiment 4 (claim 4) has a supportive effect, since here the otherwise disturbing glossy spots appear at the same locations on the surface for the two cameras.

For completely reflective surfaces, the arrangement according to exemplary embodiment 4 (claim 4) works at least for parts of the surface, and the lighting can be designed for different sized surface areas depending on the requirements.

extraneous light:

One of the advantages of the arrangements according to the invention and the procedure according to the invention is the low sensitivity to extraneous light (daylight, hall light, . . . ) at least at dull points, which is normally unpolarized. The fact that extraneous light can come from unknown directions at such points is not harmful in the case of exemplary embodiments 1 to 3 (claims 1-3), for which there are no mandatory positioning specifications in these exemplary embodiments (claims 1-3). Under certain circumstances, additional extraneous light is even helpful here. Conversely, extraneous light at such points in exemplary embodiment 4 (claim 4) is less harmful, since polarization effects on surfaces are small in the case of the unpolarized extraneous light in comparison to the polarization change effects in the case of polarized light.

Multiple Camera Pairs:

Unless otherwise stated, the considerations relate to only one single pair of polarization stereo cameras. Of course, several pairs of cameras can be used. A separate result can be created for each pair of cameras, and several such results can be merged into a common result using known methods, which results in an increase in accuracy and/or an enlargement of the evaluation area (“stitching”). One camera can also be used for several pairs of cameras.

Colour:

The invention relates equally to grayscale and color polarization cameras. With color polarization cameras, the color components are added as additional features for colored objects or when using color for lighting.

Structural features:

Structural features such as holes (1b), different roughness (1c), pores (1d), scratches (1e), etc. and albedo-related or color-related structural features can be calculated as is known from gray value or color images or from the height images created according to the invention; as soon as the point correspondences and the associated heights are known, such structural features can be directly related to the height image, whereby the structural features can also be obtained using other methods.

As soon as a reliable height image is available, well-known 3D matching methods can be used to find objects of known geometry in the height image.

Move:

Image recording and/or evaluation can take place at a standstill or in motion (camera or scene). In the arrangement according to the 4th to 7th embodiment (claim 4-7), the recordings can be made very quickly one after the other with today's technology. In the case of metallic or other achromatic surfaces, the recordings can also be separated by color using different lighting colors for L1 and L2 and the use of a color polarization camera. An example of changing scenes is security surveillance in the workspace of a conventional or cooperative robot.

Evaluation unit:

Although polarization cameras are not yet available as intelligent cameras, the image evaluation can not only take place in an evaluation unit separate from the cameras, such as a PC, but also in one of the two cameras or both or in an evaluation unit to which only the image sensors are connected.

For calibration:

The geometric calibration of stereo cameras is a topic that has been extensively discussed in the literature and is therefore not described here.

Intensity calibration can be done as follows: lamp brightness and/or camera gains are conveniently adjusted so that a centrally located point in the scene appears as equally bright in both cameras. The best way to proceed is to use a Lambertian test surface to first align the two cameras with constant lighting, and then use a shiny test surface to match the lighting L1 and L2 in pairs. Then you can still implement a site-specific standardization for the image evaluation by moving the test surface(s).

A geometric calibration of the light sources can be avoided: Assuming that the two light sources have exactly the same intensity and are the same in all directions, then the impression of brightness is still dependent on the distance of the light source to the surface point. (Within the reflection lobe, the impression of brightness is independent of the distance to the camera, small aperture relative to the reflection lobe on the camera; usually true). Here an asymmetry arises, which can in principle be taken into account in the context of the correspondence analysis: Correspondence hypotheses are geometry hypotheses, so if the illumination geometry is known, the distance of the surface element to the illuminations can be calculated for each correspondence hypothesis. However, this complication can be avoided by not evaluating the components polarized or unpolarized, but rather their percentage shares (“shares” for short) and comparing them in the two cameras.

The invention also includes methods for determining geometric properties of a surface, having method steps that correspond to the technical features of the devices of the exemplary embodiments.

Combination with known methods:

A combination with known methods is of course possible. For example, for special features, you could also operate classic stereo with the same cameras with unpolarized lighting.

Non-patent literature:

The publications from the Internet were downloaded no later than August 20, 2020.

[Sony] Polarization Image Sensor Polarsens https://www.sony-semicon.co.jp/products/common/pdf/IMX250_253MZR_MYR_Flyer_en.pdf

[JAI]

https://www.automation.com/en-us/products/product01/jai-releases-go-5100mppge-go-series-polarization As of July 9, 2019

[Huang et al.] Huang et al. Target enhanced 3D reconstruction using polarization coded structured light. Vol. 25, No. 2 |23 Jan 2017| OPTICS EXPRESS 1173.

http://wangkaiwei.org/file/publications/oe2017_xiao.pdf, https://www.osapublishing.org/DirectPDFAccess/7C029034-CFC2-CFA7-D68784028B3458A9_357437/oe-25-2-1173.pdf?da=1&id= 357437&seq=0&mobile=no

[Zhu + Smith] Dizhong Zhu and William A.P. Smith:: Depth from a polarization + RGB stereo pair. (ArXiv 2019) https://arxiv.org/pdf/1903.12061.pdf

[Zhao et al.] Jinyu Zhao, Yusuke Monno, and Masatoshi Okutomi: Polarimetric Multiview Inverse Rendering. (ArXiv 2020) https://arxiv.org/pdf/2007.08830.pdf

[Conon et al.] N. Conon et al.: Improving Image Matching using by Reducing Surface Reflections using Polarizing Filter Techniques. Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLII-2, 2018. https://pdfs.semanticscholar.org/3b93/9d75b02986a3ccbf823d7f4a4c03a53487e9.pdf

[Cui et al.] Zhaopeng Cui et al: Polarimetric Relative Pose Estimation. https://openaccess.thecvf.com/content_ICCV_2019/papers/Cui_Polarimetric_Relative_Pose_Estimation_ICCV_2019_paper.pdf

QUOTES INCLUDED IN DESCRIPTION

This list of documents cited by the applicant was generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Patent Literature Cited

• DE 102019105358 A1 [0048]

Claims (12)

Arrangement for determining geometric properties of a surface (1), characterized by at least one illumination (L0) with polarized light directed onto the surface (1), and by two polarization cameras (C1; C2) in a stereo arrangement. arrangement according to claim 1 , Having a device for random or quasi-random intensity and / or color modulation of the polarized light. arrangement according to claim 1 or 2 with multiple lights, where the multiple lights are directed at the surface from different directions. arrangement according to claim 3 , wherein the two polarization cameras have a first polarization camera (C1) and a second polarization camera (C2) and the at least one illumination (L0) comprises at least a first and a second illumination (L1; L2) with polarized light, which are arranged such that , if rays from the first illumination (L1) are reflected on a surface point (P) in the glancing angle to the second polarization camera (C2), rays from the second illumination (L2) are also reflected on at least approximately the same surface point P in the glancing angle to the first polarization camera (C1) be reflected. arrangement according to claim 4 , wherein the first illumination (L1) is arranged annularly around the first polarization camera (C1) and the second illumination (L2) is arranged annularly around the second polarization camera (C2). arrangement according to claim 4 , wherein the first illumination (L1) is arranged real or virtual in the beam path of the first polarization camera (C1) and the second illumination (L2) is arranged real or virtual in the beam path of the second polarization camera (C2). arrangement according to claim 4 , Wherein the first illumination (L1) and/or the second illumination (L2) according to claim 4 are identical to the at least one lighting (L0). claim 1 . arrangement according to claim 4 , wherein the arrangement comprises, in addition to the first illumination (L1) and second illumination (L2), a third illumination with polarized light, the third illumination being arranged such that rays of the third illumination on the surface point (P) outside the glancing angle to the first Polarization camera (C1) and are reflected outside of the grazing angle to the second polarization camera (C2). Arrangement according to one of Claims 1 until 3 , having an evaluation unit, wherein the evaluation unit is adapted to calculate an unpolarized light component from recordings of the two polarization cameras (C1, C2) and to determine corresponding points on the surface using the unpolarized light component. arrangement according to claim 4 , the arrangement being adapted to sequentially switch the plurality of lights from different directions. arrangement according to claim 4 or 10 , having an evaluation unit, wherein the evaluation unit is adapted to calculate a polarized light component and its polarization direction from recordings of the first polarization camera (C1) and the second polarization camera (C2) and to determine corresponding points on the surface using the polarized light component and its polarization direction . arrangement according to claim 8 , having an evaluation unit, wherein the evaluation unit is adapted, from recordings of the first polarization camera (C1) and the second polarization camera (C2), an unpolarized light component in an illumination with the first illumination (L1) and/or second illumination (L2) and/or to determine the third illumination, to determine a first polarized light component from recordings by the first polarization camera (C1) when illuminated with the second illumination (L2), from recordings by the second polarization camera (C2) to determine a second polarized light component when illuminated with the first illumination ( L1) and to determine corresponding points on the surface from the unpolarized light component, the first polarized light component and the second polarized light component.

Images