DE102016109131B4 - Method for the three-dimensional detection of an object - Google Patents

Abstract

The invention relates to a method for three-dimensional detection of an object, in which - the object is successively illuminated with a plurality of light patterns that differ in their focus position, - is detected by the object reflected light of the light pattern by means of a camera, wherein the focus position of the camera the focus position of the respective light pattern is adjusted and the camera generates an image of the object for each focal position, - the three-dimensional shape and / or the spatial position of the object is determined on the basis of sharp regions of the light pattern in the images and the focal positions.

Description

The present invention relates to a method for three-dimensional detection of an object.

The three-dimensional detection of objects is needed in many applications. For example, industrial manufacturing processes can be controlled and controlled in a simplified manner if the object to be tested or the object to be processed is known in its three-dimensional form and / or its spatial position.

For the three-dimensional detection of objects, various methods are known, for example, active methods, such as time-of-flight (TOF) measurements, which, for. B. be carried out by means of laser scanners. Alternatively, passive methods may be used, in which z. As stereo cameras are used to view the object from different angles and in this way to draw conclusions about the three-dimensional properties of the object.

Disadvantageously, the technical outlay for time-of-flight methods is very high. With stereo cameras, the disadvantage can occur that certain areas of the object are occluded for a lens of the stereo camera (occlusion), so that complete three-dimensional object contours can only be achieved in idealized applications.

From the US Pat. No. 6,219,461 B1 It is known to take a variety of pictures using structured lighting. The WO 2015/177784 A2 teaches using contrasts of an image to determine sharp areas of a projected pattern. The EP 0 890 822 A2 discloses focusing different colors at different distances to perform a depth measurement. From the DE 10 2014 119 126 B3 is a strip projector with a gel lens known. In "K. Mishra: Recent Developments in Optofluidic Lens Technology "(DOI: 10.3390 / mi7060102) describes variable focal length lenses. Furthermore, from the DE 10 2011 052 802 A1 the projection of light using a variety of micro-optics known. The US Pat. No. 6,229,913 B1 teaches the acquisition of depth information using projected light patterns. From the US 2011/0229840 A1 For example, a method is known in which a light pattern on an object is detected by a telecentric optical system. The optical system is moved mechanically. In this process, sharp areas are determined in different images based on their contrast. Finally, the reveals US 5,608,529 A a device for measuring three-dimensional objects, the device projecting light patterns onto objects.

It is the object underlying the invention to provide a method and a device for the three-dimensional detection of an object, which allow a concealment-free three-dimensional object detection in a simple manner.

This object is solved by the subject matters of the independent claims.

• – das Objekt nacheinander mit mehreren Lichtmustern beleuchtet wird, wobei sich die Lichtmuster in ihrer Fokuslage unterscheiden,
• – von dem Objekt zurückgestrahltes Licht der Lichtmuster mittels einer Kamera erfasst wird, wobei die Fokuslage der Kamera an die Fokuslage des jeweiligen Lichtmusters angepasst wird und die Kamera für jede Fokuslage ein Bild des Objekts erzeugt,
• – anhand von Kontrastunterschieden zwischen den Bildern und anhand von scharfen Bereichen des Lichtmusters in den Bildern und der Fokuslagen die dreidimensionale Form und/oder die räumliche Position des Objekts ermittelt wird, und
• – das Lichtmuster zumindest lokal selbstunähnlich ist.

According to claim 1, a method for the three-dimensional detection of an object is specified, in which

• The object is successively illuminated with a plurality of light patterns, the light patterns differing in their focus position,
• The light of the light pattern reflected back from the object is detected by means of a camera, wherein the focus position of the camera is adapted to the focal position of the respective light pattern and the camera generates an image of the object for each focal position,
• - Determining the three-dimensional shape and / or the spatial position of the object based on contrast differences between the images and on the basis of sharp areas of the light pattern in the images and the focal positions, and
• - The light pattern is at least locally selbstunähnlich.

In this respect, a-priori information can be used to determine the spatial location of sharp image content. Thus, it can be determined both on the basis of the a priori known focus position of the projected light pattern as well as on the basis of the (also known) focus position of the camera, how far certain areas of the object are removed from the camera. The light patterns can be projected in particular with little or limited depth of field, so that the light patterns are sharply displayed only in areas of the object that have a small distance from the focus position of the light pattern.

In a corresponding manner, the camera can be designed such that the images of the object generated by the camera have only a limited or limited depth of focus (also called depth of field). If it is known that the camera has focused on a certain distance, then it can be assumed that sharp areas of the light pattern and / or the object are at the distance to the camera, on which the camera has focused. In this way it can be determined that the sharp areas of the object are in the known distance or in the known distance.

The focal position of a projected light pattern is a location in the space in which the image of the projected light pattern is in focus. The focal point of the camera is a place called, in an object is located when it is sharply imaged by the camera. A focus position of a light pattern can thus be indicated, for example, as a distance measured by a projector and a focus position of the camera, for example, as a distance measured from the camera.

Based on the known focus positions, the spatial position and / or the three-dimensional shape of the object can accordingly be determined. The focal positions, by means of which the spatial position is determined, may comprise at least one focus position of the camera and at least one focal position of a light pattern. Alternatively, the focal positions, on the basis of which the spatial position is determined, include focal positions of the camera and / or focal positions of the light patterns.

The light patterns can be output by a projection unit and projected onto the object.

The course of the method according to the invention will be shown in more detail below.

First, the object is illuminated with a first light pattern having a first focus position, wherein the camera is adjusted such that the focus position of the camera corresponds to the first focus position of the light pattern. The distance between the first focus position of the camera is known. Sharp areas can then be determined in the image captured by the camera, which can be done, for example, by detecting the contrast in different image areas. In the case of the sharp image areas, it is particularly assumed that these image areas are in the first focus position. Accordingly, the distance and the direction of the sharp area of the object in space can be determined by the known distance of the first focus position and the arrangement of the sharp area in the camera image.

Subsequently, a second pattern of light is projected onto the object, which has a second focus position. The second focus position may, for example, be further away from the camera than the first focus position. The camera is then adjusted so that its focus position is adapted to the second focal position of the second light pattern. The now sharply appearing areas of the object or areas of the object in which the light pattern is sharp are then detected and it can again be determined on the basis of the known distance of the second focus position and the arrangement of the now sharp areas in the second recorded camera image which regions of the object have a distance from the camera corresponding to the distance between the camera and the second focus position.

In a corresponding manner, a multiplicity of different images are recorded with the camera according to the invention, each image being recorded with a different focal position of the projected light patterns and a focus position of the camera adapted to the focal position of the light patterns.

In this way, the object can be scanned in a plurality of steps so that information about the three-dimensional shape and / or the spatial position of the object are present after the projection of the different light patterns and the recording of the corresponding images.

By setting to a specific focus position for both the camera and the light pattern, the effect can be enhanced that the light pattern can be sharply imaged only with a defined focal position of the camera, whereby the accuracy of the depth resolution can be increased.

The light patterns advantageously differ only by a different focus position.

Additional information about the three-dimensional shape of the object can also be calculated from distortions of the light pattern when projecting onto the object.

The three-dimensional shape and the spatial position of the object can finally be output as depth information, for example via an interface.

Advantageous developments of the invention are described in the description, the drawings and the dependent claims.

According to a first advantageous embodiment, a first depth map is determined from the sharp regions of the light pattern in the images and the focus position of the images. The first depth map thus uses the a-priori information about the focus position (i.e., the focal plane) of the images and the light patterns.

The focal position is preferably a plane, ie. H. the light pattern is sharply displayed in a focal plane. Alternatively, the focal position (of light pattern and / or camera) may also define a curved surface which is, for example, hemispherical. Such a curved surface can be used for rapid detection of curved objects.

According to the invention, the three-dimensional shape and / or the spatial position of the object is also determined on the basis of contrast differences between the images. The light patterns and / or, for example, the boundaries of the object may contribute to the contrast differences. As a contrast, a different brightness and / or a different color of adjacent regions of an image generated by the camera is defined. Differences in contrast are accordingly differences in the contrast between different images in each case corresponding positions. For example, a position or boundary of the object can be considered as recognized when the contrast for a focal position increases particularly strong, since then it can be assumed that the light pattern is focused and provides a high contrast. Accordingly, the contrast difference between a sharp-light image and a non-sharp-light image is particularly large at such a location. The contrast differences can be used together with a respective focal position (the camera and / or the projected light pattern) to determine the three-dimensional shape and / or the spatial position of the object.

Preferably, a second depth map is determined from the contrast differences between the images. The contrast differences can be preprocessed separately from the information of the first depth map separately. For each point (i.e., each pixel) of one of the images taken by the camera, a depth value may be entered in the second depth map.

According to a further advantageous embodiment, the three-dimensional shape and / or the spatial position of the object is also determined based on color differences between the images. The color of the images captured by the camera at corresponding positions may change depending on whether, for example, the light pattern is focused. Therefore, between different images with different focus positions, a color difference may arise at positions corresponding to each other. So z. For example, there is a particularly large color difference between images showing a blurred light pattern and an image showing a sharp light pattern. Due to the color difference of the image with a sharp light pattern compared to the images with a blurred light pattern, the image can be detected with a sharp light pattern. Due to the knowledge that the light pattern is displayed sharply, it can be deduced that in the image with a sharp light pattern, the object must also be in focus, which in turn enables the distance of the object to be determined.

In contrast to the use of contrast differences, the color differences can be determined by respectively corresponding image areas of different images, i. H. from images taken at different focal positions of the camera. In the case of such a procedure, color differences that may exist within a single image are therefore not taken into account. The color differences can be used together with a respective focal position (the camera and / or the projected light pattern) for determining the three-dimensional shape and / or the spatial position of the object.

Preferably, mutually corresponding pixels in the images of the camera can be used for the contrast differences and / or the color differences. Alternatively, pixels are used, each corresponding to the same position on the object.

More preferably, a third depth map is determined from the color differences between the images. The third depth map can be created independently of the other depth maps. For each point of the images, a depth value can again be entered in the third depth map.

Due to the use of both the a-priori information, the contrast differences, as well as the color differences, so different measures are used simultaneously as a distance ruler for the process. In this way, a particularly fault tolerant and thus robust and industrially applicable method for the three-dimensional detection of objects can be created. In addition, no complex equipment is required for all measured variables, so that a compact realization is possible. Another advantage is that the acquisition of the various measured quantities can be done in a short time, so that the method allows a real-time output of three-dimensional data.

According to a further advantageous embodiment, a result depth map is created from at least two of the depth maps by means of data fusion. In the case of data fusion, for example, averaging can take place between the values of two or three depth maps. For example, a weighted averaging can take place, wherein z. For example, if the first depth map or the second depth map experiences the heaviest weighting, it is assumed that this depth map contains the most accurate information.

Preferably, the weighting factors can also be adjustable. In this way, the method can be adapted, for example, to the intended use and / or the objects to be measured.

According to a further advantageous embodiment, different colors are used for successively projected light patterns. For example, the light patterns may be projected from a matrix of RGB LEDs (Red-Green-Blue LEDs). A light pattern can each consist entirely of one color. By different light patterns with different colors, as explained later, the chromatic aberration of the camera can be exploited. In addition, such a color can be used for the light pattern, which has a particularly strong contrast to the color of the object.

According to a further advantageous embodiment, the focus position of the camera is changed synchronously with the focus position of the light patterns. This means that the camera adjusts its focus position in such a way that it can sharply represent the projected light patterns. When the focus position of the light pattern changes, a change in the focus position of the camera can take place at the same time. Such a synchronization can be carried out, for example, via a synchronization line, the synchronization accelerating the passage through all depth planes or focal positions.

Alternatively or additionally, a delay in the change in the focal positions of the camera or the light pattern can be integrated, so z. B. the focus position of the camera of the focus position of the light pattern repeatedly briefly follows (or vice versa). In this way, additional depth information can be obtained by interpolation. For this purpose, the camera can take additional pictures when the focal positions of the camera and the light patterns differ.

According to a further advantageous embodiment, the focal position of the light patterns is changed at least every 100 ms, preferably at least every 10 ms, particularly preferably at least every 1 ms. Due to such a rapid change in the focus position, the entire object can be scanned in a short time.

According to the invention, the light patterns are at least locally self-similar. The light patterns can also be completely self-dissimilar. This means that the object is illuminated with a light pattern whose pattern structures repeat at the earliest within a certain distance or not at all within the light pattern. The self-similar pattern structures can make it easier to identify corresponding image areas or object areas in different images, thus facilitating, for example, the correct assignment of contrast differences or color differences.

The light patterns can be two-dimensionally correlated illumination patterns, in particular optimal two-dimensionally correlated illumination patterns. The light patterns may have, for example, rectangular or square areas with different brightnesses.

Alternatively, however, the light patterns may also include periodic structures, such as line patterns.

According to a further advantageous embodiment, the light patterns have a depth of focus which is less than 30%, preferably less than 10%, of the distance between the camera and the object. Generally speaking, the depth of focus of the light patterns and the depth of focus of the camera may be limited.

The camera may have the same or less depth of field in capturing the images as the light patterns. The depth of field of the projected light patterns and / or the camera may also be less than 1% or 0.1% of the distance between the camera and the object. Lower depth of focus can provide increased accuracy in detecting the distance of the object from the camera. This allows more accurate depth maps to be created.

An intentional reduction of the depth of field of the light patterns represents a departure from the usual use of light patterns (eg in light-section methods), which are projected with as much depth of field as possible in order to illuminate objects at different distances with a sharp light pattern.

In this case, the depth of field is understood to mean a depth of field that extends from the near point of the camera (or the projection unit) to the far point of the camera (or the projection unit). The near and far points can be calculated in particular from the hyperfocal distance, the lens focal length of the camera (or the projection unit) and the object's distance.

• – einer Projektionseinheit, welche ausgebildet ist, das Objekt nacheinander mit mehreren Lichtmustern zu beleuchten, die sich in ihrer Fokuslage unterscheiden,
• – einer Kamera, die von dem Objekt rückgestrahltes Licht der Lichtmuster erfasst, und
• – einer Steuereinheit, welche eingerichtet ist, die Fokuslage der Kamera an die Fokuslage des jeweiligen Lichtmusters anzupassen, mittels der Kamera für jede Fokuslage ein Bild des Objekts zu erzeugen und anhand von Kontrastunterschieden zwischen den Bildern und anhand von scharfen Bereichen des Lichtmusters in den Bildern und der Fokuslagen die dreidimensionale Form und/oder die räumliche Position des Objekts zu ermitteln.

Erfindungsgemäß sind die Lichtmuster zumindest lokal selbstunähnlich.The invention further relates to a device for three-dimensional detection of an object, with

• A projection unit which is designed to illuminate the object successively with a plurality of light patterns which differ in their focus position,
• A camera which detects light of the light patterns reflected by the object, and
• A control unit which is set up, the focus position of the camera at the focal position of the adapt the respective light pattern, to generate an image of the object for each focal position by means of the camera and to determine the three-dimensional shape and / or the spatial position of the object based on contrast differences between the images and on the basis of sharp areas of the light pattern in the images and the focal positions.

According to the invention, the light patterns are at least locally self-similar.

In particular, the camera may be a high dynamic range (HDR) camera. The camera and the projection unit are preferably arranged in a common housing and / or adjacent to each other or adjacent to each other. The camera and the projection unit can look at the object from substantially the same direction.

Incidentally, the statements made with regard to the method according to the invention apply correspondingly, in particular with regard to advantages and advantageous embodiments.

In accordance with an advantageous embodiment of the device according to the invention, the camera has an objective with a membrane lens, in particular the camera comprises an objective with a silicone membrane lens and a piezoactuator.

The membrane lens can be actuated in such a way that the focus position of the camera can be changed in less than 100 ms, preferably in less than 10 ms, particularly preferably in less than 1 ms. Here, a change in the focus position is understood to mean that the focus position of the camera is adapted to a new or "neighboring" focal position of the projection unit. After the focus position of the camera has been adapted to the focus position of the projection unit, the camera can be used to generate an image of the object. This means that the camera can generate an image of the object at least every 100 ms, preferably at least every 10 ms, particularly preferably at least every 1 ms.

According to a further advantageous embodiment, the camera is not a stereo camera. For example, the camera will have exactly one lens. The camera can accordingly have exactly one light pickup, exactly one light sensor and / or exactly one light path. By using just one lens, occlusions that can occur with stereo cameras are avoided.

As an alternative to the camera with exactly one lens and a stereo camera could be used if z. B. the object geometry and / or the imaging geometry allows this.

According to a further advantageous embodiment, the camera has a, in particular distance-dependent, chromatic aberration. It can therefore be used deliberately a camera with a lens with particularly large chromatic aberration. In this way, for. B. only areas of the object are focused, which have a predetermined color. It is also possible to focus only the (monochrome) light pattern due to the chromatic aberration. Consequently, a certain independence from the color of the object is achieved, since the object itself can be partially "hidden" by the chromatic aberration.

The chromatic aberration can also be distance-dependent and z. B. increase with increasing distance. The chromatic aberration can be selected such that the focal length of the lens differs between red shades (wavelength> 630 nm) and blue shades (wavelength <490 nm) by at least 1%, preferably by at least 5%, particularly preferably by at least 10% ,

Alternatively or additionally, a, in particular distance-dependent, chromatic aberration may be provided in the projection unit.

According to a further advantageous embodiment, the projection unit comprises a sequentially switchable illumination matrix of illumination units, wherein the illumination matrix emits light through a matrix of micro-optics. The illumination matrix may be a matrix of LEDs. The LEDs are preferably RGB LEDs. Each LED can have exactly one micro-optic, e.g. As a microlens, be assigned. By means of the micro-optics, the light patterns can be generated, the micro-optics each having a different focal position.

As an alternative to the micro-optics, it is also possible to provide precisely one light source which emits the light patterns by means of an adjustable optical system with different focal positions. The adjustable optics may include a diaphragm lens with a piezo actuator.

Hereinafter, the invention will be described purely by way of example with reference to the accompanying drawings. Show it:

1 an inventive device for three-dimensional detection of an object; and

2 a locally self-similar light pattern.

1 shows a device 10 , which for the three-dimensional detection of an object 12 is trained. The object 12 is spaced from the device 10 arranged. The device 10 includes an imaging optics 14 containing a variety of microlenses 15 which has different light patterns 16 project.

An exemplary light pattern is more detailed in FIG 2 shown. The light pattern shown there 16 is a two-dimensional array of square illumination points 18 with different brightness. The light pattern 16 has no periodic structure and no line pattern, but is self-similar.

As in 1 are shown adjacent to the imaging optics 14 a variety of LEDs 20 arranged, which emit light of different color, each with an LED 20 through one of the microlenses 15 the imaging optics 40 radiates through. The microlenses 15 the imaging optics 14 are formed such that different microlenses 15 the light pattern 16 in each case at different distances from the device 10 sharp and with a shallow depth of field.

The LEDs 20 be from an LED driver 22 driven. The microlenses 15 , the LEDs 20 and the LED driver 22 form a projection unit.

The device 10 further comprises a receiving optics 24 , wherein by the receiving optics 24 passing light from a silicone membrane lens 26 is distracted. The receiving optics 24 and the silicone membrane lens 26 together form a lens. The refractive power of the silicone membrane lens 26 can by a piezo actuator 28 be changed in very short time intervals, for example in less than 1 ms. That through the silicone membrane lens 26 passing light is applied to an HDR CMOS sensor 30 (High Dynamic Range CMOS sensor) projected. The HDR CMOS sensor 30 creates images of the object 12 ,

A flow control 32 provides by means of a synchronization line 34 that the focal point of the imaging optics 14 and the focal position of the receiver optics 24 , Silicone membrane lens 26 , Piezo actuator 28 and HDR CMOS sensor 30 each camera correspond.

By means of a signal processor 36 the images of the camera are evaluated and 3D data 38 generated. The 3D data 38 be through an interface 40 output.

In operation, the device generates a first pattern of light 16 with a focus position that is in a first focal plane 42a lies. At the time of projection of the light pattern 16 in the focal plane 42a the camera is also on the focal plane 42a focused and produces a corresponding image. The focal positions of the light pattern 16 and the camera correspond to the generation of the image and are in the focal plane 42a ,

In an analogous manner, further light patterns will follow 16 in other focal planes 42b . 42c . 42d . 42e . 42f and 42g projected. The focal position of the camera is synchronized to the currently used focal plane 42b to 42g in focus. After focusing, the camera produces one image at a time.

Thus, in the example chosen here, seven images are generated, each image being evaluated as to whether there are sharp areas of the light pattern 16 and / or sharp areas of the object 12 can be recognized. Based on the sharp areas and the known focal position 42 is done by means of the signal processor 36 generates a first depth map.

Additionally, contrast differences between the seven images are determined to create a second depth map. A third depth map is generated based on color differences between the images. Also for the contrast differences and the color differences become the known focal positions 42 used.

After determining the three depth maps, the signal processor performs 36 an averaging over the three depth maps and gives the resulting 3D data 38 over the interface 40 out.

LIST OF REFERENCE NUMBERS

10

contraption

12

object

14

imaging optics

15

microlens

16

light pattern

18

lighting point

20

LED

22

LED driver

24

receiving optics

26

Silicone membrane lens

28

Piezo actuator

30

HDR CMOS sensor

32

flow control

34

synchronization line

36

signal processor

38

3D data

40

interface

42, 42a-g

focal plane

Claims (16)

Method for the three-dimensional detection of an object ( 12 ), in which - the object ( 12 ) successively with several light patterns ( 16 ), whereby the light patterns ( 16 ) in their focus position ( 42 ), - of the object ( 12 ) reflected light of the light pattern ( 16 ) is detected by means of a camera, wherein the focus position ( 42 ) of the camera to the focus position ( 42 ) of the respective light pattern and the camera for each focus position ( 42 ) an image of the object ( 12 ), - based on contrast differences between the images and on the basis of sharp areas of the light pattern ( 16 ) in the images and the focal positions ( 42 ) the three-dimensional shape and / or the spatial position of the object ( 12 ), - the light patterns ( 16 ) are at least locally self-similar. A method according to claim 1, characterized in that from the sharp areas of the light pattern ( 16 ) in the images and the focal positions of the images a first depth map is determined. A method according to claim 1 or 2, characterized in that from the contrast differences between the images, a second depth map is determined. Method according to at least one of the preceding claims, characterized in that the three-dimensional shape and / or the spatial position of the object ( 12 ) is determined by color differences between the images. A method according to claim 4, characterized in that from the color differences between the images, a third depth map is determined. Method according to claims 2, 3 and 5, characterized in that from at least two of the depth maps by means of data fusion a result depth map is created. Method according to at least one of the preceding claims, characterized in that for successively projected light patterns ( 16 ) different colors are used. Method according to at least one of the preceding claims, characterized in that the focus position ( 42 ) of the camera in sync with the focus position ( 42 ) the light pattern ( 16 ) is changed. Method according to at least one of the preceding claims, characterized in that the focus position ( 42 ) the light pattern ( 16 ) is changed at least every 100 ms, preferably at least every 10 ms, more preferably at least every 1 ms. Method according to at least one of the preceding claims, characterized in that the light patterns ( 16 ) have a depth of field that is less than 30%, preferably less than 10%, of the distance between the camera and the object. Contraption ( 10 ) for the three-dimensional detection of an object ( 12 ), with - a projection unit, which is designed, the object ( 12 ) successively with several light patterns ( 16 ), with the light patterns in their focal position ( 42 ), a camera that is separated from the object ( 12 ) reflected light of the light pattern ( 16 ), and - a control unit ( 32 . 34 . 36 ), which is set up, the focus position ( 42 ) of the camera to the focus position ( 42 ) of the respective light pattern ( 16 ), by means of the camera for each focal position ( 42 ) an image of the object ( 12 ) and contrast differences between the images and sharp areas of the light pattern ( 16 ) in the images and the focal positions ( 42 ) the three-dimensional shape and / or the spatial position of the object ( 12 ), the light patterns ( 16 ) are at least locally self-similar. Apparatus according to claim 11, characterized in that the camera is a lens with a membrane lens, in particular a silicone membrane lens ( 26 ) with a piezo actuator ( 28 ), wherein the membrane lens is actuatable such that the focus position ( 42 ) of the camera in less than 100 ms, preferably in less than 10 ms, more preferably in less than 1 ms, is variable. Apparatus according to claim 11 or 12, characterized in that the camera has exactly one lens. Device according to at least one of claims 11 to 13, characterized in that the camera has a, in particular distance-dependent, chromatic aberration. Device according to at least one of Claims 11 to 14, characterized in that the projection unit has a sequentially switchable illumination matrix of illumination units ( 20 ), wherein the illumination matrix transmits light through a matrix of micro-optics ( 15 ) emitted. Device according to at least one of claims 11 to 15, characterized in that the camera and the projection unit are arranged in a common housing and / or adjacent to each other or adjacent to each other.

Images