DE102019116506A1 - Stereo camera system - Google Patents

Abstract

The present invention relates to a stereo camera system for measuring three-dimensional surface topologies. With this, depth information of objects moving on a conveyor in front of the stereo camera system can be recorded. The stereo camera system can be part of a production monitoring device.

Description

The present invention relates to a stereo camera system for measuring three-dimensional surface topologies. With this, depth information of objects moving on a conveyor in front of the stereo camera system can be recorded. The stereo camera system can be part of a production monitoring device.

Production monitoring devices with a conveyor device and a camera for optically scanning the products conveyed on the conveyor device are known and are used, for example, in package sorting systems or production systems.

Usually, a measuring device, such as. B. a light curtain, the height of the product or object to be scanned is determined. The camera has an adjustable lens which is set to a previously detected height of an object to be scanned in order to scan its surface correctly. The camera can have a zoom lens whose focal length is variable so that it can be adjusted accordingly for scanning objects of different heights. It is also possible to use a lens with a fixed focal length, the focus of which is tracked so that different heights are sharply imaged on the sensor.

With zoom lenses, several lenses are moved relative to one another. The zoom lens has a movable mechanism within the lens that moves several lenses relative to one another. With a zoom lens, the scale can be kept constant over the adjustment path.

If a lens with a fixed focal length is used, the entire lens must be shifted along the optical axis so that the focus area is at the desired height. The image scale increases when the optical length is reduced.

Both a zoom lens and a fixed focal length lens have mechanical parts that need to be moved. These movements are subject to wear. In industrial applications, usage cycles of several million are required. These usually far exceed the life of these moving parts.

If, in addition to the optical, height information is also to be evaluated in the product monitoring device, various methods have become established.

One possibility is the so-called light section method. A thin straight strip of light is thrown onto the object to be measured and an area camera then records this strip of light. The surface of the object can be calculated back using the distortion that the object creates on the light strip.

Another option is stereoscopy. A camera system records the object from two different viewing directions and can determine the topological surface information of the object based on the offset of corresponding image points. However, the disadvantage of this method is the restriction to one focal plane.

Such stereo camera systems have two camera modules with which the same object area can be scanned from two different viewing directions. The camera modules are provided with a lens in order to image the object area on a sensor of the respective camera module. An optical axis of the objective is an axis of symmetry of the objective, which runs approximately transversely to the surfaces of one or more lenses contained in the objective. The main plane of the objective on the object side is regarded as the objective plane. A sensor plane is the plane in which the light-sensitive sensor surface lies. An object plane is the plane that is sharply imaged from the lens onto the sensor surface.

In conventional stereo camera systems, the lens plane, the object plane and the sensor plane are usually arranged parallel to one another. The parallel alignment of the object plane to the objective plane and to the sensor plane is referred to below as the parallelism arrangement.

A base angle is decisive for the resolution of a three-dimensional surface topology with a stereo camera system α , which is the angle that is spanned by the viewing directions of the two camera modules of the stereo camera system on the object area. The base angle α is essentially determined by the size of a base B. , which is the distance between the two sensor surfaces of the two camera modules S1, S2. This means the larger the base B. is, the larger the base angle α and accordingly the height resolution of the stereo camera system is better ( 8th ).

Unless otherwise stated, the following is based on the assumption that the distance between the sensor plane SP (sensor plane), objective plane LP (lens plane) and object plane OP (object plane) remains constant, so that only the base B. the base angle α certainly.

Lenses have an opening angle γ which indicates the maximum angle of incidence of the incident light to the optical axis, within which an object can be imaged essentially without distortion ( 8th ). Depending on the design of the lens, the maximum opening angle of the lens can vary. In the following, the beam path is referred to as the beam through the objective, which includes all the beams within the opening angle and extends between the sensor surface and the object.

In the case of a stereo camera system with the parallelism arrangement explained above, however, the base cannot be selected to be arbitrarily large, since if the base is too large, the sections in the object area limited by the angle of incidence only overlap slightly or not at all, and thus the sharply mapped section of the object area small or nonexistent.

Only an overlapping area, i.e. the area of the object that is detected by both sensors at the same time, can be used for an evaluation of the three-dimensional surface topology. The edge areas of this overlap area are at a large opening angle of the lens γ pictured. The distortion is therefore greatest in the edge areas of the overlap area. Furthermore, the imaging performance, which can be characterized, among other things, by a color error and a modulation transfer function (MTF), is worse in the edge area of the overlap area than in the center of the overlap area. The color error describes the loss of color reproduction. The MTF describes a loss of detail in contrast.

In "Scheimpflug stereocamera for particle image velocimetry in liquid flows", Ajay K. Prasad and Kirk Jensen, APPLIED OPTICS 34, 30 (1995) , describes a stereo video camera with Scheimpflug condition and an area sensor. The system is used to determine the velocity distribution of small particles in liquids.

With such a Scheimpflug camera, the parallelism arrangement is canceled by tilting the lens plane LP relative to the sensor plane, so that a large base is possible without the object area being restricted by the limited angle of incidence. With a Scheimpflug camera, the so-called Scheimpflug condition is fulfilled, according to which the objective plane, the sensor plane and the object plane intersect in a common cutting axis. A Scheimpflug angle β is spanned by the object level OP and the sensor level SP ( 9 ). If the Scheimpflug condition is met, the object area is sharply mapped onto the sensor by the lens. In the literature, a cutting axis is sometimes described at an infinite distance from the stereo camera system. In this special case, the Scheimpflug condition would correspond to the parallelism arrangement. For the following consideration, however, the Scheimpflug condition is not fulfilled if these planes or axes intersect at infinity. They have to intersect at a finite point so that the Scheimpflug condition is fulfilled.

With stereo camera systems with Scheimpflug condition, on the other hand, a wider base angle can be selected than with stereo camera systems without Scheimpflug condition with the same objectives and image quality. If the base is large, the angle of incidence can be reduced and thus optimized by inclining the lenses. This means that there are also lenses with a small opening angle γ suitable. Due to the smaller opening angle, the condition is that the image quality, in particular the MTF, is more constant over the overlap area than with stereo camera systems without Scheimpflug condition. A linear distortion along the overlap area is accepted as this can be corrected by rectification. The larger the base is chosen, the more the sensor plane and lens plane must be tilted relative to the object plane in order to reduce the angle of incidence.

Another problem with stereo camera systems without Scheimpflug condition is back reflection. Due to the parallel arrangement with a narrow base angle, the lens not only images the object on the sensor, but also the sensor on the object. This image of the lens can be imaged back onto the sensor by reflection on a reflective surface of the object. The sensor thus records its own mirror image. By tilting, e.g. by Scheimpflug, a broad base angle can be chosen, which avoids such back reflection. Even if the reflection continues, the image of the reflection can e.g. also represent a neutral background. The effects of reflection are thus reduced.

The advantages of stereo camera systems with Scheimpflug condition are explained above. However, with stereo camera systems with Scheimpflug condition, effects occur which negatively influence the quality of the stereoscopy when using area sensors. This is due to the so-called keystone effect, which describes the distortion of the projected sensor surface.

A two-dimensional image of the object area is recorded with an area sensor. In conventional stereo camera systems, one edge of the area sensor is parallel to the base arranged. The main axes of the recorded two-dimensional image thus also extend either parallel to the base and transversely to the base. The direction parallel to the base is hereinafter referred to as the base direction and the direction transverse to the base is hereinafter referred to as the transverse direction.

When the object area is imaged by a stereo camera system, the image is compressed both in the transverse direction and in the base direction. This compression is caused by the fact that the individual points of a flat object are spaced differently from the sensor surface. The further two points are away from the sensor surface, the smaller the distance between the points is mapped. The compression becomes stronger the larger the base is. A transverse compression occurs in the transverse direction and a base compression occurs in the base direction.

For example, a square object area with the smallest possible base and a central arrangement of the camera systems is mapped almost distortion-free. With a large base, however, the image of the square forms the shape of a trapezoid or keystone. The distance between the corners that are further away from one another is shown smaller than the distance between the corners that are closer to the objective.

The effect occurs with stereo camera systems both with and without Scheimpflug condition. In the case of those without Scheimpflug condition, this effect is also limited by the limitation of the basis. Since stereo camera systems with Scheimpflug condition can have a particularly large base, the effect is particularly pronounced here.

The keystone effect causes a loss of resolution. The resolving power is the minimum distance that two point objects must have so that they can be perceived as separate objects. Due to the compression, two points that are further away from the sensor surface are mapped closer together. From a certain distance from the lens, these points can no longer be perceived as separate. The resolving power parallel to the base is hereinafter referred to as the basic resolving power and the resolving power across the base is hereinafter referred to as the transverse resolving power.

Furthermore, the keystone effect increases the computational effort. In order to use the images for an evaluation of the three-dimensional surface topology, both the base and the transverse compression must be calculated from the image data by rectification. Rectification is a process for eliminating geometric distortions in image data that can arise, for example, from central perspective recordings. The rectification shifts the individual pixels in such a way that when the pixels of the two sensors are superimposed, the same position is displayed on the object. Furthermore, the distortion is corrected so that the calculated disparity is linearly dependent on the height profile of the object.

In stereoscopy, each point of an image of a camera module is assigned a point of an image of the second camera module. If the sensor is inclined in relation to the object, a rectangular sensor detects a trapezoid T1 of the object surface due to the keystone effect ( 10 ). The other area sensor detects a second trapezoid T2 of the object area. When assigning the points, these trapezoids overlap, the long front side coinciding with the short front side of the other trapezoid. This results in a hexagonal overlap area (hatched in 10 ).

The disadvantage here is that part of the information recorded is lost from the non-overlapping areas. Only the overlap area can be used to evaluate the three-dimensional surface topology.

Furthermore, the opening angle of the lens must be large enough that the overlap area is completely covered. This is the case when a diagonal D of the hexagonal overlap area is captured by the objective.

A method for scanning surfaces with a stereo camera with at least two camera modules emerges from DE 10 2015 11 11 20 A1. The camera modules are arranged in such a way that they each capture an image of a common area of a surface to be scanned. The camera modules can have a line camera with several different rows of sensors. A separate line image is recorded with each sensor line. The individual line images are superimposed by means of a geometric image.

Stereo camera systems with line sensors and a parallelism arrangement have the same problems as stereo camera systems with area sensors and a parallelism arrangement. As described above, the base cannot be chosen to be arbitrarily large, since if the base is too large, the sections in the object area limited by the angle of incidence only overlap slightly or not at all and the sharply mapped section of the object area is therefore small or no longer present .

In principle, stereo camera systems are conceivable that meet neither the Scheimpflug condition nor the parallelism arrangement. Thus, only the object plane OP and the sensor plane SP could be parallel to one another, while the objective plane LP is tilted relative to the other two planes in such a way that the object area is not restricted by the limited angle of incidence. An imaging plane, which corresponds to the plane on which the objective images the object planes in focus, is arranged at an angle to the object plane or the object axis. Only in the intersection of the imaging plane and the sensor plane is the object area sharply mapped on the camera sensor ( 11 ). The greater the tilt of the objective axis relative to the object axis, the narrower is a focus area in which the object is sharply imaged. The greater the depth of field TS, the wider the focus area. In the case of a narrow focus area, sufficient information is only available from a small section of the object for an evaluation of the three-dimensional surface topology.

However, a sharp image is very important for an image of the three-dimensional surface topology. It is also desirable to use the entire surface of a sensor.

In the following, stereo camera systems without Scheimpflug condition are understood to mean stereo camera systems which have the parallelism arrangement.

A system is known from “Line cameras for the inspection of the sealing surfaces of an O-ring groove”, Photonik 5/2012, in which three line cameras are operated in parallel in order to image one side of a groove each. The side walls of the groove are imaged by line cameras with Scheimpflug conditions.

From the DE 10 2013 103 897 A1 shows a camera module for line-by-line scanning of an object. The camera module comprises a line-shaped sensor and a lens for imaging the object on the sensor. The camera module has several line sensors. The line sensors are arranged at different distances from the lens, whereby image lines with different distances from the lens are mapped onto the respective line sensors.

The invention is based on the object of creating a stereo camera system with which a good spatial resolution is achieved over a large scanning area in a simple manner.

Another object of the invention is to create a stereo camera system which allows three-dimensional objects to be reliably scanned.

Another object of the invention is to create a stereo camera system with which a large image area can be captured in a simple manner.

An additional object of the invention is to provide a stereo camera system that is compact.

Another object of the invention is to create a stereo camera system that images various object planes well.

One or more objects are achieved by the subjects of the independent claims. Advantageous developments and preferred embodiments form the subject of the subclaims.

A stereo camera system according to the invention for measuring three-dimensional surface topologies of an object has at least two line-shaped sensor areas and at least one lens for imaging an object onto the sensor areas. The objective is arranged in such a way that an object area is mapped onto two independent beam paths, each on a sensor area. The stereo camera system is characterized in that the Scheimpflug condition is fulfilled along at least one of the beam paths.

As already explained, a stereo camera system taking into account the Scheimpflug condition can have a larger base than a stereo camera system with a parallel arrangement without the images being recorded by the lens. The larger base increases the resolution of the three-dimensional surface topology.

Such a stereo camera system can be used to scan an object area line-by-line transversely to the line direction, the stereo camera system being moved relative to the object. The direction parallel to the line sensors is referred to as the line direction and the direction transverse to the line sensors is referred to as the scanning direction. The distance between two points that are located in the scanning direction is recorded with the same resolution regardless of the keystone effect and the position along the line direction. The image is not compressed along the scanning direction. This increases the resolution along the scanning direction.

Trapezoidal formation is also prevented. Both line sensors can work with the entire Sensor line cover the same area. No information is lost. The overlap area of line sensors is called the overlap line in the following.

In the case of a line sensor, the opening angle of the lenses only needs to encompass the overlapping line. With the same sensor length, the overlap line is smaller than the diagonal of the overlap area of two area sensors, which, as described above, forms a hexagonal shape. As a result, the opening angle required for a stereo camera system with Scheimpflug condition and area sensors is greater than with a stereo camera system with Scheimpflug condition and line sensors. In the case of a stereo camera system with Scheimpflug condition and line sensors, this results in less distortion than in the case of a stereo camera system with Scheimpflug condition and area sensors.

By reducing the compression, the required rectification is also reduced. This reduces the computational effort and the image data is thus available more quickly.

In summary, it can be stated that by using stereo camera systems with Scheimpflug conditions, the basis can be selected to be large without distortions occurring as a result, as is the case with stereo camera systems without Scheimpflug conditions. A large base allows precise height resolution. The Scheimpflug condition allows small opening angles of the lenses with good image quality. However, the Scheimpflug condition causes unwanted distortions in both directions in area sensors due to the keystone effect. When using line sensors, the distortion only occurs in one direction, which means that the quality of the image is higher. In addition, the required opening angle of the lenses is smaller with line sensors than with area sensors, since the projected width of the line sensor is smaller than the projected diagonal of the area sensors. In addition, in contrast to area sensors, all image points of the sensor can be used.

Therefore, with a stereo camera system that satisfies the Scheimpflug condition and has line sensors, images can be generated with comparatively simple means that enable a high height resolution.

The Scheimpflug condition is preferably fulfilled along all beam paths. In principle, the beam paths can have different arrangements of the sensor areas and lenses. Because all beam paths meet the Scheimpflug condition, a larger base can be selected than with stereo camera systems in which only one beam path fulfills the Scheimpflug condition.

The beam paths are preferably arranged mirror-symmetrically. Adjustment of the camera system is simplified by the mirror-symmetrical arrangement. Asymmetries are easily noticed during the adjustment and can be corrected. The evaluation of the image data is also simplified because each sensor is equidistant from the object area. The recorded pixels are thus also offset by the same distance.

One or more of the beam paths are preferably directed with at least one mirror. This allows the beam paths to be folded by mirrors. The image of the measurement object is reflected one or more times by mirrors, whereby the length of the beam path is retained. However, the volume that is taken up by the arrangement of the optical lenses is in some cases considerably reduced. In particular, a large base for the stereo camera system can be selected and yet the two sensor halves remain close to one another. This means that more compact designs are possible than with structures without beam path folding.

When the beam path is folded, a distinction can be made between a fold on the object side, in which the mirrors that fold the beam path are arranged between the objective and the object, and a sensor-side fold in which the mirrors that fold the beam path are arranged between the lens and the sensor , and a double-sided folding in which the mirrors that fold the beam path are arranged both between the objective and the object and between the objective and the sensor. With object-side folding, the working distance, the distance between the lens and the object, can be reduced.

The beam paths are preferably projected onto a separate part of a single sensor. These parts each form a sensor area of one of the at least two camera modules. The sensor can be divided into several sensor areas, each of which is assigned a beam path. For example, in a stereo camera system with two beam paths, the sensor is divided into two areas. If the stereo camera system has three beam paths, the sensor is divided into three parts. This also increases the system stability. In addition, it enables a compact design with simple adjustment.

Alternatively, the camera can be designed in such a way that the beam paths are each projected onto a separate sensor and that a mirror is provided for each beam path and the beam path is configured in a Scheimpflug arrangement. This design allows a compact shape with larger sensor area than with a single sensor. This increases the resolution.

Another alternative is characterized in that the beam paths are each projected onto a separate sensor and that two mirrors are provided for each beam path and the beam path is configured in a Scheimpflug arrangement. This construction enables a compact shape.

The line-shaped sensor areas are preferably arranged next to one another on a line, each of which is represented by a line sensor. This line arrangement can simplify the adjustment, since the line-shaped sensor areas can be attached to a common carrier. For example, if the carrier is tilted, the sensor areas do not have to be realigned to one another.

In the case of stereoscopic methods, it is useful if the offset takes place in only one direction. This means that the sensors are arranged parallel or perpendicular to a sensor axis. This is supported by the line arrangement.

According to a further modification, the line-shaped sensor areas can be arranged parallel to one another. Here, too, the two beam paths are only offset in one direction.

It is possible for the line-shaped sensor areas to be pixel lines of an area sensor. An existing stereo line camera system can be easily retrofitted.

A common objective can be provided for all beam paths. In this way, particularly compact stereo line camera systems can be implemented. By reducing the number of lenses, costs can also be kept low.

The beam paths can have different distances between the object plane and the sensor plane. In this case, it can be expedient to compensate for the optical path due to the different distances by means of one or more glass elements which is / are arranged in the respective shorter beam paths.

The speed of the light inside the glass element is slowed down by the glass element. This means that the light needs a longer time to reach the sensor. This has the same effect as an optical path extension. Different distances can thus be compensated.

According to a further aspect of the invention, a stereo line camera system for measuring three-dimensional surface topologies comprises at least one color sensor, the color sensor having several pixels that are sensitive to different colors, and at least one hyperchromatic lens is provided for imaging the object on the sensor. The color sensor is arranged in relation to the lens in such a way that different focal planes are mapped onto the pixels of the same color.

With hyperchromatic objectives, the focus depends on the wavelength. Each color line of the sensor is thus assigned to a different focal plane.

This can be used to measure three-dimensional surface topologies. Each different height of the object area has a different focal plane. This means that each of the heights is sharply mapped on the sensor in a specific color range. The resolution of the height corresponds to that of the color resolution. This procedure is referred to below as hyperchromatic depth determination. It is particularly suitable for perceiving small differences in height.

The hyperchromatic depth determination is preferably used together with the stereoscopic method with the Scheimpflug condition. With the hyperchromatic depth determination fine height differences are recorded, whereas with the stereoscopic method height differences are detected over a larger area.

The color lines are preferably tilted in relation to the lens. The tilting takes place around an axis that is parallel to the individual color lines. The Scheimpflug condition is retained along a line. This tilting changes the distance between a color line and the lens, which also shifts the focal plane. In this way, the maximum height difference that can be measured by the hyperchromatic depth determination can be effectively varied.

The color channels of the color sensor are preferably evaluated separately. This allows the different focus levels to be calculated separately.

According to a further aspect of the invention, a stereo line camera for measuring three-dimensional surface topologies comprises at least two area sensors, the area sensors being designed in such a way that they have several parallel rows of pixels arranged next to one another, and at least two lenses for Imaging of the object on one of the area sensors. The stereo line camera is characterized by the fact that an object area with two independent beam paths is imaged simultaneously on one of the sensors so that a line-shaped object area is imaged on the pixel lines of an area sensor, with the beam paths that intersect the line-shaped object area in the line plane , the Scheimpflug condition is met, and that the area sensor is tilted about an axis in such a way that the area is not parallel to an object area to be scanned, so that the individual lines of the area sensor map different levels of the line-shaped object area.

As a result of the tilting, each line of image points has a different distance from the lens. The Scheimpflug condition is retained along a line. Each line thus has its own focus level. Each different height of the object area is thus sharply imaged on a different pixel line. The resolution of the height corresponds to the line resolution. This method is referred to below as inclinational depth determination (Latin: inclinatio = inclination).

The maximum difference in height that can be measured by the inclination depth determination can be varied by changing the tilt.

The resolution of the altitude can be improved by combining the inclination depth determination with the stereoscopic method with Scheimpflug condition. The height is now recorded using two different methods, which reduces a measurement error.

The sensors are preferably color sensors and the objective is a hyperchromatic objective. As a result, the individual focal planes that arise from the tilting of the area sensor can be subdivided again, which results in a finer resolution of the focal planes.

The image data are preferably rectified in-line. Here, the image data of the images of the stereo image pair are matched line by line, with both an offset and a different distortion being corrected by the two lenses. Through this rectification, which is also possible off-line, the position information is adapted to the respective distortion, so that the image data are displayed without distortion and can be further calculated. Since a further conversion step of the position information is not necessary, the speed of the calculations increases.

Line sensors with charge-coupled device (CCD) sensors have shift registers in which detected brightness values are stored. These shift registers are read out serially. In general, when reading out the shift register of the individual lines, uniform position information is assigned to the respective image points.

The pixel lines of sensors are preferably synchronized in such a way that they record line-synchronously and have a simultaneous image start. This significantly reduces the effort involved in rectification. In particular in combination with the rectification, a recorded image point of a camera module is assigned the same point on the object at the same time as a second recorded image point of the second camera module. This has the advantage that, during the stereo calculation, each line of a sensor area is automatically assigned the appropriate line of the second sensor area. This means that the stereo calculation is only necessary in one dimension.

Another aspect of the present invention relates to a surface topology acquisition device. The surface topology detection device has at least one stereo camera system, as explained above, in order to detect three-dimensional surface topologies of a measurement object. The surface topology detection device comprises a transport device for transporting the measurement object or the camera system, a synchronization device in order to synchronize the speed of the transport with the line-shaped scans of the three-dimensional surface topology, and an evaluation device in order to evaluate the measurements of the three-dimensional surface topology.

A transport device can for example be a conveyor belt.

The synchronization device outputs a signal to the stereo camera system and / or to the evaluation unit in order to synchronize the speed of the transport with the measurement of the three-dimensional surface topology. If the signal goes to the stereo camera system, the signal can trigger a measurement. If the signal reaches the evaluation unit, the speed of the transport device can be determined. With a known image generation frequency, it can be determined how far one image line is from the next.

The synchronization device can, for example, be an incremental encoder that is triggered by the transport device. Is the Transport device a conveyor belt, the wheel position can be determined by the synchronization device, for example, by a wheel that is coupled to the conveyor belt. A signal is then output for certain wheel positions. Alternatively, the synchronization device could have a camera unit which detects markings on the conveyor belt and uses them to output a signal.

The synchronization device can also be designed as a module of the evaluation unit. If the image generation frequency and the speed of the transport device are known, the distance between two recorded image lines can be determined by calculation.

The evaluation unit is a module on a computing unit and calculates the three-dimensional surface topology from the recorded image data.

• 1 ein erstes Ausführungsbeispiel einer Kamera mit zwei Objektiven und zwei Sensoren,
• 2a ein zweites Ausführungsbeispiel einer Kamera mit zwei Objektiven, vier Spiegeln und einem Sensor,
• 2b eine Abwandlung des zweiten Ausführungsbeispiels einer Kamera mit zwei Objektiven, vier Spiegeln und zwei Sensoren,
• 2c eine weitere Abwandlung des zweiten Ausführungsbeispiels einer Kamera mit zwei Farbfiltern, zwei Objektiven, vier Spiegeln und zwei Sensoren,
• 3 ein drittes Ausführungsbeispiel einer Kamera mit zwei Objektiven, zwei Spiegeln und zwei Sensoren,
• 4 ein hyperchromatisches Objektiv, wobei verschiedene Fokusebenen auf unterschiedliche Farb-Zeilen fallen,
• 5 ein viertes Ausführungsbeispiel einer Kamera mit drei Objektiven, vier Spiegeln und einem Farb-Zeilensensor,
• 6 eine Abwandlung des vierten Ausführungsbeispiels einer Kamera mit einem Objektiv, vier Spiegeln, einem Glaselement und einem Farb-Zeilensensor,
• 7 eine Oberflächentopologie-Erfassungseinrichtung, welche eine erfindungsgemäße Stereokamera verwendet, in einem Blockschaltbild,
• 8 eine schematische Anordnung von Sensoren und Objektiven zweier Kameramodule einer Stereokamera, welche nicht der Erfindung entspricht,
• 9 eine schematische Anordnung von Sensoren und Objektiven zweier Kameramodule einer Stereokamera, und
• 10 projizierte Sensorflächen auf einer Objektebene, und
• 11 eine Oberflächentopologie-Erfassungseinrichtung, welche eine erfindungsgemäße Stereokamera verwendet, in einem Blockschaltbild.

The invention is explained in more detail below using the examples shown in the drawings. The drawings show schematically in a side view:

• 1 a first embodiment of a camera with two lenses and two sensors,
• 2a a second embodiment of a camera with two lenses, four mirrors and a sensor,
• 2 B a modification of the second embodiment of a camera with two lenses, four mirrors and two sensors,
• 2c another modification of the second embodiment of a camera with two color filters, two lenses, four mirrors and two sensors,
• 3 a third embodiment of a camera with two lenses, two mirrors and two sensors,
• 4th a hyperchromatic lens, with different focal planes falling on different color lines,
• 5 a fourth embodiment of a camera with three lenses, four mirrors and a color line sensor,
• 6th a modification of the fourth embodiment of a camera with a lens, four mirrors, a glass element and a color line sensor,
• 7th a surface topology detection device which uses a stereo camera according to the invention, in a block diagram,
• 8th a schematic arrangement of sensors and lenses of two camera modules of a stereo camera, which does not correspond to the invention,
• 9 a schematic arrangement of sensors and lenses of two camera modules of a stereo camera, and
• 10 projected sensor areas on an object plane, and
• 11 a surface topology detection device which uses a stereo camera according to the invention, in a block diagram.

In a first embodiment, it comprises a stereo camera system according to the invention 25th two camera modules 1a and 1b ( 1 ).

Any camera module 1 is from one lens 2 and a line sensor 3 educated. The line sensors 3 each capture with a sensor area 13 one from the lens 2 the figure shown on this. The two camera modules 1 are arranged in such a way that they share a common object area from two different viewing directions 6th scan.

With every camera module 1 runs a camera axis 5 each through a center point of the sensor area 13 and a center point of the lens 2 . The two camera axes 5 thus run in the direction of view of the sensor areas 13 to the object area 6th .

The two camera axes 5 lie in an optical plane. The two line sensors are also located in this optical plane 3 , or their sensor areas 13 . This optical plane is congruent with the plane of the drawing 1 .

The base B. is the distance between the centers of the two sensor areas 13 . The camera axes 5 intersect in an object axis 10 and limit the base angle α .

Since the camera system has line sensors 3 are used, are only one sensor axis for further consideration 8th , a lens axis 9 and an object axis 10 relevant, which are the straight lines that lie in the optical plane and at the same time in the sensor plane defined above, the objective plane or the object plane.

The line sensor 3 takes an image of the object area 6th which through the lens 2 on the line sensor 3 is mapped on. The line sensor 3 is a CCD sensor and comprises a shift register and a one-dimensional array of photodetectors, each photodetector serving to receive a pixel. With every line sensor 3 runs the sensor axis 8th congruent with the sensor line.

The lenses 2 the camera modules 1 are like this to the respective line sensor 3 arranged that the object area 6th sharp in the plane of the line sensor 3 is mapped. The objective 2 can be formed from a single lens or from multiple lenses. An optical axis of the lens 2 is an axis of symmetry of the lens 2 which is perpendicular to the lens axis 9 runs in the optical plane.

The object area 6th lies on the object axis 10 and gets sharp on the sensor areas 13 pictured.

The sensor axis 8th , the lens axis 9 and the object axis 10 intersect at a common point, which, however, is not at infinity. The Scheimpflug condition is thus fulfilled.

A second exemplary embodiment ( 2a) explained, wherein the same elements as in the first embodiment are provided with the same reference numerals. The explanations given above apply to the same elements, unless otherwise stated below.

The second embodiment includes a stereo camera system according to the invention 25th again two camera modules 1a and 1b. Any camera module 1 includes a lens 2 , two mirrors 11 , 12th and a line sensor area 13 . The camera axis 5 is through the two mirrors 11 , 12th diverted. The line sensor area 13 of each camera module 1a and 1b lies in a line one behind the other and together forms a line sensor 3 .

The mirror 11 , 12th are so between lens 2 and line sensor area 13 arranged that the object area 6th on the line sensor area 13 falls.

The sensor axis 8th is at the mirror 11 mirrored. Sensor axis 8 '(not shown) is the mirrored sensor axis 8th . The sensor axis 8 'is in turn on the mirror 12th mirrored. Sensor axis 8 ″ (not shown) is the mirrored sensor axis 8 ', or the double mirrored sensor axis 8th .

One Base B. of the stereo camera system 25th is the distance between the center points of the projections of the two line sensors 3 where the projection of a line sensor is the place where the line sensor 3 would be the image of the lens area 6th not be mirrored.

The sensor axis 8 ", the lens axis 9 and the object axis 10 intersect at a common point, which, however, is not at infinity. The Scheimpflug condition is thus fulfilled.

The second embodiment differs from the first embodiment in that there are two mirrors 11 , 12th the foci of the object area per camera module 1a, 1b 10 deflect so that they are on a common axis, the respective sensor axis 8th , lie. This means that no two separate sensors are required, rather both beam paths can be directed to one line sensor 3 projected into two line sensor areas 13 is divided.

In this embodiment, the sensor axis is 8th parallel to the object axis 10 . But it is also conceivable that these two axes are not parallel. Both camera modules are then not constructed with mirror symmetry.

Alternatively, the stereo camera system 25th a distance between the two line sensor areas 13 on ( 2 B) . Any line sensor 3 now includes a line sensor area 13 . This exemplary embodiment has certain similarities to the hyperscope, which, however, is not subject to the Scheimpflug conditions.

Another possibility of this embodiment is that the two optical axes intersect, so that the first image of the object area 6th is projected onto the opposite line sensor 3b or line sensor area 13b. Without the Scheimpflug condition, this arrangement is known as a pseudoscope.

Another embodiment is that the stereo camera system 25th a structure according to the second embodiment ( 2c ), but the sensor areas are 13 not along the sensor axis 8th arranged in a line next to each other, but perpendicular to the sensor axis 8th arranged parallel to each other. These sensor areas 13 can, as explained in more detail below, each by the different color lines of a line sensor 3 be trained. Both beam paths are perpendicular to the line color sensor 3 so arranged offset from one another that the two different beam paths are slightly offset on the line color sensor 3 to meet. The two mirrors 11 are arranged next to each other. A color filter is then placed along each beam path 26th positioned. The image from one camera module 1a thus hits the line color sensor as a red image, for example 3 and the image of the other camera module 1b as, for example, a green image on the line color sensor 3 . This allows a line color sensor 3 simultaneously two, or even several, images of the object area 6th take up. The surface of the object should not be in the color of one of the color filters 26th otherwise the Image information of the surface can be filtered out. The beam path with the color filter of a first color thus hits the sensor line that detects the first color. The beam path with the color filter of the second color hits the sensor line that detects the second color.

Object surfaces that are white or have gray levels are preferred. White or grayscale have the same intensity values for each color in the color spectrum. The light gets through the respective color filter 26th , the integrated intensity of the light is reduced across all colors, regardless of the color of the color filter. Become the color channels of the line sensor 3 evaluated as gray levels, the resulting images of both beam paths differ in their gray level intensity only due to the different viewing angles to the object and not due to the different color of the beam path.
In this embodiment the filters are color 26th arranged in front of objective 2a and 2b ( 2c ). However, you can also choose between mirrors 12th and lens 2a, 2b or between mirrors 11 and 12th or between line color sensors 3 and mirror 11 be arranged.

Alternatively, instead of the color filter 26th Different polarizing filters can also be used along the beam path, provided the line sensor is used 3 accordingly differently polarized images of the object area 6th can record separately. Each beam path of the stereo camera system 25th thus has a different polarization when it hits the line sensor 3 meets. If the phase and the intensity of the images are recorded, this method can also be used in holography. The surface of the object must not change the polarization, otherwise the polarization filters filter out image information that is necessary to calculate the height information.

A third exemplary embodiment ( 3 ), the same elements as in the first and second exemplary embodiment being provided with the same reference numerals. The explanations given above apply to the same elements, unless otherwise stated below.

The third embodiment includes a stereo camera system 25th again two camera modules 1a and 1b. Any camera module 1 includes a lens 2 , a mirror 11 and a line sensor 3 . The camera axis 5 is through the mirror 11 diverted.

The mirror 11 is so between lens 2 and line sensor 3 arranged that the object area 6th on the line sensor 3 falls.

The sensor axis 8th is at the mirror 11 mirrored. Sensor axis 8 '(not shown) is the mirrored sensor axis 8th . The two sensor axes 8th of the two camera modules 1 are not necessarily parallel to one another in this exemplary embodiment.

One Base B. of the stereo camera system 25th is the distance between the center points of the projections of the two line sensors 3 where the projection of a line sensor is the place where the line sensor 3 would be the image of the lens area 6th not be mirrored.

The sensor axis 8 ', the lens axis 9 and the object axis 10 intersect at a common point, which, however, is not at infinity. The Scheimpflug condition is thus fulfilled.

In contrast to the second exemplary embodiment, the two line sensor areas are located 13 no longer in a line behind each other. Furthermore, through the mirror 11 the sensors 3 perpendicular or almost perpendicular to the object area 6th be arranged, whereby a very compact design can be selected.

A fourth exemplary embodiment ( 5 ), the same elements as in the first exemplary embodiment being provided with the same reference numerals. The explanations given above apply to the same elements, unless otherwise stated below.

In the fourth exemplary embodiment, a stereo camera system according to the invention comprises 25th three camera modules 1a, 1b and 1c. Instead of a simple line sensor, the stereo camera system comprises 25th a three-line sensor 15th . The three-line sensor 15th comprises three parallel sensor areas arranged side by side 13 . The sensor axes 8th are arranged perpendicular to the optical plane.

Each camera module 1a and 1c includes a lens 2 , two mirrors 11 , 12th and one of the three line sensor areas 13 . The camera module 1b comprises a lens 2 and one of the three line sensor areas 13 . The camera modules 1 feel a common object area from three different perspectives 6th from.

In the case of the outer camera modules 1a and 1c, the sensor axis 8th at the mirror 11 mirrored. Sensor axis 8 '(not shown) is the mirrored sensor axis 8th . The sensor axis 8 'is in turn on the mirror 12th mirrored. Sensor axis 8 ″ (not shown) is the mirrored sensor axis 8 ', or the double mirrored sensor axis 8th .

Will be three camera modules 1 with a stereo camera system 25th are used, there are three base lengths B. for every combination of two camera modules 1 possible. One Base B. is the distance between the center points of the projections of the two line sensors 3 where the projection of a line sensor is the place where the line sensor 3 would be the image of the lens area 6th not be mirrored.

The sensor axis 8th , or 8 ", the lens axis 9 and the object axis 10 intersect at a common point, which, however, is not at infinity. The Scheimpflug condition is thus fulfilled.

Alternatively, the stereo camera system comprises 25th instead of the three lenses 2a to 2c only a single lens 2 , which encloses all three camera modules 1a, 1b and 1c. To compensate for this, the middle camera module 1b has a glass element 19th on ( 6th ).

The fourth exemplary embodiment differs from the preceding exemplary embodiments in that the line sensor areas 13 are arranged parallel to each other and not in a line one behind the other.

In an alternative embodiment of the above-mentioned exemplary embodiments, the lens of each camera module is 1 a hyperchromatic lens 14th whose focus is particularly dependent on the respective wavelength (see 4th ). Furthermore, there is the three-line sensor 15th a three-line color sensor 15th which has a red 16a, green 16b and blue line 16c. Through the hyperchromatic lens 14th indicates the respective red, green and blue lines (16a, 16b, 16c) of the three-line color sensor 15th each have a different focal plane 17a, 17b and 17c. As a result, each color line 16a, 16b and 16c measures a different height measuring area 18a, 18b and 18c. The resolution of the height corresponds to that of the color resolution.

Preferably the color lines 16 in relation to the lens 14th tilted. The tilting takes place around an axis that is parallel to the individual color lines 16 is. Along a line of color 16 the Scheimpflug condition is retained. This tilting changes the spacing of a color line 16 to the lens 14th which also makes the level of focus 17th shifts.

Another possibility of the invention is that the camera modules 1 instead of a line sensor 3 have an area sensor. The area sensor is a two-dimensional array of photodetectors, each photodetector corresponding to one pixel. In this exemplary embodiment, the area sensor is used as a series of parallel line sensors 3 considered.

The area sensor is tilted about a tilt axis. The tilt axis is parallel to the individual line sensors 3 . The individual lines and their assigned sensor axes 8th , or 8 'or 8 ", are arranged in such a way that they represent the objective axis at one point 9 and the object axis 10 cut and thus meet the Scheimpflug condition.

The amount of the angle between the normal of the area sensor and the optical axis is less than 90 °. Each line of the area sensor has a different height measuring range 18th on. The adjacent height measuring ranges lead to an extension of the height range. The same camera structure can therefore be used for objects with different heights. Each height of the object is assigned at least one corresponding line of the area sensor with the associated height measurement area.

In an alternative embodiment of the above-mentioned embodiments, instead of a straight mirror 11 Such curved mirrors are used, so that the images of the measurement object 1 distortion-free on the line sensor 3 hit.

The above-mentioned exemplary embodiments each include two or three camera modules 1 . In principle, any number, but at least two camera modules can be provided for the stereoscopic determination of surface topologies. The higher the number of camera modules, the smaller the statistical error. However, the costs also increase. The number of base lengths results from the triangle number. For example, there are two camera modules 1 only one base possible with three camera modules 1 there are three, with four there are six possibilities and with five there are ten different base lengths.

The following is a surface topology acquisition device 20th explained ( 7th ).

The surface topology detection device comprises a stereo camera system 25th , as in one of the above embodiments according to one of the 1 to 6th for the measurement of three-dimensional surface topologies of a measurement object 4th , a transport facility 21st for transporting the measurement object 4th , a synchronization device 22nd to determine the speed of transport with the measurement of the three-dimensional To synchronize surface topology, and an evaluation device 23 to evaluate the measurements of the three-dimensional surface topology.

The transport device 21st is in this embodiment a conveyor belt.

The synchronization facility 22nd is an incremental encoder in this embodiment 22nd . The incremental encoder is on the conveyor belt 21st coupled so that it rotates when the conveyor belt moves. A clock signal is output when rotating through a predetermined angle of rotation. The conveyor belt thus covers a predetermined distance between two successive clock signals. These clock signals are sent to an evaluation device 23 transmitted. The evaluation device 23 is the same with the chamber modules 1 connected to the sensors arranged in the chamber modules 3 to obtain captured image signals. The evaluation device 23 is designed in such a way that it indicates the point in time at which the respective camera modules reach the measurement object 4th feel, control. This allows the evaluation device 23 according to the incremental encoder 22nd received clock signal the scanning of the measurement object 4th with the movement of the conveyor belt or the measuring object 4th synchronize. This synchronization is preferably designed in such a way that the measurement object is in the direction of movement 7th is scanned with the same spatial distance. Several successive one-dimensional scans can form a two-dimensional image in the evaluation device 23 be put together. Thus, three-dimensional information is obtained from a surface topology.

The captured images can be corrected. On the one hand, a displacement error can be corrected that arises from the object being measured 7th is moved relative to the camera and the individual pixels of the line camera are successively switched sensitive. As a result, at high speeds, the image has a certain inclination to the direction of movement 7th on.

Furthermore, the images can be rectified. The rectification eliminates geometric distortions in the image data. These can arise from the Scheimpflug arrangement, among other things.

The correction of the rectification can also be carried out directly in-line during the image registration. The data is corrected immediately after an image line has been recorded. After digitizing the sensor signal, the image signals are preprocessed on the camera in an FPGA (Field Programmable Gate Array). If the parameters of the distortion of the two camera modules are known, e.g. from a calibration of the system, the same correction model can be applied in the FPGA as conventionally on a PC. A further correction in the arithmetic unit is therefore not necessary, which increases the processing speed.

The images corrected in this way are used for a depth reconstruction of the recorded surface of the measurement object 4th used. A typical method for depth reconstruction is stereo triangulation. Each point of the first image is assigned a second point in the second image. The distance between the two points depends on the actual spatial depth of the point, more precisely the distance from the sensor 3 to the object area 6th and the base B. of the stereo camera system 25th . Depth information is assigned to each point of the first image.

Alternatively, blocks, i.e. a group of pixels, e.g. a 3x3 matrix, can be assigned to each other. This process is called block matching.

For each recorded image line, an image line containing depth information is calculated. Are several lines of a measurement object 4th recorded, the lines with the depth information can be combined in order to generate a three-dimensional surface topography of the measurement object. The distances between the individual lines can be determined by information from the synchronization device 22nd be calculated.

Instead of multiple line sensors 3 Line sensor areas 13 is used, the image data of all camera modules are used 1 on the same line sensor 3 generated. The resulting digital double image is used in the evaluation device 23 divided. The resulting two images are corrected as described above and used for the depth construction.

Becomes a three-line color sensor 15th used to record the measurement object, a separate image is generated for each color. For each color, as described above, the independent images are corrected separately and used for the depth construction. Since each color has a different height measuring range 18a, 18b and 18c when using a hyperchromatic objective, an additional estimate of the depth profile of the measuring object surface can be calculated via image definition will. Each point of an image becomes a point in the other two color domain images, but of the same camera module 1 assigned. The position of the point does not differ in the three images. The image sharpness of the point depends on the depth of the real point on the measurement object 4th and color different. If a point is deep, so it is shown in focus in the blue image, in the green image at medium depth and in the red image at low depth. A sharpness detection can be carried out, for example, by comparing the contrast of the images in a specific pixel region. If a point is sharper, the contrast is higher.

In combination with the stereoscopic calculation, a fine determination of the surface topology is possible.
As an alternative to the individual calculation of the individual color images, a gray-scale image can also be calculated from a composite RGB image, which is then analyzed.

If a surface sensor is used to record the measurement object, an independent image is generated for each line. As described above, the independent images are corrected separately for each line and used for the depth construction. The sharpness of a real point on the measuring object is then shown in each independent image 4th certainly. In this way the line can be determined which recorded the point most sharply. Since the position of the line and the focal length are known, the distance to the point can be determined. If all points are connected to one another, a height profile of the measuring object is created 4th .

Besides the three line sensors mentioned above 15th Other combinations are also conceivable, such as a sensor with three lines of RGB sensor points, a sensor with three lines of sensor points that also detects gray levels, a sensor with three lines of sensor points that only detects gray levels, etc.

The sensors mentioned above can, as mentioned above, be CCD sensors, but also other types of sensors, e.g. Complementary metal-oxide-semiconductor (CMOS) sensors are conceivable.

List of reference symbols

1

Camera module

2

lens

3

sensor

4th

Measurement object

5

Camera axis

6th

Object area

7th

Shift direction

8th

Sensor axis

9

Lens axis

10

Object axis

11

Mirror 1

12

Mirror 2

13

Line sensor area

14th

Hyperchromatic lens

15th

Three-line sensor

16

Color line

17th

Focus plane

18th

Height measuring range

19th

Glass element

20th

Surface topology acquisition device

21st

Transport device

22nd

Synchronization facility

23

Evaluation device

25th

Stereo camera system

26th

Color filter

α

Base angle

β

Scheimpflug angle

γ

Opening angle of the lens

B.

Base

QUOTES INCLUDED IN THE DESCRIPTION

This list of the documents listed by the applicant was generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Patent literature cited

• DE 102013103897 A1 [0037]

Non-patent literature cited

• "Scheimpflug stereocamera for particle image velocimetry in liquid flows", Ajay K. Prasad and Kirk Jensen, APPLIED OPTICS 34, 30 (1995) [0017]

Claims (18)

Stereo camera system for measuring three-dimensional surface topologies of an object with at least two line-shaped sensor areas and at least one lens for imaging the object onto the sensor areas, the sensor areas and the lens (s) being arranged in such a way that an object area with two independent beam paths is simultaneously on one of the sensor areas is imaged, characterized in that the Scheimpflug condition is met along at least one of the beam paths . Stereo camera system Claim 1 , characterized in that the Scheimpflug condition is met along all beam paths . Stereo camera system Claim 1 or 2 , characterized in that one or more of the beam paths are directed with at least one mirror. Stereo camera system according to one of the Claims 1 to 3 , characterized in that the beam paths are each projected onto a separate part of a single sensor, which is divided into the number of beam paths. Stereo camera system according to one of the Claims 1 to 3 , characterized in that the beam paths are each projected onto a separate sensor and that at least one mirror is provided for each beam path and the beam path for each beam path is configured in a Scheimpflug arrangement. Stereo camera system according to one of the Claims 1 to 3 , characterized in that the beam paths are each projected onto a separate sensor and that two mirrors are provided for each beam path and the beam path for each beam path is configured in a Scheimpflug arrangement. Stereo camera system according to one of the Claims 1 to 6th , characterized in that the line-shaped sensor areas are arranged side by side on a line, each of which is represented by a line sensor. Stereo camera system according to one of the Claims 1 to 5 , characterized in that the line-shaped sensor areas are arranged parallel to one another. Stereo camera system Claim 8 , characterized in that the line-shaped sensor areas are pixel lines of an area sensor. Stereo camera system according to one of the Claims 1 to 9 , characterized in that a common lens is arranged in all beam paths . Stereo camera system according to one of the Claims 1 to 10 , characterized in that a glass element is arranged in at least one of the beam paths . Stereo camera system for measuring three-dimensional surface topologies of an object, in particular according to one of the Claims 1 to 11 with - at least one color sensor, the color sensor having several pixels that are sensitive to different colors, - at least one hyperchromatic lens for imaging the object on the sensor, characterized in that the color sensor is arranged in relation to the lens in such a way that different focal planes each are mapped onto the pixels of the same color. Stereo camera system Claim 12 , characterized in that color channels of the color sensor are evaluated separately. Stereo camera system for measuring three-dimensional surface topologies of an object with - at least two area sensors, the area sensors being designed in such a way that they have several parallel rows of pixels arranged next to one another, - at least two lenses for imaging the object on one of the area sensors, characterized in that an object area with two independent beam paths is imaged simultaneously on one of the sensors, so that a line-shaped object area is imaged on the pixel lines of an area sensor, with the Scheimpflug condition in the line plane of the beam paths that intersect the line-shaped object area, in particular according to one of the Claims 1 to 11 is fulfilled, and that the area sensor is tilted about an axis in such a way that the area is not parallel to an object area to be scanned, so that the individual lines of the area sensor map different planes of the line-shaped object area. Stereo camera system Claim 14 , characterized in that the sensors are color sensors and the lens is a hyperchromatic lens Claim 11 or 12th are. Stereo camera system according to one of the Claims 1 to 15th , characterized in that the recorded data are registered in-line in order to correct the rectification. Stereo camera system according to one of the Claims 1 to 16 , characterized in that the pixel lines of sensors are synchronized in such a way that they record line-synchronously and have a simultaneous image start. Surface topology detection device with at least - one stereo camera system according to one of the Claims 1 to 17th for measuring three-dimensional surface topologies of a measurement object, - a transport device for transporting the measurement object, - a synchronization device to synchronize the speed of the transport with the measurements of the three-dimensional surface topology, and - an evaluation device to evaluate the measurements of the three-dimensional surface topology.

Images