DE102009015500B4 - Method and device for determining a change in position of an object by means of a stereo camera - Google Patents

Abstract

A method for determining a change in position of an object by means of a stereo camera, which is connected to the object defined and provides a stereo image, which consists of at least a first stereo partial image (STB1) and a second stereo partial image (STB2),
A stereo camera plane (SKEn) at a first time (n) via a shift of the first stereo partial image (STB1 n) to the second stereo partial image (STB2n) and a stereo camera plane (SKEn + 1) at a second time (n + 1) via a displacement of the first stereo subpicture (STB1n + 1) to the second stereo subpicture (STB2n + 1) are determined so that an area (F _{n, shk1} ) of contours _lying in the stereo camera plane (SKEn) at the first time point (s) and Area (F _{n, shk2} ) of contours that are in the stereo camera plane (SKEn) at the second time point (n + 1) are maximum and the difference from the distance of the stereo camera level (SKEn) at the first time (n) to a stereo camera level and the distance of the stereo camera level (SKEn + 1) ...

Description

The The invention relates to a method and a device for determination a change of position an object using a stereo camera.

The Problem of the orientation of moving objects arises many fields of application, eg. B. in the automotive industry and in the area autonomous Robot. An established standard procedure is the orientation using the Global Positioning System (GPS). There is one, however Series of cases, in which a position determination using the GPS is not applied can be. So z. B. a position determination using GPS in areas where there is insufficient connection to adequate Many satellites occur in confined spaces with a shielding effect through the walls, under water, in mines, in tunnels and under bridges not possible.

A Alternative for orientation is the stereo image acquisition and Stereo image processing with the help of a stereo camera, which with the is connected to a moving object. Previous procedures are taking effect back to the approach, the spatial Distance of an object to the stereo camera to different To determine times. From the relative movement of the individual detected objects is then estimated the movement of the camera. Consequently These previous methods are based on the successive determination from single stereo points or epipolar points in stereo images at different times. Spatial coordinates of stereo points can due to the "X. Armangue, H. Araújo, S. Salvi, A review of egomotion by means of differential epipolar Geometry applied to the movement of a mobile robot, Pattern Recognition 36 (2003), 2927-2944 " Calculated method, which determines the spatial context of a for example, right stereo partial image and a left stereo partial image the stereo camera by means of a Euclidean transformation produces.

One The main problem of the previous methods is the necessary recognition the stereo points in the next Time step. Another problem is the assignment of objects, which are over extend different depth planes relative to the camera plane. Out they can have a different distance of the object and one Camera movement are difficult to distinguish. The use of individual Stereo points also has the disadvantage of being easy to one wrong assignment of corresponding stereo points can occur. Thus, the calculation of the change in position of the camera from these individual stereo points inaccurate, if not enough stereo points were used to determine stereo point correspondences. The calculation of the change in position the camera over However, many stereo points has the disadvantage that the calculations are very time consuming and thus no real-time capability of orientation allow more. When considering many stereo points also raises the problem that stereo points unconsidered have to stay which is not a counterpart in the next Have time step. Likewise have to Stereo points in the next time step were found and the no stereo to the last time step disregard stay. The necessary arithmetic operations cause additional Computation time.

Farther have to the stereo points in temporally successive stereo images be determined extremely accurately, as accurate as possible allow the object. A summation of stereo points to minimize the error is also problematic because the errors are not the condition of a satisfy additive noise. Another problem of the methods based on stereo point correspondences is also that you are not limited to one level and so determines stereo points whose neighborhood is in another plane can lie.

From "Horn, J .; Russ, J .: Localization of a mobile robot by matching 3D-laser-range-images and predicted sensor images Intelligent Vehicles; 94 symposium, Proceedings of the Volume, Issue, 24-26 Oct. 1994 Page (s): 345-350 "is a A method is known which detects the position of moving robots over the Extraction of the vertical surface determined from the processing of a 3D laser range image. in this connection is an environment model requirement. This requirement is normally not fulfilled, when it comes to explorations with a robot, such. B. underwater exploration and mapping.

In "Mark Fiala, A. Basu, Robot navigation using panoramic tracking, pattern recognition 37 (2004) Page (s): 2195-2215 "becomes one Procedure for the Robot navigation described on a panoramic track based. This procedure requires that a particular object or panorama in the stereo images can be tracked. This condition for example, is not satisfied if a tracked object falls out of sight or if a tracked object is through another object in the next image is covered.

An alternative to determining the position change by finding stereo point correspondences in temporally successive images is a stereo image evaluation with regard to the lines or contours existing in the stereo images, which are generated and compared in temporally successive stereo images. This results difficult conditions for image processing, especially if there are few contours in the images, if similar contours are present in the images, or if contours occur at different distances. Furthermore, problems arise when rapid changes in position of the camera occur and contour-generating objects are hidden in temporally successive images.

The Mallet, A; Lacroix, S .; Gallo, L .: Position estimation in outdoor environments using pixel tracking and stereovision; IEEE International Conference to Robotics and Automation, 2000, Proceedings; ICRA '00; Vol. 4, page (s): 3519-3524 " a method of exclusion inappropriate image features.

Also the "Franke, U .; Joos, A .: Real-time stereo vision for urban traffic scence understanding; IEEE, 2000; Intelligent Vehicles Symposium; IV 2000, Proceedings often he IEEE; page (s): 273-278 " a method of exclusion inappropriate image features.

The Franke, Uwe, et. al .: 6D-vision: Fusion of stereo and motion for robust environment perception; Springer. 2005; Lecture notes in computer science; Pattern recognition, Vol. 3663/2005, pages 216-223 ", that points whose image depth is known from the stereo system, over the time recordings are tracked.

Of the The invention is therefore based on the technical problem of a method and a device for determining a change in position of an object by means of a stereo camera, which has a robust, fast and good quality contour recognition a picture change the contours in chronological successive stereo images, being from the picture change the detected contours a change in position the stereo camera and thus determines a change in position of the object can be.

The solution the problem arises from the objects with the features of claims 1 and 11. Further advantageous embodiments of the invention will become apparent from the dependent claims.

• – eine Stereokameraebene (SKEn) zu einem ersten Zeitpunkt (n) über eine Verschiebung des ersten Stereoteilbilds (STB1n) auf das zweite Stereoteilbild (STB2n) und eine Stereokameraebene (SKEn + 1) zu einem zweiten Zeitpunkt (n + 1) über eine Verschiebung des ersten Stereoteilbilds (STB1n + 1) auf das zweite Stereoteilbild (STB2n + 1) derart bestimmt werden, dass eine Fläche (F_n,shk1) von Konturen, die in der Stereokameraebene (SKEn) zum ersten Zeitpunkt (n) liegen, und eine Fläche (F_n,5hk2) von Konturen, die in der Stereokameraebene (SKEn) zum zweiten Zeitpunkt (n + 1) liegen, maximal sind und die Differenz von dem Abstand der Stereokameraebene (SKEn) zum ersten Zeitpunkt (n) zu einer Stereokamerabildebene und dem Abstand der Stereokameraebene (SKEn + 1) zum zweiten Zeitpunkt (n + 1) zur Stereokamerabildebene minimal ist,
• – wobei eine Abbildungsänderung als eine Transformation (T) mindestens des ersten, zu einem ersten Zeitpunkt (n), aufgenommenen Stereoteilbilds (STB1n) auf ein erstes, zu einem zweiten Zeitpunkt (n + 1), aufgenommenen Stereoteilbilds (STB1n + 1) bestimmt wird, bei der eine maximale Übereinstimmung der Konturen in den zu dem ersten und dem zweiten Zeitpunkt (n, n + 1) aufgenommenen, ersten Stereoteilbildern (STB1n, STB1n + 1) auftritt,
• – wobei die Konturen zu dem ersten Zeitpunkt (n) in der Stereokameraebene (SKEn) und die Konturen zu dem zweiten Zeitpunkt (n + 1) in der Stereokameraebene (SKEn + 1) liegen,
• – wobei eine Lageänderung der Stereokamera über einen Zusammenhang zwischen Lageänderung der Stereokamera und einer Abbildungsänderung von Konturen ermittelt wird.

Here, the determination of a change in position of an object by means of a stereo camera, which is connected to the object defined and provides a stereo image, which consists of at least a first stereo partial image and a second stereo partial image, wherein

• A stereo camera plane (SKEn) at a first time (n) via a shift of the first stereo partial image (STB1n) to the second stereo partial image (STB2n) and a stereo camera plane (SKEn + 1) at a second time (n + 1) via a displacement of the first stereo partial image (STB1n + 1) to the second stereo partial image (STB2n + 1) are determined so that a surface (F _{n, shk1} ) of contours _lying in the stereo camera plane (SKEn) at the first time point (n) and a surface (F _{n, 5hk2} ) of contours _lying in the stereo camera plane (SKEn) at the second time point (n + 1) are maximum and the difference from the distance of the stereo camera plane (SKEn) at the first time point (s) to a stereo camera plane and Distance of the stereo camera plane (SKEn + 1) to the second time point (n + 1) to the stereo camera plane is minimal,
• - wherein an image change as a transformation (T) of at least the first, at a first time (n), recorded stereo partial image (STB1n) to a first, at a second time (n + 1), recorded stereo partial image (STB1n + 1) is determined in which maximum coincidence of the contours occurs in the first stereo sub-images (STB1n, STB1n + 1) recorded at the first and second times (n, n + 1),
• The contours being at the stereo camera plane (SKEn) at the first time point (s) and at the stereo camera plane (SKEn + 1) at the second time point (n + 1),
• - Wherein a change in position of the stereo camera is determined by a relationship between the change in position of the stereo camera and a change in the image of contours.

As a result, the relationship between the image change of contours and the change in position of the stereo camera is used in an advantageous manner. Furthermore, the transformation between two stereo partial images recorded at different times can be uniquely determined by means of the evaluation of contours, in particular of extended contours. Another advantage of the method is that occlusion of contours does not lead to misinterpretation, since hidden contours are not used to determine the transformation. By selecting a stereo camera plane where maximum coincidence of the contours occurs in the first stereo image and the second stereo image, the method advantageously ensures that the image changes are tracked by contours in a stereo camera plane having the most contours. If very many contours are present in a stereo camera plane at a first point in time, then it can be assumed with great certainty that at a second point in time many contours will also be present in the corresponding stereo camera plane. Thus, it is advantageously ensured that the maximum number of existing contours is used to transform between two stereo partial images to calculate. The determination of the change in position of the stereo camera on the basis of the image change of contours further avoids the search for stereo point correspondences and thus reduces the time required for the calculation of the change in position of the stereo camera.

In a preferred embodiment the contours are displayed in a contour image, which over the Difference of a first binary image and a second, opposite the first binary image shifted, binary image is generated, wherein the first and second binary image via a comparison of a first and a second gray scale image having predetermined thresholds and / or selection of bit planes of the gray value images and / or the division the grayscale images by large Values and subsequent Forming the integer values of the quotients can be calculated. This ensures advantageous that for the contour generation Contours of objects and / or linear formations are used, the stand out due to strong brightness or texture differences in the image. It also ensures that even areas where small fluctuations the texture and / or brightness, used for contour generation can be. in this connection the contour generation takes place from brightness or texture differences, which, if they are identically determined, follow one another in chronological order Do not change the image significantly.

In a further preferred embodiment the contours are displayed in a contour image, which over the Calculation of edges is generated in a gray scale image, where the calculation of edges over the determination of gray value gradient amounts of the gray value image and a comparison the gray value gradient amounts with a predetermined gray level gradient amount and / or the application of edge detection filters. This will be advantageous Ensures that the edges of objects used for contour generation become. The determination of the gray value gradient amounts can thereby from a direction-dependent Gray gradient, for example, from the gradient in positive x-direction of the picture. Alternatively, it is also possible to have one directional To determine gray value gradient amount, for example, by several directional Grauwertgradientenbeträge be added.

In a further preferred embodiment the contours are displayed in a contour image, which has a Absolute amount of a difference of one by one or more smoothing functions smoothed Gray scale image and the unsmoothed one Gray value image and the comparison of the absolute value of the difference with a predetermined smoothing threshold is produced. As a result, the effect is exploited advantageously, that edges in the difference compared to surfaces with uniform Brightness stronger be highlighted because they are smeared by the smoothing. Farther is advantageously ensured that contours are selected can, which are brighter than the surroundings or darker than the surroundings.

In a further preferred embodiment the contours are displayed in a contour image, which over the Difference between a first gray scale image and a second gray scale image, the via contour-enhancing image processing methods derived from an unprocessed gray value image generated is, with the second opposite the first gray value image in horizontal or vertical image direction is moved. This ensures in an advantageous manner, that particular vertical or horizontal lines highlighted as contours can be. this makes possible a simplified determination of the image change.

In a further preferred embodiment a resulting contour image is generated by the over different Method generated algebraically and / or logically linked contour images and be compared with a predetermined contour selection value. This can advantageously the amount of selected contours to be controlled.

In a further preferred embodiment becomes a first contour image from the first stereo image and a second contour image generated from the second stereo partial image, wherein an area of contours from a surface comparison of contour surfaces of the first contour image and of contour surfaces of one or more Pixel in the x and / or y direction shifted second contour image is determined, wherein a logical AND of the first and second Contour image is analyzed. This will be advantageous made sure the match by the number of pixels in the logical AND result binary image which are occupied by a logical one, is given.

In a further advantageous embodiment, the transformation of at least the first, at a first time recorded stereo partial image on a first, at a second time, recorded stereo partial image from at least two contour operation of the group horizontal contour shift, vertical contour shift, contour magnification, contour reduction and Konturrotation about the axis determined perpendicular to the image plane, wherein the quantitative determination of the optimal operation parameters that result in a maximum coincidence of the contours, via an evaluation of all combinations of operating parameters of the contour operations. About the Evaluation of all combinations of operating parameters ensures that the best global transformation and, hence, a precise change in position is determined.

In an alternative embodiment is the transformation of at least the first, at a first time, recorded stereo partial image at a first, at a second time, recorded stereo partial image from at least one contour operation the group horizontal contour shift, vertical contour shift, Contour augmentation, Contour reduction and contour rotation around the axis perpendicular to the Image plane determined, with the quantitative determination of the optimal Operation parameters, which is a maximum match of contours result, by a sequential determination of the optimal operation parameters for each a contouring operation takes place. By decoupling the search from optimal operation parameters becomes an accelerated determination the transformation allows.

In a further preferred embodiment are the operation parameters for a transformation of at least the first, at a first time, recorded stereo partial image at a first, at a second time, recorded stereo partial image and / or the displacement of the first stereo partial image on the second stereo part, where a maximum match the contours generated from the stereo part images occurs in a surgical and / or shift parameter interval, wherein the operation and / or shift parameter interval go through sequentially and the search is aborted once a local maximum of the match is found. This ensures in an advantageous manner, that for determining a transformation and / or displacement, at the maximum match of generated contours, not the complete search space of Operational parameters must be analyzed and thus an acceleration the determination of a transformation and / or a shift is reached.

• – eine Stereokameraebene (SKEn) zu einem ersten Zeitpunkt (n) über eine Verschiebung des ersten Stereoteilbilds (STB1n) auf das zweite Stereoteilbild (STB2n) und eine Stereokameraebene (SKEn + 1) zu einem zweiten Zeitpunkt (n + 1) über eine Verschiebung des ersten Stereoteilbilds (STB1n + 1) auf das zweite Stereoteilbild (STB2n + 1) derart bestimmt werden, dass eine Fläche (F_n,shk1) von Konturen, die in der Stereokameraebene (SKEn) zum ersten Zeitpunkt (n) liegen, und eine Fläche (F_n,shk2) von Konturen, die in der Stereokameraebene (SKEn) zum zweiten Zeitpunkt (n + 1) liegen, maximal sind und die Differenz von dem Abstand der Stereokameraebene (SKEn) zum ersten Zeitpunkt (n) zu einer Stereokamerabildebene und dem Abstand der Stereokameraebene (SKEn + 1) zum zweiten Zeitpunkt (n + 1) zur Stereokamerabildebene minimal ist,
• – wobei eine Abbildungsänderung als eine Transformation (T) mindestens des ersten, zu einem ersten Zeitpunkt (n), aufgenommenen Stereoteilbilds (STB1n)
• – auf ein erstes, zu einem zweiten Zeitpunkt (n + 1), aufgenommenen Stereoteilbilds (STB1n + 1) bestimmt wird, bei der eine maximale Übereinstimmung der Konturen in den zu dem ersten und dem zweiten Zeitpunkt (n, n + 1) aufgenommenen, ersten Stereoteilbildern (STB1n, STB1n + 1) auftritt,
• – wobei die Konturen zu dem ersten Zeitpunkt (n) in der Stereokameraebene (SKEn) und die Konturen zu dem zweiten Zeitpunkt (n + 1) in der Stereokameraebene (SKEn + 1) liegen,
• – wobei die Bildverarbeitungseinheit eine Lageänderung der Stereokamera über einen Zusammenhang zwischen Lageänderung der Stereokamera und einer Abbildungsänderung von Konturen ermittelt.

In order to carry out the method for determining a change in position of an object, a device comprises at least one stereo camera, which is connected to the object in a defined manner and supplies a stereo image which comprises at least a first stereo partial image and a second stereo partial image, and an image processing unit

• A stereo camera plane (SKEn) at a first time (n) via a shift of the first stereo partial image (STB1n) to the second stereo partial image (STB2n) and a stereo camera plane (SKEn + 1) at a second time (n + 1) via a displacement of the first stereo partial image (STB1n + 1) to the second stereo partial image (STB2n + 1) are determined so that a surface (F _{n, shk1} ) of contours _lying in the stereo camera plane (SKEn) at the first time point (n) and a surface (F _{n, shk2} ) of contours _lying in the stereo camera plane (SKEn) at the second time point (n + 1) are maximum and the difference from the distance of the stereo camera plane (SKEn) at the first time point (s) to a stereo camera plane and Distance of the stereo camera plane (SKEn + 1) to the second time point (n + 1) to the stereo camera plane is minimal,
• Wherein an image change is performed as a transformation (T) of at least the first stereo partial image (STB1n) taken at a first time (n)
• Is determined to a first, at a second time (n + 1), recorded stereo partial image (STB1n + 1), in which a maximum coincidence of the contours in the at the first and the second time (n, n + 1) recorded, first stereo part images (STB1n, STB1n + 1),
• The contours being at the stereo camera plane (SKEn) at the first time point (s) and at the stereo camera plane (SKEn + 1) at the second time point (n + 1),
• - The image processing unit determines a change in position of the stereo camera via a relationship between change in position of the stereo camera and a mapping change of contours.

Regarding further advantageous embodiment of the device for determination a change of position of an object, reference is made to the preceding statements on the method claims.

The Invention will be described below with reference to a preferred embodiment explained in more detail. It demonstrate:

1 a flowchart of the method for determining a change in position of an object by means of a stereo camera,

2 a schematic flow diagram for generating contour images,

3 a schematic flow diagram of the generation of a resulting contour image,

4 a coincidence of contours of all planes in case of pure contour rotation,

5 a match of contours of all planes with pure contour shift,

6 a schematic flow chart for determining the contour shift,

7 a schematic flowchart for determining the contour enlargement or reduction,

8th a schematic representation of the determination of the stereo camera translation from a change in the image of a point,

9 a schematic flowchart for determining the change in position of the stereo camera with coupled stereo camera translation and rotation,

10 a schematic representation of the mapping of contours in different planes in stereo camera rotation and

11 a schematic flowchart for determining the Konturrotation.

In 1 a flowchart of the method for determining a change in position of an object by means of a stereo camera is shown. Here, the change in position of the camera is not determined from as many corresponding stereo points, but it is derived via a mapping change of contours in the stereo images, the change in position of the stereo camera. For this purpose, a similarity relationship between the image change of contours and the position change of the stereo camera is evaluated.

A not shown stereo camera defined with the not shown Object is connected, it provides at time n a first stereo part STB1n and a second stereo sub-picture STB2n. At time n + 1 the stereo camera, not shown, delivers the first stereo image STB1n + 1 and the second stereo subpicture STB2n + 1. Preferably the time interval Δn the times chosen very small, for example, Δn = 1/30 Hz ≈ 33.33 ms, so no big one temporal change of position of the object not shown between the recording times n and n + 1 can take place.

The defined compound is preferably solid and rigid, wherein the change of position the camera equal to the change in position of the object. In another embodiment, the compound firmly, allows but a rotation of the stereo camera relative to the object, the Rotation is detectable, so that a transformation of relative change of position the stereo camera can be recalculated on the object is.

in the includes the following specific embodiment the stereo camera a right camera and a left camera. The to Time n and n + 1 delivered first stereo subpictures STB1n, STB1n + 1 images indicate images of the right camera (right stereo images) and the second stereo part images supplied at time n and n + 1 STB2n, STB2n + 1 images from the left camera (left stereo parts).

As in 1 illustrated, the determination of a change in position of an object by means of a stereo camera can be divided into several steps. In a first step, a right contour image BK1n is generated at the time n from the right stereo partial image STB1n. Analogously, a generation of a left contour image BK2n and the generation of right and left contour images BK1n + 1, BK2n + 1 at the time n + 1 From the contour image pairs BK1n and BK2n at time n then surfaces F _{n, shk1} of contours, which in a stereo camera SKEn lie, determined.

Analogously, the determination of areas F _{n, shk2} of contours takes place at time n + 1. Then, two optimum stereo camera planes SKEn * and SKEn + 1 * are selected at time n and n + 1, for which areas F _{n, shk1} and F _{n + 1, shk2} for an optimal running index k1 and k2 are maximum, and at the same time the maximum of the distances between the optimum stereo camera planes SKEn *, SKEn + 1 * from the stereo camera plane in which the contour surfaces at time n and n + 1 are maximum is low. After this, the optimum contour images BKn *, BKn + 1 *, which contain only contours from the stereo camera planes SKEn *, SKEn + 1 * thus determined, are considered. A transformation T is then determined which produces a maximum match of the optimum contour images BKn * and BKn + 1 *, wherein, for example, the optimal contour image BKn + 1 * is transformed. The transformation T is composed of different contour operations that are applied to the optimal contour image BKn + 1 *. These include in particular the operations of contour shifting, contour rotation and contour enlargement or reduction .DELTA.V. The degree of agreement after a transformation T is determined on the basis of surfaces F _{n, shk3} . After determining the optimal transformation T *, which produces a maximum coincidence of the contours, the change in position ΔP _{SK of} the stereo camera is calculated from the contour operations and the associated operating parameters and from the known geometry of the stereo camera with the aid of the relation f ₁ (T *), for example using trigonometric functions. With the thus determined change in position ΔP _{SK of} the stereo camera, the position change ΔP _{object is} calculated with the aid of the relation f ₂ (ΔP _SK ).

In the first step of contour generation, in an image processing unit (not shown) at the respective times n and n + 1 from the right and left stereo images STB1n, STB2n, STB1n + 1 and STB2n + 1, the corresponding right and left contour images BK1n, BK2n, BK1n + 1 and BK2n + 1 generated. In the following explanations, contour images are binary contour parts of the. In particular, pixels belonging to a contour are assigned the binary value 1, while pixels which do not belong to the contour are assigned the binary value 0.

The Creating a contour or generating simple lines is one thing necessary step to determine the image change of contours between two different times. Preferably Contours and / or lines are generated which are used to determine the image change are optimally suitable. Contours do not necessarily indicate this the edges of objects, but ideally characterize geometric conditions in the picture and help to get essential characteristics from the picture Extract and reduce to a few contour sections.

at Contour generation takes into account certain criteria. determined For example, contours should be as large, fine, and closed as possible. Because these properties are particularly favorable for the accuracy and the sensitivity of a later investigation of contour match are suitable for contour images. Favorable are in particular curved and / or linear Contouring as possible have a closed shape. Contour matches are in the later Course of the method for determining an optimal stereo camera level SKEn *, SKEn + 1 * and analyzed to determine a transformation T, for example, one stereo partial image BK2n each before analysis moved or transformed.

Great contours with a big one Number of contour pixels result in the analysis of contour matches an extremely safe result in determining the optimal stereo camera levels SKEn *, SKEn + 1 * and / or transformation T. In particular, this is a misinterpretation determination of optimal stereo camera levels SKEn *, SKEn + 1 * and / or the transformation T hardly possible if larger contour parts, from a big one Number of contour pixels are connected. This applies in particular even if confounding factors like For example, there are occlusions. In particular, by the correct contour generation also misrecognitions better recognized become.

For the investigation different contours, several methods for contour generation are used, their results are then summarized in a resulting contour image become.

in principle be very small differences in stochastics and / or smallest Gray scale differences for the contour generation used. In a second approach will be for the Contour generation, for example, the contours of objects, linear formations and / or stronger Brightness or texture differences are used in the image. This approach is especially chosen if surfaces exist, which have very homogeneous distributions of gray values. In one third approach will be the surface boundaries of objects that stand out in the picture, for contour generation used. Furthermore, the contour generation includes the calculation of edges trains of objects.

At the At the beginning of the contour generation, image areas are considered characterized by a relatively uniform contour or texture and that only a very small gray value intensity difference to immediate Environment, for example, a difference of 3% of the gray scale intensity of the environment. For this finds a conversion of the output image into a binary image, in exclusively belonging to a contour Pixels with the binary value 1 are occupied, over several Binarisierungsverfahren instead.

In In a first binarization process, the gray value image is entered via a or several thresholds divided into gray value intervals. In order to can In particular, objects are binarized, extending only in one interval the gray values are off. In particular, objects are also binarized, about gray values depict that over a predetermined lower gray value limit.

An object image BI, which contains, for example, objects which map in a limited gray-scale interval I1 to I2 = I1 + ΔI, is obtained by examining the following criterion BI = (B> I1 <I2)> 0 <1 Formula 1.

in this connection B denotes the original gray value image. A pixel becomes if he a gray value intensity between I1 and I2 has, with the binary value 1 (corresponds to object pixel) occupied. Otherwise it will be with the binary value 0 (corresponds to ambient pixels) occupied. A contour image BK is obtained an object image BI by taking the absolute value of the difference BD of the Object image BI and an object image shifted by one pixel calculated. The shift by one pixel can be to the right (in the positive x direction of the image), to the left (in the negative x direction of the image), up (in the positive y-direction of the image) or down (in the negative y-direction of the image). Also a meaningful Combination of shifts, for example to the left and to below and / or shifting by more than one pixel are possible. Alternatively it is it also possible first the displacement of the object image BI and the difference formation perform, then a threshold criterion analogous to formula 1 on the difference image BD apply.

In a second Binarisierungsverfahren he This is followed by the generation of a contour image BK via the decomposition of an original image B according to the bit planes of the gray value intensities and a subsequent difference formation. A first binary bitplane image is generated by, for example, occupying only the pixels of the bitplane image with the binary value 1, in which the most significant bit is assigned the value 1. For example, if the gray scale intensities are from 0 to 255, they will capture all pixels that have a gray scale intensity greater than 127. A next binary bit-plane image is generated by occupying only the pixels of the bit-plane image with the binary value 1, in which the second most significant bit is assigned the value 1. This decomposition process can be continued into all bit planes of the gray value intensity. A contour image BK is obtained by calculating the difference of the bit plane image and a bit plane image shifted by one pixel analogously to the threshold value method and applying a restriction to the difference image BD using a threshold value method.

The object surfaces in object images BI and Bitebenenbildern, for example, by Threshold method and decomposition of the original picture B after the Bite levels are usually large. It is then advantageous these object surfaces with object surfaces, reflect the other characteristics, through logical links to combine. This can be done, for example, by ANDing a first object image BI1, which by a first method, for example a thresholding method was generated, with a first bitplane image BI2, which by a second method, for example, a decomposition of the original image B after the bit planes was created, happened.

This results in an object image BI, for example, as BI = BI1 AND BI2 Formula 2.

Analogous to the thresholding method, the difference of the object image BI and of an object image shifted by one pixel and calculated the difference image BD is a restriction applied to the value 1 using a thresholding method.

In a third Binarisierungsverfahren the generation of a Contour image BK by the division of the gray scale intensities of the Original picture B by big Values.

there become the gray value intensities of a picture, for example taking the value range from 0 to 255, divided by a number, for example 130. From the result of Division, the integer result is formed, which is also called Entier-Operation referred to as. This results in pixels with a gray value intensity greater than 129 a value of 1, for Pixels with lower gray scale intensities than 130 have a value of 0. For example, to isolate a range of 80 to 130 two divisions, once performed by 80 and once by 130, where the integer result of the division by 130 then of the integer Result of the division is subtracted by 80.

alternative it is also possible first the displacement of the object image BI and the difference formation perform, then a threshold criterion analogous to formula 1 on the difference image BD apply.

at It should be noted in the presented binarization procedure that the presented binarization procedures seemingly arbitrary contours can result. This is due in particular to the fact that the presented Binarisierungsverfahren generate contours consisting of difference values and not arise out of special objects. The so produced Contours can however, as evidence for the later one Determination of the image change be used when the picture taking conditions, for example the lighting conditions, between the recording times, for example n and n + 1, do not change, the pictures are taken similarly and the gray scale intensities or at least the gray value intensity differences not change significantly in the pictures. Under these conditions, contours are always created the same and can be so at different times and / or in the reproduce right contour image BK1n and defined in the second contour image BK2n. For the Determination of the image change In particular, it does not determine whether the contours of objects or derived from small variations in texture or brightness.

Another method for contour generation involves the consideration of object boundaries. For example, gradient information of the original image B is considered. Strong gradients in the greyscale intensities (edges) are emphasized in particular by simultaneously displacing an original image B in the x direction, in the y direction and in the x and y direction by, for example, one pixel in a first step and thus image Bx, By and bxy receives. If the absolute value of the difference of the original picture B and the shifted original pictures Bx, By, Bxy is formed, then difference pictures BDx = | B - Bx |, BDy = | B - By |, BDxy = | B - Bxy | which represent gradients of gray level intensities in different directions. To emphasize weaker edges even more clearly, the difference image can still be squared. To generate a contour image, for example, those pixels in the difference images BDx, BDy and BDxy assigned the binary value 1 whose value is after the difference formation above a predetermined threshold S5. A contour image BK thus arises according to BK = (BDx>S5)> 0 <1 Formula 3.

Around weaker To highlight edges clearly, for example, S5 = 1 is selected. Alternatively it will a contour image from difference images BDx, BDy and BDxy via the Choice of thresholds according to analog generated to formula 1.

This Method may be with the same or different predetermined thresholds for the Difference pictures BDy and BDxy are performed, wherein on the Threshold selection controls the number of generated contour pixels can be. Other methods for generating gradient images include known image processing operations, for example the sobel operator or the roberts operator.

One Another method of contouring integrates a smoothing operation smooth, which is applied to the original image B and a smoothed image Bs = smooth (B, const_s) generated. The smoothing operation can be smooth for example, be designed as a median operation, whereby the smoothed Image Bm = median (B, const_m) is generated. The smoothing constants For example, const_s and const_m are integers that are the strength of smoothing determine.

In the smoothed images Bs and Bm, sharp transitions of highlighting objects are smeared, but the gray scale intensities are still approximately the same as the original image. Denotes BDs the absolute value of the difference of original image B and smoothed image Bs according to BDs = | B - Bs | Formula 4 then in an image BDsb, which according to BDsb = BDs> 0 Formula 5 is calculated, outlines are highlighted that are brighter than the immediate environment. Contours that are darker than the surroundings will be corrected accordingly BDsd = BDs <0 Formula 6 highlighted. Specifically, highlighting outlines for the absolute amount BDm is the difference of the original image B and the smoothed image Bm, resulting in the images BDmd and BDmb.

A contour image BK results, for example, from the absolute value BDm of the difference between the original image B and the smoothed image Bm BK = BDm> Ts <(Ts + 1) - Ts Formula 7 where Ts represents an integer threshold over which the pixel of the difference image BDm is assigned the binary value of 1. Analogously, contour images can be generated from the absolute amount BDs of the difference of original image B and smoothed image Bs and from the images BDmd, BDmb, BDsb, Bdsd, whereby different integer thresholds can be used.

In a further embodiment A contour image BK results from the absolute value BDm of the difference from original image B and smoothed Image Bm analogous to formula 1.

It is also possible several smoothing methods to apply sequentially.

Contours can also be summarized in a picture BDs ^ 2 and the contrast can be increased if in accordance with BDs ^ 2 = (B - Bs) ^ 2 Formula 8 the difference between original image B and smoothed image Bs is squared.

2 shows a schematic flow diagram for generating contour images BK ₁ , BK ₂ , BK ₃ , BK ₄ and BK ₅ via various methods. In 2 is the original image B so B _O called. In order to generate the contour images BK ₁ , BK ₂ , BK ₃ , the original image B _{O is} smoothly smoothed by the operation which generates the image B _S. To generate the contour image BK ₁ , a bit plane image B _{M is} generated from the smoothed image B _S , which contains, for example, only the pixels whose uppermost bit has been assigned the binary value 1. Thereafter, the bit plane image B _M thus generated is shifted by one pixel by means of the shift operation, and the absolute value of the difference from the bit plane image B _M and the shifted bit plane image B _{M is} calculated, which then yields the contour image BK ₁ . Analogously, a contour image BK _{2 is} determined, wherein instead of the generation of a bit plane image, a binary image B _{E is formed} by a division by a constant and the Entier operator. Similarly, the contour image BK _{3 is} determined, in which case a binary image B _{K is} generated by means of a threshold value method. The generation of a contour image BK ₄ comprises in a first step the AND operation of two original images B _O smoothed with different smoothing constants q and w. With the aid of a threshold criterion and the threshold S3, a binary image B _G is then generated, for which then the absolute value of the difference of the binary image B _G from the shifted image B _{G is} calculated analogously to the generation of the contour image BK ₁ net, which gives the contour image BK ₄ . Analogously to the generation of the contour image BK ₄ , the contour image BK _{5 is} generated, in this case, however, an addition operation of two original images B _o smoothed with different smoothing constants q and w is performed instead of the AND operation. The smoothing operation smooth ensures in particular that the contours in the contour images BK ₁ , BK ₂ , BK ₃ , BK ₄ , BK _{5 are} particularly emphasized.

at it is particularly advantageous for the contour generation to be vertical and / or to highlight horizontal contours. Vertical contours are generated, for example, by generated in different ways Object images BI (for example, by the application of thresholds and / or division of images by constants) a shift operation for example, 1 pixel in the x-direction is performed. For those with multiple procedures Obtained in this way object images BI then becomes the absolute amount the difference between the edited image and the shifted, Object image determined. In the last step, the difference images are, for example, a Or link connected and mainly generated vertical lines. Analog can be moved to contour images BK to produce primarily horizontal contours. This is done for this purpose but a shift in the y direction.

With the generation of vertical and horizontal contours results the possibility of the Determine contour displacements in the x and y directions independently of each other. On the basis of vertical contours, for example, the determination a right-left shift of contours independent of top-down shifting of contours are considered. This will be a reduction of the determination of the contour shifts of one multiplicative complexity in an additive complexity possible. In a multiplicative complexity, all combinations of Operations, such as contour shifts in vertical and horizontal image direction, while considering the operations in an additive complexity individually and the overall result as a superposition of operations is determined. this makes possible advantageously a time savings.

In an advantageous, concrete embodiment is a resulting Contour image K generated by the over different methods generated contour images BK, the different Contain contours algebraically and / or logically linked. A link can while about, for example an AND or an OR or an addition of the over different methods generated contour images are generated. The Result of an AND operation of different contour images BK then provides contours, for example which are very essential. An AND operation is especially then advantageous if any of the various methods for contour generation create a lot of contours that are partly uncharacteristic. This is for example in the contour generation with the help of roberts- and / or sobel operator's case.

By linking contour images BK by means of an addition operation, weaker contours, for example, which are only contained in individual contour images BK, are also included. This is particularly advantageous if areas exist in the image in which too few contours are available to determine a picture change with sufficient certainty. In 3 is a schematic block diagram of the generation of a resulting contour image K shown in the four generated via different methods contour images BK ₀ , BK ₁ , BK ₂ , BK _{3 are} linked via an OR link to a resulting contour image K.

Here, for the generation of the first contour image BK _0, an original image B is shifted by, for example, 1 pixel, and the difference of the original image B and the shifted original image is calculated. Then the contour image BK _{0 is} calculated by means of the division by a constant const and the restriction to values between 0 and 1. Analogously, the generation of the contour image BK ₁ , wherein the contour image BK _{1 is determined} by means of a threshold value method with the threshold S ₁ from the difference of the original image B and the shifted original image. To generate the contour image BK ₂ , the original image B is smoothed smoothly by means of the smoothing function, and the difference between the smoothed image and the original image B is also formed. From the difference, the contour image BK _{2 is} generated by means of a threshold value method with the threshold S ₁ . In order to generate BK ₃ , a bit plane image is generated in a first step by occupying pixels with the binary value 1 whose seventh bit was assigned a binary 1 in the dual gray value representation. This bitplane image is then shifted by 1 pixel. The absolute value of the difference image gives the contour image BK ₃ .

In a post-processing, the resulting contour image K or the above Various methods generated contour images BK using a Contour dilution operation be edited, the pixel width of the contour on one Pixel reduced. A contour dilution operation is over for example given the known in image processing operation of erosion.

Of course, in addition to the methods for contour production already presented, it is also possible to use further methods for producing the contour. These can be sto consider chastic properties and / or texture changes in contour generation.

There for one high accuracy in the determination of contour displacements, contour rotations and / or contour magnifications or reductions ΔV Sufficient number of contours must be present, the number of generated Contours after a link controlled by contour images BK adaptive. For example, they are too few Contours in a contour image contain BKn, then may need. also weaker Contours are determined.

In an advantageous, concrete embodiment, the number of contour pixels in the resulting contour image K is controlled by threshold value methods with adaptive thresholds. A first resulting contour image K1 is, for example, according to K1 = Kinf> S6 <S7 Formula 9 generated. Kinf designates the addition of several contour images BK, which were emitted via different binarization processes. S6 and S7 denote predetermined thresholds. If too many contour pixels are present in the first resulting contour image K1, then a second resulting contour image K2 may occur K2 = Kinf> S7 <S8 Formula 10 where S8>S7> S6.

There Contours in a contour image BK always by the binary value 1, have contours in kinf, depending on whether they are were generated in one or more binarization methods, values between 1 and the number of binarization methods used.

In a preferred embodiment, the generation of a third resulting contour image K3 according to K3 = Kinf> Simport Formula 11 where Simport is an adaptive contour selection value. A high contour selection value Simport generates fewer contours in the third resulting contour image K3 than a lower contour selection value Simport. A typical contour selection value Simport is S6 = 1. In a preferred embodiment, the contour selection value Simport is controlled over a desired area of contours of the third resulting contour image K3. If there is too little area of contours (number of contour pixels) in the third resulting contour image K3 to allow a reliable determination of the image change in the image, a smaller contour selection value S6 is selected to include more contours. The reduction of the contour selection value S6 is continued until the number of contour pixels has reached a desired number. With a value of S6 = 0, the contours generated by all binarization methods are taken over into the third resulting contour image K3.

Farther can when adding contours through different methods for generating contours via the use of the contour selection value S6, which is larger than 1 is selected, contours which are particularly important and distinctive. This will change the picture in the picture primarily determined from the important contours. More Contours should be over adding a reduction of the contour selection value S6, if the accuracy of the result of the determination of the image change not enough. However, the contour selection value S6 must not chosen low be, because due to image conditions very many contours exist could be. If this is similar and / or have recurring structures, there is a risk that misinterpretations of the image change, especially in occlusions, respectively. In this case, the contour selection value S6 must be set higher.

Of course you can Kinf instead of an addition of the results of the various Binarization also a multiplicative and / or mixed shortcut the results of various Binarisierungsverfahren included. This makes it possible, in particular, to increase weak differences. Analogous can then selection procedure for otherwise linked Contour images BK are applied.

Around one possible even distribution of To achieve contours within the entire image, the number becomes the generated contour pixels for certain Subregions of the image adaptively controlled. So, for example at the edge surfaces, the most sensitive for a distinction made by stereo camera translation and stereo camera shares Contour shifts are a sufficiently large number of contour pixels be ensured. The number of contour pixels and their uniform distribution over the entire picture are important to use in later steps of the procedure the matches of contours of the right and the left left contour image and also the agreement of contour images at different times after an application of Contour operations can be clearly determined.

After the generation of contour images at time n BK1n, BK2n, contours are extracted from these images, which are in a stereo camera plane SKEn in space. This is a Flächenver equal to all contour surfaces, or even parts thereof, of the right contour image BK1n and performed by one or more pixels in the x and / or y direction left contour image BK2n performed. This area comparison can be realized, for example, by ANDing the right contour image BK1n and the left, shifted contour image. As in 1 For this purpose, the left contour image BK2n is iteratively shifted by a total displacement shk1 = (Delta_xk1, Delta_yk1), where shk1 consists of displacements in the x direction delta_x and y direction delta_y. This is done for a predetermined number of total shifts, for example, entered in a list. The initial entry of the list is indicated with and the end entry with. A run index k1 iteratively runs from k1 = ... and indexes certain list entries, which in turn contain information about the desired shift in the x direction delta_x and y direction delta_y. From the magnitude of the shift of the left contour image BK2n in the x-direction and y-direction, the distance of a stereo camera plane SKEn_k1 from the stereo camera, in which the determined number of matching contour pixels lies, can be determined simultaneously. For each value of the running variable k1, the area _comparison results in an area F _{n, shk1} which corresponds to the number of contour points that result in the respective stereo camera plane SKEn_k1. In the case of an AND operation, the area F _{n, shk1 indicates} the number of contour points which are present at the same pixel in both contour images, the right-hand contour image BK1n and the left, displaced contour image BK2n. Of course, it is also possible to move the right contour image BK1n and then to compare with the left contour image BK2n.

Similarly, at time n + 1, the areas F _{n + 1, shk2} and the corresponding stereo camera planes SKEn + 1_k2 are determined from the contour images BK1n + 1 and BK2n + 1 of the next time step.

After the areas F _{n, shk1} and F _{n + 1, shk 2 have been} determined for all displacements shk1 and shk2, an optimal stereo camera level SKEn * is selected so that it contains as many distinct contours as possible. For this purpose, a shift shk1 * from the entire set shk1 and a shift shk2 * from the entire set shk2 is selected, which simultaneously optimizes three criteria. The shk1 * and shk2 * shifts are selected as the shifts where the areas _{Fn, shk1} * and _{Fn + 1, shk2} * are maximum, and at the same time the absolute value of the difference of shk1 * and shk2 * is minimum. Thus, optimal stereo camera levels SKEn * and SKEn + 1 * are selected for both times n and n + 1, which are approximately equidistant from the stereo camera and contain as many contours as possible.

Stereo Camera levels SKEn and SKEn + 1, which approximate equidistant from the stereo camera plane, are in the Hereafter referred to as corresponding stereo camera planes.

To Determining the optimal stereo camera level SKEn * becomes the right one Contour image BK1n on the contours, which in the optimal stereo camera plane SKEn * lie, reduces and thus yields an optimal contour image BKn *. Similarly, the right contour image BK1n + 1 is applied to the contours lie in the optimal stereo camera level SKEn + 1 *, reduced and thus gives an optimal contour image BKn + 1 *. Only this optimal Contour images BKn * and BKn + 1 * will then be in chronological order n on n + 1. Of course it is also possible, the optimal Contour images BKn *, BKn + 1 * from the left contour images BK2n, BK2n + 1 to produce.

To Generation of the optimal contour images BKn * and BKn + 1 * is in one third step, a transformation T sought, the one maximum match of contours in the selected Stereo camera levels SKEn *, SKEn + 1 * before and after the change in position the stereo camera produces. The transformation T presses the picture change of contours at different times n and n + 1. To is the optimal contour image BKn + 1 * with a transformation T transformed and the transformed optimal contour image T (BKn + 1 *) is compared with the optimal contour image BKn *. A transformation T is only accepted if approximately all contour pixels, for example 95%, after the transformation T match, otherwise, a transformation T over the contour images BKn, BKn + 1 calculated from other corresponding stereo camera planes SKEn and SKEn + 1 were generated. The match can be done for example via a AND operation the optimal contour image BKn * and the transformed optimal Contour image T (BKn + 1 *) are determined.

to Determining the transformation T be different contour operations performed on the optimal contour image BKn + 1 *.

In a first operation, the contours of the optimal contour image BKn + 1 * are rotated in the image plane (contour rotation). In a second operation, the contours of the optimal contour image BKn + 1 * are shifted pixel by pixel in the x and y direction (contour shift). In a third operation, the contours of the optimal contour image BKn + 1 * are enlarged or reduced (contour enlargement or reduction ΔV). From the proportions of the individual contour operations can then determine the change in position of the stereo camera. Beispielswei From the contour enlargement or reduction ΔV, the forward and backward movement of the stereo camera can be determined.

The contour operations are usually determined sequentially. In a first step, the contour rotation around the vertical image axis is determined. This corresponds to the contour rotation in the image plane. For this purpose, not only the contours contained in the optimal contour images BKn * and BKn + 1 * but in particular all contours of the contour images BK1n and BK1n + 1 or all contours of the contour images BK2n and BK2n + 1 can be used. This is permissible because, for the pure rotation about the vertical image axis, the contours in all stereo camera planes SKEn and SKEn + 1 are rotated by the same rotation angle and the distance of the stereo camera planes SKE from the stereo camera for the contour rotation does not matter. In 4 For example, the number of binary values 1 from an AND operation of a right-hand contour image BK1n and the right-hand contour image BK1n + 1 rotated by a rotation angle over the rotation angle is shown as a solid line. The dashed line in 4 represents the number of binary values 1 from an AND operation of a left contour image BK2n and the left contour image BK2n + 1 rotated by the rotation angle. If all contours are considered for determining the contour rotation, the number of binary 1 values is obtained from the AND operation for the right contour images BK1n, BK1n + 1 and the left contour images BK2n, BK2n + 1 a sharp maximum.

For determining the contour shift, however, the same conditions do not apply if the contours of all stereo camera planes SKE are considered. In 5 For example, the number of binary values 1 from an AND operation of a right-hand contour image BK1n and the right-hand contour image BK1n + 1 shifted by an amount Delta_x in the x-direction over the amount of displacement Delta_x is shown as a solid line. The dashed line in 5 represents the number of binary values 1 from an AND operation of a left contour image BK2n and the left contour image BK2n + 1 shifted in the x direction by the amount Delta_x. In this case, there is no sharp maximum but a plurality of local maxima, where the shift between the dashed line and the solid line corresponds to the distance of the left and right cameras of the stereo camera, for example. For the determination of the contour shift, therefore, only contours are to be considered which lie in very similar stereo camera planes SKEn and SKEn + 1. When determining the contour shift, the optimum contour images BKn *, BKn + 1 * are preferably to be considered, since they lie in an optimum stereo camera plane SKEn *, SKEn + 1 *. As in 6 In the determination of the contour shift, the optimal contour image BKn + 1 * in the x and / or y direction is shifted by the amounts Delta_x and Delta_y until a maximum match occurs. Analogously to the determination of the optimum stereo camera plane SKEn *, SKEn + 1 *, the optimum contour image BKn + 1 * is shifted by a total displacement shk = (Delta_xk, Delta_yk), whereby the total displacement shk consists of shifts in the x direction Delta_xk and y direction Delta_yk , This is done for a predetermined number of total shifts, for example, entered in a list. The start entry of the list is marked with and the end entry with. A run index k runs iteratively from k = to ... and indexes certain list entries, which in turn contain information about the displacements in the x direction delta_x and y direction delta_y.

Then the area becomes F _{n, shk} = F (BKnUNDshift (BKn + 1, shk)) Formula 12 The surface F _{n, shk represents} the number of binary values 1 from an AND operation of the optimal contour image BKn * and the shifted optimal contour image BKn + 1 *. The area F _{n, shk} is therefore identical in this case with the number of contour pixels that overlap after the contour shift.

A maximum match occurs for the optimal total _shift sh *, for which the area F _{n, shk becomes} maximum. The maximum will be according to the condition sh * = max_k (F _{n, shk} ) with k from [an ... en] Formula 13 certainly. In this case, only a small area of the optimum contour images BKn * and BKn + 1 * is preferably considered, for example a section of the contour image center. In 6 is further shown that the result of the AND operation F _{n, shk is compared} with the result of the AND operation for shk-1. To save computation time, the iteration may be aborted if a local maximum of match has been found. This is the case, for example, if the area F _{n, shk-1 was} significantly larger than the area F _{n, shk} and F _{n, shk-2} . For this purpose, at least the areas F _{n, shk-1} and F _{n, shk-2} are stored in a buffer.

Become not the optimal contour images BKn * and BKn + 1 * for the determination the contour shift is selected, so contours are preferably from corresponding stereo camera levels SKEn and SKEn + 1 are selected to determine the contour shift, the lie near the stereo camera, as for such stereo camera planes the maximum of the match much more pronounced is.

In a third step, the determination of the contour enlargement or reduction ΔV takes place. In 7 FIG. 3 is a schematic flowchart for calculating a contour enlargement or reduction .DELTA.V. In this case, a contour enlargement or reduction ΔV of the optimum contour image BKn + 1 * is performed. After the contour enlargement or reduction .DELTA.V, the reduced contour image is extracted or the enlarged contour image is contracted in order to produce an equal size with the optimum contour image BKn *. Only then is it possible to compare the enlarged or reduced optimal contour image BKn + 1 * with the optimal contour image BKn *.

For this is the optimal contour image BKn + 1 * preferably in a central Part and divided into an edge that is so large that the maximum allowable contour enlargement or reduction ΔV not leads to a loss of information. This is the case, for example, if at a maximum contour enlargement of the central part exceeded the given by the CCD sensor image size becomes.

The optimal contour image BKn + 1 *, or its central part, will be around a contour enlargement or reduction ΔV enlarged or downsized, this being for a predetermined number of contour enlargements or reductions ΔV takes place, the for example, are listed in a list. The initial entry the list is marked with and the end entry with. A running index k is running iteratively from k = an ... en and indexes certain list entries that turn certain contour enlargements or Reductions ΔV describe.

The enlarged or reduced optimal contour image BKn + 1 * is compared after a corresponding contraction or extraction with the optimal contour image BKn *, for example via an AND operation. With an AND operation, an area F _{n, ΔVk} , which is identical to the number of contour pixels resulting from the contour enlargement or reduction ΔV of the optimum contour image BKn + 1 * with the optimum contour image BKn *, results analogously to formula 12. cover.

In 7 is further shown that the result of the AND operation F _{n, .DELTA.Vk is compared} with the result of the AND operation for shk-1. To save computation time, the iteration may be aborted if a local maximum of match has been found.

Preferably the determination of the contour enlargement or reduction ΔV takes place after the Determination of contour shift and contour rotation or if contour shift and Konturotation already approximate are determined. If not the optimal contour images BKn * and BKn + 1 * for the determination of the contour enlargement or reduction ΔV is selected, it is advantageous to have contours from very close to the stereo camera plane, corresponding stereo camera levels SKEn and SKEn + 1, because in these the effect of the contour enlargement or reduction .DELTA.V especially is clear.

As a general rule should be for optimal determination of the change in position of the object stereo camera levels SKE selected which contain many contours on the one hand, but on the other hand also a strong picture change from a time n to the time n + 1 included. In a another embodiment can therefore the size of the image change not only for the optimal contour images BKn * and BKn + 1 * are examined, but for Contour images consisting of non-optimal corresponding stereo camera planes SKEn, SKEn + 1 were generated, and then contour images BKn and BKn + 1 selected be the one possible size Contains number of contour pixels and their contours as possible size picture change Experienced.

Of course it is it also possible the contour image BK1n or optimal generated at the first time n Contour image BKn * to transform and with the second time n + 1 generated contour image BK1n + 1 or optimal contour image BKn + 1 * to compare. Are both possibilities of transformation determination, ie transformation from the contour image BK1n + 1 to the contour image BK1n and transformation of contour image BK1n + 1 onto the contour image BK1n used, so can control the transformation determination by a comparison of the two specific transformations take place.

Out the geometric relationships in the image, in particular the contour shifts, the Konturrotationen and the contour enlargements and reductions ΔV can be the change in position the stereo camera determine if the geometry of the stereo camera exactly known and a calibration of the focal plane is done.

With The known stereo camera model can be any three-dimensional point on the associated Pixels in the right and left stereo subpictures STB1n, STB2n and STB1n + 1, STB2n + 1, for example, via homogeneous coordinates and a corresponding scaling factor.

In 8th the determination of a stereo camera translation from a mapping change of a point M is shown. Here, the ratio of the distance D of the point M of the focal plane and the focal length f of the stereo camera is equal to the ratio of the stereo camera shift KV to Ver shift the pixel u _n of M in the right stereo partial image STB1n recorded at the first time n to the pixel u _{n + 1} of M for the right stereo partial image STB 1n + 1 taken at the next instant n + 1. The displacement of the pixels u _n - u _{n + 1} is normalized to the pixel size of the image given by the width of the CCD element. For the resulting stereo camera shift KV results KV = D × (un - un + 1) / f Formula 14.

Analogously, relationships can be developed for the stereo camera rotation and the forward or backward movement of the stereo camera. For example, from the optimal contour enlargement or reduction ΔV * and the distance of the stereo camera planes SKEn + 1 * or SKEn *, the forward or backward movement ΔZ of the stereo camera via ΔZ = ΔV * × dist (SKEn + 1 *) Formula 15 where dist (SKEn + 1 *) denotes the distance of the optimum stereo camera plane SKEn + 1 * from the stereo camera. It should be noted here that ΔV * is positive for a contour enlargement and negative for a contour reduction.

at the determination of the stereo camera translation, the stereo camera rotation and the forward or backward movement For example, the stereo camera may have the origin of the world coordinate system in the right stereo subpicture STB1n of the stereo camera at time n be placed. A change of position the entire stereo camera will then be relative to this world coordinate system expressed. The left stereo subpicture STB2n of the stereo camera at time n results, for example, over an Euclidean transformation, which in "X. Armangue, H. Araújo, S. Salvi, A review of egomotion by means of differential epipolar geometry applied to the movement of a mobile robot, Pattern Recognition 36 (2003), 2927-2944 " is.

If R is the rotation matrix and Tr is the translation matrix and ΔV is the contour enlargement or reduction, then the position change of the stereo camera between the times n and n + 1 is given by Pn + 1 = ΔV × (R × Pn + Tr) Formula 16 where Pn denotes, for example, the Cartesian coordinates (xn, yn, zn) of the stereo camera at time n. In this case, R is a 3 × 3 matrix whose columns contain information about the stereo camera rotation about the axes (x, y, z) of the world coordinate system. Tr is a vector containing partial translations of the stereo camera translation along the axes of the world coordinate system. The magnification V is here considered to be the same for the x and y direction of the stereo partial image, since the CCD matrix sensor points can be regarded as squares.

Analogous for the determination of the stereo camera translation, the stereo camera rotation and the forward or backward movement the stereo camera from a point M, the stereo camera translation, the stereo camera rotation and the forward and backward movement also from the determined image changes the contours are calculated.

In some cases, image changes, in particular contour shifts, can be generated in the contour images BKn and BKn + 1 between the different times n and n + 1 by stereo camera rotation and stereo camera translation. In order to improve discrimination of contributions of the stereo camera rotation and stereo camera translation to the contour shift, it is preferable to use the left upper margin and the lower right margin of the optimum contour images BKn * and BKn + 1 *. This margin selection is based on the following consideration. If a contour shift is the result of a stereo camera rotation, then the amount of the contour shift is the same for the contours in all image areas, in particular also in the border areas. If the contour shift is not the sole result of a stereo camera rotation, the amounts of contour shifting of the contours in different image areas are different. This is especially true for the right and left or upper and lower edges of an image. With the choice of the edge regions can thus be distinguished by stereo camera translation of a generated by stereo camera rotation contour shift, since in a stereo camera rotation of almost the same value of a contour shift occurs at both edges. In stereo camera translation, a difference in the value of the contour shift occurs because of the different angles at which the right and left edges are imaged. For example, when moving a camera on the right side of a stereo camera, the movement of contours that are imaged on near-axis rays differs from the contours of contours that are imaged over near-edge rays. Two contours are displayed in different sizes in a right camera of a stereo camera plane, for example, at a translation at the right and at the left edge of the picture and different shifts in the contour pictures BK1n and BKn + 1 before and after the stereo camera translation. On a rotation of an exemplary right camera of a stereo camera, however, there are no differences between the displacement of contours, which are imaged on near-axis rays, and the displacement of contours that are imaged over near-edge rays. Two contours of a plane are displayed at a rotation of the example right camera of a stereo camera on the right and left edge of the same size and equal in the contour images BKn and BKn + 1 before and after the rotation of the stereo camera.

If the considered stereo camera levels SKEn * and SKEn + 1 * are far from the stereo camera are removed and the contours contemplated near the Picture center, so is a distinction between the contributions from stereo camera translation and stereo camera rotation to contour shifting difficult. In this case, the contribution of stereo camera rotation for contour shifting are determined exactly by all at the time n detected stereo camera levels SKEn with the corresponding, At the time n + 1 determined, stereo camera levels SKEn + 1 compared. Of the Comparison can be made, for example, via an AND operation of the from the stereo camera levels SKEn or SKEn + 1 generated contour images BKn and BKn + 1 be realized. For Contour pictures BKn and BKn + 1 from the stereo camera levels SKEn and SKEn + 1, where most of the contours overlap, and in particular the result of the AND operation is maximum, then can be generated by the stereo camera rotation Contour shift can be optimally determined. This is special then the case when the maximum of the AND is clearly pronounced, especially for different distant stereo camera levels SKEn, SKEn + 1.

Especially may also be the mean of that across all stereo camera levels determined contribution of the stereo camera rotation to the contour shift be determined.

With the contribution of the stereo camera rotation to the contour shift so precisely determined let yourself in a second step, the contribution of stereo camera translation to the contour shift from the edge zones of corresponding Contour images BKn, BKn + 1 determine because the difference between the contour shift due to the stereo camera rotation and actually detected contour shifts from the stereo camera translation result.

In 9 Figure 4 is a schematic flow diagram for simultaneously determining contour shifts resulting from stereo camera rotation and stereo camera translation. For this purpose, the region of the upper left edge liob and the lower right edge reun are selected in an, for example, optimal contour image BKn *. Accordingly, the corresponding regions liob and reun are selected in the contour image BKn + 1 *. Analogous to the determination of the optimum stereo camera plane SKEn + 1 *, both regions liob and reun of the optimal contour image BKn + 1 * are shifted by a total displacement shk = (Delta_xk, Delta_yk), whereby the total displacement shk results from shifts in the x direction Delta_x and y direction Delta_y exists. This is done for a predetermined number of total shifts, for example, entered in a list. The start entry of the list is marked with and the end entry with. A run index k iteratively runs from k = to ... and indexes certain list entries, which in turn contain information about the desired shift in the x direction delta_x and y direction delta_y.

The shifted region liob of the optimal contour image BKn + 1 * is then compared with the region liob of the optimal contour image BKn *, for example via an AND link. With an AND operation, analogous to formula 12, an area F _{n, shk is obtained} which is identical to the number of contour pixels which, after the shift, lie in the region liob of the optimal contour image BKn * and in the region liob of the optimal contour image BKn Cover + 1 *.

In 9 is further shown that the result of the AND operation F _{n, shk is compared} with the result of the AND operation for shk-1. To save computation time, the iteration may be aborted if a local maximum of match has been found. Analogously, the optimum contour images BKn * and BKn + 1 * are traversed for the region reun. Since the stereo camera rotation causes the same contour shifts in the regions liob and reun, the difference in the determined shifts provides information about the contour shifts generated by stereo camera translation.

In an advantageous embodiment, the determination of the contribution of the stereo camera rotation to the contour shift, in particular if there is no stereo camera translation, can take place via the following procedure. At least three contours are selected by means of different stereo camera planes SKEn + 1, which are parallel to the stereo camera plane and are far apart from each other. In this case, the stereo camera plane closest to the stereo camera stabilization plane must be at least so far away that the circle of the same distance in the stereo camera rotation can be approximated by piecewise straight lines from the length of the contour. Otherwise, the distortion of the contour in the image to the stereo camera level must be taken into account, which affects the real-time capability of the process. Regardless of the distance of the selected stereo camera levels SKEn + 1, stereo camera rotation will occur a contour shift of all contours in the different stereo camera levels SKEn + 1 by the same amount. This is schematically in 10 where three contours Cont. 1, Cont. 2, Cont. 3 are mapped to a stereo camera plane at time n and to a rotated stereo camera plane at time n + 1. The contour image BKn + 1 at time n + 1 resulting from the at least three stereo camera planes is then shifted in the x-direction by an amount Delta_x and linked to the contour image BKn resulting from the at least corresponding stereo camera planes at time n, for example via an AND -Shortcut. Since the same shift amounts result for all contours Kont. 1, Kont. 2 and Kont. 3 in a stereo camera rotation, the estimated value of the stereo camera rotation can be determined from the contour shift at which the result of the link is maximum.

In 11 the schematic procedure of determining a pure stereo camera rotation is shown. Analogously to the determination of the optimum stereo camera plane SKEn + 1 *, for example, a contour image BKn + 1, which contains at least 3 differently distant stereo camera planes SKE, is shifted by a total shift shk = (Delta_xk, Delta_yk), whereby the total shift shk results from shifts in the x-direction Delta_x and y-direction Delta_y exists. This is done for a predetermined number of total shifts, for example, entered in a list. The start entry of the list is marked with and the end entry with. A run index k runs iteratively from k = to ... and indexes certain list entries, which in turn contain information about the displacement in the x direction delta_x and y direction delta_y.

The shifted contour image BKn + 1, which contains at least 3 different far-removed planes, is then compared with the contour image BKn, which contains at least 3 corresponding planes, for example via an AND link. With an AND operation, analogous to formula 12, there results an area F _{n, shk} , which is identical to the number of contour pixels that are present after the displacement of the contour image BKn + 1, which contains at least 3 different far-removed planes, and contour image BKn , which contains at least 3 corresponding levels, cover.

In 11 is further shown that the result of the AND operation F _{n, shk is compared} with the result of the AND operation for shk-1. To save computation time, the iteration may be aborted if a local maximum of match has been found.

Becomes the change of position the stereo camera of two, for example, optimal, contour images BKn * and BKn + 1 *, all possibilities of contour shifting are determined in x and y directions, all possible angles of contour rotation and all possibilities the contour enlargement or reduction ΔV consider. The one for this The number of variations considered is usually large, which means an analysis time consuming becomes. Since the determination of the contour shift in different directions, the Konturrotation and the contour enlargements and reductions .DELTA.V not independently carried out In a preferred embodiment, measures are taken used to speed up the investigation.

These affect the restriction of the contour types, for the contour shifts, contour rotation and / or contour enlargement or Reductions ΔV be determined. In particular, such contours are determined the one separation of the contributions of contour displacements, contour rotations and contour enlargement or Reductions ΔV enable.

For example vertical contours are generated, which are used to advantage can, to determine contour shifts in the x-direction particularly sensitively. Similarly, horizontal contours are generated and used to contour displacements to determine particularly sensitive in the y-direction. This allows the Search range of the contour shift are limited, for example on 3 pixels. Furthermore, by the separation of contour shifts in x-direction and y-direction the number of possible contour shifts in the x-direction anx of the number of possible contour shifts be separated in y direction any. Instead of the multiplicative number the total variations anx x any will now only be the sum of the individual variations anx + any, considered.

In a further embodiment Only a subset of the optimal contour images BKn * and BKn + 1 * considered. To contour displacements in the x direction especially sensitive to determine the subregions are preferably small in x-direction and big in to choose y-direction, for example, 5 pixels in the x-direction and H pixels in the y-direction, where H denotes the number of pixels of the image in the y direction. Analog can be for the determination of the contour shift in the y direction is a subarea chosen which is small in the y direction and large in the x direction.

The selection of the partial region can take place only in the optimal contour images BKn * and BKn + 1 *, but it can also already take place in the resulting contour image K or in the contour images BK generated by different methods. Preferably, the choice of partial area is combined with the generation of horizontal and / or vertical contours. The analysis can be done largely parallel for several image areas, resulting in a shorter computing time.

A another possibility The acceleration results when the distances to the run indices k belong Values are roughly selected in a first pass. Then one can desired Range of values, the number of run indices is chosen to be small. At the optimal run index kopt1 determined at the first pass related Values will now in a second step, a finer resolution of belonging to the run indexes Values selected. Here, the number of run indexes can be increased or remain the same.

A another possibility the acceleration results when in a first pass only certain run indices, for example every second, and those to them Associated with run indexes Values selected become. Analog can then be determined by the one to the in the first pass, optimal run index kopt1 belonging Values in a second step a finer resolution of the values associated with the run indexes chosen become. Here, the number of run indexes can be increased or stay the same.

In front the second run can be tested, whether with the help of roughly chosen resolution an optimum could be determined. With a contour shift for example, one of the determined in the first pass Contour shift by one or more pixels deviating contour shift analyzed, with the ideally deviating contour shifts be analyzed in all directions. If the results of the match with the contour shift deviating from the optimum contour shift for all deviating contour shifts are worse than the results the match with the optimum contour shift determined in the first pass, so the second pass is no longer performed.

A Further acceleration results if before the determination of Contour displacements and / or contour enlargements or reductions ΔV the optimum Konturotation in the vertical image plane is determined. For that, As previously presented, the contour rotation is determined by using contour rotation values for multiple stereo camera levels be determined and, for example, the mean of all thus determined Konturrotationswerte is formed. Since the Konturrotation regardless of the determination of contour displacements and / or contour enlargements or reductions ΔV can be determined is also the multiplicative number of Overall variations from all contour notations and contour shifts Significantly reduced because only one contra-rotation is considered must become.

In a preferred embodiment becomes the separation of the determination of contour displacements, contour rotation and / or contour enlargement or Reductions ΔV be used to obtain a first result of the contour shifts, the Konturrotation and / or the contour enlargements and reductions ΔV to determine. After that you can do a coupled search over a restricted search area performed around the first result become.

Furthermore, in order to accelerate the determination of contour rotation, contour displacements and / or contour enlargements or reductions ΔV, a control of the running indices k can be undertaken. For the individual searches, the search is stopped when you have reached a clear optimum. This is in the 6 . 7 . 9 . 11 shown.

For example the search is aborted when the number of contour pixels, the for covering different times n and n + 1, for run indices before the optimal running index kopt and for running indices after the optimal one Running index kopt significantly smaller than the number of overlapping contour pixels for the optimal run index kopt.

Of course they are Other methods for determining kopt conceivable, in particular one-dimensional Optimization methods such as the so-called Golden Section method.

Become for the Determining the contour operations not the optimal contour images BKn * and BKn + 1 * are selected, so preferably contour images BKn and BKn + 1 are to be selected, the from corresponding stereo camera levels SKEn and SKEn + 1 generated were.

The Running indices k, k1, k2 can in the context of the respective conformity analyzes over an equal great or different size Interval run.

Of course it is the procedure not only for Grayscale images applicable. Possible is in particular a use of the method for color images, for example, RGB images.

Claims (11)

A method for determining a change in position of an object by means of a stereo camera, which is connected to the object defined and provides a stereo image, the at least one first stereo a stereo camera plane (SKEn) at a first time (s) via a shift of the first stereo partial image (STB1 n) to the second stereo partial image (STB2n) and a stereo camera level (SKEn + 1) at a second time point (n + 1) via a shift of the first stereo partial image (STB1n + 1) to the second stereo partial image (STB2n + 1) such that a surface (F _{n, shk1} ) of contours formed in the Stereocamera level (SKEn) at the first time (s) are located, and a surface (F _{n, shk2} ) of contours that are in the stereo camera level (SKEn) at the second time (n + 1) are maximum and the difference of the distance Stereo camera plane (SKEn) at the first time (s) to a stereo camera plane and the distance of the stereo camera plane (SKEn + 1) to the second time point (n + 1) to the stereo camera plane is minimal, - wherein an image change as a transformation (T) of at least the first , at a first time (n), recorded stereo partial image (STB1n) to a first, at a second time (n + 1), recorded stereo partial image (STB1n + 1) is determined, in which a maximum coincidence of the contours in the first and the second stereo temporal images (STB1n, STB1n + 1) recorded at the second time point (n, n + 1), the contours being at the first time point (n) in the stereo camera plane (SKEn) and the contours at the second time point (n. + 1) lie in the stereo camera plane (SKEn + 1), - wherein a change in position of the stereo camera is determined by a relationship between the change in position of the stereo camera and a change in the image of contours. Method according to claim 1, characterized in that that the contours are displayed in a contour image (BK), which over the Difference of a first binary image and a second, opposite the first binary image shifted, binary image is generated, wherein the first and second binary image via a comparison of a first and a second gray scale image having predetermined thresholds and / or selection of bit planes of the gray value images and / or the division the grayscale images by large Values and subsequent Calculation of the integer values of the quotients can be calculated. Method according to one of the preceding claims, characterized in that the contours are represented in a contour image (BK) which is over the calculation of edges is generated in a gray value image, wherein the calculation of edges over the determination of gray value gradient amounts of the gray value image and a Comparison of gray level gradient amounts with a predetermined one Grayscale gradient amount and / or the application of edge detection filters he follows. Method according to one of the preceding claims, characterized in that the contours are represented in a contour image (BK) which is over an absolute amount of a difference of one by one or more Smoothing functions of smoothed gray value image and the unsmoothed gray value image and comparing the absolute value of the difference with a predetermined one Glättungsschwellwert is produced. Method according to one of the preceding claims, characterized in that the contours are represented in a contour image (BK) which is over the difference between a first gray scale image and a second gray scale image, the above contour enhancing Image processing method from an unprocessed gray value image are derived, the second opposite to the first gray value image in horizontal or vertical image direction is moved. Method according to one of the preceding claims, characterized characterized in that a resulting contour image (K) is generated, by having over different Method generated contour images (BK) algebraically and / or logically connected and compared with a predetermined contour selection value (Simport) become. Method according to one of the preceding claims, characterized in that a first contour image (BK1) of the first stereo partial image (STB1) and a second contour image (BK2) of the second stereo partial image (STB2) is generated, wherein a surface (F _{n, snk1} , F _{n, shk2} ) of contours from a surface comparison of contour surfaces of the first contour image (BK1) and of contour surfaces of the second contour image (BK2) displaced by one or more pixels in the x and / or y direction, a logical AND being determined. Linking the first and second contour image (BK1n, BK2n) is analyzed. Method according to one of the preceding claims, characterized characterized in that the transformation (T) of at least the first, at a first time (s), recorded stereo partial image (STB1n) on a first, at a second time (n + 1) recorded Stereo partial image (STB1n + 1) from at least two contour operations the group horizontal contour shift, vertical contour shift, Contour augmentation, Contour reduction and contour rotation around the axis perpendicular to the Image plane is determined, the quantative determination of the optimal Operational paramaters, the maximum coincidence of the contours surrender, over an evaluation of all combinations of operating parameters of Contouring operations. Method according to one of claims 1 to 7, characterized that the transformation (T) of at least the first, to a first Time (s), recorded stereo partial image (STB1 n) on a first, at a second time (n + 1), recorded stereo partial image (STB1n + 1) from at least one contour operation of the group horizontal contour shift, vertical contour shift, contour enlargement, contour reduction and contour rotation around the axis perpendicular to the image plane whereby the quantitative determination of the optimal operation parameters, the one maximum match of the contours, by a sequential determination of the optimal Operation parameters for one operation each. Method according to one of the preceding claims, characterized characterized in that the operation parameters for a transformation (T) at least a first, at a first time (s), recorded stereo partial image (STB1n) to a first, at a second time (n + 1), recorded stereo partial image (STB1n + 1) and / or the operational parameters of the displacement of the first stereo subpicture (STB1n) on the second stereo subpicture (STB2n) the one maximum match the contours generated from the images occur in an operation parameter interval lie where the operation parameter interval pass sequentially and the search is aborted once a local maximum matches is found. Apparatus for determining a change in position of an object, comprising at least one stereo camera defined to be connected to the object and providing a stereo image consisting of at least a first stereo partial image (STB1n) and a second stereo partial image (STB2n), and an image processing unit, wherein Stereo camera plane (SKEn) at a first time (s) via a shift of the first stereo partial image (STB1n) to the second stereo partial image (STB2n) and a stereo camera level (SKEn + 1) at a second time point (n + 1) via a displacement of the first stereo partial image (STB1n + 1) to the second stereo partial image (STB2n + 1) are determined so that a surface (F _{n, shk1} ) of contours _lying in the stereo camera plane (SKEn) at the first time (n) and a surface (F _{n, shk2} ) of contours that are in the stereo camera plane (SKEn) at the second time point (n + 1) are the maximum and the difference from the distance of the stereo camera plane (SKEn) to the first Z point (s) to a stereo camera plane and the distance of the stereo camera plane (SKEn + 1) at the second time point (n + 1) to the stereo camera plane is minimal, - wherein an image change as a transformation (T) of at least the first, at a first time (n ) is recorded on a first, at a second time (n + 1), recorded stereo partial image (STB1n + 1), in which a maximum coincidence of the contours in the at the first and the second time (n, n + 1), the contours at the first point in time (n) in the stereo camera plane (SKEn) and the contours at the second point in time (n + 1) in the stereo camera plane (FIG. SKEn + 1), wherein the image processing unit determines a change in position of the stereo camera via a relationship between the change in position of the stereo camera and a change in the image of contours.

Images