DE102008019795A1 - Method for adapting an object model to a three-dimensional point cloud by correcting erroneous correspondences - Google Patents

Abstract

The invention relates to a method for adapting an object model (OM) to a three-dimensional point cloud (PW), in which the point cloud (PW) is generated from two images (B1, B2) by means of a stereo method (S1) and a clustering method (FIG. S2) is applied to the point cloud (PW) in order to identify points belonging to a respective cluster (CL), and subsequently a model adaptation (S3) is carried out in which at least one object model (OM) overlays at least one cluster (CL) and an optimal position of the object model (OM) with respect to the cluster (CL) is determined and wherein a correction (S4) of misallocations of points is performed by means of the adapted object model (OM), wherein at least two iterations (S2), the model matching (S3) and the correction (S4) of misalignments are performed, starting from the second iteration from the previous iteration, a probability of the location for at least one of the P determined in the point cloud (PW, PW ') and in the formation of correspondences in the stereo method (S1) is taken into account.

Description

The The invention relates to a method for adapting an object model to a three-dimensional point cloud, in which the point cloud by means of a stereo method of two images and generates a clustering method is applied to the point cloud to points belonging to each cluster and then model matching is carried out at the at least one object model superimposed on at least one cluster and an optimal position of the object model with respect to the cluster, and wherein a misallocation of points by means of the adapted object model is corrected.

Object models are used to identify objects and their three-dimensional orientation. When adapting an object model to a 3D point cloud, known methods ( Schmidt, J., Voters, C, Krüger, L., Gövert, T., Hermes, C., 2007. 3D Scene Segmentation and Object Tracking in Multiocular Image Sequences. Proc. Int. Conf. to Computer Vision Systems (ICVS), Bielefeld, Germany. ) often to ambiguities (false positive assignments). The object is found multiple times in the point cloud, although it does not exist that often or not at all. Another problem related to model matching is the inaccuracy of the fit. Currently common stereo methods are usually based on the search of features (edges, points, corners, pixel blocks, etc.) in a left and a right image and the subsequent assignment of the same / similar features to each other. Alternatively, the contents of local image windows are often examined with regard to their similarity. The so-called disparity value is then determined by determining the offset of the associated features or image windows in the left and right images to each other. On the assumption of a calibrated camera system, triangulation can then be used to assign a depth value to the associated pixel from the disparity value. In some cases, incorrect depth values occur due to incorrect assignment. This often happens in repetitive structures in the image, such. Finger fingers, forest, etc. in edge-based stereo methods. The resulting from the misallocation 3D points are referred to as mismatches or outliers. Depending on the choice of features, this effect occurs more or less frequently, but can never be ruled out without further assumptions. These erroneous correspondences adversely affect the adaptation of the object model, since they lead to a deterioration in the representation of the scene by the 3D point cloud.

Various methods are known in the literature dealing with the problem of mismatches. Most of the methods try to detect the outliers and then eliminate them. The disadvantage here is the decreasing number of 3D points or the resulting loss of information. Other methods Hirschmuller, H., 2005. Accurate and Efficient Stereo Processing by Semi-Global Matching and Mutual Information, Proc. IEEE Conf. at Computer Vision and Pattern Recognition, San Diego, USA. ] in turn try to suppress the problem, for example, by accepting sections of smooth surfaces. By such smoothness assumptions fine structures are no longer recognizable, resulting in a loss of information. In addition, these methods provide good results only where smooth surfaces are really a factor.

It It is an object of the invention to provide an improved method of adaptation of an object model to indicate a three-dimensional point cloud.

The The object is achieved by a method having the features of claim 1.

advantageous Further developments are the subject of the dependent claims.

at a method of adaptation according to the invention of an object model to a three - dimensional point cloud becomes the Point cloud generated by means of a stereo method of two images and a clustering method applied to the point cloud to identifying points belonging to each cluster.

To generate the three-dimensional point cloud, any stereo method may be used, for example, a spatio-temporal stereo method as described in US Pat. Schmidt, J. Wähler, C., Krüger, L., Gövert, T. Hermes, C. 2007. 3D Scene Segmentation and Object Tracking in Multiocular Image Sequences. Proc. Int. Conf. On Computer Vision Systems (ICVS), Bielefeld, Germany ] is described. This document also describes a clustering method which can be used, for example, in the method according to the invention.

At the Clustering become points of the point cloud that is assumed that they belong to the same object, to so-called clusters summarized.

Following the clustering, a model adaptation takes place in which at least one object model is superimposed on at least one cluster and an optimal position of the object model with respect to the cluster is determined. In particular, a model adaptation method can be used here, as in [ Zhang, Z., 1992. Iterative Point Matching for Registration of Free-Form Curves, INRIA Technical Report 1658 ] is described.

As a result Model matching may mislead a cluster assigned points are identified and eliminated. Likewise erroneously outside of the considered cluster isolated or lying in another cluster points, so-called Outliers, as belonging to the considered cluster identified and the assignment be corrected accordingly.

According to the invention at least two iterations using the stereo method, the clustering method, model matching and correction of misallocations, being after correcting the misallocations in the stereo method is fed back to the correspondence assignment at this Place not only for this point, but if necessary for further affected points. It will from the second iteration for at least one of the points especially for one in the previous iteration as misidentified and corrected point a probability of his whereabouts in the three-dimensional point cloud, as he is possible on the surface of the Object model or cluster. When forming correspondences in the stereo method is now not only the similarity of image areas but also considers the probability of residence, which improves the quality of the point cloud, since fewer misalignments occur.

For example When looking at fingers of one hand, they are very similar to each other are a point with a higher probability the right finger instead of a neighboring finger assigned.

An advantageous embodiment of the solution according to the invention is to segment out image areas which exhibit repetitive structures in which misregistrations are particularly frequently performed by stereo methods, which leads to a deeply flawed depth map of the scene. Repetitive structures in the image are detected, for example, by the determination of significant local maxima in amplitude spectra obtained by Fourier transformation on image regions. The 3D points belonging to the image pixels marked in this way form a cluster, ie the clustering takes place in this exemplary embodiment by the described detector for repetitive image structures. The model of the scene preferably consists of one or more levels. The frequencies at which maxima occur in the amplitude spectrum are used to model visual rays based on the camera parameters, which in turn define a family of planes parallel to the real object plane. The normal vector of this group of planes therefore defines the normal vector of the object plane. With the aid of the initial stereo analysis, the object level z. For example, after transformation of the set of planes into the disparity space spanned by the coordinates u (horizontal image coordinate), v (vertical image coordinate) and d (disparity), the maximum accumulation of 3D points in this (uvd) space is determined (see black dots in 3 ).

Also, in addition or as an alternative, at least one of the two images can be used to create a so-called probability map, which is referred to below as the attention map, as is customary in the literature. This is in detail in [ Tanaka, M., Hotta, K., Kurita, T., Mishima, T., 2008. Dynamic Attention Map by Ising Model for Human Face Detection, Int. Conf. On Pattern Recognition, Brisbane, Australia. ]. A classifier is used to create the attention map, for example the one in [ Wähler, C., Anlauf, JK, 1999. A Time Delay Neural Network Algorithm for Estimating Image-pattern Shape and Motion, Image and Vision Computing 17, pp. 281-294 ] described TDNN. This classifier is previously trained with sample images of a sample object. With the attention map, a statement can be made as to whether an object similar to the example object is in the scene or how many of these objects are present in the scene. The Attention Map also allows you to tell how likely it is that such an object is in a particular location.

The Attention Map can now be supportive in the clustering process and / or taken into account during model adaptation. In the clustering method, the attention map serves the selection more appropriately Cluster for the subsequent model adaptation. In the model adaptation the Attention-Map can be used to calculate an initial pose of the model, starting from the model adaptation is made. This improves the convergence behavior the model adaptation. A good initialization leads to the Model adaptation also causes the problem of local minima which leads to a suboptimal adaptation can.

in the Below is an embodiment of the invention based a drawing explained in more detail.

there shows:

1 a flowchart of a method for adapting an object model to a dreidi metric point cloud.

In 1 1 is a flow chart of a method for adapting an object model OM to a three-dimensional point cloud PW. First, two stereoscopically recorded images B1, B2 are supplied to a stereo method S1. This forms correspondence pairs from points of both pictures B1, B2. The result is a three-dimensional point cloud PW. With a clustering method S2, the point cloud PW is segmented, that is to say formed cluster CL with associated points.

On following the clustering process S2, a model adaptation S3 is found instead of at least one object model OM is superimposed on at least one cluster CL and an optimal position of the object model OM with respect of cluster CL.

With the objects O now identified in their position and pose during a correction S4 of misallocations to a cluster CL incorrectly identified points and identified be eliminated. Likewise, mistakenly isolated from the considered cluster CL or in one other cluster CL lying points, so-called outliers, identified as belonging to the considered cluster CL and the assignment are corrected accordingly, with a corrected point cloud PW 'and corrected clusters CL' arise.

It be at least two iterations of the invention Procedure performed. It will after the correction the misalignments S4 fed back into the stereo method S1. This is from the second iteration for at least one of the points, especially for one in the previous iteration as misassigned and corrected point a probability of his whereabouts in the three-dimensional point cloud PW '. When forming correspondences in the stereo method S1 now not only the similarity of image areas but also takes into account the probability of residence, whereby the quality of the point cloud PW improves, because fewer misalignments occur.

In 2 A second embodiment of the method for adapting the object model OM to the three-dimensional point cloud PW is shown. The procedure is the same as in 1 Shown. In addition, however, the steps clustering method S2 and / or model adaptation S3 are supported by means of a probability map AM, which according to the practices in the literature is referred to below as attention map AM.

The Attention Map AM is in a creation step S6 from at least one of the two images B1, B2 created. To create the attention map AM a classifier KL is used. This classifier KL will previously with example pictures of an example object BO in a training step S5 trains. With the Attention Map AM can make a statement about it whether it is an object similar to the example object BO O is in the scene or how many of these objects O in the scene available. The attention-map AM also allows a statement about how likely it is that yourself such an object O is located at a certain location.

At the Clustering method S2 serves the attention map AM of the selection suitable cluster CL for the subsequent model adaptation S3. In the model adaptation S3, the attention map AM can in particular used to calculate an initial pose of the object model OM from which model adaptation S3 is made.

AT THE

Attention Map, probability map

B1, B2

image

BO

example object

CL, CL '

cluster

O

object

OM

object model

PW, PW '

point cloud

S1

Stereo process

S2

clustering methods

S3

model fit

S4

correction of misallocations

S5

training step

S6

creating step the attention map

QUOTES INCLUDE IN THE DESCRIPTION

This list The documents listed by the applicant have been automated generated and is solely for better information recorded by the reader. The list is not part of the German Patent or utility model application. The DPMA takes over no liability for any errors or omissions.

Cited non-patent literature

• - Schmidt, J., Voters, C, Krüger, L., Gövert, T., Hermes, C., 2007. 3D Scene Segmentation and Object Tracking in Multiocular Image Sequences. Proc. Int. Conf. to Computer Vision Systems (ICVS), Bielefeld, Germany. [0002]
• Hirschmuller, H., 2005. Accurate and Efficient Stereo Processing by Semi-Global Matching and Mutual Information, Proc. IEEE Conf. at Computer Vision and Pattern Recognition, San Diego, USA. [0003]
• - Schmidt, J. Wähler, C., Krüger, L., Gövert, T. Hermes, C. 2007. 3D Scene Segmentation and Object Tracking in Multiocular Image Sequences. Proc. Int. Conf. On Computer Vision Systems (ICVS), Bielefeld, Germany [0008]
• - Zhang, Z., 1992. Iterative Point Matching for Free Form Curves Registration, INRIA Technical Report 1658 [0010]
• - Tanaka, M., Hotta, K., Kurita, T., Mishima, T., 2008. Dynamic Attention Map by Ising Model for Human Face Detection, Int. Conf. On Pattern Recognition, Brisbane, Australia. [0015]
• - Wähler, C., Anlauf, JK, 1999. A Time Delay Neural Network Algorithm for Estimating Image-pattern Shape and Motion, Image and Vision Computing 17, pp. 281-294 [0015]

Claims (3)

Method for adapting an object model (OM) to a three-dimensional point cloud (PW), in which the point cloud (PW) is generated from two images (B1, B2) by means of a stereo method (S1) and a clustering method (S2) is applied to the Point cloud (PW) is applied to identify points belonging to each cluster (CL), and then a model adaptation (S3) is performed, wherein at least one object model (OM) at least one cluster (CL) superimposed and an optimal position of the object model (OM) with respect to the cluster (CL), and wherein a correction (S4) of misallocations of points is carried out by means of the adapted object model (OM), characterized in that at least two iterations are performed with the stereo method (S1) , the clustering method (S2), the model adaptation (S3) and the correction (S4) of misalignments are performed, starting from the second iteration from the previous iteration a probability of the whereabouts for at least one of the points in the point cloud (PW, PW ') determined and in the formation of correspondences in the stereo method (S1) is taken into account. Method according to claim 1, characterized in that that a trained by means of at least one example object (BO) Classifier (KL) to create a probability map (AM) is used, wherein the probability map (AM) on at least one of the images (B1, B2) applied while a number and / or Probability of residence of at least one example object (BO) similar object (O) in the image (B1, B2) is determined, where the probability map (AM) in the clustering process (S2) and / or during the model adaptation (S3) becomes. Method according to claim 2, characterized in that that by means of the probability map (AM) an initial pose of the object model (OM) is determined, starting from the model adaptation (S3) is made.