DE102016116572A1 - Alignment of point clouds to the modeling of interiors - Google Patents

Abstract

The present invention relates to a method for determining an orientation of a source point cloud relative to a target point cloud, wherein the source and the target point cloud each represent partial models of interiors of a building structure and each aligned with respect to a gravitational axis and at least have a small overlap with respect to the actual building geometry. In a first step, a source signature histogram of the source point cloud is cross-correlated with a target signature histogram of the target point cloud to determine rotation of the source point cloud. In a second step, a translation histogram is calculated based on pairs of points, each point pair comprising a source point from the source point cloud and a target point from the target point cloud, in order to determine a displacement of the source point cloud.

Description

Technical area

The present invention relates to a method for aligning two at least partially overlapping point clouds.

The digitization of interiors in the form of three-dimensional (3D) scan recordings has made much progress in recent years, in particular thanks to the development of accurate 3D space scanners and 3D image processing apparatus, as well as the design of efficient algorithms for merging multiple individual scan scans into a comprehensive 3D scan. Model. Such 3D interior models are used in several new technologies, for example, in the navigation of interiors or in the orientation of various service robots. Similarly, existing work areas, such as building maintenance, will benefit from being able to record accurate 3D models in a short time. In view of these advantages, in particular larger interiors as well as shopping centers, museums and company buildings are suitable for 3D mapping. Such interiors have room elements, such as ceilings, floors, walls or corridors, on.

Such large structures are usually not scanned with a single shot. When scanning, there are technical limitations, such as drifts (small billing errors that accumulate over time), limited storage capacity and limited processing power. There are also physical constraints that include the movement constraints of the scanner; For example, wheel-mounted scanning devices can not climb stairs or steps. In addition, there may be time constraints, such as access restrictions, which require scanning to take place in sections during certain windows of time in which access to the interior is granted. Furthermore, scan updates must be performed regularly, and it would be very cumbersome if the entire structure would have to be scanned again.

These scanning limitations mean that multiple, overlapping sub-models need to be merged to create a single, comprehensive 3D model by aligning them. Thus, the alignment of multiple, overlapping point clouds is performed in one or more post-processing steps. In the case of point clouds, these and in the following may be in particular structural models, in particular models of structures of interior spaces of buildings.

State of the art

There is already a branch of research that deals with the alignment of 3D models, which are often stored in the form of grids or point clouds. The problem is called "point cloud alignment", point set registration, or scan registration.

Here, the point clouds, with which interior spaces of buildings are modeled, can contain several thousands or millions of points. Each point is captured as a data structure that contains the Cartesian coordinates of the point in 3D. Optionally, additional information about areas or structures on which the point is located may be captured in the data structure. For example, a point of a point cloud may include information indicating that the point is located on a surface, such as a wall surface, or on a structure, such as a pillar, a beam, or a pillar. In a second example, a point of a point cloud may be placed on the intersection of two surfaces (eg walls, floors or ceilings) or on the intersection of three surfaces, namely on a corner. According to these examples, the particular point includes information regarding its associated areas or three-dimensional structures, for example, by detecting a surface normal vector and / or a surface curvature index in the data structure that may indicate the existence of an edge or corner of the interior at the point.

Advances in 3D mapping and reconstruction have been promoted by the development of accurate space scanners, which have even been implemented as hand-held scanners. Existing mapping systems include wheel base based scanners, backpack mounted scanners, and handheld scanners. Such systems are capable of capturing interior and exterior spaces in point clouds in great detail.

Also, many reconstruction algorithms based on two-dimensional camera images have been developed. An example is the system of Akbarzadeh et al. "Towards Urban 3D Reconstruction from Video" at Third International Symposium on 3D Data Processing, Visualization, and Transmission; 2006 , which reconstructs urban exterior spaces based on video recordings. Another example is the system of Agarwal et al. "Building Rome in a Day," Communications of the ACM, Vol. 10, pages 105-112 , which reconstructs urban exterior spaces based on many camera shots.

The problem of aligning two point clouds refers to two sets of data, also called point sets or point clouds, namely the target (also called a scene in German) which serves as a reference and remains motionless, and the source (in German "source"; also referred to as a model), with respect to which a spatial transformation is sought after which the source is oriented in relation to the target. Theoretically, the source point cloud and the target point cloud are interchangeable in the application of the alignment algorithm.

In general, the problem of the point cloud alignment can be divided into two operations, namely the coarse alignment and the fine alignment. In the coarse alignment, there are no restrictions on the initial alignment, and the alignment result is substantially correct. Fine alignment requires good pre-alignment and provides very accurate alignment between source and target. The Iterative Closest Point (ICP) algorithm is selected by default for fine alignment. The following section deals with the problem of coarse alignment. In this case, any initial alignment of the source and target point cloud is possible, provided that the point clouds are already aligned with respect to the gravitational axis or the gravitational direction of the earth, i. have a common local Z axis.

Several approaches have been developed for the task of coarse alignment of different point clouds. Various approaches for different alignment tasks are particularly well suited, as described in more detail in the following discussion.

Approaches for the coarse alignment of two point clouds have so far been assigned to two categories. The first category includes algorithms based on a voting method wherein the parameter space is quantized with six degrees of freedom and an alignment of each dot triplet is used as the voting method in the parameter space. These methods are guaranteed to determine the correct orientation, however, these methods require that the point clouds over a large area or preferably even completely overlap. Furthermore, these methods are partly inefficient due to a complete case distinction and the high number of degrees of freedom. Above all, such methods suffer from the so-called "Curse of Dimensioning" problem. Even with the simplified problem, according to which there is only an X-Y displacement and a rotation angle θ to be determined (the Z axis being along the gravitational direction), these algorithms are still not suitable for registration of large point cloud data sets. The second category contains methods for calculating point correspondences, which are then used to estimate a transformation according to which the best possible assignment of the source points to the target points is determined. For point clouds with only a small overlap area, however, so-called feature matching, which is essential for the determination of correspondences, can lead to many incorrect correspondences, so that the alignment result is not reliably determined correctly. Furthermore, certain steps, such as the so-called keypoint detection and the so-called descriptor computation require a high computational effort, since the next adjacent points within a large, disordered point cloud must be determined.

The comparatively new approach of the Four-Point Congruent Set (4PCS) belongs to both of the above categories. It searches for matches in the feature sets of the two point clouds in sets of four points each. A single correspondence is sufficient to determine a transformation with six degrees of freedom. According to this approach, the alignment is determined by the Largest Common Point Set, or LCP, looking for the largest subset of the source point cloud which, after transformation, is a maximum distance δ away from the target. In other words, the solutions that maximize the overlap area are preferred. However, this is particularly inconvenient for records containing many similar passages of a scanned building.

Another, relatively new category of point cloud alignment algorithms involves probabilistic methods. In such probability methods, no exact correspondences are calculated, but "soft" relationships between pairs of points are described by means of probability values.

An example of a probabilistic alignment technique is described by B. Jian and BC Vemuri in the article "Robust point set registration using Gaussian mixture models," which is described in Pattern analysis and machine intelligence, IEEE Transactions on 33 (8) (2011), pages 1633-1648 was published and GMMREG is named here. According to this method, a so-called Gaussian Mixture Model (GMM) is adapted to the source or target point cloud. The alignment problem is considered as a minimization problem of the histogram distances, with the GMM model of the source on the Transformation parameter is parameterized. Then GMMREG uses an iterative approach to determine the transformation parameters.

Another example of the probabilistic orientation is this "Coherent Point Drift" by A. Myronenko and X. Song described in "Point set registration: Coherent point drift," and published in Pattern analysis and machine intelligence, IEEE Transactions on 32 (12) (2010), pages 2262-2275 , This method also uses GMM, however, the GMM model is adjusted only to the target point cloud, with each target point representing one component of the mixture. Each source point is considered a sample of the GMM model with a conditional probability described by a Gaussian probability density function. Transformation parameters are estimated using a Maximum Likelihood (ML) using the Expectation Maximization algorithm (EM).

Probabilistic approaches have the advantage that so-called "outliers" are explicitly taken into account. Furthermore, the probabilistic approaches offer a great deal of flexibility in various types of transformations, such as mirroring, affine transformations, and scaling. However, this flexibility with respect to the transformation types offers no advantage in rigid body transformations in two- or three-dimensional space. Furthermore, the iterative nature of these methods makes them not quite insensitive to initial alignment. In addition, these methods have a high computational complexity, so that the calculations are sometimes very time-consuming, especially for larger data sets.

From the foregoing discussion, it can be seen that the effectiveness of the known methods is compromised when used for the alignment of point clouds with a smaller overlap area. Furthermore, some known approaches to the initial alignment of the point clouds are sensitive, so that their results do not reliably indicate the correct orientation. Finally, many known algorithms are not applicable to very large point clouds unless there is a drastic scan of the points in a preprocessing step. However, such a scan is not desirable because it will lose important information about the geometry (shape) of the parts of the building.

Presentation of the invention

Starting from the known state of the art, it is an object of the present invention to provide an improved method for determining alignment of a source point cloud relative to a target point cloud.

The above object is achieved by a method described herein for determining an alignment of a source point cloud relative to a target point cloud, wherein the source and the target point cloud each represent a partial model of interiors of a building structure and each aligned with respect to a gravitational axis. In a first step of the method, a source signature histogram of the source point cloud is cross-correlated with a target signature histogram of the target point cloud to determine rotation of the source point cloud. In a second step of the method, a translation histogram is calculated based on pairs of points, wherein each pair of points comprises a source point from the source point cloud and a target point from the target point cloud to determine a displacement of the source point cloud.

In this case, the determination of the correct rotation and displacement (hereinafter also referred to as rotation and displacement values) with respect to the initial alignment of the source and target point cloud is insensitive. Insofar as the source and target are already aligned along the gravitational axis, the method will give a correct orientation regardless of the initial rotation angle between source and target and independent of the distance between the point clouds. In particular, the point clouds may initially be in the same room. Further, according to the method, alignment of large point clouds with millions of points can be calculated efficiently. These benefits are made possible by the adoption of insights into interior spaces. Furthermore, a relaxation of the rigid coupling used in previous methods between the determination of the translation and the rotation is exploited. This partial decoupling of the rotation evaluation from the translation evaluation leads to the complexity of the overall evaluation being considerably reduced. Finally, the proposed approach has the advantage that point clouds that have a small overlap area, with, for example, less than 30% overlap between source and target, and even point clouds with a particularly small overlap area, with, for example, less than 10% overlap, can be aligned with the method ,

The method can be carried out with a scanning device. By way of example, a scanning device may be a wheeled or mounted substructure, backpack-mounted scanning devices and / or hand-held scanners. A scanning device may include a computing device, such as a microcontroller or a computer for carrying out the method. Furthermore, a scanning device can have optical and / or electronic components for scanning an interior of a building. Alternatively, the method may be performed with a stationary computing device, such as a desktop or laptop computer.

For the calculation of the source signature histogram, a source signature can be formed with a set of surface normal vectors, wherein each surface normal vector of the source signature is associated with a respective source point. Similarly, for the calculation of the target signature histogram, a target signature may be formed with a set of surface normal vectors, each surface normal vector of the target signature being associated with a target point, respectively. By forming source or target signatures each from a set of surface normal vectors respectively associated with source points and target points, the distinctness of the rotational orientation of the submodels is increased, so that the correct orientation rotation of the source point cloud can be reliably determined.

Furthermore, the source signature and target signature can each be calculated only on the basis of those points whose respectively associated surface normal vector is aligned substantially parallel to a ground plane. For example, the ground plane runs along or parallel to a floor of the building to be scanned. Here and below, mathematical terms such as "parallel" and / or "vertical" are not to be interpreted in the mathematically strict sense, but in particular within the scope of the measurement or alignment tolerances. For example, two planes whose surface normal vectors include an angle of at most or at least 5 °, preferably at most or at least 3 ° and particularly preferably at most or at least 1 °, can be regarded as parallel or perpendicular to each other. The reduction in the size of the signatures reduces the complexity of calculating the rotation, so that alignment of the point clouds can be performed with fewer (arithmetic) operations. At the same time, the calculation time is reduced.

A result of cross-correlating the source signature histogram with the target signature histogram may include a number of peaks that coincide with a number of dominant directions of the interiors of the building structure. The dominant directions may be dictated by the orientation of the walls of the interiors, for example. In this case, a further step of calculating a rotation-dependent translation histogram for each dominant direction may be provided. Further, a step of comparing the calculated rotation-related translation histograms may be provided to determine the rotation of the source point cloud. This ensures that the correct rotation of the source point cloud is determined, regardless of the initial orientation of the point clouds. In particular, it is ensured that in the case of building structures that exhibit a pronounced symmetry between the dominant directions, no false rotation is chosen, which corresponds to one of the usually three other "wrong" dominant directions.

The comparing of the calculated rotation-related translation histograms may comprise a step of comparing the maximum values of each of the rotation-related translation histograms. Further, the comparing may comprise a step of selecting the rotational translation translational histogram having the largest maximum value, the associated rotation forming the rotation of the source point cloud. The determination of the rotation of the source point cloud allows an invariance with respect to the initial orientation, since the corresponding pairs of points corresponding to the incorrect rotations are not assigned to the same class. Therefore, the maximum values of rotation-dependent translational histograms serve as another reliable indicator of correct rotation.

In the calculation of the translation histogram, primitive properties of the source or target points can be used, such as their respective surface normal vector, surface curvature index and / or Z value, to allow discriminating class assignment, as described in more detail below.

In particular, the pairs of points used to calculate the translation histogram may only include those source points and target points whose respective associated surface normal vectors differ substantially from each of the dominant directions of the interiors of the building structure. Thus, the number of points taken into account in the translation histogram can be drastically reduced be reduced, since all points that lie on inner surfaces such as interior walls, as well as all points that lie on the edges between wall and floor or between wall and ceiling, must not be taken into account in the calculation.

Alternatively or additionally, the pairs of points used to calculate the translation histogram can only include those source points and target points whose respective associated surface curvature index has a high absolute value. Thus, the number of points taken into account in the translation histogram can be drastically reduced since all points located on a flat interior surface, such as a wall, floor and / or ceiling, need not be taken into account in the calculation. This measure not only reduces the computational effort, but also significantly contributes to the calculation of better translation histograms.

Alternatively or additionally, the pairs of points used to calculate the translation histogram may only include those pairs of points that comprise a source point and a target point whose associated normal normal vectors are substantially parallel to each other. Such a selection can be justified by the fact that the rotational alignment between the source and target point clouds has already been calculated so that the surface normal vectors of corresponding source and target points must also have corresponding directions. Thus, the number of points taken into account in the translation histogram can be drastically reduced since only those pairs of points whose respective surface normal vectors are substantially parallel to one another must be taken into account in the calculation.

The source point cloud and target point cloud may each be divided into slices that include all source points that are within a threshold of a reference plane parallel to the ground plane. The reference plane can be arranged at a distance from the ground plane and in particular from the ground. Furthermore, it is possible that the reference plane is arranged at a distance from a ceiling plane running along the ceiling. It is possible to compute two or more slice-related translation histograms, where the calculation is based on pairs of points that are in the same slice. Finally, the two or more slice-related translation histograms can be summed up to form the translation histogram. The discs may be arranged adjacent to each other. Furthermore, all disks can have the same thickness. Alternatively, the discs may at least partially have different thicknesses. Thus, the complexity of calculating each of the slice-related translation histograms is significantly reduced, so that the summed translation histogram can be calculated as a result with fewer operations. At the same time, the calculation time is reduced.

Each of the above-listed discriminatory class assignment measures individually contributes to efficiently calculating the translation histogram. Thus, the orientation shift of the source point cloud with respect to the target point cloud can be performed in fewer steps. At the same time, the calculation time is reduced.

If the source and / or target point cloud are each not already aligned with respect to the gravitational axis in an initial orientation, they can be aligned in a preprocessing step with respect to the gravitational axis. Accordingly, a rotation of the source and / or target point cloud for orientation with respect to the axis of gravity takes place beforehand by means of a principal component analysis. Subsequently, an adaptation of the local Z coordinates of the source point cloud is carried out on the basis of a comparison result between the ground level of the target point cloud and the ground level of the source point cloud. Thus, interiors stored in any conceivable initial orientation can be efficiently pre-aligned along the gravitational axis and subsequently aligned with each other, so to speak in a main alignment step, to create a comprehensive interior model.

According to this embodiment, the rotation step after the main component analysis may comprise a further adjustment step. In this case, all points of the source or target point cloud are selected with a surface normal pointing essentially along the gravitational axis. Subsequently, the median value of the surface normals of these points is determined and a fine rotation is performed so that the median value corresponds to an adjusted gravitational axis.

After determining the rotation and displacement values that can be used to align the source point cloud with respect to the target point cloud, the method may include a further step of aligning the source point cloud relative to the target point cloud. In this case, the points of the source point cloud are aligned on the basis of the determined rotation and the determined displacement.

The method may include a further step of applying a fine-alignment algorithm, such as the Iterative Closest Point (ICP) algorithm. After a coarse alignment of the point clouds comprising the rotation and the displacement has been determined, a fine alignment of the source point cloud relative to the target point cloud is calculated and applied to all points of the source point cloud.

Brief description of the figures

Preferred further embodiments of the invention are explained in more detail by the following description of the figures. Showing:

1 a perspective view of a source point cloud and a target point cloud, which have not yet been aligned with respect to each other.

2A a perspective view of the point clouds according to 1 that have been aligned to provide a comprehensive representation of the building structure underlying the point clouds;

2 B another perspective view of the point clouds according to 1 that have been aligned along one of the "wrong" dominant directions (not according to the invention), so that no comprehensive representation of the building structure underlying the point clouds is granted;

3 a perspective view of a simplified embodiment with a source point cloud and a target point cloud in an initial orientation;

3A one on 3 based, modified embodiment in which the source and target point clouds have an overlapping region comprising a three-dimensional volume, which should not be or at least not completely filled with building structures;

4 a perspective view of in 3 illustrated embodiment after rotation alignment;

4A a cross section of in 3 illustrated embodiment with the associated surface normal vectors of the source and target points;

5A a graph of a source signature histogram of an example source signature overlaid with a target signature histogram of an example target signature;

5B a graph of the cyclic cross-correlation of the source signature histogram with the target signature histogram, respectively according to 5A ;

6 a perspective view of in 3 illustrated embodiment after translation alignment;

7A to 7C a pictorial explanation of the calculation of a translation histogram in the X direction according to a simplified example, based on source / target point pairs;

8th a part of a point cloud, by means of which structures in the building interior are recorded or reproduced; and

9 a flowchart of the steps of an embodiment, according to which the source and target point clouds initially arbitrary, ie not aligned with respect to the global axis of gravitation.

Detailed description of preferred embodiments

In the following, preferred embodiments will be described with reference to the figures. In this case, identical, similar or equivalent elements in the different figures are provided with identical reference numerals, and a repeated description of these elements is partially omitted in order to avoid redundancies.

Where applicable, all individual features illustrated in the embodiments may be combined and / or interchanged without departing from the scope of the invention.

1 shows a first embodiment of two large point clouds, namely a target point cloud 100 and a source point cloud 200 , The point clouds 100 . 200 form a discrete representation of an interior of a building structure 10 , also called 'building', wherein the interior at least partially by the target point cloud 100 and at least partially through the source point cloud 200 is modeled. It represents the target point cloud 100 and the source point cloud 200 usually different parts of the interior of the building structure 10 But it is also possible that the target point cloud 100 and the source point cloud 200 the same part of the building structure 10 in the event that this part is to be rescanned or updated.

The target and source point cloud 100 . 200 are originally aligned with respect to a gravitational axis (Z-direction), but not yet in relation to each other. In fact, the target and source point clouds can 100 . 200 initially placed in any conceivable arrangement, as long as they are aligned with respect to the same Z-axis. The point clouds can 100 . 200 initially different rooms, or fill but the same space, ie be arranged overlapping. In this case, the point clouds are 100 . 200 Although overlapping, but not aligned with respect to each other, so that their composition is not as a representation of the interior of the building structure 10 can be understood, which would be desirable.

The point clouds 100 . 200 have only a small overlap area, which with the region 110 the target point cloud 100 or the region 210 the source point cloud 200 roughly indicated.

2A shows the result of correct alignment between the target and source point clouds 100 . 200 , In particular, it is shown that the source point cloud 200 with respect to the stationary target point cloud 200 was rotated and shifted, so the composition of the point clouds 100 . 200 all recorded forms in the interior of the building structure 10 accurately reproduces. In the process, the overlapping regions became 110 . 210 aligned with each other so that in each region 110 . 210 at least some target points of the target point cloud 100 essentially overlapping with corresponding source points of the source point cloud 200 are arranged.

In a second, simplified embodiment according to 3 are the target point cloud 100 as well as the source point cloud 200 as interiors of a building structure 10 represented, which were discretized by means of a 3D lattice. As in the first embodiment, the target and source point clouds are 100 . 200 Already aligned with respect to a local Z-axis, ie they are aligned with respect to the axis of gravity 17 the earth, perpendicular to a ground plane 15 with a Z value equal to zero. In this simplified embodiment, the point clouds include 100 . 200 a plurality of points arranged in a rectangular grid, which together form an outer surface of the modeled interiors. Further, dots may be disposed along the outer edges as well as on each of the outer corners of the modeled interiors.

Within these point clouds 100 . 200 (in the 3 not shown), various adjoining structures may be arranged. As in 8th Both floor and ceiling surfaces can be shown essentially parallel to the floor level 15 arranged, as well as wall surfaces 20 which is essentially perpendicular to the ground plane 15 be arranged. Furthermore, columns, beams or other internal structures 22 be present in every conceivable orientation. Similar to the outer structures of the point clouds 100 . 200 , the structures become at the inside of the point clouds 100 . 200 modeled by points, for example, arranged in a grid on the surface of said structures. Furthermore, points may be arranged along the edges of the inner structures as well as at corners or intersections of two or more inner structures. For example, the empty spaces between the structures modeled with dots represent the rooms, halls and corridors of the building 10 represents.

Out 3 it can be seen that the source point cloud 200 with respect to the target point cloud 100 originally not aligned. Further, the point clouds 100 . 200 characterized in this idealized embodiment by a small overlap area. In particular, the target point cloud has 100 a section 105 with a cutting edge 110 on. Similarly, the source point cloud points 200 a gearing 205 with a gearing edge 210 on, with the cutout 105 and the gearing 205 are formed so that they engage in a correct orientation. In this embodiment, the cut edge 110 and the gearing edge 210 each complementary levels, each containing the interior of the building structure 10 cut through and essentially perpendicular to the ground plane 15 are aligned.

When the point clouds 100 . 200 as in the illustration in 6 aligned with each other, those on the toothed edge 210 arranged sources with the on the cutting edge 110 substantially overlap arranged target points. In other words: the cutting edge 110 of the interlocking clipping 105 has several contiguous levels, with multiple contiguous levels of the gearing edge 210 the interlocking interlocking 205 correspond. Thus, those source points are aligned with respect to corresponding target points, each on different inner and outer surfaces of the through the point clouds 100 . 200 discretized interior of the building 10 are arranged. In particular, those source points are aligned with respect to corresponding target points, each on different structures of the point clouds 100 . 200 are arranged, which coincide with the respective point cloud edges 110 . 210 overlap.

Unlike the in 3 illustrated, idealized embodiment, it is possible that the overlapping region of the source and the target point cloud 100 . 200 is not limited to an edge region consisting of adjoining planes, but rather may comprise a volume in three-dimensional space. In the standard case, the overlapping area comprises one volume 110 once as part of the target point cloud 100 was scanned, with the same volume 210 also as part of the source point cloud 200 was scanned once.

This is off 3A it can be seen which an overlapping region 110 the target point cloud 100 as a three-dimensional volume whose upper surface is indicated by a dashed line. The volume corresponds to this 110 to an overlapping region 210 the source point cloud 200 , in the form of toothing 205 , In other words: the volume 110 represents a region of the building interior that once as part of the target point cloud 100 was scanned, with the same volume once during generation of the source point cloud 200 was scanned, and indeed around the gearing 205 to build. When the source and the target point clouds 100 . 200 as shown in 6 are aligned on the overlapping region 210 arranged points of the source point cloud 200 with respect to corresponding points of the target point cloud 100 that in the volume 110 are arranged to be aligned.

Unlike in the 3 and 3A illustrated, idealized embodiments, it is possible that the overlapping regions 110 . 210 are generally not confined to a single area or to a single volume of building interiors. Rather, the overlapping regions 110 . 210 comprise two or more separated areas within the point clouds, each corresponding to each other. Similarly, the overlapping regions 110 . 210 include two or more separate volumes in three-dimensional space, within which the source and target point clouds 100 . 200 overlap with each other after alignment. Generally speaking, the overlapping regions 110 . 210 comprise a combination of one or more overlapping levels and one or more overlapping volumes.

The first step in aligning the source point cloud 200 with respect to the target point cloud 100 includes the determination of a rotation angle of the source point cloud 200 with respect to the target point cloud 200 , Incidentally, the result of correct rotational alignment of the second embodiment is shown in FIG 4 shown.

To clarify how the correct rotation orientation of the source point cloud 200 in relation to the target point cloud 100 is determined, shows 4A one parallel to the ground level 15 oriented cross-section of the source and target point clouds 100 . 200 , The outlines of the through the source and the target point cloud 100 . 200 modeled interior 10 are marked with dashed lines. It should be noted that in 4A an example is shown that only the points on the outer edge of the point clouds 100 . 200 and on their inner, overlapping edges 110 . 210 shows. However, the fewest building interiors only consist of an outer edge without any internal structures. In practice, the point clouds include 100 . 200 several points, not only at the outer edge of the point clouds, but also along the inner structures of the point clouds 100 . 200 modeled interior 10 are arranged, for example, to internal structures of the building 10 like interior walls, floors, ceilings, columns or beams.

The determination of the rotational orientation is based on a primitive, point-related property, namely the surface normal, also called the surface normal vector. After points belonging to a point cloud on different structures in the interior of the structure represented by the point clouds 10 are arranged, each point is associated with one or more surfaces of the respective structure or structures. With respect to each surface, those surface normals associated with the individual points arranged on the respective surface can be calculated. The surface normal vector associated with a point can thus be calculated by combining the surface normals of all surfaces associated with the point.

In the in the 4A illustrated simplified embodiment are arranged in the cross section points of the point clouds 100 . 200 each represented with their associated surface normal vectors, shown here in two dimensions. The normal vectors in 4A can in 2D be shown because the walls of the in the 3 . 4 and 6 shown embodiments are aligned substantially transverse to the ground plane, so that the normal vectors of the walls of the building 10 completely in the plane parallel to the ground plane cross-sectional plane according to 4A lie.

For point clouds representing submodels of structures with the sub-models already aligned with respect to the axis of gravity, the walls represent the dominant structures of the interiors so that the orientation of the walls can be considered as a reliable reference with respect to rotational orientation. To the rotation orientation of the source point cloud 200 First, those points of the source point cloud are to be determined 200 and the target point cloud 100 which are lying on a wall with a sufficient probability because their respective surface normals _nk = [ _nk (x), _nk (y), _nk (z)] ^T are substantially parallel to the ground plane 15 aligned, or in other words because its abs (n _k (z)) ≈ 0. Thus, only those points are selected whose associated surface normal vector is almost parallel to the ground plane. In the following, only relevant points are considered.

Next, for each relevant point k, the angle φ _k in the ground plane is calculated as follows: Φ _k = atan (n _k (y) / n _k (x)).

Accordingly, a source signature of the source point cloud is formed from the angles φ _{k of} those source points having a sufficiently small value abs (n _k (z)). Similarly, a target signature of the target point cloud is formed from the angles φ _{k of} those target points having a sufficiently small value abs (n _k (z)).

In the in 4A In the example shown, all the normal vectors shown in the figure are taken into the source signature or target signature because they are all substantially parallel to the ground plane 15 are aligned. This is especially true only because the in 4A points all on essentially parallel to the axis of gravity 17 aligned wall structures are arranged so that the surface normal vectors of these points substantially parallel to the ground plane 15 are aligned. But this would not be the case for points on the lower or upper surfaces of the 3 and 3A arranged square point clouds are arranged. Namely, such points are located on a floor or ceiling structure of the modeled interiors, and are used in determining the rotation of the source point cloud 200 not considered.

Next, a source signature histogram of the source signature is calculated from the calculated angles φ _k with a bin size of 0.5 ° over the entire frequency range of 360 °, including all possible rotations in the XY plane. The result is a source signature histogram h _s with 360 x 0.5 = 720 classes. This step is repeated for the target point cloud target signature to calculate a target signature histogram h _t . A histogram, in particular the source signature histogram and / or the target signature histogram, can be a frequency distribution here. The results of these calculations (applied to another dataset, ie not in the 3 . 4 and 6 illustrated embodiments) are in 5A shown. Should greater accuracy be required, the so-called Kernel Density Estimation (KDE) method can be used to achieve a finer subdivision of the classes.

Thus, the area normal vectors of the relevant points become the target point cloud 100 or the source point cloud 200 used to create a signature histogram as in each case 5A shown to form. Then, the source and target signature histograms h _s , h _{t are} cyclically cross-correlated with each other. The result is a rotation-dependent signal, as in 5B whose maximum value is the correct rotation θ ^ of the source point cloud 200 around the gravitational axis or Z-axis 17 features. With the application of this result, the points of the source point cloud become 200 around the gravitational axis 17 rotated by the angle θ ^.

To the source point cloud 200 in relation to the target point cloud 100 to align, in the following step, a shift of the source point cloud 200 determined in the X and Y directions. The result of correct translation alignment of the second embodiment is in 6 shown.

After the rotation has already been calculated, the displacement or so-called translation in the X and Y directions is constant for corresponding point pairs (ie points in the source and target that represent the same physical point). The correct translation of the source point cloud 200 along the X or Y axis can therefore be determined by means of a bivariate translation histogram (also called displacement histogram). The translation histogram may include displacement values in the X and Y directions for all source / target point pairs, with one source / target point pair being exactly one point from the source point cloud 200 and exactly one point from the target point cloud 100 includes. The maximum value among the histogram classes characterizes the correct shift, because in the class corresponding to the correct shift, as many hits accumulate as there are actually corresponding source / target pairs of points. By contrast, all non-corresponding pairs of points are distributed over the entire displacement space and thus quite unlikely to gather in a particular, incorrect class.

The calculation of the translation histogram of the pairs of points is in the 7A to 7C shown. It shows 7A like a shift of a selected source point 200E the source point cloud 200 to different target points 100A . 100B . 100C . 100D and 100E is calculated and the respective shifts of a class of the histogram 300 be assigned. This step is for the source points 200D . 200B repeated, like that 7B and 7C demonstrate. The shifts between corresponding pairs of points, which are in 7 marked with solid lines of the same class 302 assigned, so that this class is a maximum or maximum. A normalized translational histogram of the pairs of points represents a probability density of the correct displacement 7A to 7C The example shown thus represents the class 302 the most likely X shift. It should be noted that the example according to 7 represents only a translation histogram of the X-shift. In practice, a bivariate translational histogram with respect to both Directions, namely X and Y, calculated according to this principle.

This approach has some advantages over known algorithms based on a voting technique. After the rotation alignment has already been determined in a previous step, it is no longer necessary to evaluate all point-triplet pairs of the source and target point clouds for the determination of the translation orientation, resulting in a computation complexity of | S | ³ · | T | ³ operations, where S and T represent respectively the source and target point sets. Instead, the decoupling of the rotation determination from the displacement determination means that each individual source / target pair of points can 'choose' for itself so that a maximum of | S | · | T | Calculation steps are required.

To further reduce computational complexity while increasing the distinctiveness of the translation histogram, there are some additional measures that can be used alone or in combination. In particular, according to one embodiment, not all points of the target point cloud become 100 or the source point cloud 200 taken into consideration. In other words, certain points in the calculation of the translation histogram may be disregarded to make the voting procedure discriminative. In this case, the translation histogram is formed only on the basis of those source / target pairs whose points individually meet for themselves and / or in combination with certain criteria. These measures, called common discriminative voting, are explained in more detail below.

In contrast to the rotation determination, the corner points are more important for the determination of the translation than those points which are located on interior walls. This can be explained by the fact that an ambiguity arises when a source point is compared with a plurality of target points arranged on a wall. As a first measure for reducing the computation complexity in the translation determination, it is therefore possible to filter out all points with too low a curvature value on the basis of the surface curvature index (which arises anyway in the surface normal calculation). In other words, all points lying on essentially flat planes are not taken into account in the calculation of the translation histogram, so that only those points which have a high curvature value, for example because they lie on a corner or edge, are included. Thus, the number of points taken into account in the translation histogram can be drastically reduced.

Points with a high curvature value are not only corner points, but also points that lie on the edges between inner walls and ceilings or between inner walls and floors. These points can lead to spurious peaks in the translation histogram. Thus, the method can be improved by using surface normals which enable the two further selection methods described below: Firstly, only those points whose surface normals essentially differ from each of the four dominant directions of the inner walls may be exposed to source / target Make up a couple. Thus, the points on an edge between a wall surface and a ceiling surface or on an edge between a wall surface and a bottom surface are also eliminated, because at these points the surface normal resembles one of the four dominant directions too much. Second, only those source / target pairs of points may be considered whose associated surface normals have a small V-angle, for example less than 45 °, 30 °, 10 °, 5 °, 2 °, 1 °, 0.5 ° or 0.1 °. In other words, point pairs may only be formed if their surface normals are essentially parallel to one another. These selection mechanisms lead to a dramatic improvement in the translation histogram, with a corresponding increase in the maximum value at the correct X, Y shift values and thus an improved uniqueness of the result.

In order to make the voting process even more discriminative, another primitive point feature can be used, namely the height value or the Z coordinate. After the source and target point clouds are already aligned with respect to the gravitational axis, corresponding points of the source and target point clouds will have similar height values. It may be advantageous to incorporate an error tolerance for the Z-values of the pairs of points, since, for example, ramps, such as wheelchair ramps, in the interior of the building structure 10 can lead to the rooms of a floor not being exactly at the same height. The source and target point clouds 200 . 100 are each divided into slices, all of which include source points located within a threshold of a reference plane parallel to the ground plane. Two points can only form a pair of points if they are in the same disk. This is the case if their Z-values differ by less than a certain fault tolerance, which essentially corresponds to the maximum height of any ramps present in the interior. For each slice, an associated slice-referenced translation histogram is calculated, with the calculations in a multi-kernel system running concurrently or simultaneously can be done. After calculating the disc-associated translational histograms associated with each disc, they can be summed to form the translation histogram. Thus, the efficiency of calculating the translation histogram can be increased.

The discriminative selection methods described above can be used singly or in combination in the calculation of the translation histogram. Using all these discriminative voting techniques, a reduction in computation time of over 95% and an increase in the quality of the resulting translation histogram can be achieved.

Since the interior walls of most building structures are oriented transversely in four dominant directions, the result of the cross-correlation of the source and target signature histograms normally has four peaks, e.g. 5B you can see. In other words, for most structures, there is some ambiguity at multiples of 90 ° in calculating the rotational orientation of the source point cloud relative to the target point cloud. Like in the 2 B This may result in a rotational orientation being judged to be "correct" by an alignment procedure, even though it belongs to one of the "wrong" dominant directions and thus is false. In the case of a wrong rotation orientation, in the subsequent calculation of the translation histogram, all actually corresponding pairs of points would not be assigned to the same class. As a result, translational histograms, which are calculated from one of the three 'wrong' dominant directions, will not have as high a peak as the 'right' dominant direction.

Therefore, another embodiment will be described, which is particularly suitable in the alignment of interior models of buildings 10 whose cross-correlation of the source and target signature histograms has four similarly sized peaks. According to this exemplary embodiment, four rotation-dependent translational histograms are calculated following the rotation determination, namely one rotation each, which is marked with a peak and thus corresponds to one of the four dominant directions. The maximum values of the four rotation-related translation histograms are then compared, and the associated rotation is identified as orientation rotation of the source point cloud. Further, from the rotation-related translation histogram corresponding to the proper dominant direction, the X and Y coordinates of the peak representing the shift values for the alignment of the source point cloud can be read out.

Should the building have more or fewer than four dominant directions such that the cross-correlation of the source and target signature histograms yields a number of more or less than four peaks, the above embodiment can be adapted to compute as many rotational translational histograms as that Result of cross-correlation peaks (ie, how the number of dominant directions of the building).

According to another embodiment, two point clouds may have any initial orientation, ie, not initially along the axis of gravitation 17 are aligned, aligned and joined together. As in 9 are shown in a preprocessing step 530 the source and target point clouds are each aligned along the axis of gravity. This is accomplished by performing a principal component analysis on each of the point clouds 200 . 100 is carried out. Assuming that the sub-models of the building interior have comparatively flat profiles, ie that the modeled building interior extends farther in the X and Y directions than in the Z direction, the third main component of the principal component analysis can be identified as a gravitational direction with reasonable computation effort , It then takes place in advance in each case one rotation 510 the source and the target point cloud for alignment with respect to the axis of gravity 17 ,

In another part 520 the preprocessing 530 the local Z-coordinates of the source point cloud are adjusted. In particular, those points are selected that are not on a wall. In other words, all points whose surface normal has a low Z component, ie, where abs (n _k (z)) ≈ 0, are disregarded in this process. From the remaining points, ie all points that lie on a floor or ceiling surface, a histogram is formed on the basis of their Z coordinates for the source and for the target point cloud. Finally, the histograms of the source and target point clouds are cross-correlated with each other to shift the source point cloud 200 in the Z direction. The source point cloud is accordingly shifted along its Z-axis so that the local Z-axis of the source point cloud corresponds to the local Z-axis of the target point cloud.

Upon completion of the preprocessing step described above, the source and target point clouds may be aligned according to any of the above-described embodiments, which assumes a common local axis of gravity of the point clouds. In particular, a determination of the rotation then takes place 610 around the Z axis 17 and a determination of the displacement 620 into the XY Directions according to one of the embodiments described above.

In a further embodiment, the rotation step comprises 510 after the main component analysis a further adjustment step. In this case, all points of the source or target point cloud are selected, which lie on a floor or ceiling surface of the modeled interior. This is achieved by selecting only those points with a surface normal pointing essentially along the gravitational axis. A surface normal is considered to be pointing essentially along the gravitational axis if it makes an angle of less than 15 °, 10 °, 7.5 °, 5 °, 4 °, 3 °, 2 °, 1 with the already determined Z-axis °, or 0.5 ° forms. From the surface normals of the selected points a median value is determined which points to the exact axis of gravity. A further fine rotation of the source and target point clouds can be carried out accordingly.

In the embodiments described here with two point clouds discretized by means of rectangular grids 100 . 200 which the outer contours and / or interiors of the building structure 10 represent the discretization of point clouds 100 . 200 be done differently, for example by means of a non-rectangular grid, a triangular grid, a randomized, regular distribution of the points, a three-dimensional regular or irregular grid structure or any other suitable discretization.

LIST OF REFERENCE NUMBERS

10

Building structure or structure

15

Ground level (XY level)

17

Gravity axis (Z-axis)

20

wall surface

22

modeled structures in the building interior

100

Target point cloud

105

interlocking clipping

110

overlapping region of the target

200

Source point cloud

205

interlocking interlocking

210

overlapping region of the source

300

Translation histogram

302

most likely X-shift

500

Entry at point clouds not yet aligned with respect to the gravitational axis

510

Rotation orientation with respect to Z-axis

520

bottom alignment

530

preprocessing

600

Entry into point clouds already aligned with respect to the gravitational axis

610

Determination of the rotation of the source point cloud with respect to the target point cloud

620

Determination of the shift of the source point cloud with respect to the target point cloud

QUOTES INCLUDE IN THE DESCRIPTION

This list of the documents listed by the applicant has been generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Cited non-patent literature

• Akbarzadeh et al. "Towards Urban 3D Reconstruction from Video" at Third International Symposium on 3D Data Processing, Visualization, and Transmission; 2006 [0008]
• Agarwal et al. "Building Rome in a Day," Communications of the ACM, Vol. 10, pages 105-112 [0008]
• IEEE Transactions on 33 (8) (2011), pages 1633-1648 [0015]
• "Coherent Point Drift" by A. Myronenko and X. Song described in "Point set registration: Coherent point drift," and published in pattern analysis and machine intelligence, IEEE Transactions on 32 (12) (2010), pages 2262-2275 [ 0016]

Claims (14)

Method for determining an alignment of a source point cloud ( 200 ) relative to a target point cloud ( 100 ), the source and target point clouds ( 200 . 100 ) partial models of interiors of a building structure ( 10 ) and in each case with respect to a gravitational axis ( 17 ), with the steps: cross-correlation ( 5B ) of a source signature histogram ( 5A ) the source point cloud ( 200 ) with a target signature histogram ( 5A ) of the target point cloud ( 100 ) to determine a rotation of the source point cloud ( 610 ); and calculating a translation histogram ( 300 ) based on pairs of points ( 100A -E, 200A -E), wherein each pair of points comprises a source point from the source point cloud and a target point from the target point cloud, in order to determine a displacement of the source point cloud (FIG. 200 ) to investigate ( 620 ). The method of claim 1, wherein for the calculation of the source signature histogram, a source signature is formed with a set of surface normal vectors each associated with a source point, and for the calculation of the target signature histogram a target signature is formed with a set of surface normal vectors each associated with a target point , Method according to claim 2, wherein the source signature and target signature are each calculated only on the basis of those points whose respectively associated surface normal vector is aligned substantially parallel to a ground plane. Method according to one of the preceding claims, wherein a result of the cross-correlation ( 5B ) has a number of peaks associated with a number of dominant directions of the building structure ( 10 ), comprising the further steps of: calculating a rotation-related translation histogram for each rotation corresponding to one of the peaks, and comparing the calculated rotation-related translation histograms to determine the rotation of the source point cloud. The method of claim 4, wherein comparing the calculated rotation-related translation histograms further comprises: Comparing the maximum values of each of the rotation-related translation histograms with each other and Select the rotational translation translational histogram that has the largest maximum value so that the associated rotation forms the rotation of the source point cloud. Method according to claim 4 or 5, wherein the pairs of points used to calculate the translation histogram comprise only those source points and target points whose respective surface normal vector differs substantially from each of the dominant directions. Method according to one of the preceding claims, wherein the pairs of points used to calculate the translation histogram comprise only those source points and target points whose respective associated surface curvature index has a high absolute value. Method according to one of the preceding claims, wherein the pairs of points used to calculate the translation histogram comprise only those pairs of points which comprise a source point and a target point, whose respectively associated surface normal vectors extend substantially parallel to one another. The method of claim 1, wherein the source point cloud and the target point cloud are each divided into slices that include all source points that are within a threshold distance from a reference plane parallel to the ground plane, wherein two or more slice-related translation histograms are based on Point pairs that are in the same slice can be calculated and the two or more slice-related translation histograms are finally summed to form the translation histogram. A method according to claim 9, wherein the discs are arranged adjacent to each other and / or wherein all discs have the same thickness. Method according to one of the preceding claims, wherein the source and / or target point cloud in each case in an initial orientation is not yet oriented with respect to the gravitational axis, with a preprocessing step ( 530 ) for aligning the point clouds with respect to the axis of gravitation, comprising: rotation ( 510 ) of the source and / or target point cloud for alignment with respect to the axis of gravity using a principal component analysis, and fitting ( 520 ) of the local Z coordinates of the source point cloud based on a comparison result between the ground level of the target point cloud with the ground level of the source point cloud. The method of claim 11, wherein the rotation step ( 510 ) comprises, according to the principal component analysis, a further adjustment step, wherein all points of the source or target point cloud have a substantially along the Gravity axis pointing surface normal are selected, and wherein a fine rotation is performed so that the median value of this surface normal corresponds to an adjusted gravitational axis. Method according to one of the preceding claims, with a further step of aligning the source point cloud in relation to the target point cloud, wherein the source points of the source point cloud are aligned on the basis of the determined rotation and the determined displacement. Method according to one of the preceding claims, comprising a further step of using an algorithm, for example the Iterative Closest Point (ICP) algorithm, for determining the fine alignment of the source point cloud relative to the target point cloud after a coarse alignment of the point clouds comprising the rotation and the displacement was determined.

Images