WO2010042466A1

WO2010042466A1 - Apparatus and method for classifying point cloud data based on principal axes

Info

Publication number: WO2010042466A1
Application number: PCT/US2009/059619
Authority: WO
Inventors: Kevin Scott Williams; Lesa Suzanne Williams; Jim Baber Williams
Original assignee: Kevin Scott Williams; Lesa Suzanne Williams; Jim Baber Williams
Priority date: 2008-10-06
Filing date: 2009-10-06
Publication date: 2010-04-15

Abstract

A method of extracting features from a point cloud and apparatus therefore are described. The method comprises extracting one or more surface patches from the point cloud and determining one or more principal axes of objects from the point cloud based on at least one of the one or more surface patches. The method further comprises performing, constrained by one of the one or more determined principal axes, a subsequent extraction of one or more surface patches.

Description

APPARATUS AND METHOD FOR CLASSIFYING POINT CLOUD DATA BASED ON PRINCIPAL AXES

BACKGROUND

[001] Civil and mechanical engineering projects, GIS (Geographical Information Systems) mapping programs, military simulations, and numerous other applications all require accurate three dimensional (3D) computer models of real-world objects.

[002] Most prior art methods for creating 3D models involve extensive manual measurement and modeling. The measuring component may be achieved either through direct measurement (such as surveying) of the objects themselves or through measuring images of the objects using the science of photogrammetry. The modeling component typically involves manually inputting the measurements into computer modeling programs such as computer-aided design (CAD) software, GIS, or other similar solid modeling packages. This process is labor intensive and error prone.

[003] Light Distancing and Ranging (LIDAR) is a relatively new technology for improving upon this 3D-modeling process. LIDAR systems scan objects or scenes using laser range-finding technology to construct a "point cloud" consisting of 3D point measurements of the scene. These points can then be used to guide the process of feature extraction, e.g., as set forth in Dimsdale, et al. U.S. Patent 7,184,036.

[004] The most common (and successful) prior art means for performing feature extraction from a point cloud is to manually partition the points that fall on a surface of interest into a single group (this process of partitioning points into groups of points that fall on a single surface is called "classification"). Several commercial software packages exist that allow a user to select a group of points using a variety of graphical user interface (GUI) interactions and then to assign the group of points to a single feature. The existent software typically allows a user to fit a parametric surface (such as a plane or cylinder) to the group of points. Finally, the user can create a coherent solid object model of the scanned scene by topologically connecting these surfaces. Though this process works quite well, it is extremely labor intensive and therefore cost- prohibitive for many modeling projects. [005] There have also been a number of attempts to automate this process by finding local regions of points belonging to a common geometric feature and then automatically fitting parametric surfaces to those points. One method that has been attempted, e.g., as set forth in Kacyra et al., U.S. Patent 6,473,079, involves the classical image-processing technique of using convolution-style edge filters over either the depth information or the normal information to find edges between adjacent surfaces in a LIDAR point cloud. Another method for classifying points by the surfaces they fall on is the use of random sample consensus (RANSAC), e.g., as set forth in M. A. Fischler and R. C. Bolles (June 1981), "Random Sample Consensus: A Paradigm for Model Fitting with Applications to Image Analysis and Automated Cartography". Comm. of the ACM 24: 381 — 395. Yet another method for classifying points is region-growing, e.g., as set forth in Franz Rottensteiner and Christian Briese: "A New Method for Building Extraction in Urban Areas from High-Resolution Lidar Data," Symposium der ISPRS-Comm. Ill, Graz, 9 - 13 September 2002, International Archives of Photogrammetry and Remote Sensing, Volume XXXIV / 3A, pp. 295 - 301, to find planar or curved regions. The Hough transform has also been applied to this problem of classifying surfaces within a point cloud as well, e.g., as set forth in Vosselman, G. and Dijkman, 2001, "3D Building Model Reconstruction from Point Clouds and Ground Plans," in Proceedings of the ISPRS Workshop on Land Surface Mapping.

[006] Unfortunately, all of these automated search methods encounter significant problems when attempting to classify 3D points in complex LIDAR "scenes" (a scene is a scan of an object or collection of objects). In the presence of noise or clutter, the aforementioned techniques will usually misclassify significant numbers of points. A more robust method for identifying features and classifying LIDAR point cloud data is needed in order for automated feature extraction to become an economically attractive alternative to manual feature extraction.

SUMMARY OF THE INVENTION

[007] The present embodiments provide a method of and apparatus for extracting features from a point cloud. [008] A method embodiment comprises extracting one or more surface patches from the point cloud and determining one or more principal axes of objects from the point cloud based on at least one of the one or more surface patches. The method further comprises performing, constrained by one of the one or more determined principal axes, a subsequent extraction of one or more surface patches. In one or more further embodiments, the method further comprises tagging extracted line segments according to whether or not the segment resulted from occlusion within the point cloud.

DESCRIPTION OF THE DRAWINGS

[009] One or more embodiments are illustrated by way of example, and not by limitation, in the figures of the accompanying drawings, wherein elements having the same reference numeral designations represent like elements throughout and wherein:

[0010] FIG. 1 is a high-level block diagram of a computer system usable in conjunction with an embodiment;

[0011] FIG. 2 is a high-level process flow diagram of an embodiment; [0012] FIG. 3 is a another high-level process flow diagram of an embodiment; [0013] FIG. 4 is a graphic depiction of analyses;

[0014] FIG. 5 is a graphic depiction of rotation and translation of orthogonal patches according to an embodiment; and

[0015] FIG. 6 is a graphic depiction of object occlusion and lack thereof;

DETAILED DESCRIPTION

[0016] One or more embodiments of the present invention apply to extracting features from point cloud data containing objects with one or more common axes. This characteristic is very typical amongst many man-made objects. For instance, a parking garage may have a number of parallel horizontal planes (the roof and floor of each level), a number of parallel vertical planes (comprising the walls) oriented in the north- south direction, and a number of parallel vertical planes oriented in the east-west direction. Therefore, in the terminology of this patent application, the parking garage is said to have three "principal axes" or dominant orientations: one up-down principal axis, a north-south principal axis, and an east-west principal axis. Nearly all structures within the parking garage will align with one of those axes. One or more embodiments of the present invention seek to exploit this expectation of finding multiple parallel features within the same object.

[0017] The method works by first finding and identifying large, high-confidence features within a point cloud dataset using one or more techniques such as RANSAC, region-growing, the Hough transform, etc. Once these large features have been detected, the "principal axes", or dominant orientations, of each object within the scene are determined by clustering the axes of the features found in the first operation. A subset of these principal axes is selected and is used to constrain the next search. This selection process may be performed either through manual processes, or through an automated, rule-based process (such as a rule to keep the largest vertical surfaces), or a combination of the two processes. A second search of the point cloud is then performed with the objective of finding smaller features. This second search may have looser (less restrictive) tolerance requirements such as fewer required points or relaxed distance tolerances, but this subsequent search is constrained to finding parametric surfaces aligned with the dominant orientations of the surfaces found in the first step. The looser tolerances allow the second search to find smaller features, while the axial constraints retain a higher level of confidence when extracting objects with regularly-oriented features.

[0018] The following paragraphs describe one or more embodiments of the current invention in terms of finding planar regions within a scanned LIDAR dataset, but the same process may be used to find axially oriented, parametric surfaces such as cylinders, cones, etc. One or more embodiments of the method may be implemented in software, e.g., a set of instructions for execution by a computer system, hardware, firmware, and a combination thereof.

[0019] Figure 1 depicts a block diagram of a computer system 100 in accordance with an embodiment of the present invention, wherein the computer system is programmed, e.g., executes a set of instructions stored, for example, in memory 102, with a method according to an embodiment of the present invention. Computer system 100 comprises a memory 102, a central processing unit (CPU) 104, a display 106, a storage device 108, a user input device 110, and a bus 112. Bus 112 or another similar communication mechanism communicates information between the components of computer system 100, i.e., memory 102, CPU 104, display 106, storage device 108, and user input device 110. Memory 102, such as a volatile and/or non- volatile memory, stores a set of instructions to be executed by CPU 104. Memory 102 also may be used for storing temporary variables or other intermediate information during execution of instructions to be executed by CPU 104.

[0020] CPU 104 processes information and instructions, e.g., stored in memory 102 and/or storage device 108, according to at least some embodiments.

[0021] Computer system 100 further comprises display 106, such as a liquid crystal display (LCD), cathode ray tube (CRT), or other display technology, for displaying information to a user. In at least some embodiments, display 106 is not included as a part of computer system 100. In at least some further embodiments, computer system 100 may be configured to be removably connected with display 106.

[0022] In at least some embodiments, storage device 108 may comprise a static and/or a dynamic memory storage device such as a hard drive, optical and/or magnetic drive, etc. for storing information and/or instructions.

[0023] Figure 2 depicts a high-level functional process flow of processing a point cloud and classifying points according to the features upon which the points fall in accordance with an embodiment. A first step 202 is to perform an unconstrained search for large parametric surfaces, e.g., using random sample consensus (RANSAC), a 3D version of a Hough transform, edge-detection, region-growing, etc. The goal of the initial search for parametric surfaces is to positively identify prominent features with very high confidence. Therefore, acceptance criteria such as minimum number of points, goodness-of-fit, etc. are set very high to ensure that only surfaces with a high degree of confidence are included. The result of step 202 is a set of surface "patches", where a patch is defined as a plurality of points that are all deemed to fall on the same surface.

[0024] After determining a set of high-confidence surface patches in step 202, the process flow proceeds to a step 204 to determine the dominant orientations, or principal axes, for objects in a point cloud. The principal axes are determined based on the axes of the previously determined surface patches, and are later used to constrain the search for additional, smaller features.

[0025] The process flow proceeds to a step 206 wherein the axes of each original patch are "snapped" to the closest principal axis if the points belonging to the patch are within a distance threshold predetermined by the user. Snapping in this context is defined as replacing the original parametric axis of the surface with the newly discovered principal axis (as illustrated in Figure 4, 406). For example, a plane is typically represented as a combination of an origin, which is a single point in space (x₀, y₀, Z₀) through which the plane passes, and a normal vector (A₀, B₀, C₀) which is perpendicular to the plane (the "axis" of the plane). Snapping the plane to a principal axis simply means replacing the original axis parameters (A₀, B₀, C₀) with the parameters of the principal axis, while leaving the origin of the plane (xo, y₀, Z₀) untouched. Snapping the plane in this manner creates a more uniform resultant model with parallel surfaces. In at least some embodiments, the resultant model comprises exactly parallel surfaces. This procedure has the potential either to improve the accuracy of the resultant model (by "smoothing out" small noise effects between truly parallel surfaces), or to decrease the true accuracy of the model (by forcing surfaces that are not truly parallel into alignment with each other).

[0026] From this set of principal axes, a subset is selected that is expected to be the most likely set of orientations for finding smaller features 208. The subset selection may be based on a-priori rules (such as "keep all near-vertical axes", "keep axes representing the greatest number of 3D data points", "keep principal axes formed from the largest clusters of original axes", etc.). In at least some embodiments, the subset selection may be a manual process whereby a user selects, e.g., by manipulating user input device 110 (Figure 1), the axes that are most representative of the smaller features that remain to be extracted. In at least some embodiments, the subset selection may be an automated process based on a pre-determined set of criteria.

[0027] Though the choice to set high acceptance requirements (i.e., restrictive tolerances) for the initial feature search provides a high confidence in the extracted features, a potential side effect of this choice is an increased probability that the points belonging to those large features will initially be under-classified, e.g., some points that truly should belong to the feature are not classified as such because they did not meet the acceptance criteria. Therefore, the process flow proceeds to a step 210 wherein the classification of points surrounding these features is repeated using looser tolerances in order to include all points belonging to the prominent features. Process step 210 is similar to step 202, except that the predetermined tolerance value(s) are loosened (i.e., the number of points required to form a surface is decreased, and / or the distance tolerance for surface fitting is relaxed). In at least some embodiments, step 210 (i.e., the reclassification process) will catch some of the under-classified points from step 202 and correctly classify them.

[0028] The process flow proceeds to a step 212 wherein the method comprises searching for smaller, subtler features (i.e., features defined by fewer points) in the point cloud. In accordance with an embodiment of the present invention, smaller features are more reliably extracted by constraining the search to find only features with axes aligned with the principal axes. Points in the point cloud that are not found to belong to the initially-extracted prominent features are searched for additional surface features. Again, the search procedure is similar to step 202, but it is constrained to finding features aligned with the principal axes, and the acceptance criteria such as a minimum number of classified points or a minimum area threshold may be lowered in step 212 to allow inclusion of smaller features.

[0029] The process flow proceeds to step 214 wherein boundary outlines (edges) for each cluster of contiguous, same-class points identified in step 212 are extracted. This may be performed using any number of edge-tracing or contouring algorithms. These edges may or may not be altered through generalization, squaring, or other methodologies designed to enhance their suitability. Additionally, these edges may or may not be projected onto the computed surfaces assigned to the point cluster.

[0030] The process flow then continues to step 216, where the edges from step 214 are determined to be either 1) "real" edges, or 2) "occluded" edges caused by an occluding object. Figure 6 illustrates the difference between real edges and occluded edges. Depiction 602 shows two 3D boxes, with one in front of, or occluding, the other. Surface 602a is partially occluded by surface 602b. Depiction 604 depicts an exploded view where the surfaces (602a and 602b) are shown spaced apart from each other. The real edges are shown as solid lines (604a). The real, but hidden lines that complete the surface are shown as heavy dotted lines (604b). The occluded edge caused by the occlusion of surface 602b is shown as a medium dashed line (604c). This occluded edge is not a real edge in the surface, but rather an artificial visibility border caused by the perspective of the scanner when recording the image.

[0031] Standard edge-tracing or contouring algorithms cannot tell the difference between real edges and occluded edges when outlining the point clusters. However, this information can be useful for knowing how to extend the surfaces to create a complete model, whether this is done manually or through an automated process.

[0032] When the original outlines are created, each separate line segment comprising the outline represents a boundary between two surfaces. In at least some embodiments, the determination of the "real" vs. "occluded" status of each line segment or edge is based on the depth difference between these two adjoining surfaces with a minimum threshold determining whether an edge is classified as real or occluded. Here, depth is defined as the distance of a point from the scanner. In other embodiments, the edges are classified based on a depth difference between points belonging to each surface. In a further method, the intersection of the two surfaces is computed, and the computed intersection is projected along the scanner's line of sight onto the image plane. The edge line is also projected along the scanner's line of sight onto the image plane, and the distance between the two linear features is computed. If the intersection does not exist (due to parallel surfaces), or if the distance between the two linear features surpasses a threshold, the surface with the greater distance from the scanner is considered to be occluded by the nearer surface, and the line segment of the occluded surface is labeled as "occluded" (604c), while the line segment of the nearer surface is labeled "real" (604d).

[0033] In at least some embodiments, the line segment is identified as "occluded" or "real" by storage or assignment of a particular value in memory.

[0034] The resulting outlines may then be graphically displayed according to the "real" or "occluded" status to guide a user in interpreting the results and aid in the reconstruction of a complete CAD model. [0035] In at least some embodiments, steps 214 and 216 may be considered optional and not executed.

[0036] Figure 3 depicts a detailed functional process flow diagram of a method for determining the principal axes of step 204 (Figure 2).

[0037] The process flow begins with a clustering process 302 wherein CPU 104 clusters the axes from each of the previously discovered surface patches based upon angular criteria, i.e., axes which are nearly parallel are clustered together. For example, mathematical normals of discovered planar surface patches are considered as the axes, and the normals are "clustered" or "grouped" such that patches with nearly parallel normals are grouped together. In at least some embodiments, methods for clustering data comprise k-means clustering, fuzzy c-means clustering, the QT clustering algorithm, etc. In alternative embodiments, differing methods may be employed to achieve the desired goal of grouping similar axes together.

[0038] After the surface patches have been grouped according to similar axes, a single axis is computed for each cluster based upon a best fit to the surface patches associated with the cluster. In at least some embodiments, determining the best-fit axis may comprise weighted averaging, statistical selection methods such as selection of the largest surface or median angled axis, vector addition, etc. However, a preferred method is outlined in Figure 3, and depicted graphically (in two dimensions (2D)) in Figure 4. In accordance with the Figure 3 method, the best axis is obtained by finding the centroid of each surface patch and computing the offset of that centroid from a common point (such as 0,0,0). If the offset is (x_offset, Yoffset, z_Off_Set) then all points associated with the patch are translated by (-x₀ff_set, -yoffset, -z₀ff_Set) such that the patch points are centered on the common point. After all of the patches in the cluster are translated according to step 304, a regression analysis is performed through all of the translated points to fit a single surface to the points. In step 306, the axis of the fitted single surface is then considered the best representative axis for the entire cluster of axes and is termed a "principal axis". This process is performed for each cluster of axes, and the resultant principal axes are stored, e.g., in memory 102 and/or storage device 108, and used for future extraction work in step 322. [0039] Figure 4 illustrates the rationale behind the translation-regression approach using a 2D example. Depiction 402 shows two patches of points (one whose points are represented by hollow squares, and one whose points are represented by solid black squares) whose individual regression lines (indicated by reference lines 402a and 402b) are approximately parallel to each other. Since the two patches are offset from each other in space, a regression through both sets of points in the original locations is meaningless as indicated by reference line 402c. However, if the two sets of points are translated such that the centroids of the patches (indicated generally by reference numerals 402d and 402e) coincide as illustrated in depiction 404, a regression line 404a through the combined set of points provides a weighted average axis for both sets of points, which is also referred to as a "principal axis" 404a. Principal axis 404a is used to create improved surfaces in the original point space by constraining the original surfaces to align with the principal axis, as illustrated in depiction 406.

[0040] If the scanned object is known, e.g., based on user knowledge, to have orthogonal features (such as walls that are perpendicular to each other or to floors), the knowledge may be exploited to improve upon the estimation of principal axes and to enforce exact orthogonal relationships (step 308) between principal axes as shown in Figure 3. According to this embodiment, each principal axis is adjusted in turn according to all other axes that are orthogonal to the principal axis. In step 310, a principal axis (A) is selected and every other principal axis is compared to axis A to see if the compared other principal axis is approximately orthogonal to axis A. When a second principal axis (B) is found that is nearly orthogonal to axis A, in step 312 a mathematical rotation matrix is formed and applied to rotate axis B by exactly 90 degrees such that axis B falls as close as possible to axis A. In accordance with an embodiment, the 90-degree rotation occurs around an imaginary axis C defined by the cross product of axes A and B (AxB). All of the points associated with the surface patches associated with axis B are then rotated using the matrix. Next in step 314, all of the points are translated such that each rotated patch centroid falls on the common point as described above. The process is repeated for every principal axis that is nearly orthogonal to axis A. Additionally in step 314, the points associated with patches associated with axis A are translated (with no rotation) to the same common point. Finally in step 316, a regression analysis is performed to fit a surface through all of the transformed points, and the axis of the fitted surface then replaces the initial estimate for the principal axis A. The process is then repeated for all principal axes.

[0041] Figure 5 illustrates in 2D the method described above in conjunction with Figure 3. Depiction 502 shows two patches that are approximately orthogonal to each other - one with hollow points, and another with solid points, with separate regression lines through each patch as depicted by 502a and 502b. A regression through both sets of points, depicted by 502c, in the original location of the points produces meaningless results. As depicted in 504, simply translating the points such that the centroids of the points coincide and regressing through all points as described in Figure 4 also produces meaningless results as illustrated by line 504a. However, if the hollow points are rotated 90 degrees such that the hollow points align as closely as possible to the solid points, then a regression through both sets of points (after translation to a common centroid) becomes meaningful as illustrated in depictions 506 and 508. The resulting regression axis illustrated in depiction 508a reflects the knowledge that the two patches are supposed to be exactly orthogonal, and represents the best estimate of the true axis, i.e., the axis of the surface scanned by the LIDAR, for the solid points using appropriately weighted information from both patches.

[0042] In step 318, the principal axes found using the preceding methods are then stored, e.g., in memory 102 and/or storage device 102 of computer system 100, and may be used for future extraction of smaller features from the point cloud. Additionally in step 320, the orthogonal vectors discovered for each principal axis may be used for subsequent generalization or squaring of the boundaries or outlines of resulting surface features.

[0043] It will be readily seen by one of ordinary skill in the art that the disclosed embodiments fulfill one or more of the advantages set forth above. After reading the foregoing specification, one of ordinary skill will be able to affect various changes, substitutions of equivalents and various other embodiments as broadly disclosed herein. It is therefore intended that the protection granted hereon be limited only by the definition contained in the appended claims and equivalents thereof.

Claims

That which is claimed is:

1. A method of extracting features from a point cloud, the method comprising: extracting one or more surface patches from one or more points in the point cloud; determining one or more principal axes of objects from the point cloud based on at least one of the one or more surface patches; and performing, constrained by the principal axes, a subsequent extraction of one or more surface patches.

2. The method of claim 1, further comprising: extracting surface outlines; and labeling one or more line segments comprising the outlines according to whether or not the segment resulted from occlusion by another object within the point cloud.

3. The method of claim 2, wherein the one or more line segments are found to be either occluded segments or real segments, and labeling information is stored within memory for subsequent use.

4. The method of claim 2, wherein the one or more line segments are found to be either occluded segments or real segments, and labeling information is displayed graphically for subsequent use.

5. The method of claim 1 , further comprising: replacing one or more original surface parameters with one or more parameters from one of the one or more principal axes.

6. The method of claim 1 , further comprising: generalizing and squaring outlines of resulting surface patches based on one of the one or more principal axes.

7. A method of analyzing a group of surface patches to determine one or more principal axes within the group of surface patches, the method comprising: clustering one or more surface patches based on an angular similarity of a determined surface patch axis; translating points associated with one or more clustered surface patches to a common origin; and deriving a principal axis of the one or more clustered surface patches based on a regression surface through the translated points.

8. The method of claim 7, wherein the clustering comprises clustering one or more surface patches based on whether the surface patch axis of a first surface patch is nearly parallel with the surface patch axis of a second surface patch.

9. A method of analyzing a group of surface patches to determine principal axes within the group of surface patches, the method comprising: clustering patches with nearly parallel axes; translating points associated with clustered patches to a common origin; rotating points associated with surface patches that are nearly orthogonal to the clustered patches by 90 degrees such that the points align as closely as possible to the axis of the clustered patches; translating the rotated points to the common origin; and determining a principal axis based on a regression through the translated points.

10. A memory comprising a set of instructions which, when executed by a processor, causes the processor to extract one or more surface patches; determine one or more principal axes of objects from the point cloud based on at least one of the one or more surface patches; and perform a subsequent extraction of one or more surface patches constrained by the principal axes.

11. The memory of claim 10, wherein the set of instructions further comprise instructions which, when executed by the processor, cause the processor to extract surface outlines; and label the line segments comprising the outlines according to whether or not the segment resulted from occlusion by another object within the point cloud.