WO2013044484A1

WO2013044484A1 - Method and apparatus for robust similarity comparison of 3d models

Info

Publication number: WO2013044484A1
Application number: PCT/CN2011/080382
Authority: WO
Inventors: Jiang Tian; Kangying Cai; Jun TENG
Original assignee: Thomson Licensing
Priority date: 2011-09-29
Filing date: 2011-09-29
Publication date: 2013-04-04

Abstract

A method and apparatus for robust similarity comparison in three-dimensional (3D) models is provided. According to an implementation, at least two candidate 3D models are received, and normal vectors are calculated for each of the 3D models. Based on the computed normal vectors, a representative shape for each of the 3D models is derived. A rough determination on the similarity measure is performed based on the derived representative shapes. A refinement step is further performed to provide a refined decision on the similarity comparison of the 3D models.

Description

METHOD AND APPARATUS FOR ROBUST SIMILARITY

COMPARISON OF 3D MODELS

TECHNICAL FIELD

The present invention relates to 3D model applications. More particularly, it relates to similarity comparisons of 3D models.

BACKGROUND

Repetitive patterns are very common in our daily life. Figure 1 shows an example of the repetitive structures 12A, 12B and 14A, 14B of a castle model 10. The representation of repetitive patterns is an important step for numerous 3D model processing applications. A 3D model compression algorithm, which is based on exploiting repetitive patterns, yields a significant improvement in the compression ratio. If two objects A and B are known to be repetitive patterns in advance, compression of object A and the residue of "A minus B" is sufficient for compression application.

How to test the similarity between A and B becomes a crucial step. The basic idea is to extract numerical data that describes the objects under some identified geometric aspect and to infer the similarity of the models from the distance of these numerical descriptions in some metric space.

3D models similarity measurement also finds applications in content based searching in large 3D object repositories. Example application domains include computer aided design/computer aided manufacturing (CAD/CAM), virtual reality (VR), entertainment, and so on. For content based 3D model search, in order to measure how similar two objects are, it is necessary to compute distances between pairs of descriptors using a similarity measure, and a small distance means high similarity and vice versa.

One way to build a numerical descriptor is to rely on Principal Component Analysis (PCA) registration. In order to construct the descriptor, an object is placed and oriented into the canonical coordinate/reference frame using PCA with its origin at the center of mass of the object and its axes with the principal axes. The PCA based descriptor can be defined as the 3D points' coordinates. Then the averaged pairwise (e.g., nearest neighbor) points distance can be utilized to measure the similarity between two objects. However, it will be appreciated by those of skill in the art that this descriptor is simply not robust enough. In some situations, although two point cloud models are obtained from scanning the same object, the difference between their corresponding descriptors is not small enough, and these two point cloud models will not be considered as obtained from the same real object. One reason is that PCA is based on statistics and has an implicit assumption of Gaussian distributions of its input data points. This assumption may not be true for some data because of scan noise. Another restriction is that PCA is limited to orthogonal linear combinations only.

Iterative Closest Point (ICP) is an algorithm employed to minimize the difference between two clouds of points. This algorithm starts with two point clouds and an estimate of the aligning rigid body transform. It then iteratively refines the transform by alternating the steps of choosing corresponding points across the point clouds, and finding the best rotation and translation that minimizes an error metric based on the distance between the corresponding points. ICP can be used to register 3D scan models, and hence to compare the similarity between two models. Unfortunately, due to its iterative nature, it is usually computationally expensive. The distribution of the normal vectors of the points or polygon that form a 3D object can be used to describe its global shape. Those of skill in the art understand that a normal vector, or simply a "normal," is a vector that is perpendicular to the surface of a polygon or a point. For a point cloud model, the normal at each point is estimated as the normal to the fitting plane obtained by applying the least square method to the k nearest neighbors of the point.

In some systems, the normal vector of each polygon of a model is mapped onto the unit sphere in which the point on the unit sphere has the same normal as the polygon normal. Next, the unit sphere is divided into cells. Each cell corresponds to a range of orientations. The sampling scheme for the discretization of the unit sphere is longitude ( 6>) and latitude ( ), as shown in Figure 2.

Then, all the cells of the unit sphere are mapped onto a rectangular array to form a 1 D shape signature of the 3D models. A simplified example is shown in Figure 3. Figure 3 (i) and (ii) show the front and back hemisphere, respectively. The numbers on the hemisphere indicate the corresponding positions on the array shown in FIG. 3 (iii).

The similarity measure between two models is defined as:

x,y

where xand y are the corresponding columns and rows in Figure 3(iii) and Ν^χ, γ) and N₂ (x, y) are the number of normal vectors within the corresponding (x, y) range for the two models. This method is scale and translation invariant; however, it requires rotational normalization. Considering all these factors, the present invention proposes a more robust similarity testing frame based on orientation-based model descriptor, which is scale, translation, and rotation invariant. SUMMARY

According to an implementation, the method for robust similarity comparison of 3D models, includes computing normal vectors for the two three-dimension models; deriving a representative shape for each of the two three-dimensional models using the computed normal vectors; and determining whether the two models are similar based on the derived representative shapes of the two three-dimension models.

According to another aspect of the invention, the apparatus for robust similarity comparison of 3D models includes a connection interface configured to receive input signals, enable a user interface and/or provide outputs signals, a processor in signal communication with the connection interface; and a memory in signal communication with the processor. The processor is preferably configured to: compute normal vectors for the two three-dimension models, derive a representative shape for each of the two three-dimensional models using the computed normal vectors, and determine whether the two models are similar based on the derived representative shapes of the two three- dimension models.

These and other aspects, features and advantages of the present principles will become apparent from the following detailed description of exemplary embodiments, which is to be read in connection with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

The present principles may be better understood in accordance with the following exemplary figures, in which:

FIG. 1 is exemplary 3D model of a castle showing repetitive structures for identification in accordance with an implementation of the present invention;

FIG. 2 is a graphical representation of the conversion from polygon normal to the unit sphere in accordance with an implementation of the present invention;

FIG. 3 is a representation of the mapping from the unit sphere to a rectangular array in accordance with an implementation of the present invention;

FIG. 4A is a flow diagram of the method for robust similarity comparison in 3D models according to the present invention;

FIG. 4B is a flow diagram of a preferred implementation of the method shown in FIG. 4A;

FIG. 5 is a flow diagram of the representative shape determination performed in the method of FIGS. 4A and 4B, according to an implementation of the present invention;

FIG 6 is a flow diagram of the PCA based analysis performed in the method of FIG. 4B, according to an implementation of the present invention;

FIGS 7A and 7B are exemplary 2D representations to provide simpler explanation of the method for robust 3D model similarity comparison according to an implementation of the present invention;

FIG. 8 is a block diagram of an apparatus configured to perform the robust 3D model similarity comparison according to an implementation of the invention; FIGS. 9A and 9B show an exemplary diagram of a calculation of the convex hull; and

FIG. 10A, 10B and 10C show block diagrams of an apparatus configured to perform the robust 3D model similarity comparison according to an implementation of the invention.

DETAILED DESCRIPTION

FIG. 4A shows a flow diagram of the method 40 for robust similarity 3D model comparison according to an implementation of the invention. Initially two candidate 3D models A and B are received or selected in step 42 as the case may be. All normal vectors of models A and B are then computed in step 44. In some other cases, for example, to save computation resources, a subset of the normal vectors for each model are computed. Based on the calculated normal vectors, a representative shape SA for model A and SB for model B is derived in step 46. Such a representative shape captures the global shape of the model but with less complexity. Therefore, similarity measurement on the representative shapes could provide a rough determination on the similarity of the models. Step 47 determines whether SA is similar to SB. If the decision is no, models A and B are identified in step 50 as having different global shapes and thus are not similar. If SA is similar to SB, models A and B are identified in step 56 as having similar global shapes and thus are roughly similar. Depending on the

application, a further optional refinement step 58 may be necessary, wherein such a refinement step further examine the similarity between model A and model B, for example, by directly examining model A and model B instead of their representative shapes. An advantage of the present method is that the rough determination based on the representative shapes help filter out the cases where the two models are largely dissimilar. Because normally the rough determination process incurs much lower computational complexity than the refinement step, overall the whole comparison process can be achieved with low computational complexity without sacrificing comparison performance.

Fig. 4B shows one embodiment of method 40 in FIG. 4A for robust similarity 3D model comparison according to the invention. In this embodiment, a volume V_A for representative shape S_A and a volume V_B for the representative shape S_B are computed and a determination is made in step 48 as to whether V_A is close to Vs? The determination step 48 calculates a difference |V_A-V_B| between V_A and V_B and

determines if that difference is smaller than a predetermined threshold V_t . The value of the threshold V_t is determined according to the volume that is calculated. If V_A is not close to V_B, i.e. | V_A-V_B | > V_th, models A and B are identified in step 50 as having different global shapes, i.e. these two models are not similar, and the process ends. If V_A is close to V_B, i.e. | V_A-V_B | <= V_th, a PCA based analysis of both S_A and S_B is performed in step 52. If the PCA based analysis results in a false output, models A and B are identified in step 50 as having different global shapes, i.e. not similar and the process ends. If the PCA based analysis results in a true output, models A and B are identified in step 56 as having similar global shape. Depending on the application, an optional refinement step 58 may be performed on the models to provide a refined decision as to whether model A and model B are similar.

According to a preferred implementation of the present invention, the

representative shape determination step 46 is performed in accordance with the steps shown in FIG. 5. Initially, the normal vectors of all or part of 3D discrete points of the model (A and B) are mapped (60) onto the respective point of the unit sphere that has the same normal as the 3D discrete point. Next, the unit sphere is partitioned (62) into multiple cells , i = 1...N_c, where each cell C, corresponds to a range of normal orientations. This partitioning is independent from the result of the prior mapping step. For each cell, C, is marked as mapped in step 64 if there is a normal of some 3D point mapped onto this region. Then a 3D convex hull H of mapped cells, or more precisely, convex hull of the reference points of mapped cells, is computed in step 66 for each model. Denote the convex hull for model A and B as H_A and H_B, respectively. Here it will be noted that the geometric center of each mapped cell is used as the reference point, which means that one mapped cell contributes only one vertex for the convex hull. That is, the convex hull computation does not take into account the number of normal vectors, the present invention is primarily concerned with whether or not some cells are mapped. Those of skill in the art will appreciate that the convex hull of a set of points P in n dimensions is the intersection of all convex sets containing P. Figures 9A and 9B show an example of this concept. Figure 9A shows an input image of a set P of points in n-dimensional space. The problem is to find the smallest convex polygon containing all the points of the set P. FIG. 9B shows an example of the output image showing the computed convex hull containing all the points in the set P. Various methods and/or processes can be employed to compute the convex hull without departing from the scope of the present invention. In this embodiment, the convex hull H_A and H_B are employed as the representative shapes S_A and S_B, respectively.

Referring again to FIG. 4B, once the above calculation for the convex hull of mapped cells for each model is complete, the volume V_A and V_B is computed for each of the convex hulls H_A and H_B in step 48. When the convex hull of the mapped cells is used as SA and SB, the threshold V_t in the embodiment of step 48 can be set to be around 0.01 -0.05.

Fig. 6 shows an example of the PCA analysis performed in accordance with an implementation of step 52 and corresponding to the embodiment shown in Fig. 5 for step 46. Initially, the corresponding vertices PA and Ps of the convex hull H_A and H_B are recorded in step 70. Next the PCA analysis of PA and Ps are taken and are

transformed into their canonical reference frames PA ' and Ps' in step 72. This may be considered as a normalization step (i.e., post estimation) in which the models are transformed into a canonical coordinate frame. The translation could be accomplished by translating the center of gravity of a model to the origin; the rotation is accomplished by rotating the x-, y-, and z-axis to align with the three principal directions which are calculated from the PCA analysis. In the exemplary implementation referring to the two models A and B, this means that the two convex hulls are rotated into comparable positions so that they are prepared for the next alignment step. The canonical reference frames are then aligned in step 74.

Finally, a variable D as the averaged pairwise (nearest neighbor) points distance between P/ and Ps' is computed in step 76. One embodiment of the computation of D is to examine every point in P_A' and find the corresponding nearest neighbor point in P_B' to form a pair. The average of the distances between all the pairs denoted as D_A->B would be the averaged pairwise points distance D between P_A' and PB', i.e. D = D_A->B. A similar embodiment is to examine every point in P_B' to find corresponding nearest neighbor point in P_A' when calculating the averaged pairwise points distance, i.e. D = D_B->_A. A different embodiment would be to take the average of the two distances calculated in the above embodiments as the averaged pairwise points distance D between P_A' and P_B', i.e. D = (D_A->_B + DB->A)/2.

Step 78 further determines whether D exceeds a user defined threshold a. When D < a, the output is True, and for all other cases, the output is False. The user defined threshold a can vary depending on the maximum value of the partitioning cell size, but for example could be 0.005. By way of further example, an acceptable range of user defined threshold a could be 0.005 ~ 0.01.

Figures 7A and 7B show a 2D example of the process described above. First, the unit circle is partitioned by the circular dots into multiple cells. Second, all normal vectors of model A and model B are mapped onto the unit circle (positioned indicated by triangles). For model A and B, both of them have 4 representative normal vectors (V_Ai, V_A2, V_A3, and V_A4 for model A; V_Bi, V_B2, V_B3, and V_B for model B). The cell on the unit circle that has a point with a same normal as these 8 representative normal vectors is marked by a triangle of different pattern at the middle point of this cell. These cells are marked as mapped cells. Third, the middle points, i.e. the reference points of the mapped cells of each model A or B are connected to form the convex hull for the mapped cells for model A or B. Denote by H_A and H_B their corresponding convex hull, by P_A and P_B the corresponding vertices of H_A and H_B, wherein P_A = {PAI , PA2, PA3, PA , and P_B = {P_Bi, P_B2, P_B3, P_B4}. The comparison of V_A and V_B described above is carried out by comparing the area of H_A and H_B. The PCA analysis of P_A and P_B is performed by first finding out the principle directions of P_A and P_B, which are used to identify their canonical reference frames represented by the crosses 70A and 70B, respectively, shown in Figure 7B. These two frames are then aligned together for the averaged pairwise points distance calculation. Referring back to FIG. 4B, if the PCA based analysis of SA and SB has a true output, the two models, A and B, are determined to have similar global shape. Next, a refinement algorithm is employed to refine the results in step 58 to determine if model A is similar to model B or not.

As will be appreciated from the above, this normal distribution based method can be considered as a quick filtering step to quickly identify and discard all cases where the difference between two models is too large (i.e., larger than a threshold). If the two models pass through this filtering stage, a refinement step 58 will be used to refine the result. In a preferred embodiment, Iterative Closest Point (ICP) is employed to perform the refinement. Those of skill in the art would understand that ICP algorithm iteratively revises the transformation (i.e., translation, rotation) needed to minimize the distance between these two models. The inputs of ICP algorithm are the two models A and B, initial estimation of the transformation, criteria for terminating the iteration. In one of the implementation in accordance to the invention, the termination criteria is set to be when the number of iterations exceeds N, where N is for example around 1000; or the average distance between the two models is smaller than a ratio (β) of the diagonal length of the smaller bounding box of these two models, where β is preferably set to be around 0.5% - 1 %. The output is whether these two models are similar or dissimilar.

Compared with PCA based descriptor, the present method is more robust. PCA based method is a "one shot" analytical solution. For noisy data, if the resulting principal axes from PCA are not accurate, it is hard to obtain reliable results. In comparison, the method of present invention will not reject two models' similarity property as "easily" as PCA based descriptor does. A multi-steps procedure which increases the reliability is utilized in this invention. Compared with ICP method, the present invention is computationally more efficient since ICP iteratively revises the transformation (translation, rotation) needed to minimize the distance between the points of two raw scans.

FIG. 8 shows an example of a processing apparatus 80 which can be configured to perform the 3D model similarity comparison according to an implementation of the invention. A connection interface 82 is provided which can receive input signals (e.g., 3D model candidate information), output resulting signals, and can also include a user interface connection for keyboard or other user interface device. A processor 84 is in signal communication with the connection interface 82, and also with a memory 86 and a data storage device 88. A display 90 can be provided to assist the user by providing a graphical user interface or other visual display to show the performance of the 3D model similarity comparison of the invention.

In accordance with one implementation and as described above with respect to Figure 4, the processor 84 is configured to compute normal vectors for the two three- dimension models, derive a representative shape for each of the two three-dimensional models using the computed normal vectors, and determine whether the two models are similar based on the derived representative shape of the two three-dimension models. The processor further performs a refinement on the two models when it is determined that the two models are similar. In one embodiment, the refinement is performed using an iterative closest point (ICP) algorithm. As illustrated in Fig.4B and described above, the processor further calculates a volume of each of the representative shapes, and based on the calculated volumes, determines whether the two models are similar. In a preferred embodiment, the processor determines whether the volume of each of the representative shapes is close to each other, performs a principal component analysis (PCA) based analysis of each of the representative shapes when it is determined that the volumes are close, and identifies the models as being similar when the PCA based analysis results in a true output.

In one embodiment, the processor is further configured to perform the process described above with respect to Figures 5 and 6.

The processor can be implemented as one unit or by employing several processing modules/units. In an exemplar implementation as shown in Figure 10A, the processor can include a normal vector calculator 102, a representative shape generator 104, a representative shape similarity comparison unit 106 and a refinement unit 108. In one embodiment, the normal vector calculator 102 is configured to realize the step of 44; the representative shape generator 104 is configured to perform step 46; the

representative shape similarity comparison unit 106 is configured to perform step 47 and the refinement unit 108 is configured to perform step 58, as described above with respect to Figure 4A.

In accordance with one implementation of the present invention as shown in

Figure 10B, the representative shape generator 104 can be implemented by employing a unit sphere partition unit 1 10, a normal vector mapping unit 1 12, a mapped cell identifying unit 1 14 and a convex hull calculator 1 16. The unit sphere partition unit 1 10 is configured to perform step 62; the normal vector mapping unit 1 12 is configured to perform step 60; the mapped cell identifying unit 1 14 is configured to perform step 64 and the convex hull calculator 1 16 is configured to perform step 66 as described above with respect to Figure 5.

An example implementation of the representative shape similarity comparison unit 106 is shown in Figure 10C, which includes a volume computing and comparing unit 1 18 and a PCA analyzer 120. The volume computing and comparing unit 1 18 is configured to perform step 48 and the PCA analyzer 120 is configured to perform step 52 as described above with respect to Figure 4B.

These and other features and advantages of the present principles may be readily ascertained by one of ordinary skill in the pertinent art based on the teachings herein. It will thus be appreciated that those skilled in the art will be able to devise various arrangements that, although not explicitly described or shown herein, embody the present principles and are included within its spirit and scope.

Claims

1. A method for similarity comparison of two three-dimension models, the method comprising the steps of:

computing (44) normal vectors for the two three-dimension models;

deriving (46) a representative shape for each of the two three-dimensional models using the computed normal vectors; and

determining (47) whether the two models are similar based on the derived representative shapes of the two three-dimension models.

2. The method of claim 1 , wherein said deriving step (46) comprises:

mapping (60, 62, 64) the computed normal vectors onto respective cells of a unit sphere, the unit sphere being partitioned into multiple cells, each cell corresponding to a range of normal orientations and having a reference point; and

calculating (66) a convex hull of the mapped cells as the representative shape for each model.

3. The method of claim 1 , further comprising

performing a refinement step (58) on the two models when the determining step outputs that the two models are similar.

4. The method of claim 3, wherein the refinement step uses an iterative closest point (ICP) algorithm.

5. The method of claim 1 , wherein the determining step (47) comprises

calculating a volume of each of the representative shapes;

based on the calculated volumes, determining (48, 50, 52, 56) whether the two models are similar.

6. The method of claim 5, further comprising the steps of:

determining whether the volume of each of the representative shapes is close to each other;

performing (52) a principal component analysis (PCA) based analysis of each of the representative shapes when it is determined that the volumes are close; and

identifying (56) the models as being similar when the principal component analysis (PCA) based analysis results in a true output.

7. The method of claim 6, wherein the volume of each of the three-dimension model is determined to be close to each other if their difference is smaller than a pre-determined threshold.

8. The method of claim 6, further comprising the step of identifying (50) that the two models have different global shapes when the volumes of the representative shapes are not close to each other.

9. The method of claim 6, wherein said performing a principal component analysis (PCA) based analysis further comprises the steps of: recording (70) corresponding vertices of the representative shapes; performing (72) a principal component analysis (PCA) of the recorded vertices and transforming recorded vertices into corresponding canonical reference frames; aligning (74) the reference frames;

determining (76, 78) if the averaged pairwise points distance between the reference frames is less than a predetermined threshold;

outputting a true result when the averaged pairwise points distance between the reference frames is less than the predetermined threshold; and

outputting a false result when averaged pairwise points distance between the reference frames is equal to or greater than the predetermined threshold.

10. The method of claim 9, further comprising the step of performing an iterative closest point (ICP) based refinement on the two models when the results of the PCA based analysis outputs a true result.

1 1 . An apparatus for similarity comparison of three-dimensional models, the apparatus comprising:

a connection interface configured to perform at least one of receiving input signals, enabling a user interface and providing outputs signals;

a processor in signal communication with the connection interface; and a memory in signal communication with the processor;

wherein said processor is configured to compute normal vectors for the two three-dimension models, derive a representative shape for each of the two three- dimensional models using the computed normal vectors, and determine whether the two models are similar based on the derived representative shapes of the two three- dimension models.

12. The apparatus of claim 1 1 , further comprising a data storage (88) connected to the processor.

13. The apparatus of claim 1 1 , wherein the processor is further configured to:

map the computed normal vectors onto respective cells of a unit sphere, the unit sphere being partitioned into multiple cells, each cell corresponding to a range of normal orientations and having a reference point; and

calculate a convex hull of the mapped cells as the representative shape for each model.

14. The apparatus of claim 1 1 , wherein the processor is further configured to perform a refinement on the two models when it is determined that the two models are similar.

15. The apparatus of claim 14, wherein the processor is further configured to perform refinement using an iterative closest point (ICP) algorithm.

16. The apparatus of claim 1 1 , wherein the processor is further configure to calculate a volume of each of the representative shapes, and based on the calculated volumes, determine whether the two models are similar.

17. The apparatus of claim 16, wherein said processor is further configured to determine whether the volume of each of the representative shapes is close to each other, perform a principal component analysis (PCA) based analysis of each of the representative shapes when it is determined that the volumes are close, and identify the models as being similar when the PCA based analysis results in a true output.

18. The apparatus of claim 17, wherein the volume of each of the three-dimension model is determined to be close to each other if their difference is smaller than a predetermined threshold.

19. The apparatus of claim 17, wherein said processor is further configured to identify that the two models have different global shapes when the volumes of the

representative shapes are not close to each other.

20. The apparatus of claim 17, wherein said processor is further configured to perform the principal component analysis (PCA) based analysis by:

recording corresponding vertices of the representative shapes;

performing a principal component analysis (PCA) of the recorded vertices and transforming recorded vertices into corresponding canonical reference frames;

aligning the reference frames;

determining if the averaged pairwise points distance between the reference frames is less than a predetermined threshold;

outputting a true result when the averaged pairwise points distance between the reference frames is less than the predetermined threshold; and outputting a false result when averaged pairwise points distance between the reference frames is equal to or greater than the predetermined threshold.