WO2007069260A1 - Method and apparatus for determining similarity between surfaces - Google Patents
Method and apparatus for determining similarity between surfaces Download PDFInfo
- Publication number
- WO2007069260A1 WO2007069260A1 PCT/IL2006/001451 IL2006001451W WO2007069260A1 WO 2007069260 A1 WO2007069260 A1 WO 2007069260A1 IL 2006001451 W IL2006001451 W IL 2006001451W WO 2007069260 A1 WO2007069260 A1 WO 2007069260A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- point
- probe
- cloud
- points
- model
- Prior art date
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/60—Type of objects
- G06V20/64—Three-dimensional objects
- G06V20/653—Three-dimensional objects by matching three-dimensional models, e.g. conformal mapping of Riemann surfaces
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/11—Technique with transformation invariance effect
Definitions
- the present invention relates to automatic object recognition and, more particularly, but not exclusively to a method and apparatus for determining similarity between surfaces, e.g., for matching purposes.
- Matching of surfaces has recently become an important task of computer vision and is needed in a variety of applications, such as biometric, security and medical applications.
- security systems for example, surface matching can be used for face recognition, so as to grant or deny access to select individuals, to alert when an individual is recognized, or to track an individual as the individual travels amongst a plurality of people.
- home automation systems can employ surface matching to distinguish among residents of a home, so that the features of the system can be customized for each resident.
- surface matching can be employed for registration an image scan so as to provide a basis for localizing or tracking a medical instrument with respect to anatomical features or other elements in the image.
- the 2D approach suffers from sensitivity to illumination conditions and object orientation. Since the image represents the light reflected from the object's surface at a single observation angle, different illumination conditions can result in different images, which are likely to be recognized as different objects.
- the 3D approach provides face geometry information, which is typically independent of viewpoint and lighting conditions, and as such is complementary to the two-dimensional image.
- Three-dimensional information carries the actual geometry of the surface of the object, including depth information which allows easy segmentation of the surface from the background.
- the fundamental question in the 3D approach is how to efficiently yet accurately quantify the similarity between a given reference surface (a model), and some other surface (a probe), which is potentially a deformed version of the model. Typically, such quantification is achieved by calculating the so-called
- distance between the model and the probe. It is desired, typically, to capture the distinction of the intrinsic properties of the model and probe (which are associated with their metric structure), while ignoring the extrinsic properties that describe the way the surfaces deform.
- a deformation that preserves the intrinsic structure of the surface is referred to as "an isometry”.
- Euclidian spaces for example, translation, rotation and reflection of a body are isometries, because they are not associated with the structure of the body.
- isometries may also include bending of surfaces.
- U.S. Patent No. 6,947,579 discloses a three-dimensional face recognition technique in which three- dimensional topographical data of a geometric body in a form of a triangulated manifold is converted into a series of geodesic distances between pairs of points of the manifold.
- a bending invariant representation of the geometric body is then provided by forming a low dimensional Euclidean representation of the geodesic distances.
- the bending invariant representation is suitable for matching the geometric body with other geometric bodies.
- the matching is performed by calculating the distance between the respective geometric bodies, based on the bending invariant representations thereof.
- the presence or absence of a match is determined by thresholding the calculated distance.
- MDS multidimensional scaling
- dimensionality reduction and texture mapping see, e.g., A. Elad and R. Kimmel, "On bending invariant signatures for surfaces,” IEEE Trans. PAMI, 2003, 25(10):1285-1295; S. T. Roweis and L. K. Saul, "Nonlinear Dimensionality Reduction by Locally Linear Embedding," Science, 2000, 290(5500):2323-2326; and G. Zigelman and R. Kimmel and N. Kiryati, “Texture mapping using surface flattening via multi-dimensional scaling,” IEEE Trans. Visualization and computer graphics, 2002, 9(2): 198-207).
- MDS multidimensional scaling
- dimensionality reduction and texture mapping see, e.g., A. Elad and R. Kimmel, "On bending invariant signatures for surfaces," IEEE Trans. PAMI, 2003, 25(10):1285-1295; S. T. Roweis and L. K. Saul, “Nonlinear Dimensionality Reduction by Locally Linear
- a method of determining similarity between a non-planar probe surface represented by a probe point-cloud and a non-planar model surface represented by a model point-cloud comprises: calculating an extremal value of an objective function describing embedding of the probe surface into an embedding space having a non-constant sectional curvature; and determining similarity between the probe surface and the model surface based on the extremal value.
- a method of searching a library of non-planar model surfaces for at least a partial match to a non-planar probe surface the non-planar probe surface being represented by a probe point-cloud and each non-planar model surface being represented by a model point-cloud
- the method comprising, for each non-planar model surface, performing the method of claim 1 so as to determine similarity between the probe surface and the model surface, thereby determining the presence or absence of at least a partial match.
- the method further comprises calculating a matrix of geodesic distances corresponding to the probe point-cloud.
- the calculation of the extremal value comprises multiresolution optimization.
- the method further comprises selecting a subset of the probe point-cloud corresponding to a predetermined patch of the probe surface, wherein the calculation of the extremal value is based on the subset.
- apparatus for determining similarity between a non-planar probe surface represented by a probe point-cloud and a non-planar model surface represented by a model point- cloud, the method comprising: an extremal value calculator, for calculating an extremal value of an objective function describing embedding of the probe surface into an embedding space having a non-constant sectional curvature; and a similarity determinator operable to determine the similarity between the probe surface and the model surface based on the extremal value.
- the apparatus further comprises a matrix calculator for calculating a matrix of geodesic distances corresponding to the probe point-cloud.
- the geodesic distances are calculated using the fast marching method.
- the extremal value calculator is operable to employ multiresolution optimization.
- the apparatus further comprises a subset selector for selecting a subset of the probe point- cloud corresponding to a predetermined patch of the probe surface, wherein the extremal value calculator is operable to use the subset for calculating the extremal value.
- the difference between a geodesic distance characterizing the pair in the point-cloud and a geodesic distance characterizing the pair in the patch is higher than a predetermined threshold.
- the sum of geodesic distances in the patch from each point in the pair to a boundary of the patch is higher than a geodesic distance characterizing the pair in the point-cloud.
- the objective function is selected so as to allow calculation of a partial embedding distance between the probe surface and the model surface.
- the objective function is selected so as to allow calculation of a Gromov-Hausdorff distance between the probe surface and the model surface.
- the embedding space is a two-dimensional space.
- the embedding space is a polyhedral approximation of the model surface.
- the polyhedral approximation comprises a triangulated manifold.
- the method further comprises constructing the polyhedral approximation from the model point-cloud.
- the apparatus further comprises a polyhedron constructor for constructing the polyhedral approximation from the model point-cloud.
- the objective function comprises a sum of local distortions, each characterizing embedding of a pair of points in the probe point-cloud into the embedding space.
- the sum is a weighted sum.
- At least a few of the local distortions in the sum comprise a function of the difference between a first geodesic distance defined in the probe surface and a second geodesic distance, corresponding to the first geodesic distance and being defined in the embedding space.
- the first geodesic distance characterizes a pair of points of the probe point-cloud and the second geodesic distance characterizes images of the pair of points.
- the first geodesic distance characterizes a pair of points of the embedding space and the second geodesic distance characterizes images of the pair of points.
- the first geodesic distance is a distance between a first point of the probe point-cloud and an image of a first point of the embedding space
- the second geodesic distance is a distance between a second point of the embedding space and an image of a second point of the probe point-cloud.
- the calculation of the extremal value comprises interpolating geodesic distances in the polyhedral approximation of the model surface.
- the extremal value calculator comprises an interpolator for interpolating geodesic distances in the polyhedral approximation of the model surface.
- the extremal value is calculated by an iterative procedure.
- the iterative procedure traverses the triangulated manifold while ensuring that images of points in the point-clouds are always in a plane of one triangle of the triangulated manifold.
- a direction of the traverse is determined by a procedure selected from the group consisting of the steepest decent method, the fixed step gradient method, the conjugate gradient method, the Newton method and any combination thereof.
- the function of the difference between the first geodesic distance and the second geodesic distance is a non-linear function.
- the non-linear function comprises finite norms of the local distortions. According to still further features in the described preferred embodiments the non-linear function is constrained by a constraint in an infinite norm representation.
- the method further comprises employing penalty barrier method for de-constraining the non-linear function.
- the probe surface and the model surface represent faces.
- Implementation of the method and system of the present invention involves performing or completing selected tasks or steps manually, automatically, or a combination thereof.
- several selected steps could be implemented by hardware or by software on any operating system of any firmware or a combination thereof.
- selected steps of the invention could be implemented as a chip or a circuit.
- selected steps of the invention could be implemented as a plurality of software instructions being executed by a computer using any suitable operating system.
- selected steps of the method and system of the invention could be described as being performed by a data processor, such as a computing platform for executing a plurality of instructions.
- FIG. 1 is a flowchart diagram of a method suitable for determining similarity between surfaces, according to various exemplary embodiments of the present invention
- FIG. 2 is a schematic illustration of an apparatus for determining similarity between surfaces, according to various exemplary embodiments of the present invention
- FIGs. 3a-b show a model surface ( Figure 3 a) for which sufficient geometrical information is given, and a probe surface ( Figure 3 b) for which the known geometrical information is only partial;
- FIGs. 4a-b show a model surface (Figure 4a) and probe surface ( Figure 4b), where there is sufficient geometrical information for both the model surface and the probe surface;
- FIGs. 5a-c are schematic illustrations of a propagation procedure, according to various exemplary embodiments of the present invention.
- FIG. 6 is a flowchart diagram of an algorithm for traversing a triangulated manifold, according to various exemplary embodiments of the present invention.
- FIGs. 7a-d show. example of data hierarchy construction for multiresolution optimization, according to various exemplary embodiments of the present invention
- FIGs. 8a-b are schematic illustrations of a procedure for finding a virtual
- source point characterized by predetermined distances from triangle vertices according to various exemplary embodiments of the present invention
- FIGs. 9a-b are schematic illustrations of partial embeddings of a planar patch into a plane, according to various exemplary embodiments of the present invention.
- FIGs. 10a-b shows normalized distance between a densely sampled unit sphere and spheres of different radii calculated according to a preferred embodiment of the present invention (top line), and using canonical form distances (bottom lines);
- FIGs. 1 la-c show the normalized distances for radii in the range of 0.88 to 1.1, calculated according to a preferred embodiment of the present invention with 50 points (Figure Ha), and using the canonical form distances with 100 (Figure l ib) and 500 (Figure Ha) points.
- FIGs. 12a-f are schematic illustrations of six articulated objects and approximate isometries used according to embodiments of the present invention were applied for determining similarity;
- FIGs. 13a-d show dissimilarity matrices obtained using embodiments of the present invention ( Figures 13a-b) and using canonical form distances ( Figures 13c-d);
- FIGs. 14a-c show proximity pattern representing the dissimilarity matrices of Figures 13a, 13c and 13d, respectively;
- FIGs. 15a-f are schematic illustrations of six probe surfaces with partial geometrical information, used according to embodiments of the present invention for determining similarity to the articulated objects of Figures 12a-f;
- FIGs. 16a-d show model facial surfaces used according to embodiments of the present invention in a face recognition experiment
- FIGs. 17a-b show similarity matrices obtained using embodiments of the present invention for model facial surfaces in the face recognition experiment.
- FIGs. 18a-b show local maps of objective function describing embeddings of probe facial surfaces of two different subjects into a model surface, according to embodiments of the present invention. DESCRIPTION OF THE PREFERRED EMBODIMENTS
- the present embodiments comprise a method and apparatus which can be used for determining similarity between surfaces. Specifically, but not exclusively, the present embodiments can be used for matching a probe surface to a model surface. For example, the present embodiments can be used for searching a library of model surfaces for a model surface which at least partially matches a probe surface.
- non-planar surface is typically known as a metric space (S, ds) induced by a smooth connected and compact Riemannian 2-manifold S, characterized by a geodesic distance function d$(s a ,S b ) between every two points s a and Sb on S.
- the value of d$(s a ,$ b ) is the length of the minimal geodesic of S 1 which passes through the points s a and S h ⁇ it is recognized, however, that practically only a sampled version of the surface in known.
- non-planar surface refers herein to a sampled surface represented by an appropriate point-cloud.
- surface is used herein as an abbreviation of the term “non-planar surface”.
- sectional curvature is known in the art and refers to a mathematical object which controls the rate of geodesic deviation.
- the sectional curvature determines the Riemannian curvature tensor i? ⁇ vp ⁇ -
- the sectional curvature K is related to i? ⁇ vp ⁇ via i? ⁇ vp ⁇ ⁇ K(g ⁇ p g ⁇ v — g ⁇ g pv ), where g ⁇ v is the metric tensor of the space.
- Figure 1 is a flowchart diagram of a method suitable for determining similarity between surfaces, according to various exemplary embodiments of the present invention.
- the method steps described hereinbelow can be executed either contemporaneously or sequentially in many combinations or orders of execution.
- the ordering of the flowchart diagrams is not to be considered as limiting.
- two or more method steps, appearing in the following description or in the flowchart diagrams in a particular order can be executed in a different order (e.g., a reverse order) or substantially contemporaneously.
- several method steps described below are optional and may not be executed.
- the method determines the method determines similarity between a reference surface, referred to herein as the model surface S, and another surface referred to herein as the probe surface Q.
- the method quantifies the similarity between the surfaces, typically by assigning a similarity measure ⁇ (S, Q) to the pair of surfaces S and Q, so as to allow determining, for example, whether or not Q is a deformed version of S.
- the method of the present embodiments is particularly useful in face recognition applications, in which case both S and Q are facial surfaces.
- the method can determine, for example, whether or not the two facial surfaces 5" and Q represent two different expressions or posing angles of the same person.
- low values of ⁇ can correspond to a situation in which the facial surfaces S and Q represent the same face
- high values of ⁇ can correspond to a situation in which the facial surfaces S and Q represent different faces.
- the method can be executed a plurality of times, where in each execution the method determines similarity between Q and a different facial surface of the library.
- non-planar surfaces of the present embodiments are represented by point-clouds.
- the point-cloud representing the model surface is a set of N points s j (j- 1, ..., N) which is referred to herein as the model point-cloud
- the dimensions N and M of the model and probe point-cloud, respectively, can be equal or different.
- the method of the present embodiments is capable of determining similarity between non-planar surfaces even when the number of points in the point cloud is relatively small, e.g., less than 200 points, more preferably less than 100 points, say about 50 points.
- the method begins at step 10 and, optionally and preferably continues to step 11, in which a subset of the probe point-cloud is selected.
- the subset corresponds to a predetermined patch Q ' e Q of the probe surface.
- This embodiment is useful when it is desired to determine similarity between the patch Q ' and the model surface S.
- the selection of subset can be according to any criterion or set of criteria.
- the subset includes only points for which the metric on Q ' (expressed by a proper matrix of geodesic distances) is consistent with the metric on Q.
- a preferred technique for selecting the subset is provided hereinunder.
- the probe point-cloud is preferably redefined to include only the points of the subset.
- the matrix Ds which corresponds to the model point-cloud, is already known.
- the calculation of geodesic distance matrices is well known in the art.
- the calculation of Ds and/or D Q is performed using the fast marching method (FMM), found, e.g., in J. A.
- FMM is an efficient numerical method to compute a first-order approximation of the geodesic distances. Given a set of points ⁇ q j ⁇ on Q a distance map T(x, y) from these points to other points on the surface is obtained as the solution of an Eikonal equation,
- the FMM allows computing of the distance map with an O(MlogM) complexity.
- a similar technique can be applied for the set ⁇ sy ⁇ of «S with complexity of O(N logN).
- an extremal value (a minimum or a maximum) of an objective function F is calculated.
- the objective function F preferably describes the embedding of the probe surface Q (or a patch thereof, Q ') into an embedding space, which preferably has a non-constant sectional curvature.
- the present embodiments contemplate any embedding space other than a spherical, hyperbolic and Euclidian space.
- the embedding space is a two-dimensional space, i.e., the dimensionality of the embedding space is preferably the same as the dimensionality of the probe surface Q.
- the embedding space is a polyhedral approximation of the model surface S.
- the embedding space can be a triangulated manifold, a quadrangulated manifold or it can be expressed using any other polygonal representation.
- the embedding space when it is a triangulated manifold, it is associated with a triangulation set Ts, defined as Ts ⁇ ⁇ (m n ,m l2 ,m n ),...,(m N ⁇ ,m N 2> m N 3 ⁇ > sucn ⁇ 13 * tne m °del point-cloud S is approximated as a polyhedron consisting of N T triangular faces, where the t-th face is described by the vertices s m ,s m ,s m .
- Ts triangulation set
- step 14 is preceded by step 13 in which a polyhedral approximation is constructed from the model point-cloud.
- a region-growing algorithm e.g., a ball-pivoting algorithm or the like.
- the objective function F is selected so as to allow the determination of the similarity between Q and S.
- F is selected such that its extremal (e.g., minimal) value can serve as a similarity measure ⁇ .
- Q —> S is a mapping from Q to S.
- the arguments of the objective function are images in the model surface of points from the probe point cloud.
- the arguments of the objective function can be both sets ⁇ s' j ⁇ and ⁇ s) ⁇ .
- the arguments are images
- they do not necessarily coincide with points from the point-cloud.
- the images are obtained by interpolating the respective point cloud.
- the s s - do not necessarily coincide with points from the model point-cloud.
- F is a function of ⁇ s) ⁇
- the images s) are interpolated in the polyhedral approximation of S.
- the extremal value of F can be calculated using any numerical optimization procedure known in the art. Typically, an iterative procedure is employed. The numerical optimization procedure is preferably selected in accordance with the type of the selected objective function. For example, when the F comprises I 2 -norms, the iterative procedure is preferably such that can efficiently locate extremal points of I 2 - norm optimization problems.
- Representative examples of numerical optimization procedure suitable for the present embodiments include, without limitation, the fixed step gradient method, the conjugate gradient method, the biconjugate gradient method, the Newton method or any approximation thereof including the truncated Newton method, one of the variety of quasi-Newton methods, including BFGS and limited memory BFGS, the Chebyshev method, the minimal residual method and any combination thereof.
- Preferred expressions for the objective function F and preferred procedures for calculating the extremal value of F are provided hereinunder and exemplified in the Examples section that follows.
- step 15 the similarity between the surfaces is determined, based on the obtained extremal value. This can be done, e.g., by subjecting the extremal value to a thresholding procedure so as to determine whether or not the two surfaces are similar. Alternatively, the level of similarity between the surfaces can be determined, e.g., by assigning to the two surface a similarity measure ⁇ as further detailed hereinabove.
- the method ends at step 16.
- Figure 2 is a schematic illustration of an apparatus 20 for determining similarity between probe surface Q and model surface S, according to various exemplary embodiments of the present invention. Apparatus 20 preferably designed and configured for executing selected steps of the method described herein.
- Apparatus 20 comprises a extremal value calculator 22, which calculates the extremal value of the objective function F as explained hereinabove and further detailed hereinunder.
- Apparatus 20 further comprises a similarity determinator 24 which, based on extremal value, determines the similarity between the surfaces Q and S, as further detailed hereinabove.
- apparatus 20 further comprises a matrix calculator 26 for calculating the matrix D Q and optionally the matrix Ds, as further detailed hereinabove.
- Apparatus 20 can also comprise a subset selector 28 which selects a subset of the probe point-cloud corresponding to the predetermined patch Q ⁇ as further detailed hereinabove.
- Subset selector 28 preferably communicates with extremal value calculator 22 such that the subset selected by selector 28 is used for calculating the extremal value of F.
- apparatus 20 further comprises a polyhedron constructor 30 which constructs a polyhedral approximation from the model point- cloud and feeds extremal value calculator 22 with the polyhedral approximation.
- Calculator 22 uses the polyhedral approximation as the embedding space as further detailed hereinabove.
- F is typically selected such that its extremal value can serve as a similarity measure ⁇ .
- the objective function is selected such that ⁇ is a partial embedding distance Q) between the probe surface Q (or a patch Q ' thereof) and the model surface S.
- the partial embedding distance can be defined as:
- the objective function is selected such that ⁇ is a Gromov-Hausdorff distance do ⁇ (S, Q) between Q and S.
- the Gromov-Hausdorff distance can be written as [N. J. Kalton and M. I. Ostrovskii, "Distances between Banach spaces," Forum Math, 1999, l l(l):17-48; F. Memoli and G. Sapiro, "A Theoretical and Computational Framework for Isometry Invariant Recognition of Point Cloud Data," Foundations of Computational Mathematics, 2005, 5(3):313-347]:
- dis ( ⁇ , ⁇ ) ⁇ ( E Q 4 ⁇ where the distortion measure dis ( ⁇ , ⁇ ) characterizes the difference between ⁇ and ⁇ ⁇ , or between ⁇ and ⁇ ⁇ .
- dis ( ⁇ , ⁇ ) can be defined as:
- the objective function F comprises a sum of local distortions, where at least a few of the local distortions characterize the embedding of a pair of points in Q into the embedding space. Preferably, but not obligatorily, the sum is a weighted sum.
- a local distortion is a local function of a difference between two geodesic distances.
- the local distortion is defined to characterize mapping of a specific geodesic path. More specifically, a local distortion is a local function of the difference between a first geodesic distance defined in the probe surface and a corresponding second geodesic distance defined in the embedding space.
- the images q ⁇ , q) are preferably interpolated in a similar way to the images s ⁇ and s' ⁇ , e.g., by identifying image points using vectors of barycentric coordinates, as further detailed hereinabove.
- Equation 1 can also be written as a minimization of maximum distortion in an infinite norm ( i ⁇ ) representation.
- F max w tJ d s (s ⁇ ,s' j )-d Q (q t , q j )
- Equation 1 With such selection for F, Equation 1 becomes the following min-max ⁇ 00 -norm constrained problem:
- Equation 9 have M+ N+ l variables and at most 0.5(M 2 + N 2 + MN- M- N) inequality constraints.
- the extremal value of the objective function F can be calculated by an iterative procedure.
- the method proceeds by iteratively updating their locations on S so as produce a decreasing sequence of function values.
- Equation 7 the expression of Equation 7.
- Equation 8 the problem is first de-constrained, for example, by employing a penalty-barrier method [D. Bertsekas, "Nonlinear programming,” published by Atlanta Scientific]. Once an unconstrained problem is provided it can be solved similarly to the way described below.
- the method selects, at each iterative step, a set of directions d ; w and a step size ⁇ (k) such that the displacement of uf* along df ) by a ⁇ decreases the value of F.
- the iteration index k will be omitted hereinafter.
- the directions d ⁇ can be selected using any method known in the art. Representative examples include, without limitation, the steepest decent method, the fixed step gradient method, the conjugate gradient method, the Newton method and any combination thereof. These methods are well known in the art and are found in many textbooks (to this end see, e.g., D. Bertsekas, supra).
- the step size ⁇ is preferably a variable step size selected so as to ensure a sufficient decrease of the objective function.
- ⁇ is a parameter satisfying 0 ⁇ ⁇ ⁇ 1
- s"j represent the points s' t displaced by ⁇ along d ; .
- Typical values for the parameters ⁇ and ⁇ are, without limitation, about 0.3 for ⁇ , and about 0.5 for ⁇ .
- the initial value ⁇ 0 is selected sufficiently large (e.g., about one tenth of the diameter of the surface. As used herein the term "about” refers to + 10 %.
- the method preferably traverses the triangulated manifold while ensuring that the images s ⁇ are always in a plane of one of the triangles. This can be achieved by following a poly-linear path on the triangulated manifold.
- a point u; on the manifold can be updated by propagating along a straight line in the direction d, until the first intersection with the triangle boundary and proceeding along a line inside the triangle adjacent to the intersected edge. This propagation procedure is continued until the total length of the path is ⁇ .
- a representative example of such propagation procedure is illustrated in Figures 5a-c.
- the final point u' is distant ⁇ from point u along a poly-linear path.
- a preferred algorithm for traversing the manifold while preserving the barycentric representation is illustrated in the flowchart diagram of Figure 6.
- step 58 the algorithm continues to step 58 in which «" and d are transformed to the barycentric coordinates of t ⁇ Since w" is located on b which is shared by t and t ⁇ its transformation can readily be achieved using the same barycentric representation in f up to vertex numbering.
- the transformation of d to the barycentric coordinates of t' is performed by rotating f about b such that t and f are coplanar, and finding a direction vector d 1 in the coordinate system of f, which defines the same planar direction as d in t.
- the transformation of d is completed with the replacement d — > d.
- the calculation of extremal value comprises multiresolution optimization.
- the advantage of multiresolution optimization is that it can reduce or eliminate effects of convergence to a local extremum.
- An additional advantage of multiresolution optimization is a significant speedup of convergence.
- the optimization problem (e.g., minimization of F) is converted to a hierarchy of optimization problems.
- the initial optimization problem is a coarse version of the problem which contains a small number of variables (image points, in the above example).
- the coarse level solution is interpolated to the next resolution level, and is used as an initialization for the optimization at that level. The process is repeated until the desired resolution is obtained.
- Such a multiresolution optimization facilitates the initiation of the iteration, because small local extrema tend to disappear at coarse resolution levels, thus reducing the risk of local convergence.
- the main components of the multiresolution scheme are the hierarchy of data which defines optimization problems at different resolution levels and the interpolation procedure, which allows to pass from coarse level to a finer one.
- Q M ⁇ - Q M ⁇ ⁇ — ⁇ Q M
- ⁇ Q M be an Z-level hierarchy of data.
- the points on the (M)st resolution level are obtained by removing part of the points on the /-th level.
- Such a hierarchy can be constructed, for example, is the farthest point sampling
- M ⁇ points can be selected either randomly or according to a predetermined criterion. For example, in face recognition applications, it is usually easy to roughly locate points such as the nose and the eyes.
- new points are added, preferably in the following manner: q M +1 is preferably selected as the most distant point from Q M , and so on.
- q ⁇ +k argmax ⁇ g ⁇ d Qu (q,, ⁇ q ⁇ ,-,q M ⁇ +k ⁇ ) ⁇
- Figures 7a-d show example of data hierarchy construction for multiresolution optimization, where Figures 7a-c show three resolution levels constructed using the farthest point sampling algorithm, and Figure 7d shows geodesic distances from the coarsest grid points.
- ⁇ 1 can be initialized with such a point on the polyhedron approximation that the geodesic distances from it to the points s[,...,s M ' are as close as possible to the geodesic distances from q x to q u ...,q M .
- s'j can be expressed as:
- the minimum can be found by exhaustively searching over all Si, sz, ..., SN, or even a coarser subset of the model point-cloud. This is because the complexity of such a search is negligible compared to the complexity of the iterative minimization process.
- the approximation error of such selection has the order of the radius of the largest triangle in the polyhedron.
- the average is replaced by a weighted sum, where the weights are chosen according to proximity of « a and Ui 0 to each of the enclosing triangle vertices.
- the weights are chosen according to proximity of « a and Ui 0 to each of the enclosing triangle vertices.
- two interpolation steps are employed.
- a first such interpolation step an approximation to the geodesic distances ds(s ⁇ , S bj ) between the arbitrary point s' ⁇ and each of the three vertices sy of t ⁇ .
- the computation of each of these distances can be regarded as the problem of finding an approximation to ds(s ⁇ , sv) given d ⁇ (S h Sv)-
- the solution of this problem is preferably obtained by the following numerical procedure which is referred to herein as "the three-point geodesic distance approximation".
- the original triangle can always be brought to this canonical coordinate system by a simple pre-computed transformation.
- the actual vertex coordinates Sj in a triangle are not required by the procedure.
- serves as the approximation to the geodesic distance c? s (x 0 ,x) . In the above particular case, the two coincide.
- a preferred procedure for selecting the weights is provided hereinafter.
- the least-squares solution can be performed by any way known in the art, such as, but not limited to, using Newton steps.
- the initialization is preferably achieved as follows. First, simultaneous solutions of different pairs of equations from Equations 13 are found, resulting in at most six different points. Then, the minimization procedure is initialized by the one yielding the smallest value of I . Solution of a pair of quadratic equations has an analytic expression. For example, the first two equations yield X 0
- Lemma 1 Let the vertices x, be non-collinear and let w contain at least two non-zero weights. Then, the first-order derivatives of Xo&iw) with respect to d and w are given by:
- the three-point geodesic distance approximant d s and it derivatives can be calculated from the following equations:
- V x ⁇ (x;X,d,w) 2(x-x 0 )
- V w jJ(x;X,d,w) 2J w (xo -x) (EQ. 21)
- V d ⁇ (x;X,d,w) 2J d (x 0 -x).
- the derivatives of ⁇ ?J with respect to « a are obtained using the chain rule, .
- a C 1 continuity for dj is enforced such that a change of w a by a unit yector ⁇ « a and a change of U] 0 by a unit vector ⁇ M b displace the point x inside the triangles t a and t b by the same Euclidean distance. Since barycentric coordinates are defined up to scale, C l continuity can be enforced by selecting an appropriate scaling for w a and « b -
- the solution is preferably selected consistent with the case when the weight is slightly larger than zero.
- the second inconsistent solution can be avoided by setting the weight to some small positive constant ⁇ rather than strictly to zero.
- two weights vanish, which makes the least squares problem under-determined.
- the approximate geodesic distance from s ⁇ to s' b is preferably calculated in two steps. In a first step, a triplet of approximate geodesic distances d s [s' a ,s r ) from
- distance d s (s' a ,s'b) from s ⁇ to s' b is calculated by operating in the triangle t b - d s (s' a ,s'b) is calculated using the expression ⁇ 5
- V u ⁇ f V U ⁇ i
- the procedure can be used for executing method step 11 (see Figure 1).
- the procedure can also be used by subset selector 28 of apparatus 20.
- the subset preferably includes only points for which the metric on Q ' is consistent with the metric on Q. Specifically, geodesies of Q that pass through points not in Q ' may give rise to inconsistent distances and are preferably removed from the subset.
- M is the size of the point-cloud representing the patch Q '.
- the point-cloud of Q ' is already known (this is a common situation, for example, in face recognition, where one has the full probe surface but wishes to deliberately remove some parts of it).
- the pairs of points satisfying qj) ⁇ ⁇ ⁇ , where ⁇ is a predetermined parameter are preferably excluded.
- FIGS 9a-b are schematic illustrations of particular situations in which a planar patch is partially embedded of in a plane.
- Figure 9a illustrates the partial embedding after the exclusion of inconsistent distances.
- the solid line in Figure 9a corresponds to an inconsistent distance between two points that is preferably excluded from the subset, and the dashed line corresponds to the consistent geodesic distance between the two points.
- Figure 9b illustrates the embedding result when the inconsistent distances are not excluded.
- Embodiments of the present invention have been employed for the purpose of determining similarity between a unit two-dimensional sphere sampled at 3200 points, and spheres with radii ranging from 0.5 to 2.5 sampled at a smaller number of points according to the farthest point sampling strategy with a random seed. Ten random samplings were used for each radius.
- the partial embedding distances were computed according to a preferred embodiment of the present invention using the multi-resolution minimization algorithm.
- Figures 10a-b show the normalized distances as a function of the sphere radius. Shown in Figures 10a-b is the calculation of rfp E (top line of Figure 10a), 4>E (top line of Figure 10b) and the five calculations of dc E canonical form distance (bottom lines of Figures 10a and 10b). The numbers on the plots stand for the number of points used for distance computation.
- Figures l la-c show the normalized distances for radii in the range of 0.88 to
- the partial embedding distance appears to be extremely sensitive to the geometry of the surfaces. A change as small as 0.1 % in the spherical patch radius
- Embodiments of the present invention have been employed for calculating similarity measure between surfaces and their approximate isometries for different articulated objects.
- the surfaces were represented by polyhedral meshes obtained by triangulation.
- Figures 12a-f illustrate the articulated objects used in the present example.
- Figures 12a-f Shown in Figures 12a-f are illustration of human (Figure 12a), dog (Figure 12b), giraffe (Figure 12c), crocodile (Figure 12d), paper (Figure 12e) and spider (Figure 12f), and their approximate isometries. Also shown in Figures 12a-f are the parameters of the corresponding polyhedral approximation (number of vertices, faces and edges) for each surface.
- the canonical form distance was computed between the surfaces, by comparing a full-density canonical form to canonical forms sampled at
- M 100, 250 and 600 points as well as to full-density canonical forms.
- Moment signatures and the iterative closest point algorithm were used to compare between the canonical forms.
- Figures 13a-d show dissimilarity matrices obtained using d GH ( Figure 13 a), d OH ( Figure 13b), dc ⁇ with M- 600 ( Figure 13c) and full-density J CF ( Figure 13d) as the similarity measure.
- Each point in Figures 14a-c represents an object, and an Euclidean distance between two points represents dissimilarity between the corresponding objects.
- the clusters obtained using embodiments of the present invention Figure 14a
- the objects dog and paper exhibit larger intra-cluster dissimilarities, which we attribute to larger deviations from isometricity.
- Embodiments of the present invention have been employed for calculating similarity measure between model surfaces and probe surfaces for which the known geometrical information is only partial.
- the model surfaces were the polyhedral meshes of Example 2, see Figures 12a-f, and the probe surfaces are illustrated in Figures 15a-f.
- the solid parts of the surfaces represent the known geometrical information of the probe surfaces.
- the similarity measures used in the present example were the I 2 - and £ ⁇ - norms partial embedding distances, dp E and d PE , respectively.
- the objective function and its gradient were implemented in C.
- Embodiments of the present invention have been applied to a toy face recognition problem of 20 faces taken from the Notre Dame database [Chang et al., " Face recognition using 2D and 3D facial data,” ACM Workshop on Multimodal User Authentication, 2003, pages 25-32; Flynn et al., "Assessment of time dependency in face recognition: an initial study,” Audio- and Video-Based Biometric Person Authentication, 2003, pages 44-51].
- Figures 17a-b present the similarity matrix using d ?E ( Figure 17a) and dj E
- Figures 18a-b show local maps of the objective function as calculated for two different subjects into a model surface. Shown in Figures 18a-b are the embedding of subject 1, probe 4 into model 1 ( Figure 18a), and the embedding of subject 2, probe 4 into model 1 ( Figure 18b).
- r-covering when used in connection to a particular continuous space S, refers to a continuous subset S r c S such that:
- Bs(so,r) ⁇ s e S: ds(s,$o) ⁇ r ⁇ is an open ball of radius r centered at SQ in S.
- the term "bounded continuous surface” refers to a continuous surface S for which sup Sis ' e s ds(s, s') is finite. S is referred to as a "compact surface” if each of its open covers has a finite sub-cover. IfS is compact it is necessary bounded.
- ⁇ -isometry refers to a mapping ⁇ : S -> Q, for which for every q & Q there exists s e S such that dQ(q, ⁇ (s)) ⁇ ⁇ .
- ⁇ -isometric spaces refers to spaces for which there exists an ⁇ - isometry.
- An image ⁇ (5) of a space S in a space Q is referred to as an " ⁇ -Hausdorff approximation" of S in Q if the mapping ⁇ : S -> Q is an ⁇ -isometry.
- the Gromov-Hausdorff distance is a finite metric on the space M of continuous compact surfaces, in which a point is defined as an equivalence class of a continuous surface and its isometries.
- Another property of the Gromov-Hausdorff distance is the near-isometric similarity, which can be formulated as follows: (i) if d G u(S, Q) ⁇ ⁇ , where S, Q are continuous surfaces, then there exists a 2 ⁇ -Hausdorff approximation of S in Q.
- the converse version of the near-isometric similarity can be formulated as follows: (ii) if there exists an ⁇ -Hausdorff approximation of S in Q, then d G u(S, Q) ⁇ 2 ⁇ .
- An additional property of the Gromov-Hausdorff distance is sampling consistency which can be formulated as follows: Given Q r an r-covering of a continuous surface Q and S ⁇ an r -covering of a continuous surface S, r ⁇
- dp E measures the metric distortion obtained trying to embed the probe surface Q into the model surface S in the "most isometric" way.
- ⁇ ip E can be considered as a part of the Gromov-Hausdorff distance JQ H -
- Q ' c Q is a fixed countable dense set. Assume, without loss of generality, that such a sequence is ⁇ ⁇ n ) itself.
- ⁇ is a distance-preserving embedding of Q ' into S. Since Q ' is dense, ⁇ can be extended to a distance-preserving embedding of the entire Q into S (see, for example, Proposition 1.5.9 in Burago et ah, "A course in metric geometry," American Mathematical Society, 2001).
- dp ⁇ Another property of dp ⁇ is the near-isometric similarity, which can be formulated as follows: if there exist an ⁇ -Hausdorff approximation of a continuous surface S in a continuous surface Q then d ⁇ i ⁇ , Q) ⁇ 2 ⁇ . This property follows straightforwardly from the fact that dpsiS, Q) ⁇ dan(S, Q). It can be shown that the converse version of this property also applies.
- J PE J PE
- S, Q an r'- covering of a continuous surface Q
- S r an /-'-covering of a continuous surface S, d PE (S, Q) - r ⁇ d PB (S r' , Q r ) ⁇ d PE (S, Q) + r'.
- the proof of the sampling consistency property is presented below and is divided into two separate statements: (i) dpE(S, Q) - r ⁇ d PE (S, Q r ) ⁇ d PE (S, Q); and (ii) d PE (S, Q) ⁇ Q) ⁇ d ?E (S, Q) + r ⁇
- fifpE can be used for partial matching of surfaces.
- x 0 is a local minimum of £(x o ;XA,yv) , hence, where g stands for the gradient V x ⁇ .
- g stands for the gradient V x ⁇ .
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
- Analysing Materials By The Use Of Radiation (AREA)
- Investigating Or Analyzing Materials By The Use Of Ultrasonic Waves (AREA)
Abstract
A method for determining similarity between a non-planar probe surface and a non-planar model surface is disclosed. The method comprises calculating an extremal value of an objective function describing embedding of the probe surface into an embedding space having a non-constant sectional curvature; and determining similarity between the probe surface and the model surface based on the extremal value.
Description
METHOD AND APPARATUS FOR DETERMINING SIMILARITY BETWEEN
SURFACES
FIELD AND BACKGROUND OF THE INVENTION The present invention relates to automatic object recognition and, more particularly, but not exclusively to a method and apparatus for determining similarity between surfaces, e.g., for matching purposes.
Matching of surfaces has recently become an important task of computer vision and is needed in a variety of applications, such as biometric, security and medical applications. In security systems, for example, surface matching can be used for face recognition, so as to grant or deny access to select individuals, to alert when an individual is recognized, or to track an individual as the individual travels amongst a plurality of people. In like manner, home automation systems can employ surface matching to distinguish among residents of a home, so that the features of the system can be customized for each resident. In medical applications, surface matching can be employed for registration an image scan so as to provide a basis for localizing or tracking a medical instrument with respect to anatomical features or other elements in the image.
Humans have a remarkable ability to identify objects, such as faces of individuals, in a rapid and seemingly effortless fashion. It develops over several years of childhood and results in the intelligence to recognize thousands of faces and other objects throughout our lifetime. This skill is quite robust, and allows humans to correctly identify others despite changes in appearance, like aging, hairstyle, facial hair and expression. For decades, building automatic systems to duplicate human face identification capability has been an attractive goal for many academic researchers and commercial companies around the world. Various attempts in the past were hampered by a lack of appropriate image acquisition means, efficient identification algorithms with required accuracy, and computation power to implement such algorithms. In general, modern object recognition approaches can be divided into two wide categories: 2D approaches, using only image information (which can be either grayscale or color), and 3D approaches, which incorporate three-dimensional information as well.
While simpler in data acquisition (which permits real-time surveillance applications, such as face recognition from a video-taped crowd in pubic places), the 2D approach suffers from sensitivity to illumination conditions and object orientation. Since the image represents the light reflected from the object's surface at a single observation angle, different illumination conditions can result in different images, which are likely to be recognized as different objects.
The 3D approach provides face geometry information, which is typically independent of viewpoint and lighting conditions, and as such is complementary to the two-dimensional image. Three-dimensional information carries the actual geometry of the surface of the object, including depth information which allows easy segmentation of the surface from the background. The fundamental question in the 3D approach is how to efficiently yet accurately quantify the similarity between a given reference surface (a model), and some other surface (a probe), which is potentially a deformed version of the model. Typically, such quantification is achieved by calculating the so-called
"distance" between the model and the probe. It is desired, typically, to capture the distinction of the intrinsic properties of the model and probe (which are associated with their metric structure), while ignoring the extrinsic properties that describe the way the surfaces deform. A deformation that preserves the intrinsic structure of the surface is referred to as "an isometry". In Euclidian spaces, for example, translation, rotation and reflection of a body are isometries, because they are not associated with the structure of the body. In non- Euclidian spaces, isometries may also include bending of surfaces.
U.S. Patent No. 6,947,579, the contents of which are hereby incorporated by reference discloses a three-dimensional face recognition technique in which three- dimensional topographical data of a geometric body in a form of a triangulated manifold is converted into a series of geodesic distances between pairs of points of the manifold. A bending invariant representation of the geometric body is then provided by forming a low dimensional Euclidean representation of the geodesic distances. The bending invariant representation is suitable for matching the geometric body with other geometric bodies. The matching is performed by calculating the distance between the respective geometric bodies, based on the bending invariant
representations thereof. The presence or absence of a match is determined by thresholding the calculated distance.
The above approach is based on known algorithms such as multidimensional scaling (MDS), dimensionality reduction and texture mapping (see, e.g., A. Elad and R. Kimmel, "On bending invariant signatures for surfaces," IEEE Trans. PAMI, 2003, 25(10):1285-1295; S. T. Roweis and L. K. Saul, "Nonlinear Dimensionality Reduction by Locally Linear Embedding," Science, 2000, 290(5500):2323-2326; and G. Zigelman and R. Kimmel and N. Kiryati, "Texture mapping using surface flattening via multi-dimensional scaling," IEEE Trans. Visualization and computer graphics, 2002, 9(2): 198-207).
Also known are methods in which graph theory is employed in the context of representation of discrete metric spaces [N. Linial and E. London and Y. Rabinovich, "The geometry of graphs and some its algorithmic applications," Combinatorica, 1995, 15(2):333-344]. A graph is embedded in Euclidean space such that the distances between nodes in the graph are close to the geometric distance of the vectors representing the Euclidean space.
However, when dealing with abstract metric spaces like graphs, or even points clouds with local distances, smoothness of the underlying geometry can not be assumed, and the embedding is a much harder problem. One of the caveats of the Euclidean embedding approach is the fact that such approach introduces a metric distortion, because a Riemannian surface cannot be perfectly represented in a finite- dimensional Euclidean space. One of the simplest examples is the i£1)3 graph which cannot be embedded in Euclidean space without distortion.
There is thus a widely recognized need for, and it would be highly advantageous to have, a method and apparatus for determining similarity between surfaces devoid of the above limitations.
SUMMARY OF THE INVENTION
According to one aspect of the present invention there is provided a method of determining similarity between a non-planar probe surface represented by a probe point-cloud and a non-planar model surface represented by a model point-cloud, the method comprises: calculating an extremal value of an objective function describing embedding of the probe surface into an embedding space having a non-constant
sectional curvature; and determining similarity between the probe surface and the model surface based on the extremal value.
According to another aspect of the present invention there is provided a method of searching a library of non-planar model surfaces for at least a partial match to a non-planar probe surface, the non-planar probe surface being represented by a probe point-cloud and each non-planar model surface being represented by a model point-cloud, the method comprising, for each non-planar model surface, performing the method of claim 1 so as to determine similarity between the probe surface and the model surface, thereby determining the presence or absence of at least a partial match. According to further features in preferred embodiments of the invention described below, the method further comprises calculating a matrix of geodesic distances corresponding to the probe point-cloud.
According to still further features in the described preferred embodiments the calculation of the extremal value comprises multiresolution optimization. According to still further features in the described preferred embodiments the method further comprises selecting a subset of the probe point-cloud corresponding to a predetermined patch of the probe surface, wherein the calculation of the extremal value is based on the subset.
According to yet another aspect of the present invention there is provided apparatus for determining similarity between a non-planar probe surface represented by a probe point-cloud and a non-planar model surface represented by a model point- cloud, the method comprising: an extremal value calculator, for calculating an extremal value of an objective function describing embedding of the probe surface into an embedding space having a non-constant sectional curvature; and a similarity determinator operable to determine the similarity between the probe surface and the model surface based on the extremal value.
According to further features in preferred embodiments of the invention described below, the apparatus further comprises a matrix calculator for calculating a matrix of geodesic distances corresponding to the probe point-cloud. According to still further features in the described preferred embodiments the geodesic distances are calculated using the fast marching method.
According to still further features in the described preferred embodiments the extremal value calculator is operable to employ multiresolution optimization.
According to still further features in the described preferred embodiments the apparatus further comprises a subset selector for selecting a subset of the probe point- cloud corresponding to a predetermined patch of the probe surface, wherein the extremal value calculator is operable to use the subset for calculating the extremal value.
According to still further features in the described preferred embodiments for each pair of points in the subset, the difference between a geodesic distance characterizing the pair in the point-cloud and a geodesic distance characterizing the pair in the patch is higher than a predetermined threshold. According to still further features in the described preferred embodiments for each pair of points in the subset, the sum of geodesic distances in the patch from each point in the pair to a boundary of the patch is higher than a geodesic distance characterizing the pair in the point-cloud.
According to still further features in the described preferred embodiments the objective function is selected so as to allow calculation of a partial embedding distance between the probe surface and the model surface.
According to still further features in the described preferred embodiments the objective function is selected so as to allow calculation of a Gromov-Hausdorff distance between the probe surface and the model surface. According to still further features in the described preferred embodiments the embedding space is a two-dimensional space.
According to still further features in the described preferred embodiments the embedding space is a polyhedral approximation of the model surface.
According to still further features in the described preferred embodiments the polyhedral approximation comprises a triangulated manifold.
According to still further features in the described preferred embodiments the method further comprises constructing the polyhedral approximation from the model point-cloud.
According to still further features in the described preferred embodiments the apparatus further comprises a polyhedron constructor for constructing the polyhedral approximation from the model point-cloud.
According to still further features in the described preferred embodiments the objective function comprises a sum of local distortions, each characterizing embedding of a pair of points in the probe point-cloud into the embedding space.
According to still further features in the described preferred embodiments the sum is a weighted sum.
According to still further features in the described preferred embodiments at least a few of the local distortions in the sum comprise a function of the difference between a first geodesic distance defined in the probe surface and a second geodesic distance, corresponding to the first geodesic distance and being defined in the embedding space.
According to still further features in the described preferred embodiments the first geodesic distance characterizes a pair of points of the probe point-cloud and the second geodesic distance characterizes images of the pair of points.
According to still further features in the described preferred embodiments the first geodesic distance characterizes a pair of points of the embedding space and the second geodesic distance characterizes images of the pair of points.
According to still further features in the described preferred embodiments the first geodesic distance is a distance between a first point of the probe point-cloud and an image of a first point of the embedding space, and the second geodesic distance is a distance between a second point of the embedding space and an image of a second point of the probe point-cloud.
According to still further features in the described preferred embodiments the calculation of the extremal value comprises interpolating geodesic distances in the polyhedral approximation of the model surface. According to still further features in the described preferred embodiments the extremal value calculator comprises an interpolator for interpolating geodesic distances in the polyhedral approximation of the model surface.
According to still further features in the described preferred embodiments the extremal value is calculated by an iterative procedure. According to still further features in the described preferred embodiments the iterative procedure traverses the triangulated manifold while ensuring that images of points in the point-clouds are always in a plane of one triangle of the triangulated manifold.
According to still further features in the described preferred embodiments a direction of the traverse is determined by a procedure selected from the group consisting of the steepest decent method, the fixed step gradient method, the conjugate gradient method, the Newton method and any combination thereof. According to still further features in the described preferred embodiments the function of the difference between the first geodesic distance and the second geodesic distance is a non-linear function.
According to still further features in the described preferred embodiments the non-linear function comprises finite norms of the local distortions. According to still further features in the described preferred embodiments the non-linear function is constrained by a constraint in an infinite norm representation.
According to still further features in the described preferred embodiments the method further comprises employing penalty barrier method for de-constraining the non-linear function. According to still further features in the described preferred embodiments the probe surface and the model surface represent faces.
The present invention successfully addresses the shortcomings of the presently known configurations by providing a method and apparatus for determining similarity between surfaces. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. Although methods and materials similar or equivalent to those described herein can be used in the practice or testing of the present invention, suitable methods and materials are described below. In case of conflict, the patent specification, including definitions, will control. In addition, the materials, methods, and examples are illustrative only and not intended to be limiting.
Implementation of the method and system of the present invention involves performing or completing selected tasks or steps manually, automatically, or a combination thereof. Moreover, according to actual instrumentation and equipment of preferred embodiments of the method and system of the present invention, several selected steps could be implemented by hardware or by software on any operating system of any firmware or a combination thereof. For example, as hardware, selected steps of the invention could be implemented as a chip or a circuit. As software,
selected steps of the invention could be implemented as a plurality of software instructions being executed by a computer using any suitable operating system. In any case, selected steps of the method and system of the invention could be described as being performed by a data processor, such as a computing platform for executing a plurality of instructions.
BRIEF DESCRIPTION OF THE DRAWINGS
The invention is herein described, by way of example only, with reference to the accompanying drawings. With specific reference now to the drawings in detail, it is stressed that the particulars shown are by way of example and for purposes of illustrative discussion of the preferred embodiments of the present invention only, and are presented in the cause of providing what is believed to be the most useful and readily understood description of the principles and conceptual aspects of the invention. In this regard, no attempt is made to show structural details of the invention in more detail than is necessary for a fundamental understanding of the invention, the description taken with the drawings making apparent to those skilled in the art how the several forms of the invention may be embodied in practice.
In the drawings:
FIG. 1 is a flowchart diagram of a method suitable for determining similarity between surfaces, according to various exemplary embodiments of the present invention;
FIG. 2 is a schematic illustration of an apparatus for determining similarity between surfaces, according to various exemplary embodiments of the present invention; FIGs. 3a-b show a model surface (Figure 3 a) for which sufficient geometrical information is given, and a probe surface (Figure 3 b) for which the known geometrical information is only partial;
FIGs. 4a-b show a model surface (Figure 4a) and probe surface (Figure 4b), where there is sufficient geometrical information for both the model surface and the probe surface;
FIGs. 5a-c are schematic illustrations of a propagation procedure, according to various exemplary embodiments of the present invention;
FIG. 6 is a flowchart diagram of an algorithm for traversing a triangulated manifold, according to various exemplary embodiments of the present invention;
FIGs. 7a-d show. example of data hierarchy construction for multiresolution optimization, according to various exemplary embodiments of the present invention; FIGs. 8a-b are schematic illustrations of a procedure for finding a virtual
"source" point characterized by predetermined distances from triangle vertices according to various exemplary embodiments of the present invention;
FIGs. 9a-b are schematic illustrations of partial embeddings of a planar patch into a plane, according to various exemplary embodiments of the present invention. FIGs. 10a-b shows normalized distance between a densely sampled unit sphere and spheres of different radii calculated according to a preferred embodiment of the present invention (top line), and using canonical form distances (bottom lines);
FIGs. 1 la-c show the normalized distances for radii in the range of 0.88 to 1.1, calculated according to a preferred embodiment of the present invention with 50 points (Figure Ha), and using the canonical form distances with 100 (Figure l ib) and 500 (Figure Ha) points.
FIGs. 12a-f are schematic illustrations of six articulated objects and approximate isometries used according to embodiments of the present invention were applied for determining similarity; FIGs. 13a-d show dissimilarity matrices obtained using embodiments of the present invention (Figures 13a-b) and using canonical form distances (Figures 13c-d);
FIGs. 14a-c show proximity pattern representing the dissimilarity matrices of Figures 13a, 13c and 13d, respectively;
FIGs. 15a-f are schematic illustrations of six probe surfaces with partial geometrical information, used according to embodiments of the present invention for determining similarity to the articulated objects of Figures 12a-f;
FIGs. 16a-d show model facial surfaces used according to embodiments of the present invention in a face recognition experiment;
FIGs. 17a-b show similarity matrices obtained using embodiments of the present invention for model facial surfaces in the face recognition experiment; and
FIGs. 18a-b show local maps of objective function describing embeddings of probe facial surfaces of two different subjects into a model surface, according to embodiments of the present invention.
DESCRIPTION OF THE PREFERRED EMBODIMENTS
The present embodiments comprise a method and apparatus which can be used for determining similarity between surfaces. Specifically, but not exclusively, the present embodiments can be used for matching a probe surface to a model surface. For example, the present embodiments can be used for searching a library of model surfaces for a model surface which at least partially matches a probe surface.
The principles and operation of a method and apparatus according to the present invention may be better understood with reference to the drawings and accompanying descriptions. Before explaining at least one embodiment of the invention in detail, it is to be understood that the invention is not limited in its application to the details of construction and the arrangement of the components set forth in the following description or illustrated in the drawings. The invention is capable of other embodiments or of being practiced or carried out in various ways. Also, it is to be understood that the phraseology and terminology employed herein is for the purpose of description and should not be regarded as limiting.
Surface matching techniques which are based on embeddings a surface into an embedding space of high dimensionality introduce metric distortions which increase the matching error. For example, in U.S. Patent No. 6,947,579, the surfaces to be matched are first embedded into a Euclidean space of three or more dimensions to produce bending-invariant canonical forms, and, subsequently, high-dimensional moments are used to compare between the bending-invariant canonical forms. Similar procedures have also been employed by replacing the Euclidean space with a non-flat space with a constant sectional curvature (e.g., a spherical or hyperbolic space) for which the geodesic distances are expressed analytically. However, in these techniques, the errors arising from the embedding step limit the distinguishing capability of the comparison. Such errors are inevitable and typically cannot be neglected because, generally, the surfaces of interest have intricate geometry which cannot be isometrically embedded into a space with a zero or another constant sectional curvature.
While conceiving the present invention it has been hypothesized and while reducing the present invention to practice it has been realized that similarity between
non-planar surfaces can be determined by embedding at least one of the surfaces into an embedding space having a non-constant sectional curvature.
The term "non-planar surface" is typically known as a metric space (S, ds) induced by a smooth connected and compact Riemannian 2-manifold S, characterized by a geodesic distance function d$(sa,Sb) between every two points sa and Sb on S. The value of d$(sa,$b) is the length of the minimal geodesic of S1 which passes through the points sa and Sh¬ it is recognized, however, that practically only a sampled version of the surface in known. Such a sampled surface is represented by a point-cloud which is a set of points Sj (J - I, ..., N), on the Riemannian 2-manifold, and which is sufficient for describing the topology of the manifold. Being represented as a discrete set of points, the sampled version of the surface is characterized by a discrete set of geodesic distances, rather than by a continuous function. Such a discrete set of geodesic distances is conveniently described by an NxN matrix Ds = (ds(si,Sj)). Thus, unless explicitly stated, the term "non-planar surface" refers herein to a sampled surface represented by an appropriate point-cloud. The term "surface" is used herein as an abbreviation of the term "non-planar surface".
The term "sectional curvature" is known in the art and refers to a mathematical object which controls the rate of geodesic deviation. The sectional curvature determines the Riemannian curvature tensor i?μvpσ- For example, for two-dimensional spaces, the sectional curvature K is related to i?μvpσ via i?μvpσ ~ K(gμp gσv — gμσ gpv), where gμv is the metric tensor of the space.
Referring now to the drawings, Figure 1 is a flowchart diagram of a method suitable for determining similarity between surfaces, according to various exemplary embodiments of the present invention.
It is to be understood that, unless otherwise defined, the method steps described hereinbelow can be executed either contemporaneously or sequentially in many combinations or orders of execution. Specifically, the ordering of the flowchart diagrams is not to be considered as limiting. For example, two or more method steps, appearing in the following description or in the flowchart diagrams in a particular order, can be executed in a different order (e.g., a reverse order) or substantially contemporaneously. Additionally, several method steps described below are optional and may not be executed.
In various exemplary embodiments of the invention the method determines the method determines similarity between a reference surface, referred to herein as the model surface S, and another surface referred to herein as the probe surface Q. The method quantifies the similarity between the surfaces, typically by assigning a similarity measure ε(S, Q) to the pair of surfaces S and Q, so as to allow determining, for example, whether or not Q is a deformed version of S. The method of the present embodiments is particularly useful in face recognition applications, in which case both S and Q are facial surfaces. In these embodiments, the method can determine, for example, whether or not the two facial surfaces 5" and Q represent two different expressions or posing angles of the same person. Thus, low values of ε can correspond to a situation in which the facial surfaces S and Q represent the same face, while high values of ε can correspond to a situation in which the facial surfaces S and Q represent different faces. When there is more than one model surface, as, for example, in the case of a library of facial surfaces, the method can be executed a plurality of times, where in each execution the method determines similarity between Q and a different facial surface of the library.
As stated, non-planar surfaces of the present embodiments are represented by point-clouds. The point-cloud representing the model surface is a set of N points sj (j- 1, ..., N) which is referred to herein as the model point-cloud, and point-cloud representing the probe surface is a set of M points qj (J = X, ..., M) which is referred to herein as the probe point-cloud. The dimensions N and M of the model and probe point-cloud, respectively, can be equal or different. As demonstrated in the Examples section that follows, the method of the present embodiments is capable of determining similarity between non-planar surfaces even when the number of points in the point cloud is relatively small, e.g., less than 200 points, more preferably less than 100 points, say about 50 points.
The method begins at step 10 and, optionally and preferably continues to step 11, in which a subset of the probe point-cloud is selected. The subset corresponds to a predetermined patch Q ' e Q of the probe surface. This embodiment is useful when it is desired to determine similarity between the patch Q ' and the model surface S. The selection of subset can be according to any criterion or set of criteria. For example, in various exemplary embodiments of the invention the subset includes only points for which the metric on Q ' (expressed by a proper matrix of geodesic distances) is
consistent with the metric on Q. A preferred technique for selecting the subset is provided hereinunder. In the embodiments in which step 11 is employed, the probe point-cloud is preferably redefined to include only the points of the subset.
According to a preferred embodiment of the present invention the method continues to step 12 in which a matrix DQ = {dQ{q\,q]j) of geodesic distances d.Q corresponding to the probe point-cloud is calculated. Typically, the matrix Ds, which corresponds to the model point-cloud, is already known. In the event in which Ds is not known, step 12 preferably comprises the calculation of Ds = (dsi$i,Sj)). The calculation of geodesic distance matrices is well known in the art. In various exemplary embodiments of the invention the calculation of Ds and/or DQ is performed using the fast marching method (FMM), found, e.g., in J. A. Sethian, "A fast marching level set method for monotonically advancing fronts," Proc. Nat. Acad. ScL, 1996, 93(4): 1591-1595; and R. Kimmel and J. A. Sethian "Computing geodesic on manifolds," Proc. US National Academy of Science, 1998, 95:8431-8435, the contents of which are hereby incorporated by reference. FMM is an efficient numerical method to compute a first-order approximation of the geodesic distances. Given a set of points {qj} on Q a distance map T(x, y) from these points to other points on the surface is obtained as the solution of an Eikonal equation, ||V7| = 1; T(qj) = 0 . The FMM allows computing of the distance map with an O(MlogM) complexity. A similar technique can be applied for the set {sy} of «S with complexity of O(N logN).
The method continues to step 14 in which an extremal value (a minimum or a maximum) of an objective function F is calculated. The objective function F preferably describes the embedding of the probe surface Q (or a patch thereof, Q ') into an embedding space, which preferably has a non-constant sectional curvature. In other words, the present embodiments contemplate any embedding space other than a spherical, hyperbolic and Euclidian space. According to a preferred embodiment of the present invention the embedding space is a two-dimensional space, i.e., the dimensionality of the embedding space is preferably the same as the dimensionality of the probe surface Q. As will be appreciated by one of ordinary skill in the art, this corresponds to a zero co-dimension of Q in the embedding space. This is in sharp distinction to other surface matching techniques in which the co-dimension of the probe surface in the embedding space is at least 1 and oftentimes 2 or more.
In various exemplary embodiments of the invention the embedding space is a polyhedral approximation of the model surface S. Thus, the embedding space can be a triangulated manifold, a quadrangulated manifold or it can be expressed using any other polygonal representation. For example, when the embedding space is a triangulated manifold, it is associated with a triangulation set Ts, defined as Ts ~ {(mn,ml2,mn),...,(mN γ,mN 2>m N 3^ > sucn ^13* tne m°del point-cloud S is approximated as a polyhedron consisting of NT triangular faces, where the t-th face is described by the vertices sm ,sm ,sm . Typically, such or similar polyhedral approximation of the model surface S is already provided before the method is executed. In other embodiments, e.g., when only a point-cloud of S is known, step 14 is preceded by step 13 in which a polyhedral approximation is constructed from the model point-cloud. This can be done in any technique known in the art, such as, but not limited to, a region-growing algorithm, e.g., a ball-pivoting algorithm or the like.
Many types of objection functions are contemplated. Generally, the objective function F is selected so as to allow the determination of the similarity between Q and S. Typically, F is selected such that its extremal (e.g., minimal) value can serve as a similarity measure ε. For example, when the embedding space is a polyhedral approximation of the model surface, F can selected such that: z(S, Q) = mm F, (EQ. 1) where the minimization is performed over the arguments of the objective function F.
In various exemplary embodiments of the invention the arguments of the objective function is the set {s)} (J = I, ..., M), where s) satisfy s) = yAjjj) and where ψ. Q —> S is a mapping from Q to S. In other words, in these embodiments, the arguments of the objective function are images in the model surface of points from the probe point cloud. In other embodiments, the arguments of the objective function is the set {q'i} (/ = 1, ..., N), where q) satisfy q'\ — φ(s,) and where φ: S — > Q is a mapping from S to Q. Alternatively, the arguments of the objective function can be both sets {s'j} and {s)}.
In the embodiments in which the arguments are images, they do not necessarily coincide with points from the point-cloud. According to a preferred embodiment of the present invention the images are obtained by interpolating the respective point cloud. For example the ss- do not necessarily coincide with points from the model
point-cloud. Thus, when F is a function of {s)}, the images s) are interpolated in the polyhedral approximation of S. For example, when a triangulated manifold is used, an image point s' can be identified using a vector of barycentric coordinates (t, u), where t e {1, ..., NT} is an index identifying the triangle to which s' belongs, and u = (uι, U2, M3) is a barycentric 3 -vector satisfying Wj ≥ O and u\+U2+U3 = l. The Euclidean displacement vector s' of the point s' is related to the barycentric 3 -vector via s' = M1S1,,, +«2 S »! +M 3 s /« > where s'm (i - 1, 2, 3) are the vertices of triangle t.
The extremal value of F can be calculated using any numerical optimization procedure known in the art. Typically, an iterative procedure is employed. The numerical optimization procedure is preferably selected in accordance with the type of the selected objective function. For example, when the F comprises I2 -norms, the iterative procedure is preferably such that can efficiently locate extremal points of I2 - norm optimization problems. Representative examples of numerical optimization procedure suitable for the present embodiments include, without limitation, the fixed step gradient method, the conjugate gradient method, the biconjugate gradient method, the Newton method or any approximation thereof including the truncated Newton method, one of the variety of quasi-Newton methods, including BFGS and limited memory BFGS, the Chebyshev method, the minimal residual method and any combination thereof. Preferred expressions for the objective function F and preferred procedures for calculating the extremal value of F are provided hereinunder and exemplified in the Examples section that follows.
Once the extremal value of F is calculated, the method continues to step 15 in which the similarity between the surfaces is determined, based on the obtained extremal value. This can be done, e.g., by subjecting the extremal value to a thresholding procedure so as to determine whether or not the two surfaces are similar. Alternatively, the level of similarity between the surfaces can be determined, e.g., by assigning to the two surface a similarity measure ε as further detailed hereinabove. The method ends at step 16. Reference is now made to Figure 2 which is a schematic illustration of an apparatus 20 for determining similarity between probe surface Q and model surface S, according to various exemplary embodiments of the present invention. Apparatus 20 preferably designed and configured for executing selected steps of the method
described herein. Apparatus 20 comprises a extremal value calculator 22, which calculates the extremal value of the objective function F as explained hereinabove and further detailed hereinunder. Apparatus 20 further comprises a similarity determinator 24 which, based on extremal value, determines the similarity between the surfaces Q and S, as further detailed hereinabove. According to a preferred embodiment of the present invention apparatus 20 further comprises a matrix calculator 26 for calculating the matrix DQ and optionally the matrix Ds, as further detailed hereinabove. Apparatus 20 can also comprise a subset selector 28 which selects a subset of the probe point-cloud corresponding to the predetermined patch Q \ as further detailed hereinabove. Subset selector 28 preferably communicates with extremal value calculator 22 such that the subset selected by selector 28 is used for calculating the extremal value of F. Preferably, apparatus 20 further comprises a polyhedron constructor 30 which constructs a polyhedral approximation from the model point- cloud and feeds extremal value calculator 22 with the polyhedral approximation. Calculator 22 uses the polyhedral approximation as the embedding space as further detailed hereinabove.
Following is a description of several types of objective functions, according to various exemplary embodiments of the present invention.
As stated, F is typically selected such that its extremal value can serve as a similarity measure ε. Thus, in one embodiment, the objective function is selected such that ε is a partial embedding distance
Q) between the probe surface Q (or a patch Q ' thereof) and the model surface S. The partial embedding distance can be defined as:
dpE (S, Q) ≡ (EQ. 2)
where P ^ QxQ is defined the set of all pairs of points for which the metric on the patch Q ' is consistent with the metric on Q, P = {(q, q1): dQ <(q,qx) = dQ(q,q')}.
The selection ε = dPE is particularly useful in situations in which one or both surfaces are given only partially. Such situation is illustrated in Figures 3a-b. Shown in Figures 3a-b is a model surface S (Figure 3a) for which sufficient geometrical information is given, and a probe surface Q (Figure 3b) for which the known geometrical information is only partial. It was found by the present Inventors of the
present invention that ε = d?E is also suitable for the case in which sufficient geometrical information is given for both surfaces. Such situation is illustrated in Figures 4a-b, showing sufficient geometrical information for both the model surface S (Figure 4a) and the probe surface Q (Figure 4b). In this embodiment, P = QxQ, and Equation 2 reduces to:
dFE(S,Q) = -^inf^disψ , (EQ. 3)
where dis ψ, defined as dis ψ=
q') ~ ds(ψ(q), ψ(q% As will can be understood from the above definition, dis ψ measures the distortion of the mapping ψ from Q to S. If, for example, dis ψ = 0, the mapping ψ is an isometry and the surfaces S and Q are isometric. A preferred expression for the objective function in the embodiment in which ε = JPE is provided hereinunder.
In another embodiment, the objective function is selected such that ε is a Gromov-Hausdorff distance doκ(S, Q) between Q and S. For bounded metric spaces, the Gromov-Hausdorff distance can be written as [N. J. Kalton and M. I. Ostrovskii, "Distances between Banach spaces," Forum Math, 1999, l l(l):17-48; F. Memoli and G. Sapiro, "A Theoretical and Computational Framework for Isometry Invariant Recognition of Point Cloud Data," Foundations of Computational Mathematics, 2005, 5(3):313-347]:
dis(φ,ψ)}, (EQ 4Λ
where the distortion measure dis (φ, ψ) characterizes the difference between φ and ψ~ι, or between ^and φ~ι. dis (φ,ψ) can be defined as:
The selection ε(S, Q) = don(S, Q) is particularly useful in situations in which sufficient geometrical information is given for both surfaces (e.g., the situation in Figures 4a-b). A preferred expression for the objective function in the embodiment in which ε = don is provided hereinunder.
It is to be understood that similarity measures other than ε = dps and ε = dan are not excluded from the scope of the present invention.
As will be appreciated by one ordinarily skilled in the art, Equation 1 above describes a minimization problem in which the images s) = ψ(qj) serve as optimization variables. This is advantageous because there is no need to optimize over all possible permutations of _?,• and qj. In various exemplary embodiments of the invention the objective function F comprises a sum of local distortions, where at least a few of the local distortions characterize the embedding of a pair of points in Q into the embedding space. Preferably, but not obligatorily, the sum is a weighted sum. A local distortion is a local function of a difference between two geodesic distances. Thus, unlike the distortions dis ψ, dis φ and dis(φ,ψ), in which a supremum is taken on the respective set(s) so as to globally characterize the mappings from one surface to the other, the local distortion is defined to characterize mapping of a specific geodesic path. More specifically, a local distortion is a local function of the difference between a first geodesic distance defined in the probe surface and a corresponding second geodesic distance defined in the embedding space. For example, when the embedding space is an approximation of S, the local distortion is preferably defined as η{ds - do), where η is preferably a non-linear function {e.g., the selection η(x) -x2 correspond to the I2 norm and the selection η(x) = \x\ corresponds to the Ix norm), and ds and UQ are geodesic distances between pair of points in S and Q, respectively. A preferred expression for the objective function F in the embodiment in which ε = JPE is:
F(s[,..., sM' ) = ∑wiJτ\(ds(s'i ,s'J ) -dQ(qi,qJ))^ (EQ. 6)
U where Wy are the weights of the sum over i and j. When F is expressed as a non- weighted sum, Wij = 1 V i, j. The weights can be set so as to select the subset which corresponds to the patch Q' <≡ Q. Specifically, Wy can be selected such that Wy = 1 for (qι, qj) in the subset, and 0 otherwise. According to the aforementioned notations, (qh qj) is a pair of points in the point-cloud of Q, while (s'j, sj') = (ψiqϊ), ψ(qjj) are the images of (qh qj) as mapped to S.
A preferred expression for the objective function F in the embodiment in which ε = JGH is:
N F(s[,..., sM' ;q\ ,..., q'N ) = ^η^g^1, , q'j ) - ds(Sι,Sj))
where ij = I, 2, ..., N are model point-cloud indices, k,l = 1, 2, ..., M are probe point- cloud indices, and #',-, g'j are images in the probe surface of points of the model point- cloud, e.g., q) = φ(sj). The images q\, q) are preferably interpolated in a similar way to the images s\ and s'ι, e.g., by identifying image points using vectors of barycentric coordinates, as further detailed hereinabove.
Equation 1, with the expressions of Equations 2 or 3 for the objective function F, can be viewed as a minimization problem with in a finite norm £p , where l≤p≤∞ . It is to be understood, however, that although practical considerations may favor the choice of finite norms, it is not intended to limit the scope of the present invention to finite norm representations. Thus, Equation 1 can also be written as a minimization of maximum distortion in an infinite norm ( i ∞ ) representation. For example, when ε = 4Έ, F can be defined as F = max wtJ ds (s\ ,s'j)-dQ (qt , qj )| .
With such selection for F, Equation 1 becomes the following min-max ^00 -norm constrained problem:
™% β s.t. μtøVi) - dQ(qhqj)\ ≤ ε, (EQ. 8)
with M+ 1 variables and at most M(M- l)/2 inequality constraints.
Similarly, for ε = dou, another objective function can be defined such that the following min-max ^00 -norm constrained problem is obtained:
ε
\d^hs\) - dQ{qk,q% ≤ z, (EQ. 9) where j > i and 1 > L Equations 9 have M+ N+ l variables and at most 0.5(M2 + N2 + MN- M- N) inequality constraints.
As stated, the extremal value of the objective function F can be calculated by an iterative procedure. Thus, starting with some initial values of the optimization variables of F, the method proceeds by iteratively updating their locations on S so as produce a decreasing sequence of function values. Following is a description of a preferred iterative procedure in the embodiment in which the objective function has the form of Equation 6, where the optimization variables are the image points s'j, j = 1,..., M. One of ordinary skills in the art, provided with the details described herein would know how to adjust the procedure for other types of objective functions, e.g., the expression of Equation 7. When it is desired to solve the constrained problems (e.g., Equations 8 or 9), the problem is first de-constrained, for example, by employing a penalty-barrier method [D. Bertsekas, "Nonlinear programming," published by Atlanta Scientific]. Once an unconstrained problem is provided it can be solved similarly to the way described below.
Let i?'f) = 0,(fc),u[fc)) be the optimization variables at the A--th iteration. According to the presently preferred embodiment of the invention the method selects, at each iterative step, a set of directions d; w and a step size α(k) such that the displacement of uf* along df ) by a^ decreases the value of F. For clarity of presentation, the iteration index k will be omitted hereinafter. The directions d\ can be selected using any method known in the art. Representative examples include, without limitation, the steepest decent method, the fixed step gradient method, the conjugate gradient method, the Newton method and any combination thereof. These methods are well known in the art and are found in many textbooks (to this end see, e.g., D. Bertsekas, supra).
The step size α is preferably a variable step size selected so as to ensure a sufficient decrease of the objective function. In various exemplary embodiments of the invention α is adaptively selected, at each iterative step. This can be done, for example, be a procedure known as "the Armijo rule". According to this procedure, beginning from an initial value α = α0, α is successively reduced by a factor β (0 < β < 1), until the following condition is fulfilled: F(sl ',...,sM)-F(sl,...,sM H ) ≥ ~σα]Td[VH.F(>i,...,i), (EQ. 10)
where σ is a parameter satisfying 0 < σ < 1, and s"j represent the points s't displaced by α along d;. Typical values for the parameters β and σ are, without limitation, about 0.3 for σ, and about 0.5 for β. The initial value α0 is selected sufficiently large (e.g., about one tenth of the diameter of the surface. As used herein the term "about" refers to + 10 %.
A similar rule can be applied when the update is performed for a single point per iteration, yielding a block-coordinate descent algorithm.
During the iterative procedure, the method preferably traverses the triangulated manifold while ensuring that the images s\ are always in a plane of one of the triangles. This can be achieved by following a poly-linear path on the triangulated manifold. Thus, a point u; on the manifold can be updated by propagating along a straight line in the direction d, until the first intersection with the triangle boundary and proceeding along a line inside the triangle adjacent to the intersected edge. This propagation procedure is continued until the total length of the path is α. A representative example of such propagation procedure is illustrated in Figures 5a-c. In Figure 5 a, a point u is displaced along direction d in a triangle t = ABC until the edge BC of the triangle is reached. In Figure 5b an adjacent triangle f = BCD sharing edge BC with triangle t is rotated about BC and the direction vector d is transformed to the barycentric coordinates of triangle t' (d — > d). In Figure 5c the final point u' is distant α from point u along a poly-linear path. Thus, the barycentric representation of the image points s' is preserved.
A preferred algorithm for traversing the manifold while preserving the barycentric representation is illustrated in the flowchart diagram of Figure 6. The algorithm begins at step 50 and continues to step 51 in which a step size α, a propagation direction d, and a point $' = (t, u) identified by a triangle t and barycentric vector u are inputted. The algorithm proceeds to step 52 in which a query barycentric vector «' = u + a d is computed. The algorithm continues to decision step 53 in which it determines whether or not «' is inside triangle t. If «' is inside t, the algorithm continues to step 54 in which the query barycentric vector is accepted and s1 is updated to a new point s' = (t, «') in t. From step 54 the algorithm continues to ending step 60. If M1 is outside t, the algorithm continues to step 55 in which the algorithm finds a new step size α' for which the vector u" = u + α1 d lies on a boundary b of t. From step 56
the algorithm continues to an additional decision step in which the algorithm determines whether or not there exist a triangle adjacent to t at b. If there is no such adjacent triangle, the algorithm continues to step 57 in which sx is updated to a new point s' ~ (t, M") in t. From step 57 the algorithm continues to ending step 60. If there is a triangle f adjacent to t at boundary b, the algorithm continues to step 58 in which «" and d are transformed to the barycentric coordinates of t\ Since w" is located on b which is shared by t and t\ its transformation can readily be achieved using the same barycentric representation in f up to vertex numbering. The transformation of d to the barycentric coordinates of t', is performed by rotating f about b such that t and f are coplanar, and finding a direction vector d1 in the coordinate system of f, which defines the same planar direction as d in t. The transformation of d is completed with the replacement d — > d.
Since at each iteration the Armijo rule produces a decreasing sequence of step sizes, it is sufficient to compute the displacement of s}k) by αod, and cache the entire path. As will be appreciated by one of ordinary skill in the art this allows to save path computations for subsequent smaller steps.
In various exemplary embodiments of the invention the calculation of extremal value comprises multiresolution optimization. The advantage of multiresolution optimization is that it can reduce or eliminate effects of convergence to a local extremum. An additional advantage of multiresolution optimization is a significant speedup of convergence.
Thus, according to a preferred embodiment of the present invention the optimization problem (e.g., minimization of F) is converted to a hierarchy of optimization problems. The initial optimization problem is a coarse version of the problem which contains a small number of variables (image points, in the above example). The coarse level solution is interpolated to the next resolution level, and is used as an initialization for the optimization at that level. The process is repeated until the desired resolution is obtained. Such a multiresolution optimization facilitates the initiation of the iteration, because small local extrema tend to disappear at coarse resolution levels, thus reducing the risk of local convergence.
The main components of the multiresolution scheme are the hierarchy of data which defines optimization problems at different resolution levels and the interpolation procedure, which allows to pass from coarse level to a finer one.
Formally, let QM <=- QM^ ^ — ^ QM, ~ QM be an Z-level hierarchy of data. The points on the (M)st resolution level are obtained by removing part of the points on the /-th level. The corresponding distance matrices D1, ..., DI = DQ are created as sub- matrices of DQ. Such a hierarchy can be constructed, for example, is the farthest point sampling
(FPS) strategy [Eldar et ah, "The farthest point strategy for progressive image sampling", IEEE Trans. Image Processing, 1997, 6(9):1305-1315]. As the coarsest resolution level QM , M\ points can be selected either randomly or according to a predetermined criterion. For example, in face recognition applications, it is usually easy to roughly locate points such as the nose and the eyes. At the next resolution level, new points are added, preferably in the following manner: qM +1 is preferably selected as the most distant point from QM , and so on. Generally, a sequence of new points qMι+k can be added according to q^+k = argmax^g^ dQu (q,,{qι ,-,qMι+k^}) ■
Taking the first Mi points from the sequence produced in this manner, one obtains QM . Figures 7a-d show example of data hierarchy construction for multiresolution optimization, where Figures 7a-c show three resolution levels constructed using the farthest point sampling algorithm, and Figure 7d shows geodesic distances from the coarsest grid points.
Assume that at the /-th resolution level, QM = {qχ,-,qM } were embedded into the model surface S using the iterative minimization algorithm described above. As a result, a set of images ψ(QM ) = {sv,...,sM '} on the polyhedron approximation of S was obtained. At the next resolution level, a larger set Qu "M is embedded into S, by solving the minimization problem for ψ(gw ) = {sY,...,sM '} . The initialization for the first Mi points is readily available from the solution at the previous level. The initial locations for the remaining points s1',- (i - Mι + l, Mi + 2, ..., M/+1) are preferably interpolated. For example, ^1, can be initialized with such a point on the polyhedron approximation that the geodesic distances from it to the points s[,...,sM' are as close as possible to the geodesic distances from qx to qu...,qM . Formally, s'j can be expressed as:
Practically, the minimum can be found by exhaustively searching over all Si, sz, ..., SN, or even a coarser subset of the model point-cloud. This is because the complexity of such a search is negligible compared to the complexity of the iterative minimization process.
Following is a description of a preferred procedure for interpolating the geodesic distances d^/\, s'j) on the polyhedron approximation of the model surface S, according to a preferred embodiment of the present invention. Although the geodesic distance between two arbitrary points s\ and Λ is generally unavailable, each point lies inside a triangle with vertices on the grid. Let si, s%, S3 and sy, s?, Sy be the vertices of the triangles ta and fø enclosing s'a and s'b, respectively. Using the barycentric representation described above, the arbitrary points j'a and s\ ate identified as (tα, Ua) and (tα, «t>), respectively, where αa and Jt0 are the corresponding barycentric 3-vectors. Note that the mutual geodesic distances ds(Si, ay-), ij'~ 1, 2, 3 are known, because they are geodesic distances between points of the model point-cloud. In one embodiment, the geodesic distance ds(s\, s'b) is approximated as the average of the above mutual geodesic distances: j 3 ds(s'α >s'b ) = T ∑ds ψ i » Sf ) . (EQ. 12)
5 /=1 By the triangle inequality, the approximation error of such selection has the order of the radius of the largest triangle in the polyhedron.
In another embodiment, the average is replaced by a weighted sum, where the weights are chosen according to proximity of «a and Ui0 to each of the enclosing triangle vertices. Such approximation makes the interpolant ds{s'a, s'b) a smooth function of «a and U0. The computational complexity of this procedure is relatively low.
In an additional embodiment, two interpolation steps are employed. In a first such interpolation step an approximation to the geodesic distances ds(s\, Sbj) between the arbitrary point s'α and each of the three vertices sy of tø. The computation of each of these distances can be regarded as the problem of finding an approximation to
ds(s\, sv) given d∑(Sh Sv)- The solution of this problem is preferably obtained by the following numerical procedure which is referred to herein as "the three-point geodesic distance approximation".
As a starting point, consider the particular case where s\ is one of the polyhedron vertices, In this case, the distances dj = ds(Si, s'b) for / = 1, 2, 3 can be obtained from the matrix Ds- It will be assumed without loss of generality that the triangle S1, S2, S3 lies in 9t2 with ^1 at the origin and S2 on the horizontal axis, denoting by £1 = (0, 0)τ, xp. = (x2, 0)τ and X3 = (x3, y3)τ the planar coordinates of the corresponding vertices. As will be appreciated by one ordinarily skilled in the art, the original triangle can always be brought to this canonical coordinate system by a simple pre-computed transformation. Thus, the actual vertex coordinates Sj in a triangle are not required by the procedure.
Assume that there exists a (virtual) source point x0 = (Jc0, j>0)τ in the plane of the triangle, whose Euclidean distance from x, is du Figures 8a-b illustrate a procedure for finding such virtual "source" point. When the surface S is planar (Figure 8a), there exists such a point consistent with the data, and it can be found by computing the intersection of three circles of radii dt centered at the corresponding triangle vertices
Xj-
(X0 -X3)V(J) 0 -J3)2 - J3 2.
Once x0 is determined, the Euclidean distance ||χo -χ|| serves as the approximation to the geodesic distance c?s(x0,x) . In the above particular case, the two coincide.
When the surface has non-zero Gaussian curvature (Figure 8b), only an approximate source x0 can be found, yet every pair of circles typically intersects at two distinct points. According to a preferred embodiment of the present invention the virtual source is found by a weighted least-squares solution of Equations 13, e.g.: xo(x,d,w) = argmin£(xo;X,d,w), (EQ. 14) xo where
φco;X,d,w) = ∑whxQ -Xif -df ) , (EQ. 15)
K~ (*u £L, xa), d = ψ\ 4\4l) , and w = (wl5 W2, W3)1 is a vector of weights controlling the influence of each of the vertices (The vector w and its components W1, W2 and W3, should not be confused with the set of weights
appearing in Equation 6 above). A preferred procedure for selecting the weights is provided hereinafter.
The least-squares solution can be performed by any way known in the art, such as, but not limited to, using Newton steps. In the embodiment in which Newton steps are employed, the gradient g and the Hessian matrix H of ^(xo;X,d,w)with respect to £0 are given by g = 4(wTf -x-X-w®f)
H = 4wτf .I + 8^(x0 -X,.Xx0 -X/)τ (EQ- 16)
where (f), =||xo -x;||2 -rf,2 , and W O f= (W1 Zj, W2^2, W3 /j) denotes the element-wise
(Hadamard) product. When the vertices X[ are not collinear, which normally happens in a valid triangulation, the Hessian is full rank for w; ≠ 0. When one of the weights, say W3, is zero, the first two equations of Equations 13 are satisfied and hence the first terms of f vanish. Consequently, the first term in the Hessian is zero, whereas the second term is a sum of two outer products of linearly independent vectors, which is full rank. When only one weight, say W1 is non-zero, the Hessian assumes the form H = 8W1(X0 -X1Xx0 -Xi)τ ■> which is a rank one matrix. Since in this case the solution χ0 admits only the first equation of Equations 13, the minimum of ^(xo;X,d,w) is achieved on the circle described by the said equation, and the null vector of H corresponds to the tangential direction, in which the function does not change.
The initialization is preferably achieved as follows. First, simultaneous solutions of different pairs of equations from Equations 13 are found, resulting in at most six different points. Then, the minimization procedure is initialized by the one yielding the smallest value of I . Solution of a pair of quadratic equations has an analytic expression. For example, the first two equations yield
X0
wherever two, one, or none of the solutions exist. Since by its virtue such an initialization is sufficiently close to the optimal x0 , convergence is achieved in about two or three iterations.
Lemma 1. Let the vertices x, be non-collinear and let w contain at least two non-zero weights. Then, the first-order derivatives of Xo&iw) with respect to d and w are given by:
Jw = Vwx0 = -GwH
= Vdx^ =-«d -1 (EQ. 18)
GHH where GΛV = Vwgτ and Gd = Vdgτ are given by
Gw - 4F(XO1T -X) . ( T \ (EQ- 19)
Gd = -4w(x0lτ -x)
W and F are, respectively, diagonal matrices containing the elements of w and f on the diagonal, and 1 = (1, 1, l)τ.
For the proof of lemma 1, see the Appendix section that follows.
Having the approximant of the virtual source xo(x,d,w) and its first-order derivatives, the three-point geodesic distance approximant ds and it derivatives can be calculated from the following equations:
42(x;X,d,w) = ||x-xo(x,d,w||2, (EQ. 20)
Vx^(x;X,d,w) = 2(x-x0)
VwjJ(x;X,d,w) = 2Jw(xo -x) (EQ. 21)
Vd^(x;X,d,w) = 2Jd(x0 -x).
When s'α = (tα, wa) is given in normalized barycentric coordinates (M1 + w2 + W3 = 1), x can be expressed as x = -ΛTuα = «1χ1 +«2χ2 +«3X3 , where X is a pre- computed matrix defining the geometry of the triangle tα. Hence, the present embodiments successfully provide a tool for calculating the approximate geodesic distance dj(s'α,sb' ) given A =
),ds 2{s2,s\ ),dj{s3,s\ )f and a vector of weights w.
The derivatives of <?J with respect to «a are obtained using the chain rule,
.
Following is a preferred procedure for calculating the weights vector w. According to a preferred embodiment of the present invention the weights w are proportional to «a, the barycentric coordinates of s'Λ. Since inside the triangle u\ > 0, the weights are preferably set to some positive values inside the triangle. When s'a is located on the edge s2s3 , u\ = 0 and W1 is preferably set to zero. This cancels the influence of the vertex ^1 in the least squares problem. Hence, the same vertices participate with the same weights when <?f is computed both from the triangle s\, sz, s3 and the adjacent triangle si, S3, S4, yielding a continuous transition of rff across the edge.
In various exemplary embodiments of the invention a C1 continuity for dj is enforced such that a change of wa by a unit yector Δ«a and a change of U]0 by a unit vector ΔMb displace the point x inside the triangles ta and tb by the same Euclidean distance. Since barycentric coordinates are defined up to scale, C l continuity can be enforced by selecting an appropriate scaling for wa and «b-
Since the least-squares problem (Equation 14) is under-determined when one of the weights is set to zero and generally yields two distinct solution, the solution is preferably selected consistent with the case when the weight is slightly larger than zero. In practice, the second inconsistent solution can be avoided by setting the weight to some small positive constant ε rather than strictly to zero. At the triangle vertices, two weights vanish, which makes the least squares problem under-determined. In this case, however, there is no need for the least squares solution, since the geodesic distances between the triangle edges to s'b are known. The approximate geodesic distance from s\ to s'b is preferably calculated in two steps. In a first step, a triplet of approximate geodesic distances ds[s'a ,sr) from
s'a to Sr, f = 1, 2, 3, is calculated using the expression dj [s'a ,sr )= dj (uα ;X,d;. ,w = ua ) where X is a matrix with the canonical coordinates of s\, ^2, S3, and d/ = (ds(sι,si,),ds(s2,si,),d$(s3,sr))Υ . In a second step, the approximate geodesic
distance ds(s'a ,s'b) from s\ to s'b is calculated by operating in the triangle tb-
ds(s'a ,s'b) is calculated using the expression <5|(j:'α, ^)= dj(ab;X',d,w = ub) , where d is a vector of distances defined as d = (ds(sa,sv\dj(sa,s2,\dj(sa,sy))i: , and X1 is a matrix with the canonical coordinates Of^1-, S2 1 J S3'.
Using the chain rule, it is straightforward to show that the derivatives of. dς {s'a ,s'b ) with respect to ua and U0 are given by
Following is a description of a procedure for selecting the subset which corresponds to the patch Q ' c Q, according to a preferred embodiment of the present invention. The procedure can be used for executing method step 11 (see Figure 1). The procedure can also be used by subset selector 28 of apparatus 20. As stated, the subset preferably includes only points for which the metric on Q ' is consistent with the metric on Q. Specifically, geodesies of Q that pass through points not in Q ' may give rise to inconsistent distances and are preferably removed from the subset. Formally, index pairs (/,/) e {1, 2, ..., M }X{1, 2, ..., M) for which <ig(#,, qj) ≠ d<2' {qi, qj) are excluded from the subset, where M is the size of the point-cloud representing the patch Q '. Typically, the point-cloud of Q ' is already known (this is a common situation, for example, in face recognition, where one has the full probe surface but wishes to deliberately remove some parts of it). In this case the pairs of points satisfying
qj)\ ≥ δ, where δ is a predetermined parameter, are preferably excluded. When the point-cloud of Q ' is not known, the minimal geodesies may change due to the change in the boundary of Q. In this case pairs of for which dQ ') + dQ< (qj, dQ ')\ < dQ< (qi, qj), where dQ ' is the boundary of Q 1 are preferably excluded. Figures 9a-b are schematic illustrations of particular situations in which a planar patch is partially embedded of in a plane. Figure 9a illustrates the partial embedding after the exclusion of inconsistent distances. The solid line in Figure 9a corresponds to an inconsistent distance between two points that is preferably excluded from the subset, and the dashed line corresponds to the consistent geodesic distance
between the two points. Figure 9b, illustrates the embedding result when the inconsistent distances are not excluded.
Additional objects, advantages, and novel features of the present invention will become apparent to one ordinarily skilled in the art upon examination of the following examples and appendix, which are not intended to be limiting. Additionally, each of the various embodiments and aspects of the present invention as delineated hereinabove and as claimed in the claims section below finds experimental and theoretical support in the following examples and appendix.
EXAMPLES
Reference is now made to the following examples, which together with the above descriptions, illustrate the invention in a non limiting fashion.
EXAMPLE l
Partial Embedding Distance Between Spherical Patches
Embodiments of the present invention have been employed for the purpose of determining similarity between a unit two-dimensional sphere sampled at 3200 points, and spheres with radii ranging from 0.5 to 2.5 sampled at a smaller number of points according to the farthest point sampling strategy with a random seed. Ten random samplings were used for each radius. The similarity measures used in the present example were the I2 -norm partial embedding distance, dpE , using M= 100 points, and the £∞ -norm partial embedding distance, dpE, using 100, 250 and 500 points. The partial embedding distances were computed according to a preferred embodiment of the present invention using the multi-resolution minimization algorithm.
As a reference, a canonical form distance dcF [see U.S. Patent No. 6,947,579 and Elad et ah, supra] sampled at M= 100, 200, 250, 300 and 500 points was also calculated.
Figures 10a-b show the normalized distances as a function of the sphere radius. Shown in Figures 10a-b is the calculation of rfpE (top line of Figure 10a), 4>E (top line of Figure 10b) and the five calculations of dcE canonical form distance (bottom lines
of Figures 10a and 10b). The numbers on the plots stand for the number of points used for distance computation.
Figures l la-c show the normalized distances for radii in the range of 0.88 to
Ll, calculated using d$E with 50 points (Figure Ha), and using dcv with 100 (Figure lib) and 500 (Figure Ha) points. Also indicated in Figures l la-c are intervals of sphere radii that fall below the sensitivity threshold of the distance measure are indicated. The grayed areas in Figures l la-c represent the variance of the distance computed for the ten different samplings of the spheres. This variance makes spheres within a certain interval of radii around R= I indistinguishable by the distance function.
The partial embedding distance appears to be extremely sensitive to the geometry of the surfaces. A change as small as 0.1 % in the spherical patch radius
(from 1.000 to 0.999) increases d?E by a value exceeding the variance due to sampling. Similar results are achieved using the £ x -norm partial embedding distance, rfpE and slightly inferior with the ^00 -norm partial embedding distance, dPE . For comparison, with the same number of points (100), dcv was unable to discern between spheres differing in radius by more than 10 %. Increasing the number of points to 500 makes dc? sensitive to radius differences of about 2.64 %, which is still one order of magnitude below the sensitivity of the similarity measure of the present embodiments. It is therefore demonstrated that the present embodiments are capable of accurately determining similarity between surfaces.
The objective function (Equation 6) and its gradient implemented in C were evaluated in about 40 ms on a Pentium® IV PC for M= 100 points. The entire computation of d?E using non-optimized MATLAB® software and a C code took was completed in about 10-60 seconds.
EXAMPLE 2 Gromov-Hausdorff Distance Between Isometric Surfaces
Embodiments of the present invention have been employed for calculating similarity measure between surfaces and their approximate isometries for different articulated objects. The surfaces were represented by polyhedral meshes obtained by triangulation.
Figures 12a-f, illustrate the articulated objects used in the present example.
Shown in Figures 12a-f are illustration of human (Figure 12a), dog (Figure 12b), giraffe (Figure 12c), crocodile (Figure 12d), paper (Figure 12e) and spider (Figure 12f), and their approximate isometries. Also shown in Figures 12a-f are the parameters of the corresponding polyhedral approximation (number of vertices, faces and edges) for each surface.
The similarity measures used in the present example were the I2 -norm
Gromov-Hausdorff distance, 6?QH, and the £∞ -norm Gromov-Hausdorff distance, dGH .
The embedding spaces consisted of the full-density meshes and the probe surfaces were meshes sampled at M= N= 50 points. Optimization was performed in accordance with preferred embodiments of the present invention using the multi- resolution scheme. Geodesic distances were computed using the fast marching method for triangulated domains.
As a reference, the canonical form distance was computed between the surfaces, by comparing a full-density canonical form to canonical forms sampled at
M= 100, 250 and 600 points as well as to full-density canonical forms. Moment signatures and the iterative closest point algorithm were used to compare between the canonical forms.
Figures 13a-d show dissimilarity matrices obtained using dGH (Figure 13 a), dOH (Figure 13b), dcψ with M- 600 (Figure 13c) and full-density JCF (Figure 13d) as the similarity measure. As shown the present embodiments successfully recognize all the objects using dGH as well as dm , although the sampling included a relatively small number of points (M= N= 50). On the other hand, the best result of dCF with full-density canonical forms and iterative closest point yielded about 11 % equal error rate.
Figures 14a-c show proximity pattern representing the dissimilarities between the objects using dGH (Figure 14a), JCF with M= 600 (Figure 14b) and full-density C/CF (Figure 14c) as the similarity measure. Each point in Figures 14a-c represents an object, and an Euclidean distance between two points represents dissimilarity between the corresponding objects. As shown the clusters obtained using embodiments of the present invention (Figure 14a) are significantly tighter as compared to the clusters
obtained using do?. The objects dog and paper exhibit larger intra-cluster dissimilarities, which we attribute to larger deviations from isometricity.
The discriminative power of dlH , dQH and dcψ was quantified using a separability measure σ defined as:
where Sni denotes the /th instance of the nth object, and d is the Euclidean distance between the corresponding points, σ expresses the "worst-case" ratio between the inter-cluster and the intra-cluster distances. Thus, higher values of σ correspond to higher discriminative power between the objects. It was found that the embodiments of the present invention were superior to the reference method. For example, for CIQH the separability measure was σ = 3.47, while for dcF the separability measure was σ < 0.58.
Increasing the number of points from 100 to 600 in dcp increases the computational complexity with small or no benefit in accuracy. Starting from about 250 points, the use of iterative closest point in the computation of the canonical form distance gives negligible accuracy improvement compared to the faster moment signature comparison. It is therefore concluded that in the present experiment, the discriminative power of the canonical form distance is essentially limited by metric distortion introduced by the embedding of the surface into a three-dimensional Euclidian space.
EXAMPLE 3 Partial Embedding Distance Between isometric surfaces
Embodiments of the present invention have been employed for calculating similarity measure between model surfaces and probe surfaces for which the known geometrical information is only partial. The model surfaces were the polyhedral meshes of Example 2, see Figures 12a-f, and the probe surfaces are illustrated in Figures 15a-f. The solid parts of the surfaces represent the known geometrical information of the probe surfaces.
The similarity measures used in the present example were the I2 - and £∞ - norms partial embedding distances, dpE and dPE , respectively. The embedding spaces consisted of the full-density meshes and the probe surfaces were meshes sampled at M= N= 50 points. Optimization was performed in accordance with preferred embodiments of the present invention using the multi-resolution scheme. The objective function and its gradient were implemented in C.
Perfect matching was achieved by both similarity measures, yielding separability measure σ of 1.62 and 1.29 for <s?pE and dm , respectively.
EXAMPLE 4
Face Recognition Experiment
Embodiments of the present invention have been applied to a toy face recognition problem of 20 faces taken from the Notre Dame database [Chang et al., " Face recognition using 2D and 3D facial data," ACM Workshop on Multimodal User Authentication, 2003, pages 25-32; Flynn et al., "Assessment of time dependency in face recognition: an initial study," Audio- and Video-Based Biometric Person Authentication, 2003, pages 44-51].
As models, four different faces were used. Each model contained approximately 4000 points and was represented using global parameterization. As probes, faces of the same subjects with different facial expressions were used (four probes per subject). The probes were cropped using geodesic mask [Bronstein et al., "Three-dimensional face recognition," Intl. J. Computer Vision, 2005, 64(l):5-30] (leaving the nose and the forehead region) and subsampled using farthest point sampling to 53 points. The similarity measures used in the present example were the I2 - and £∞ - norms partial embedding distances,
and d¥E , respectively. All distances were computed numerically using a version of parametric fast marching method. For 53 points, the computation time of the objective function and its gradient is about 20 ms on a Pentium® IV PC. Figures 16a-d show four model facial surfaces, referred to below as models 1-
4, respectively.
Figures 17a-b present the similarity matrix using d?E (Figure 17a) and djE
(Figure 17b). The bars on the right hand side of Figures 17a-b are on logarithmic scale. In both cases a perfect discrimination between different subjects was achieved with equal error rate which is consistent with zero, although the sampling included a relatively small number of points (53). It is noted that such discrimination ability cannot be achieved using the canonical form approach even when thousands of sample points are used.
Figures 18a-b show local maps of the objective function as calculated for two different subjects into a model surface. Shown in Figures 18a-b are the embedding of subject 1, probe 4 into model 1 (Figure 18a), and the embedding of subject 2, probe 4 into model 1 (Figure 18b).
The bar on the right hand side of Figures 18a-b depicts local values F(k) of the objective function at point t. F(K)
F(K) describes the contribution of the &-th point to the objective function. As shown, the objective function is significantly higher when a probe of a different subject is embedded into the model surface.
APPENDIX
Reference is now made to the following appendix, which together with the above descriptions, illustrates the invention in a non limiting fashion. Various embodiments and aspects of the present invention as delineated hereinabove and as claimed in the claims section below find theoretical support in the following appendix.
Mathematical Definitions and Notations The term "r-covering" when used in connection to a particular continuous space S, refers to a continuous subset Sr c S such that:
s≡Sr where Bs(so,r) = {s e S: ds(s,$o) < r} is an open ball of radius r centered at SQ in S. The space (S, ds\sr) has a restricted metric ds\sr obtained by restricting the metric ds to the subset 5"": ds\s r = ds(s, s') Vs,s' e 5"".
The term "bounded continuous surface" refers to a continuous surface S for which supSis'es ds(s, s') is finite. S is referred to as a "compact surface" if each of its open covers has a finite sub-cover. IfS is compact it is necessary bounded.
The term "ε-isometry" refers to a mapping φ: S -> Q, for which for every q & Q there exists s e S such that dQ(q, φ(s)) ≤ ε.
The term "ε-isometric spaces" refers to spaces for which there exists an ε- isometry.
An image φ(5) of a space S in a space Q is referred to as an "ε-Hausdorff approximation" of S in Q if the mapping φ: S -> Q is an ε-isometry.
Properties of the Gromov-Hausdorff Distance
The Gromov-Hausdorff distance dGu (Equation 4) enjoys the following properties which make it useful for determining similarities between surfaces.
The Gromov-Hausdorff distance is a finite metric on the space M of continuous compact surfaces, in which a point is defined as an equivalence class of a continuous surface and its isometries. In other words, JGH satisfies the following axioms: (i) doniS, Q) ≥ 0; (ii) dGn(S, Q) if and only if S and Q are isometric; (iii) dQdβ, Q) = dGH(Q, S); and (iv) dGH(R, Q) ≤ dGn(R, S) + dGH(S, Q), also known as the triangle inequality, where S, Q and R are continuous surfaces. Another property of the Gromov-Hausdorff distance is the near-isometric similarity, which can be formulated as follows: (i) if dGu(S, Q) ≤ ε, where S, Q are continuous surfaces, then there exists a 2ε-Hausdorff approximation of S in Q. The converse version of the near-isometric similarity can be formulated as follows: (ii) if there exists an ε-Hausdorff approximation of S in Q, then dGu(S, Q) ≤ 2ε. An additional property of the Gromov-Hausdorff distance is sampling consistency which can be formulated as follows: Given Qr an r-covering of a continuous surface Q and S^ an r -covering of a continuous surface S,
r\
Thus, the Gromov-Hausdorff distance between sampled versions of two surfaces is close to the Gromov-Hausdorff distance between the original surfaces, and the discrepancy vanishes as the sample radius decreases.
The above properties of the Gromov-Hausdorff distance are known in the art and their proofs are found in many publications (to this end see, e.g., F. Memoli and G. Sapiro, "A theoretical and computational framework for isometry invariant recognition of point cloud data," University of Minnesota, IMA preprint 2004; and I. Borg and P. Groenen, "Modern multidimensional scaling - theory and applications," Springer- Verlag, Berlin Heidelberg New York, 1997].
Properties of the Partial Embedding Distance
The partial embedding distance dpE (Equations 2-3) of the present embodiments enjoys several properties which make it useful for determining similarities between surfaces. Intuitively, dpE measures the metric distortion obtained trying to embed the probe surface Q into the model surface S in the "most isometric" way. <ipE can be considered as a part of the Gromov-Hausdorff distance JQH-
As will be proven below, although dpE is not a metric per se it satisfies some of the metric axioms: (i) positive definiteness, CIPE(S,Q) > 0; (ii) minimality dpv(S,Q) = 0 if and only if Q is isometrically embeddable into S; (iii) and a non-symmetric version the triangle inequality dpE(S, Q) ≤ dpE(S,R) + dpE(R,Q), where S, Q and R are continuous surfaces.
The positive definiteness property follows directly from the definition of dpE since dis ψ > q.
The proof of the minimality property is as follows. If Q is isometrically embeddable into S, there exists a mapping ψ. Q → S with dis ψ = 0, which proves the first direction of this property. For the second direction, if dpE(S,Q) = 0 then there exists a sequence of mappings { ψn: Q — » S} satisfying dis ψn < Hn. Fix a countable dense set Q ' c Q. Using the Cantor diagonalization procedure, a subsequence |ψM ] can be chosen such that for every q' e Q } the sequence |ψn [q')j converges in S, where
Q ' c Q is a fixed countable dense set. Assume, without loss of generality, that such a sequence is { ψn) itself. A mapping ψ. Q' → S can then be defined as the point-wise limit of { ψn}, namely ψ(q') = limn→∞ψn(q') for every qx ^ Q \ Since \ds(ψn(q"), ψn(q')) - dQ(q", q')\ ≤ dis ψn → 0, it follows that:
for all q, q' e Q '. Thus, ψ is a distance-preserving embedding of Q ' into S. Since Q ' is dense, ψ can be extended to a distance-preserving embedding of the entire Q into S (see, for example, Proposition 1.5.9 in Burago et ah, "A course in metric geometry," American Mathematical Society, 2001). The proof of the triangle inequality property is as follows. Let dpE(R, Q) ~ §ι and dpE(S, R) = δ2. Then, there exist two sequences of mappings {<pn: Q -> R) and {ψn: R -> S}, satisfying dis φn < 2h\ + Hn and dis ψn ≤ 2δ2 + Hn. Denote by {ζn'- Q — > S} the sequence of the compositions ζ,, = ψ,, oφn . Invoking the triangle inequality for real numbers, one obtains: ds{ζMUS-dQ{q,q'^ds{ζMU^)hd R(ψM<Pn{^
<disφn +disψn (EQ. A3)
<2(δ1 +δ2) + -. n for all q, q' e Q '. This proves the triangle inequality property because dis ζn ≤ 2(δi + δ2 + \hi) and dPE(S, Q) ≤ δι + δ2. Q.E.D
Another property of dp^ is the near-isometric similarity, which can be formulated as follows: if there exist an ε-Hausdorff approximation of a continuous surface S in a continuous surface Q then d^iβ, Q) ≤ 2ε. This property follows straightforwardly from the fact that dpsiS, Q) < dan(S, Q). It can be shown that the converse version of this property also applies.
An additional property of JPE is the sampling consistency: given Q r an r'- covering of a continuous surface Q and Sr an /-'-covering of a continuous surface S, dPE(S, Q) - r ≤ dPB(Sr', Qr) ≤ dPE(S, Q) + r'. The proof of the sampling consistency property is presented below and is divided into two separate statements: (i) dpE(S, Q) - r ≤ dPE(S, Qr) < dPE(S, Q); and (ii) dPE(S, Q) ≤
Q) ≤ d?E(S, Q) + r\
The right side of inequality (i) is straightforward, since . (EQ. A4)
The left side of inequality (i) can be shown by invoking the triangle inequality property which yields dPE(S,Q) ≤ dpE(S,Qr) + dPE(Qr,Q). It remains to show that dpE(Qr,Q) ≤ r. Let the mapping ψ. Q -> Qr be defined in the following way: for
q e O/, ψ(q) = q, and for q e Q\Qr ψ(q) = argmin g< e Q dQ(q, #'). It is noted that the minimum exists because Qr can be replaced by its finite sub-covering. From the fact that O/ is an r-covering, it follows that dg(#, ψ{q)) ≤ r. Let q\, qi be two points in Q; if both qi and q2 are in Qr, then \dQ(qι, qi) - dQ(ψ(qι), K^))I == 0. If both ^1 and #2 are in Qr, invoking the triangle inequality one obtains |dρ(#i, qi) - dQ(ψ(q{), ψ{q2))\ ≤ dQ(q\, ψiqi)) + dQ(q2, ψiqij) ≤ 2r. Similarly, when only qx is in Qr, WQ(QU qi) ~ d<2(ψ(qι),
≤ r. Hence, dis ψ < r.
The left side of inequality (ii) is straightforward. To show the right side of inequality (ii), let ψ be a mapping ψ. Q -^ S. A new mapping ψ:Q-* Sr> is defined in the following way:
Since .S r' is an r'-covering of S, sup | ^β(^,g')-^(ψ(^Xψ(^'))l
< sup \ dQ(q,q')-ds(y(q),ψ(q')) \ +2r' (EQ- A6)
for all ψ, and consequently ^(5""', Q) < dPE(S, Q) + r\ Q.E.D. A further property of <ipε is referred to as the partial matching property, and is formulated as follows: given Qr an r-covering of a continuous surface Q, d?E(Qr, Q) - 0. The proof of this property follows directly from the fact that Qr is a sub-space of Q with a restricted metric d<^ς[(q, q') = dq(q, q') for all q, q' in Qr. Thus, there exists a trivial isometric embedding of 0/ into £λ It is noted that by virtue of this property (unlike the corresponding property of the Gromov-Hausdorff distance dGUQ, Qr) ≤ r), fifpE can be used for partial matching of surfaces.
Proof of Lemma 1
The proof of lemma 1 is presented below for the derivation of Jw (Equation 18). One of ordinary skill in the art would know how to derive Jd according to the similar principles. The proof of lemma 1 starts with the derivative with respect to w.
By Equation 14, x0 is a local minimum of £(xo;XA,yv) , hence, where g stands for the gradient Vx ^ . Adding an infinitesimal perturbation to w results in
g(xo;X,d,w+dw)≠O, yet there exists an infinitesimal perturbation dx0 , such that g(x0 + d&0 ; X, d, w + dyv) = 0. From the first-order Taylor expansion, it follows that: g(x0 + dx0 ; X, d, w + d\v) « g(x0 ; X, d, w)
+ H(xo;X,d,wVxo + Gj(xo;X5d,w>/w, (EQ' A?) where H is given in Equation 16. Substituting g(xo;X,d,w)= 0 and neglecting second- order terms, the following equation is obtained:
Jw can then be straightforwardly obtained from Equation A9. Q. E. D
It is appreciated that certain features of the invention, which are, for clarity, described in the context of separate embodiments", may also be provided in combination in a single embodiment. Conversely, various features of the invention, which are, for brevity, described in the context of a single embodiment, may also be provided separately or in any suitable subcombination.
Although the invention has been described in conjunction with specific embodiments thereof, it is evident that many alternatives, modifications and variations will be apparent to those skilled in the art. Accordingly, it is intended to embrace all such alternatives, modifications and variations that fall within the spirit and broad scope of the appended claims. All publications, patents and patent applications mentioned in this specification are herein incorporated in their entirety by reference into the specification, to the same extent as if each individual publication, patent or patent application was specifically and individually indicated to be incorporated herein by reference. In addition, citation or identification of any reference in this application shall not be construed as an admission that such reference is available as prior art to the present invention.
Claims
1. A method of determining similarity between a non-planar probe surface represented by a probe point-cloud and a non-planar model surface represented by a model point-cloud, the method comprising: calculating a extremal value of an objective function describing embedding of said probe surface into an embedding space having a non-constant sectional curvature; and determining similarity between the probe surface and the model surface based on said extremal value.
2. A method of searching a library of non-planar model surfaces for at least a partial match to a non-planar probe surface, the non-planar probe surface being represented by a probe point-cloud and each non-planar model surface being represented by a model point-cloud, the method comprising, for each non-planar model surface, performing the method of claim 1 so as to determine similarity between the probe surface and said model surface, thereby determining the presence or absence of at least a partial match.
3. The method of claim 1 or 2, further comprising calculating a matrix of geodesic distances corresponding to the probe point-cloud.
4. The method of claim 1 or 2, wherein said calculation of said extremal value comprises multiresolution optimization.
5. The method of claim 1 or 2, further comprising selecting a subset of the probe point-cloud corresponding to a predetermined patch of the probe surface, wherein said calculation of said extremal value is based on said subset.
6. Apparatus for determining similarity between a non-planar probe surface represented by a probe point-cloud and a non-planar model surface represented by a model point-cloud, the method comprising: a extremal value calculator, for calculating a extremal value of an objective function describing embedding of said probe surface into an embedding space having a non-constant sectional curvature; and a similarity determinator operable to determine the similarity between the probe surface and the model surface based on said extremal value.
7. The apparatus of claim 6, further comprising a matrix calculator for calculating a matrix of geodesic distances corresponding to the probe point-cloud.
8. The method or apparatus of claim 3 or 7, wherein said geodesic distances are calculated using the fast marching method.
9. The apparatus of claim 6, wherein said extremal value calculator is operable to employ multiresolution optimization.
10. The apparatus of claim 6, further comprising a subset selector for selecting a subset of the probe point-cloud corresponding to a predetermined patch of the probe surface, wherein said extremal value calculator is operable to use said subset for calculating said extremal value.
11. The apparatus or method of claim 10, wherein for each pair of points in said subset, the difference between a geodesic distance characterizing said pair in said point-cloud and a geodesic distance characterizing said pair in said patch is higher than a predetermined threshold.
12. The apparatus or method of claim 10, wherein for each pair of points in said subset, the sum of geodesic distances in said patch from each point in said pair to a boundary of said patch is higher than a geodesic distance characterizing said pair in said point-cloud.
13. The apparatus or method of claim 1, 2 or 6, wherein said objective function is selected so as to allow calculation of a partial embedding distance between the probe surface and the model surface.
14. The apparatus or method of claim 1, 2 or 6, wherein said objective function is selected so as to allow calculation of a Gromov-Hausdorff distance between the probe surface and the model surface.
15. The method or apparatus of claim I5 2 or 6, wherein said embedding space is a two-dimensional space.
16. The method or apparatus of claim 1, 2 or 6, wherein said embedding space is a polyhedral approximation of the model surface.
17. The method or apparatus of claim 16, wherein said polyhedral approximation comprises a triangulated manifold.
18. The method of claim 16, further comprising constructing said polyhedral approximation from the model point-cloud.
19. The apparatus of claim 16, further comprising a polyhedron constructor for constructing said polyhedral approximation from the model point-cloud.
20. The method or apparatus of claim 1, 2 or 6, wherein said objective function comprises a sum of local distortions, each characterizing embedding of a pair of points in the probe point-cloud into said embedding space.
21. The method or apparatus of claim 20, wherein said sum is a weighted sum.
22. The method or apparatus of claim 20, wherein at least a few of said local distortions in said sum comprise a function of the difference between a first geodesic distance defined in the probe surface and a second geodesic distance, corresponding to said first geodesic distance and being defined in said embedding space.
23. The method or apparatus of claim 22, wherein said first geodesic distance characterizes a pair of points of the probe point-cloud and said second geodesic distance characterizes images of said pair of points.
24. The method or apparatus of claim 22, wherein said first geodesic distance characterizes a pair of points of the embedding space and said second geodesic distance characterizes images of said pair of points.
25. The method or apparatus of claim 22, wherein said first geodesic distance is a distance between a first point of the probe point-cloud and an image of a first point of said embedding space, and said second geodesic distance is a distance between a second point of said embedding space and an image of a second point of the probe point-cloud.
26. The method of claim 16, wherein said calculation of said extremal value comprises interpolating geodesic distances in said polyhedral approximation of the model surface.
27. The apparatus of claim 16, wherein said extremal value calculator comprises an interpolator for interpolating geodesic distances in said polyhedral approximation of the model surface.
28. The method or apparatus of claim 17, wherein said extremal value is calculated by an iterative procedure.
29. The method or apparatus of claim 28, wherein said iterative procedure traverses said triangulated manifold while ensuring that images of points in the point- clouds are always in a plane of one triangle of said triangulated manifold.
30. The method or apparatus of claim 29, wherein a direction of said traverse is determined by a procedure selected from the group consisting of the steepest decent method, the fixed step gradient method, the conjugate gradient method, the Newton method and any combination thereof.
31. The method or apparatus of claim 22, wherein said function of said difference between said first geodesic distance and said second geodesic distance is a non-linear function.
32. The method or apparatus of claim 31, wherein said non-linear function comprises finite norms of said local distortions.
33. The method or apparatus of claim 31, wherein said non-linear function is constrained by a constraint in an infinite norm representation.
34. The method of claim 33, further comprising employing penalty barrier method for de-constraining said non-linear function.
35. The method or apparatus of claim 1, 2 or 6, wherein said probe surface and said model surface represent faces.
Priority Applications (4)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
AT06821644T ATE507537T1 (en) | 2005-12-16 | 2006-12-17 | METHOD AND DEVICE FOR DETERMINING SIMILARITY BETWEEN SURFACES |
US12/086,623 US8280150B2 (en) | 2005-12-16 | 2006-12-17 | Method and apparatus for determining similarity between surfaces |
DE602006021627T DE602006021627D1 (en) | 2005-12-16 | 2006-12-17 | METHOD AND DEVICE FOR DETERMINING THE SIMILARNESS BETWEEN SURFACES |
EP06821644A EP1969523B1 (en) | 2005-12-16 | 2006-12-17 | Method and apparatus for determining similarity between surfaces |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US75083705P | 2005-12-16 | 2005-12-16 | |
US60/750,837 | 2005-12-16 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2007069260A1 true WO2007069260A1 (en) | 2007-06-21 |
Family
ID=37846983
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/IL2006/001451 WO2007069260A1 (en) | 2005-12-16 | 2006-12-17 | Method and apparatus for determining similarity between surfaces |
Country Status (5)
Country | Link |
---|---|
US (1) | US8280150B2 (en) |
EP (1) | EP1969523B1 (en) |
AT (1) | ATE507537T1 (en) |
DE (1) | DE602006021627D1 (en) |
WO (1) | WO2007069260A1 (en) |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US8280150B2 (en) | 2005-12-16 | 2012-10-02 | Technion Research & Development Foundation Ltd. | Method and apparatus for determining similarity between surfaces |
CN106267774A (en) * | 2015-05-25 | 2017-01-04 | 腾讯科技(深圳)有限公司 | Moving state identification method and apparatus |
US10970655B2 (en) | 2010-05-03 | 2021-04-06 | Technion Research & Development Foundation Ltd | Surgery planning based on predicted results |
Families Citing this family (14)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6947579B2 (en) * | 2002-10-07 | 2005-09-20 | Technion Research & Development Foundation Ltd. | Three-dimensional face recognition |
WO2009124139A1 (en) * | 2008-04-01 | 2009-10-08 | The Government Of The United States Of America, As Represented By The Secretaty Of The Navy | Methods and systems of comparing face models for recognition |
US20130229412A1 (en) * | 2010-12-03 | 2013-09-05 | Sony Corporation | 3d data analysis apparatus, 3d data analysis method, and 3d data analysis program |
US9324040B2 (en) * | 2013-01-30 | 2016-04-26 | Technion Research & Development Foundation Limited | Training ensembles of randomized decision trees |
WO2014178051A2 (en) * | 2013-04-30 | 2014-11-06 | Mantisvision Ltd. | Selective 3d registration |
US9438412B2 (en) * | 2014-12-23 | 2016-09-06 | Palo Alto Research Center Incorporated | Computer-implemented system and method for multi-party data function computing using discriminative dimensionality-reducing mappings |
EP3307142B1 (en) | 2015-06-10 | 2021-03-17 | Tyto Care Ltd. | Apparatus and method for inspecting skin lesions |
EP3280075A1 (en) * | 2016-08-03 | 2018-02-07 | Rohde & Schwarz GmbH & Co. KG | Measurement system and a method |
CN108549873B (en) * | 2018-04-19 | 2019-12-24 | 北京华捷艾米科技有限公司 | Three-dimensional face recognition method and three-dimensional face recognition system |
JP2022539843A (en) * | 2019-07-08 | 2022-09-13 | ウェイモ エルエルシー | Object detection in point clouds |
CN112102377B (en) * | 2020-08-04 | 2023-02-03 | 广东工业大学 | Chebyshev-based ICP point cloud global optimal registration method and device |
JP7528383B2 (en) | 2021-03-11 | 2024-08-05 | グーグル エルエルシー | System and method for training a model for predicting dense correspondences in images using geodesic distances - Patents.com |
CN117422710A (en) * | 2023-12-14 | 2024-01-19 | 泉州师范学院 | Point cloud geodesic measuring method |
CN117544963B (en) * | 2024-01-04 | 2024-03-26 | 四川大学 | Method and equipment for identifying radiation source of cross-mode communication signal based on FTGan-Yolo |
Family Cites Families (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6807290B2 (en) * | 2000-03-09 | 2004-10-19 | Microsoft Corporation | Rapid computer modeling of faces for animation |
US6947579B2 (en) * | 2002-10-07 | 2005-09-20 | Technion Research & Development Foundation Ltd. | Three-dimensional face recognition |
EP1969523B1 (en) | 2005-12-16 | 2011-04-27 | Technion Research and Development of Foundation, Ltd. | Method and apparatus for determining similarity between surfaces |
WO2007119870A1 (en) * | 2006-04-14 | 2007-10-25 | Nec Corporation | Checking device and checking method |
-
2006
- 2006-12-17 EP EP06821644A patent/EP1969523B1/en active Active
- 2006-12-17 US US12/086,623 patent/US8280150B2/en active Active
- 2006-12-17 WO PCT/IL2006/001451 patent/WO2007069260A1/en active Application Filing
- 2006-12-17 AT AT06821644T patent/ATE507537T1/en not_active IP Right Cessation
- 2006-12-17 DE DE602006021627T patent/DE602006021627D1/en active Active
Non-Patent Citations (2)
Title |
---|
ALEXANDER M BRONSTEIN ET AL: "Three-Dimensional Face Recognition", INTERNATIONAL JOURNAL OF COMPUTER VISION, KLUWER ACADEMIC PUBLISHERS, BO, vol. 64, no. 1, 1 August 2005 (2005-08-01), pages 5 - 30, XP019216473, ISSN: 1573-1405 * |
ELAD A ET AL: "On bending invariant signatures for surfaces", IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, IEEE, NEW YORK, NY, US, vol. 25, no. 10, October 2003 (2003-10-01), pages 1285 - 1295, XP011101853, ISSN: 0162-8828 * |
Cited By (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US8280150B2 (en) | 2005-12-16 | 2012-10-02 | Technion Research & Development Foundation Ltd. | Method and apparatus for determining similarity between surfaces |
US10970655B2 (en) | 2010-05-03 | 2021-04-06 | Technion Research & Development Foundation Ltd | Surgery planning based on predicted results |
CN106267774A (en) * | 2015-05-25 | 2017-01-04 | 腾讯科技(深圳)有限公司 | Moving state identification method and apparatus |
CN106267774B (en) * | 2015-05-25 | 2019-05-24 | 腾讯科技(深圳)有限公司 | Moving state identification method and apparatus |
Also Published As
Publication number | Publication date |
---|---|
EP1969523B1 (en) | 2011-04-27 |
ATE507537T1 (en) | 2011-05-15 |
EP1969523A1 (en) | 2008-09-17 |
US20090028442A1 (en) | 2009-01-29 |
DE602006021627D1 (en) | 2011-06-09 |
US8280150B2 (en) | 2012-10-02 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
EP1969523B1 (en) | Method and apparatus for determining similarity between surfaces | |
Guo et al. | 3D object recognition in cluttered scenes with local surface features: A survey | |
Petitjean | A survey of methods for recovering quadrics in triangle meshes | |
Bronstein et al. | A Gromov-Hausdorff framework with diffusion geometry for topologically-robust non-rigid shape matching | |
Lucchese et al. | A frequency domain technique for range data registration | |
EP2048599B1 (en) | System and method for 3D object recognition | |
Tazir et al. | CICP: Cluster Iterative Closest Point for sparse–dense point cloud registration | |
JP2008547094A (en) | Recognition system and method for 2D images using 3D class model | |
CN101369309A (en) | Human ear image normalization method based on active apparent model and outer ear long axis | |
Davies et al. | Advanced methods and deep learning in computer vision | |
Salah et al. | Registration of three-dimensional face scans with average face models | |
Kakadiaris et al. | 3D Face Recognition. | |
Paradowski et al. | Local keypoints and global affine geometry: Triangles and ellipses for image fragment matching | |
Davies | The dramatically changing face of computer vision | |
Ahmed et al. | Point cloud registration using virtual interest points from Macaulay’s resultant of quadric surfaces | |
Hong et al. | Database-guided simultaneous multi-slice 3D segmentation for volumetric data | |
CN112767457A (en) | Principal component analysis-based plane point cloud matching method and device | |
WO2006061365A1 (en) | Face recognition using features along iso-radius contours | |
Seeger et al. | Feature extraction and registration: An overview | |
SANDOVAL et al. | Robust sphere detection in unorganized 3D point clouds using an efficient Hough voting scheme based on sliding voxels | |
Sa et al. | Depth grid-based local description for 3D point clouds | |
Sintunata et al. | Skewness map: estimating object orientation for high speed 3D object retrieval system | |
Unel et al. | 3D object recognition using invariants of 2D projection curves | |
Ahmed et al. | Point cloud registration with virtual interest points from implicit quadric surface intersections | |
Guizilini et al. | Iterative continuous convolution for 3d template matching and global localization |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application | ||
WWE | Wipo information: entry into national phase |
Ref document number: 12086623 Country of ref document: US |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
WWE | Wipo information: entry into national phase |
Ref document number: 2006821644 Country of ref document: EP |
|
WWP | Wipo information: published in national office |
Ref document number: 2006821644 Country of ref document: EP |