Classifications

• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
  • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
  • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
  • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
  • G06V40/168—Feature extraction; Face representation
  • G06V40/169—Holistic features and representations, i.e. based on the facial image taken as a whole
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F18/00—Pattern recognition
  • G06F18/20—Analysing
  • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
  • G06F18/213—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
  • G06F18/2134—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods based on separation criteria, e.g. independent component analysis
  • G06F18/21342—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods based on separation criteria, e.g. independent component analysis using statistical independence, i.e. minimising mutual information or maximising non-gaussianity

Definitions

• the present invention relates generally to image recognition.
• the present invention relates to a logic arrangement, data structure, system and method for acquiring data, and more particularly to a logic arrangement, data structure, system and method for acquiring data describing at least one characteristic of an object, synthesizing new data, recognizing acquired data and reducing the amount of data describing one or more characteristics of the object (e.g., a human being).
• the PCA technique encodes pairwise relationships between pixels, the second-order statistics, correlational structure of the training image ensemble, but it ignores all higher-order pixel relationships, the higher-order statistical dependencies.
• a generalization of the PCA technique known as independent component analysis ("ICA") technique learns a set of statistically independent components by analyzing the higher-order dependencies in the training data in addition to the correlations. See A. Hyvarinen et al., Independent Component Analysis. Wiley, New York, 2001, which is hereby incorporated by this reference.
• the ICA technique does not distinguish between higher-order statistics that rise from different factors inherent to an image formation — factors pertaining to scene structure, illumination and imaging.
• the ICA technique has been employed in face recognition and, like the
• One of the objects of exemplary embodiments of the present invention is to overcome the above-described deficiencies.
• Another object of the present invention is to provide a method, system, storage medium, and data structure for generating an object descriptor.
• such method can include steps of computing a response of an image to a basis tensor, flattening the image response, extracting a coefficient vector from the image response, and comparing the extracted coefficient to a plurality of different parameters stored in rows of a coefficient matrix.
• a computer system can be provided which includes a storage arrangement (e.g., a memory), and a processor which is capable of receiving data associated with an image, and provided in communication with the storage arrangement.
• the storage arrangement can store computer-executable instructions for performing a method of processing data. For example, a response of an image to a basis tensor may be determined, the image response can be flattened, a coefficient vector can be extracted from the image response, and the extracted coefficient may be compared to a plurality of different parameters stored in rows of a coefficient matrix.
• a computer-readable medium having stored thereon computer-executable instructions for performing a method.
• the method includes steps of computing a response of an image to a basis tensor, flattening the image response, extracting a coefficient vector from the image response, and comparing the extracted coefficient to a plurality of different parameters stored in rows of a coefficient matrix.
• a method of processing data includes steps of applying a multilinear independent component analysis to image data to create a factorial code; and generating a representation of the data having a plurality of sets of coefficients that encode people, viewpoints, and illuminations, wherein each set is statistically independent.
• a data structure having set of coefficient vectors for a target image.
• the vectors include an identifier of the target, a viewpoint of the target, and an illumination direction of the target.
• Figure IA illustrates exemplary Principal Components Analysis ("PCA”) eigenvectors (e.g., eigenfaces), which are the principal axes of variation across all images;
• PCA Principal Components Analysis
• Figure IB shows a partial visualization of an exemplary "TensorFaces" representation
• Figure 2A illustrates independent components for image data which can be used with the exemplary embodiment of the present invention
• FIG. 2B shows a partial visualization of an exemplary multilinear independent component analysis ("MICA") representation which can be used with the exemplary embodiment of the present invention
• Figure 3A shows an image representation of the set of coefficient vectors used by the either by "TensorFaces” or MICA representation to represent a particular image which can be used with the exemplary embodiment of the present invention
• Figure 3B shows an image response of a new image to a basis tensor
• Figure 4 A shows the full set of images for one of the subjects with viewpoints arrayed horizontally and illuminations arrayed vertically which can be used with the exemplary embodiment of the present invention
• Figure 4B shows a scan of subjects from which the images in Figure 4A were rendered
• Figure 5 shows a block diagram of one exemplary embodiment of a computer system that performs the MICA procedure according to the present invention
• Figure 6 shows a block diagram of one exemplary embodiment of a method of processing image data using a multilinear independent component analysis according to the present invention.
• FIGS. 7A-7F are block diagrams of sample tensors and equivalent mode-1, mode-2 and mode-3 flattened tensors according to an exemplary embodiment of the present invention.
• the same reference numerals and characters, unless otherwise stated, are used to denote like features, elements, components or portions of the illustrated embodiments.
• the present invention will now be described in detail with reference to the figures, it is done so in connection with the illustrative embodiments. It is intended that changes and modifications can be made to the described embodiments without departing from the true scope and spirit of the subject invention as defined by the appended claims.
• Exemplary embodiments of the present invention relate to and may utilize a nonlinear, multifactor model of image ensembles that generalizes conventional ICA technique.
• the ICA technique employs linear (matrix) algebra
• the exemplary embodiment of a Multilinear ICA ("MICA") procedure according to the present invention uses multilinear (tensor) algebra.
• the MICA procedure is able to learn the interactions of multiple factors inherent to image formation and separately encode the higher order statistics of each of these factors.
• the MICA procedure can also encode higher order dependencies associated with the different factors.
• the multilinearICA procedure of the exemplary embodiment of the present invention can be understood in the context of the mathematics of PCA, multilinearPCA, and ICA techniques.
• the principal component analysis of an ensemble of 7 2 images can be determined by performing a singlularvalue decomposition ("SVD") on a /] x/ 2 data matrix D whose columns are the "vectored” I ⁇ -pixel "centered” images.
• Figure 1 A illustrates the exemplary PCA eigenvectors 300 (eigenfaces), which are the principal axes of variation across all images.
• Figure IB shows a partial visualization of the 75 x 6 x 6 x 8560
• Each vectored-centered image can be obtained by subtracting the mean image of the ensemble from each input image, and likely identically arranging the resulting pixels into a column vector.
• the matrix D e IR 1 x ⁇ is a two-mode mathematical object that has two associated vector spaces, a row space and a column space.
• ⁇ ⁇ 2 > ... > ⁇ p > 0 the singular values of D, and an orthogonal row space represented by the right matrix Ve lR 2 ⁇ 2 .
• the eigenvectors U are also called the
• Principal component (or Karhunen-Loeve) directions of D.
• Exemplary optimal dimensionality reduction in matrix principal component analysis can be obtained by a truncation of the singular value decomposition (i.e., deleting eigenvectors associated with the smallest singular values).
• the analysis of an ensemble of images resulting from the confluence of multiple factors related to scene structure, illumination, and viewpoint is a problem of multilinear algebra. See M.A.O. Vasilescu et al., "Multilinear analysis of image ensembles: Tensorfaces," In Proc. European Conf. on Computer Vision (ECCV 2002), pp. 447-460, Copenhagen, Denmark, May 2002.
• the image ensemble can be represented as a higher-order tensor.
• This image data tensor ⁇ D should be decomposed in order to separate and parsimoniously represent the constituent factors.
• an N-mode SVD procedure may be used as a multilinear extension of the above-mentioned conventional matrix SVD technique.
• a tensor also known as an n-way array or multidimensional matrix or n-mode matrix, is a higher order generalization of a vector (first order tensor) and a matrix (second order tensor).
• first order tensor first order tensor
• second order tensor second order tensor
• a tensor can be defined as a multi-linear mapping over a set of vector spaces.
• the tensor can be represented in the following manner: ⁇ e IR 1 ** 1 **-* 1 " t w ere ⁇ is a tensor.
• the order of the tensor ⁇ is N.
• a tensor can be formed by a group of primitives. Each primitive is a set of mode vectors, such that a first primitive is a set of mode- 1 vectors, a second vector is a set of mode-2 vectors, an n th primitive is a set of mode-n vectors, etc.
• the primitives can be row vectors of a matrix, column vectors of a matrix, index of a vector, etc.
• An element of tensor ⁇ can be denoted as ⁇ l- ⁇ .. l ⁇ n.. l •N or
• Scalars are denoted by lower case letters (a, b, %), vectors by bold lower case letters (a, b %), matrices by bold upper-case letters (A, B %), and higher-order tensors by italicized bolded upper-case letters (A, B %) or uppercase caligraphic letters ( ⁇ , (B.).
• column vectors can be referred to as mode-1 vectors, and row vectors are referred to as mode-2 vectors.
• Mode-n vectors of an ⁇ th order tensor ⁇ e IR l 2X ' X N are the J,-dimensional vectors obtained from the tensor ⁇ by varying index iiza while maintaining the other indices as fixed.
• the mode- vectors are the column vectors of matrix A ⁇ ⁇ / j i/i -J- i "*'-- / " ) that can result from flattening the tensor ⁇ , as shown in Figures 12A - 12F.
• the flattening procedure shall be described in further detail below.
• Figures 7 A - 7C illustrate third order tensors 1200, 1210, 1220, respectively, each having dimensions I] x I x I 3 .
• Figure 7D shows the third order tensor 1200 after having been mode-1 flattened to obtain a matrix 1250 containing mode-1 vectors of the third order tensor 1200.
• the third order tensor 1200 of Fig. 12A is a cube type structure, while the matrix 1250 is a two dimensional type structure having one index, i.e., I , embedded (to a certain degree) within the matrix 1250.
• Figure 7E shows a matrix 1260 containing mode-2 vectors of the third order tensor 1210 after it has been mode-2 flattened.
• This third order tensor 1210 is a cube type structure, while the matrix 1260 is a two dimensional type structure having one index, e.g., I 3 , imbedded (to a certain degree) with the data.
• Figure 7F shows the third order tensor 1220 after having been mode-3 flattened to obtain a matrix 1270 containing mode-3 vectors of the third order tensor 1220.
• Such third order tensor 1220 is a cube type structure, while the matrix 1270 organizes is a two dimensional type structure having one index, e.g., L, imbedded (to a certain degree) with the data.
• a generalization of the product of two matrices can be the product of the tensor and matrix.
• the mode-n product of a tensor and a matrix is a special case of the inner product in multilinear algebra and tensor analysis.
• the mode-n product is often denoted using Einstein summation notation, but for purposes of clarity, the mode-n product symbol can be used.
• the mode-n product may have the following properties:
• a singular value decomposition can be expressed as a rank decomposition as is shown in the following simple example:
• an SVD can be a combinatorial orthogonal rank decomposition, but that the reverse is not true.
• rank decomposition is not necessarily singular value decomposition.
• the N-mode SVD can be expressed as an expansion of mutually orthogonal rank-1 tensors, as follows:
• Tensor Z known as the core tensor, is analogous to the diagonal singular value matrix in conventional matrix SVD (although it does not have a simple, diagonal structure).
• the core tensor governs the interaction between the mode matrices Ui,..., U N - Mode matrix U jail contains the orthonormal vectors spanning the column space of matrix D( n) resulting from the mode-n flattening of ⁇ .
• the ICA technique can be applied in two exemplary ways: Architecture I applies the ICA technique to D r , each of whose rows is a different vectorized image, and finds a spatially independent basis set that reflects the local properties of faces. On the other hand, architecture II applies the ICA technique to D and finds a set of coefficients that are statistically independent while the basis reflects the global properties of faces.
• the ICA technique starts essentially from the factor analysis or PCA solution shown in equation (1) of a pre-whittened data set, and computes a rotation of the principal components such that they become independent ⁇ components. See J. Friedman et al., "The Elements of Statistical Learning: Data Mining, Inference, and Prediction," Springer, New York, 2001.
• Figure 2A illustrates independent components C p j xe i s for image data 320.
• Various objective functions such as those based on mutual information, negentropy, higher- order cumulants, etc., can be presented using conventional procedures for computing the independent components along with different optimization methods for extremizing these objective functions.
• Dimensionality reduction with the ICA technique is usually performed in the PCA preprocessing stage.
• a mathematically equivalent way of representing the above expressions that will serve better in the next section is, as follows:
• ICA (UW 7 )(W 7 ⁇ V 7 ) (7) CK.
• ICA can be applied to D and it rotates the principal components directions such that the coefficients are statistically independent, as follows:
• Figure 5 shows a block diagram of an exemplary embodiment of a computer system 100 according to the present invention which may be used to implement the MICA procedure according to the present invention.
• the system 100 can include an image capturing system 120 in communication with a processor 130.
• the system may further include a storage arrangement (e.g., a memory 110) in communication with t ie-processor 130.
• the memory 110 may store a MICA application 112 that includes compute-executable instructions that are capable of being executed by the processor 130 on data captured by the image capturing system 120.
• the memory 110 may further store an image data base used to compare image data processed according to the MICA procedure.
• Figure 6 shows an exemplary embodiment of a flow chart of the method 200 of the present invention.
• a response of an image to a basis tensor is computed (step 210).
• the image response is flattened (step 220).
• a coefficient vector may be extracted from the image response (step 230).
• the extracted coefficient may be compared to a plurality of different parameters stored in coefficient rows of a matrix (step 240).
• the exemplary MICA procedure may be implemented in the exemplary embodiments of the present invention.
• architecture I can apply the MICA procedure, and may result in a factorial code. It can locate a representation in which each set of coefficients that encodes people, viewpoints, illuminations, etc., is statistically independent.
• Architecture II finds a set of independent bases across people, viewpoints, illuminations, etc.
• N-mode ICA U n W .
• a multilinear ICA decomposition can be performed of the tensor ⁇ of vectored training images ⁇ d ,
• each of the mode matrices contains a set of statistically independent coefficients
• architecture II yields B, a set of independent bases across people, viewpoints, illuminations, etc.
• MICA has the same mathematical form as in equation (15). However, the core tensor S and mode matrices C ⁇ ...C N are computed according to equations (9)-(l l). This architecture results in a set of basis vectors that are statistically independent across the different modes.
• "training" facial images dj is decomposed into a reduced-dimensional basis matrix B PCA , and a matrix C containing a vector of coefficients c, / associated with each vectored image d d .
• the first coefficient vector encodes the person's identity, the second encodes the view point, the third encodes the illumination direction, etc.
• Figure 3B shows an image response 350 of a new image to a basis tensor.
• the image response to the basis tensor ⁇ 2 Sx 4 C p ⁇ xe ⁇ s can be computed as
• R ⁇ p ⁇ xels (C(pi X els)S(p lxe ls)) d n ⁇ w . (17)
• ⁇ is a multi-modal response to the different factors that make up the image.
• This tensorial response has a particular structure shown in Figure 3B, which can be exploited in order to extract the people parameters associated with d new .
• this image was constructed from the following unknown set of coefficient vectors: an illumination coefficient vector c / , a view point coefficient vector c legally, and a person coefficient vector c p .
• the face recognition problem is the classification of the computed person coefficient vector.
• R ⁇ e ople ..l ⁇ V n C p ...
• the image response ⁇ 2 ⁇ can be reorganized as a matrix whose columns are multiples of the people parameters c p in equation (18).
• the reorganization of the ' image responses is achieved by flattening along the people mode.
• the matrix R ⁇ pe0p i e) has rank 1, it's column are multiples of c ⁇ Therefore, the people rank of ⁇ is 1 ; hence, an SVD of R ⁇ peo pi e) can extract c p .
• a flattening along the viewpoint mode or the illumination mode etc. results in the matrices R( vie wpoints) and can be observed whose columns are multiples of c v and c respectively.
• These coefficient vectors, the viewpoint coefficient vector, the illumination coefficient vector etc. are extracted by computing a singular value decomposition on the respective matrices.
• Figure 4A shows the full set 360 of 225 exemplary images for one of the subjects with viewpoints arrayed horizontally and illuminations arrayed vertically.
• the images boxed may be used for training.
• the image set 360 can be rendered from a 3D scan 370 of the subject shown as the top left scan in Figure 4B.
• the 75 scans shown in Figure 4B may be recorded using a CyberwareTM 3030PS laser scanner and are part of the 3D morphable faces database created at the University of Dortmund.
• the independent Component Analysis (“ICA") technique may minimize the statistical dependence of the representational components of a training image ensemble.
• the ICA technique generally is not able to distinguish between the different factors related to scene structure, illumination and imaging, which are inherent to image formation.
• a nonlinear, multifactor ICA procedure according to the present invention can be utilized (as described above) that generalizes the ICA technique.
• the exemplary Multilinear ICA ("MICA") procedure according to the present invention of image ensembles can learn the statistically independent components of multiple factors.
• the MICA procedure according to the present invention generally may exploit multilinear (tensor) algebra.

Applications Claiming Priority (2)

Publications (1)

Family

ID=34794392

Family Applications (1)

Country Status (5)

Families Citing this family (14)

Family Cites Families (67)

• 2005
  • 2005-01-13 US US11/571,341 patent/US7693299B2/en not_active Expired - Fee Related
  • 2005-01-13 WO PCT/US2005/001671 patent/WO2005067572A2/en active Application Filing
  • 2005-01-13 JP JP2006549695A patent/JP2007518199A/ja active Pending
  • 2005-01-13 CA CA002553452A patent/CA2553452A1/en not_active Abandoned
  • 2005-01-13 EP EP05705897A patent/EP1709572A2/de not_active Withdrawn

Non-Patent Citations (1)

Also Published As

Similar Documents

Legal Events