WO2005013066A2 - Agencement logique, structure de donnees, systeme et procede permettant une representation multilineaire d'ensembles de donnees combinees aux fins de synthese, de rotation et de compression - Google Patents

Agencement logique, structure de donnees, systeme et procede permettant une representation multilineaire d'ensembles de donnees combinees aux fins de synthese, de rotation et de compression Download PDF

Info

Publication number
WO2005013066A2
WO2005013066A2 PCT/US2004/024000 US2004024000W WO2005013066A2 WO 2005013066 A2 WO2005013066 A2 WO 2005013066A2 US 2004024000 W US2004024000 W US 2004024000W WO 2005013066 A2 WO2005013066 A2 WO 2005013066A2
Authority
WO
WIPO (PCT)
Prior art keywords
data elements
viewpoint
primitives
tensor
descriptors
Prior art date
Application number
PCT/US2004/024000
Other languages
English (en)
Other versions
WO2005013066A3 (fr
Inventor
Manuela O. Vasilescu
Demetri Terzopoulos
Original Assignee
New York University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by New York University filed Critical New York University
Publication of WO2005013066A2 publication Critical patent/WO2005013066A2/fr
Priority to US11/200,479 priority Critical patent/US7379925B2/en
Publication of WO2005013066A3 publication Critical patent/WO2005013066A3/fr

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating 3D models or images for computer graphics
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
    • G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
    • G06V10/76—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries based on eigen-space representations, e.g. from pose or different illumination conditions; Shape manifolds

Definitions

  • the present invention relates generally to a logic arrangement, data structure, system and method for acquiring and manipulating data, and more particularly to a logic arrangement, data structure, system and method for acquiring and manipulating data describing the surface appearance of an object using at least one characteristic of the object, synthesizing new data, rotating an image of the object and reducing the amount of data describing one or more characteristics of the object (e.g., a group of coins or an ear of corn).
  • One of the objects of the present invention is to provide a logic arrangement, data structure, storage medium, system and method for generating an object descriptor.
  • such data structure can include a plurality of first data elements that have information regarding at least one characteristic of the at least one object.
  • the information of the first data elements is capable of being used to obtain the object descriptor.
  • the object descriptor is related to the at least one characteristic and a further characteristic of the at least one object, and is capable of being used to generate a plurality of second data elements which contain information regarding the further characteristic of the at least one object based on the object descriptor.
  • Each of the at least one object is one of an identity of an object, a viewpoint, an illumination, and a pixel.
  • the data structure for identifying a sample object based upon a sample object descriptor can include a plurality of first data elements including information defined by at least two first primitives.
  • the first data elements capable of being used to obtain at least one of a plurality of object descriptors.
  • a plurality of second data elements including information which is defined by at least two second primitives.
  • the second data elements are capable of being used to obtain the sample object descriptor, and wherein the at least one of the object descriptors are configured to be compared to the sample object descriptor for determining whether the sample object descriptor is potentially identifiable as one of the object descriptors.
  • Each of the plurality of object descriptors is associated with a respective one of a plurality of objects, wherein the sample object is one of an identity of an object, a viewpoint, an illumination, and a pixel.
  • a method for reducing a dimensionality of one of at least two object descriptors collects a plurality of data elements which are defined by at least two primitives, and obtains the one of the object descriptors based on the information of the data elements. The method also reduces the dimensionality of the one of the object descriptors, wherein each of the object descriptors except for the one of the object descriptors having the reduced dimensionality maintain full dimensionality.
  • the one of the object descriptors is one of an identity of an object, a viewpoint, an illumination, and a pixel.
  • the data structure is adapted for generating an object descriptor.
  • the data structure includes a plurality of data elements which are defined by at least two primitives.
  • the information related to the data elements is capable of being used to obtain the object descriptor using an orthonormal decomposition procedure.
  • the object descriptor is one of an identity of an object, a viewpoint, an illumination, and a pixel.
  • FIG. 1 is a block diagram of a data analysis system according to an exemplary embodiment of the present invention
  • Fig. 2 is a flow diagram of an exemplary embodiment of a process according to the present invention which analyzes multilinear data
  • Fig. 3 is a flow diagram of an exemplary embodiment of a core tensor computation procedure of the process of Fig. 2 which performs an N-mode SVD algorithm for decomposing an N-dimensional tensor;
  • Fig. 4 is a flow diagram of an exemplary embodiment of a process of Fig. 2 which synthesizes the remaining viewpoints for an object;
  • Fig. 6 is a flow diagram of an exemplary embodiment of an object recognition procedure of the process of Fig. 2 which recognizes an unidentified object from a known viewpoint as one of a group of objects
  • Fig. 7 is a flow diagram of an exemplary embodiment of a viewpoint recognition procedure of the process of Fig. 2 which recognizes an unknown viewpoint of a known object
  • Fig. 8 is a flow diagram of another exemplary embodiment of a process according to the present invention which analyzes multilinear data
  • Fig. 9 is a flow diagram of an exemplary embodiment of the object recognition procedure of the process of Fig. 8 which recognizes an unidentified object given an unknown object image;
  • Fig. 10 is a flow diagram of an exemplary embodiment of the viewpoint recognition procedure of the process of Fig. 8 which recognizes of an unidentified viewpoint of a known object;
  • Fig. 11 is a flow diagram of an exemplary embodiment of a data reduction process of the process of Fig. 8 which dimensionally reduces the amount of data describing an object displaying an viewpoint;
  • Figs. 12A -12F are block diagrams of sample tensors and equivalent mode-1, mode-2 and mode-3 flattened tensors according to an exemplary embodiment of the present invention;
  • Fig. 13 is a flow diagram of another exemplary embodiment of a process according to the present invention which analyzes multilinear data
  • Fig. 14 is a flow diagram of an exemplary embodiment of a core matrix computation procedure of the process of Fig. 13 which performs an SVD matrix algorithm for decomposing a matrix
  • SVD matrix algorithm for decomposing a matrix
  • Fig. 15 is a flow diagram of an exemplary embodiment of a process of Fig. 13 which synthesizes the remaining viewpoints for a new object.
  • the same reference numerals and characters, unless otherwise stated, are used to denote like features, elements, components or portions of the illustrated embodiments.
  • the present invention will now be described in detail with reference to the figures, it is done so in connection with the illustrative embodiments. It is intended that changes and modifications can be made to the described embodiments without departing from the true scope and spirit of the subj ect invention as defined by the appended claims.
  • FIG. 1 illustrates an exemplary embodiment of a data analysis system 100 for use in the collection and analysis of data describing various characteristics of different objects
  • a central server 102 is provided in the system 100, which provides therein a central processing unit (“CPU") 104, a data storage unit 106 and a database 108.
  • the central server 102 is connected to a communications network 110, which is in turn connected to an data capturing system 112.
  • the data capturing system 112 can include at least one camera (not shown for the sake of clarity).
  • a first client server 114 is provided in the system 100, which provides therein a CPU 116, a data storage unit 118, and a database 120.
  • the first client server 1 14 is connected to the communications network 110.
  • a second client server 124 is also provided in the system 100, which situates a CPU 126, a data storage unit 128, and a database 130.
  • the second client server 124 is also connected to the communications network 110. It should be understood that the central server 102, the image capture system 112, the first client server 114 and the second client server 124 can forward data messages to each other over the communications network 110.
  • the data capturing system 112 can be a "VICON" system which employs at least four video cameras.
  • the VICON system can be used to capture human limb motion and the like.
  • a multilinear data analysis application can be stored in the data storage unit 106 of the central server 102.
  • This multilinear data analysis application is capable of recognizing an unknown object, an unknown viewpoint of an object, an unknown illumination, an unknown viewpoint, and the like.
  • Such application can also synthesize a known viewponit that has never before been recorded of an object, as well as an illumination which has previously not been recorded of an object. Further the application can reduce the amount of stored data that describes an object or viewpoint by using dimensionality reduction techniques, and the like.
  • the multilinear data analysis application preferably utilizes a corpus of data, which is collected using the data capturing system 112 from different objects.
  • the corpus of data is stored in the database 108 of the server 102, and can be organized as a tensor D, which shall be described in further detail as follows.
  • a tensor also known as an n-way array or multidimensional matrix or n-mode matrix, is a higher order generalization of a vector (first order tensor) and a matrix (second order tensor).
  • a tensor can be defined as a multi-linear mapping over a set of vector spaces. The tensor can be represented in the following manner:
  • a e IR ⁇ x X "" X N where A is a tensor.
  • the order of the tensor A is N.
  • a tensor is formed by a group of primatives.
  • Each primative is a set of mode vectors, such that a first primative is a set of mode-1 vectors, a second vector is a set of mode-2 vectors, an n th primative is a set of mode-n vectors, etc.
  • the primatives can be row vectors of a matrix, column vectors of a matrix, index of a vector, etc.
  • An element of tensor A is denoted as ',. , . occidental...
  • mode-1 vectors column vectors are referred to as mode-1 vectors
  • row vectors are referred to as mode-2 vectors.
  • Mode-; vectors of an ⁇ order tensor A & IR
  • the mode-?. vectors are the column vectors of matrix A (n) e ? / " ⁇ C l/2 - / "-' /n+1 - /w) ma t can result from flattening the tensor A, as shown in Figures 12A - 12F. The flattening procedure shall be described in further detail below.
  • Figures 12A - 12C show third order tensors 1200, 1210, 1220, respectively, each having dimensions Ii x I 2 x I 3 .
  • Figure 12D shows the third order tensor 1200 after having been mode-1 flattened to obtain a matrix 1250 containing mode-1 vectors of the third order tensor 1200.
  • the third order tensor 1200 of Fig. 12A is a cube type structure, while the matrix 1250 is a two dimensional type structure having one index, i.e., I 2 , imbedded (to a certain degree) within the matrix 1250.
  • Figure 12E shows a matrix 1260 containing mode-2 vectors of the third order tensor 1210 after it has been mode-2 flattened.
  • This third order tensor 1210 is a cube type structure, while the matrix 1260 is a two dimensional type structure having one index, e.g., I 3 , imbedded (to a certain degree) with the data.
  • Figure 12F shows the third order tensor 1220 after having been mode-3 flattened to obtain a matrix 1270 containing mode-3 vectors of the third order tensor 1220.
  • Such third order tensor 1220 is a cube type structure, while the matrix 1270 organizes is a two dimensional type structure having one index, e.g., Ii, imbedded (to a certain degree) with the data.
  • a generalization of the product of two matrices can be the product of the tensor and matrix.
  • the entries of the tensor B are computed by ' ⁇ iX ⁇ i ...iggi j servicei, M ...i N ⁇ j a 'V-'»- ⁇ i n '» + ⁇ -' ⁇ W ⁇ . canal' » " '»
  • the mode-?, product of a tensor and a matrix is a special case of the inner product in multilinear algebra and tensor analysis.
  • the mode-n product is often denoted using Einstein summation notation, but for purposes of clarity, the mode-rc product symbol will be used.
  • an SVD is a combinatorial orthogonal rank decomposition, but that the reverse is not true; in general, rank decomposition is not necessarily singular value decomposition.
  • a client interface application can be stored in the data storage units 118, 128 of the first and second client servers 114, 124, respectively.
  • the client interface application preferably allows the user to control the multilinear data analysis application described previously.
  • the client interface application can instruct the multilinear data analysis application to generate new data describing a particular characteristic of a known object that may be different from those characteristics of the known object which were already observed.
  • the client interface application can instruct the multilinear data analysis application to generate new data describing a particular characteristic of the remainder of the population of observed objects that are different from those characteristics of the remainder of the population already observed.
  • the client interface application can instruct the multilinear data analysis application to recognize an unknown object from the population of observed objects, recognize a characteristic of a known object from the characteristics of the known object already observed, dimensionally reduce the amount of data stored to describe a characteristic of a known object, etc.
  • the object can be any physical object and the characteristic may be a viewpoint.
  • the object can be any physical object and the characteristic may be an illumination.
  • the multilinear data analysis application may transmit to the client interface application certain information describing the requested characteristic or object.
  • FIG. 2 illustrates flow diagram of an exemplary embodiment of a process 200 which is indicative of the multilinear data analysis application.
  • the process 200 is configured to recognize the unknown object, recognize the unknown viewpoint from which the known object is being observed, generate a known viewpoint from which the object has not been observed, etc.
  • the multilinear data analysis application utilizes the corpus of image data, which is collected using the data capturing system 1 12 from different objects. This corpus of image information is stored in the database 108 of the server 102, and describes the surface appearance of an object, including complex details such as self-occlusion and self-shadowing.
  • the corpus of image information can be organized as a tensor D. It should be understood that the corpus of image information can also be organized as a matrix D or a vector d. For example, if the information is organized as a matrix D, the process 200 preferably remains the same, but the underlying tensor procedures could be converted to matrix procedure equivalents. It should also be noted that representing the data contained in the tensor __ ) may integrate multiple indices into a singular matrix index. Likewise, if the information is organized as a vector d, the process 200 preferably remains the same, but the underlying tensor procedures could be converted to vector procedure equivalents. It should also be noted that representing the data contained in the tensor D may integrate multiple indices into a singular vector index.
  • the corpus of image data is preferably collected from different objects from at least one viewpoint which forms the tensor D.
  • Each viewpoint can be repeated multiple times, and a image cycle can be segmented from each image sequence.
  • the collected image data can be passed through a low-pass fourth-order Butterworth filter at a cut off frequency of 6 Hz, and missing data may be interpolated with a cubic spline.
  • Illumination represents the image information of the objects.
  • the illuminations are stored in the tensor D.
  • Such tensor D can have the form of a jR G ⁇ MxT s where G is the number of objects, M is the number of viewpoint classes, and T is the number of illuminations.
  • each viewpoint can be repeated ten (10) times.
  • images can be recorded using the VICON system that employs five cameras. The cameras can be positioned to record various viewpoints of an object.
  • the process 200 collects image information or data on various objects from different viewpoints, e.g., new image data.
  • the image data is collected as a group of vectors.
  • Each of the group of vectors represents an object observed from a viewpoint. If each of the possible the viewpoints and the object are known, the data can be integrated into the tensor D. If the viewpoint or object are not known, such data would likely not be integrated into the tensor D until those pieces of information are determined.
  • the data describing an unknown viewpoint or object is organized as a new data tensor D p a of a new object or a new data vector d of a new object.
  • the new data tensor _9 P;a includes more than one new data vector d.
  • Each new data vector d of the new data tensor D Vfi describes the image of object p perforniing viewpoint a.
  • the process 200 solves for a core tensor Z which can be generally used for defining the inter-relationships between the orthonormal mode matrices.
  • This step represents an N-mode singular value decomposition ("SVD") process 204, shown in Fig. 3, and described in further detail herein. It should be noted that the N-mode SVD procedure of step 204 is an orthonormal decomposition procedure.
  • the N-mode SVD procedure of step 204 solves for the core tensor Z. When this procedure of step 204 determines the core tensor Z, the process 200 advances to step 205.
  • an alternate n- mode orthonormal decomposition procedure is used in place of the n-mode SVD procedure.
  • step 205 the process 200 analyzes the data collected in the step 202.
  • the tensor D can take the form of a _ r i? GW r tensor, where G is the number of objects, M is the number of viewpoint classes, and T is the number of illuminations.
  • the object matrix P [pi ... pate...p G ] r , whose object-specific row vectors p ⁇ n span the
  • the matrix P contains the object or human image signatures.
  • the illumination matrix J whose row vectors which span the space of illuminations are preferably the eigenimages, the image variation.
  • the product Z x J transforms the eigenimages into tensorimages, a tensor representaion of the variation and co-variation of modes (objects and viewpoint classes).
  • the product Zx 3 J also characterizes how the object's parameters and viewpoint parameters interact with one another.
  • the tensor _ ⁇ Z x 2
  • a x 3 J is a viewpoint specific tensorimage, which contains a set of basis matrices for all the images associated with particular viewpoints.
  • the tensor C xi P ⁇ 3 J is an object/signature specific tensorimage, which preferably contains a set of basis matrices for all the images associated with particular objects (with particular object image signatures).
  • the core tensor Z, the matrix A, and the matrix J generated by the N-mode SVD procedure of step 204 of the tensor D define a generative model.
  • step 206 the process 200 determines whether it has been instructed by the client interface application to synthesize new data describing at least one known viewpoint that was never before recorded of a new object. If the process 200 has received such instruction, step 208 is executed to perform advances to an object generation procedure, as shown in further detail in Fig. 4 and described herein. When the object generation procedure of step 208 is complete, the process 200 advances to step 226.
  • step 210 the process 200 determines if it was instructed by the client interface application to synthesize new data describing a new viewpoint that was never before recorded of the remainder of the population of observed objects. If the process 200 has received such instruction, the process 200 continues to a viewpoint generation procedure of step 212, as shown in further detail in Fig. 5 and described herein. When the viewpoint generation procedure of step 212 is completed, the process 200 is forwarded to step 226.
  • step 214 the process 200 determines if it was instructed by the client interface application to recognize an unknown object that has been observed to perform a known viewpoint as one of the population of observed known objects. If the process 200 has received such instruction, the process 200 is directed to an object recognition procedure of step 216, as shown in greater detail in Fig. 6 and described infra. Once the object recognition process 216 is completed, the process 200 advances to step 226.
  • the process 200 is capable of recognizing an unknown object that has been observed performing an unknown viewpoint as one of the population of observed known objects.
  • the process 200 determines if it was instructed by client interface application to recognize an unknown viewpoint of a known object as one of the viewpoints already observed of the known object. If the process 200 has received such an instruction, the process 200 continues to a viewpoint recognition procedure of step 220, as shown in Fig. 7 and described infra. When the object recognition procedure of step 220 is completed, the process 200 is forwarded to step 226. Then in step 226, the process 200 determines whether a data set for a new object should be integrated into the tensor D or if the client interface application has transmitted a new instruction. In particular, if a data set for a new object is available, the process 200 advances to step 202. Otherwise, the process 200 received the new instruction from the client interface application, so the process 200 continues to step 206.
  • Fig. 3 illustrates the exemplary details N-mode SVD procedure of step
  • ⁇ -mode SVD procedure of step 204 is related to and grows out of a natural generalization of the SVD procedure for a matrix.
  • a matrix D G Ii? 2 is a two-mode mathematical object that has two associated vector spaces, e.g., a row space and a column space.
  • TJl e IR II XJ 5 a diagonal singular value matrix ⁇ e R JIXJ2 ⁇ nd a n orthogonal row
  • the tensor D can be an order-N tensor comprising N spaces, where N is preferrably greater than 2.
  • N-mode SVD is a natural generalization of SVD that orthogonalizes these N spaces, and decomposes the tensor as the mode-., product of N-orthonormal spaces.
  • Z ) x ⁇ U] x 2 U2...X,, U leverage...X ⁇ U ⁇ ,
  • a matrix representation of the N-mode SVD can be obtained by: D ( apparently - 5 n Z ⁇ n) (UAN + ⁇ ®Uotwithstanding +2 ®...®U ⁇ ®U 1 ®...®U réelle_ 1 ) r where ® is the matrix Kronecker product.
  • the core tensor Z can be analogous to the diagonal singular value matrix in conventional matrix SVD. It is important to realize, however, that the core tensor does not have a diagonal structure; rather, Z is in general a full tensor.
  • the procedure of step 204 begins at step 302 by setting an index n to one (1). This allows the process 204 to begin computing an initial matrix from the tensor D.
  • the procedure of step 204 advances to step 304.
  • the procedure of step 204 continues to step 306.
  • the procedure of step 204 sets the matrix U effet to be a left matrix of the SVD.
  • step 204 goes on to step 308, in which it is determined whether the index n is equal to the order of the tensor, i.e. N. If the index n is equal to the order of the tensor, the procedure of step 204 advances to step 312. Otherwise, the process 204 is forwarded to step 310. In step 310, the index n is incremented by one, and then the procedure of step 204 is directed to step 304. In step 312, the core tensor Z is solved for as follows: ⁇ 2 v 2 ⁇ ... ⁇ n ⁇ l... ⁇ N ⁇ ⁇ N .
  • step 204 When the core tensor Z is selected, the procedure of step 204 is completed.
  • Fig. 4 illustrates the details of the object generation procedure of step 208, which synthesizes the remaining viewpoints, which were never before seen, for a new object.
  • the remaining viewpoints are generated given the new image data tensor _D Pja of the new object observed from the viewpoint a, which includes at least one viewpoint.
  • new data tensor D p>a is a 1 x 1 x T tensor.
  • step 402 of this procedure flattens the new data tensor _9 p>a in the object mode, yielding a row vector d a ⁇ .
  • the matrix T) p , a( o b ject) is
  • step 404 in which it is determined if the object is observed from a single viewpoint. If the object is observed from a single viewpoint, the procedure of step 208 is forwarded to step 406. If the object is observed from at least two viewpoints, the procedure of step 208 advances to step 408.
  • step 406 the image signature for the object given a single observed viewpoint is computed.
  • the procedure of step 208 is completed.
  • step 408 the image signature for the object given at least two observed viewpoints is determined. If several different viewpoints d a , k are observed, the image signature can be computed as follows:
  • step 410 the procedure of step 208 synthesizes a complete set of images for the object.
  • the process 208 exits.
  • Fig. 5 illustrates details of the viewpoint generation procedure of step 212, which synthesizes an observed new viewpoint that has never before been seen for the remainder of the objects represented in the object matrix P.
  • the observed viewpoint for the remainder of the objects represented in the object matrix P is generated given the new image data tensor D p , a of at least one object from the new viewpoint a.
  • step 501 of this procedure flattens the new data tensor Z)p, a in the viewpoint mode, yielding a row vector d p ⁇ .
  • the matrix D ⁇ ( iewp o int is generated, and in particular a
  • the procedure determines as to whether the new image data tensor D p>a represents one object from the new viewpoint in step 502. If the new image data tensor D v>& represents one object from the new viewpoint, the procedure of step 212 advances to step 504. If the new image data tensor Z) p>a represents more than one object from the new viewpoint, the procedure of step 212 is forwarded to step 506. In step 504, the associated viewpoint parameters are determined based on the new image data tensor D P) a, which represents one object from the new viewpoint. If a known object, e.g., an object that is already recorded in the image database, performs a new type of viewpoint d p , it is possible to compute the associated viewpoint parameters
  • step 212 When the associated viewpoint parameters are computed, the P p( viewpoints) procedure of step 212 is directed to step 508.
  • step 506 the associated viewpoint parameters are computed based on the new image data tensor D Vfi , which represents more than one object from the new viewpoint. If several different objects are observed performing the same new viewpoint d p/c , the viewpoint parameters are computed as follows:
  • step 508 the new viewpoints are obtained for the remainder of the objects represented in the object matrix P.
  • the procedure of step 212 is completed.
  • Fig. 6 illustrates an object recognition procedure of step 216 for recognizing an unidentified object from a known viewpoint.
  • Multilinear analysis can provide basis tensors that map certain observed images into the space of object parameters (thereby enabling the recognition of objects from image data) or the space of viewpoint parameters (thereby enabling the recognition of viewpoint from image data).
  • the object recognition process 216 begins at step 602, in which the signature p of an unknown object from a known viewpoint is computed.
  • the new image vector d of a known viewpoint a can be mapped into the object signature space, by computing
  • step 604 in which an index variable n and a variable match are initialized.
  • the index variable n can be initialized to one (1) and the variable match may be initialized to negative one (-1).
  • step 606 is performed in which, the signature p is compared to an object signature p n . This signature is compared against each of the object signatures p mousse in P. Then the magnitude of the difference between the signature p and the signature p n , i.e.
  • step 608 it is determined whether a process-computed magnitude of the difference between the signature p and the signature p n is smaller than any magnitude computed up to this point. If the magnitude of the difference between the signature p and the signature p n is smaller than any difference computed up to this point, the process 216 advances to step 610. Otherwise, the process 216 is forwarded to step 612. In step 610, the variable match is set to be equal to the index n.
  • the variable match generally signifies the index of the recognized object, such that the signature p most closely matches the signature p matc -
  • step 612 it is determined if the index n is equal to G. If that is the case, the procedure of step 216 advances to step 616, otherwise the procedure of step 216 is forwarded to step 614.
  • step 614 the index n is incremented by one (1), and the procedure is returned to step 606, such that each of the objects in the object matrix P from 1 to G is objected to the comparison.
  • step 616 the signature Pmatch is identified as the signature that most closely approximates the signature p.
  • the variable match is an indexed array, which records the indices of multiple signatures that most closely match the signature p. Once the signature p match is identified, the procedure of step 216 is completed.
  • Fig. 7 illustrates the details of a viewpoint recognition procedure of step 220 for recognizing an unknown viewpoint of a known object.
  • a multilinear analysis yields basis tensors that map the observed images into the space of viewpoint parameters, thus enabling the recognition of viewpoints from the image data.
  • step 702 computes the vector a of a l ⁇ iown object from an unknown viewpoint.
  • the new image data vector d of a known object p can be
  • step 220 advances to step 704, in which an index variable m and a variable match are initialized.
  • the index variable m can be initialized to one (1), and the variable match may be initialized to negative one (-1).
  • step 706 the process 220 is forwarded to step 706, in which the vector a is compared to a viewpoint parameter vector a m . h particular, the vector a is compared against each of the viewpoint parameter vectors a m in A, in turn, as the index m is incremented.
  • the magnitude of the difference between the vector a and the viewpoint parameter vector a m i.e.
  • , is also determined.
  • step 708 the procedure of step 220 determines whether process computed magnitude of the difference between the vector a and the viewpoint parameter vector a m is smaller than any difference computed up to this point. If the magnitude of the difference between the vector a and the viewpoint parameter vector a m is smaller than any difference computed up to this point, the procedure of step 220 advances to step 710. Otherwise, the procedure of step 220 is forwarded to step 712. In step 710, the procedure of step 220 sets the variable match is set to be equal to the index m.
  • the variable match generally signifies the index of the recognized viewpoint, such that the vector a most closely matches the viewpoint parameter vector
  • step 712 it is determined if the index m is equal to M. If that is the case, the procedure of step 220 advances to step 716, otherwise the procedure is forwarded to step 714.
  • Step 714 indicates that the index m is incremented by one (1), and the procedure advances to step 706, such that the index m increments through each of the viewpoints in the viewpoint matrix A from 1 to M.
  • the viewpoint parameter vector a matC h is identified as the signature that most closely approximates the vector a.
  • the variable match can be an indexed array, which records the indices of multiple viewpoints that most closely match the vector a.
  • FIG. 8 illustrates a flow diagram of an exemplary embodiment of a process implementing a multilinear data analysis application 800 according to the present invention.
  • the multilinear data analysis application 800 may be configured to recognize an unknown object, an unknown viewpoint, and dimensionally reduce the amount of data describing illuminations, etc.
  • the multilinear data analysis application 800 utilizes a corpus of image data, which is collected using the data capturing system 112 from different objects.
  • the corpus of image information can be stored in the database 108 of the server 102. This corpus of image information may describe the illuminations, the viewpoints, and the objects captured in images made of pixels.
  • The- corpus of image information is organized as a tensor D.
  • the tensor D takes the form of a JR GXIXEXP tensor, where G is the number of objects, I is the number of illuminations, E is the number of viewpoints, and P is the number of pixels.
  • the corpus of image information can also be organized as a matrix D or a vector d.
  • the process 800 preferably remains the same, but the underlying tensor procedures could be converted to matrix procedure equivalents.
  • representing the data contained in the tensor D may integrate multiple indices into a singular matrix index.
  • the process 800 preferably remains the same, but the underlying tensor procedures could be converted to vector procedure equivalents. It should also be noted that representing the data contained in the tensor D may integrate multiple indices into a singular vector index.
  • three viewpoints can be collected for each object.
  • Each viewpoint may be captured in four different illuminations, i.e. light positions.
  • the four different illuminations may be one light from the center, one light from the right, one light from the left, and two lights one from the right and one from the left.
  • the three different viewpoints may be center, 34 degrees to the right, and 34 degrees to the left.
  • further similar viewpoints are collected for each object such that each viewpoint is captured in four different illuminations.
  • the four different illuminations are one light from the center, one light from the right, one light from the left, and two lights one from the right and one from the left.
  • the two different viewpoints are 17 degrees to the right, and 17 degrees to the left.
  • each viewpoint is captured in three different illuminations and five different viewpoints.
  • the three different illuminations are one light from the center, one light from the right, and one light from the left.
  • the five different viewpoints are center, 17 degrees to the right, 17 degrees to the left, 34 degrees to the right, and 34 degrees to the left.
  • step 802 provides that the multilinear data analysis application 800 collects image information describing the illumination, viewpoint, and object.
  • New image data is collected describing the illumination of individual pixels of viewpoints of objects. If each of the illuminations, each of the viewpoints, and each of the pixels for a given object are known, the data is integrated to the tensor D. Otherwise, the data cam ot be integrated into the tensor D until those pieces of information are determined.
  • the data describing an unknown illumination, pixel, viewpoint or object is organized as a new data vector d.
  • the new data vector d describes an image having certain illumination and viewpoint data.
  • the multilinear data analysis application 800 solves for the core tensor Z.
  • this step can be an N-mode SVD procedure 304 as shown in Fig. 3 and described below in relation to Fig. 3.
  • the N-mode SVD procedure 304 solves for the core tensor Z with N being equal to 4.
  • the multilinear data analysis application 800 advances to step 806.
  • the tensor D takes the form of a fR GxlxExP tensor, where G is the number of objects, I is the number of illuminations, E is the number of viewpoints, and P is the number of pixels.
  • the N-mode SVD process 804 decomposed the tensor D as follows: I ) — £ ⁇ ⁇ X. ⁇ U objects ⁇ 2 '-'ilium ⁇ 3 «J viewpoints ⁇ 4 U ixels
  • the G x I x E x P core tensor Z governs the interviewpoint between the factors represented in the 4 mode matrices:
  • the G x G mode matrix U 0b j ects spans the space of object parameters
  • the I x I mode matrix Uj ⁇ um spans the space of illumination parameters
  • the E x E mode matrix U V j eW p o i nts spans the space of viewpoint parameters.
  • the P x P mode matrix U p j xe ⁇ _ orthonormally spans the space of images.
  • the multilinear data analysis incorporates aspects of a linear principal component analysis ("PCA") analysis.
  • PCA linear principal component analysis
  • Each column of U 0b j e c t s is an "eigenimage".
  • These eigenimages are preferably identical to the conventional eigenfaces, since the eigenimages are computed by performing the SVD on the mode-4 flattened data tensor D so as to yield the matrix D objects .
  • the core tensor Z can transform the eigenimages in U p j xe
  • the PCA basis vectors or eigenimages represent only the principal axes of variation across images.
  • the image image database can include V • I • E images for each object which vary with illumination and viewpoint.
  • the PCA output represents each object as a set of V • I • E vector- valued co-efficients, one from each image in which the object appears.
  • Each column in the tensor D is a basis matrix that comprises N eigenvectors.
  • the first eigenvector depicts the average object, and the remaining eigenvectors capture the variability across objects for the particular combination of illumination and viewpoint associated with that column.
  • Each image is represented with a set of coefficient vectors representing the object, view point, illumination and viewpoint factors that generated the image.
  • Multilinear decomposition allows the multilinear data analysis application 800 to construct different types of basis depending on the instruction received from the client interface application.
  • step 814 of Fig. 8 causes the multilinear data analysis application 800 to determine whether the client interface application has instructed the multilinear data analysis application 800 to recognize an unknown object that has been observed displaying a l ⁇ iown viewpoint as one of the population of observed l ⁇ iown objects. If the multilinear data analysis application 800 has received such instruction, the multilinear data analysis application 800 advances to an object recognition procedure of step 816, shown in greater detail in Fig. 9 and described infra. When the object recognition procedure of step 816 is completed as the multilinear data analysis application 800 advances to step 826.
  • step 818 the multilinear data analysis application 800 determines whether the client interface application has instructed the multilinear data analysis application 800 to recognize an unknown viewpoint of a known object as one of the viewpoints already observed of such known object. If the multilinear data analysis application 800 has received such instruction, the multilinear data analysis application 800 advances to an viewpoint recognition procedure of step 820, as shown in greater detail in Fig. 10 and described infra. When the viewpoint recognition procedure of step 820 is completed, the multilinear data analysis application 800 is forwarded to step 826.
  • step 822 the multilinear data analysis application 800 determines whether the client interface application has instructed the multilinear data analysis application 800 to dimensionally reduce the amount of data describing illuminations. If the multilinear data analysis application 800 has received such instruction, the multilinear data analysis application 800 advances to a data reduction procedure of step 824, as shown in greater detail in Fig. 11 and described infra. Once the data reduction procedure of step 824 is complete, the multilinear data analysis application 800 advances to step 826. Finally, in step 826, the multilinear data analysis application 800 determines whether a data set for a new object should be collected or if the client interface application transmitted new instruction.
  • Fig. 9 illustrates a flow diagram of the details of the object recognition procedure of step 816 for recognizing an unidentified object given an unknown image image: the new data vector d.
  • the multilinear data analysis preferably yields a basis tensor (as defined below) that maps all images of an object to the same point in the object parameter space, thus creating a many-to-one mapping.
  • the object recognition procedure of step 816 begins at step 902, in which the matrix U 0 je c ts is extracted.
  • the matrix U 0bjects contains row vectors cj of coefficients for each object p.
  • the procedure of step 816 advances to step 906 where this procedure initializes indexes i and e to one (1).
  • the object recognition procedure of step 816 indexes into the basis tensor B to obtain a sub-tensor 2?j je . This is performed for a particular illumination i and viewpoint e to obtain the subtensor .Behaving dimensions G ⁇ 1 x 1 P.
  • step 910 the subtensor B i e is flattened along the object mode.
  • the subtensor B i e is flattened along the object mode to obtain the G P matrix
  • step 912 an index variable p and a variable match are initialized.
  • the index variable p is initialized to one (1)
  • the variable match is initialized to negative one (-1).
  • the procedure of step 816 advances to step 914, in which, the projection operator B ⁇ X (object) is used to project the new data vector d into a set of candidate coefficient vectors.
  • each of the set of candidate coefficient vectors c,- ,e is compared against the object-specific coefficient vectors c p .
  • the comparison can be made according to the following equation:
  • the best matching vector c ⁇ can be the one that yields the smallest value of j
  • Step 920 provides that the variable match is set to be equal to the index p.
  • the variable match signifies the index of the most closely matched object, such that the set of candidate coefficient vectors c, >e most closely matches the object- specific coefficient vectors c m atch-
  • step 922 it is determined if the index p is equal to G. If that is the case, the procedure of step 816 advances to step 928; otherwise, the procedure of step 816 advances to step 924.
  • step 924 the index p is incremented by one (1), and the procedure of step 816 advances to step 914, such that the procedure tests each of the objects in the object matrix U 0 bj ect from 1 to G.
  • step 928 it is determined if the index e is equal to E. If that is the case, the procedure of step 816 sets the index e equal to one (1) and advances to step 930; otherwise, the procedure of step 816 advances to step 934.
  • step 934 the index e is incremented by one (1), and the procedure of step 816 advances to step 908, such that the procedure tests each of the objects in the object matrix U v i e p o i nts from 1 to E.
  • step 930 it is determined if the index i is equal to I. If that is the case, the procedure of step 816 advances to step 926; otherwise, the procedure of step 816 advances to step 936.
  • step 936 the index i is incremented by one (1), and the procedure of step 816 advances to step 908, such that the procedure tests each of the objects in the object matrix U ⁇ um from 1 to I.
  • the object match can be identified as the object protrayed in the new data vector d.
  • the variable match can be an indexed array, that records the indices of multiple objects most closely matching the objects portrayed in the new data vector d. Once the object match is identified, the procedure of step 816 is completed.
  • Fig. 10 illustrates a flow diagram of the details of the viewpoint recognition procedure of step 820 for recognizing an unidentified viewpoint given an unknown image image: the new data vector d.
  • the viewpoint recognition procedure of step 820 is largely the same as the object recognition procedure of step 816.
  • the viewpoint recognition procedure of step 820 begins in step 1002, in which the matrix Uviewpoints is extracted, in a manner similar to that used to extract U 0 bjects in step 902. h particular, the matrix U v j ewpo in ts contains row vectors c ⁇ e of coefficients for each viewpoint e.
  • step 820 advances to step 1004, in which the basis tensor B is generated.
  • the procedure of step 820 advances to step 1006 where this procedure initializes indexes i and p to one (1).
  • the viewpoint recognition procedure of step 820 indexes into the basis tensor B to obtain a sub-tensor _5 p . This is performed for a particular object p and illumination i to obtain the subtensor _5 P j having dimensions 1 1 x E x P.
  • step 1010 the subtensor _5 P) j is flattened along the viewpoint mode.
  • the subtensor B p ⁇ is flattened along the viewpoint mode to obtain the E P matrix B p ,j (viewpoints).
  • step 1012 an index variable e and a variable match are initialized.
  • the index variable e is initialized to one (1)
  • the variable match is initialized to negative one (-1).
  • the comparison can be made according to the following equation:
  • step 1018 it is determined whether the set of candidate coefficient vectors c p j is the closest match to the viewpoint coefficient vectors c e up to this point.
  • the best matching vector c e can be the one that yields the smallest value of
  • Step 1020 provides that the variable match is set to be equal to the index p.
  • the variable match signifies the index of the most closely matched viewpoint, such that the set of candidate coefficient vectors c p most closely matches the viewpoint coefficient vectors C mate ⁇ ..
  • step 1022 it is determined if the index e is equal to E. If that is the case, the procedure of step 820 advances to step 1028; otherwise, the procedure of step 820 advances to step 1024.
  • step 1024 the index e is incremented by one (1), and the procedure of step 820 advances to step 1014, such that the procedure tests each of the viewpoints in the viewpoint matrix U v i ew p o i nts from 1 to E.
  • step 1028 it is determined if the index p is equal to G. If that is the case, the procedure of step 820 sets the index p equal to one (1) and advances to step 1030; otherwise, the procedure of step 820 advances to step 1034.
  • step 1034 the index p is incremented by one (1), and the procedure of step 820 advances to step 1008, such that the procedure tests each of the objects in the object matrix U 0bje t from l to G.
  • step 1030 it is determined if the index i is equal to I. If that is the case, the procedure of step 820 advances to step 1026; otherwise, the procedure of step 820 advances to step 1036.
  • step 1036 the index i is incremented by one (1), and the procedure of step 820 advances to step 1008, such that the procedure tests each of the illuminations in the illumination matrix Ujn um from 1 to I.
  • step 1026 the object match can be identified as the object protrayed in the new data vector d.
  • the variable match can be an indexed array, that records the indices of multiple objects most closely matching the objects portrayed in the new data vector d.
  • Fig. 11 illustrates a flow diagram of the details for the data reduction procedure step 824 for dimensionally reduce the amount of data describing illuminations.
  • the truncation of the mode matrices yields an exemplary reduced- dimensionality approximation D
  • the truncation of the mode matrices results in the approximation of the tensor D with reduced ranks R ⁇ ⁇ I ⁇ , R ⁇ ⁇ , ⁇ ⁇ -, RN ⁇ IN that has a bounded error
  • the procedure step 824 advances to step 1104.
  • D a best rank-(R l5 R 2 , ..., R N ) approximation
  • D' Z' X] U' ⁇ ⁇ U' 2 ... /v U , with orthonormal / diligent x R n mode matrices U' «.
  • the data reduction procedure step 824 begins in step 1102, where an index n is initialized to one (1).
  • step 1104 the mode matrix U n is truncated to R n columns. All data in the mode matrix U n beyond the R n column can be removed from the matrix U n .
  • the procedure step 824 advances to step 1106, in which it is determined whether the index n is equal to N. If that is the case, the procedure step 824 advances to step 1110; otherwise, the procedure step 824 is forwarded to step 1108.
  • step 1108 the index n is incremented by one (1), and the procedure step 824 proceeds to step 1104.
  • step 1110 the index n is initialized to one (1), and the procedure step 824 advances to step 1112, in which the
  • step 824 advances to step 1114, in which the tensor U ' n k+1
  • the matrix U' j +1 n(n) is computed as the Ii ⁇ Ri matrix whose columns are the first R ⁇ columns of the left
  • step 1118 it is determined whether the index n is equal to N. If that is the case, the procedure step 824 advances to step 1122; otherwise the procedure step 824 advances to step 1120, in which the index n is incremented by one (1) and the procedure step 824 advances to step 1112. Then in step 1122, it is determined whether the mode matrices have converged. The mode matrices have converged if T II U /£+1 U *
  • the procedure step 824 is completed.
  • Fig. 13 illustrates a flow diagram of an exemplary embodiment of a process implementing a multilinear data analysis application 1300 which is indicative of the multilinear data analysis application.
  • the process 1300 is configured to synthesize a known viewpoint never before recorded of the object.
  • the multilinear data analysis application utilizes the corpus of viewpoint data, which is collected using the data capturing system 112 from different objects as described above in relation to Fig. 2.
  • This corpus of image information is stored in the database 108 of the server 102, and describes illumination of at least one object from at least one viewpoint.
  • the corpus of image information can be organized as a matrix D and is preferably collected from different objects as described above in relation to Fig. 2.
  • the corpus of image information can also be organized as a tensor D or a vector d.
  • the multilinear data analysis application 1300 is similar to the multilinear data analysis application 200 of Fig. 2, except that the data utilized by the multilinear data analysis application 1300 takes is organized as the matrix D, not as the tensor D.
  • the process 1300 collects image information or data on various objects from different viewpoints, e.g., new image data. If the viewpoint and object are known, the data can be integrated into the matrix D. If the viewpoint or object are not l ⁇ iown, such data would likely not be integrated into the matrix D until those pieces of information are determined.
  • the data describing an unknown viewpoint or object is organized as a new data matrix D p or a new data vector d.
  • the new data matrix D p can include more than one new data vector d.
  • Each new data vector d p>a of the new data matrix D p describes the image of object p performing viewpoint a.
  • the matrix D can take the form of a nt x m matrix, where n is the number of objects, t is the number of image samples, and m is the number of viewpoints.
  • the first column of the matrix D stacks the mean first viewpoint of every object, the second column stacks the mean second viewpoint of every object and the third stacks the mean third viewpoint of every object, as follows:
  • the columns of the matrix D are the average first viewpoint, second viewpoint and tthhiirrdd ⁇ viewpoint of the i th object.
  • Each image is defined as the illumination of each pixel.
  • the process 1300 decomposes the matrix D into a core matrix Z, an object matrix P, and a viewpoint matrix A.
  • the core matrix Z can be used for defining the inter-relationships between an objects matrix P and a viewpoint matrix A.
  • This step represents a singular value decomposition ("SVD") process 1304, shown in Fig. 14, and described in further detail herein.
  • the SVD procedure of step 1304 is an orthonormal procedure that solves for the core matrix Z, the object matrix P, and the viewpoint matrix A, which minimizes
  • step 1304 determines the core matrix Z
  • the process 1300 advances to step 1305.
  • step 1305 the process 1300 analyzes the data collected in the step 1302.
  • the object matrix P [pi ... P »" -P G ] ⁇ , whose row vectors pj are object specific, encodes the invariancies across viewpoints for each object.
  • the object matrix P contains the object or human image signatures pi.
  • step 1306 the process 1300 determines whether it has been instructed by the client interface application to synthesize new data describing at least one l ⁇ iown viewpoint that was never before recorded of an object. If the process 1300 has received such instruction, step 1308 is executed to perform advances to an object generation procedure, as shown in further detail in Fig.
  • step 1326 the process 1300 determines whether a data set for a new object should be integrated into the matrix D or if the client interface application has transmitted a new instruction. In particular, if the data set for a new object from the viewpoint is available, the process 1300 advances to step 1302. Otherwise, the process 1300 received the new instruction from the client interface application, so the process 1300 continues to step 1306.
  • the matrix V is then truncated to the first r-columns of the matrix V.
  • the matrix A is then truncated to the first r-columns of the matrix V.
  • Fig. 15 illustrates the details of the object generation procedure of step 1308, which synthesizes the remaining viewpoints, which were never before seen, for an new object.
  • the remaining viewpoints are generated given new image data D new of the new object observed from a viewpoint.
  • the new signature model of the new object is the matrix
  • step 1510 it is determined whether the error function E has converged. If the error function E has not converged, the procedure of step 1308 continues to step 1512, where the index t is incremented by one ( 1 ) and this procedure advances to step 1504. If the error function E has converged, this procedure advances to step 1514.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Software Systems (AREA)
  • Physics & Mathematics (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • Computer Hardware Design (AREA)
  • Health & Medical Sciences (AREA)
  • Computer Graphics (AREA)
  • Computing Systems (AREA)
  • Databases & Information Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Multimedia (AREA)
  • Debugging And Monitoring (AREA)
  • Compression, Expansion, Code Conversion, And Decoders (AREA)
  • Processing Or Creating Images (AREA)
  • Image Processing (AREA)

Abstract

L'invention a trait à une structure de données, à un procédé, à un support de stockage et à un agencement logique permettant de collecter et d'analyser des données multilinéaires décrivant diverses caractéristiques d'objets différents. En particulier, il est possible de reconnaître un objet inconnu ou un point de vue inconnu d'un objet, ainsi que de synthétiser un nouveau point de vue connu qui n'a jamais été enregistré auparavant, et de réduire le volume de données stockées décrivant un objet ou un point de vue, au moyen de techniques de réduction de la dimensionnalité et analogues.
PCT/US2004/024000 2003-07-25 2004-07-26 Agencement logique, structure de donnees, systeme et procede permettant une representation multilineaire d'ensembles de donnees combinees aux fins de synthese, de rotation et de compression WO2005013066A2 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US11/200,479 US7379925B2 (en) 2003-07-25 2005-08-08 Logic arrangement, data structure, system and method for multilinear representation of multimodal data ensembles for synthesis, rotation and compression

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US49013103P 2003-07-25 2003-07-25
US60/490,131 2003-07-25

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US11/200,479 Continuation-In-Part US7379925B2 (en) 2003-07-25 2005-08-08 Logic arrangement, data structure, system and method for multilinear representation of multimodal data ensembles for synthesis, rotation and compression

Publications (2)

Publication Number Publication Date
WO2005013066A2 true WO2005013066A2 (fr) 2005-02-10
WO2005013066A3 WO2005013066A3 (fr) 2009-04-09

Family

ID=34115362

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2004/024000 WO2005013066A2 (fr) 2003-07-25 2004-07-26 Agencement logique, structure de donnees, systeme et procede permettant une representation multilineaire d'ensembles de donnees combinees aux fins de synthese, de rotation et de compression

Country Status (1)

Country Link
WO (1) WO2005013066A2 (fr)

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6680735B1 (en) * 2000-10-04 2004-01-20 Terarecon, Inc. Method for correcting gradients of irregular spaced graphic data
US6717576B1 (en) * 1998-08-20 2004-04-06 Apple Computer, Inc. Deferred shading graphics pipeline processor having advanced features

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6717576B1 (en) * 1998-08-20 2004-04-06 Apple Computer, Inc. Deferred shading graphics pipeline processor having advanced features
US6680735B1 (en) * 2000-10-04 2004-01-20 Terarecon, Inc. Method for correcting gradients of irregular spaced graphic data

Also Published As

Publication number Publication date
WO2005013066A3 (fr) 2009-04-09

Similar Documents

Publication Publication Date Title
US7603323B2 (en) Logic arrangement, data structure, system and method for multilinear representation of multimodal data ensembles for synthesis recognition and compression
US7379925B2 (en) Logic arrangement, data structure, system and method for multilinear representation of multimodal data ensembles for synthesis, rotation and compression
KR102648770B1 (ko) 홍채 식별을 위한 딥 뉴럴 네트워크
CN107203745B (zh) 一种基于跨域学习的跨视角动作识别方法
Zhang et al. Gabor wavelet associative memory for face recognition
Sahambi et al. A neural-network appearance-based 3-D object recognition using independent component analysis
CN110569768A (zh) 人脸模型的构建方法、人脸识别方法、装置及设备
US8885893B1 (en) System and method for adaptive face recognition
KR20020050126A (ko) 1차 및 2차 고유특징을 사용하여 조명 및 시각에 영향받지않는 안면 기술 방법
WO2005067572A2 (fr) Procede, systeme, support d'enregistrement et structure de donnees pour la reconnaisance d'images par analyse de composante independante multilineaire
CN113095149A (zh) 一种基于单张人脸图像的全头部纹理网络结构及生成方法
Chowdhary et al. Singular value decomposition–principal component analysis-based object recognition approach
Belahcene et al. Local descriptors and tensor local preserving projection in face recognition
Kekre et al. CBIR feature vector dimension reduction with eigenvectors of covariance matrix using row, column and diagonal mean sequences
Lewandowski et al. View and style-independent action manifolds for human activity recognition
CN111598995A (zh) 一种基于原型分析的自监督多目三维人体姿态估计方法
US7822693B2 (en) Logic arrangement, data structure, system and method for multilinear representation of multimodal data ensembles for synthesis, recognition and compression
Yang et al. Development of a fast panoramic face mosaicking and recognition system
WO2005013066A2 (fr) Agencement logique, structure de donnees, systeme et procede permettant une representation multilineaire d'ensembles de donnees combinees aux fins de synthese, de rotation et de compression
CN108921106B (zh) 一种基于capsule的人脸识别方法
Dickinson et al. Geons
Zhang et al. Face, image, and analysis
Alikarami et al. Sparse representation and convolutional neural networks for 3D human pose estimation
CN109859268A (zh) 基于生成查询网络的物体被遮挡部分成像方法
Nualtim et al. Face Synthesis and Partial Face Recognition from Multiple Videos

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A2

Designated state(s): AE AG AL AM AT AU AZ BA BB BG BR BW BY BZ CA CH CN CO CR CU CZ DE DK DM DZ EC EE EG ES FI GB GD GE GH GM HR HU ID IL IN IS JP KE KG KP KR KZ LC LK LR LS LT LU LV MA MD MG MK MN MW MX MZ NA NI NO NZ OM PG PH PL PT RO RU SC SD SE SG SK SL SY TJ TM TN TR TT TZ UA UG US UZ VC VN YU ZA ZM ZW

AL Designated countries for regional patents

Kind code of ref document: A2

Designated state(s): GM KE LS MW MZ NA SD SL SZ TZ UG ZM ZW AM AZ BY KG KZ MD RU TJ TM AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IT LU MC NL PL PT RO SE SI SK TR BF BJ CF CG CI CM GA GN GQ GW ML MR NE SN TD TG

121 Ep: the epo has been informed by wipo that ep was designated in this application
WWE Wipo information: entry into national phase

Ref document number: 11200479

Country of ref document: US

NENP Non-entry into the national phase

Ref country code: DE

WWP Wipo information: published in national office

Ref document number: 11200479

Country of ref document: US

122 Ep: pct application non-entry in european phase