JP5552519B2 - Construction of face feature vector - Google Patents

Description

  The present disclosure relates generally to the field of face recognition. More specifically, this disclosure describes a number of techniques for combining multiple types of face recognition descriptors into a single element face feature vector. The face feature vector can be used for face recognition. Examples of such applications include, but are not limited to, managing (sorting and annotating) images (still images and video images) in iPhoto® and Aperture®. is not. (IPHOTO and APERTURE are registered trademarks of Apple Inc.)

  In general, face recognition operations scan a person's face, extract or detect a specific set of parameters from them, and these parameters are already assigned identification information or otherwise known Check against a library of known face data. The data set to which the new image parameters are compared is often characterized or described by the model. In practice, these models define a group of parameter sets, and all images that fall within a given group are classified as those of the same person.

  In order to be robust (eg, stable against image noise, human poses, and scene illumination) and accurate (eg, resulting in a high recognition rate), a particular parameter set is a typical person It is necessary to code the information describing the face in a way that is repeatable and unchanging with respect to variations in the face, and at the same time distinguishing one person from another. This need is a central problem facing all face recognition systems. Thus, it is beneficial to identify a mechanism (method, apparatus, and system) for defining a set of parameters that provides robust and accurate face recognition.

  In various embodiments, the present invention provides an apparatus (eg, a personal computer), method, and computer for generating a novel facial feature vector that can be used to identify a detected face in a digital image.・ Provide program code. The method includes implementing (or executing) the computer program code to obtain landmark detection information for the first face in the first image (eg, by face detection techniques). The landmark detection information is applied to the first and second shape models to generate first and second shape feature vectors, and is applied to the first and second texture models to apply the first and second textures. A feature vector can be generated. All of these four feature vectors can be combined to form a face feature vector.

  In one embodiment, the first shape model is a two-dimensional shape model of the detected face, and the second shape model is a three-dimensional shape model of the detected face. The first and second shape models can be linear or non-linear independently of each other.

  In another embodiment, the landmark detection information can be normalized before being used to generate the first and second texture feature vectors. In some embodiments, the first texture feature vector may be based on a specific region in the normalized landmark detection information (this region is normalized landmark detection information). Contains less information than all of

  In yet another embodiment, a morphing operation can be applied to the normalized landmark detection information before generating the second texture feature vector using the normalized landmark detection information.

  In yet another embodiment, the similarity measure can be determined by comparing these two facial feature vectors. This similarity measure can be used to determine whether two face feature vectors are likely to represent the same face. In this and similar embodiments, the similarity measure can be based on the Mahalanobis distance measure.

FIG. 5 illustrates in face diagram form face feature vector generation and runtime face recognition operations according to one embodiment. FIG. FIG. 3 illustrates in block diagram form a composition of a shape model and a texture model according to one embodiment. FIG. 6 illustrates in a block diagram form a face feature vector generation operation according to another embodiment. Fig. 4 illustrates the computation of a local image descriptor according to one embodiment. Fig. 4 illustrates dense image descriptor operations according to one embodiment. Fig. 4 illustrates a region of a dense distortion image descriptor according to one embodiment. Fig. 4 illustrates the structure of a face feature vector according to one embodiment. 2 shows a face recognition operation according to one embodiment in flowchart form. FIG. 6 illustrates an exemplary receiver operating characteristic (ROC) curve illustrating the disclosed facial feature vector identification performance according to one embodiment. FIG. 1 illustrates in block diagram form an exemplary electronic device that can be used to implement one or more operations in accordance with the present disclosure.

  The present disclosure relates to systems, methods, and computer-readable media for determining and applying a face recognition parameter set. In general, a unique combination of face recognition identifiers is identified to be more robust (eg, stable to image noise, human poses, and scene illumination) and more accurate (eg, compared to prior art recognition methods) A technique for constructing a “facial feature vector” that has been found to yield a high recognition rate. More specifically, the face feature vector can be generated by a combination of a shape descriptor and a texture descriptor. In one implementation, the face feature vector includes a two-dimensional (2D) shape of the face, a three-dimensional (3D) shape of the face, the entire face or global texture, and details or local texture information (eg, skin Information).

  In the following description, numerous specific details are set forth for purposes of explanation in order to provide a thorough understanding of the concepts of the present invention. As part of this description, some of the drawings in this disclosure represent structures and devices in block diagram form in order not to obscure the present invention with details that are well known to those skilled in the art. Further, the language used in this disclosure is selected primarily for readability and teaching purposes, and is not selected to define or limit the subject matter of the present invention. In order to make a decision, it is necessary to rely on the claims. Reference to “one embodiment” or “one embodiment” in this disclosure includes in the at least one embodiment of the invention the particular feature, structure, or characteristic described in connection with that embodiment. It is not to be understood that all references to “one embodiment” or “an embodiment” all refer to the same embodiment.

  Of course, in developing any actual implementation (such as in any development project), to achieve a developer's specific purpose (eg, conformance to system-related and business-related constraints). Many decisions must be made and these objectives will vary from implementation to implementation. Similarly, it will be appreciated that such development efforts can be complex and time consuming, but nevertheless will be routine for those skilled in the art of face recognition having the benefit of this disclosure. Become.

  Referring to FIG. 1, a face feature vector generation and runtime face recognition operation 100 according to one embodiment is shown in block diagram form. First, the input image 105 is processed according to the face detector 110, and a landmark image 115 is generated. As used herein, the term “landmark image” refers to an image of a face from which landmark points have been detected. Landmark features can include the location of one or more facial features such as eyes, eyebrows, nose, mouth, and cheeks. The input image 105 can be, for example, an image obtained from a digital still camera or a video camera. The face detector 110 can use any method suitable for the designer's objectives / constraints. Exemplary face detection techniques include, but are not limited to, knowledge base, feature invariance, template matching, and appearance based methods. The exact method for detecting the face is not important for the following description, and will not be described further in this specification. Although not limited, in one embodiment, the landmark image 115 can be a grayscale image in which the detected features are noticeable. For ease of explanation, it will be assumed below that the input image (eg, image 105) contains only a single face. However, it should be understood that the disclosed technology does not inherently include such limitations.

  The landmark image 115 can be applied to one or more shape models 120 and one or more texture models 125. As shown, shape model 120 generates shape descriptor 130 and texture model 125 generates texture descriptor 135. It should be appreciated that the shape model 120 and the texture model 125 are typically generated off-line using a library of known images and can be linear or non-linear independently of each other. These models can also include "geometry constrained part-based models" where the landmark points have their own appearance model. Descriptors 130 and 135 may be combined in any manner that meets the developer's objectives and / or constraints, according to block 140. As an example, operation 140 can concatenate each of the given shape and texture descriptors. In another embodiment, operation 140 can generate a set of linear combinations of descriptor elements. In yet another embodiment, shape descriptors 130 can be combined in one way and texture descriptors 135 can be combined in a different way to concatenate each combination. In yet another embodiment, one or more descriptors are combined as generated by the respective model, while other descriptors are subject to additional processing (eg, dimension reduction, smoothing) before being combined. ). Regardless of how they are combined, the result of operation 140 is a face feature vector 145. The face feature vector 145 can be maintained in the storage 150 (eg, a permanent magnetic or solid disk unit). In practice, the facial feature vector 145 can be incorporated into the input image 105 (eg, in its metadata) and / or kept in a separate data store that references the image 105.

  Once generated, the face feature vector 145 can be used by the application 155 to identify the face of the corresponding image (eg, in the image 105). For example, the application 155 may retrieve an image 160 in which the associated face feature vector <f> is associated with or identified from the face 'F'. Once extracted, the facial feature vector 145 can be compared 165 with the facial feature vector <f>, and if they are sufficiently similar (eg, by some convenient measure), the image 105 is It can be said that it contains 'F'. In one embodiment, the application 155 may be a user level graphics application (eg, iPhoto or Aperture). In another embodiment, application 155 can be incorporated into a face recognition framework that can be used by user-level applications. In yet another embodiment, some or all of the applications 155 can be incorporated into dedicated image processing hardware.

  Referring to FIG. 2, shape model 120 includes a two-dimensional (2D) model 200 and a three-dimensional (3D) model 205 (generating 2D and 3D shape descriptors 210 and 215, respectively), while a texture model. It can be seen that 125 includes a global texture model 220 and a local texture model 225 (which generates global and local texture descriptors 230 and 235, respectively).

とすることができ、ここで、

（これ以降ベクトルｉと記載する）は、（モデルが形状モデルであるかテクスチャ・モデルであるかに応じて）画像又は画像点を表し、Ｂは、基底ベクトル（通常、直交する）のセットを表し、

（これ以降ベクトルｃと記載する）は、モデル係数のセットを表し、

（これ以降ベクトルｍと記載する）は、（モデルが形状モデルであるかテクスチャ・モデルであるかに応じて）平均形状ベクトル又は平均テクスチャ・ベクトルを表す。（トレーニング）画像のセットが与えられると、例えば、主成分分析（ＰＣＡ）、独立成分分析（ＩＣＡ）、線形判別分析（ＬＤＡ）、弾性バンチ・グラフ・マッチング（ＥＢＧＭ）、トレース変換、アクティブ外観モデル（２Ｍ）、ベイズ・フレームワーク、サポート・ベクター・マシン（ＳＶＭ）、隠れマルコフ・モデル（Ｈ８）、及び固有顔といったあらゆる技術を用いて、基底ベクトルＢ及び平均形状／テクスチャ・ベクトルｍを決定することができる。Ｂを構成する基底ベクトルの数が、モデルの正確性をある程度まで決定する。従って、Ｂの大きさは、所望の正確性を達成するように設計者が選択することができる。１つの実装形態においては、１０個の基底ベクトルで十分な場合があるが、別の実装形態においては、２０、５０、又は７５個の基底ベクトルが必要となる場合がある。 In one embodiment, the 2D model 200, the 3D model 205, and the global texture model 200 are linear models of the form:

And where

(Hereinafter referred to as vector i) represents an image or image point (depending on whether the model is a shape model or a texture model) and B represents a set of basis vectors (usually orthogonal) Represent,

(Hereinafter referred to as vector c) represents a set of model coefficients,

(Hereinafter referred to as vector m) represents an average shape vector or average texture vector (depending on whether the model is a shape model or a texture model). Given a set of (training) images, for example, Principal Component Analysis (PCA), Independent Component Analysis (ICA), Linear Discriminant Analysis (LDA), Elastic Bunch Graph Matching (EBGM), Trace Transformation, Active Appearance Model (2M), base vector B and average shape / texture vector m are determined using any technique such as Bayesian framework, support vector machine (SVM), hidden Markov model (H8), and eigenface be able to. The number of basis vectors that make up B determines the accuracy of the model to some extent. Thus, the size of B can be selected by the designer to achieve the desired accuracy. In one implementation, 10 basis vectors may be sufficient, while in other implementations, 20, 50, or 75 basis vectors may be required.

  Referring to FIG. 3, a block diagram for one embodiment of a face feature vector construction operation 300 is shown. As described above with respect to FIGS. 1 and 2, the input image 105 is provided to the face detector 110 that generates the landmark image 115. In the illustrated embodiment, the landmark image 115 can be provided directly to the 2D and 3D shape models 200 and 205. Assuming that these models can be characterized by Equation 1, for the 2D shape model 200, the vector i represents the landmark image 115, B represents the set of basis vectors of the 2D model, and the vector c represents the 2D model. Represents a set of coefficients (ie, 2D descriptor 210), and vector m represents an average 2D shape vector. Similarly, for 3D shape model 205, vector i also represents landmark image 115, B represents a set of 3D model basis vectors, and vector c represents a set of 3D model coefficients (ie, 3D descriptor 215). , Vector m represents an average 3D shape vector.

  Next, a normalization operation 305 can be performed on the landmark image 115 to generate a normalized image 310. One of ordinary skill in the art can adjust the image's landmark features (eg, eyebrows, eyes, nose, mouth, and chin) to appear at specific locations within a frame of a given size. It will be understood that this refers to a process that can be performed.

  Once normalized, the image 310 can be provided to the global texture model 220 to generate a global texture descriptor 230. If Equation 1 characterizes the global texture model 220, vector i represents the normalized image 310, B represents the set of texture model basis vectors, and vector c represents the texture model. Represents a set of coefficients (ie, global texture descriptor 230), and vector m represents the average texture vector.

を評価することである。 After the base vectors (B) and mean vectors (vector m) for the 2D model 200, 3D model 300, and global texture model 220 are determined offline and stored for runtime use, the formula By solving 1 for vector c, the model coefficients (representing 2D descriptor 210, 3D descriptor 215, and global texture descriptor 230) can be determined. Since B is not necessarily a square matrix, the simple algebraic solution of Equation 1 for determining the vector c may not be available. Thus, vector c can be determined at runtime by any one of a number of optimization procedures. One such procedure has the following relationship:

Is to evaluate.

  As an example, if each of the landmark image 115 and the normalized image 305 is represented by a (128 × 128) element array, the vector i may be a (16,384 × 1) vector. It can be seen from Equation 1. Further, when ‘n1’ represents the number of basis vectors in B, B is a matrix of (16,384 × n1), and vector m is a vector of (16,384 × 1). In this example, the 2D descriptor 210, the 3D descriptor 215, and the global texture descriptor 230 are (n1 × 1) vectors. In one embodiment, the 3D model coefficients can be obtained using the technique described in co-pending US patent application Ser. No. 13 / 299,211 entitled “3D Object Recognition”.

  Referring again to FIG. 3, the normalized image 310 can also be provided to the local texture model 225. As shown, the local texture model 225 itself can include a local image descriptor 315, a dense image descriptor 320, and a dense distorted image descriptor 325.

  Referring to FIG. 4a, in one embodiment, the local image descriptor 315 may be based on the texture of one or more surrounding small regions or tiles 400 of landmark features ( Only one exemplary region is listed in FIG. 4a). The exact number of tiles depends on the resolution of the image and the designer's objectives / constraints, but for a (128 × 128) normalized image, 10-20 tiles have been found to be sufficient. Yes. The size of each tile can be determined based on training data and can vary on a fixed number of scales, where each point can have multiple tiles of different sizes. is there. It will be appreciated that the settings used in practice can be based on those that provide the best recognition performance (within established design constraints). As examples, local image descriptors include gradient histograms (HoG), speed-up robust features (SURF), scale-invariant feature transformations (SIFT), binary robust binary robust independent features (BREF), and oriented BRIEF (ORB). ) Or similar type descriptors. Referring to FIG. 4b, in one embodiment, the dense image descriptor 320 determines the image descriptor based on the entire image. For example, a selected descriptor operation (eg, HoG or SIFT) can be applied to each of a plurality of regions (eg, 20 regions 405 such as a 5 × 4 grid) covering the image 305. The result of the local image descriptor 315 is a j element descriptor. The result of the dense image descriptor is a k-element descriptor.

  Although both local image descriptor 315 and dense image descriptor 320 have been described as using gradient vector descriptors, this is not required. For example, other descriptors such as intensity-based descriptors and image texture bases may be used. Further, the local image descriptor 315 may use one technique (eg intensity) and the dense image descriptor 320 may use another technique (eg gradient vector).

  In addition to using the normalized image 310 directly, the local texture model 225 may use a distorted version of the image 310. Referring again to FIG. 3, the normalized image 310 can be applied to the distortion operator or deformation operator 330 to generate a distortion image or deformation image 335. In one embodiment, the distortion operator 330 adjusts the face by out-of-plane rotation so that the distortion image 335 approximates a true front image of the subject's face. Referring to FIG. 5, similar to the operations described for generating dense image descriptor 320, the entire distorted image 335 is region-wise (eg, 500) as described above for dense image descriptor 320. Can be evaluated. In one embodiment, the dense distortion image descriptor is an l-element descriptor. The operation is similar, but the dense distortion image descriptor 325 need not use the same technique or the same number of regions / tiles used to generate the dense image descriptor 320.

である。 Referring back to FIG. 3, the combination operation 340 may perform any two, any three, or any combination of the generated local image descriptor, dense image descriptor, and dense distortion image descriptor. In combination, an intermediate local texture descriptor 345 can be generated. The combinatorial operation 340 can handle the whole of each descriptor, or only a part of each descriptor, or can handle the whole of one descriptor and only part of another descriptor. The combination operation 340 is a concatenation of each of the local image descriptor (j element), the dense image descriptor (k element), and the dense distortion image descriptor (l element). Can do. In such an embodiment, the intermediate local texture descriptor 345 is (j + k + l). In one implementation,

It is.

として表現することができる変換と見なすことができ、ここで、

（これ以降ベクトルｙと記載する）は、ローカル・テクスチャ記述子２３５を表し、Ｍは、所望の変換を行う基底ベクトル（通常、直交する）のセットを表し、

（これ以降ベクトルｘと記載する）は、中間のローカル・テクスチャ記述子３４５を表す。顔の大規模な集合についてベクトルｙの分布を知っている場合には、実質的に同じ情報を表す、より少ない数の要素（次元）を識別及び保持することができる。変換行列Ｍは、既知の多数の最適化技術（例えば、メトリック学習、特徴選択、又は主成分分析）のいずれかを用いて、オフラインで決定することができる。決定されると、Ｍは、実行時の使用のために格納することができる。上記で始めた数値例を続けると、中間のローカル・テクスチャ記述子３４５（ベクトルｘ）が３，０００個の要素を有し、Ｍがこの次元をｎ２次元まで縮小する場合には、ベクトルｙは（ｎ２×１）のベクトルであり、Ｍは（ｎ２×３，０００）の行列であり、ベクトルｘは（３，０００×１）のベクトルである。 In order to reduce the size of this descriptor to a value that is easier to compute in real time, a dimension reduction operation 350 can be performed to generate a local texture descriptor 235. Alternatively, dimension reduction can be performed on these components prior to combining the individual components (315, 320, 325) at 340. The dimension reduction is as follows:

Can be viewed as a transformation that can be expressed as

(Hereinafter referred to as vector y) represents the local texture descriptor 235, M represents the set of basis vectors (usually orthogonal) that perform the desired transformation,

(Hereinafter referred to as vector x) represents an intermediate local texture descriptor 345. If the distribution of the vector y is known for a large set of faces, a smaller number of elements (dimensions) representing substantially the same information can be identified and retained. The transformation matrix M can be determined offline using any of a number of known optimization techniques (eg, metric learning, feature selection, or principal component analysis). Once determined, M can be stored for runtime use. Continuing with the numerical example started above, if the intermediate local texture descriptor 345 (vector x) has 3,000 elements and M reduces this dimension to n2 dimensions, then the vector y is (N2 × 1) vector, M is a (n2 × 3,000) matrix, and vector x is a (3,000 × 1) vector.

  Referring again to FIG. 3, after each of the descriptors 210, 215, 230, and 235 is determined, these descriptors can be combined by the operator 140 to create a face feature vector 145. As shown in FIG. 6, the face feature vector includes a field for the 2D shape descriptor 210 (600), a field for the 3D shape descriptor 215 (605), and a global texture descriptor 230. A field (610) and a field (615) for the local texture descriptor 235 may be included.

さらに、ローカル画像記述子３１５、密な画像記述子３２０、及び密な歪み画像記述子３２５の組み合わせが、３，０００要素を有する中間のローカル・テクスチャ記述子３４５を生成し、次元縮小演算３５０が、数式３によって特徴付けられて、次元の数をｎ２次元に縮小する場合には、次元縮小演算３５０についての例示的なモデル・パラメータは、表２に示されるようなものになる。

最後に、組み合わせ演算子１４０が２Ｄ記述子２１０、３Ｄ記述子２１５、グローバル・テクスチャ記述子２３０、及びローカル・テクスチャ記述子２３５の各々を連結する場合には、顔特徴ベクトル１４５は、（（３ｎ１＋ｎ２）×１）のベクトルである。 Referring again to the numerical example that began above, the 2D model 200, 3D model 205, and global texture model 220 are linear models of the form given by Equation 1, and the model input image is (128 × 128) If there are n1 basis vectors in each of the 2D model, the 3D model, and the global texture model, exemplary model parameters are as shown in Table 1.

In addition, the combination of local image descriptor 315, dense image descriptor 320, and dense distorted image descriptor 325 generates an intermediate local texture descriptor 345 having 3,000 elements, and dimension reduction operation 350 Characterized by Equation 3, when reducing the number of dimensions to n2 dimensions, exemplary model parameters for the dimension reduction operation 350 would be as shown in Table 2.

Finally, if the combination operator 140 concatenates each of the 2D descriptor 210, 3D descriptor 215, global texture descriptor 230, and local texture descriptor 235, the face feature vector 145 is ((3n1 + n2 ) × 1) vector.

  Referring to FIG. 7, a face recognition operation 700 using facial features according to one embodiment is shown. Initially, facial feature vectors for unknown and known faces / identities are obtained (blocks 705 and 710). A similarity metric is then applied to these vectors (block 715) and a check is made to determine if the metric matches (block 720). If the two facial feature vectors are sufficiently similar (“Yes” branch of block 720), a determination is made that the unknown facial feature vector represents the same identity associated with the known facial feature vector. (Block 725). If the two facial feature vectors are not similar enough to match (the “No” branch of block 720), a further check is made to determine if another known facial feature vector is available. (Block 730). If there are no more face feature vectors associated with the known identity (the “No” branch of block 730), the unknown face feature vector (ie, obtained during operation by block 705) is unknown. Can be concluded (block 735). If there are more facial feature vectors associated with known identities (“Yes” branch of block 730), for example, the “next” known facial feature vector can be obtained from storage 150 (block 740), and then The operation 700 resumes at block 715.

ここで、ベクトルｘは第１の顔特徴ベクトル（例えば未知の顔に関連する顔特徴ベクトル）を表し、ベクトルｙは第２の顔特徴ベクトル（例えば既知の顔に関連する顔特徴ベクトル）を表し、Ｓ（ ）は類似性又は比較演算を表し、Ｗは重み行列を表す。本質的に、重み行列Ｗは、比較演算の際に、顔特徴ベクトルにおける各々の要素がどの程度重要又は有意であるかを特定する。既知の身元に関連する大量の顔特徴ベクトルを用いて、評価基準学習技術を適用し、Ｗをオフラインで決定することができる。Ｗが分かると、Ｗは、図７による実行時の使用のために格納することができる。例として、顔特徴ベクトルが５００個の要素を有する場合、即ち（５００×１）のベクトルによって表される場合には、Ｗは、（５００×５００）要素の重み行列となる。 In one embodiment, the similarity metric (see block 715) may be a distance metric along the Hamming distance line. In the case of vectors of large dimensions such as the face feature vectors described herein, it has been found that the Mahalanobis distance measure described by Equation 4 is an effective similarity measure.

Here, the vector x represents a first face feature vector (for example, a face feature vector related to an unknown face), and the vector y represents a second face feature vector (for example, a face feature vector related to a known face). , S () represents similarity or comparison operation, and W represents a weight matrix. In essence, the weight matrix W specifies how important or significant each element in the face feature vector is during the comparison operation. Using a large number of facial feature vectors associated with known identities, an evaluation criterion learning technique can be applied to determine W offline. Once W is known, it can be stored for runtime use according to FIG. As an example, if the face feature vector has 500 elements, i.e., represented by a (500 x 1) vector, W is a (500 x 500) element weight matrix.

  Referring to FIG. 8, the receiver operating characteristic (ROC) curve 800 shows the individual components that make up the face feature vector: (1) standard 2M descriptor (810), dense gradient descriptor (815). FIG. 6 illustrates the performance of the face feature vector (805) disclosed herein for a local gradient descriptor (820) and a dense distortion gradient descriptor (825) used alone. As can be seen, the use of facial feature vectors in accordance with the present disclosure provides higher performance than these other descriptors.

  Referring now to FIG. 9, a simplified functional block diagram of an exemplary electronic device 900 according to one embodiment of the present invention is shown. The electronic device 900 includes a processor 905, a display 910, a user interface 915, graphics hardware 920, a device sensor 925 (eg, proximity sensor / ambient light sensor, accelerometer, and / or gyroscope), microphone 930, One or more audio codecs 935, one or more speakers 940, a communication circuit 945, a digital image capture unit 950, one or more video codecs 955, a memory 960, a storage 965, and a communication bus 970 Can be included. The electronic device 900 can be, for example, a personal digital assistant (PDA), a portable music player, a mobile phone, a notebook computer, a laptop computer, or a tablet computer.

  The processor 905 executes instructions necessary to execute or control a number of function operations performed by the apparatus 900 (eg, face feature vector construction and runtime face recognition operation 100 or face recognition operation 700, etc.). Can do. The processor 905 can drive the display 910 and receive user input from the user interface 915, for example. User interface 915 allows a user to interact with device 900. For example, the user interface 915 can take various forms such as buttons, keypads, dials, click wheels, keyboards, display screens, and / or touch screens. The processor 905 can also be a system-on-chip, such as found in a portable device, and can include a dedicated graphics processing unit (GPU). The processor 905 may be based on a reduced instruction set computer (RISC) or compound instruction set computer (CISC) architecture, or any other suitable architecture, and may include one or more processing cores. Can be included. Graphics hardware 920 may be dedicated computing hardware for processing graphics and / or support processor 905 for processing graphics information. In one embodiment, the graphics hardware 920 can include a programmable graphics processing unit (GPU).

  The sensor and camera circuit 950 may be at least in part by one or more video codecs 955 and / or processors 905 and / or graphics hardware 920 and / or by dedicated image processing units incorporated within the circuit 950. Still images and video images that may be processed can be captured. Images so captured can be stored in memory 960 and / or storage 965. Memory 960 may include one or more different types of media used by processor 905 and graphics hardware 920 to perform the functions of the device. For example, the memory 960 can include a memory cache, read only memory (ROM), and / or random access memory (RAM). Storage 965 includes media for holding audio, image, and video files, computer program instructions or software, selection information, device profile information, and any other suitable data. Storage 965 can be, for example, optical media such as magnetic disks and tapes (fixed, floppy, and removable), CD-ROMs and digital video disks (DVDs), and electrically programmable read-only. One or more permanent storage media may be included, including semiconductor memory devices such as memory (EPROM) and electrically erasable programmable read only memory (EEPROM). Using memory 960 and storage 965, computer program instructions or code organized in one or more modules and written in any desired computer programming language can be tangibly maintained. For example, when executed by the processor 905, such computer program code may implement one or more of the methods described herein.

  Various modifications of materials, components, circuit elements, and details of the illustrated method of operation are possible without departing from the scope of the claims. For example, although the models described herein were in linear form, such limitations are not specific to the disclosed technique. Further, the various models can be different, some models can be linear and others can be non-linear. In addition, the combination operations (eg, 140 and 340) are not limited to concatenation operations and they need not be the same. Any combination that suits the designer's purpose can be used. For example, linear combinations, selection of a subset of descriptor values, and their weighted combinations are all feasible. Further, when the dimension of the model descriptor does not require dimensional reduction (for example, operations 315, 320, and 325), it is not necessary to perform this operation.

  Finally, it should be understood that the above description is intended to be illustrative and not restrictive. For example, the above-described embodiments can be used in combination with each other. Many other embodiments will be apparent to those of skill in the art upon reviewing the above description. Accordingly, the scope of the invention should be determined by reference to the claims, and by the full scope of equivalents to which such claims are applicable. In the claims, the terms "including" and "in which" mean plain English for the terms "comprising" and "wherein", respectively. Is used as an equivalent term.

100: face feature vector generation and runtime face recognition calculation 105: input image 110: face detector 115: landmark image 120: shape model 125: texture model 130: shape descriptor 135: texture descriptor 140: calculation 145 : Face feature vector 150: storage 155: application 160: image 165: comparison 200: 2D shape model 205: 3D shape model 210: 2D shape descriptor 215: 3D shape descriptor 220: global texture model 225: local texture Model 230: Global texture descriptor 235: Local texture descriptor 300: Face feature vector construction operation 305: Normalization operation 310: Normalized image 315: Local image descriptor 320: Dense image descriptor 325: Dense distortion image description 330: Distortion operator 335: Distortion image 340: Combination operation 345: Intermediate local texture descriptor 350: Dimension reduction operation 400: Tile 405, 500: Region 600: 2D shape field 605: 3D shape field 610: Global texture Field 615: Local texture field 800: Receiver operating characteristic curve 805: Face feature vector 810: Standard 2M descriptor 815: Dense gradient descriptor 820: Local gradient descriptor 825: Distorted gradient descriptor 900: Electronic device 905: Processor 910: Display 915: User interface 920: Graphics hardware 925: Device sensor 930: Microphone 935: Audio codec 940: Speaker 945: Communication circuit 950: Digital The image capturing unit (sensor / camera circuit)
955: Video codec 960: Memory 965: Storage 970: Communication bus

Claims (14)

Computer code for obtaining landmark detection information for a first face in a first image;
Computer code for generating normalized landmark detection information based at least in part on the landmark detection information;
Computer code for generating a first shape model feature vector based at least in part on the landmark detection information, wherein the landmark detection information is applied to a two-dimensional model of the first face. Including such computer code,
Computer code for generating a second shape model feature vector based at least in part on the landmark detection information, comprising applying the landmark detection information to the three-dimensional model of the first face The computer code and
Computer code for generating a first texture model feature vector based at least in part on the landmark detection information, wherein the first code is generated at least in part on the normalized landmark detection information. Computer code comprising generating a texture model feature vector of
Computer code for generating a second texture model feature vector based at least in part on the landmark detection information , wherein at least in part, the normalized landmark detection information and a specific morphing operation The computer code comprising generating distortion landmark detection information based on
The first face feature is obtained by combining the first shape model feature vector, the second shape model feature vector, the first texture model feature vector, and the second texture model feature vector. Computer code forming a vector;
Computer code for storing the first facial feature vector in a storage device;
A permanent computer readable medium characterized by comprising: The computer code for generating a first texture model feature vector is:
Identifying a plurality of regions less than all of the normalized landmark detection information in the normalized landmark detection information;
Generating a first texture model feature vector based on the plurality of regions;
The persistent computer readable medium of claim 1 , comprising computer code for: The computer code for generating a second texture model feature vector comprises computer code for generating a second texture model feature vector based at least in part on the distortion landmark detection information. The persistent computer readable medium of claim 1 , comprising: The computer code for generating a second texture model feature vector combines the first texture model feature vector and the second texture model feature vector to form a first face feature vector before carrying out the computer code for, characterized by further comprising computer code for reducing the dimensionality of the second texture model feature vector, persistent computer according to claim 1 A readable medium.   The computer code for acquiring landmark detection information includes computer code for acquiring landmark detection information about the first face in the first image by face detection calculation. The persistent computer readable medium of claim 1.   The computer code for combining the first shape model feature vector, the second shape model feature vector, the first texture model feature vector, and the second texture model feature vector A computer for concatenating the first shape model feature vector, the second shape model feature vector, the first texture model feature vector, and the second texture model feature vector; The persistent computer readable medium of claim 1, comprising code. The computer code for storing the first facial feature vector in a storage device is:
Incorporating the first facial feature vector into the metadata of the first image;
Storing the first image having the first face feature vector in a storage device;
The persistent computer readable medium of claim 1, comprising computer code for: Extracting the first face feature vector from the storage device;
A second facial feature vector further corresponding to a known person is extracted from the storage device,
Comparing the first face feature vector and the second face feature vector to generate a similarity value;
If the similarity values indicate matching, it is determined that the first face corresponds to the known person;
The persistent computer readable medium of claim 1, further comprising computer code for: A storage device storing a plurality of images;
A memory that is communicatively coupled to the storage device and that stores the computer code of claim 1;
A programmable control unit, communicatively coupled to the storage device and the memory, and configured to retrieve and execute the computer code stored in the memory;
Including electronic devices. Computer code for obtaining a landmark image identifying a plurality of aspects of the first face for the first face in the first image;
Computer code for generating a normalized landmark image based at least in part on the landmark image;
Computer code for generating a distorted landmark image based at least in part on the normalized landmark image;
Computer code for generating a first shape model feature vector based at least in part on the landmark image, the computer code comprising applying the landmark image to a two-dimensional model of the first face Computer code,
Computer code for generating a second shape model feature vector based at least in part on the landmark image, the computer code comprising applying the landmark image to a three-dimensional model of the first face Computer code,
Computer code for generating a first texture model feature vector based at least in part on the normalized landmark image, the plurality of regions in the normalized landmark image The computer code including using gradient vector operations;
Computer code for generating a second texture model feature vector based at least in part on the distorted landmark image , wherein the first and second are based at least in part on the normalized landmark image. Generating the second descriptor and generating a third descriptor based at least in part on the distorted landmark image;
The first face feature is obtained by combining the first shape model feature vector, the second shape model feature vector, the first texture model feature vector, and the second texture model feature vector. Computer code forming a vector;
Computer code for storing the first facial feature vector in a storage device;
A permanent computer readable medium characterized by comprising: The persistent computer readable medium of claim 10 , wherein the plurality of regions are less than all of the normalized landmark images . The computer code for generating a second texture model feature vector combines the first, second, and third descriptors to form the second texture model feature vector The persistent computer readable medium of claim 10 , further comprising computer code. The computer code for combining the first, second, and third descriptors comprises computer code for reducing the dimensions of the combined first, second, and third descriptors. The persistent computer readable medium of claim 12 , further comprising: A storage device storing a plurality of images;
A memory operatively coupled to the storage device and having stored the computer code of claim 10 ;
A programmable controller that is communicatively coupled to the storage device and the memory and configured to execute the computer code stored in the memory;
A computer system including:

Images