JP2004038937A - Method and device for face description and recognition using high-order eigen-component - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a method and device for face description and recognition using a high-order eigen-component. <P>SOLUTION: High-order eigen-components are proposed to describe detailed regional information on a particular facial component. A formula is given to calculate a high-order transform matrix for projection. High-order component features can be used individually to describe a facial component or combined with first-order component features. The combination of first-order component features and high-order component features of eyes, eyebrows, nose, mouth and outline, which depends on the significance of identity information on the corresponding area, has better face description capability than first-order eigen-face features or combined first-order and high-order eigen-face features. <P>COPYRIGHT: (C)2004,JPO

Description

【００１３】
すべての顔成分は平均の成分と、次式のベクトルだけ異なる。
【数２】

【００１４】
データの共分散行列は従って次式で定義される。
【数３】

ここで、
【数４】

【００１５】
Ｑの次元はｗｈ×ｗｈであり、ｗとｈはそれぞれ成分の幅と高さである点に留意されたい。この行列の大きさは巨大であるが、有限個数の成分ベクトルＭだけを加算するため、この行列の階数はＭ−１を超えることはない。ｖ_ｉ ^（１）が

（ｉ＝１，２，．．．Ｍ）の固有ベクトルであるならば、λ_ｉ ^（１）を

の固有値として

であり、従って上式の左辺にＡ^（１）を乗じることにより

は

の固有ベクトルになることに留意する。
【数５】

【００１６】
従って、固有ベクトルｖ_ｉ ^（１）、および固有値λ_ｉ ^（１）は次式によって得られる。
【数６】

【００１７】
しかし

の大きさはＭ×Ｍに過ぎない。従ってｕ_ｉ ^（１）を

の固有ベクトルとして次式を得る。
【数７】

【００１８】
固有値λ_ｉ ^（１）は、固有ベクトルｕ_ｉ ^（１）が張る新しい座標空間に沿った分散である。これ以降、ｉの次数が大きいほど固有値λ_ｉ ^（１）を減少するものと仮定する。固有値は指数的に減少する。従って、

を計算することにより（ここに

かつ１≦ｋ≦Ｍ_１）、顔成分Γ^（１）をＭ_１＜＜Ｍ次元だけに射影することができる。ｗ_ｋ ^（１）は新座標系におけるΓ^（１）のｋ番目の座標である。この前提でＷ^（１）を一次成分特徴と呼ぶ。ベクトルｕ_ｋ ^（１）は実際には画像であり、一次固有成分と呼ぶ。
【００１９】
【数８】

これによれば、次式を得る。
【数９】

【００２０】
Ｕ^（１）はＭ_１×Ｐ行列であるため、逆行列は得られない。しかし、その擬似逆行列を用いて逆行列を近似することができる。

を次式のように

の擬似逆行列とする。
【数１０】

これによれば、次式を得る。
【数１１】

ここに、

はＷ^（１）およびＵ^（１）から再構成された行列である。
【００２１】
次いで、次式が残余成分Г_ｉ ^（２）を得るために実行される。
【数１２】

【００２２】
残余の顔成分ベクトルは依然として個々の成分の情報を豊富に含んでいるため、顔成分特徴を再び残余成分から抽出する必要がある。

、λ_ｉ ^（２）を

の固有値、ｖ_ｉ ^（２）を

の対応する固有ベクトルとすると、

である。上述の議論に基づいて、

の固有ベクトルは

である。従って、

を計算することにより、残余成分Γ^（２）をＭ_２＜＜Ｍ次元だけに射影することができる。ここに、
【数１３】

かつ１≦ｋ≦Ｍ_２である。ｕ_ｋ ^（２）は残余成分の固有ベクトルであるため、ｕ_ｋ ^（２）を二次固有成分、ｗ_ｋ ^（２）を二次成分特徴と呼ぶ。
【００２３】
【数１４】

とすれば、式（１３）は以下のように書ける。
【数１５】

また、
【数１６】

とすれば次式を得る。
【数１７】

【００２４】
Ｕ_２は定数変換行列であって一度しか計算されないため、計算効率には影響しない。顔成分は次式で記述できる。
【数１８】

ここに、

である。Ω（Φ）を計算する際の計算負荷は、固有成分Ｕから成分特徴を計算するだけの場合と比べて増大はしない。
【００２５】
残余成分を二次残余成分と呼び、元の成分を一次残余成分と呼ぶ。
【００２６】
同じ方法により、三次、四次、・・・、ｎ次固有成分を導くことができる。対応する次数の残余成分を射影することにより、三次、四次、・・・、ｎ次成分特徴が得られる。これらの高次成分特徴を用いて、成分の類似度を射影間の重み付ユークリッド距離として定義することができる。図２にｉ次固有成分Ｕ^（ｉ）および対応する変換行列Ｕ_ｉの計算手順を示す。同図においてＰｓｅｕｄｏ＿Ｉｎｖ（Ｂ）は行列Ｂの擬似逆行列を計算する関数である。
【００２７】
二つの顔Ｈ_１とＨ_２の非類似度を、固有成分（例：固有目、固有眉、固有鼻、固有口および固有輪郭）と固有顔の射影から生成された各種の顔成分特徴間の合成距離として定義する。
【数１９】

ａ１＝０の場合、顔画像の類似度は二次特徴だけを用いて測定される。
【００２８】
上述の方法により、顔画像を記述する方法は以下のように考察できる。
１）成分ウインドウ左上隅から始めて成分ウインドウの右下隅までラスタスキャン方式により前記顔成分を走査して一次元ピクセル配列を得る。
２）前記一次元ピクセル配列から平均成分を減算する。
３）前記減算された一次元ピクセル配列に前記一次および高次固有成分を乗ずる。
４）得られた成分の特徴を顔の記述として用いる。
５）前記特徴を符号化表現としてコーディングする。
【００２９】
本方法により、人物の顔の識別情報の重要度応じて異なる注目度で重み付けを行ない、人物の顔を効果的かつ空間効率良く記述することができる。
【００３０】
次に、本発明による顔特徴抽出の全体の演算を、図３および４を参照して説明する。演算は図４に示される学習モード演算（ステップ＃２２〜＃３１）、および図３に示される試験モード演算（ステップ＃３２〜＃４２）を含む。学習モードでは、まず、多くの顔サンプルを学習し、累積し、かつ一次平均の係数を得るための演算を行なう。
【００３１】
学習モードはステップ＃２２から開始し、ステップ＃３１まで続く。学習モードは、試験モードで使用される種々のパラメータを生成するために設けられている。
【００３２】
ステップ＃２２において、複数のサンプル顔画像が入力される。
【００３３】
ステップ＃２３において、各サンプル顔画像は、右目、左目、右眉、左眉、鼻、口、輪郭のような、複数の顔部分に分割され、各部分は解析されて、基本顔成分Φ_ｉを得る。顔成分Φ_ｉは人の顔の識別性の重要度に基づいて重み付けがなされる。
【００３４】
ステップ＃２４において、鼻部分のような同じ部分の顔成分が複数のサンプル顔画像から収集される。鼻部分の顔成分は鼻成分と呼ばれる。各部分の収集された顔成分が平均されて、式（１）を用いて一次平均顔成分Φを得る。同じ演算が実行されて、異なる顔成分に対する一次平均顔成分を得る。一次平均顔成分Φは、ステップ＃３４における試験モード演算で使用される。
【００３５】
ステップ＃２５において、各サンプル顔画像の任意の顔成分、例えば鼻の顔成分が、鼻の一次平均顔成分Φで引き算され、式（２）によって与えられる鼻のベクトルを得る。異なる顔成分の各々に対して、同じ演算が実行される。
【００３６】
ステップ＃２２〜＃２５はまとめて、学習用顔画像を解析する解析ステップと呼ばれる。
【００３７】
ステップ＃２６において、式（４）、（５）、（６）、（７）、および（８）が実行され、一次固有成分Ｕ^（１）を得る。一次固有成分Ｕ^（１）は、ステップ＃３５における試験モード演算で使用される。
【００３８】
ステップ＃２７において、式（１０）を用いて逆行列Ｕ^（１）＋が生成される。逆行列Ｕ^（１）＋は、Ｕ^（１）Ｔにほぼ等しい。逆行列Ｕ^（１）＋は、ステップ＃３７における試験モード演算で使用される。
【００３９】
ステップ＃２８において、逆行列Ｕ^（１）＋は、式（１１）を用いて顔成分の差を得るために使用される。こうして、ステップ＃２２で収集された元の顔画像の基本顔成分に対する顔成分の差が得られる。ステップ＃２８で生成されたデータは差顔成分と呼ばれる。
【００４０】
ステップ＃２９において、差顔成分と一次平均顔成分との差が得られる。
【００４１】
ステップ＃２７〜＃２９はまとめて、一次固有成分を解析する解析ステップと呼ばれる。
【００４２】
ステップ＃３０において、ステップ＃２９で得られる差を用いて、二次Ｋ−Ｌ係数Ｕ^（２）（これは二次固有成分とも呼ばれる）が、式（４）、（６）、（７）、（１４）、および（１６）を用いて計算される。二次Ｋ−Ｌ係数Ｕ^（２）は、Ｋ−Ｌ変換（ステップ＃４０）における試験モードで使用される。
【００４３】
試験モードは、ステップ＃３２から始まり、ステップ＃４２まで続く。一次成分特徴Ｗ^（１）およびニ次成分特徴Ｗ^（２）を生成するために、試験モードが設けられている。
【００４４】
ステップ＃３２において、試験対象であるひとつの顔画像が入力される。
【００４５】
ステップ＃３３において、入力された顔画像は、右目、左目、右眉、左眉、鼻、口、輪郭のような、複数の顔部分に分割され、各部分は解析されて、基本顔成分Φ_ｉを得る。顔成分Φ_ｉは人の顔の識別の重要度に基づいて重み付けがなされる。
【００４６】
ステップ＃３４において、基本顔成分が一次平均顔成分Ψで引き算され、第１差Γ^（１）（一次平均残余成分とも呼ばれる）を得る。第１差Γ^（１）はステップ＃３５およびステップ＃３９で利用される。
【００４７】
ステップ＃３２〜＃３４はまとめて、試験用顔画像を解析する解析ステップと呼ばれる。
【００４８】
ステップ＃３５において、第１差Γ^（１）および一次固有成分Ｕ^（１）を用いて、式（９）で示されるＫ−Ｌ変換が実行され、一次成分特徴Ｗ^（１）を得る。
【００４９】
ステップ＃３６において、一次成分特徴Ｗ^（１）が生成される。一次成分特徴Ｗ^（１）は、ステップ＃３２で入力された試験用顔の特徴を表わす。一次成分特徴Ｗ^（１）は試験用顔を表わす情報として使用され得るが、比較的大きいサイズのデータを有する。次に説明する更なる計算が実行されれば、データサイズを減らすことが出来る。
【００５０】
ステップ＃３７において、式（１１）を用いて、Ｋ−Ｌ逆変換が実行されて、再構成行列

を生成する。
【００５１】
ステップ＃３８において、再構成行列

が生成される。
【００５２】
ステップ＃３９において、式（１２）が実行され、第１差Γ^（１）と再構成行列

との差を生成して、第２差Γ^（２）を得る。これは通常第２残余成分Γ_ｉ ^（２）と呼ばれる。
【００５３】
ステップ＃３７〜＃３９はまとめて、一次成分特徴を解析する解析ステップと呼ばれる。
【００５４】
ステップ＃４０において、第２差Γ^（２）および二次Ｋ−Ｌ係数Ｕ^（２）を用い、式（１７）によるＫ−Ｌ変換が実行されて、二次成分特徴Ｗ^（２）を生成する。
【００５５】
ステップ＃４１において、二次成分特徴Ｗ^（２）が生成される。二次成分特徴Ｗ^（２）は、試験モードにおいて入力された試験用顔の特徴を表わす情報として用いられる。
【００５６】
図３および４に示されるフローチャートは、サンプル顔画像および試験用顔画像を捕らえるカメラが接続されたコンピュータによって実行される。２つのコンピュータのセットを準備することも可能である。一方のセットは学習モード演算用とし、他方のセットは試験モード演算用として用いる。各セットにコンピュータとカメラを含むことも可能である。学習モード演算用セットは、ステップ＃２２〜＃３０を実行する様プログラムされている一方、試験モード演算用セットは、ステップ＃３２〜＃４２を実行する様プログラムされているようにすることも出来る。試験モード演算用セットにおいて、記憶装置が設けられて、一次平均顔成分Ψ、一次固有成分Ｕ^（１）、逆行列Ｕ^（１）＋、およびニ次固有成分Ｕ^（２）のような、学習モード演算用セットで得られた情報を前もって格納することも可能である。
【００５７】
本発明は、成分主体の特徴を用いることにより、人物の顔記述を極めて効率的に得ることが出来る。高次固有成分が学習用成分を用いて一度だけ計算すれば良いため、高次成分特徴は一次成分特徴と同程度の効率で得られる。しかし、詳細な局部的な識別情報は高次成分特徴により明らかになるため、目、眉、鼻、口および輪郭の一次成分特徴と高次成分特徴にさまざまな注目度の重み付けを組み合わせれば、一次固有顔特徴のみ、または一次固有顔特徴と高次固有顔特徴との組合わせだけの場合よりも性能の良い顔記述を得ることが出来る。
【００５８】
本発明は、インターネット・マルチメディア、データベース検索、ビデオ編集、電子図書館、監視および追跡、および顔認識と認証を利用するその他広範な用途に利用可能な顔記述を極めて効果的かつ効率的に提供する。
【図面の簡単な説明】
【図１】一次特徴Ｗ^（１）の計算手順を示すフローチャートである。
【図２】ｉ次固有成分Ｕ^（ｉ）および対応する変換行列Ｕ_ｉの計算手順を示すフローチャートである。
【図３】学習モード演算のフローチャートである。
【図４】試験モード演算のフローチャートである。[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to a method and apparatus for facial description and recognition using higher-order eigencomponents. The present invention can be used for face description and recognition for content-based image retrieval, human face identification and authentication, monitoring and tracking for banks, security systems and videophones, electronic libraries and Internet multimedia databases. .
[0002]
[Prior art]
Human face recognition is an active area in the field of computer vision. Face recognition will play an important role in multimedia database searching and various other applications. In recent years, there has been considerable progress in the face detection and recognition problem, and various techniques have been proposed. Of these, neural networks, elastic template matching, Karhunen-Loeve expansion, algebraic moments and isopycnic lines are typical methods.
[0003]
Among these methods, principal component analysis (PCA) or Karhunen-Loeve expansion is an important trend. The eigenface method is derived from PCA, is computationally convenient, and the discriminating ability is consistently accurate. Previous studies have shown that the PCA approach can naturally distinguish between different types of information. An eigenvector having a large eigenvalue acquires information common to the partial structure of the face, and an eigenvector having a small eigenvalue acquires information unique to each face. Studies have shown that only the information contained in eigenvectors with large eigenvalues can be generalized to unlearned new faces.
[0004]
[Problems to be solved by the invention]
An advantage of the eigenface method is that an eigenvector having a large eigenvalue has information corresponding to the basic shape and structure of the face. This means that main features of a person's face can be described using features extracted from eigenvectors having large eigenvalues. However, this is also a weak point of PCA. If only features extracted from eigenvectors with large eigenvalues are taken into account, it is not possible to obtain facial details corresponding to individual faces. If these details of individual faces can be described by common features of the face of the person, the face description of the person will be more accurate.
[0005]
One of the drawbacks of the eigenface method is that the contribution ratios of the face components are all equal. The identification information is not evenly distributed over the entire face but exists mainly in the center of the face, such as the eyes, eyebrows, nose, mouth, and contour. The cheek portion contains little identification information and is susceptible to changes in lighting conditions and changes in facial expressions. If the importance of discrimination is available for the face component, the recognition of the face of a person may be performed more accurately.
[0006]
[Means for Solving the Problems]
The eigenface method is efficient for extracting general features of a face, such as a shape and a structure. In order to capture the details of the face that would be lost if the eigenvectors with lower eigenvalues were truncated, it would be necessary to obtain a face reconstructed with features from the eigenvectors with higher eigenvalues. The reconstructed face image gives a residual image between the original image and the reconstructed matrix. These residual faces are considered high-passed face images and still contain a wealth of individual face detail information. To describe these residual faces, the eigenface method can be applied again to these residual faces. The acquired eigenvectors with large eigenvalues will reveal features common to residual faces. According to this method, a higher-order eigenvector having a large eigenvalue can be acquired and a corresponding feature can be extracted. A face can be efficiently described by combining features obtained from eigenfaces of different orders.
[0007]
Similarly, primary and higher order principal components (eigencomponents) of the face component can be obtained and the features of the corresponding face region can be described. By combining and using these features obtained from eigen components of different orders, individual face components can be described efficiently. Finally, a person's face can be represented by a combination of eigencomponents of different orders weighted with various degrees of attention. Depending on the various applications, the functionality (advantages and disadvantages) of the various components will, of course, vary. Different weights need to be assigned to each component.
[0008]
[Action]
The present invention provides a method for interpreting a person's face that can be used for image retrieval (query with sample faces), person identification and authentication, monitoring and tracking, and other face recognition applications. To describe facial features, we propose the concept of higher-order eigencomponents based on the inventors' observations and derivations. First, all face images are normalized to a standard size. Then calculate the vertical position of the eyes and move the face to the appropriate position. When all of these preprocessings are completed, the eigencomponents and higher-order eigencomponents can be obtained from the set of learning face images. In order to search for a face image in the face database, features of the image onto which the eigencomponents and higher-order eigencomponents are projected can be calculated from the selected eigencomponents and higher-order eigencomponents. A face can be described by combining these features. With this description, the similarity can be measured using the Euclidean distance. In order to increase the accuracy of the similarity, it is necessary to weight the features.
[0009]
BEST MODE FOR CARRYING OUT THE INVENTION
The present invention provides a method for extracting a higher-order eigencomponent feature by combining features of components of different orders and expressing a face.
[0010]
Using the normalized face image, the eigen component and the higher-order eigen component can be obtained as follows.
[0011]
First, face components (eyes, eyebrows, nose, mouth, contour, etc.) are obtained using predetermined blocks of the normalized face image.
[0012]
An average component Ψ is defined by the following equation with respect to a face component Φ _i which is a one-dimensional vector of the face component represented by the raster scan.
(Equation 1)

[0013]
All face components differ from the average component by the vector of:
(Equation 2)

[0014]
The covariance matrix of the data is thus defined by:
[Equation 3]

here,
(Equation 4)

[0015]
Note that the dimension of Q is wh × wh, where w and h are the width and height of the component, respectively. Although the size of this matrix is huge, since only a finite number of component vectors M are added, the rank of this matrix does not exceed M-1. v _i ⁽¹⁾ is

(I = 1, 2,... M), λ _i ⁽¹⁾

As the eigenvalue of

Therefore, by multiplying the left side of the above equation by A ⁽¹⁾ ,

Is

Note that the eigenvector of
(Equation 5)

[0016]
Therefore, the eigenvector v _i ⁽¹⁾ and the eigenvalue λ _i ⁽¹⁾ are obtained by the following equations.
(Equation 6)

[0017]
However

Is only M × M. Therefore, u _i ⁽¹⁾

The following equation is obtained as the eigenvector of
(Equation 7)

[0018]
The eigenvalue λ _i ⁽¹⁾ is the variance along the new coordinate space spanned by the eigenvector u _i ⁽¹⁾ . Hereinafter, it is assumed that the eigenvalue λ _i ⁽¹⁾ decreases as the order of _i increases. The eigenvalues decrease exponentially. Therefore,

By calculating (here

And 1 ≦ k ≦ M ₁ ), and the face component Γ ⁽¹⁾ can be projected only to M ₁ << M dimensions. w _k ⁽¹⁾ is the k-th coordinate of Γ ⁽¹⁾ in the new coordinate system. On this premise, W ⁽¹⁾ is called the primary component feature. Vector u _k ⁽¹⁾ is actually an image and is called the primary eigencomponent.
[0019]
(Equation 8)

According to this, the following equation is obtained.
(Equation 9)

[0020]
Since U ⁽¹⁾ is an M ₁ × P matrix, an inverse matrix cannot be obtained. However, the inverse matrix can be approximated using the pseudo inverse matrix.

As follows:

Is a pseudo inverse matrix.
(Equation 10)

According to this, the following equation is obtained.
[Equation 11]

here,

Is a matrix reconstructed from W ⁽¹⁾ and U ⁽¹⁾ .
[0021]
Then, the following equation is executed to obtain the residual component _{ｉ i} ⁽²⁾ .
(Equation 12)

[0022]
Since the remaining face component vectors still contain a wealth of information on individual components, it is necessary to extract face component features from the remaining components again.

, Λ _i ⁽²⁾

_Eigenvalues, ^{v i (2)}

Let the corresponding eigenvectors be

It is. Based on the above discussion,

The eigenvector of is

It is. Therefore,

, The residual component Γ ⁽²⁾ can be projected only to the M ₂ << M dimension. here,
(Equation 13)

And is 1 ≦ k ≦ _{M 2.} Since u _k ⁽²⁾ is the eigenvector of the residual component, u _k ⁽²⁾ is called a secondary eigen component and w _k ⁽²⁾ is called a secondary component feature.
[0023]
[Equation 14]

Then, equation (13) can be written as follows.
[Equation 15]

Also,
(Equation 16)

Then, the following equation is obtained.
[Equation 17]

[0024]
Since U ₂ is not calculated only once a constant conversion matrix does not affect the computational efficiency. The face component can be described by the following equation.
(Equation 18)

here,

It is. The calculation load when calculating Ω (Φ) does not increase as compared with the case where only the component feature is calculated from the eigencomponent U.
[0025]
The residual component is called a secondary residual component, and the original component is called a primary residual component.
[0026]
The same method can be used to derive the third, fourth,. By projecting the residual components of the corresponding order, tertiary, quaternary,..., N-order component features are obtained. Using these higher order component features, the similarity of the components can be defined as a weighted Euclidean distance between the projections. FIG. 2 shows a procedure for calculating the _i- ^th eigencomponent U ⁽ⁱ⁾ and the corresponding transformation matrix U _i . In the figure, Pseudo_Inv (B) is a function for calculating a pseudo inverse matrix of the matrix B.
[0027]
The dissimilarity of the two faces H ₁ and H _2, specific components: between (eg specific day, specific eyebrows, specific nasal, specific ports and specific contour) and generated from the projection of eigenface various facial component feature Defined as composite distance.
[Equation 19]

When a1 = 0, the similarity of the face images is measured using only the secondary features.
[0028]
The method of describing a face image by the above method can be considered as follows.
1) The face component is scanned by a raster scan method from the upper left corner of the component window to the lower right corner of the component window to obtain a one-dimensional pixel array.
2) Subtract an average component from the one-dimensional pixel array.
3) Multiply the subtracted one-dimensional pixel array by the primary and higher order eigencomponents.
4) The features of the obtained components are used as a description of the face.
5) Coding the features as a coded representation.
[0029]
According to the present method, weighting is performed with different degrees of attention according to the importance of identification information of a person's face, and a person's face can be described effectively and space-efficiently.
[0030]
Next, the overall calculation of the facial feature extraction according to the present invention will be described with reference to FIGS. The calculation includes a learning mode calculation (steps # 22 to # 31) shown in FIG. 4 and a test mode calculation (steps # 32 to # 42) shown in FIG. In the learning mode, first, many face samples are learned, accumulated, and an operation for obtaining a coefficient of a primary average is performed.
[0031]
The learning mode starts from step # 22 and continues to step # 31. The learning mode is provided to generate various parameters used in the test mode.
[0032]
In step # 22, a plurality of sample face images are input.
[0033]
In step # 23, each sample face image is divided into a plurality of face parts such as a right eye, a left eye, a right eyebrow, a left eyebrow, a nose, a mouth, and a contour, and each part is analyzed to obtain a basic face component Φ _i. Get. The face component Φ _i is weighted based on the degree of importance of human face discrimination.
[0034]
In step # 24, face components of the same part, such as the nose, are collected from a plurality of sample face images. The face component of the nose is called the nose component. The collected face components of each part are averaged to obtain a primary average face component Φ using equation (1). The same operation is performed to obtain a primary average face component for different face components. The primary average face component Φ is used in the test mode calculation in step # 34.
[0035]
In step # 25, an arbitrary face component of each sample face image, for example, the nose face component is subtracted by the nose primary average face component Φ to obtain a nose vector given by equation (2). The same operation is performed for each of the different face components.
[0036]
Steps # 22 to # 25 are collectively called an analysis step of analyzing the learning face image.
[0037]
In step # 26, equations (4), (5), (6), (7), and (8) are executed to obtain a first-order eigencomponent U ⁽¹⁾ . The primary eigencomponent U ⁽¹⁾ is used in the test mode calculation in step # 35.
[0038]
In step # 27, an inverse matrix U ^{(1) +} is generated using equation (10). The inverse matrix U ^{(1) +} is approximately equal to U ^{(1) T.} The inverse matrix U ^{(1) +} is used in the test mode calculation in step # 37.
[0039]
In step # 28, the inverse matrix U ^{(1) +} is used to obtain the difference between the face components using equation (11). Thus, the difference between the face component and the basic face component of the original face image collected in step # 22 is obtained. The data generated in step # 28 is called a difference face component.
[0040]
In step # 29, the difference between the difference face component and the primary average face component is obtained.
[0041]
Steps # 27 to # 29 are collectively called an analysis step of analyzing the primary eigencomponent.
[0042]
In step # 30, using the difference obtained in step # 29, the second-order KL coefficient U ⁽²⁾ (also referred to as a second-order eigencomponent ⁾ is calculated by using equations (4), (6), and (7). , (14), and (16). The secondary KL coefficient U ⁽²⁾ is used in the test mode in the KL conversion (Step # 40).
[0043]
The test mode starts from step # 32 and continues to step # 42. A test mode is provided to generate primary component features W ⁽¹⁾ and secondary component features W ⁽²⁾ .
[0044]
In step # 32, one face image to be tested is input.
[0045]
In step # 33, the input face image is divided into a plurality of face parts such as a right eye, a left eye, a right eyebrow, a left eyebrow, a nose, a mouth, and a contour, and each part is analyzed to obtain a basic face component Φ. _{Get i} . The face component Φ _i is weighted based on the importance of identifying a human face.
[0046]
In step # 34, the basic face component is subtracted by the primary average face component Ψ to obtain a first difference Γ ⁽¹⁾ (also called primary average residual component). The first difference Γ ⁽¹⁾ is used in step # 35 and step # 39.
[0047]
Steps # 32 to # 34 are collectively called an analysis step of analyzing the test face image.
[0048]
In step # 35, using the first difference Γ ⁽¹⁾ and the first-order eigencomponent U ⁽¹⁾ , the KL conversion represented by the equation (9) is performed to obtain a first-order component feature W ⁽¹⁾ .
[0049]
In step # 36, a primary component feature W ⁽¹⁾ is generated. The primary component feature W ⁽¹⁾ represents the feature of the test face input in step # 32. The primary component feature W ⁽¹⁾ can be used as information representing a test face, but has data of a relatively large size. If further calculations described below are performed, the data size can be reduced.
[0050]
In step # 37, the inverse KL transformation is performed using the equation (11), and the reconstruction matrix

Generate
[0051]
In step # 38, the reconstruction matrix

Is generated.
[0052]
In step # 39, equation (12) is executed, and the first difference Γ ⁽¹⁾ and the reconstruction matrix

And a second difference Γ ⁽²⁾ is obtained. This is usually called the second residual component _{ｉ i} ⁽²⁾ .
[0053]
Steps # 37 to # 39 are collectively called an analysis step of analyzing the primary component characteristics.
[0054]
In step # 40, using the second differential gamma ⁽²⁾ and the secondary K-L coefficient ^{U (2),} and K-L transform according to equation (17) is executed, it generates a secondary component, wherein ^{W (2)} I do.
[0055]
In step # 41, a secondary component feature W ⁽²⁾ is generated. The secondary component feature W ⁽²⁾ is used as information representing the feature of the test face input in the test mode.
[0056]
The flowcharts shown in FIGS. 3 and 4 are executed by a computer to which a camera for capturing the sample face image and the test face image is connected. It is also possible to prepare a set of two computers. One set is used for learning mode calculation, and the other set is used for test mode calculation. Each set can include a computer and a camera. The learning mode calculation set is programmed to execute steps # 22 to # 30, while the test mode calculation set may be programmed to execute steps # 32 to # 42. . In the test mode calculation set, a storage device is provided for learning, such as a primary average face component Ψ, a primary eigencomponent U ⁽¹⁾ , an inverse matrix U ^{(1) +} , and a secondary eigencomponent U ^(2). It is also possible to store the information obtained by the mode calculation set in advance.
[0057]
According to the present invention, a face description of a person can be obtained extremely efficiently by using the feature mainly composed of components. Since the higher-order eigen component only needs to be calculated once using the learning component, the higher-order component feature can be obtained with approximately the same efficiency as the primary component feature. However, since the detailed local identification information is revealed by the higher-order component features, if the primary component features and the higher-order component features of the eyes, eyebrows, nose, mouth, and contour are combined with various attention weightings, It is possible to obtain a face description with better performance than when only the primary unique face features or only the combination of the primary unique face features and the higher-order unique face features is used.
[0058]
The present invention provides a face description that can be used very effectively and efficiently for Internet multimedia, database searching, video editing, electronic libraries, surveillance and tracking, and a wide variety of other applications utilizing face recognition and authentication. .
[Brief description of the drawings]
FIG. 1 is a flowchart showing a calculation procedure of a primary feature W ⁽¹⁾ .
FIG. 2 is a flowchart showing a calculation procedure of an _i- ^th eigencomponent U ⁽ⁱ⁾ and a corresponding transformation matrix U _i .
FIG. 3 is a flowchart of a learning mode calculation.
FIG. 4 is a flowchart of a test mode calculation.

Claims (18)

A method of extracting features of each component of a face to describe the face,
Processing a learning mode operation,
Analyzing a plurality of learning face images;
Calculating a primary eigencomponent U ⁽¹⁾ using the analyzed learning face image;
Calculating a secondary eigencomponent U ⁽²⁾ using the analyzed learning face image;
Processing the test mode operation,
Analyzing the test face image;
Obtaining a secondary component feature W ⁽²⁾ of the test face image using the secondary eigen component U ⁽²⁾ ;
A method that includes A method of extracting features of each component of a face to describe the face,
Processing a learning mode operation,
Analyzing a plurality of learning face images to generate a primary residual component Γ ⁽¹⁾ of the learning face;
Calculating a primary eigencomponent U ⁽¹⁾ using a primary residual component Γ ⁽¹⁾ of the learning face;
Analyzing the primary eigencomponent U ^{(1 )} to generate a secondary residual component Γ ⁽²⁾ of the learning face;
Calculating the secondary eigen component U ⁽²⁾ using the secondary residual component Γ ⁽²⁾ of the learning face;
Processing the test mode operation,
Analyzing the test face image to generate a primary residual component 試 験⁽¹⁾ of the test face;
Obtaining a primary component feature W ⁽¹⁾ of the test face image using the primary eigen component U ⁽¹⁾ and the primary residual component Γ ⁽¹⁾ of the test face;
Analyzing the primary component feature W ^{(1 )} to generate a secondary residual component Γ ⁽²⁾ of the test face;
Obtaining a secondary component feature W ⁽²⁾ of the test face image using the secondary eigen component U ⁽²⁾ and the secondary residual component Γ ⁽²⁾ of the test face;
A method that includes The step of analyzing a plurality of learning face images,
Dividing each sample face image into face portions to obtain a face component Φ _i of the face portions;
Averaging the face components of each face part to obtain a primary average face component Ψ,
Subtracting a primary average face component か ら from the face component to generate a primary residual component 学習⁽¹⁾ of the learning face;
The method for extracting features of a face component according to claim 2, comprising: The step of analyzing the primary eigencomponent,
Reconstruction matrix

Obtaining a
Reconstruction matrix

Subtracting from the primary residual component 学習^{(1) of} the learning face to generate a secondary residual component Γ ⁽²⁾ of the learning face;
3. The method of extracting facial component features according to claim 2, comprising: The step of analyzing the test face image,
Dividing the test face image into face portions to obtain face components Φ _i of the face portions;
Subtracting the primary average face component Ψ from the face component Φ _i to generate a primary residual component Γ ⁽¹⁾ of the test face;
The method for extracting features of a face component according to claim 2, comprising: The step of analyzing the test face image,
Reconstruction matrix

Obtaining a
Reconstruction matrix

Subtracting from the primary residual component 試 験^{(1) of} the test face to generate a secondary residual component Γ ⁽²⁾ of the test face. Method. The method of claim 3, wherein the face component Φ _i of the face portion of the learning face image is weighted. The method for extracting features of a face component according to claim 5, wherein the face component Φ _i of the face portion of the test face image is weighted. A device for extracting a feature of a component to describe a face,
A component operable to process a learning mode operation,
A component operable to analyze a plurality of learning face images;
A component operable to calculate a primary eigencomponent U ⁽¹⁾ using the analyzed learning face image;
A component comprising: a component operable to calculate a secondary eigencomponent U ⁽²⁾ using the analyzed learning face image;
A component operable to process a test mode operation,
A component operable to analyze the test face image;
A component comprising: a component operable to obtain a secondary component feature W ⁽²⁾ of the test face image using the secondary eigencomponent U ⁽²⁾ ;
An apparatus comprising: An apparatus for extracting a feature of each component of the face to describe the face,
A component operable to process a learning mode operation,
A component operable to analyze a plurality of learning face images and generate a primary residual component Γ ⁽¹⁾ of the learning face;
A component operable to calculate a primary eigencomponent U ⁽¹⁾ using a primary residual component Γ ⁽¹⁾ of the learning face;
A component operable to analyze the primary eigencomponent U ^{(1 )} to generate a secondary residual component Γ ⁽²⁾ of the learning face;
A component operable to calculate a secondary eigencomponent U ⁽²⁾ using the secondary residual component Γ ⁽²⁾ of the learning face;
A component operable to process a test mode operation,
A component operable to analyze the test face image and generate a primary residual component Γ ⁽¹⁾ of the test face;
A component operable to obtain a primary component feature W ⁽¹⁾ of the test face image using the primary eigen component U ⁽¹⁾ and the primary residual component Γ ⁽¹⁾ of the test face;
A component operable to analyze the primary component feature W ^{(1 )} to generate a secondary residual component Γ ⁽²⁾ of the test face;
A component operable to obtain a secondary component feature W ⁽²⁾ of the test face image using the secondary eigen component U ⁽²⁾ and the secondary residual component Γ ⁽²⁾ of the test face; A component unit to be provided;
An apparatus comprising: The apparatus operable to analyze a plurality of learning face images,
A component operable to divide each sample face image into face portions to obtain a face component Φ _i of the face portions;
A component operable to average the face components of each face portion to obtain a primary average face component Ψ,
A component operable to subtract the primary average face component か ら from the face component to generate a primary residual component 学習⁽¹⁾ of the learning face;
The apparatus for extracting features of a face component according to claim 10, comprising: The apparatus operable to analyze a primary eigencomponent,
Reconstruction matrix

A component operable to obtain
Reconstruction matrix

A component operable to subtract from the primary residual component 学習^{(1) of} the learning face to generate a secondary residual component Γ ⁽²⁾ of the learning face;
The apparatus for extracting features of a face component according to claim 10, comprising: The apparatus operable to analyze a test face image,
A component operable to divide the test face image into face portions and obtain a face component Φ _i of the face portions;
A component operable to subtract the primary average face component Ψ from the face component Φ _i to generate a primary residual component 試 験⁽¹⁾ of the test face;
The apparatus for extracting features of a face component according to claim 10, comprising: The apparatus operable to analyze a test face image,
Reconstruction matrix

A component operable to obtain
Reconstruction matrix

A component operable to subtract from the primary residual component Γ ^{(1) of} the test face to generate a secondary residual component Γ ⁽²⁾ of the test face;
The apparatus for extracting features of a face component according to claim 10, comprising: The apparatus for extracting features of a face component according to claim 11, wherein the face component Φ _i of the face portion of the learning face image is weighted. 13. The apparatus for extracting features of a face component according to claim 12, wherein the face component Φ _i of the face portion of the test face image is weighted. An apparatus for extracting a feature of each component of the face to describe the face,
A component operable to process a learning mode operation,
A component operable to analyze a plurality of learning face images and generate a primary residual component Γ ⁽¹⁾ of the learning face;
A component operable to calculate a primary eigencomponent U ⁽¹⁾ using a primary residual component Γ ⁽¹⁾ of the learning face;
A component operable to analyze the primary eigencomponent U ^{(1 )} to generate a secondary residual component Γ ⁽²⁾ of the learning face;
An apparatus comprising: a component operable to calculate a secondary eigencomponent U ⁽²⁾ using a secondary residual component Γ ⁽²⁾ of a learning face. An apparatus for extracting a feature of each component of the face to describe the face,
A storage component for storing a primary average face component Ψ, a primary eigencomponent U ⁽¹⁾ , an inverse matrix U ^{(1) +} , and a secondary eigencomponent U ⁽²⁾ ;
A component operable to process a test mode operation,
A component operable to analyze the test face image and generate a primary residual component Γ ⁽¹⁾ of the test face;
A component operable to obtain a primary component feature W ⁽¹⁾ of the test face image using the primary eigen component U ⁽¹⁾ and the primary residual component Γ ⁽¹⁾ of the test face;
A component operable to analyze the primary component feature W ^{(1 )} to generate a secondary residual component Γ ⁽²⁾ of the test face;
A component operable to obtain a secondary component feature W ⁽²⁾ of the test face image using the secondary eigen component U ⁽²⁾ and the secondary residual component Γ ⁽²⁾ of the test face; A component unit to be provided;
An apparatus comprising:

Images