JPH10171988A - Pattern recognizing/collating device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To accurately perform a collation to a person even about any face photograph by previously obtaining a conversion to make a model and a fluctuation amount be orthogonal to each other from many face pictures and face photographs, extracting a characteristic using this conversion and collating a face not attaching to any group of models to a photograph. SOLUTION: A covariance calculating unit 3 fetches picture data from an input pattern input means 1 and a model pattern input means 1, and calculates a covariance matrix as regarding one picture data as a vector. A characteristic extracting unit 4 calculates a conversion matrix from the covariance matrix, and at the same time, performs a vector conversion as regarding one picture data as a vector and calculates a characteristic vector. Then, the selection of a characteristic space in that the space of the model pattern and the space of model-input varied vector are orthogonal is performed, and the collating processing of the face picture and the face photograph by the input pattern input means 1, the model pattern input means 2, the covariance calculating unit 3 and the determination unit 5.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a pattern recognition / collation apparatus used for user identification using a face photograph or the like and information compression for low bit rate communication.
Pattern recognition / identification using human faces for ID recognition, license carrier authentication, man-machine interface, and pattern recognition / matching devices for security. -It relates to a collation device.

[0002]

2. Description of the Related Art A method that has recently attracted attention in the technical field of pattern recognition (for example, face image recognition and voice recognition) uses a secondary statistic (covariance) calculated from a set of models registered in a database. Based on the distribution of the pattern, that is, the portion occupied by the set of patterns in the data space, the feature is extracted from the pattern. For example, the well-known KL (Karhune
The n-Loeve) expansion method uses KL expansion to extract features. Reference M.Turk & A.Pentland: “Face Recogniti
on Using Eigenfaces "Proceedings of IEEE, CVPR91.
, And many other methods are in accordance with this.

In the KL method, a model image M and an input pattern I are represented by M = Σγ _i E _i , I = Σγ _i 'E _i (8) (γ _i is the i component of M, γ _i ′ is the i component) (the sum is taken for i = 1.
Is approximated by a linear combination of the plurality of basis vectors E _i (i = 1... P), and collation is performed between the approximate data.

In the KL method, eigenvectors corresponding to p (for example, about 100) eigenvalues of the covariance matrix obtained from w pieces of teaching pattern data are used as the basis vectors. If the space is constituted by the basis vectors, (1) the projected teaching data is separated best, that is, it is easy to distinguish. (2) Irregular components (fluctuations) such as noise included in the pattern can be removed. It is believed to have the advantage of: A point to be noted in this KL method is that the distribution of the pattern vector estimated based on the statistic obtained from the model pattern set has generality, that is, it is roughly applied to the input pattern. It is assumed that

In fact, it has been experimentally confirmed that, for example, in face image recognition, when the variation of the input pattern from the model is not so large, a very accurate recognition rate can be achieved.

[0006]

However, the above conventional method has a problem that a sufficient recognition rate cannot be provided when the difference between the input pattern and the model is large. This occurs, for example, when the environment such as the lighting conditions at the time of shooting changes greatly between an input image and a model image in image recognition, and is a serious problem that often occurs in reality.

The cause of the above problem in the conventional method is that it is assumed that the general distribution of the pattern can be estimated using only the statistics of the model pattern included in the database.

[0008]

In order to solve this problem, in the present invention, in addition to the statistic obtained from the model set, a statistic capturing a change of the input pattern from the model is learned in advance and used. . Therefore, in the present invention, the model vector covariance input means for inputting the covariance matrix C _m of the model pattern (estimation of the statistical properties of the pattern), and the covariance matrix of the variation from each model pattern to the corresponding input pattern A model-input variation covariance input means for learning and inputting C _p (statistical information indicating the nature of change) in advance;
The weighted average of the model vector covariance matrix and the model-input variation covariance matrix is calculated according to C _s ≡αC _m + (1−α) C _p (α is a real number of 0 <α <1) (1) , A covariance weighted average generating means for newly generating a matrix C _s , and C _s as C _s = (AQ ^1/2 ) (Q ^1/2 A ^T ) (2) (Q ^1/2 is Q Where A ^T is the transposed matrix of A) (A is the normalized eigenvector matrix of C _s ) (Q is the diagonal matrix of the corresponding eigenvalues) and the matrix Q ^-1/2 A ^T The first diagonalizing means to be obtained and the model vector covariance matrix C _m are converted into a matrix D≡Q ^−1/2
The matrix DC _m D ^T obtained by conversion using the ^{_{A T DC m D T = BPB}} T ··· (3) (B is DC _m D ^T normalized eigenvector matrix) (P consists corresponding eigenvalues diagonal matrix a second diagonalization means for obtaining a spectral decomposition matrix B as), these matrices Q ^-1/2 a ^T, using the ^{B H≡WB T Q -1/2 a T ···} ( 4) Generate and hold a matrix H according to (W≡diag (α ₁ , α ₂ ,..., Α _n ), (α _i is an appropriate nonnegative number)). M′≡HM, I′≡HI (5).

In the pattern recognition / collation apparatus of the first aspect, the evaluation value of {M′−I ′} (‖ * ‖ is a Euclidean distance) with respect to the feature vector of the input face is There is a determination means for selecting a model having the smallest feature vector as a recognition result.

Further, the pattern recognition / collation apparatus according to claim 2 further determines the similarity of the feature vector between the model and the input pattern by (M ′ · I ′) / (| M ′ || I ′ |) (7) ((* ・ *) is the inner product of vectors, | * | is the size of the vector) and whether or not the input pattern corresponds to the model depends on whether or not this value is a certain value or more Is determined.

In the above, the weighted averaged covariance C
_s and the model vector covariance C _m are converted to diagonalizing means 1 and 2
By performing diagonalization using, and performing feature extraction using the obtained matrix H, it is possible to control the change of the input pattern from the model pattern so as to be orthogonal to the space occupied by the model set. As a result, even when the difference between the model and the input pattern is large, the model corresponding to the input can be correctly matched by ignoring features in a direction orthogonal to the space occupied by the model in the final process of recognition and matching. Hereinafter, the detailed mechanism will be described.

[0012]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS According to the first aspect of the present invention, there is provided a model pattern input means for inputting a model pattern M (also referred to as a model vector), and an input pattern I to be recognized (also referred to as an input vector). , An input pattern input means for inputting a covariance matrix C _m of model vectors, and a covariance matrix C _p of variation from an individual model pattern to a corresponding input pattern. The input model-input variation covariance input means, the model vector covariance matrix input from the model vector covariance input means, and the model-input variation covariance matrix input from the model-input variation covariance input means; weighted average _{C s ≡αC m + (1-} α) ( the α 0 <α <1 real number) C _p taken according (1), generates a new matrix C _s of And covariance weighted average generating means that, the matrix C _s a ^{_{C s = (AQ 1/2) (}} Q 1/2 A T) ··· (2) (A output of the covariance weighted average generating means C _s normalized eigenvector matrix) (Q is a diagonal matrix consisting of corresponding eigenvalues) (Q ^1/2 the Q of the square root matrix, a ^T is the spectral decomposed as transposed matrix of a), than this matrix D≡ First diagonalization means for obtaining Q ^-1/2 A ^T (Q ^-1/2 is the inverse matrix of the square root matrix of the matrix Q), and a matrix DC _{m obtained} by transforming the model pattern covariance matrix C _m by the matrix D the D ^T spectrally decomposed as ^{_{DC m D T = BPB T ···}} (3) (B normalized eigenvector matrix of DC _m D ^T) (P consists corresponding eigenvalues diagonal matrix), the matrix B using a second diagonalization means of obtaining, output Q ^-1/2 a ^T of the first and second diagonalization means, the B H≡WB ^T Q ^-1/2 a ^T ·· (4) (W≡diag (α 1, α 2, ··· α n), (α i is the number) of suitable non-negative) generates and holds the matrix H in accordance with the model pattern M runtime recognition From the input pattern I and M ′ 抽出 HM, I′≡HI... (5), and a feature of the input pattern I extracted by the feature extracting means. The distance {M′−I ′} (‖ * ‖ is the Euclidean distance) between the vector I ′ and the feature vector M ′ of the model pattern M (6) finds the model pattern having the smallest feature vector. A determination unit for determining (recognizing) which model the input pattern corresponds to, by orthogonalizing the model space and the variation space,
This has the effect that it can be recognized by comparing with the model pattern except for the variation of the input pattern.

According to the second aspect of the present invention, the similarity between the feature vector of the model pattern and the feature vector of the input pattern is determined by (M ′ · I ′) / (| M ′ || I ′ |) (7) ((* · *) Is the inner product of the vectors, | * | is the size of the vector), and it is determined whether the input pattern and the model are the same depending on whether or not this value is a certain value or more. It is provided with a judging means, and has an effect that by making the model space and the variation space orthogonal, it is possible to collate with the model pattern except for the variation of the input pattern.

(First Embodiment) A first embodiment of the present invention relates to a face image recognition apparatus for selecting a face image that matches an input face image input from a video camera or the like from a model image photograph database. is there. Hereinafter, a case where the pattern recognition / collation apparatus of the present invention is applied to face image recognition will be described with reference to FIG.

The input pattern input means 1 comprises a video camera for photographing an input face, a digitizer for converting an analog video signal of the video camera into a digital signal, and an image memory for storing the digital video signal. The model pattern input means 2 includes an image scanner for scanning and inputting a model image photograph, and a database for storing the model image photograph input from the image scanner as a digital image file. The covariance calculation unit 3 is a calculation device that inputs image data from the input pattern input unit 1 and the model pattern input unit 2 and calculates a covariance matrix by regarding one image data as a vector. The feature extraction unit 4 is a calculation device that calculates a transformation matrix from a covariance matrix, and also converts one image data as a vector and performs a vector transformation using the transformation matrix to calculate a feature vector. The determination unit 5 is a calculation device that calculates a distance and an angle between feature vectors. These computing devices may be configured using a general-purpose processor, or may be configured using a dedicated processor such as a DSP.

The input pattern input means 1, the model pattern input means 2, the covariance calculation unit 3 and the feature extraction unit 4 perform an off-line process such as orthogonalizing the space of the model pattern and the space of the model-input variation vector. A feature space is selected, and an input pattern input unit 1, a model pattern input unit 2, a covariance calculation unit 3, and a determination unit 5 perform online recognition processing.

First, the offline processing will be described. Generally, the dimension of the space occupied by the pattern in the data space (for example, the dimension of the portion occupied by the face image in the entire image space) is the original dimension of the space (for example, 100,000 if the number of pixels is 100,000) Is often much smaller (eg, 100 dimensions). Similarly, the model-input variation vector also occupies a low dimensional space in the data space. First, the covariance matrix C _m indicating a statistical trend of the model patterns, and inputs from the model vector covariance input means. The covariance matrix C _{m ^{is, C m ≡ΣMM T ··· (9}} ) using a model pattern input from the model pattern input means (M ^T is the transpose matrix of the matrix M, sum all specimens models {M Can be calculated directly according to｝, but there is no problem if it is obtained by another method. What is necessary is just to show the distribution (covariance) of the model pattern. In this embodiment, the model pattern covariance C _m is calculated using a model human face image set which has been entered into the database of the image scanner equipped {M} is a model pattern input unit 2. Here, the average vector of the model vectors is set to 0 for simplicity. Otherwise, set M to (M-
M _a ) (M _a is the average of the set {M}).

Further, a covariance matrix C _p indicating the statistical property of the difference (M−I) between the input pattern and the corresponding model is input from the model-input fluctuation covariance input means to learn the input fluctuation tendency. . The covariance C _p is calculated using the difference between a sample group of the input pattern input from the input pattern input unit and the corresponding model, C _p ≡Σ (MI) (MI) ^T (10) ( (M−I) ^T is the transpose of the matrix (M−I) (the sum is taken over all obtained (M, I) pairs), but can be calculated directly but obtained by other methods It may be. What is necessary is just to show the distribution (covariance) of the difference vector. In this embodiment, the model-input variation vector covariance C _p is calculated according to Equation 10 from the actual input of the input face vector {I} (input pattern) and the difference from the corresponding model face pattern.

The calculation of the two covariances C _m and C _p is performed by a common covariance calculation unit 3. These pieces of covariance information are sent to the feature extraction unit 4.

The feature extraction unit 4 first calculates a weighted average C _s of the two covariances according to Equation 1, and generates and holds a feature extraction matrix H through simultaneous diagonalization of C _s and C _m .
In the case of the image recognition device, a feature vector {M ′} is extracted from the model image vector {M}, and the feature extraction unit holds the feature vector.

The covariance weighted average generation means included in the feature extraction unit 4, two obtained covariance, i.e., the model vector covariance C _m and models - a weighted average of the input variation covariance C _p C _s ≡ αC _m + (1−α) C _p (1) (α is a real number of 0 <α <1) to generate a matrix C _s . α needs to be determined according to the characteristics of the video camera or the image scanner, and it is necessary to determine the optimum value in accordance with the image quality of the model image. For example, the initial value of α may be set to 0.5, and the value may be changed little by little so as to improve the recognition rate.

Next, a feature space suitable for pattern recognition is selected using these covariances, that is, a specific mechanism of feature extraction is determined. Here, feature extraction from a pattern refers to projecting an original pattern (N-dimensional) into a lower-dimensional space (for example, K-dimensional, K <N). Therefore, choosing a feature space is such a K
This is to select K orthogonal coordinate axes (vectors) constituting a dimensional space, and thus feature extraction is to apply a linear transformation (matrix) composed of such vectors. Therefore, the conversion simultaneously diagonalizing the matrix C _s and model vector covariance C _m, orthogonalizing the spatial area as the variation vector of the model patterns. The principle is as follows.

The matrix C _s is obtained by the first diagonalization means as C _s = (AQ ^1/2 ) (Q ^1/2 A ^T ) (2) (A is a normalized eigenvector matrix of C _s ) ( Q is a diagonal matrix composed of the corresponding eigenvalues (Q ^1/2 is a square root matrix of Q, A ^T is a transposed matrix of A), and a matrix D≡Q ^{-1/2 AT} is output. You.

On the other hand, the covariance C _m conversion D DC _m D
After being mapped to ^T , DC _m D ^T = BP B ^T (3) (B is a normalized eigen vector matrix of DC _m D ^T ) (P is a pair of corresponding eigenvalues) (Angular matrix), and a matrix B is output.

The feature extracting means calculates a matrix Q of these outputs.
^{Based on -1/2} A ^T and B, H≡WB ^T Q ^-1/2 A ^T (4) (W≡diag (α ₁ , α ₂ ,... Α _N ), α _i is Generate a matrix H according to an appropriate non-negative number) and hold it. This matrix H is a matrix for performing feature extraction. α _i is a coefficient for weighting the feature, and is determined by a method of obtaining an optimum value while trying to improve the recognition rate.

Here, the matrix L is defined as L≡B ^T Q ^-1/2 A ^T (11)

The matrix H is obtained by performing a constant multiple conversion on each component after the application of the matrix L. Now, when the matrix L is applied to the model vector M and the input vector I according to M′≡LM, I′≡LI (12), that is, feature extraction is performed,
, The matrix C _s and the model vector covariance C _m are respectively LC _s → C _s ′ = αC _m ′ + (1−α) C _p ′ = α {(LM) (LM) ^T + (1−α)} (L (M−I)) (L (M−I)) ^T = αΣ (LMM ^T L ^T ) + (1−α) Σ (L (M−I) (M−I) ^T L ^T ) = αLC ^{_{m L T + (1-α}} ) LC p L T = L (αC m + (1-α) C p) L T = LC s L T = B T Q -1/2 A T AQA T AQ -1 / ² B = E (E is a unit _{matrix) ··· (13) L C m} → C m '= Σ (LM) (LM) T = Σ (B T DM) (M T D T B) = B T DC ^{_{m D T B = B T BPB}} T B = P ··· unit matrix E as (14), is converted to a diagonal matrix P.

[0028] At the same time, the model from equation 1 - Input variation covariance C _p be ^{_{E = LC s L T = αLC}} m L T + (1-α) LC p L T = αP + (1-α) C p 'C p '= (E−αP) / (1−α) (15) (P is a diagonal matrix, α is a real number of 0 <α <1, and E is a unit matrix).

It is apparent from the step of equation 15 that the model vector covariance C _m ′ and the model-input variation covariance C _p ′ have a common eigenvector by the transformation according to equation 12. Further, from Equation 15, the former eigenvalues are arranged in descending order by x ₁ >
x _2> x _3>. . . > X _N (all non-negative), the eigenvalues of the corresponding axes are y ₁ = (1−αx ₁ ) / (1−α), y ₂ = (1−αx ₂ ) / (1−α), ... y _N = (1−αx _N ) / (1−α), so that y _N > y _N−1 >. . . > Y ₁
The order of the eigenvalues is completely reversed.

Since the eigenvalue of the covariance matrix indicates the variance in the direction of the corresponding eigenvector, that is, the spread (square) of the distribution, the space occupied by the model pattern and the model-input variation vector is expressed by the following equation (12). Will share all axes of the distribution and reverse the order of magnitude of spread in the axial direction. That is, it can be said that the space of the model pattern and the space of the model input variation vector are orthogonal.
The matrix H emphasizes this orthogonalization by further expanding the difference in distribution spread in each axis direction after the transformation of the matrix L.

The above is the off-line process, whereby the average fluctuation tendency of the input pattern from the model is grasped, and the specific feature extraction mechanism suitable for recognition is determined. In the case of pattern recognition (matching with a registered model), the input of the model pattern is also input in advance offline through the model pattern input means 2 including a database function.

When executing pattern recognition, the feature extraction unit calculates a feature vector I 'according to I'≡HI for a new input face image I which is taken in from the video camera and subjected to predetermined processing. The determination unit is {M′−I ′} (‖ * ‖ is the Euclidean distance) (6) A model face having a feature vector that minimizes M ′｝, and output as a recognition result.

(Second Embodiment) A second embodiment of the present invention relates to an input face image input from a video camera or the like,
This is a face image collation device that collates input image photographs input from an image scanner or the like and determines whether they match. Hereinafter, a case where the pattern recognition / collation apparatus of the present invention is applied to face image collation will be described with reference to FIG.

The model pattern covariance C _m is calculated using a model human face image set which has been entered into the database of the image scanner equipped {M} is a model pattern input unit 2. Calculation of C _m is according to ^{_{C m ≡ΣMM T ··· (9)}} (M T is the transpose matrix of the matrix M, the sum is taken for all of the specimens models {M}.).

The input pattern input means 1 comprises a video camera, a digitizer and an image memory. Model-
The input variation vector covariance C _p is the input face vector {I}
(Input pattern) is actually input, and is calculated by Expression 10 from the difference from the corresponding model face pattern. Or calculation of the two covariance C _m and C _p is carried out by a common covariance calculation unit 3. These pieces of covariance information are sent to the feature extraction unit 4. In the feature extraction unit 4, first,
A weighted average C _s of the two covariances is calculated according to Equation 1, and a feature extraction matrix H is generated and held through simultaneous diagonalization of C _s and C _m as described in detail above. The above is the offline processing.

At the time of matching, the feature extraction unit 4 calculates a feature vector I 'according to I'≡HI for a new input face image I which has been taken in from the video camera and subjected to predetermined processing. The input face image is
The lighting and the like are sufficiently controlled so that the best image can be captured.

In the face image matching device, the model face vector M
At the time of execution, the feature extraction unit 4 extracts the feature vector M ′ according to M ′ 実 行 HM at the time of execution. As for the model image, a photograph is input by a scanner, and therefore, a variation due to a difference in photographing conditions of the photograph is directly input.

In the determination unit 5, (M ′ · I ′) / (| M ′ || I ′ |) (7) ((* · *) is the inner product of the vectors, | * | Is the size of the vector), and outputs whether the collation is correct based on whether the value is equal to or greater than a predetermined value.

The best image taken on the spot and the image of the photograph containing the variation are input, and they are mapped to a space where the model and the variation are separated, and correspond to the cosine of the angle between the two vectors. The amount is determined to determine whether the person and the photograph match. By appropriately adjusting the weight of W when determining the transformation H, it is possible to select and collate features that are easily distinguishable.

[0040]

As described above, according to the present invention, a transform that makes the space occupied by the model pattern and the space occupied by the model-input variation vector orthogonal to each other is obtained and applied to face image recognition and collation. Since the model-input variation vector is the deviation of the input pattern from the corresponding model pattern,
If the input and the model are collated in the space where the model pattern exists after the conversion, this shift can be removed. Also, from a large number of face images and face photographs, a transform that orthogonalizes the model and the variation is obtained in advance, features are extracted using this transform, and faces that do not belong to the model set are compared with the photos, Such a face photograph can be compared with a person with high accuracy.

Therefore, face image recognition and collation can be realized with much higher accuracy than the conventional method, and the effect is very large.

[Brief description of the drawings]

FIG. 1 is a diagram showing a configuration of a face image recognition / collation device according to the present invention.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 Input pattern input means 2 Model pattern input means 3 Covariance calculation unit 4 Feature extraction unit 5 Judgment unit

Claims (2)

[Claims] 1. A model pattern input means for inputting a model pattern M (also referred to as a model vector), an input pattern input means for inputting an input pattern I to be recognized (also referred to as an input vector), and a covariance of the model vector Model-vector covariance input means for inputting a matrix C _m , model-input fluctuation covariance input means for previously learning and inputting a covariance matrix C _p of variation from an individual model pattern to a corresponding input pattern, and the model the vector covariance input means input from model vector covariance matrix model - from the input variation covariance input means model - a weighted average of the input variation covariance matrix _{C s ≡αC m + (1-} α ) C _p (α is a real number of 0 <α <1) (1), and a covariance weighted average generating means for newly generating a matrix C _s ; The matrix C _s of the output of the weighted average generation means is represented by C _s = (AQ ^1/2 ) (Q ^1/2 A ^T ) (2) (A is a normalized eigenvector matrix of C _s ) A diagonal matrix composed of eigenvalues (Q ^1/2 is a square root matrix of Q, A ^T is a transposed matrix of A), and a matrix D≡Q ^-1/2 A ^T (Q ^{-1 / 2} is the inverse matrix of the square root matrix of the matrix Q), and a matrix DC _m D ^T obtained by transforming the model pattern covariance matrix C _m by the matrix D is DC _m D ^T = BPB ^T. (3) second diagonalization means for performing spectral decomposition to obtain a matrix B as (B is a normalized eigenvector matrix of DC _m D ^T ) (P is a diagonal matrix composed of corresponding eigenvalues), Using the outputs Q ^-1/2 A ^T and B of the first and second diagonalizing means, H≡WB ^T Q ^-1/2 A ^T (4) (W≡diag (α ₁ , α _2, ... _n), (alpha _i is generated and holds the matrix H according to the number)) suitable nonnegative, M'≡HM from the model pattern M and the input pattern I runtime recognition, I'≡HI ··· (5) A feature extracting means for extracting respective feature vectors M ′ and I ′ according to the following formula: A distance 距離 M′−I between the feature vector I ′ of the input pattern I and the feature vector M ′ of the model pattern M extracted by the feature extracting means '‖ (‖ * ‖ Is the Euclidean distance) (6) is provided with a judging means for finding a model pattern having the smallest feature vector and judging (recognizing) which model the input pattern corresponds to. And a pattern recognition / collation device. 2. The similarity between a feature vector of a model pattern and a feature vector of an input pattern is expressed by (M ′ · I ′) / (| M ′ || I ′ |) (7) ((* · *) (Inner product, | * | is the magnitude of a vector), and determining whether or not the input pattern and the model are the same based on whether or not this value is a certain value or more. The pattern recognition / collation device according to claim 1.