JP2009140513A - Pattern characteristic extraction method and device for the same - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a feature vector transformation technique for suppressing a reduction in feature amount effective for discrimination and performing more efficient feature extraction when a feature vector effective for discrimination is to be extracted from an input pattern feature vector and feature dimensions are to be compressed. <P>SOLUTION: An input pattern feature amount is decomposed into element vectors. For each of the feature vectors, a discriminant matrix obtained by discriminant analysis is prepared in advance. Each of the feature vectors is projected into a discriminant space defined by the discriminant matrix and the dimensions are compressed. According to the feature vector obtained, projection is performed again by the discriminant matrix to calculate the feature vector. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

The present invention relates to an image feature extraction method, an image feature extraction apparatus, and a program therefor in the field of pattern recognition, and extracts a feature vector effective for recognition from an input feature vector and converts a feature vector for compressing a feature dimension. Regarding technology.

Conventionally, in the field of pattern recognition, feature vectors are extracted from input patterns,
For example, a feature vector effective for identification is extracted from the feature vector, and the similarity of patterns such as characters and human faces is determined by comparing the feature vectors obtained from the patterns. Yes.

For example, in the case of face recognition, the pixel value of the face image normalized by the eye position or the like is raster scanned to convert it into a one-dimensional feature vector, and this feature vector is used as an input feature vector to perform principal component analysis. (Non-Patent Document 1: Moghdamdam et al., “Probabil.
istic Visual Learning for Object Detecti
on ”, IEEE Transactions onPattern Analysis
s and Machine Intelligence, Vol. 17, no.
7, pp. 696-710, 1997) and linear discriminant analysis for principal components of feature vectors (Non-Patent Document 2: W. Zhao et al., “Discriminant Analysis”).
is of Principal Components for Face Reco
gitionion, "Proceedings of the IEEE Third
International Conference on Automatic Fa
ce and Gesture Recognition, pp. 336-341
1998), the dimension is reduced, and the obtained feature vector is used to identify an individual using a face.

In these methods, a covariance matrix, an intra-class covariance matrix, and an interclass covariance matrix are calculated for learning samples prepared in advance, and basis vectors obtained as solutions of eigenvalue problems in those covariance matrices are calculated. Using these basis vectors, the features of the input feature vector are converted.

Here, the linear discriminant analysis will be described in more detail.
In the linear discriminant analysis, when there is an N-dimensional feature vector x, the intra-class covariance matrix SW of the M-dimensional vector y (= WTx) obtained when this feature vector is transformed by a certain transformation matrix W This is a method for obtaining a transformation matrix W that maximizes the ratio of the matrix SB. As an evaluation function of such a dispersion ratio, (Equation 1) of the evaluation formula is defined using a determinant.

Here, the intra-class covariance matrix ΣW and the inter-class covariance matrix ΣB are the C classes ωi (i = 1, 2,..., C; the number of data ni in the set of feature vectors x in the learning sample. ) Are covariance matrices between a class and a covariance matrix Σi, and are represented by (Equation 2) and (Equation 3), respectively.

Here, mi is an average vector of class ωi (Equation 4), and m is an average vector of x in the entire pattern (Equation 5).

If the prior probability P (ωi) of each class ωi reflects the number of samples ni in advance, P
(Ω i) = ni / n may be assumed. Otherwise, if we can assume equiprobability, P (
ωi) = 1 / C.

The transformation matrix W that maximizes (Equation 1) is an eigenvalue problem of the column vector wi.
Determined as a set of generalized eigenvectors corresponding to the large eigenvalues. The transformation matrix W obtained in this way is called a discriminant matrix.

As for the conventional linear discriminant analysis method, for example, Non-Patent Document 5: “Pattern identification” (
Richard O. Duda et al., Directed by Morio Onoe, New Technology Communications, 20
01, pp. 113-122).

When the number of dimensions of the input feature vector x is particularly large, or when less learning data is used, Σ
W is not regular, and the eigenvalue problem of (Equation 6) cannot be solved by a normal method.

Further, as described in Patent Document 1: Japanese Patent Laid-Open No. 7-296169, it is known that a high-order component having a small eigenvalue of a covariance matrix has a large parameter estimation error.
This adversely affects recognition accuracy.

For this reason, the aforementioned W.S. Zhao et al. Performed principal component analysis of input feature vectors,
Discriminant analysis is applied to principal components with large eigenvalues. That is, as shown in FIG.
After extracting the principal component by projecting the input feature vector using the basis matrix obtained by principal component analysis, the principal component is projected using the discriminant matrix obtained by discriminant analysis as the basis matrix, which is effective for discrimination Extract feature vectors.

Further, in the feature transformation matrix calculation method described in Patent Document 1: Japanese Patent Laid-Open No. 7-296169, the number of dimensions is reduced by deleting higher-order eigenvalues and corresponding eigenvectors of the total covariance matrix ΣT. Discriminant analysis is applied in the reduced feature space. In this sense, in the sense that deleting higher-order eigenvalues and corresponding eigenvectors of the total covariance matrix performs discriminant analysis in a space of only principal components having large eigenvalues by principal component analysis.
Similar to Zhao's method, higher-order features are removed, and stable parameter estimation is achieved.

Japanese Patent Laid-Open No. 7-296169

Moghdamdam et al., “Probabilistic Visual Learning for Object Detection”, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 17, no. 7, pp. 696-710, 1997)

W. Zhao et al., “Discriminant Analysis of Principal Components for Face Recognition,” Proceedings of the IEEE International Conferencing Conference. 336-341, 1998)

Kernel-based Optimized Features Vectors Selection and Discriminant Analysis for Face Recognition, “Proceeding of IAPR International Conf.

Generalized Discrimination Analysis Usage a Kernel Approach, “Neural Computation, Vol. 12, pp 2385-2404, 2000

"Pattern identification" (Richard O. Duda et al., Translated by Morio Onoe, New Technology Communications, 2001, pp. 113-122)

However, the principal component analysis using the total covariance matrix ΣT only selects an axis that is sequentially orthogonal to the axis direction in which the variance in the feature space is large, and the feature axis is independent of the pattern identification performance. Is selected. For this reason, a feature axis effective for pattern identification is lost.

For example, the feature vector x is composed of three elements (x = (x1, x2, x3) T
), The variance of x1 and x2 is large, but it is a feature unrelated to pattern identification, and x3 is effective for pattern identification, but when the variance is small (interclass variance / intraclass variance, that is, the Fisher ratio is large, (Each dispersion value itself is sufficiently smaller than x1 and x2)
If the principal component analysis is performed on 2 and only two dimensions are selected, the feature space related to x1 and x2 is selected, and the contribution of x3 effective for identification is ignored.

Explaining this phenomenon with reference to the figure, (a) in FIG. 3 is a distribution of data viewed from a direction substantially perpendicular to the plane stretched by x1 and x2, and black circles and white circles represent data points of different classes. And When viewed in the space spanned by x1 and x2 (in this figure, a plane), the black circle and the white circle cannot be distinguished, but when viewed from the feature axis of x3 perpendicular to this plane as shown in FIG. Can be separated. However, if an axis with a large variance is selected, the plane spanned by x1 and x2 is selected as the feature space, which is equivalent to trying to make a distinction with reference to FIG. Become.

This is a phenomenon that cannot be avoided by the conventional technique that deletes a space having a small eigenvalue of the principal component analysis or the (all) covariance matrix.

In view of the problems of the prior art as described above, the present invention is based on input pattern feature vectors.
To provide feature vector conversion technology for extracting feature vectors effective for discrimination and suppressing reduction of feature amounts effective for discrimination and performing more efficient feature extraction when compressing feature dimensions is there.

According to the present invention, in a pattern feature extraction method for compressing a feature dimension using linear transformation, a pattern feature is expressed by a plurality of feature vectors xi, and each feature vector xi is subjected to linear discriminant analysis. A discriminant matrix WT is obtained by linear discriminant analysis for a feature vector y obtained by previously obtaining a discriminant matrix Wi of each feature vector to be obtained and further combining each vector yi obtained by linearly transforming the vector xi using the discriminant matrix.
Is obtained in advance, and the feature dimension is compressed by converting the feature vector of the pattern by the linear transformation specified by the discriminant matrix Wi and the discriminant matrix WT.

In the pattern feature extraction method, the pattern feature is divided into a plurality of feature vectors xi, and the linear transformation yi = Wi is used for each feature vector xi by using the discriminant matrix Wi.
A feature vector yi is calculated by performing Txi, and a linear transformation z = WTTTy is calculated using a discriminant matrix WT for the vector y obtained by combining the calculated feature vectors yi, and the feature vector z
By calculating the number of dimensions of the pattern feature.

In the pattern feature extraction method, the matrix W specified by the respective discriminant matrices Wi and W T is calculated in advance, and the input feature vector x
The number of dimensions of the pattern feature may be compressed by calculating a linear transformation z = WTx of the feature vector x combined with i and the matrix W, and calculating the feature vector z.

When the present invention is applied to an image, an image feature is extracted by extracting a feature amount from the image and compressing a feature dimension of the obtained feature using a linear transformation. In the extraction method, a plurality of predetermined sample point sets Si in the image are extracted as feature vectors xi composed of pixel values obtained from the plurality of sample points, and each feature vector xi is obtained by linear discriminant analysis. A discriminant matrix Wi for each feature vector obtained in advance, and a vector xi obtained by linearly transforming the vector xi using these discriminant matrices to obtain a discriminant matrix WT in advance by linear discriminant analysis. An image sample is obtained by linear transformation specified by the discriminant matrix Wi and the discriminant matrix WT. By converting the feature vector for each case, and extracts a feature from the image.

As one method, for a plurality of feature vectors xi composed of a plurality of sample points,
A linear transformation yi = WiTxi is performed using the discriminant matrix Wi to calculate a feature vector yi, and a linear transformation z = WTTy is calculated using the discriminant matrix WT for a vector y obtained by combining the calculated feature vectors yi. By calculating the feature vector z, a feature amount may be extracted from the image.

In the above-described image feature extraction method, the matrix W specified by the respective discrimination matrices Wi and WT is calculated in advance, and the matrix W is used to calculate the linearity of the vector x and the matrix W that are the combined feature vectors xi. The feature quantity may be extracted from the image by calculating the transformation z = WTx and calculating the feature vector z.

In addition, the image is divided into a plurality of predetermined local regions, feature amounts are extracted for each of the plurality of local regions, the feature amounts are expressed as feature vectors xi, and linear for each feature vector xi. A feature matrix y obtained by previously obtaining a discriminant matrix Wi of each feature vector obtained by discriminant analysis and further combining each vector yi obtained by linearly transforming the vector xi using the discriminant matrix is discriminated by linear discriminant analysis. A matrix WT is obtained in advance, and a feature quantity may be extracted from an image by converting a feature vector of a local region by linear transformation specified by the discriminant matrix Wi and the discriminant matrix WT.

In the image feature extraction method described above, the feature vector xi of the local region of the image is subjected to linear transformation yi = WiTxi using the discriminant matrix Wi to calculate the feature vector yi, and the vector y obtained by combining the calculated feature vectors yi For, a linear transformation z = WTTY is calculated using the discriminant matrix WT, and a feature quantity is extracted from the image by calculating a feature vector z.

Alternatively, in the image feature extraction method described above, the matrix W specified by the respective discrimination matrices Wi and WT is calculated in advance, and the matrix W is used to calculate the linearity of the vector x and the matrix W combined with the feature vector xi. The feature quantity may be extracted from the image by calculating the transformation z = WTx and calculating the feature vector z.

As an effective implementation method of an image feature extraction method characterized by extracting a feature amount from an image of the present invention, the feature amount is extracted from the image, and the obtained feature is compressed using a linear transformation. In the image feature extraction method, the image feature is extracted by a two-dimensional Fourier transform of the image, the real and imaginary components of the two-dimensional Fourier transform are extracted as a feature vector x1, and the power spectrum of the two-dimensional Fourier transform And the power spectrum is extracted as a feature vector x2, and for each feature vector xi (i = 1, 2), a discriminant matrix Wi of each feature vector obtained by linear discriminant analysis is obtained, and further, For a feature vector y that combines each vector yi obtained by linearly transforming the vector xi using a discriminant matrix, Discriminant analysis obtained in advance discriminant matrix WT by, and converting the feature vector by linear transformation specified by the discriminant matrix Wi and the discriminant matrix WT.

In addition, in the image feature feature extraction method, the feature is extracted from the image by extracting the feature amount from the image and compressing the feature dimension by using the linear transformation to obtain the obtained feature. Then, the real and imaginary components of the two-dimensional Fourier transform are extracted as the feature vector x1, the power spectrum of the two-dimensional Fourier transform is calculated, the power spectrum is extracted as the feature vector x2, and each feature vector xi (i = 1, 2) Each vector y obtained by obtaining a discriminant matrix Wi of each feature vector obtained by linear discriminant analysis and linearly transforming vector xi using those discriminant matrices.
For a feature vector y combined with i, a discriminant matrix WT is obtained in advance by linear discriminant analysis, and the real component of the Fourier component is obtained by linear transformation specified by the discriminant matrix Wi and the discriminant matrix WT for the principal component of the feature vector xi. A feature quantity is extracted from the image by converting the feature vector x1 for the imaginary component and the feature vector x2 for the power spectrum of the Fourier component so as to reduce the dimensions.

In the image feature extraction method described above, a basis matrix Φ1 (= (W1TΨ1T) represented by a transformation matrix Ψ1 that transforms a feature vector x1 of a real component and an imaginary component by Fourier transform into a principal component and a discriminant matrix W1 for the principal component T), the discriminating feature of the principal component of the feature vector x1 is calculated by linear transformation y1 = Φ1Tx1, the magnitude of the obtained feature vector y1 is normalized to a predetermined magnitude, and the power by Fourier transformation Using the transformation matrix Ψ2 for converting the feature vector x2 based on the spectrum into the principal component and the basis matrix Φ2 (= (W2TΨ2T) T) represented by the discrimination matrix W2 for the principal component, the feature vector x
The feature feature y of the principal component of 2 is calculated by linear transformation y2 = Φ2Tx2, the magnitude of the obtained feature vector y2 is normalized to a predetermined magnitude, and the feature vector y that combines the two feature vectors y1 and y2 , Using the discriminant matrix WT, a linear transformation z = WTTTy is calculated, and a feature vector z is calculated to extract a feature quantity from the image.

By extracting pattern features according to the present invention, feature vectors effective for discrimination by discriminant analysis are extracted for each element vector from the input pattern feature vectors, and the obtained feature vectors are extracted again using a discriminant matrix by discriminant analysis. When compressing the feature dimension,
It is possible to perform feature vector conversion for suppressing feature reduction effective for discrimination and performing more efficient feature extraction.

This is especially effective when the number of learning samples required for performing discriminant analysis is limited despite the large amount of pattern features, and is effective for identification without necessarily using principal component analysis. It is possible to reduce the number of feature dimensions while suppressing loss of features.

It is a block diagram which shows the structure of the pattern feature extraction apparatus by embodiment of this invention. It is a figure for demonstrating a prior art. It is a figure for demonstrating distribution of a pattern feature. It is a block diagram which shows the structure of the pattern feature extraction apparatus by 2nd embodiment by this invention. It is a figure for demonstrating embodiment by this invention. It is a figure for demonstrating embodiment by this invention. It is a block diagram which shows the structure of the face image matching system by 3rd embodiment by this invention. It is a figure for demonstrating embodiment by this invention. It is a figure for demonstrating embodiment by this invention. It is a figure for demonstrating embodiment by this invention. It is a figure for demonstrating embodiment by this invention. It is a figure for demonstrating embodiment by this invention. It is a figure for demonstrating embodiment by this invention. It is a figure for demonstrating embodiment by this invention. It is a figure for showing an example of face description in a 5th embodiment of the present invention. It is a figure which shows an example of the rule at the time of using the binary expression grammar (Binary Representation Syntax) in the 5th embodiment of this invention. It is explanatory drawing for extracting the Fourier feature (FourierFeature) in the 5th Embodiment of this invention. It is a figure for showing an example of the scanning method of the Fourier spectrum in the 5th Embodiment of this invention. It is a table for showing an example of the scanning rule of the Fourier spectrum in the fifth exemplary embodiment of the present invention. It is a table which shows an example of the scanning area | region in the Fourier space for the CentralFourierFeature element in the 5th Embodiment of this invention. It is a figure which shows an example of the block diagram in the 5th Embodiment of this invention.

(First embodiment)
Embodiments of the present invention will be described in detail with reference to the drawings. FIG. 1 is a block diagram showing a pattern feature extraction apparatus using the pattern feature extraction apparatus of the present invention.

Hereinafter, the pattern feature extraction apparatus will be described in detail.
As shown in FIG. 1, the pattern feature extraction apparatus according to the present invention includes a first linear conversion unit 11 that linearly converts an input feature vector x1, a second linear conversion unit 12 that linearly converts an input feature vector x2, A third linear conversion unit 13 that performs linear conversion using the feature vector converted by the linear conversion unit 11 and the linear conversion unit 12 as an input is provided. Each of the above-described linear conversion means performs base conversion by discriminant analysis using the discriminant matrix previously obtained by learning stored in the corresponding discriminant matrix storage means 14, 15, 16.

The input feature vectors x1 and x2 are feature amounts extracted according to their purposes in character recognition and face recognition, for example, directional features calculated from image gradient characteristics and image pixel values themselves. There are multiple elements such as shading characteristics. At this time, for example, N1 direction features are input as one feature vector x1, and the other N2 grayscale values are input as a feature vector x2.

The discriminant matrix storage means 14 and the discriminant matrix storage means 15 perform linear discriminant analysis on the feature vector x1 and the feature vector x2, and store the discriminant matrices W1 and W2 obtained thereby, respectively.

As described above, the discriminant matrix includes the intra-class covariance matrix ΣW ((Expression 2)) and the interclass covariance matrix ΣB according to the class of the feature vectors in the learning sample prepared in advance.
((Equation 3)) may be calculated. The prior probability P (ωi) of each class ωi may be set to P (ωi) = ni / n, reflecting the number of samples ni.

A discriminant matrix can be obtained in advance by selecting an eigenvector wi corresponding to a large eigenvalue represented by (Equation 6) for these covariance matrices.

For each feature vector x1, x2, M is smaller than the input feature dimensions N1 and N2.
If one-dimensional and M2-dimensional bases are selected, then M1,
M2-dimensional feature vectors y1 and y2 can be obtained.

Here, the matrix sizes of W1 and W2 are M1 × N1 and M2 × N2, respectively.
If the feature space dimensions M1 and M2 of the projected feature space are significantly reduced, the feature dimensions can be efficiently reduced, and this is effective in reducing the amount of data and speeding up. However, if the feature dimensions are significantly reduced Causes degradation of discrimination performance. This is because a feature quantity effective for discrimination is lost by reducing the number of feature dimensions.

For this reason, the dimension number M1 or M2 of the feature vector is an amount that is easily affected by the balance with the number of learning samples, and is preferably determined based on experiments.

In the third linear conversion means 13, y1 calculated by the first and second linear conversion means
, Y2 as an input feature vector y, and projection onto the discrimination space is performed. The discriminant matrix W3 registered in the discriminant matrix storage means 16 is obtained from the learning sample in the same manner as when the first and second discriminant matrices are calculated. However, the input feature vector y is a vector in which elements are arranged as represented by the following (Equation 8).

As in (Expression 7), the basis matrix W3 (the size of the matrix is L × (M1 + M2)), and L
The dimensional feature vector y is projected by (Equation 9) to obtain a feature vector z as an output.

In this way, by dividing each feature vector and performing linear discriminant analysis on the learning samples of feature vectors with a small number of dimensions, it is possible to suppress estimation errors that are likely to occur in high-dimensional feature components and to perform discrimination. Effective features can be captured.

In the above-described example, the case where three linear conversion units are provided and processing is performed in parallel and stepwise is shown. However, the linear discrimination unit can be realized basically by including a product-sum operation unit. It is also possible to realize that the discriminant matrix to be read out is switched using the switching linear conversion means in accordance with the input feature vector for performing the linear conversion. In this way, the use of one linear conversion means can reduce the scale of a necessary arithmetic unit.

Further, the calculation of the output feature vector z can be expressed as (Equation 10) as can be seen from (Equation 7), (Equation 8), and (Equation 9).

That is, the linear transformation using each discriminant matrix can be combined into a linear transformation using one matrix. The number of product-sum operations when performing stepwise operations is L × (M1 + M2) + M1N1 + M
When 2N2 is combined into one matrix, L × (N1 + N2), for example, N
When 1 = N2 = 500, M1 = M2 = 200, and L = 100, 240,000 product-sum operations are required in a stepwise operation, and the latter operation requires 100,000 product-sum operations. Therefore, the amount of calculation is smaller in the case of performing the batch operation like the latter, and high-speed calculation is possible. As can be seen from the equation, when the final number of dimensions L is reduced, it is more effective to use the batch calculation method because the calculation amount can be reduced.

(Second embodiment)
In the above example, when merging features with different types of features such as directional features and shade features, repeated discriminant analysis is performed on the feature vectors that have undergone discriminant analysis for each feature. However, a plurality of elements for one feature may be divided into a plurality of feature vectors, each element set may be discriminated and analyzed as an input feature, and the projected vector may be further discriminated and analyzed.

In the second embodiment, a face image feature extraction apparatus will be described.
In the face image feature extraction apparatus according to the second invention, as shown in FIG. 4, an image feature decomposing means 41 for decomposing the grayscale feature of the input face image, and a linear conversion means for projecting the feature vector according to the discriminant matrix corresponding to the feature vector. 42 and discriminant matrix storage means 43 for storing the respective discriminant matrices.

Regarding the technique for extracting features of a face image, the above-mentioned W.W. As shown in the paper by Zhao et al., There is a method in which the gray value is used as a vector feature after the face image is aligned by the eye position or the like.

In the second invention, the gray value of the pixel of the image is handled as the input feature as the original feature, but the image size is set to the coordinates of (14, 23), (29, 23), for example, the center position of the left and right eyes. Normalized 42 × 54 pixels = 2352 dimensions and a large image feature. In such a large feature dimension, it is difficult to perform accurate feature extraction even if linear discriminant analysis is performed directly using a limited number of learning samples. By performing discriminant analysis on the feature and obtaining a discriminant matrix, deterioration of the feature that occurs when principal component analysis or the like is applied is suppressed.

One of the methods for decomposing the image feature is to divide the image. For example, as shown in FIG. 5, the image is divided into 9 parts each having a size of 14 × 18 pixels (= 252 dimensions). , Local images having respective sizes are set as feature vectors xi (i = 1, 2, 3,..., 9), and each partial image is subjected to discriminant analysis using a learning sample. A discriminant matrix Wi corresponding to is obtained.

Note that by providing overlap between regions when dividing an image, the feature quantity based on the correlation between pixels in the boundary region can be reflected in the feature vector. You may do it.

The number of feature dimensions is 252 dimensions, which is significantly smaller than the original image. By using several hundreds of images of each person and a count of 1,000 facial images as samples, it is possible to use discriminant analysis. The basis matrix can be calculated with accuracy. If this is as large as the original feature (2352 dimensions), it is expected that more than a thousand face image samples will be required to obtain performance with the feature based on discriminant analysis. Since it is difficult to collect such large-scale image data, it cannot be realized.

If, for example, each local region is compressed into 20-dimensional features using the first-stage discrimination features, the output feature vectors will be 9 regions × 20 dimensions = 180-dimensional feature vectors. By further performing discriminant analysis on this feature vector, the number of dimensions can be efficiently compressed to about 50 dimensions, for example. The discriminant matrix of the second stage is also stored in the discriminant matrix storage means 43, and the linear transformation means 42 receives the 180-dimensional vector of the discriminant features of the first stage as input and performs discriminant analysis again. The first-stage discriminant matrix and the second-stage discriminant matrix may be calculated in advance as shown in (Equation 10), but the 252 dimensions × 9 area is changed to the 20 dimensions × 9 area. In the case of compressing and converting the 180 dimensions to 50 dimensions, it is more efficient to calculate in two stages because the memory used and the amount of calculation are less than half.

By applying discriminant analysis locally and stepwise in this way, facial features with high discrimination ability can be extracted. Speaking of character recognition, for example, when trying to identify “large” and “dog”, if a component with a large eigenvalue is extracted by principal component analysis of the entire character image,
The feature of “｀” that distinguishes between “large” and “dog” is likely to be lost. (For this reason, certain high-order features are used for similar character recognition rather than features with large eigenvalues by principal component analysis. Sometimes happen). The effectiveness of extracting discriminant features by dividing into local regions is
It is similar to the phenomenon of similar character identification in character recognition, and by limiting the features that are easy to identify spatially, it is possible to ensure accuracy per unit dimension rather than performing discriminant analysis of principal components as a whole. It is thought that it becomes.

Further, the image feature dividing unit 41 may sample and divide the entire image, instead of dividing the image for each local region to form a feature vector. For example, the primary feature is 9
In the case of dividing into nine 252-dimensional nine vectors, sampling is performed from a 3 × 3 region as shown in FIG. That is, the sampled image becomes a reduced image with a slight difference in position. By raster scanning this reduced image, it is converted into nine feature vectors. A discriminant component may be calculated using such a feature vector as a primary vector, and the discriminant component may be integrated to perform discriminant analysis again.

(Third embodiment)
Another embodiment of the present invention will be described in detail with reference to the drawings. FIG. 7 is a block diagram showing a face image matching system using the face metadata generation apparatus of the present invention.

Hereinafter, the face image matching system will be described in detail.
As shown in FIG. 1, in the face image matching system according to the present invention, a face image input unit 71 that inputs a face image, a face metadata generation unit 72 that generates face metadata, and the extracted face metadata are stored. A face metadata accumulating unit 73, a face similarity calculating unit 74 for calculating a face similarity from the face metadata, a face image database 75 for accumulating face images, and an image registration request /
In response to a search request, there are a control unit 76 that controls input of images, generation of metadata, accumulation of metadata, and calculation of face similarity, and a display unit 77 of a display that displays face images and other information. Is provided.

The face metadata generation unit 72 includes a region cutout unit 721 that cuts out a face region from the input face image, and a face pattern feature extraction unit 722 that extracts a face feature of the cut out region, and includes a face feature vector. To generate metadata related to the face image.

When registering a face image, an image input unit 71 such as a scanner or a video camera is used to input a face photograph or the like after matching the size and position of the face. Alternatively, a person's face may be input directly from a video camera or the like. In this case, it is better to detect the face position of the input image and automatically normalize the size of the face image using the face detection technique as shown in the above-mentioned Mohaddam literature. Would be good.

Further, the input face image is registered in the face image database 75 as necessary. Simultaneously with the registration of the face image, face metadata is generated by the face metadata generation unit 72 and stored in the face metadata storage unit 73.

At the time of search, a face image is input by the face image input unit 71 as in registration, and the face metadata generation unit 72 generates face metadata. The generated face metadata is once registered in the face metadata accumulation unit 73 or directly sent to the face similarity calculation unit 74.

In the search, when it is confirmed whether or not a face image input in advance is in the database (face identification), the similarity with each piece of data registered in the face metadata storage unit 73 is calculated. Based on the result with the highest similarity, the control unit 76 selects a face image from the face image database 75, displays the face image on the display unit 77, etc., and determines the identity of the face in the search image and the registered image. Confirm.

On the other hand, when confirming whether the face image specified in advance by the ID number or the like matches the searched face image (face identification), it is determined whether the face image with the specified ID number matches or not. When the degree of similarity is lower than the predetermined similarity, it is determined that they do not match, and when the degree of similarity is high, it is determined that they match, and the result is displayed on the display 77. indicate. If this system is used for entry management, entry control can be performed by controlling the automatic door by sending an opening / closing control signal from the control unit 76 to the automatic door instead of displaying a face image. .

As described above, the face image matching system operates, but such an operation can also be realized on a computer system. For example, a metadata generation program and a similarity calculation program for performing metadata generation as will be described in detail below are stored in a memory, respectively, and executed by a program control processor, thereby realizing face image matching. can do.

Next, the operation of the face image matching system, particularly the face metadata generation unit 72 and the face similarity calculation unit 74 will be described in detail.

(1) Face metadata generation The face metadata generation unit 72 uses the image I (x, y) normalized for position and size,
Extract facial features. The normalization of the position and size is, for example, when the eye position is (16, 24), (3
1, 24), the image may be normalized so that the size is 46 × 56 pixels. Hereinafter, a case where an image is normalized to this size will be described.

Next, a plurality of local regions of the preset face image of the face image are cut out by the region cutout unit 721. For example, for example, one of the above images is normalized (this is expressed as f (x,
and y)) and the other is a 32 × 32 pixel region g (x, y) in the central region centered on the face.
). What is necessary is just to cut out so that the position of both eyes may become the position of (9,12) and (24,12).

The center area of the face is cut out as described above by cutting out a range that is not affected by the hairstyle, etc., when the hairstyle changes (for example, before and after bathing when using face matching with a home robot). This is to extract stable features even if the hairstyle changes, but if the hairstyle does not change (such as when identifying a person in a scene in a video clip) Since collation performance can be expected to improve by collating with hairstyles included, face images are cut out for large face images that contain hairstyles and small face images at the center of the face.

Next, the face image feature extraction unit 722 performs Fourier transform on the two extracted regions f (x, y) by two-dimensional discrete Fourier transform to extract the feature of the face image.

FIG. 8 shows a more detailed configuration of the face image feature extraction means 722. In this face image feature extracting means, Fourier transform means 81 for performing discrete Fourier transform on the normalized and cut-out image,
A Fourier power calculation means 82 for calculating the power spectrum of the Fourier frequency component obtained by Fourier transform, and a real vector and an imaginary component of the Fourier frequency component calculated by the Fourier transform means 81 are regarded as a one-dimensional feature vector. The linear transformation means 83 for extracting the discriminant feature from the principal component of the feature vector, the base matrix storage means 84 for storing the base matrix for the transformation, and the discriminating feature of the main component in the same manner as the power spectrum. A linear transformation means 85 for extraction and a basis matrix storage means 86 for storing a basis matrix for the transformation are provided. Further, the discriminating features of the real component and the imaginary component of the Fourier feature and the discriminating feature of the power spectrum are normalized to a vector of size 1, respectively, and the discriminating feature of the vector is obtained by integrating the two feature vectors. And a discriminant matrix storage unit 89 for storing a discriminant matrix for the discriminant feature.

With this configuration, after extracting the Fourier frequency features, the main component discriminating features are respectively obtained for the feature vector having the real part and the imaginary part of the Fourier frequency component as elements and the feature vector having the power spectrum as an element. The feature amount of the face is calculated by calculating and calculating the discriminating feature again with respect to the feature vector obtained by integrating them.

Hereinafter, each operation will be described in more detail.
In the Fourier transform means 81, the input image f (x, y) (x = 0, 1, 2,... M
−1, y = 0, 1, 2,. . . , N−1), a two-dimensional discrete Fourier transform is performed according to (Equation 11), and its Fourier feature F (u, v) is calculated. This method is widely known and is described, for example, in the literature (Rosenfeld et al., “Digital Image Processing”, pp. 20-26.
The description is omitted here.

The Fourier power calculation means calculates the Fourier power spectrum | F (u, v) | by obtaining the magnitude of the Fourier feature F (u, v) according to (Equation 12).

The two-dimensional Fourier spectrum F (u, v) or | F (u, v) |
Transforms an image of only two-dimensional real components, so that the obtained Fourier frequency components are symmetrical. Therefore, these spectral images F (u, v), | F (u, v) | are u = 0, 1,. . . , M−1; v = 0, 1,. . . , N−1 M × N components, half of which are u = 0, 1,. . . , M−1; v = 0, 1,. . . , N-1 M ×
The N / 2 components and the remaining half of the components are substantially equivalent components. For this reason, the subsequent processing may be performed using half the component as the feature vector. Naturally, by omitting components that are not used as elements of the feature vector in the calculation of the Fourier transform unit 81 or the Fourier power calculation unit 82, the calculation can be simplified.

Next, the linear conversion means 83 handles the feature quantity extracted as the frequency feature as a vector. The subspace defined in advance is determined by a base vector (eigenvector) obtained by preparing a face image set for learning and discriminating and analyzing the principal components of the frequency feature vectors of the corresponding cutout region. For how to obtain this basis vector, see W.W. Since various methods including the Zhao document are generally well-known methods described in the document, the description thereof is omitted here. The reason why the discriminant analysis is not performed directly is that the dimension number of the feature vector obtained by the Fourier transform is too large to handle the discriminant analysis directly. The extraction of feature vectors at the first stage is one selection. Further, a basis matrix obtained by repeating discriminant analysis may be used here.

That is, the principal component discriminant matrix Φ1 stored in the base matrix storage means 84 is preliminarily learned by performing discriminant analysis of the principal component of the feature vector x1 obtained by making the real component and the imaginary component of the frequency feature one-dimensional by raster scanning. Can be obtained from Here, the Fourier feature does not necessarily have to be handled as a complex number, and the imaginary number component may be handled as a real number as just another feature element.

If the basis matrix to the principal component is ψ1 and the discrimination matrix obtained by discriminating and analyzing the vector of the principal component is W1, the discrimination matrix Φ1 of the principal component is expressed by (Equation 13).

Note that the number of dimensions to be reduced by principal component analysis may be about 1/10 of the original feature Fourier feature (around 200 dimensions), and then reduced to about 70 dimensions by this discriminant matrix. This basis matrix is calculated in advance from the learning sample and used as information stored in the basis matrix storage means 84.

Similarly, the Fourier power spectrum | F (u, v) | is represented as a one-dimensional feature vector x2 by raster scanning, and the basis matrix Φ2T = Ψ2TW obtained by performing discriminant analysis of the principal components of the feature vector. 2T is obtained in advance from the learning sample.

Thus, by calculating the principal component discrimination feature for each component of the Fourier feature, the principal component discrimination feature y1 of the real component and imaginary component feature vector x1 of the Fourier component and the main vector of the power spectrum feature vector x2 The component discrimination feature y2 can be obtained.

In the normalizing means 87, the size of the obtained feature vector is normalized to a unit vector of length 1, for example. Here, since the vector length changes depending on where the origin for measuring the vector is set, it is necessary to predetermine the reference position. This is because the projected feature vector y
What is necessary is just to use it as a reference point using the average vector mi calculated | required from the learning sample of i. By using the average vector as the reference point, the feature vector is distributed around the reference point. In particular, if the Gaussian distribution is used, the feature vector is distributed isotropically. In this case, it becomes easy to limit the distribution area.

That is, the vector yio obtained by normalizing the feature vector yi to the unit vector by the average vector mi is expressed as (Equation 14).

Thus, the normalization means is provided, and the feature vector y1 related to the real number and the imaginary number of the Fourier power.
By normalizing the feature vector y2 related to power to a unit vector, the size between two feature quantities that are different kinds of feature quantities is normalized, and the distribution characteristics of the feature vectors are stabilized. Can be made. In addition, since the size in the feature space necessary for discrimination has already been normalized in the process of dimension reduction, it is less susceptible to noise than when normalized in a feature space that contains more deleted noise. This is because normalization can be realized. By this normalization, it is possible to remove the influence of a fluctuation element such as a fluctuation component proportional to the overall illumination intensity, which is difficult to remove by simple linear conversion.

The feature vectors y1o and y2o thus normalized are integrated into one feature vector y as in (Equation 8), and a basis matrix W3 obtained by performing linear discriminant analysis on the integrated feature vector y is used. Then, the output feature vector z can be obtained by projecting onto the discriminant space. The discriminant matrix W3 for this purpose is stored in the discriminant matrix storage means 89, and the linear conversion means 88 is stored.
Then, a projection calculation for this is performed, and for example, a 24-dimensional feature vector z is calculated.

Note that when the output feature vector z is quantized to, for example, 5 bits per element, the size of each element needs to be normalized. For example, the output feature vector z is normalized according to the variance value of each element. Apply.

That is, the standard deviation value σi in the learning sample of each element zi of the feature vector z is obtained and normalized such that zo = 16zi / 3σi.
For example, the value may be quantized from -16 to 15.

The normalization at this time is an operation in which each element is multiplied by the reciprocal of the standard deviation. Therefore, considering a matrix Σ having σi as a diagonal element, the normalized vector zo becomes zo = Σz. That is, since it is a simple linear transformation, Σ may be applied to the discriminant matrix W3 in advance as in (Equation 15).

Normalization in this way not only has the advantage that the range correction required for quantization can be performed, but also normalization based on standard deviation values, so when calculating the norm of the inter-pattern distance during matching By simply calculating the L2 norm, the Mahalanobis distance can be calculated, and the amount of calculation at the time of collation can be reduced.

As described above, the facial image feature extraction unit 122 has explained the case of extracting the feature vector zf from the normalized image f (x, y). However, the image g (
x, y), the feature vector zg is also obtained by the face image feature extraction means 122 in the same manner as described above.
To extract. Two feature vectors zf and feature vector zg are extracted as face feature quantity z by the face metadata generation unit.

As described above, the face metadata generation procedure can be executed by a computer using a computer program.

(2) Face similarity calculation Next, the operation of the face similarity calculation unit 74 will be described.

The face similarity calculation unit 74 calculates the similarity d (z1, z2) between the two faces using the K-dimensional feature vectors z1 and z2 obtained from the two face metadata, respectively.

  For example, the similarity is calculated from the square distance of (Equation 16).

αi is a weighting factor, and for example, if the inverse of the standard deviation of each feature dimension zi is used, the calculation is based on the Mahalanobis distance. If the feature vector is normalized in advance using (Equation 15), the basis matrix is preliminarily expressed by the variance value. Since it has been normalized, it is Mahalanobis distance as described above. Further, the similarity may be calculated by the cosine formed by each feature vector to be compared in (Equation 3).

When distance is used, the larger the value, the smaller the similarity (the face is not similar). When cosine is used, the larger the value, the greater the similarity (the face is similar). Mean).

In the above description, a case where a single face image is registered and retrieval is performed using a single face image has been described. However, a plurality of images are registered for one face, and a single face image is registered. When searching by using, for example, the degree of similarity may be calculated for each of a plurality of face metadata on the registration side.

Similarly, in the case of registration of multiple images per face and search by multiple images, the similarity to one face data can be calculated by calculating the similarity by calculating the average or minimum value of the similarity of each combination. The degree can be calculated. This means that the matching system of the present invention can be applied to face recognition in a moving image by imitating the moving image as a plurality of images.

Although the embodiments of the present invention have been described with reference to the drawings as appropriate, it is needless to say that the present invention can also be realized by a computer-executable program.

(Fourth embodiment)
Another embodiment of the present invention will be described in detail with reference to the drawings. The present invention provides a third
In this invention, the face metadata generation unit 72 is improved. In the third aspect of the present invention, the main vector for the feature vector having the real part and the imaginary part of the Fourier frequency component obtained by performing the Fourier transform on the input face image, and the feature vector having the power spectrum as an element, respectively. The feature amount of the face is calculated by calculating the discriminant feature of the component and calculating the discriminant feature again for the feature vector obtained by integrating them. In this case, since the Fourier power spectrum reflects the feature amount of the entire input image, components with a lot of noise in the input pixel (for example, pixels around the mouth whose relative position is likely to change) are also included in the power spectrum. Even if an effective feature amount is selected by discriminant analysis, sufficient performance may not be obtained. In such a case, the input image is divided into regions, Fourier transformed for each local region, and discriminant analysis is performed with the power spectrum of each local region as a feature quantity, resulting in poor local discrimination performance (within the class Discriminant analysis of the influence of the feature amount of the area
Can be reduced.

FIG. 9 is a diagram for explaining the embodiment and shows a flow of feature extraction processing. In this embodiment, for example, an area of 32 × 32 pixels is divided into four areas of 16 × 16 pixels, 16 areas of 8 × 8 pixels, 64 areas of 4 × 4 pixels, 256 areas of 2 × 2 pixels, and 1024 areas of 1 × 1 pixels (because they are substantially the same as the input image). The input image may be left as it is, and Fourier transform is performed on each of the divided regions. FIG. 10 shows a summary of this processing flow. The 1024 × 5 dimension = 5120 dimension feature quantity of all the power spectra of each region obtained in this way is extracted. Since the number of dimensions is large when the number of normal learning data is small, principal component analysis is performed in advance, and a basis of principal component analysis that reduces the number of dimensions is obtained. For example, about 300 dimensions is appropriate as the number of dimensions. Discriminant analysis is further performed on the feature vector of this number of dimensions, the number of dimensions is reduced, and a base corresponding to a feature axis with good discrimination performance is obtained. Bases corresponding to principal component analysis and discriminant analysis are calculated in advance (this is referred to as PCLDA projection base Ψ).

The 5120-dimensional feature is projected by linear calculation using the projected basis Ψ using the PCLDA basis, whereby the discriminant feature z can be obtained. Further, by performing quantization or the like, it becomes a facial feature amount. Note that the 5120-dimensional feature quantity can be reduced by considering the symmetry of the Fourier power spectrum, etc., or by removing high frequency components in advance, so that the number of dimensions can be reduced, high-speed learning, and required data. Since the amount can be reduced and high-speed feature extraction can be performed,
It is desirable to reduce the number of dimensions as appropriate.

By blocking the region and multiplexing the Fourier spectrum in this way, the feature amount having the universality of parallel movement and the local feature amount are sequentially converted from the feature amount equivalent to the image feature (in the case of 1024 divisions). You can have multiple expressions. By selecting feature quantities effective for discrimination from the multiple redundant feature expressions by discriminant analysis, it is possible to obtain feature quantities that are compact and have good discrimination performance. The Fourier power spectrum is a non-linear operation with respect to an image, which can calculate an effective feature amount that cannot be obtained simply by applying a discriminant analysis that processes an image by a linear operation. Here, the case where the linear discriminant analysis is performed on the principal component has been described, but the kernel discriminant analysis (Kernel Fisher Discriminator).
inant Analysis, KFDA or KernelDiscriminan
t Analysis: KDA, Generalized Discriminan
t Analysis: Discriminant analysis using kernel technique called GDA)
The second stage feature extraction may be performed using. For example, for kernel discriminant analysis, Q.I. Liu et al. (Non-Patent Document 3: “Kernel-based Optim”
iZed Feature Vectors Selection and Discr
Iminant Analysis for Face Recognition, “P
laced of of IAPR International Conference
ce on Pattern Recognition (ICPR), Vol. II,
pp. 362-365, 2002) and G.R. Baudat's document (Non-Patent Document 4)
: "Generalized Discriminant Analysis Using
a Kernel Approach, “Neural Computation,
Vol. 12, pp 2385-2404, 2000), which are described in detail. By extracting features using kernel discriminant analysis in this way, the effect of feature extraction by nonlinearity can be further exhibited, and effective features can be extracted.

However, since a large feature vector of 5120 dimensions is handled in this case, a large amount of memory and a large amount of learning data are required even when principal component analysis is performed. In FIG. 11, in order to avoid such a problem, principal component analysis / discriminant analysis is individually performed for each block, and then discriminant analysis (Linear Discriminant Analysis: LDA) is performed in two stages, thereby reducing the amount of calculation. can do. In this case, each area has 1024 dimensions (
When the symmetry is halved in consideration of the symmetry, the principal component analysis and the discriminant analysis are performed using the 512-dimensional feature value to obtain the basis matrix Ψi (i = 0, 1, 2,..., 5). Then, the feature vector is normalized using each average value, and the second-stage LDA projection is performed. By performing processing for each block in this way, the number of data and computer resources required for learning can be reduced, and learning optimization time can be reduced. If it is desired to perform the calculation at high speed, the vector normalization process is omitted, and the PCLDA projection base matrix and the LDA projection base matrix are calculated in advance, so that the calculation speed can be increased.

FIG. 12 is a diagram for explaining another embodiment, and shows a flow of feature extraction processing.
In this embodiment, such region division is performed in a plurality of steps (two steps in the figure), and the power spectrum is multiplexed in a multiple manner so that the translation universality of the Fourier power spectrum of the local region and the reliability of the local region are taken into consideration. Extracted with multiple resolutions, extracted as feature quantities for discriminant analysis, and extracted features using the best feature space found in discriminant analysis.

For example, when the input image f (x, y) is 32 × 32 pixels, as shown in FIG. 10, the power spectrum | F (u, v) | of the entire image and four regions of 16 × 16 pixels obtained by dividing the power spectrum | F (u, v) | Respective power spectrums | F11 (u, v) |, | F12 (u, v) |
| F13 (u, v) |, | F14 (u, v) |, | F21 (u, v) |, divided into 16 regions of 8 × 8 pixels, | F21 (u, v) |,. A feature vector is extracted from | F216 (u, v) |. However, considering the symmetry of the Fourier power spectrum of the real image,
One half of that may be extracted. In order to avoid an increase in the size of the feature vector in discriminant analysis, the feature vector may be configured without sampling high-frequency components for discrimination. For example, it is possible to reduce the number of necessary learning samples or reduce the processing time required for learning or recognition by sampling a quarter of the spectrum corresponding to the low frequency component to construct a feature vector. it can. If the number of learning data is small, discriminant analysis may be performed after the principal component analysis is performed in advance and the number of feature dimensions is reduced.

Now, using the feature vector x2f extracted in this way, discriminant analysis is performed using a learning set prepared in advance, and its base matrix Ψ2f is obtained. In FIG. 9, the extraction of discriminant features from the principal components (Principal Component Linear Discriminator).
An example in which projection of (Nant Analysis: PCLDA) is performed is shown. The feature vector x2f is projected using the basis matrix Ψ2f, the average and size of the projected feature vector are normalized, and the feature vector y2f is calculated.

Similarly, for the feature vector x2f that integrates the real and imaginary components of the Fourier frequency, the feature vector is projected by linear arithmetic processing using the basis matrix Ψ1f to obtain a feature vector with a reduced number of dimensions. A feature vector y1f whose size is normalized is calculated. The feature vector obtained by integrating these is projected again using the discriminant basis Ψ 3 f to obtain a feature vector zf. By quantizing this to, for example, 5 bits, a facial feature amount is extracted.

When the input is a face image normalized to a size of 44 × 56 pixels, the above processing is performed on the 32 × 32 pixels in the central portion to extract the face feature amount and 44 × of the entire face.
As for the area of 56 pixels, the entire area of 44 × 56 pixels and 4 areas of 22 × 28 pixels, 1
A facial feature amount is extracted for each region divided into 16 pixels of 1 × 14 pixels. FIG. 13 shows another embodiment. When PCLDA is performed by combining the real component, the imaginary component, and the power spectrum for each local region, or when the real component and the imaginary component are combined as shown in FIG. This is an example in which the power spectrum and PCLDA are individually projected and finally LDA projection is performed.

(Fifth embodiment)
Another embodiment according to the present invention will be described in detail with reference to the drawings.

2 illustrates an embodiment of a facial feature description method and facial feature descriptor using the present invention. In FIG.
As an example of facial feature description, ISO / IEC FDIS 15938-3 “Inform”
application technology multimedia content des
creation interface- Part3: Visual in DD ”
L Expression Grammar (Description Definition Language Re
The description of the facial feature value is expressed using a presentation syntax.

Here, the description of the facial feature named AdvancedFaceRecognition is “FourierFeature” and “CentralFourie” respectively.
rFeature "and has an element named“ FoilerFeature ”or“ Ce ”
ntualFourierFeature is an unsigned 5-bit integer indicating that each element can have 24 to 63 dimensions. FIG. 16 shows a binary representation grammar (Binary Representation Syn) for the data representation.
tax) is used to represent the rules, FourierFeature, Cent
The size of an array element of ralFourierFeature is an unsigned 6-bit integer n
It is stored in umOfFourierFeature and numOfCentralFourier, and indicates that each element of FourierFeature and CentralFourierFeature is stored as a 5-bit unsigned integer.

Such a facial feature descriptor using the present invention will be described in more detail.
● numOfFourierFeature
This field specifies the size of the array of FourierFeatures. The acceptable range of values is 24 to 63.
NumOfCentralFourierFeature
This field defines the size of the CentralFourierFeature array. The acceptable range of values is 24 to 63.
● FourierFeature
This element represents a facial feature based on the hierarchical LDA of the Fourier characteristics of the normalized facial image.
The normalized face image is obtained by converting the size of the original image into an image of 56 lines having 46 luminance values in each line. The center position of both eyes in the normalized image is 2 for the right eye and the left eye, respectively.
It must be located in the 16th and 31st columns of the 4th row.

The elements of the FourierFeature are extracted from two feature vectors. One is a Fourier spectrum vector x1f, and the other is a multi-block Fourier intensity vector x2f. FIG. 17 illustrates the process of extracting Fourier features. Given a normalized image, the following five processing steps must be performed to extract its elements.

(1) Extraction of Fourier spectrum vector x1f (2) Extraction of multi-block Fourier intensity vector x2f (3) Projection of feature vector using PCLDA basis matrices Ψ1f and Ψ2f and normalization to unit vectors y1f and y2f (4) Projection of combined Fourier vectors of unit vectors using LDA basis matrix Ψ3f (5) Quantization of projection vector Zf

(STEP-1) Extraction of Fourier spectrum vector The Fourier spectrum F (u, v) for a given normalized image f (x, y) is expressed by
8) Calculate according to equation.

Here, M = 46 and N = 56. The Fourier spectrum vector x1f is defined by a set of components obtained by scanning the Fourier spectrum. FIG. 18 shows a Fourier spectrum scanning method. Scanning is performed on two regions in Fourier space, region A and region B. The scanning rules are summarized in FIG. Here, SR (u, v) represents the upper left coordinates of the region R, and ER (u, v) represents the lower right point of the region R. Therefore, the Fourier spectrum vector x1f is expressed by (Equation 19).

ｘ１ｆの次元数は６４４次元である。

The number of dimensions of x1f is 644 dimensions.

STEP 2) Extraction of multi-block Fourier intensity vector A multi-block Fourier intensity vector is extracted from the Fourier intensity of the partial image of the normalized face image. Three types of images are used as partial images: (a) the whole image, (b) a quarter image, and (c) a 16th image.

(A) Whole image The whole image f10 (x, y) can be obtained by removing columns on both sides of the image boundary of the normalized image f (x, y) and cutting out to an image size of 44x56. This is given by equation (20).

(B) 1/4 image The 1/4 image is obtained by converting the entire image f10 (x, y) into 4 blocks fk1 (x, y) (k = 1,
It can be obtained by dividing equally into 2, 3, 4).

ここで、ｓｋ１＝（ｋ−１）％２、ｔｋ１＝（ｋ−１）／２ である。

Here, sk1 = (k−1)% 2 and tk1 = (k−1) / 2.

(C) 1/16 image A 1/16 image is obtained by changing f10 (x, y) into 16 blocks fk2 (x, y) (k = 1, 2,
3,..., 16) and obtained by the following equation.

ここで、ｓｋ２＝（ｋ−１）％４、ｔｋ２＝（ｋ−１）／４ である。

Here, sk2 = (k−1)% 4 and tk2 = (k−1) / 4.

From these images, the Fourier intensity | Fkj (u, v) | is calculated as in the following equation (23).

Ｍｊは各々の部分画像の幅を表し、Ｍ０＝４４， Ｍ１＝２２， Ｍ２＝１１である。
Ｎｊは部分画像の高さを表し、Ｎ０＝５６， Ｎ１＝２８， Ｎ２＝１４である。

Mj represents the width of each partial image, and M0 = 44, M1 = 22, and M2 = 11.
Nj represents the height of the partial image, and N0 = 56, N1 = 28, and N2 = 14.

Multi-block Fourier intensity vectors are: 1) Whole image (k = 1), 2) 1/4 image (k = 1, 2, 3, 4), and 3) 1/16 image (k = 1, 2) , ...,
16) in order of scanning and scanning the low frequency region of each intensity | Fkj (u, v) |. The scanning area is defined in FIG.

Therefore, the multi-block Fourier intensity vector x2f is expressed by Equation (24).

ｘ２ｆの次元数は８５６次元である。

The number of dimensions of x2f is 856 dimensions.

STEP 3) PCLDA Projection and Vector Normalization Fourier spectrum vector x1f and multi-block Fourier intensity vector x2f are projected using PCLDA basis matrices Ψ1f and Ψ2f, respectively, and unit vectors y1f and y2f
Normalize to The normalized vector ykf (k = 1, 2) is given by

ここで、ＰＣＬＤＡ 基底行列Ψｋｆと平均ベクトルｍｋｆは、ｘｋｆの主成分の判別
分析によって得られる基底行列と射影して得られる平均ベクトルであり、予め計算してあ
るテーブルを参照する。ｙ１ｆとｙ２ｆ の次元数はそれぞれ７０次元と８０次元である
。

Here, the PCLDA basis matrix Ψkf and the average vector mkf are average vectors obtained by projecting with the basis matrix obtained by discriminant analysis of the principal component of xkf, and refer to a previously calculated table. The number of dimensions of y1f and y2f is 70 dimensions and 80 dimensions, respectively.

STEP 4) LDA Projection of Combined Fourier Vectors Normalized vectors y1f and y2f are connected to form a 150-dimensional combined Fourier vector y3f, and projected using an LDA basis matrix. The projection vector zf is given by the following equation.

STEP 5) Quantization The elements of zf are rounded to a 5-bit unsigned integer range using the following equation.

The quantized elements are stored as an array of FourierFeatures. Four
ierFeature [0] represents the quantized first element w0f, and Fourier
Feature [numOfFourierFeature-1] is the numOfF
ourerFeature-th element wfnumOfFourierFeature-
Corresponding to 1.

● CentralFourierFeature
This element represents a facial feature based on a hierarchical LDA of Fourier characteristics of the center portion of the normalized facial image. CentralFourierFeature is a FourierFeatur
Extract by the same method as re.

The center portion g (x, y) is obtained by cutting out the image to a size of 32 × 32 pixels from the start point (7, 12) of the image f (x, y) as shown in the following equation.

STEP 1) Extraction of Fourier Spectral Vector The Fourier spectrum G (u, v) of g (x, y) is calculated by the equation (29).

ここで、Ｍ＝３２， Ｎ＝３２である。２５６次元のフーリエスペクトルベクトルｘ１
ｇ は、フーリエスペクトルＧ（ｕ，ｖ）を図２０で定義したように走査することによっ
て得ることができる。

Here, M = 32 and N = 32. 256-dimensional Fourier spectrum vector x1
g can be obtained by scanning the Fourier spectrum G (u, v) as defined in FIG.

STEP 2) Extraction of multi-block Fourier intensity vector Multi-block Fourier intensity vector x2g is converted into (a) center portion g10 (x, y),
(B) 1/4 image gk1 (x, y) (k = 1, 2, 3, 4), and (c) 1/16 image gk2 (x, y) (k = 1, 2, 3, 4). .., 16) are extracted from the Fourier intensity.

  (A) Central part

  (B) 1/4 image

ここで、ｓｋ１＝（ｋ−１）％２、ｔｋ１（ｋ−１）／２ である。

Here, sk1 = (k−1)% 2, tk1 (k−1) / 2.

  (C) 1/16 image

ここで、ｓｋ２＝（ｋ−１）％４、ｔｋ２＝（ｋ−１）／４ である。

Here, sk2 = (k−1)% 4 and tk2 = (k−1) / 4.

The Fourier intensity | Gkj (u, v) | of each image is calculated as shown in (Expression 33).

ここで、Ｍ０＝３２，Ｍ１＝１６， Ｍ２＝８， Ｎ０＝３２， Ｎ１＝１６， Ｎ２
＝８である。マルチブロックフーリエ強度ベクトルｘ２ｇは、図２０に定義するようにそ
れぞれの強度｜Ｇｋｊ（ｕ，ｖ）｜を走査することによって得られる。

Here, M0 = 32, M1 = 16, M2 = 8, N0 = 32, N1 = 16, N2
= 8. The multi-block Fourier intensity vector x2g is obtained by scanning each intensity | Gkj (u, v) | as defined in FIG.

The processing of STEP 3-5) is the same as that of FourierFeature. Cent
The basis matrices Ψ1g, Ψ2g, Ψ3g and the average vectors m1g, m2g for ralFourierFeature are also referred to as prepared in advance as a table.

The size of the CentralFourierFeature array is numOfCent
Limited to ralFourierFeature.

The facial feature description data obtained in this way is a facial feature description data having a high recognition performance while having a compact description length, and is an efficient expression for storing and transmitting data.

Note that the present invention may be realized by a program operable by a computer. In this case, in the fifth embodiment, the functions shown in steps 1 to 5 in FIG. 17 are described by a computer-readable program, and the program is made to function on the computer. Is feasible. Further, when the example described in FIG. 17 is configured as an apparatus, all or part of the functions described in the block diagram of FIG. 21 may be realized.

11: First linear conversion means 11
12: Second linear conversion means 12
13: Third linear conversion means 13
14: First discriminant matrix storage means 14
15: Second discriminant matrix storage means 15
16: Third discriminant matrix storage means 16

Claims (12)

A pattern feature extraction method for extracting facial features from an input normalized image,
Calculating a Fourier spectrum vector by calculating a Fourier spectrum for the input normalized face image using a predetermined calculation formula; and
Extracting a multi-block Fourier intensity vector from the Fourier intensity of the partial image of the normalized image;
Projecting a feature vector using a discriminant matrix obtained by linear discriminant analysis of a Fourier spectrum vector and the multi-block intensity vector to obtain respective normalized vectors;
Concatenating respective normalized vectors, performing projection using a second discriminant matrix obtained by linear discriminant analysis of the coupled combined Fourier vectors, and obtaining a projection vector obtained by projection;
A pattern feature extraction method comprising:
The step of obtaining the Fourier spectrum vector is characterized in that the input normalized image is subjected to Fourier transform, and a vector having components obtained by sampling real and imaginary components of the obtained Fourier spectrum is extracted as a Fourier spectrum vector. 3. The pattern feature extraction method according to claim 2.
The step of extracting the multi-block Fourier intensity vector includes dividing the input normalized image into a plurality of partial images, calculating a Fourier intensity of each partial image, and a vector having the Fourier intensity of each partial image as an element. The pattern feature extraction method according to claim 1, wherein the pattern feature is extracted as a multi-block Fourier intensity vector.
Obtaining the normalized vector using a discriminant matrix obtained by linear discriminant analysis of a principal component of the Fourier spectrum vector, projecting the Fourier spectrum vector, and normalizing the size of the projected vector;
Projecting a multi-block Fourier intensity vector using a discriminant matrix obtained by linear discriminant analysis of the principal components of the multi-block Fourier intensity vector, and normalizing the magnitude of the projected vector, The pattern feature extraction method according to any one of claims 1 to 3.
A pattern feature extraction device that extracts facial features from an input normalized image,
Fourier spectrum generation means for calculating a Fourier spectrum vector by calculating a Fourier spectrum for an input normalized face image using a predetermined calculation formula;
A multi-block Fourier intensity vector extraction means for extracting a multi-block Fourier intensity vector from the Fourier intensity of the partial image of the normalized image;
Normalization vector generation means for performing projection of a feature vector using a discriminant matrix obtained by linear discriminant analysis of a Fourier spectrum vector and the multi-block intensity vector, and obtaining respective normalized vectors;
Projection vector generating means for connecting the respective normalized vectors, performing projection using a second discriminant matrix obtained by linear discriminant analysis of the coupled combined Fourier vectors, and obtaining a projection vector obtained by the projection;
A pattern feature extraction apparatus comprising:
The Fourier spectrum vector generation means performs Fourier transform on the input normalized image, and extracts a vector having components obtained by sampling real and imaginary components of the obtained Fourier spectrum as Fourier spectrum vectors. Item 6. The pattern feature extraction apparatus according to Item 5.
The multi-block Fourier intensity vector extraction means divides the input normalized image into a plurality of partial images, calculates a Fourier intensity of each partial image, and multi-blocks a vector having the Fourier intensity of each partial image as an element. The pattern feature extraction apparatus according to claim 5, wherein the pattern feature extraction device is extracted as a Fourier intensity vector.
The normalized vector generation means projects a Fourier spectrum vector using a discriminant matrix obtained by linear discriminant analysis of the principal component of the Fourier spectrum vector, and normalizes the magnitude of the projected vector. And
A multi-block Fourier intensity vector means for projecting a multi-block Fourier intensity vector using a discriminant matrix obtained by linear discriminant analysis of the principal component of the multi-block Fourier intensity vector and normalizing the size of the projected vector; The pattern feature extraction apparatus according to claim 5, wherein:
A process for obtaining a Fourier spectrum vector by calculating a Fourier spectrum for an input normalized face image using a predetermined calculation formula in a computer;
A process of extracting a multi-block Fourier intensity vector from the Fourier intensity of the partial image of the normalized image;
Projecting feature vectors using a discriminant matrix obtained by linear discriminant analysis of a Fourier spectrum vector and the multi-block intensity vector, and obtaining respective normalized vectors;
A process of concatenating the respective normalized vectors, performing projection using a second discriminant matrix obtained by linear discriminant analysis of the coupled combined Fourier vectors, and obtaining a projection vector obtained by the projection;
Pattern feature extraction program to execute
The process of obtaining the Fourier spectrum vector is characterized in that the input normalized image is Fourier transformed, and a vector having components obtained by sampling real and imaginary components of the obtained Fourier spectrum is extracted as a Fourier spectrum vector. The pattern feature extraction program according to claim 9.
The process of extracting the multi-block Fourier intensity vector includes dividing the input normalized image into a plurality of partial images, calculating a Fourier intensity of each partial image, and a vector having the Fourier intensity of each partial image as an element. The pattern feature extraction program according to claim 9, wherein the pattern feature extraction program is extracted as a multi-block Fourier intensity vector.
The process of obtaining the normalized vector is a process of projecting a Fourier spectrum vector using a discriminant matrix obtained by linear discriminant analysis of the principal components of the Fourier spectrum vector, and normalizing the size of the projected vector;
A multi-block Fourier intensity vector is projected using a discriminant matrix obtained by linear discriminant analysis of the principal components of the multi-block Fourier intensity vector, and the size of the projected vector is normalized. The pattern feature extraction program according to any one of claims 9 to 11.

Images