JP4891278B2 - Facial image feature extraction method and identification signal generation method - Google Patents

Description

  The present invention relates to a facial image feature extraction method and an identification signal creation method, and more particularly to a fast and compact facial image feature extraction method and an identification signal creation method while improving robustness against illumination fluctuations.

  Conventionally, biometric information used for personal identification includes a face, a fingerprint, a vein, and an iris. However, since personal identification using a face can be performed without contact, it is highly convenient for the user.

  In addition, it is considered that the psychological burden on the user is light because identifying the other party by looking at the face is a routine operation.

  However, even if the face image is the same person, the appearance varies greatly depending on the illumination conditions, face orientation, and facial expression, so the recognition performance varies greatly depending on the face image shooting conditions.

  The face image recognition method is roughly divided into two methods: a method that features a shading pattern of a face image and a method that performs graph matching.

  A typical example of the former is Eigenfaces that performs principal component analysis on a face image and creates a basis vector called an eigenface (Non-Patent Document 1).

  The latter includes a method of applying a Gabor Wavelet filter to a face image (Non-Patent Document 2, Non-Patent Document 3).

  The method using the Gabor Wavelet filter is used in many researches and products because it can provide stable recognition performance even with face orientation and illumination fluctuations.

  However, since the template data created from the face image is several kilobytes and the calculation amount is relatively large, it is difficult to operate in an environment with limited calculation resources such as an embedded device.

  The Discrete Cosine Transform (DCT) is one of orthogonal transforms. However, since the signal energy tends to concentrate in the low frequency region, when used for image compression, the image is more effectively than other orthogonal transforms. Can be compressed. That is, by performing DCT on an image, most of the information held in the image before conversion can be stored with DCT coefficients in the low frequency region.

  Based on such DCT features, methods using DCT for pattern matching and face image recognition have been developed (Non-Patent Documents 4 and 5).

However, since the DCT coefficient depends on the appearance of the target image, the recognition performance may be deteriorated when the illumination variation occurs in the face image.
M. Turk and A. Pentland, “Face recognition using eigenfaces,” IEEE Proc. Conf. On Computer Vision and Pattern Recognition, pp.586-591, 1991. L. Wiskott, JM Fellous, N. Kruger and C. Malsburg, “Face recognition by elastic bunch graph matching,” IEEE Trans. Pattern Anal. Mach. Intell., Vol.19, no.7, 775-779, 1997. Takaaki Hirayama, Yoshio Iwai, Masahiko Taniuchi, “Integration of face location estimation and personal identification for face recognition”, IEICE (D-II), vol.J-88-D-II, no.2, pp. 276-290, 2005. ZM Hafed and MD Levine, “Face Recognition Using the Discrete Cosine Transform”, International Journal of Computer Vision, vol.43, no.3, 167-188, 2001. HK Ekenel and R. Stiefelhagen, “Local Appearance based Face Recognition Using Discrete Cosine Transform,” EUSIPCO 2005, Antalya, Turkey, 2005.

  SUMMARY OF THE INVENTION Accordingly, the present invention provides a fast and compact facial image feature extraction method and identification signal generation method while improving robustness against illumination variation.

  The other subject of this invention becomes clear by the following description.

  The above problems are solved by the following inventions.

(Claim 1)
A step of dividing a plurality of face images of a plurality of people with odd-numbered division patterns and different block sizes;
Performing orthogonal transform on each of the divided blocks and calculating transform coefficients;
Selecting a conversion coefficient corresponding to an individual with high fitness based on a genetic algorithm (GA) from the calculated conversion coefficient;
And a step of storing the position of the selected conversion coefficient.

(Claim 2)
A step of dividing a plurality of face images of a plurality of people with odd-numbered division patterns and different block sizes;
Performing orthogonal transform on each of the divided blocks and calculating transform coefficients;
Normalizing the calculated conversion coefficient;
Quantizing the normalized transform coefficients;
Selecting a transform coefficient corresponding to an individual having high fitness based on a genetic algorithm (GA) from the quantized transform coefficient;
And a step of storing the position of the selected conversion coefficient.

(Claim 3)
3. The facial image feature extraction method according to claim 1, wherein the orthogonal transformation is DCT.

(Claim 4)
Based on the feature extraction position stored by the feature extraction method of the face image according to claim 1, 2, or 3,
Dividing a plurality of face images of a person to be registered into odd-numbered division patterns and different block sizes;
Performing orthogonal transform on each of the divided blocks and calculating transform coefficients;
And a step of creating a template from the conversion coefficient extracted based on the calculated conversion coefficient and the feature extraction position.

(Claim 5)
Based on the feature extraction position stored by the feature extraction method of the face image according to claim 1, 2, or 3,
Dividing a plurality of face images of a person to be registered into odd-numbered division patterns and different block sizes;
Performing orthogonal transform on each of the divided blocks and calculating transform coefficients;
Normalizing the calculated conversion coefficient;
Quantizing the normalized transform coefficients;
And a step of creating a template from a transform coefficient extracted based on the quantized transform coefficient and the feature extraction position.

(Claim 6)
6. The identification signal generation method according to claim 4, wherein the orthogonal transformation is DCT.

  According to the present invention, it is possible to provide a fast and compact facial image feature extraction method and identification signal generation method while improving robustness against illumination variation.

  Hereinafter, preferred embodiments of the present invention will be described.

  FIG. 1 is a flowchart for explaining an example of feature vector extraction information processing according to the present invention. Feature vector extraction information creation processing will be described with reference to FIG.

First, a face image is input (S1). For ease of explanation, input Kaoga Zosa size is described as vertical 60 × horizontal 60Pixels. In addition, since it is a process of selecting feature points to be described later, feature vectors are selected based on face images for about 3000 people .

  Next, the input face image is divided into multi-blocks (S2).

  Multi-block division is a method of dividing a face image into a plurality of different sizes.

  Conventionally, when performing DCT, which will be described later, a method of performing DCT on the entire face image and a method of performing DCT by dividing the face image into the same size (for example, 4 × 4 pixels, 8 × 8 pixels) have been employed.

  When DCT is performed on the entire face image, the features of the position and size of the facial organs such as eyes, nose, and mouth can be extracted effectively, but the appearance of the face image changes greatly when illumination variation occurs. As a result, the DCT coefficient fluctuates and the recognition performance deteriorates.

  In addition, when DCT is performed by dividing a face image into the same size, it is possible to cope with illumination fluctuations compared with the case where DCT is performed on the entire face image. As a result, the personal identification performance decreases. Therefore, the division method of the same size is not suitable for face authentication.

  Therefore, the disadvantages of the above method can be compensated by dividing the face image into multi-blocks. That is, it is possible to effectively cope with illumination fluctuation while effectively extracting the features of the position and size of the face function.

  In the multi-block division according to the present invention, when the face image data is 60 × 60 pixels, the block size is four different block sizes of 20 × 20, 20 × 12, 12 × 20, and 12 × 12. Divide into In other words, the face image is divided by odd-numbered division with the size of the entire face image being 1/3 or 1/5.

  The odd division is to divide the face image by odd numbers in the vertical and horizontal directions. For example, the face image is divided into three or five in the vertical direction and three or five in the horizontal direction.

  The reason for performing the odd division is to make it easy to extract the features of these organs because the human face is symmetrical and the nose and mouth are in the center of the face. In the even division, the nose and mouth are divided, and there is a problem that the features of those organs cannot be extracted.

  In other words, multi-block division in the present invention is to divide a plurality of face images into odd-numbered division patterns and different block sizes.

  Now, the necessity of multi-block division will be described with reference to Table 1.

  Table 1 shows the relationship between DCT coefficients and frequencies (displays up to ½ of the signal length).

  By performing DCT with a plurality of block sizes as shown in Table 1, more types of frequency components are extracted than using a single block size (for example, 8 × 8 pixels, a signal length of 8 in Table 1). be able to. In order to extract unique features from an image with a high degree of similarity such as a face image, it is considered that extracting as many frequency components as possible is effective in expressing small differences in facial features. However, as can be seen from Table 1, when the block size (signal length) to be used is a close value (for example, when 20 and 18 are used), the frequency components that can be extracted are close to each other. Will be similar. For this reason, it is considered that the block size to be used should have a certain interval.

  An example of multi-block division of face image data is shown in FIG.

  In FIG. 2, (a) is an example in which the entire face image is divided at 20 × 20 pixels per block. That is, the pattern is divided into three parts vertically and three parts horizontally. (B) is an example in which the entire face image is divided by 20 × 12 pixels per block. In other words, the pattern is divided into 3 parts vertically and 5 parts horizontally. (C) is an example in which the entire face image is divided at 12 × 20 pixels per block. That is, the pattern is divided into 5 parts vertically and 3 parts horizontally. (D) is an example in which the entire face image is divided at 12 × 12 pixels per block. That is, the pattern is divided into 5 parts vertically and 5 parts horizontally.

  Conventionally, when a face image is divided, the distortion between blocks is reduced by overlapping the boundary lines of adjacent blocks.

  However, when the image is restored by one type of block division, the boundary line is distorted unless overlapping is performed.

  In the present invention, by dividing the image into a plurality of sizes, an image divided by one type of block division is contained in one block in the other block divisions, so that no overlap is required. This is because it is not necessary to restore the face image. Furthermore, there is no problem if a frequency component can be extracted in any block.

  Here, in the present invention, the case where four types of block sizes are used is described, but there is no particular limitation, and the present invention is not limited as long as it is an influence of light and division for extracting many frequency components. It is fully applicable to. Therefore, the multi-block division can sufficiently cope with not only the increase / decrease of the types when dividing but also the movement of the position and the like.

  Next, the divided blocks are converted into frequency components by DCT, and DCT coefficients are calculated (S3). Specifically, in FIGS. 2A to 2D, all divided blocks are converted into frequency components by DCT, and DCT coefficients are calculated. The number of DCT coefficients calculated in FIG. 2A is calculated as 20 × 20 × 9 = 3600 DCT coefficients. 2B to 2D are similarly performed for each block. Since four images are used here, 3600 × 4 = 14400 DCT coefficients are calculated. .

  In S2, the multi-block division is performed and the DCT is converted into the frequency component. Therefore, the DCT in the present invention is hereinafter also referred to as “multi-block DCT”.

  Here, since the calculated DCT coefficients have different frequency components depending on the divided pattern, more frequency components are extracted from the human face image.

  DCT is often used for image compression, audio compression, and the like, and is one method for converting image information into the frequency domain. When DCT is performed on an image, the image information is collected in a small number of low frequency components in the upper left, so that effective compression can be performed by cutting high frequency components with less image information.

  Here, because of the property that image information gathers in a small number of low-frequency components, it is considered that most of the original image information can be expressed with only a small number of DCT coefficients of low-frequency components. In the present invention, such a DCT feature is applied to pattern recognition to identify a face image with a small amount of information.

  The two-dimensional DCT is given by Equation 1.

However,

M and N are the number of pixels in the vertical and horizontal directions of the image.

  Next, the DCT coefficient calculated by DCT is normalized (S4).

  In order to reduce the fluctuation of the DCT coefficient between the same persons due to the change of the face image, the DCT coefficient is normalized for each frequency component using the following formula 3.

  Next, the normalized DCT coefficient is quantized (S5).

  One DCT coefficient can be stored in a 1-byte variable so that face data created from a face image is compact. Therefore, the DCT coefficient is quantized to an integer value in the range of −127 to +127.

  Through this quantization, the DCT coefficient is suppressed to 1 byte (8 bits), and is further set to an integer value, so that the calculation speed can be further increased.

  In the present invention, the quantized DCT coefficient is referred to as a feature vector.

  Next, a feature vector is selected using a genetic algorithm (S6).

  Since the feature vector of the face image obtained by the processing up to S5 has the same number of dimensions as the number of pixels of the face image, the number of dimensions of the feature vector obtained by DCT performed with a plurality of block sizes becomes very large.

  When performing personal identification using a face image, not all DCT coefficients are required.

  In the conventional method, only the low-frequency component in which the energy of the DCT coefficient is collected is used. However, in this method, not only the DCT coefficient in which individual characteristics are strong, but also many DCT coefficients that have few individual characteristics and are unnecessary for identification are included. As a result, the identification performance is lowered. Moreover, it has been found from the experiments so far that the DCT coefficient necessary for individual identification is not only in the low frequency component but also in the high frequency component.

  Therefore, in the present invention, a minimum feature vector necessary for performing individual identification using a genetic algorithm (GA) is selected. Thus, effective personal identification can be performed using fewer feature vectors than in the past.

  That is, the genetic algorithm (GA) is a method for deriving which frequency component of the DCT coefficient is more suitable for human identification.

  The feature vector is selected by performing the genetic algorithm (GA) by the following method.

  First, encoding is performed to express the entire feature vector as one individual. The bit of each individual gene corresponds to each feature vector, and whether the feature vector is used or not is determined by turning the bit on or off.

  Next, the fitness is obtained. The GA generates a population while preferentially leaving individuals with high fitness at the time of generation update. In the present invention, the fitness of an individual is obtained from the recognition performance. In other words, a target other person acceptance rate is set, and an individual with a low individual rejection rate is set as an individual with high fitness when the other person acceptance rate is equal to or lower.

  Next, make a selection. Selection is an operation of probabilistically selecting a parent individual to be crossed from the group prior to crossing according to fitness. In the present invention, a tournament method is used for selection. The tournament method is a method in which when selecting one parent individual, two individuals are picked up from the group and an individual with high fitness is used as a parent individual.

  Next, crossover is performed. Crossover is to create a new individual by recombining genes between two selected parent individuals. In the present invention, uniform crossover is used as the crossover method.

  Next, the mutation probability is set. Mutation is an operation of changing the bit of each individual gene according to the mutation probability. It is possible to avoid falling into a local solution by mutation, and to search for a global solution. In the present invention, the mutation probability is 0.1%.

  Through the above steps, the position of the feature vector can be selected.

  In the present invention, the position of the feature vector is determined using about 3000 face images having an image size of 60 × 60 pixels. Here, the position of the feature vector selected by the GA is called a feature point.

  The feature vector determined by the GA with the target others acceptance rate being less than 0.1% was about 450 points. Therefore, the number of DCT coefficients obtained by DCT by multi-block division (60 × 60 × 4 = 14400 when four face images per person are used) is about 1/36 of the data amount of the face image. I understood that I can recognize it.

  FIG. 3 shows an example of feature points selected by the GA. A position represented by a white pixel in the figure is a feature point. Here, the feature points shown in FIG. 3 are examples when the four face images of FIGS. 2A to 2D are arranged. Specifically, FIG. 2 (a) is the upper left, FIG. 2 (b) is the upper right, FIG. 2 (c) is the lower left, and FIG. 2 (d) is the lower right. However, the arrangement is not particularly limited.

  The position of the feature point is extraction information, which is feature vector extraction information. By using this feature vector extraction information, an identification signal can be easily created.

  This feature vector extraction information can be used for any person's face image when an identification signal is generated.

  Next, the identification signal creation process will be described with reference to FIG.

  First, as in the feature vector selection process described with reference to FIG. 1, the face image of the person to be registered is input (S10). The person to be registered is a person of a template to be described later, and is a person who is a basis for creating a template that is a criterion for later authentication. The size of the face image data will be described below as 60 × 60 pixels described in the feature vector selection process.

  Next, since the process from S2 to S5 of the feature vector selection process described with reference to FIG. 1 is performed on the face image of one person to be registered, the description is omitted (S11 to S14).

  Next, the feature vector of the person to be registered is extracted based on the feature vector extraction information created by the feature vector selection process (S15). For example, feature vector extraction information stored in advance is called from a storage unit (not shown) and the process is performed together with the feature vector in S14. Specifically, the feature vector extraction information is the position of the white frame shown in FIG. 4, and the feature vector is the feature vector of the person to be registered. Therefore, the feature vector corresponding to the position of the white frame can be extracted.

  Next, the extracted quantized DCT coefficient is templated (S16). The arrangement method is not particularly limited. For example, the feature vectors corresponding to the white frames are arranged so that the white frames shown in FIG. 4 are arranged one by one from the upper left to the right.

  The template created by the above steps becomes an identification signal. This makes it possible to identify a person at the time of authentication. In addition, it is possible to call a person having a close personality or the like by determining the target person or setting a threshold value or the like by creating a template of the criminal's face image at the time of a crime or the like using only the identification signal.

  Furthermore, the present invention is applicable if the face direction with respect to the camera is about 20 to 30 degrees in the vertical and horizontal directions.

  As the DCT block size is larger, the information of the low frequency component of the signal can be extracted, so that it is less susceptible to small changes in the signal. When the face direction with respect to the camera is about 20 to 30 degrees in the vertical and horizontal directions, the positional deviation of the organ of the face is about 2 pixels. When processing is performed by dividing a face image into blocks of a local area, the displacement of the position of a facial organ due to a change in the tilt of the face is more greatly affected as the block size is smaller. This is because the facial organ that existed in the block may not exist due to the displacement.

  In the present invention, the face image is divided into blocks in the local area, but a plurality of types of block sizes are used in addition to small square blocks.

  The DCT coefficient calculated with a small block size is affected by the position shift of the facial organ, but the influence is small with a large block size, so that the influence of the entire DCT coefficient extracted from the face image by the multi-block DCT is small. In addition, since the DCT coefficient is normalized so that the fluctuation of the DCT coefficient between the same persons is reduced by Equation 4, the fluctuation of the DCT coefficient due to the change in the tilt of the face can be reduced.

  As described above, since the variation of the feature vector caused by the change in the tilt of the face is small, the authentication can be performed even when the face direction is about 20 degrees to 30 degrees with respect to the camera. .

  Further, in the present invention, the orthogonal transform is described as the conversion to the frequency component by DCT. However, the present invention is not limited to this. For example, the present invention can also be applied using the conversion to the frequency component by DST (discrete sine transform). is there.

  The actual authentication process will be described below.

  First, a face image is cut out from an image taken in the following procedure, and normalization is performed to align the position, size, rotation, and the like.

  (1) The position of the face and pupil is detected from the captured image.

  (2) Enlargement / reduction is performed so that the pupil interval is 36 pixels with reference to the detected pupil position, and a face image of 60 × 60 pixels is cut out.

  (3) Non-linear density conversion using the log function is performed on the cut face image, and further normalization is performed on the local region using the following equation (5).

Here, m and σ are average values and standard deviations of local regions, and m ₀ and σ ₀ are average values and standard deviations experimentally obtained from many face images, m ₀ = 150 and σ ₀ = 50. is there.

  Multi-block DCT is performed on the face image normalized by the above procedure, and the face image is converted into a feature vector. Further, a feature vector on the feature point determined by the genetic algorithm is extracted as a one-dimensional signal. This is face data created from each face image. A recognition rate (similarity) r is calculated using the cross-correlation expressed by the following formula 6 for this face data.

Here, N is the size (number of feature vectors) of face data, and f _i and g _i are face data (feature vectors) created from face images at the time of registration and authentication (collation), respectively.

  In order to evaluate the recognition performance of the present invention, a recognition experiment was performed using about 5000 face images taken under different illumination conditions.

  Here, in order to compare the recognition performance between the present invention and another technology, the evaluation is made with the FRR (person rejection rate) value when the FAR (other person acceptance rate) is 0.1%. FAR and FRR are each expressed by the following formula 7.

  Table 2 shows the comparison results of recognition performance. From Table 2, it can be seen that the present invention has higher recognition performance than the conventional method regardless of the presence or absence of illumination variation, and the identity rejection rate is 10% or more lower than the conventional method when there is illumination variation.

  In addition, the processing time was measured on a commercially available personal computer (Dell Dimension 8400, Intel Pentium4-3.2GHz (Dell, Intel and Pentium are registered trademarks)). The comparison between registered face data and authenticated face data was found to be very fast, 0.0007 milliseconds (1.4 million checks per second, calculated from 100 million verification times). Therefore, if a face image is converted into face data and registered in a server or the like, a face search can be performed at a very high speed.

  According to the present invention, a fast and compact face recognition method using discrete cosine transform and a genetic algorithm can be provided. In addition, a multi-block DCT can be provided to improve high-speed recognition and robustness against illumination variation. Furthermore, by quantizing DCT coefficients and selecting feature vectors using a genetic algorithm, the size of the face data created from one face image is about 450 bytes, which is much smaller than the conventional method. The effect becomes.

  Experiments using face images taken in an environment in which illumination fluctuations occur showed that the present invention is more robust against illumination fluctuations than the conventional method. Also, the recognition speed is as high as about 100 milliseconds from when a face image is taken on a commercially available personal computer to when the face is cut out and recognized.

  From the above, it is considered that a system that searches for a target face from a database that stores a large number of face images is very effective. Further, the present invention is considered to enable the development of a real-time face recognition system combined with a surveillance camera or the like because high-speed recognition is possible.

  Since the present invention places importance on compactness and high speed, the present invention is particularly suitable for mounting on a monitoring camera or the like that requires real-time authentication, such as a portable terminal that requires a small amount of memory.

The flowchart explaining an example which concerns on the feature vector extraction information processing concerning this invention (A) is a diagram showing an example of division by three vertical and horizontal divisions, (b) is a diagram showing an example of division by three vertical and five horizontal divisions, and (c) is five vertical and three horizontal divisions. (D) which shows the example of a division | segmentation by (5) is a figure which shows the example of a division | segmentation by vertical 5 division and horizontal 5 division The figure which shows an example of the feature vector extraction information which concerns on this invention The flowchart explaining an example which concerns on the template creation process which concerns on this invention

Claims (6)

A step of dividing a plurality of face images of a plurality of people with odd-numbered division patterns and different block sizes;
Performing orthogonal transform on each of the divided blocks and calculating transform coefficients;
Selecting a conversion coefficient corresponding to an individual with high fitness based on a genetic algorithm (GA) from the calculated conversion coefficient;
And a step of storing the position of the selected conversion coefficient. A step of dividing a plurality of face images of a plurality of people with odd-numbered division patterns and different block sizes;
Performing orthogonal transform on each of the divided blocks and calculating transform coefficients;
Normalizing the calculated conversion coefficient;
Quantizing the normalized transform coefficients;
Selecting a transform coefficient corresponding to an individual having high fitness based on a genetic algorithm (GA) from the quantized transform coefficient;
And a step of storing the position of the selected conversion coefficient.   3. The facial image feature extraction method according to claim 1, wherein the orthogonal transformation is DCT. Based on the feature extraction position stored by the feature extraction method of the face image according to claim 1, 2, or 3,
Dividing a plurality of face images of a person to be registered into odd-numbered division patterns and different block sizes;
Performing orthogonal transform on each of the divided blocks and calculating transform coefficients;
And a step of creating a template from the conversion coefficient extracted based on the calculated conversion coefficient and the feature extraction position. Based on the feature extraction position stored by the feature extraction method of the face image according to claim 1, 2, or 3,
Dividing a plurality of face images of a person to be registered into odd-numbered division patterns and different block sizes;
Performing orthogonal transform on each of the divided blocks and calculating transform coefficients;
Normalizing the calculated conversion coefficient;
Quantizing the normalized transform coefficients;
And a step of creating a template from a transform coefficient extracted based on the quantized transform coefficient and the feature extraction position.   6. The identification signal generation method according to claim 4, wherein the orthogonal transformation is DCT.