JP4569186B2 - Image processing apparatus and method, recording medium, and program - Google Patents

Description

  The present invention relates to an image processing apparatus and method, a recording medium, and a program, and in particular, an image processing apparatus and method, a recording medium, and an image processing apparatus and method capable of stably determining an image regardless of deformation or inclination of the image, Regarding line-up programs.

  For example, it is known to use a support vector machine (SVM) to make a robot identify a human face. The support vector machine (SVM) is well known as a powerful pattern identification technique with high generalization ability, but there are the following problems when performing face identification using this technique.

1. In order to construct a classifier with good performance, a large amount of teacher data is required during learning.
2. Learning takes a lot of computation time.
3. Only two classes can be identified.

  There is a paper of Non-Patent Document 1 as a representative one that introduces a face identification algorithm using SVM. In this paper's technique, q SVM classifiers are used to identify a registered person class and other person classes in order to identify q persons. This technology requires a large amount of face image data of the person who wants to register and other persons already registered in the teacher data for learning of each classifier. There is a problem that registration takes a long time.

  In order to solve the above problem, Jonathon Phillips et al. In Non-Patent Document 2 used a face recognition method using PCA (Principal Components Analysis) with a differential face proposed by Baback Moghaddam et al. We have proposed a more accurate face recognition algorithm that can perform face registration and additional learning in real time by class identification with SVM instead of.

  Baback et al.'S face recognition method uses a single classifier (a combination of PCA and Bayesian estimation) that identifies two classes, and identifies any number of faces. This is a method that can be registered, and this is based on the fact that the factors that change the face image can generally be classified into two types as follows.

1. intra-personal variations: Changes in facial images of the same person due to lighting conditions, facial expressions, growth, etc. extra-personal variations: Changes in facial images due to person differences

Bernd Heisele, “Face Recognition with Support Vector Machines: Global versus Component-based Approach”, Massachusetts Institute of Technology Center for Biological and Computational Learning Canmbridge, U.S.A. P. Jonathon Phillips, “Support Vector Machines Applied to Face Recognition”, NISTIR 6241, U.S. Department of commerce Technology Administration, National Institute of Standards and Technology, November 1998 Baback Moghaddam, “Beyond Eigenfaces: Probabilistic Matching for Face Recognition”, MIT Media Laboratory, U.S.A.

  In Baback et al.'S method, a large number of teacher images are created in advance by adding a difference facial class (intra-personal class) of two facial images of the same person and an extra-personal class of two facial images of another person. To learn from PCA. This learning data does not need to include the face image of the person to be registered. Thereby, it is not necessary to perform learning again when the face is registered, and the registered image for generating the difference image is merely stored as it is, so that real-time face registration processing can be performed. In this method, assuming that the distribution of each class is a Gaussian distribution, the class existence probability is obtained by a method that combines PCA and Bayesian estimation, but this assumption is sufficient in environments other than those where lighting conditions are very well prepared. It does not hold. For this reason, there is a problem that it is not possible to sufficiently cope with a change in face image in a natural environment required for a robot or the like, and satisfactory performance is often not obtained. Further, in order for the difference image to sufficiently represent the change of the two face images, it is necessary to generate the difference image after performing accurate alignment between the feature points of the two face images. In order to do this, Baback et al. Found feature positions such as the eyes and nose by pattern matching using PCA, and after performing affine transformation so that those feature positions are at the same position, a difference image is generated. In this method, there is a problem that the performance is remarkably lowered as the face deformation or inclination between the two face images increases.

  The method proposed by Jonathon et al. Uses SVM instead of PCA and Bayesian estimation for this class identification. This enables correct discrimination even when the class distribution is more complicated than the Gaussian distribution, and improves the discrimination performance in environments where the lighting conditions etc. include changes. Therefore, similarly, there is a problem that the performance is remarkably lowered as the face deformation or inclination between the two face images increases.

  The present invention has been made in view of such a situation, and makes it possible to identify stably regardless of the deformation or inclination of the face.

A first image processing apparatus according to an aspect of the present invention includes a feature point detection unit that detects a feature point of an image of input image data, and a feature amount detection that detects a local feature amount of the feature point using a Gabor filter. Detecting a feature point of the face image based on the corrected image data, a correction means for correcting the position of the local feature quantity detected by the feature quantity detection means, a feature quantity of the feature points detected in advance and calculating means for the correlative similarity between the feature quantity of the registration image being calculates all, the class of the inputted image data of the image to the degree of similarity And a determination means for determining based on the support vector machine.

When the image of the input image data is determined to be a class that does not belong to a pre-registered image, the image data further includes a registration unit that registers the feature amount according to a user instruction. Can do.

A first image processing method according to one aspect of the present invention includes a feature point detecting step for detecting an image feature point of input image data, and a feature amount detection for detecting a local feature amount of the feature point by a Gabor filter. Detecting a feature point of the face image based on the corrected image data, a correction step of correcting the position of the local feature amount detected in the feature amount detection step so that the face can be identified and the feature quantity of the feature points detected, a calculating step of calculating all correlation similarity degree between the feature quantity of the image registered in advance, the similar class of images of the image data input A determination step based on the degree by the support vector machine.

A first recording medium program according to one aspect of the present invention includes a feature point detecting step for detecting an image feature point of input image data, and a feature amount for detecting a local feature amount of the feature point by a Gabor filter. A detection step; a correction step for correcting the position of the local feature amount detected by the processing of the feature amount detection step so that the face can be identified; and feature points of the face image based on the corrected image data. A step of detecting and calculating all the correlation similarities between the feature quantity of the detected feature point and the feature quantity of the pre-registered image; and the class of the image of the input image data And a determination step of determining by the support vector machine based on the similarity.

A first program according to one aspect of the present invention includes a feature point detection step of detecting an image feature point of input image data, and a feature amount detection step of detecting a local feature amount of the feature point by a Gabor filter. A correction step of correcting the position of the local feature amount detected in the processing of the feature amount detection step so that a face can be identified; and a feature point of the face image based on the corrected image data; a feature quantity of the feature points detected in advance and calculating step correlative similarity between the feature quantity of the registration image being calculates all, the class of the inputted image data of the image to the degree of similarity And a determination step of determining by the support vector machine based on the computer.

The second image processing apparatus according to one aspect of the present invention acquires first image data for learning belonging to a pair of intra personal classes and second image data for learning belonging to a pair of extra personal classes. Obtaining means, labeling means for associating a label with the first image data and the second image data, an image based on the first image data and the second image data to which the label is attached. Detection means for detecting an image portion of a face and detecting a face feature; correction means for correcting the face feature to a position where face identification can be performed; and the corrected first image data and second image a feature point detection means for detecting a feature point of the face image based on the data, the feature quantity detecting means for detecting the Gabor filter the feature quantity of the feature point, the first image data to be paired Comprising a similarity score, a similarity calculating means for calculating a similarity of said second image data to be paired, based on the label and the degree of similarity, and a support vector calculating means for calculating a support vector.

According to a second image processing method of one aspect of the present invention, first image data for learning belonging to a pair of intra personal classes and second image data for learning belonging to a pair of extra personal classes are acquired. An obtaining step, a labeling step for associating a label with the first image data and the second image data, and an image based on the first image data and the second image data to which the label is attached. A detecting step for detecting an image portion of the face and detecting a facial feature; a correcting step for correcting the facial feature to a position where the facial feature can be identified; and the corrected first image data and second image a feature point detection step of detecting a feature point of the face image based on the data, and the feature quantity detecting step of detecting by Gabor feature amount of the feature point pairs with I A similarity calculation step for calculating the similarity of the first image data and the similarity of the second image data to be paired; and a support vector calculation for calculating a support vector based on the label and the similarity Steps.

The second recording medium program according to one aspect of the present invention includes a first image data for learning belonging to a pair of intra personal classes and a second image data for learning belonging to a pair of extra personal classes. An obtaining step for obtaining, a labeling step for associating a label with the first image data and the second image data, an image based on the first image data and the second image data to which the label is attached A detection step of detecting a face image portion and detecting a face feature; a correction step of correcting the face feature to a position where the face can be identified; the corrected first image data and second a feature point detection step of detecting a feature point of the face image based on the image data, the feature quantity detecting step of detecting by Gabor feature amount of the feature point Based on the similarity calculation step of calculating the similarity of the first image data to be paired and the similarity of the second image data to be paired, and calculating the support vector based on the label and the similarity. And a support vector calculation step.

The second program according to one aspect of the present invention acquires the first image data for learning belonging to the paired intra personal class and the second image data for learning belonging to the paired extra personal class. A labeling step for associating a label with the first image data and the second image data, and an image of a face from an image based on the first image data and the second image data to which the label is attached. A detecting step for detecting an image portion and detecting a facial feature; a correcting step for correcting the facial feature to a position where the facial feature can be identified; and the corrected first image data and second image data. a feature point detection step of detecting a feature point of the based facial image, a feature quantity detecting step of detecting by Gabor feature amount of the feature point, a pair The similarity calculation step for calculating the similarity of the first image data and the similarity of the second image data to be paired, and the support vector calculation for calculating a support vector based on the label and the similarity Causing the computer to execute the steps.

  In the present invention, the similarity between the feature amount of the feature point of the input image data and the feature amount of the feature point of the image data registered in advance is calculated. Based on the similarity, the support vector machine A class of image data is determined.

  In the present invention, the support vector is based on the labels associated with the first image data and the second image data, and the similarity between the first image data and the similarity between the second image data. Is calculated.

  According to the present invention, the class of image data can be determined. In particular, it is possible to make a stable determination regardless of the deformation or inclination of the image. It is also possible to register images in real time.

  Furthermore, according to the present invention, an apparatus capable of determining the class of image data can be provided. In particular, it is possible to provide a device that can make a stable determination regardless of the deformation or inclination of an image and can register an image in real time.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 shows a configuration example of an image processing apparatus to which the present invention is applied. The image processing apparatus 1 includes a face extraction unit 11 that extracts facial features such as eyes, mouth, and nose from an input image data image by Gabor filtering, and the eyes, mouth, and nose extracted by the face extraction unit 11. The face recognition unit 12 recognizes whether the image is a human face based on the above.

  The face recognition unit 12 includes a face image alignment unit 51, a class determination unit 52, and a learning database 53. The face image alignment unit 51 is based on the feature positions of features such as the eyes, mouth, and nose of the face supplied from the face extraction unit 11 so that they are normally positioned as human faces. The morphing process is performed. The class determination unit 52 refers to the learning data registered in the learning database 53 and determines whether or not the input image belongs to the same person class as the registered face.

  If the face extraction unit 11 determines that the input image is a face image and is further an eye image, a mouth image, or a nose image, The position in the face is detected, and the position information is supplied to the face image alignment unit 51 of the face recognition unit 12 together with the image data.

  The face image alignment unit 51 performs morphing processing such as affine transformation so that the face feature position comes to the reference position. Thus, for example, in the case of a face image seen from the front, the left and right eyes are positioned at a predetermined distance from the horizontal position, the nose is positioned below the middle of the left and right eyes, and further below the nose Morphing processing is performed so that the mouth is located at the top. As a result, the face image can be easily identified.

  The class determination unit 52 determines whether the image data of the face image aligned by the face image alignment unit 51 matches the image data of the face image registered in advance in the learning database 53. For this reason, the class determination unit 52 has a functional configuration as shown in FIG.

  That is, the class determination unit 52 includes a feature point detection unit 101, a feature amount calculation unit 102, a reading unit 103, a similarity vector calculation unit 104, a class determination unit 105, a duplication determination unit 106, a selection unit 107, an output unit 108, a display Unit 109, instruction determination unit 110, and registration unit 111.

  The feature point detection unit 101 detects a feature point of the face image based on the image data input from the face image alignment unit 51. The feature amount calculation unit 102 calculates the feature amount of the feature point detected by the feature point detection unit 101 and supplies it to the similarity vector calculation unit 104. The feature amount calculation unit 102 is configured by a Gabor filter.

  Here, the Gabor filter processing (Gabor filtering) will be described. First, it is already known that human visual cells have cells that are selective for a specific orientation. It consists of cells that respond to vertical lines and cells that respond to horizontal lines. Similarly, Gabor filtering is a spatial filter composed of a plurality of filters having orientation selectivity.

  A Gabor filter is spatially expressed by a Gabor function. The Gabor function g (x, y) is composed of a carrier s (x, y) composed of a cosine component and an envelope Wr (x, y) in a two-dimensional Gaussian analysis as shown in the following equation.

The carrier s (x, y) is expressed as the following equation (2) using a plurality of functions. Here, the coordinate value (u ₀ , v ₀ ) represents the spatial frequency, and P represents the phase of the cosine component.

  The carrier shown in the equation (2) can be separated into a real component Re (s (x, y)) and an imaginary component Im (s (x, y)) as shown in the following equation (3).

  On the other hand, an envelope composed of a two-dimensional Gaussian distribution is expressed as follows.

Here, the coordinate axes (x ₀ , y ₀ ) are the peaks of this function, and the constants a and b are the scale parameters of the Gaussian distribution. The subscript r means a rotation operation as shown in the following equation.

  Therefore, the Gabor filter is expressed as a spatial function as shown in the following equation from the above equations (2) and (4).

  The feature amount calculation unit 102 according to the present embodiment employs eight types of directions and three frequencies, and performs extraction processing of face eyes, mouths, noses, and the like using a total of 24 Gabor filters.

The response of the Gabor filter is expressed by the following equation, where G _i is the i-th Gabor filter, J _i is the result (Gabor Jet) of the i-th Gabor filter, and I is the input image.

  The calculation of equation (7) can actually be speeded up using fast Fourier transform.

  The performance of the created Gabor filter is examined by reconstructing the pixels obtained by filtering. The reconstructed image H is expressed as follows:

  An error E between the input image I and the reconstructed image H is expressed by the following equation.

  It can be reconstructed by obtaining an optimum a that minimizes the error E.

  As for Gabor filtering, the type of filter may be changed according to the recognition task.

  In low frequency filtering, it is redundant to have all the filtered images as vectors. Therefore, downsampling may be performed to reduce the vector dimension. The 24 types of downsampled vectors are arranged in a line to make a long vector (the pattern vector described above).

  The reading unit 103 reads the feature points of the registered image corresponding to the feature points detected by the feature point detection unit 101 from the registration unit 111 and outputs them to the similarity vector calculation unit 104.

  The similarity vector calculation unit 104 is registered with the input face image based on the feature amount of the input image calculated by the feature amount calculation unit 102 and the feature amount of the registered image read by the reading unit 103. The similarity vector of the face image is calculated.

  The class determination unit 105 includes a support vector machine, and based on the similarity vector supplied from the similarity vector calculation unit 104, whether the input image belongs to an intra-personal class (intra personal class) (pre-registered) Whether it belongs to an extra-personal class (extra personal class) (whether it is an image other than a pre-registered image (does not match a registered image)). When it is determined that the image input by the class determination unit 105 belongs to the intra-personal class, the overlap determination unit 106 determines whether the class determination unit 105 has performed the overlap determination. That is, it is determined whether or not it is determined that the input image matches a plurality of registered images. When the overlap determination is performed by the overlap determination unit 106, the selection unit 107 performs a process of selecting any one of registered images determined to match. When the overlap determination unit 106 determines that the overlap determination has not been performed, the selection unit 107 selects the class image determined by the class determination unit 105 as it is. The output unit 108 outputs the image selected by the selection unit 107 to a subsequent apparatus or causes the display unit 109 to display the image.

  The display unit 109 displays a message when it is determined that the image input by the class determination unit 105 belongs to the extra-personal class instead of the intra-personal class. The instruction determination unit 110 determines whether or not the user has instructed registration of the input image based on the message displayed by the display unit 109. When the registration unit 111 determines that the registration of the image input by the instruction determination unit 110 has been instructed, the registration unit 111 calculates the feature amount calculated by the feature amount calculation unit 102 together with the input face image data as necessary. sign up.

  In the class determination unit 105, a support vector machine (SVM) that is considered to have the highest learning generalization ability in the field of pattern recognition is used to identify whether or not the face is applicable. Will be described here.

  Regarding the support vector machine itself, for example, a report by B. sholkopf (B. Sholkopf, C. Burges, A. Smola, “Advance in Kernel Support Vector Learning”, The MIT Press, 1999.) can be cited. From the results of preliminary experiments conducted by the present applicant, it has been found that the face recognition method using the support vector machine gives better results than the method using principal component analysis (PCA) or neural network.

  The support vector machine is a learning machine using a linear classifier (perceptron) as a discriminant function, and can be extended to a non-linear space by using a kernel function. In the learning of discriminant functions, the margin of separation between classes is maximized, and the solution can be obtained by solving quadratic mathematical programming, so that it is theoretically guaranteed that a global solution can be reached. it can.

  Usually, the problem of pattern recognition is to obtain the discriminant function f (x) given by the following equation for the test sample x = (x1, x2,..., Xn).

  Here, the teacher label for learning of the support vector machine is set as follows.

  Then, the recognition of the face pattern in the support vector machine can be regarded as a problem of minimizing the square of the weight factor w under the constraint condition shown in the following equation.

  Problems with such constraints can be solved using Lagrange's undetermined constant method. That is, the Lagrange shown in the equation (13) is first introduced, and then, as shown in the equation (14), partial differentiation is performed for each of b and w.

  As a result, the identification of the face pattern in the support vector machine can be regarded as a quadratic programming problem expressed by the following equation.

  When the number of dimensions of the feature space is smaller than the number of training samples, a scratch variable ξ ≧ 0 is introduced, and the constraint condition is changed as follows.

  For optimization, the objective function of the following equation is minimized.

  In this equation (17), C is a coefficient that specifies how far the constraint condition is relaxed, and it is necessary to determine the value experimentally.

  The problem concerning the Lagrangian constant a is changed as follows:

  However, with this equation (18), the nonlinear problem cannot be solved. Therefore, in this embodiment, a kernel function K (x, x3) is introduced, and once mapped onto a high-dimensional space (kernel trick), linear separation is performed in that space. Therefore, it is equivalent to non-linear separation in the original space.

  The kernel function is expressed as follows using a certain mapping Φ.

  Further, the discriminant function shown in the equation (10) can also be expressed as the following equation.

  Learning can also be considered as a quadratic programming problem expressed by the following equation.

  As the kernel, a Gaussian kernel (RBF: Radius Basic Function) represented by the following equation can be used.

  Next, the recognition process of the image processing apparatus 1 will be described with reference to the flowchart of FIG. In step S31, the face extraction unit 11 extracts a face from the input image data. Specifically, facial features such as facial eyes, mouth, and nose are extracted. In step S32, the face image alignment unit 51 aligns the eyes, mouth, and nose. Specifically, the positions of the eyes, mouth, nose, etc. are adjusted so that they are positioned in a common sense positional relationship.

  In step S <b> 33, the feature point detection unit 101 detects a facial feature point of the face image data input from the face image alignment unit 51. Which point is used as a feature point is arbitrary. For example, as shown in FIG. 4, pixels located at a predetermined interval in the horizontal direction and the vertical direction (indicated by x in FIG. 4). Pixel) can be used as a feature point. In FIG. 4, the left image represents a face input image based on the input face image data, and the right image represents a registered face image based on the registered face image data.

In step S34, processing for obtaining the feature amount of the feature point is performed by a Gabor filter. That is, the feature amount calculation unit 102 calculates feature points at the feature position x _Vi (i = 1,..., M) of the input image (hereinafter, this feature point is also referred to as the feature point x _Vi if necessary). A feature quantity J _j (x _Vi ) that is a local feature quantity is calculated based on the following equation. In the following formulas, the subscript V is in x _V, k _V, indicating that x, k is a vector. X ′ _V represents a differential value of x _V.

In the above equation, I (x _Vi ) represents the pixel value (gray value) of the input face image at the feature position x _{Vi and} is convolved and integrated by Ψ _j (x _V ). Ψ _j (x _V ) is called a Gabor Kernel and constitutes a Gabor filter that extracts a local feature quantity in a specific direction and frequency indicated by k _Vj determined from the frequency parameter ν and the direction parameter μ. The output values at various frequencies and directions of the Gabor filter are collectively referred to as Gabor Jet J _j (x _Vj ), and are used as feature quantities at the feature position x _Vi . The Gabor jet represents a local feature amount around the feature point, and has a feature that it is invariant to a certain amount of positional shift or deformation of the feature point.

In step S35, the reading unit 103 reads the feature amount of the registered face. That is, the reading unit 103 reads the feature amount J _j ^M (x _Vi ) of the feature point of the registered face corresponding to the feature point x _Vi detected by the feature point detection unit 101 from the registration unit 111, and calculates the similarity vector calculation Output to the unit 104.

In step S36, the similarity vector calculation unit 104 calculates a similarity vector. Specifically, based on the following equation, the similarity d (x _Vi ) at the same feature position x _Vi is obtained by _calculating the input face feature quantity J _i ^I (x _Vi ) and the registered image feature quantity J _j ^M (x _Vi ). From this similarity d (x _Vi ), a similarity vector d {d (x _Vi )} (i = 1,..., M), which is the set, is obtained.

  The processes in steps S35 and S36 described above are performed for all registered faces registered in advance. As a result, the input face image is compared with all registered face images.

  In step S37, the class determination unit 105 determines a class using a support vector machine (SVM). The support vector machine is a distance from the boundary surface of the similarity vector (the boundary surface (the surface at the position where the value is 0) that determines the value of f (x) in Expression (10) as, for example, +1 or −1)). To determine whether it belongs to intra-personal class or extra-personal class. If there is no similarity vector determined to belong to the intra-personal class, it is determined that an unregistered person's face has been input. One support vector machine that constitutes the class determination unit 105 learns many face images, so that a newly input face image matches a registered (learned) face image (intra- It has a function to determine whether it belongs to personal class) or does not match (whether it belongs to extra-personal class).

The process in step S37 is schematically shown in FIG. That is, as shown in FIG. 4, the support vector machine performs the feature amount J ^I (x _Vj ) of the feature points of the input image I ^I (x _V ) detected by the Gabor filter and the registered image I ^Mk (x _V ). The class d is determined by calculating the similarity d (x _Vi ) of the feature quantity J ^M (x _Vi ) of the feature points and determining the similarity vector that is a set of the similarities.

  In step S38, the class determination unit 105 determines whether the input face image is an intra-personal class. When it is determined that the input face image is an intra-personal class (when it is determined that the input face image is an image registered in the registration unit 111), in step S39, the overlap determination unit 106 Determines whether the intra-personal class of a plurality of registered faces has been determined. That is, it is determined whether or not the class determination unit 105 determines that the input face image matches two or more registered faces registered in the registration unit 111. If the intra-personal class of a plurality of registered faces is determined, in step S40, the selection unit 107 selects a registered face corresponding to the similarity vector having the largest distance from the boundary surface.

  In step S39, when it is determined that the intra-personal class of a plurality of registered faces has not been determined (when it is determined that the input face image matches one registered face), Since the selection process in step S40 is not necessary, it is skipped. In step S41, the output unit 108 outputs the face image data registered in the registration unit 111 or displays the face image data on the display unit 109 or the like.

  If it is determined in step S38 that the input face image is not an intra-personal class (when it is determined to be an extra-personal class), the display unit 109 displays a message in step S42. For example, a message such as “The input person is not registered” is displayed on the display unit 109. The administrator sees this message, determines whether or not to register the input face image, and instructs to that effect when registering. At this time, if necessary, for example, a password is presented as information indicating that the administrator is an administrator having registration authority. Therefore, the instruction determination unit 110 determines whether or not registration is instructed in step S43, and when it is determined that registration of the input face image is instructed, in step S44, the instruction determination unit 110 further Then, it is determined whether or not the instruction is an instruction from the administrator. If the input password is the correct password, it is determined that the instruction is from the administrator. In this case, in step S <b> 45, the registration unit 111 executes a process of registering the feature amount calculated by the feature amount calculation unit 102. Thereafter, the process proceeds to step S41, and the output unit 108 executes a process of outputting the face image data currently registered in the registration unit 111.

  If it is determined in step S43 that registration is not instructed, or if it is determined in step S44 that registration is instructed but not an instruction from the administrator (for example, if the input password is incorrect) The registration process in step S45 is skipped and the process ends.

  The support vector machine used in the class determination unit 105 is learned and generated by, for example, the image processing apparatus shown in FIG.

  The image processing apparatus 151 includes an acquisition unit 161, a labeling unit 162, a face extraction unit 163, a face image alignment unit 164, a feature point detection unit 165, a feature amount calculation unit 166, a similarity vector calculation unit 167, and a support vector calculation. Part 168.

  The acquisition unit 161 acquires learning image data. The learning image data includes learning image data belonging to a pair of intra-personal classes and learning image data belonging to a pair of extra-personal classes. Here, a pair of images is an image of a human face, and is an image of a pair of an image before changing the lighting condition, facial expression, race, age, sex, etc. and an image after changing the image. means. In the case of face images belonging to the intra-personal class, the paired images are images of the same person, so it is possible to change the lighting conditions, facial expressions, age, etc., but the race, gender, etc. It cannot be changed. Therefore, a face image belonging to the intra-personal class is prepared in which conditions such as lighting conditions, facial expressions, or age are changed. On the other hand, since the face images belonging to the extra-personal class are face images determined to be different persons, the paired face images are face images of different persons. In this case, with respect to one face image, the other face image can be a face image in which conditions such as lighting conditions, facial expressions, race, age, and sex are changed.

  The labeling unit 162 labels the face images belonging to the intra-personal class acquired by the acquisition unit 161 and the face images belonging to the extra-personal class. For example, +1 is labeled for a face image belonging to the intra-personal class, and -1 is labeled for a face image belonging to the extra-personal class. The face extraction unit 163 detects a face image portion from the labeled images, and further detects features such as eyes, mouth, and nose from the face image. The configuration and function of the face extraction unit 163 are the same as those of the face extraction unit 11 in FIG.

  The face image alignment unit 164 performs processing for aligning the positions of features such as eyes, mouths, and noses of the image detected by the face extraction unit 163 with reference positions. The configuration and function of the face image alignment unit 164 are the same as those of the face image alignment unit 51 in FIG.

  The feature point detection unit 165 detects feature points of the image in which features such as eyes, mouth, and nose are aligned by the face image alignment unit 164. It is arbitrary what point is used as the feature point. The feature amount calculation unit 166 calculates the feature amount of the feature point detected by the feature point detection unit 165. The feature amount calculated by the feature amount calculation unit 166 is also arbitrary. The configuration and function of the feature point detection unit 165 and the feature amount calculation unit 166 are the same as those of the feature point detection unit 101 and the feature amount calculation unit 102 in FIG. 2, and the detected feature points correspond to the feature amounts.

  The similarity vector calculation unit 167 calculates a similarity vector of a pair of feature amounts calculated by the feature amount calculation unit 166. The configuration and function of the similarity vector calculation unit 167 are the same as those of the similarity vector calculation unit 104 in FIG. The support vector calculation unit 168 has two classes (intra-personal class and extra-personal class) based on the similarity vector calculated by the similarity vector calculation unit 167 and the label added by the labeling unit 162. Calculate the support vector that separates.

  Next, the learning process of the image processing apparatus 151 in FIG. 5 will be described with reference to the flowchart in FIG.

  In step S61, the acquisition unit 161 acquires learning image data belonging to the paired intra-personal class and learning image data belonging to the paired extra-personal class. This learning image data does not necessarily need to be image data of a person's face to be identified (registered). This is because the registration is performed in step S45 in FIG. Therefore, the support vector machine used in the class determination unit 105 of the image processing apparatus 1 can be shared regardless of the place where the image processing apparatus 1 is actually used, the user, and the like. Can be easily generated.

  In step S62, the labeling unit 162 performs labeling. Specifically, the labeling unit 162 receives +1 for image data input as learning image data belonging to the intra-personal class, and is input as image data for learning belonging to the extra-personal class. For image data, -1 is labeled. In step S63, the face extraction unit 163 detects the feature of the face image. In step S64, the face image alignment unit 164 executes a process of aligning the facial features detected in the process of step S63.

  That is, by the processing in steps S63 and S65, the face image portion is detected from the input image data as in the case of the face extraction unit 11 of FIG. 1 and the face image alignment unit 51 of the face recognition unit 12. Then, facial features such as eyes, mouth, and nose are further detected from the detected faces, and the detected features such as eyes, mouth, and nose are aligned to a predetermined reference position.

  In step S65, the feature point detector 165 detects feature points of the face image data. In step S66, the feature amount calculation unit 166 obtains feature amounts of feature points using a Gabor filter. These processes are the same as the processes executed in step S33 and step S34 in FIG. 3 by the feature point detection unit 101 and the feature amount calculation unit 102 in FIG. That is, the feature amount of the feature point of the image data belonging to the intra-personal class that is a pair for learning is obtained and the feature amount of the feature point of the image data belonging to the extra-personal class that is a pair for learning. Is required.

In step S67, the similarity vector calculation unit 167 calculates a similarity vector. Specifically, instead of the feature value J _j ^I (x _Vi ) of the input face in the above equation (29), one feature value of the face image belonging to the paired intra-personal class is substituted, and similarly , J _j ^M (x _Vi ) is substituted with the feature quantity of the other pair of image data for learning belonging to the intra-personal class. Then, the similarity between them is calculated based on the equation (29). Furthermore, the similarity d similarity vector d {d (x _Vi)} as a collection of _{(x Vi) (i = 1} , ···, M) is calculated. Similarly, a similarity vector is also calculated for learning image data belonging to a pair of extra-personal classes.

  In step S68, the support vector calculation unit 168 inputs the label and the similarity vector to the support vector machine, and calculates a support vector that separates the two classes. Specifically, the similarity vector calculated in step S67 is substituted for x in the above-described equation (10), and the label attached in the process of step S62 is substituted for y shown in equation (11). Is done. Thereby, f (x) in Expression (10) as a support vector is calculated.

  The support vector machine generated as described above is used as the support vector machine of the class determination unit 105 in FIG.

  As described above, in the present invention, only one support vector machine for identifying the intra-personal class and the extra-personal class is required. Since the teacher data necessary for learning the support vector machine does not need to include the registered face image, there is no need to re-learn the support vector machine. At the time of registration, since it is only necessary to store the feature amount generated from the face image for registration, face image registration processing can be performed in real time, and the amount of data to be stored hardly increases. In addition, since a support vector machine is used for the class determination unit, the generalization capability is very high, and high-precision identification is possible even when various conditions change.

  In addition, since the similarity vector of Gaboradgets of two face images is used as data to be input to the support vector machine instead of a difference image of two face images, a certain amount of positional deviation or deformation between feature points is used. Become stronger. As a result, the face is robust against changes in face deformation and tilt between the two face images, and the face recognition performance of the robot under various environments can be improved. Further, the present invention can cope with not only face identification but also two-class identification of various patterns (identification of coincidence or non-coincidence), and the deformation or inclination of the pattern to be identified. It is possible to construct a robust identification system.

  Accordingly, face registration and additional learning can be performed in real time, and a face identification system that is robust against changes in face deformation and tilt under various environments can be constructed.

  The series of processes described above can be executed by hardware or can be executed by software. In this case, for example, the image processing apparatus is configured by a personal computer as shown in FIG.

  In FIG. 7, a CPU (Central Processing Unit) 221 executes various processes according to a program stored in a ROM (Read Only Memory) 222 or a program loaded from a storage unit 228 to a RAM (Random Access Memory) 223. To do. The RAM 223 also appropriately stores data necessary for the CPU 221 to execute various processes.

  The CPU 221, ROM 222, and RAM 223 are connected to each other via a bus 224. An input / output interface 225 is also connected to the bus 224.

  The input / output interface 225 includes an input unit 226 including a keyboard and a mouse, a display including a CRT (Cathode Ray Tube) and an LCD (Liquid Crystal display), an output unit 227 including a speaker, a hard disk, and the like. A communication unit 229 including a storage unit 228 and a modem is connected. The communication unit 229 performs communication processing via a network including the Internet.

  A drive 230 is connected to the input / output interface 225 as necessary, and a removable medium 231 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory is appropriately mounted, and a computer program read from them is It is installed in the storage unit 228 as necessary.

  When a series of processing is executed by software, a program constituting the software executes various functions by installing a computer incorporated in dedicated hardware or various programs. For example, a general-purpose personal computer is installed from a network or a recording medium.

  As shown in FIG. 7, the recording medium is distributed to provide a program to the user separately from the apparatus main body, and includes a magnetic disk (including a floppy disk) on which the program is recorded, an optical disk (CD- It is not only composed of a removable medium 231 consisting of ROM (compact disk-read only memory), DVD (digital versatile disk), magneto-optical disk (including MD (mini-disk)), or semiconductor memory. The program is configured by a ROM 222 on which a program is recorded, a hard disk included in the storage unit 228, and the like provided to the user in a state of being preinstalled in the apparatus main body.

  In the present specification, the step of describing the program recorded on the recording medium is not limited to the processing performed in chronological order according to the described order, but is not necessarily performed in chronological order. It also includes processes that are executed individually.

  Further, in this specification, the system represents the entire apparatus constituted by a plurality of apparatuses.

  The present invention can be applied to a robot for identifying a human face.

It is a block diagram which shows the structural example of the image processing apparatus to which this invention is applied. It is a block diagram which shows the functional structural example of the class determination part of FIG. It is a flowchart explaining the class determination process of the class determination part of FIG. It is a figure explaining a class determination process. It is a block diagram which shows the functional structural example of the image processing apparatus which learns a support vector machine. It is a flowchart explaining the learning process of the image processing apparatus of FIG. And FIG. 16 is a block diagram illustrating a configuration example of a personal computer.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 Image processing apparatus, 11 Face extraction part, 12 Face recognition part, 51 Face image alignment part, 52 Class judgment part, 101 Feature point detection part, 102 Feature-value calculation part, 103 Reading part, 104 Similarity vector calculation part, 105 Class determination unit, 106 overlap determination unit, 107 selection unit, 108 output unit, 109 display unit, 110 instruction determination unit, 111 registration unit, 151 image processing device, 161 acquisition unit, 162 labeling unit, 163 face extraction unit, 164 Face image alignment unit, 165 feature point detection unit, 166 feature amount calculation unit, 167 similarity vector calculation unit, 168 support vector calculation unit

Claims (9)

Feature point detecting means for detecting feature points of the image of the input image data;
Feature quantity detection means for detecting a local feature quantity of the feature point by a Gabor filter;
Correction means for correcting the position so that the local feature quantity detected by the feature quantity detection means can be identified; and
Calculation means for detecting feature points of the face image based on the corrected image data, and calculating all correlation similarities between the feature quantities of the detected feature points and the feature quantities of the pre-registered image When,
Images processing device and a determination means by the support vector machine based on the class of the inputted image data of the image to the degree of similarity. When it is determined that the image of the input image data is a class that does not belong to a previously registered image, the image data further includes a registration unit that registers the feature amount according to a user instruction.
The image processing apparatus according to 請 Motomeko 1. A feature point detecting step for detecting a feature point of the image of the input image data;
A feature amount detection step of detecting a local feature amount of the feature point by a Gabor filter;
A correction step for correcting the position so that the local feature amount detected by the processing of the feature amount detection step can be identified; and
Calculating step the detected feature points of the modified facial image based on the image data, and the feature quantity of the feature points detected, it calculates all correlations similarity degree between the feature quantity of image registered beforehand When,
Images processing method comprising a determining step by the support vector machine based on the class of the inputted image data of the image to the degree of similarity. A feature point detecting step for detecting a feature point of the image of the input image data;
A feature amount detection step of detecting a local feature amount of the feature point by a Gabor filter;
A correction step for correcting the position so that the local feature amount detected by the processing of the feature amount detection step can be identified; and
A calculation step of detecting feature points of the face image based on the corrected image data, and calculating all correlation similarities between the feature amount of the detected feature points and the feature amount of the image registered in advance. When,
Recording medium input computer-readable program including a determining step by the support vector machine based on the class of the image of the image data to the similarity is recorded. A feature point detecting step for detecting a feature point of the image of the input image data;
A feature amount detection step of detecting a local feature amount of the feature point by a Gabor filter;
A correction step for correcting the position so that the local feature amount detected by the processing of the feature amount detection step can be identified; and
A calculation step of detecting feature points of the face image based on the corrected image data, and calculating all correlation similarities between the feature amount of the detected feature points and the feature amount of the image registered in advance. When,
Programs for executing the class of the inputted image data of the image and determining steps to the computer via the support vector machine based on the similarity. Acquisition means for acquiring first image data for learning belonging to a pair of intra personal classes, and second image data for learning belonging to a pair of extra personal classes;
Labeling means for associating a label with the first image data and the second image data;
Detecting means for detecting a facial feature from an image based on the first image data and the second image data with the label, and detecting a facial feature;
Correction means for correcting the face feature to a position where the face can be identified;
Feature point detecting means for detecting feature points of a face image based on the corrected first image data and second image data ;
Feature quantity detection means for detecting a feature quantity of the feature point by a Gabor filter;
Similarity calculating means for calculating the similarity of the first image data to be paired and the similarity of the second image data to be paired;
Based on the label and the degree of similarity, images processing device and a support vector calculating means for calculating a support vector. An acquisition step of acquiring first image data for learning belonging to a pair of intra personal classes, and second image data for learning belonging to a pair of extra personal classes;
A labeling step of associating a label with the first image data and the second image data;
A detection step of detecting a facial image portion from an image based on the first image data and the second image data with the label, and detecting a facial feature;
A correction step of correcting the face feature to a position where the face can be identified;
A feature point detecting step of detecting a feature point of a face image based on the corrected first image data and second image data ;
A feature amount detecting step of detecting a feature amount of the feature point by a Gabor filter;
A similarity calculation step of calculating the similarity of the first image data to be paired and the similarity of the second image data to be paired;
Based on the label and the degree of similarity, images processing method comprising a support vector calculating step of calculating a support vector. An acquisition step of acquiring first image data for learning belonging to a pair of intra personal classes, and second image data for learning belonging to a pair of extra personal classes;
A labeling step of associating a label with the first image data and the second image data;
A detection step of detecting a facial image portion from an image based on the first image data and the second image data with the label, and detecting a facial feature;
A correction step of correcting the face feature to a position where the face can be identified;
A feature point detecting step of detecting a feature point of a face image based on the corrected first image data and second image data ;
A feature amount detecting step of detecting a feature amount of the feature point by a Gabor filter;
A similarity calculation step of calculating the similarity of the first image data to be paired and the similarity of the second image data to be paired;
The label and on the basis of the similarity, a recording medium on which computer-readable program including a support vector calculating step of calculating a support vector is recorded. An acquisition step of acquiring first image data for learning belonging to a pair of intra personal classes, and second image data for learning belonging to a pair of extra personal classes;
A labeling step of associating a label with the first image data and the second image data;
A detection step of detecting a facial image portion from an image based on the first image data and the second image data with the label, and detecting a facial feature;
A correction step of correcting the face feature to a position where the face can be identified;
A feature point detecting step of detecting a feature point of a face image based on the corrected first image data and second image data ;
A feature amount detecting step of detecting a feature amount of the feature point by a Gabor filter;
A similarity calculation step of calculating the similarity of the first image data to be paired and the similarity of the second image data to be paired;
Programs to be executed based on the label and the degree of similarity, and a support vector calculating step of calculating a support vector to the computer.

Images