JP2004054442A - Face detecting device, face detecting method, and face detecting program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To perform highly reliable and quick face detection by quickly and surely absorbing the fluctuation of the position relation of face parts. <P>SOLUTION: A face detecting device 1 is provided with an image pickup part 2 and a storage part 4 and a detecting part 3. The image pickup part 2 inputs an arbitrary image. The storage part 4 stores the preliminary information of each face parts including the existence probability of the reference points of a face with each face parts as a reference and the partial space of each face parts. The detecting part 3 recognizes the face parts existing in the image based on the partial space, and calculates the reference points of the face in the image based on each face parts, and detects the face image from the image based on the reference points of the face. <P>COPYRIGHT: (C)2004,JPO

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to a face detection device, a face detection method, and a face detection program for detecting a face image in an input image, and more particularly, to a method for quickly and surely absorbing a variation in the positional relationship between face parts, and thereby achieving a highly reliable and quick process. The present invention relates to a face detection device capable of performing face detection, a face detection method, and a face detection program.
[0002]
[Prior art]
In a surveillance camera system, it is necessary to automatically recognize whether or not a person is included in an image captured by the surveillance camera. Therefore, a technique using a face detection technique called a subspace method is known.
[0003]
For example, Matthew A. See Turk, Alex Pentland, "Face Recognition Using Eigenfaces", Proc. CVPR, pp. 586-591, 1991 discloses a technique for collecting data of a large number of face images to form a partial space. In this prior art, a low-dimensional partial space representing the entire face is prepared and input is performed. The face position is detected by checking at which position in the image the partial image best fits the prepared partial space.
[0004]
Specifically, in a model expressing a pattern distribution assuming a normal distribution using a mean and a covariance matrix as parameters obtained from a face image data set prepared for learning, an eigenvector associated with a higher eigenvalue of the covariance matrix is used. The pattern is limited to a subspace to be set, and the distance between the input pattern and the subspace is measured by regarding the subspace as a face space.
[0005]
By the way, when performing face detection using such a subspace method, it is difficult to absorb this variation because the position of face parts (eyes, nose, mouth, etc.) varies greatly between individuals. In other words, in face detection, it is necessary to normalize the size of the face image. For example, if the criterion is “the distance between both eyes”, the distance from the midpoint of these eyes to other face parts (eg, “mouth”) The distance differs greatly between a long face and a round face.
[0006]
For this reason, Japanese Patent Application Laid-Open No. 9-91429 (Prior Art 1) discloses an invention in which matching with a subspace is performed while changing the aspect ratio. Japanese Patent Application Laid-Open No. 9-44676 (Prior Art 2) discloses an invention in which after detecting a plurality of face parts, the position of the face is specified based on the positional relationship between the face parts. By using the inventions disclosed in the related art 1 and the related art 2, variations in the position of the face parts can be absorbed to some extent.
[0007]
[Problems to be solved by the invention]
However, when the above-described conventional technology 1 is used, the number of matching calculations is proportional to the number of times the aspect ratio is changed, so that the calculation time becomes enormous. As a result, face detection cannot be performed quickly. There's a problem.
[0008]
In addition, when the above-described related art 2 is used, the variation in the position of the face parts can be absorbed to some extent, but the pattern of each face part is simpler than the entire face, so that the reliability of the face part detection is low. As a result, there is a problem that the reliability of face detection is reduced.
[0009]
The present invention has been made in order to solve the above-described problem, and a face detection method capable of quickly and reliably absorbing a variation in the positional relationship between face parts and performing highly reliable and quick face detection. It is an object to provide an apparatus, a face detection method, and a face detection program.
[0010]
[Means for Solving the Problems]
The present invention has been made to achieve the above object, and a face detection apparatus according to claim 1 is a face detection apparatus that detects a face image present in an input image, wherein each face part is referenced. Storage means for storing prior information of each face part including the existence probability of the reference point of the face and the subspace of each face part, and recognizing each face part present in the input image based on the prior information Detecting means for calculating a reference point of the face in the input image based on each recognized face part, and detecting a face image from the input image based on the calculated reference point of the face. And
[0011]
Further, in the face detecting apparatus according to claim 2, in the invention according to claim 1, the detecting means calculates the likeness of each face part as part information using the partial space, and the part information calculating means. Based on the part information calculated by the means and the existence probability of the face reference point based on each face part, which is one of the prior information, the existence probability of the face reference point based on each face part is calculated. The existence probability calculating means for calculating, and the existence probability of the reference point of the face based on each face part calculated by the existence probability calculating means is weighted by the importance of each face part, and each weight obtained from the weighting result is obtained. A sum calculation means for calculating the sum of the existence probabilities of the reference points of the face based on the face parts; anda maximum value of the sum of the existence probabilities calculated by the sum calculation means, Characterized in that a reference point specifying means for specifying as a reference point.
[0012]
Further, in the face detecting device according to claim 3, in the invention according to claim 1 or 2, the existence probability calculating means is one of the part information calculated by the part information calculating means and the prior information. The present invention is characterized in that the existence probability of the reference point of the face based on each face part is calculated by convolution with the existence probability of each face part.
[0013]
A face detection method according to claim 4 is a face detection method for detecting a face image present in an input image, wherein a face reference point existence probability based on each face part and a part of each face part are included. A storing step of storing prior information of each face part including a space in a storage unit; recognizing each face part present in the input image based on the prior information; and performing the input based on the recognized face parts. Calculating a reference point of the face in the image, and detecting a face image from the input image based on the calculated reference point of the face.
[0014]
According to a fifth aspect of the present invention, in the face detection device according to the fourth aspect of the present invention, the detection step includes a part information calculation step of calculating each face part likeness as part information using the partial space; Based on the part information calculated by the process and the existence probability of the reference point of the face based on each face part which is one of the prior information, the existence probability of the reference point of the face based on each face part is calculated. The existence probability calculation step of calculating, and the existence probability of the reference point of the face based on each face part calculated in the existence probability calculation step is weighted by the importance of each face part, and each weight obtained from the weighting result is obtained. A sum calculation step of calculating the sum of the existence probabilities of the reference points of the face based on the face parts; anda maximum value of the sum of the existence probabilities calculated in the sum calculation step, and calculating the maximum value for the face. Characterized in that it includes a reference point specifying step of specifying as a reference point.
[0015]
Also, in the face detection device according to claim 6, in the invention according to claim 4 or 5, the existence probability calculation step is one of the part information calculated in the part information calculation step and the prior information. The present invention is characterized in that the existence probability of the reference point of the face based on each face part is calculated by convolution with the existence probability of each face part.
[0016]
A face detection program according to claim 7, which is a face detection program for detecting a face image present in an input image, comprising: a face reference point existence probability based on each face part; A storage procedure of storing prior information of each face part including a space in a storage unit, and recognizing each face part present in the input image based on the prior information, and performing the input based on the recognized face parts. And (c) calculating a reference point of the face in the image, and detecting the face image from the input image based on the calculated reference point of the face.
[0017]
The face detection program according to claim 8 is the invention according to claim 7, wherein the detection step includes: a part information calculation step of calculating each face part likeness as part information using the subspace; and the part information calculation step. Based on the part information calculated by the procedure and the existence probability of a face reference point based on each face part, which is one of the prior information, the existence probability of a face reference point based on each face part is calculated. The existence probability calculation procedure to be calculated and the existence probability of the reference point of the face based on each face part calculated by the existence probability calculation procedure are weighted by the importance of each face part, and each weight obtained from the weighting result is obtained. A sum calculation procedure for calculating the sum of the existence probabilities of the reference points of the face based on the face parts, and a maximum value of the sum of the existence probabilities calculated by the sum calculation procedure, Characterized in that it includes a reference point specifying step of specifying as the reference point of the face.
[0018]
The face detection program according to claim 9 is the invention according to claim 7 or 8, wherein the existence probability calculation step is one of the part information calculated by the part information calculation step and the prior information. The present invention is characterized in that the existence probability of the reference point of the face based on each face part is calculated by convolution with the existence probability of each face part.
[0019]
BEST MODE FOR CARRYING OUT THE INVENTION
DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Exemplary embodiments of the present invention will be described in detail below with reference to the accompanying drawings.
[0020]
FIG. 1 is a block diagram illustrating a schematic configuration of a face detection device according to an embodiment of the present invention. As shown in FIG. 1, the face detection device 1 recognizes face parts present in an input image based on prior information of each face part stored in an imaging unit 2 and a storage unit 4, and recognizes each of these face parts. A detection unit 3 that calculates a reference point of the face in the input image based on the face parts and detects a face image from the input image based on the reference points of the face; and the presence of a reference point based on each face part A storage unit 4 for storing prior information of each face part including probabilities. The advance information stored in the storage unit 4 will be described later. Here, the face parts refer to face components such as the right eye, the left eye, the nose and the mouth.
[0021]
In the case of this example, the imaging unit 2 employs a conventionally known camera device using a CCD or the like. The detection unit 3 uses a computer device (CPU). Further, the storage unit 4 employs a hard disk attached to the computer device forming the detection unit 3. Note that the storage unit 4 may employ a computer device different from the computer device.
[0022]
As shown in FIG. 1, the detecting unit 3 includes a parts information calculating unit 3a, an existence probability calculating unit 3b, a sum calculating unit 3c, and a reference point specifying unit 3d. The part information calculation unit 3a is a processing unit that obtains the likeness of each face part using the subspace stored in the storage unit 4 for each position of the input image input by the imaging unit 2.
[0023]
Further, the existence probability calculation unit 3b uses the likelihood of each face part obtained by the parts information calculation unit 3a and the existence probability of the reference point of the face based on each face part, which is one of the advance information, respectively. Is a processing unit that calculates the existence probability of the reference point of the face based on the face part. Here, the existence probability of the reference point of the face based on each face part is determined by convolving the likelihood of each face part with the reference position existence probability of the face for each face part stored in the storage unit 4 (convolution). It is to be calculated by:
[0024]
Further, the sum calculation unit 3c first weights the existence probability of the reference point of the face based on each face part obtained by the existence probability calculation unit 3b according to the importance of each face part, and obtains the weight by performing this weighting. The processing unit obtains the sum of the probabilities of existence of the reference points of the face based on the respective face parts.
[0025]
The reference point specifying unit 3d is a processing unit that obtains the maximum value of the sum of the existence probabilities obtained by the sum calculation unit 3c, and uses this maximum value as the reference point of the face.
[0026]
A procedure for detecting a face image in an input image obtained from the imaging unit 2 using the face detection device 1 having the above configuration is as shown in a flowchart in FIG. However, prior to such face detection, the learning process of the flowchart shown in FIG. 2 needs to be performed, and various pieces of advance information obtained by this learning process need to be stored in the storage unit 4. Hereinafter, after the learning process is described, the face detection process will be described.
[0027]
FIG. 2 is a flowchart illustrating a learning processing procedure performed before face detection. As shown in the figure, a large number of face images are input in advance and stored in the storage unit 4 (step S201). Such a face image can be input by capturing a face image for learning using the image capturing unit 2. Further, another input device (not shown) such as a scanner may be used.
[0028]
When this input operation is completed, the process proceeds to step S202, and for each face image input in step S201, the reference point of the face and the position of each face part are designated, and the position of each face part is set as a reference. A face part image having a predetermined size is extracted. The reference point of the face and the position of each face part are appropriately designated by the operator.
[0029]
Next, the process proceeds to step S203, and the relative position of the reference point of the face determined in step S202 with respect to each face part position is obtained. Thereafter, the process proceeds to step S204, and the average and covariance of the plurality of relative positions obtained in step S203 are obtained. Further, based on these averages and covariances, the existence probability Qi (x) of the reference point of the face for each face part is obtained using a normal distribution.
[0030]
Specifically, assuming that the two-dimensional average column vector is m and the inverse matrix of the covariance matrix is V,
Qi (x) = Cexp {-<m, Vm>}
Asking. Note that C is a constant, and <m, Vm> indicates an inner product of m and Vm.
[0031]
Finally, the process proceeds to step S205, and learning is performed on each face part image using the above-described subspace method. That is, an average image and a unique image of each face part are created and stored in the storage unit 4. The average image and unique image of each face part and the existence probability Qi (x) of the reference point of the face for each face part correspond to the prior information described in the claims.
[0032]
After completing the learning as described above, the face image can be detected from the input image captured from the imaging unit 2. Next, the face detection procedure will be described with reference to FIG.
[0033]
As shown in step S301 in FIG. 3, if the input image for the detection target acquired by the imaging unit 2 is input to the detection unit 3, the partial images centered on the positions for all the positions in the input image Is extracted, and the subspace method is applied to each of the face parts i. When the distance between the position x of each face part i and the subspace of each face part i is di (x), The facial part likeness Pi (x) = exp (−di (x) / d0) is calculated. This is performed by the parts information calculation unit 3a of the detection unit 3.
[0034]
Specifically, regarding face part i, if i = 1 is the right eye, i = 2 is the left eye, i = 3 is the nose, and i = 4 is the mouth, these face parts are right eye, left eye, nose, and mouth. The above-mentioned subspace method is applied to the above expression to convert the likelihood of right eye P1 (x), likelihood of left eye P2 (x), likeness of nose P3 (x), and likeness of mouth P4 (x) into the above equation of Pi (x). Calculated based on
[0035]
Next, the process proceeds to step 303, where the existence probability Ri (x) of the reference point of the face using each face part i is obtained. The existence probability Ri (x) is obtained by Pi (x) * Qi (x) using the above Pi (x) and the existence probability Qi (x) of the face reference point for each face part. The operation symbol “*” represents convolution (convolution). Note that this Ri does not satisfy the condition ΣRi = 1 that is a probability, but is referred to as a probability here in the sense that the greater Ri (x), the greater the probability that x is the reference point of the face. The operation relating to step S303 is performed by the existence probability calculation unit 3b of the detection unit 3.
[0036]
FIG. 4 shows the variation of the reference point of the face with respect to each face part. For example, FIG. 4A shows that when the position K1 of the right eye is determined, the probability that the reference point of the face is within the range of the ellipse Q1 is high. Similarly, FIG. 17B shows that when the position K2 of the left eye is determined, there is a high probability that the reference point of the face is within the range of the ellipse Q2, and FIG. This indicates that when the position K3 is determined, the probability that the reference point of the face is within the range of the ellipse Q3 is high. FIG. Indicates that the probability of being within the range of the ellipse Q4 is high.
[0037]
Next, the process proceeds to step S304, where a weight corresponding to the importance is added to each existence probability Ri (x) in each face part i obtained in step S303, and the existence probability Wi · Find the sum of Ri (x). That is, R (x) = ΣWi · Ri (x) is obtained. The operation related to step S304 is performed by the sum total calculation unit 3c of the detection unit 3.
[0038]
When the total sum R (x) of the respective existence probabilities after weighting is obtained, the process proceeds to step S305, and the maximum value Rmax (x) of R (x) is obtained. Further, a position x satisfying Rmax (x) is set as a reference point of the face. That is, based on the existence probability Ri (x) of each face part i, the reference point of each face for each face part i is obtained as shown in FIG. By obtaining the maximum value Rmax (x) of R (x) for each of these reference points, the position x in a state where the face parts i are integrated is obtained as shown in FIG. 5B. The operation related to step S305 is performed by the reference point specifying unit 3d of the detection unit 3.
[0039]
As described above, the face detection device according to the present invention obtains the likelihood of each face part, and then integrates the results in consideration of positional variations. Therefore, unlike the conventional method, there is no large variation in face parts between individuals. That is, such variations can be effectively absorbed. As a result, reliable face detection can be performed, and the reliability of face detection is improved.
[0040]
In the present embodiment, a case where a CCD camera is used as the imaging unit 2 has been described. However, the present invention is not limited to this. For example, an apparatus that employs a scanner to detect a face image in a photograph will be described. It can also be applied.
[0041]
The detecting unit 3 shown in FIG. 1 is constituted by one computer device. In this case, the computer device includes a parts information calculating unit 3a, an existence probability calculating unit 3b, a sum calculating unit 3c, and a reference point specifying unit 3d. The program corresponding to each is incorporated.
[0042]
【The invention's effect】
As described above, according to the face detection device, the face detection method, and the face detection program according to the present invention, after determining the likelihood of each face part, the results are integrated in consideration of the positional variation. Therefore, unlike the conventional method, there is no large variation in face parts between individuals. That is, such variations can be effectively absorbed. As a result, reliable face detection can be performed, and the reliability of face detection is improved. Further, for example, the present invention can be incorporated in a monitoring apparatus, and the present invention can be incorporated in various known apparatuses and applied, thereby contributing to industrial development.
[Brief description of the drawings]
FIG. 1 is a block diagram showing a schematic configuration of a face detection device according to an embodiment of the present invention.
FIG. 2 is a flowchart showing a learning procedure performed before face detection.
FIG. 3 is a flowchart illustrating a procedure of face detection.
FIG. 4 is a diagram showing a variation of a face reference point for each face part.
FIGS. 5A and 5B are schematic diagrams illustrating an operation when a final reference point of a face is obtained. FIG. 5A is a schematic diagram illustrating a state before the integration of each face part, and FIG. .
[Explanation of symbols]
REFERENCE SIGNS LIST 1 face detection device 2 imaging unit 3 detection unit 3a parts information calculation unit 3b existence probability calculation unit 3c total sum calculation unit 3d reference point specification unit 4 storage unit

Claims (9)

A face detection device that detects a face image present in an input image,
Storage means for storing the prior probability of each face part including the existence probability of the reference point of the face based on each face part and the subspace of each face part;
Recognizing each face part present in the input image based on the prior information, calculating a reference point of the face in the input image based on the recognized face part, and based on the calculated reference point of the face And a detecting means for detecting a face image from the input image. The detecting means,
Part information calculation means for calculating the likeness of each face part as part information using the subspace,
Based on the part information calculated by the part information calculating means and the existence probability of the reference point of the face based on each face part, which is one of the preliminary information, the reference point of the face based on each face part Existence probability calculating means for calculating the existence probability of
The existence probability of the face reference point based on each face part calculated by the existence probability calculation means is weighted by the importance of each face part, and the face reference point based on each face part obtained from the weighting result is obtained. Sum calculation means for calculating the sum of the existence probabilities of
2. The face detection device according to claim 1, further comprising: reference point specifying means for obtaining a maximum value of the sum of the existence probabilities calculated by said total calculation means, and specifying the maximum value as a reference point of the face. apparatus. The existence probability calculation means convolves the part information calculated by the parts information calculation means with the existence probability of each face part, which is one of the pieces of prior information, to thereby obtain a reference point of a face based on each face part. The face detection device according to claim 1, wherein an existence probability is calculated. A face detection method for detecting a face image present in an input image,
A storage step of storing in the storage unit the prior probability of each face part including the existence probability of the reference point of the face based on each face part and the subspace of each face part,
Recognizing each face part present in the input image based on the prior information, calculating a reference point of the face in the input image based on the recognized face part, and based on the calculated reference point of the face And detecting a face image from the input image. The detection step,
A part information calculation step of calculating each face part likeness as part information using the subspace;
Based on the part information calculated in the part information calculation step and the existence probability of the reference point of the face based on each face part, which is one of the preliminary information, the reference point of the face based on each face part Existence probability calculation step of calculating the existence probability of
The existence probability of the reference point of the face based on each face part calculated in the existence probability calculation step is weighted by the importance of each face part, and the reference point of the face based on each face part obtained from the weighting result Sum calculation step of calculating the sum of the existence probabilities of
5. The face detection method according to claim 4, further comprising: determining a maximum value of the sum of the existence probabilities calculated in the sum calculation step, and specifying the maximum value as a face reference point. Method. The existence probability calculation step is a convolution of the part information calculated in the part information calculation step with the existence probability of each face part, which is one of the pieces of prior information, to thereby determine the reference point of the face based on each face part. The face detection method according to claim 4, wherein an existence probability is calculated. A face detection program for detecting a face image present in an input image,
A storage procedure for storing in the storage unit the prior probability of each face part including the existence probability of the reference point of the face based on each face part and the subspace of each face part,
Recognizing each face part present in the input image based on the prior information, calculating a reference point of the face in the input image based on the recognized face part, and based on the calculated reference point of the face And a detection procedure for detecting a face image from the input image. The detection procedure includes:
A part information calculation procedure for calculating the likeness of each face part as part information using the subspace,
Based on the part information calculated by the part information calculation procedure and the existence probability of the reference point of the face based on each face part, which is one of the preliminary information, the reference point of the face based on each face part Existence probability calculation procedure for calculating the existence probability of
The existence probability of the reference point of the face based on each face part calculated by the existence probability calculation procedure is weighted by the importance of each face part, and the reference point of the face based on each face part obtained from the weighting result Sum calculation procedure for calculating the sum of the existence probabilities of
8. A reference point specifying step of obtaining a maximum value of the total sum of the existence probabilities calculated by the total sum calculation procedure, and specifying this maximum value as a reference point of the face. program. The existence probability calculation step is performed by convolving the part information calculated by the part information calculation step with the existence probability of each face part, which is one of the pieces of the advance information, to obtain a reference point of a face based on each face part. 9. The face detection program according to claim 7, wherein an existence probability is calculated.

Images