JP2006293720A - Face detection apparatus, face detection method, and face detection program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a face detection technology capable of detecting a face of a person at high speed and highly precisely from among images which a user has photographed. <P>SOLUTION: The face detection apparatus for detecting a face of a person from images has: an image input part 11 for inputting the photographed images; a face area candidate extraction part 12 for applying space frequency filtering locally to the images to extract face area candidates; a face area decision part 13 for deciding whether or not a face is included in the extracted face area candidates; and a detection result output part 14 for outputting a face to a detected person on the basis of the decision result in the face area decision part 13. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a face detection technique for identifying a human face included in an image.

  With the proliferation of mobile phones with cameras and digital cameras, the number of images handled by individuals has increased. In order to classify such an image by a person image and other images, classify by number of people, search, etc., it is necessary to accurately detect only the face of the person from the image.

  Various researches have been conducted on image processing techniques for detecting a human face in an image. For example, a method of discriminating a face by learning a face and a non-face (hereinafter, described as non-face) has been studied (for example, see Non-Patent Document 1). With this method, accurate face detection with reduced false detection of similar patterns can be performed by non-face learning, but the processing speed is slow.

On the other hand, as a high-speed pattern recognition method, an AdaBoost learning method and a method using a cascade structure detector using a Haar type feature amount have been studied (for example, see Non-Patent Document 2). However, both methods have false detection of similar patterns. Moreover, in order to create a discriminator corresponding to the orientation of the face, a plurality of discriminators must be prepared, and there is a problem that erroneous detection increases cumulatively according to the number of discriminators.
Takio Kurita, Toshiharu Taguchi, Kazuhiro Hotta, Kengo Mishima, “Identification of Faces and Faces Using Kernel Discriminant Analysis”, IEICE Pattern Recognition and Media Understanding Study Group, Vol. PRMU 2001-208, pp. 17-24, 2002.1 Sayo ≡, Takayoshi Yamashita, Takuya Okamoto, Masato Kawade, “Fast Omnidirectional Face Detection”, IEICE Image Recognition and Understanding Symposium (MIRU 2004), pp. 271-276, July 2004

  As described above, when a user tries to detect a face of a person who is not necessarily front-facing from a group of images arbitrarily taken regardless of the presence / absence of a person, the number of people, a shooting environment, and an imaging device, Detection was a problem.

  An object of the present invention is to provide a face detection technique capable of detecting a human face at high speed from an image taken by a user, and a face detection technique capable of accurately detecting a human face. .

  Accordingly, in order to solve the above problem, the invention according to claim 1 is a face detection device for detecting a human face from an image, an image input unit for inputting a captured image, and a space for the image. A face area candidate extraction unit that performs frequency filtering locally and extracts face area candidates, and determines whether a face is included in the extracted face area candidates using a feature amount different from the face area candidate extraction. And a detection result output unit that outputs a detected face region based on a determination result in the face region determination unit.

  The face area candidate extraction unit may include an image scale conversion unit that converts a scale of an image input to the image input unit, and a partial region that is a detection target from the scale conversion image. A target region cutout unit to be cut out, a spatial frequency analysis unit that analyzes a spatial frequency of the partial region, a face candidate determination unit that determines whether the partial region can be a face region candidate based on the spatial frequency analysis result, And a face candidate information storage unit that stores face areas determined to be candidates for the input image and face area candidate information including a scale.

  In the invention according to claim 3, the face area determination unit is calculated by a dictionary reading unit that reads a face degree calculation dictionary obtained by learning a face and a non-face, and the face area candidate extraction unit. A target area cutout unit that cuts out a partial area to be processed based on the face area candidate information with respect to the image to which the face area candidate information is assigned, and the face degree of the target area by referring to the face degree calculation dictionary A face calculation unit for calculating, a face non-face determination unit that determines whether the target region is a face or a non-face based on the face degree, a determination result storage unit that stores a determination result for each face region candidate, It is characterized by having.

  Further, in the invention according to claim 4, the face area determination unit learns a part of the face area as a face and the other as a non-face, and determines a face and a non-face using the generated discriminator. It is characterized by performing.

  Further, the invention according to claim 5 is characterized in that the face area determination unit uses a support vector machine as a discriminator for determining a face and a non-face.

  According to a sixth aspect of the present invention, there is provided a face detection method in a face detection device for detecting a human face from an image, wherein the image input unit inputs an image input step of inputting a photographed image, and a face region candidate extraction unit. Is a face region candidate extraction step that performs spatial frequency filtering on the image locally to extract a face region candidate, and a face region determination unit determines whether the extracted face region candidate includes a face. A face area determination step that uses a feature amount different from the face area candidate extraction, and a detection result output step in which the detection result output unit outputs the detected face area based on the determination result in the face area determination unit; It is characterized by having.

  In the invention according to claim 7, the face area candidate extraction step includes: an image scale conversion step in which an image scale conversion unit converts a scale of an image input to the image input unit; and a target region extraction unit. A target region cutting step of cutting out a partial region that is a detection target from the scale-converted image, a spatial frequency analyzing unit that analyzes a spatial frequency of the partial region, a face candidate determining unit, and the spatial frequency A face candidate determination step for determining whether the partial area can be a face area candidate based on an analysis result, and a face candidate information storage unit includes a face area determined as a candidate for the input image, and a scale A face candidate information storing step for storing face area candidate information and storing the face area candidate information.

  In the invention according to claim 8, the face area determination step includes a dictionary reading step in which the dictionary reading unit reads a face degree calculation dictionary obtained by learning a face and a non-face, and a target area cutting unit includes A target area cutting step of cutting out a partial area to be processed based on the face area candidate information from the image to which the face area candidate information calculated by the face area candidate extraction unit is added; A face degree calculating step for calculating a face degree of the target area with reference to the face degree calculation dictionary, and a face non-face determining unit for determining whether the target area is a face or a non-face based on the face degree The face non-face determination step and the determination result storage unit include a determination result storage step of storing a determination result for each face area candidate.

  In the invention according to claim 9, in the face region determination step, a part of the face region is learned as a face and the other is learned as a non-face, and the determination of a face and a non-face is performed using the generated discriminator. It is characterized by performing.

  The invention described in claim 10 is characterized in that the face detection device or the face detection method described in any one of claims 1 to 9 is described in a computer program and can be executed. To do.

  In the first to tenth aspects of the present invention, the face area candidate extraction unit narrows down face area candidates from the input image, and the face area determination unit uses only feature values different from the face area candidate determination for the face area candidates. Face detection.

  According to the first to tenth aspects of the present invention, since the final face detection is performed using only the face area candidate using the feature amount different from the face area candidate determination, the face can be detected at high speed and with high accuracy. Is possible.

  Further, according to the inventions according to claims 4 and 9, since the classifier targeting the partial region of the face is used, it is possible to detect a face in a wide range of directions with one classifier and increase the number of classifiers. Will not cause false detection.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 is a diagram showing a schematic configuration of a face detection apparatus according to the present invention. As shown in FIG. 1, the face detection apparatus includes an image input unit 11, a face region candidate extraction unit 12, a face region determination unit 13, and a detection result output unit 14.

  The image input unit 11 acquires a captured image.

  The face area candidate extraction unit 12 performs scale conversion of the input image, extracts partial areas from the images of each scale and performs spatial frequency analysis, thereby extracting an area that is highly likely to be a face from the image.

  The face area determination unit 13 performs detailed face detection in an area that is a face area candidate. Specifically, a part of the face area candidate is cut out as a partial area, and the face degree is calculated by calculating the facial degree.

  The detection result output unit 14 assigns and outputs information related to the face area to the input image based on the determination result by the face area determination unit.

  FIG. 2 is a diagram illustrating a configuration of the face area candidate extraction unit 12. The face area candidate extraction unit 12 will be described with reference to FIG. As shown in FIG. 2, the face area candidate extraction unit 12 includes an image scale conversion unit 21, a detection target region extraction unit 22, a spatial frequency analysis unit 23, a face candidate determination unit 24, and a face candidate information storage unit 25. Is done.

  The image scale conversion unit 21 performs scale conversion of the image input to the image input unit 11. Here, it is assumed that the minimum image size and the scale conversion width are set in advance. Face area candidates are searched for each scale, and the process in the face area candidate extraction unit 12 ends when the search is completed for all scales.

  The detection target region cutout unit 22 cuts out a region having a predetermined size from the scale-converted image. However, the shape of the region handled here may be an arbitrary shape.

  The spatial frequency analysis unit 23 performs spatial frequency filtering on the cut out region. For example, an AdaBoost learning method and a Haar type feature amount detection method having a cascade detector structure may be used. This technique is described in the literature P.A. Viola, M .; Jones, “Rapid object Detection using a Boosted Cascade of Simple Features”, In Proc. IEEE Conf. on Computer Vision and Pattern Recognition, Kauai, USA, 2001, the details are omitted here.

  The face candidate determination unit 24 determines whether the region can be a face region candidate based on the output value of the spatial frequency analysis unit 23.

  The face candidate information storage unit 25 stores face area candidates and a scale at the time of candidate detection.

  FIG. 3 is a diagram illustrating a configuration of the face area determination unit 13. The face area determination unit 13 will be described with reference to FIG. As shown in FIG. 3, the face area determination unit 13 includes a face degree calculation dictionary 31, a dictionary reading part 32, a target area cutout part 33, a face degree calculation part 34, a face non-face determination part 35, and a determination result storage part. 36.

  It is assumed that a face degree calculation dictionary 31 for viewing details of face information is created in advance. The face degree calculation dictionary 31 uses a support vector machine or the like as a learning discrimination method at the time of creation, and learns and creates a face (hereinafter referred to as a partial face) limited to a predetermined size area and a non-face. . A learning discrimination method such as a support vector machine is used for learning discrimination between a face and a non-face. This method is described in the literature "" Support Vector Machines-Basic Mechanism and Recent Development "Mathematical Sciences NO. 444 Science Inc., JUNE 2000 Ako et al. ", And the details are omitted here.

  It is assumed that the face degree calculation dictionary 31 includes parameters used by the face degree calculation unit 34 and includes information necessary for calculating the face degree for discriminating between a face and a non-face. That is, based on the learning data, it includes the coefficient of the formula that defines the discrimination surface between the face class and the non-face class calculated in the feature space, and the facial degree is the distance from the discrimination surface represented by this equation It shall be calculated by.

  The dictionary reading unit 32 reads the face degree calculation dictionary 31 so that the face degree calculation unit 34 can use it.

  The target area cutout unit 33 cuts out a partial area to be detected from the face area candidate area based on the input image and the face area candidate information calculated by the face area candidate extraction unit 12.

  The face degree calculation unit 34 refers to the face degree calculation dictionary 31 read by the dictionary reading unit 32 and calculates the face degree of the target region by the target region cutout unit 33.

  The face non-face determination unit 35 determines whether the face degree calculated by the face degree calculation unit 34 corresponds to a face or a non-face.

  The determination result storage unit 36 stores the determination information calculated by the face non-face determination unit 35.

  The determination result is output as a detection result from the face detection device in the detection result output unit 14.

  FIG. 4 is a diagram illustrating an example of a partial face that is used as a target area when the face area determination unit 13 performs face area determination.

  An example in which a dictionary is created using the entire area of a face facing forward as a partial face is shown in (a). In this case, the direction of the detected face is only the front, and the face facing sideways and up and down is not detected.

  An example of creating a dictionary using a half face as a partial face is shown in FIG. In this case, the detected face orientation is lateral from the front, and a face in a wide angle range can be detected.

  For example, when the face rotates, when the rotation axis is parallel to the image plane, there is a region where the amount of change in appearance is small due to the rotation. Therefore, it is possible to use an area of the face area where the variation of the luminance value is small depending on the shooting angle. Specifically, when the detection target is a face image, the shape of the left half-face eye or mouth shape is small due to slight clockwise rotation fluctuation. In addition, the luminance values of the eyes, mouth, and nose when the luminance value of the cheek is used as a reference are little changed.

  Therefore, it is possible to use a template created by cutting out the left half surface area of the face normalized by the position of the eyes and mouth and performing normalization by size and luminance distribution. Therefore, for example, when it is desired to detect a face from the front direction to 30 degrees to the right, the left side including the subject's eyes, nose, and mouth may be characterized.

  The size of such a face region is determined so that only common feature points are included by examining the presence of feature points that can serve as a standard for size normalization, such as the eyes, eyebrows, nose, and mouth end points. May be.

  In this way, a dictionary may be created according to the face orientation range desired to be detected and used for detection. Also, a plurality of dictionaries may be combined so that the user can select according to the purpose when using the system.

  FIG. 5 is a diagram showing a flow of processing when face area candidate extraction is performed in the face detection apparatus. As shown in FIG. 5, the specific steps are as follows.

  Step 51: The image input to the image input unit 11 is acquired.

  Step 52: Initialize scale parameters for image reduction processing. The processing after step 53 is repeated processing while reducing the image according to the scale parameter. Here, it is assumed that the update for each repetitive processing of the scale parameter is performed with a predetermined step width up to a predetermined minimum scale.

  Step 53: Cut out the target area from the input image.

  Step 54: Perform spatial frequency filtering of the extracted region.

  Step 55: If the numerical value obtained as a result of Step 54 is a value equal to or greater than a threshold value that can be a prescribed face area candidate, it is determined as a face area candidate. . If the clipped area is a face area candidate, the process proceeds to step 56; otherwise, the process proceeds to step 57.

  Step 56: The cut out area is set as a face area candidate, and the position and scale in the image are stored in the face candidate information storage unit 25.

  Step 57: Judge whether all the areas in the image are the target areas. When the search is completed with the entire area as the target area, the process proceeds to step 58, and otherwise, the process returns to step 53, and the next area is set as the target area in a scanning manner.

  Step 58: It is determined whether the currently processed scale is the minimum scale for the input image. If it is the minimum scale, the process ends. Otherwise, the process proceeds to step 59.

  Step 59: The input image is reduced (scale conversion) by the designated step width.

  FIG. 6 is a diagram illustrating a flow of processing when performing face area determination in the face detection apparatus. As shown in FIG. 6, the specific steps are as follows.

  Step 61: The face degree calculation dictionary 31 is read.

  Step 62: The face area candidate stored in the face area candidate extraction unit 12 is read. Note that one face area candidate information is read at a time, and the steps after step 62 are repeated for the number of area candidates.

  Step 63: The scale obtained by extracting the candidate area by the face area candidate extracting unit 12 is read from the face candidate information storage unit 25, and the target area is cut out from the candidate area and converted into the read scale.

  Step 64: The face degree of the target area is calculated.

  Step 65: Determine whether the target area is a face area. The face degree of the target area calculated in step 64 is compared with a preset face degree value. If the face degree of the target area is larger, it is determined as a face, and if it is smaller, it is determined as a non-face. judge. If it is determined that the face is determined, the process proceeds to step 66; otherwise, the process proceeds to step 69.

  Step 66: It is determined whether the area determined to be a face overlaps with the area stored as a face in the determination result storage unit 36 (the storage steps are Steps 67 and 68). Proceed to step 67, otherwise proceed to step 68.

  Step 67: When there is an overlap, the facial values of both regions are compared, information on the region with the lower facial value is deleted, and the region with the higher facial value is stored as a face in the determination result storage unit. (This stored information is also used in step 6).

  Step 68: The stored area is set as a face and stored in the determination result storage unit 36 (this stored information is also used in step 66).

  Step 69: It is determined whether or not all face area candidates have been searched. If the search has been completed, the process is terminated. Otherwise, the process proceeds to Step 62.

  FIG. 7 is a diagram illustrating an example of use of the face detection apparatus. As shown in FIG. 7, the flow of processing when the image of a surveillance camera installed indoors and outdoors is used as an input image and the face detection apparatus is used in a client server type is shown.

  As a setting, the user sets an image captured by the monitoring camera as the input image. Here, the setting method may be input directly to the face detection apparatus or input via a network.

  In the face detection apparatus, a detection target area is cut out from the image while repeating the scale conversion up to the minimum scale using the captured image as an input image, and a face area candidate is searched while scanning the detection target area. If a face area candidate exists, the corresponding area and scale are stored.

  Next, when a face area candidate exists, in order to perform detailed detection from the face area candidates, the face degree calculation dictionary is read, the detection target area in the face area candidate is cut out, and the face degree of the target area is calculated. The face and the non-face are discriminated based on the face degree. The area determined as the face stores the candidate area as a face. Here, if there is an overlap in face area candidates, that is, if the face area to be stored overlaps with an already stored area, the larger face degree of both areas is updated and stored as the face area.

  If no face area candidate exists, the result is displayed as if there is no face in the input image.

  If a face area candidate exists and the area is determined to be a face, the area is displayed as a result.

  Here, the display form may be displayed as it is on the user terminal, or may be displayed by transmitting the network to a mobile terminal or a remote terminal.

  In the above-described embodiment, the face detection device is realized by, for example, a CPU included in a computer device constituting the face detection device, and can include face area candidate extraction, face area determination, and the like as application programs.

  In addition, data such as processing results obtained by face area candidate extraction, face area determination, etc., calculation results, and information necessary for the processing may be written to and read from an internal memory or an external storage device.

  In addition, a recording medium in which a program code of software for realizing the function of the face detection apparatus is recorded is supplied to the system or apparatus, and the CPU (MPU) of the system or apparatus reads the program code stored in the storage medium. It is also possible to execute. In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiment, and examples of the storage medium storing the program code include CD-ROM, DVD-ROM, and CD-R. , CD-RW, MO, and HDD.

The block diagram of a face detection apparatus. The block diagram of a face area candidate extraction part. The block diagram of a face area | region determination part. The figure which showed an example of the partial face made into a target area when performing face area determination. The figure which shows the flow of a process at the time of performing face area candidate extraction. The figure which shows the flow of a process at the time of performing face area determination. The figure which shows an example of utilization of a face detection apparatus.

Explanation of symbols

DESCRIPTION OF SYMBOLS 11 ... Image input part 12 ... Face area candidate extraction part 13 ... Face area determination part 14 ... Detection result output part 21 ... Image scale conversion part 22 ... Detection object area | region extraction part 23 ... Spatial frequency analysis part 24 ... Face candidate determination part 25 ... Face candidate information storage unit 31 ... Facial degree calculation dictionary 32 ... Dictionary reading unit 33 ... Target area extraction unit 34 ... Facial degree calculation unit 35 ... Face non-face determination unit 36 ... Determination result storage unit

Claims (10)

A face detection device for detecting a human face from an image,
An image input unit for inputting a photographed image;
A facial region candidate extraction unit that locally performs spatial frequency filtering on the image and extracts facial region candidates;
A face area determination unit that determines whether a face is included in the extracted face area candidate using a feature amount different from the face area candidate extraction;
A face detection apparatus comprising: a detection result output unit that outputs a detected face region based on a determination result in the face region determination unit. The face area candidate extraction unit
An image scale converter for converting the scale of the image input to the image input unit;
A target area cutout unit that cuts out a partial area that is a detection target from the scale-converted image;
A spatial frequency analysis unit for analyzing the spatial frequency of the partial region;
A face candidate determination unit that determines whether the partial region can be a face region candidate based on the spatial frequency analysis result;
The face according to claim 1, further comprising: a face area determined as a candidate for the input image, and a face candidate information storage unit that stores face area candidate information including a scale. Detection device. The face area determination unit
A dictionary reading unit that reads a face degree calculation dictionary obtained by learning a face and a non-face;
A target area cutout unit that cuts out a partial area to be processed based on the face area candidate information with respect to the image to which the face area candidate information calculated by the face area candidate extraction unit is attached;
A face degree calculation unit for calculating the face degree of the target region with reference to the face degree calculation dictionary;
A face non-face determination unit that determines whether the target region is a face or a non-face based on the face degree;
The face detection apparatus according to claim 1, further comprising: a determination result storage unit that stores a determination result for each face area candidate.   The face area determination unit learns a part of the face area as a face and the other as a non-face, and determines a face and a non-face using the generated discriminator. 3. The face detection device according to any one of 3.   The face detection apparatus according to claim 1, wherein the face area determination unit uses a support vector machine as a discriminator that determines a face and a non-face. A face detection method in a face detection device for detecting a human face from an image,
An image input unit for inputting a photographed image;
A face area candidate extracting unit that performs spatial frequency filtering on the image locally to extract face area candidates; and
A face area determination step in which a face area determination unit determines whether a face is included in the extracted face area candidate using a feature amount different from the face area candidate extraction;
A detection result output unit, wherein the detection result output unit includes a detection result output step of outputting a detected face region based on a determination result in the face region determination unit. The face area candidate extraction step includes:
An image scale conversion unit for converting a scale of an image input to the image input unit;
A target region cutting step, wherein the target region cutting unit cuts out a partial region that is a detection target from the scale-converted image;
A spatial frequency analysis step, wherein the spatial frequency analysis unit analyzes the spatial frequency of the partial region;
A face candidate determination unit that determines whether the partial region can be a face region candidate based on the spatial frequency analysis result; and
The face candidate information storage unit includes a face candidate information storage step of adding and storing a face area determined as a candidate for the input image and face area candidate information including a scale. The face detection method according to claim 6. The face area determination step includes
A dictionary reading step in which a dictionary reading unit reads a face degree calculation dictionary obtained by learning a face and a non-face;
A target area cutout unit that cuts out a partial area to be processed based on the face area candidate information from the image to which the face area candidate information calculated by the face area candidate extraction unit is added. Steps,
A face degree calculating unit that calculates the face degree of the target region with reference to the face degree calculation dictionary;
A face non-face determination unit for determining whether the target region is a face or a non-face based on the face degree;
The face detection method according to claim 6, wherein the determination result storage unit includes a determination result storage step of storing a determination result for each face area candidate.   The face region determination step includes learning a part of the face region as a face and the other as a non-face, and determining a face and a non-face using the generated discriminator. 8. The face detection method according to any one of the above.   A program characterized in that the face detection device or the face detection method according to any one of claims 1 to 9 described above is described in a computer program and can be executed.

Images