JP4749884B2 - Learning method of face discriminating apparatus, face discriminating method and apparatus, and program - Google Patents

Description

  The present invention relates to a learning method of a face discriminating apparatus that discriminates whether or not a digital image is a face image, a face discriminating method and apparatus, and a program therefor.

  Conventionally, the color distribution of a person's face area in a snapshot photographed by a digital camera is examined to correct the skin color, or a person in a digital image photographed by a digital video camera of a surveillance system is recognized. Has been done. In such a case, it is necessary to detect a face area corresponding to a human face in the digital image. For this reason, various methods for determining whether an image is an image representing a face have been proposed.

  For example, Non-Patent Document 1 normalizes a luminance value, which is a feature amount used when detecting a face, refers to a learning result of a neural network that has learned the face, and determines whether the image is a face image. This is a technique for determining whether or not. Non-Patent Document 2 obtains a high-frequency component such as an edge included in an image as a feature amount used for detection of an object, normalizes the feature amount, and performs machine learning (machine learning) called boosting. This is a method for determining whether or not the image is an image representing an object with reference to the learning result on the feature amount using the method (1). Since these methods of Non-Patent Documents 1 and 2 normalize the feature amount used for detecting an object such as a face, it is possible to accurately determine whether the image is an image representing the object. .

  In addition, it is calculated from each of a large number of sample image groups including a plurality of sample images known to be images representing a predetermined object and a plurality of sample images known to be images not representing the predetermined object. A reference value for discriminating between an image representing a predetermined object and an image not representing the predetermined object by inputting the feature amount, which is obtained by learning the obtained feature amount in advance by a machine learning method. This technique includes a plurality of discriminators, and discriminates that a discrimination target image is an image representing a predetermined target when a weighted sum of reference values output from the plurality of discriminators exceeds a predetermined threshold. It has been proposed by the applicant (see Patent Documents 2 to 4).

  In addition, the feature amount calculated from each of a large number of sample images including a plurality of sample images known to be images representing a face and a plurality of sample images known to be images not representing a face is obtained. A plurality of weak discriminators, which are obtained by learning in advance by a machine learning method, and that determine whether or not the discrimination target image is an image representing a face by input of a feature amount. A method has also been proposed in which when a weakly classifier is identified as an image representing a face, the image to be identified is an image representing a face when the images are represented in a cascade structure. Non-Patent Document 3).

  Using the methods described in Patent Documents 2 to 4 and Non-Patent Document 3, when a face is learned as a sample image, it is well determined whether or not the determination target image is an image representing a face. be able to.

  In addition, because the sample image is deformed stepwise by scaling the sample image stepwise or rotated stepwise, and learning is performed using the sample image obtained at each stage of deformation, Even if the reduction ratio of the face represented by the determination target image is different or slightly rotated, it can be determined whether or not the determination target image is an image representing the face.

  In the case where a plurality of discriminators or a plurality of weak discriminators are obtained by learning in advance by a machine learning method, in the learning, as a sample image representing the face, the top and bottom direction of the face and the direction of the face ( Face images with uniform head swing directions are used (see Non-Patent Document 3, FIG. 7 and the like). By using a face image with the same orientation and orientation of the face as a sample image, the facial parts such as eyes, nose, and mouth that make up the face and the outline of the face are substantially specified in the sample image representing the face. Therefore, it is easy to find features on the pattern common to the face, and an improvement in discrimination accuracy can be expected.

Further, in the learning using the face image having the same vertical direction and orientation as the sample image as described above, the face orientation of the sample image used for the learning is a face orientation that can be determined as it is. Therefore, when multi-directional face detection for detecting a face facing an arbitrary direction is to be realized, face discrimination means (apparatus) for discriminating whether or not the discrimination target image is an image representing a face by the discrimination method is provided. A plurality of face orientations are prepared and used simultaneously.
Henry A. Rowley, Shumeet Baluja, and Takeo Kanada, "Neural Network-Based Face Detection", volume 20, number 1, pages 23-38, January 1998. Rainer Lienhart, Jochen Maydt, "An Extended Set of Haar-like Features for Rapid Object Detection", International Conference on Image Processing. "High-speed omnidirectional face detection", Shihong LAO et al., Image Recognition and Understanding Symposium (MIRU2004), July 2004, P.II-271-II-276 Japanese Patent Laid-Open No. 5-282457 Japanese Patent Application No. 2003-316924 Japanese Patent Application No. 2003-316925 Japanese Patent Application No. 2003-316926

  By the way, the feature parts on the image to be learned by the face discriminating apparatus are not always the same, and in particular, differ depending on the face direction to be discriminated by the face discriminating apparatus and the type of processing on the face detection used by the face discriminating apparatus.

  For example, in a face discriminating apparatus that discriminates a side face, it is important to learn the presence of a relatively wide background unique to the side face, and a face discriminating apparatus that discriminates a front face and selects face candidates for face detection. In a face discrimination device used for rough detection processing, it is important to emphasize the robustness and to learn the simplest and most common features of the front face that have a round face shape rather than the fine features of the facial parts It becomes.

  However, learning using a face image with the same orientation and orientation of the face as a sample image as described above only aligns the orientation and orientation of the sample image representing the face. Are not properly included in the sample image, and the features on the image included in the sample image tend to be distributed among a plurality of images. Difficult to learn.

  SUMMARY OF THE INVENTION In view of the above circumstances, an object of the present invention is to provide a face discriminating apparatus that accurately learns a characteristic that is originally intended to be learned, a program therefor, and a learning method for the face discriminating apparatus for obtaining the face discriminating apparatus. It is.

  According to the learning method of the face discriminating apparatus of the present invention, a face discriminating apparatus for discriminating whether an input image is a face image including a face in a predetermined direction and a vertical direction, the face direction and the vertical direction are the predetermined direction. In the learning method of the face discriminating apparatus that learns the facial features by a machine learning method using a plurality of different learning face images that are in the direction and the top and bottom directions, the plurality of learning face images have the predetermined orientation and It is characterized by comprising only an image of a predetermined face area determined according to at least one of the types of processing for face detection used by the face discrimination device.

  The learning device of the face discriminating apparatus of the present invention is a face discriminating device for discriminating whether or not the input image is a face image including a face in a predetermined direction and a vertical direction. In the learning device of the face discriminating apparatus that learns the facial features by a machine learning method using a plurality of different learning face images that are the orientation and the top-and-bottom direction, the plurality of learning face images have the predetermined orientation and It is characterized by comprising only an image of a predetermined face area determined according to at least one of the types of processing for face detection used by the face discrimination device.

  The program according to the present invention provides a face discriminating device for discriminating whether or not an input image is a face image including a face in a predetermined direction and a vertical direction, the face direction and the vertical direction being the predetermined direction. And a program for executing a process of learning facial features by a machine learning method using a plurality of different learning face images in a vertical direction, wherein the plurality of learning face images have the predetermined orientation and the face discrimination. The apparatus comprises only an image of a predetermined face area determined according to at least one of the types of processing on face detection used by the apparatus (first program).

  According to the face discrimination method of the present invention, an input image obtained by learning facial features by a machine learning method using a plurality of different learning face images in which the face direction and the top and bottom direction are a predetermined direction and the top and bottom direction is used. In the face discriminating method for discriminating whether or not the input image is the face image using a face discriminating device that discriminates whether or not the face image includes a face in the predetermined direction and the top and bottom direction, The learning face image is composed only of an image of a predetermined face area determined according to at least one of the predetermined direction and the type of processing on face detection using the face discrimination device. It is.

  The face discriminating apparatus of the present invention has an input image obtained by learning facial features by a machine learning method using a plurality of different learning face images in which the face direction and the vertical direction are a predetermined direction and the vertical direction. In the face discriminating apparatus that discriminates whether or not the face image includes a face in the predetermined direction and the top-and-bottom direction, the plurality of learning face images are used for the face detection in which the predetermined direction and the face discriminating apparatus are used. It consists only of an image of a predetermined face area determined according to at least one of the types of processing.

  The program according to the present invention is an input image obtained by learning a facial feature by a computer learning method using a plurality of different learning face images in which the face direction and the vertical direction are a predetermined direction and the vertical direction. In a program for functioning as a face discriminating device that discriminates whether or not the face image includes a face in the predetermined direction and the top-and-bottom direction, the plurality of learning face images include the predetermined direction and the face determination The apparatus comprises only an image of a predetermined face area determined according to at least one of the types of processing on face detection used by the apparatus (second program).

  In the present invention, when the predetermined direction is horizontal, the predetermined face area can be an area surrounding the entire face contour.

  Further, in the present invention, when the predetermined direction is the front, and the type of the process is the pre-stage process in the face detection process including the pre-stage process for extracting the face candidates and the post-stage process for narrowing down the extracted face candidates. The predetermined face area may be an area surrounding the entire face contour.

  Further, in the present invention, when the predetermined direction is the front and the type of processing is the post-processing in the face detection processing including the pre-processing for extracting the face candidates and the post-processing for narrowing down the extracted face candidates. The predetermined face region may be a region surrounding only both eyes, nose and upper lip and a region surrounding all face contours, a region surrounding only both eyes, nose and upper lip, or a region surrounding only both eyes and nose.

  In the present invention, the face area is preferably a rectangular area.

  In the present invention, the face discriminating apparatus may have a structure in which a plurality of different weak discriminators are linearly coupled.

  In the present invention, it is sufficient that at least the learning face image is used as a face image used for learning. Of course, a learning non-face image may be used in addition to the learning face image as an image used for learning. Absent.

  In the present invention, as a machine learning technique, for example, a technique such as boosting or a neural network can be considered.

  “Learning face image” means a sample image used for learning that is known to be an image representing a face, and “Non-face image for learning” is used for learning that is not known to represent a face. A sample image.

  “The face direction and the top and bottom direction are the predetermined direction and the top and bottom direction” is not limited to a state in which the face direction and the top and bottom direction completely coincide with the predetermined direction and the top and bottom direction. For example, it is assumed that a rotational angle shift of about ± 15 degrees on the image plane is allowed in the vertical direction of the face.

  “Weak discriminator” is a discriminating means (module) with a correct answer rate exceeding 50%, and “a structure in which a plurality of weak discriminators are linearly combined” is such a weak discriminator connected in series. The weak classifier has a structure configured to proceed to the next weak classifier when the target image is determined to be a face image and to leave the determination process when it is determined to be a non-face image. That means. The target image determined to be a face image by the last weak classifier is finally determined to be a face image.

  The “face” has eyes, nose and mouth as components, and the mouth is composed of an upper lip and a lower lip. Ears are not included in “components”. Therefore, the face area of the learning face image may or may not include an ear. “Eyes” may or may not include eyebrows.

  The “face contour” is the contour excluding the contours of the head (including the back of the head) and neck. In the case of a frontal face, the right temple, chin, and left temple are connected. In the case of an outline or profile, it means an outline connecting points on the forehead, nose, and lower jaw.

  According to the learning method of the face discriminating apparatus of the present invention, in addition to aligning the face direction and the top-and-bottom direction of the learning face image, the face of the learning face image is classified according to the case where the characteristic parts on the image to be learned are different. Since the area is limited to a predetermined face area, a feature portion on the image to be learned can be appropriately included in the learning face image, and a plurality of features on the image included in the learning face image can be included. It is possible to obtain a face discriminating apparatus that can suppress dispersion between images and accurately learn features on an image that is originally desired to be learned.

  Further, since the face discriminating apparatus and program according to the present invention are obtained by learning based on the learning method described above, the face discriminating apparatus and the program therefor accurately learn the features on the image to be originally learned.

  Hereinafter, embodiments of the present invention will be described. FIG. 1 is a schematic block diagram showing a configuration of a face detection system to which a face discrimination device (discriminator) according to the present invention is applied. This face detection system detects a face in a digital image regardless of its position, size, orientation, and rotation angle based on a discrimination result by a discriminator. As shown in FIG. 1, the face detection system 1 has a multi-resolution image generation unit 10 that obtains a plurality of resolution images (reduced images) S1_i (i = 1, 2, 3,...) By converting the input image S0 to multi-resolution. The image normalization unit 20 obtains a normalized multi-resolution image S1′_i by normalizing the luminance dispersion of the multi-resolution image S1_i, and performs a rough face detection process on each resolution image S1′_i. A face detection pre-processing unit 30 that extracts the candidate S2, a face detection post-processing unit 40 that obtains a face S3 by performing high-precision face detection processing on the face candidate neighborhood image to narrow down the face candidate S2, and a plurality of resolution images A duplicate detection determination processing unit 50 is provided that arranges the duplicated detected faces S3 to obtain a face S3 ′.

  The multi-resolution image generation unit 10 converts the image size (resolution) of the input image S0 to be face-detected into a predetermined size, for example, a rectangular size with a short side of 416 pixels, and generates the image S1, and further the image S1 Are generated as a basic image, and a plurality of resolution images S1_i having different resolutions are generated. The reason why such an image is generated is that the size of the face included in the input image is usually unknown. On the other hand, the size of the face to be detected (image size) depends on the structure of the discriminator described later. This is because it is necessary to make them constant in relation to each other, and it is necessary to cut out partial images of a predetermined size on images with different resolutions to determine whether they are faces or non-faces. Specifically, as illustrated in FIG. 2, an image S1 is a basic image S1_1, an image S1_2 that is −1/3 times a power of 2 with respect to the image S1_1, and a 2 −1/3 power of the image S1_2 The image S1_3 that is doubled (2 to the power of 2/3 for the basic image S1_1) is generated first, and then a reduced image that is ½ times the size of each of the images S1_1, S1_2, and S1_3 Are generated, and a reduced image of 1/2 size is further generated with respect to the reduced images, so that a predetermined number of reduced images are generated. In this way, a plurality of images whose resolution is reduced by a factor of −1/3 times from the basic image at a high speed mainly by a reduction process of 1/2 times that does not require an interpolation process of luminance signals. Can be generated. For example, when the image S1_1 has a rectangular size of 416 pixels on the short side, the images S1_2, S1_3,... Have 330 pixels, 262 pixels, 208 pixels, 165 pixels, 131 pixels, 104 pixels, and 82 on the short sides, respectively. The resolution is reduced to a square size of pixels, 65 pixels,. Note that an image generated without interpolating the luminance signal in this way has a strong tendency to carry the characteristics of the image pattern as they are, and is preferable in that an improvement in accuracy can be expected in the face detection process.

  The image normalization unit 20 performs normalization by performing gradation conversion so that the luminance dispersion becomes a predetermined level so that the accuracy of face detection processing to be performed later on the multi-resolution image S1_i is improved, and has been normalized. Multi-resolution image S1′_i is obtained.

  The face detection pre-processing unit 30 performs a relatively coarse and high-speed face detection process on each resolution image S1′_i normalized by the image normalization unit 20, and provisionally selects a face candidate S2 from each resolution image S1′_i. To be extracted. FIG. 3 is a block diagram showing the configuration of the face detection pre-processing unit 30. As shown in FIG. 3, the face detection pre-processing unit 30 includes a first front face detection unit 31 that mainly detects a front face, a first left side face detection unit 32 that mainly detects a left side face, The first right side face detection unit 33 for detecting the right side face includes a plurality of weak discriminators WCi (i = 1 to N) that are linearly coupled. Discriminators 31a, 32a, 33a having a cascade structure.

  FIG. 5 shows a general processing flow in the discriminator, and FIG. 6 shows a processing flow by each weak discriminator therein.

  First, the first weak discriminator WC1 discriminates whether or not the partial image is a face with respect to a partial image of a predetermined size cut out on the resolution image S1′_i (step SS1). Specifically, as shown in FIG. 7, the weak discriminator WC1 uses four neighboring pixels for a partial image of a predetermined size cut out on the resolution image S1′_i, for example, an image of 32 × 32 pixel size. By averaging, an image of 16 × 16 pixel size and a reduced image of 8 × 8 pixel size are obtained, and a plurality of types of two predetermined points set in the plane of these three images are used as one pair. The difference value of the brightness | luminance between two points in each pair which comprises one pair group which consists of a pair is each calculated, and the combination of these difference values is made into a feature-value (step SS1-1). The predetermined two points of each pair are, for example, two predetermined points arranged in the vertical direction and two predetermined points arranged in the horizontal direction so as to reflect the characteristics of the facial shading on the image. Then, a score is calculated by referring to a predetermined score table according to a combination of difference values as feature amounts (step SS1-2), and the score calculated by itself is added to the score calculated by the previous weak discriminator. The accumulated score is calculated (step SS1-3). However, since the first weak discriminator WC1 has no previous weak discriminator, the score calculated by itself is used as the cumulative score. It is determined whether or not the partial image is a face depending on whether or not the accumulated score is equal to or greater than a predetermined threshold (step SS1-4). Here, when the partial image is determined to be a face, the process proceeds to determination by the next weak classifier WC2 (step SS2). When the partial image is determined to be a non-face, the partial image is immediately determined to be a non-face. It is determined (step SSB), and the process ends.

  Also in step SS2, as in step SS1, the weak classifier WC2 calculates the above-described feature amount representing the feature on the image based on the partial image (step SS2-1), and refers to the score table for the feature. A score is calculated from the amount (step SS2-2). Then, the score calculated by itself is added to the cumulative score calculated by the previous weak discriminator WC1 to update the cumulative score (step SS2-3), and the partial image is determined depending on whether the cumulative score is equal to or greater than a predetermined threshold. Is a face (step SS2-4). Again, when the partial image is determined to be a face, the process proceeds to determination by the next weak classifier WC3 (step SS3). When the partial image is determined to be a non-face, the partial image is immediately determined to be a non-face. (Step SSB), and the process ends. In this way, when it is determined that the partial image is a face in all N weak classifiers, the partial image is finally extracted as a face candidate (step SSA).

  Each of the face detection units 31 to 33 has a unique discriminator composed of a plurality of weak discriminators determined by a characteristic type, a score table, and a threshold value, that is, the face direction to be discriminated, that is, the front surface. A face, a left side profile, and a right side profile are identified. Further, the face detection units 31 to 33 perform the discrimination processing for the partial images as described above while rotating the resolution images 360 degrees on the planes of all the resolution images S1′_i as shown in FIG. A sub-window W for cutting out a 32 × 32 pixel size partial image on the image is set, and the sub-window W is moved by a predetermined number of pixels on the resolution image, for example, by 5 pixels, and the partial image cut out in the sub-window W By determining whether or not the image is an image, a front face, a left side face, and a right side face at every rotation angle on the plane are detected in each resolution image S1′_i, and a face candidate S2 is output. In addition, in order to improve the detection accuracy of the oblique face, a discriminator for discriminating each of the right oblique face and the left oblique face may be further provided, but it is not particularly provided here.

  The face detection post-processing unit 40 performs face detection processing with relatively high accuracy on the image near the face candidate S2 extracted by the face detection pre-processing unit 30, and the true face S3 from the image near the face candidate. Is detected. The face detection post-processing unit 40 basically has the same configuration as that of the face detection pre-processing unit 30, and as shown in FIG. 4, a second front face detection unit 41 that mainly detects a front face. The second left side face detection unit 42 that mainly detects the left side face, and the second right side face detection unit 43 that mainly detects the right side face. A plurality of weak classifiers WCi (i = 1 to N) are linearly coupled to each other to include classifiers 41a, 42a, and 43a having a cascade structure. However, it is preferable that these discriminators have higher discrimination accuracy than the discriminators in the face detection pre-processing unit 30. In the face detection post-processing unit 40, the general processing flow in the classifier and the processing flow in the weak classifier are basically the same as those in the face detection pre-processing unit 30, but the position where the sub window W is set is The image is limited to an image in a predetermined area including the face candidate S2 extracted by the face detection pre-processing unit 30, and the movement width of the subwindow W is finer than that of the face detection pre-processing unit 30, for example, one pixel at a time. It becomes. As a result, the face candidates S2 roughly extracted by the face detection pre-processing unit 30 are further narrowed down and only the true face S3 is output.

  Based on the position information of the face S3 on each resolution image S1′_i detected by the face detection post-processing unit 40, the duplication detection determination processing unit 50 detects duplicates of the faces detected on each resolution image. A process of grouping the same faces as one face is performed, and position information of the face S3 ′ detected in the input image S0 is output. Although the discriminator depends on the learning method, the size of the face that can be detected with respect to the size of the partial image generally has a certain width, and therefore, in a plurality of resolution images having adjacent resolution levels, the same face is detected. This is because it may be detected in duplicate.

  FIG. 9 is a flowchart showing the flow of processing in the face detection system. As shown in FIG. 9, when the input image S0 is supplied to the multi-resolution image generation unit 10 (step S1), an image S1 in which the image size of the input image S0 is converted to a predetermined size is generated, and the image S1 A plurality of resolution images S1_i whose resolution is reduced by a factor of 2 to 1/3 times are generated (step S2). Then, the image normalization unit 20 normalizes the luminance dispersion of each resolution image S1_i, and obtains a normalized resolution image S1′_i (step S3). The face detection pre-processing unit 30 roughly detects the face candidate S2 for each resolution image S1′_i using the front face, right side face, and left side face discriminators 31a, 32a, and 33a (step S4). Further, the face detection post-stage processing unit 40 applies the front face, right side face, and left side face discriminators 41a to the neighborhood image of the face candidate S2 extracted in step 4 in the same manner as the face detection pre-stage processing unit 30. Face detection corresponding to scrutiny is performed using 42a and 43a and narrowed down to the true face S3 (step S5). Then, the same face detected redundantly in each resolution image S1′_i is determined (step S6), and these are collectively combined into one finally detected face S3 ′.

  Next, a learning method for the classifier will be described. FIG. 10 is a flowchart showing a learning method of the classifier. Note that learning is performed for each type of discriminator, that is, for each orientation of the face to be discriminated.

  The sample image group to be learned is a plurality of sample images that are known to be faces and a plurality of samples that are known to be non-faces, standardized at a predetermined size, for example, 32 × 32 pixel size. It consists of an image. As a sample image that is known to be a face, an image in which the face orientation is the same as the face orientation to be discriminated by the discriminator and the face orientations are aligned is used. A sample image that is known to be a face is obtained by scaling in steps of 0.1 times in the range of 0.7 to 1.2 times in length and / or width for each sample image. For each sample image to be obtained, a plurality of deformation variations obtained by rotating stepwise in units of 3 degrees within a range of ± 15 degrees on a plane is used. Each sample image is assigned a weight or importance. First, the initial value of the weight of all the sample images is set equal to 1 (step S11).

  Next, when a plurality of types of pair groups consisting of a plurality of pairs are set with a predetermined two points set in the plane of the sample image and the reduced image as one pair, each of the plurality of types of pair groups is weak. A separate device is created (step S12). Here, each weak discriminator sets one pair group consisting of a plurality of pairs with a predetermined two points set in the plane of the partial image cut out in the sub-window W and the reduced image as one pair. A reference for discriminating between a face image and a non-face image using a combination of luminance difference values between two points in each pair constituting the one pair group is provided. In the present embodiment, a histogram of combinations of luminance difference values between two points in each pair constituting one pair group is used as the basis of the score table of the weak classifier.

  The creation of a classifier will be described with reference to FIG. As shown in the sample image on the left side of FIG. 11, two points of each pair constituting the pair group for creating this discriminator are a plurality of sample images that are known to be faces. The right eye on the reduced image of 16 × 16 pixel size in which the point in the center of the right eye is P1, the point in the right cheek part is P2, the point in the part between the eyebrows is P3, and the sample image is reduced by an average of four neighboring pixels The point at the center of P4, the point at the cheek on the right side is P5, and the point at the forehead part on the reduced image of 8 × 8 pixel size reduced by the average of 4 neighboring pixels is P6, the mouth part A certain point is P7, and there are five pairs of P1-P2, P1-P3, P4-P5, P4-P6, and P6-P7. Note that the coordinate positions of the two points of each pair constituting one pair group for creating a certain classifier are the same in all sample images. Then, for all sample images that are known to be faces, combinations of luminance difference values between two points of each pair constituting the above five pairs are obtained, and a histogram thereof is created. Here, the value that can be taken as a combination of luminance difference values depends on the number of luminance gradations of the image, but if it is a 16-bit gradation, there are 65536 patterns for one luminance difference value. The number of gradations is (the number of pairs), that is, 65536 to the fifth power, and a large number of samples, time, and memory are required for learning and detection. For this reason, in the present embodiment, the luminance difference value is divided and quantized by an appropriate numerical value width, and converted into an n-value (for example, n = 100).

  As a result, the number of combinations of luminance difference values is n to the fifth power, so the number of data representing the combination of luminance difference values can be reduced.

  Similarly, histograms are created for a plurality of sample images that are known not to be faces. For sample images that are known not to be faces, positions corresponding to the positions of the two predetermined points of each pair on the sample image that is known to be a face (similarly, reference numerals P1 to P7 are used). ) Is used. A histogram obtained by taking the logarithm of the ratio of the frequency values indicated by these two histograms and representing the histogram is the histogram used as the basis of the score table of the weak discriminator shown on the rightmost side of FIG. The value of each vertical axis indicated by the histogram of the weak classifier is hereinafter referred to as a discrimination point. According to this weak discriminator, an image showing the distribution of combinations of luminance difference values corresponding to positive discrimination points is likely to be a face, and the possibility increases as the absolute value of the discrimination point increases. I can say that. Conversely, an image showing a distribution of combinations of luminance difference values corresponding to negative discrimination points is highly likely not to be a face, and the possibility increases as the absolute value of the discrimination point increases. In step S12, a plurality of weak discriminators in the above-described histogram format are created for combinations of luminance difference values between two predetermined points of each pair constituting a plurality of types of pair groups that can be used for discrimination.

  Subsequently, the most effective weak discriminator for discriminating whether or not the image is a face is selected from the plurality of weak semi-divided devices created in step S12. The most effective weak classifier is selected in consideration of the weight of each sample image. In this example, the weighted correct answer rates of the weak classifiers are compared, and the weak classifier showing the highest weighted correct answer rate is selected (step S13). That is, in the first step S13, since the weight of each sample image is equal to 1, the one with the largest number of sample images for which it is simply determined correctly whether or not the image is a face by the weak classifier is as follows: Selected as the most effective weak classifier. On the other hand, in the second step S13 after the weight of each sample image is updated in step S15, which will be described later, a sample image with a weight of 1, a sample image with a weight greater than 1, and a sample image with a weight less than 1 The sample images having a weight greater than 1 are counted more in the evaluation of the correct answer rate because the weight is larger than the sample images having a weight of 1. Thereby, in step S13 after the second time, more emphasis is placed on correctly identifying a sample image having a large weight than a sample image having a small weight.

  Next, the correct answer rate of the combination of weak classifiers selected so far, that is, using the weak classifiers selected so far in combination (in the learning stage, the weak classifiers do not necessarily need to be linearly combined. ) It is ascertained whether the result of determining whether or not each sample image is a face image has exceeded a predetermined threshold value at a rate that matches the answer of whether or not it is actually a face image (step) S14). Here, the current weighted sample image group or the sample image group with equal weight may be used for evaluating the correct answer rate of the combination of weak classifiers. When the predetermined threshold value is exceeded, learning is terminated because it is possible to determine whether the image is a face with a sufficiently high probability by using the weak classifier selected so far. If it is equal to or less than the predetermined threshold value, the process proceeds to step S16 in order to select an additional weak classifier to be used in combination with the weak classifier selected so far.

  In step S16, the weak discriminator selected in the most recent step S13 is excluded so as not to be selected again.

  Next, the weight of the sample image that could not be correctly determined whether or not it is a face in the weak classifier selected in the most recent step S13 is increased, and the sample image that can be correctly determined whether or not the image is a face. Is reduced (step S15). The reason for increasing or decreasing the weight in this way is that in the selection of the next weak classifier, importance is placed on images that could not be correctly determined by the already selected weak classifier, and whether or not those images are faces is correct. This is because a weak discriminator that can be discriminated is selected to enhance the effect of the combination of the weak discriminators.

  Subsequently, the process returns to step S13, and the next effective weak classifier is selected based on the weighted correct answer rate as described above.

  As a weak discriminator suitable for discriminating whether or not it is a face by repeating the above steps S13 to S16, a combination of luminance difference values between predetermined two points of each pair constituting a specific pair group When the weak classifier corresponding to is selected and the correct answer rate confirmed in step S14 exceeds the threshold value, the type of the weak classifier and the determination condition used for determining whether or not the face is determined are determined ( Step S17), thereby completing the learning. The selected weak classifiers are linearly combined in descending order of the weighted correct answer rate to constitute one classifier. For each weak discriminator, a score table for calculating a score according to a combination of luminance difference values is generated based on the obtained histogram. Note that the histogram itself can also be used as a score table. In this case, the discrimination point of the histogram is directly used as a score.

  In the case of employing the learning method described above, the weak classifier uses a combination of luminance difference values between two predetermined points of each pair constituting a specific pair group, and a face image and a non-face image. As long as it provides a criterion for discriminating the above, it is not limited to the above-described histogram format, and may be anything, for example, binary data, threshold value, function, or the like. Further, even in the same histogram format, a histogram or the like indicating the distribution of difference values between the two histograms shown in the center of FIG. 11 may be used.

  Further, the learning method is not limited to the above method, and other machine learning methods such as a neural network can be used.

  Here, let us consider how the feature portion on the image to be learned by the discriminator differs depending on the orientation of the face to be discriminated and the type of processing on the face detection in which the discriminator is used.

  First, consider the case where the face to be identified is a profile. An image representing a profile is characterized by the presence of a relatively wide background other than the face and a relatively sharp outline of the jaw. Therefore, it is important for the discriminators 32a, 33a, 42a, and 43a that discriminate the profile to learn the presence of a relatively wide background specific to the profile and the contour shape of the jaw. In order for the classifier to learn these features, it is necessary to properly include these features in the sample image representing the face. For example, as shown in FIG. It can be considered that the image is limited to the image of the region surrounding the entire outline. Note that it is more preferable that the profile image of the profile be set so that the contour of the side surface of the face is positioned at the approximate center of the image because the balance of the arrangement of the characteristic portions included in the image is good.

  Next, consider the case where the face to be identified is a front face. In the image representing the front, the presence of both eyes, nose and mouth, and the round outline of the face are characteristic. However, in the front face, the mouth is deformed by a change in facial expression, such as a face with a widened mouth or a face with a wide open mouth, and a large change tends to appear particularly in the shape of the lower part of the upper lip.

  Therefore, like the discriminator 31a of the face detection pre-processing unit 30, the face to be discriminated is a front face, and the type of processing on the face detection used by the discriminator is the pre-processing for extracting face candidates. In the case of the preceding process in the face detection process including the subsequent process for narrowing down the extracted face candidates, it is the simplest and most common face face in the front face from the viewpoint of preventing omission of face candidates. It is important to learn the round shape. In order for the discriminator to learn this feature, it is necessary to appropriately include this feature in the sample image representing the face. For example, as shown in FIG. It is conceivable that the image is limited to the image of the region surrounding the.

  Further, like the discriminator 41a of the face detection post-processing unit 40, the face to be discriminated is a front face, and the type of processing on the face detection used by the discriminator is the pre-processing for extracting face candidates. In the case of the latter-stage process in the face-detection process including the latter-stage process for narrowing down the extracted face candidates, from the viewpoint of suppressing the influence of variations in deformation around the mouth and improving the discrimination accuracy, both eyes, nose, upper lip It is important to learn mainly the existence of In order for the discriminator to learn this feature, it is necessary that the feature image is properly included in the sample image representing the face. For example, the sample image representing the face is obtained by using both eyes and nose as shown in FIG. And only the image of the region surrounding only the upper lip, or the image of the region surrounding all the face contours as shown in FIG. 13 and the region surrounding only the eyes, nose and upper lip as shown in FIG. It is conceivable to limit to both images.

  Here, the “face outline” is an outline excluding the outline of the head (including the back of the head) and the neck. In the case of a front face, each of the right temple, chin and left temple In the case of a profile connecting points, in the case of a profile, it is a profile connecting points on the forehead, nose, and lower jaw.

  As described above, according to the learning method of the face discriminating apparatus (discriminator) according to the present embodiment, in addition to aligning the face direction and the top-and-bottom direction of the learning face image, the feature portions on the image to be learned are respectively Since the face area of the sample image representing the face is limited to a predetermined face area for each different case, the feature portion on the image to be learned can be appropriately included in the sample image representing the face, and the face It is possible to suppress the dispersion of the features on the image included in the sample image representing the image between the plurality of images, and it is possible to obtain a face discriminating apparatus that has accurately learned the features on the image to be originally learned. . In addition, since the face discriminating apparatus of the present embodiment is obtained by learning based on the learning method described above, it is a face discriminating apparatus that has accurately learned the features on the image that are originally desired to be learned.

  Note that the face area of the sample image representing the face is preferably all rectangular for convenience.

  The face discriminating apparatus and the learning method of the face discriminating apparatus according to the embodiment of the present invention have been described above, but a program for causing a computer to execute each process in the face discriminating apparatus (discriminator) is also an embodiment of the present invention. It is one of. A computer-readable recording medium that records such a program is also one embodiment of the present invention.

Block diagram showing the configuration of the face detection system 1 Diagram showing the multi-resolution image generation process Block diagram showing the configuration of the face detection pre-processing unit 30 Block diagram showing the configuration of the post-face detection processing unit 40 Diagram showing the global processing flow in the classifier The figure which shows the processing flow in the weak classifier The figure for demonstrating calculation of the feature-value in a weak discriminator Diagram for explaining resolution image rotation and sub-window movement in multiple resolution images The flowchart which shows the process performed in the face detection system 1 Flow chart showing the learning method of the classifier The figure which shows the method of deriving the histogram of the weak classifier The figure which shows the example of the sample image of the face area | region surrounding all the face outlines of a profile The figure which shows the example of the sample image of the area | region surrounding the face outline of all the front faces The figure which shows the example of the sample image of the face area which surrounds only both eyes, nose and upper lip

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Face detection system 10 Multi-resolution image generation part 20 Image normalization part 30 Face detection pre-processing part 31 1st front face detection part 32 1st left side face detection part 33 1st right side face detection part 31a, 32a, 33a Discriminator 40 Face detection post-stage processing unit 41 Second front face detection unit 42 Second left side face detection unit 43 Second right side face detection units 41a, 42a, 43a Discriminator 50 Duplicate detection determination processing unit

Claims (21)

A face discriminating apparatus for discriminating whether or not an input image is a face image including a face in a predetermined direction and a vertical direction. A plurality of different learnings in which the facial direction and the vertical direction are the predetermined direction and the vertical direction. In a learning method of a face discrimination device that learns facial features by a machine learning method using facial images
The plurality of learning face images consist only of images of predetermined face areas determined according to the predetermined direction and the type of processing on face detection in which the face discrimination device is used,
The process on face detection using the face discrimination device is a face detection process having a pre-stage process for extracting face candidates and a post-stage process for narrowing down the extracted face candidates,
When the predetermined direction is the front and the type of processing is the pre-stage processing, the predetermined face region is a region surrounding all face contours,
When the predetermined direction is the front and the type of processing is the subsequent processing, the predetermined face region is a region surrounding only both eyes, nose and upper lip. Device learning method. A face discriminating apparatus for discriminating whether or not an input image is a face image including a face in a predetermined direction and a vertical direction. In a learning method of a face discrimination device that learns facial features by a machine learning method using facial images
The plurality of learning face images consist only of images of predetermined face areas determined according to the predetermined direction and the type of processing on face detection in which the face discrimination device is used,
The process on face detection using the face discrimination device is a face detection process having a pre-stage process for extracting face candidates and a post-stage process for narrowing down the extracted face candidates,
When the predetermined direction is the front and the type of processing is the pre-stage processing, the predetermined face region is a region surrounding all face contours,
When the predetermined orientation is the front and the type of processing is the subsequent processing, the predetermined face region is a region surrounding only both eyes and nose. Learning method.   The predetermined face area is an area surrounding all face contours when the predetermined direction is horizontal and the type of the process is the pre-stage process or the post-stage process. The learning method of the face discrimination device according to 2.   4. The learning method for a face discrimination device according to claim 1, wherein the face area is a rectangular area.   5. The learning method for a face discrimination device according to claim 1, wherein the face discrimination device has a structure in which a plurality of weak discriminators different from each other are linearly coupled.   6. The learning method for a face discrimination device according to claim 1, wherein the machine learning method is boosting. A face discriminating apparatus for discriminating whether or not an input image is a face image including a face in a predetermined direction and a vertical direction. A plurality of different learnings in which the facial direction and the vertical direction are the predetermined direction and the vertical direction. In a learning device of a face discrimination device that learns facial features by a machine learning method using a facial image
The plurality of learning face images consist only of images of predetermined face areas determined according to the predetermined direction and the type of processing on face detection in which the face discrimination device is used,
The process on face detection using the face discrimination device is a face detection process having a pre-stage process for extracting face candidates and a post-stage process for narrowing down the extracted face candidates,
When the predetermined direction is the front and the type of processing is the pre-stage processing, the predetermined face region is a region surrounding all face contours,
When the predetermined direction is the front and the type of processing is the subsequent processing, the predetermined face region is a region surrounding only both eyes, nose and upper lip. Device learning device. A face discriminating apparatus for discriminating whether or not an input image is a face image including a face in a predetermined direction and a vertical direction. A plurality of different learnings in which the facial direction and the vertical direction are the predetermined direction and the vertical direction. In a learning device of a face discrimination device that learns facial features by a machine learning method using a facial image
The plurality of learning face images consist only of images of predetermined face areas determined according to the predetermined direction and the type of processing on face detection in which the face discrimination device is used,
The process on face detection using the face discrimination device is a face detection process having a pre-stage process for extracting face candidates and a post-stage process for narrowing down the extracted face candidates,
When the predetermined direction is the front and the type of processing is the pre-stage processing, the predetermined face region is a region surrounding all face contours,
When the predetermined orientation is the front and the type of processing is the subsequent processing, the predetermined face region is a region surrounding only both eyes and nose. Learning device. On the computer,
A face discriminating apparatus for discriminating whether or not an input image is a face image including a face in a predetermined direction and a vertical direction. A plurality of different learnings in which the facial direction and the vertical direction are the predetermined direction and the vertical direction. In a program that executes the process of learning facial features by machine learning using facial images,
The plurality of learning face images consist only of images of predetermined face areas determined according to the predetermined direction and the type of processing on face detection in which the face discrimination device is used,
The process on face detection using the face discrimination device is a face detection process having a pre-stage process for extracting face candidates and a post-stage process for narrowing down the extracted face candidates,
When the predetermined direction is the front and the type of processing is the pre-stage processing, the predetermined face region is a region surrounding all face contours,
The program according to claim 1, wherein the predetermined face area is an area surrounding only both eyes, nose and upper lip when the predetermined direction is the front and the processing type is the subsequent process. On the computer,
A face discriminating apparatus for discriminating whether or not an input image is a face image including a face in a predetermined direction and a vertical direction. A plurality of different learnings in which the facial direction and the vertical direction are the predetermined direction and the vertical direction. In a program that executes the process of learning facial features by machine learning using facial images,
The plurality of learning face images consist only of images of predetermined face areas determined according to the predetermined direction and the type of processing on face detection in which the face discrimination device is used,
The process on face detection using the face discrimination device is a face detection process having a pre-stage process for extracting face candidates and a post-stage process for narrowing down the extracted face candidates,
When the predetermined direction is the front and the type of processing is the pre-stage processing, the predetermined face region is a region surrounding all face contours,
The program according to claim 1, wherein the predetermined face area is an area surrounding only both eyes and nose when the predetermined direction is front and the type of processing is the subsequent processing. An input image is obtained by learning facial features by a machine learning method using a plurality of different learning face images in which the face direction and the vertical direction are the predetermined direction and the vertical direction. In a face discrimination method using a face discrimination device that discriminates whether or not a face image includes a face,
The plurality of learning face images consist only of images of predetermined face areas determined according to the predetermined direction and the type of processing on face detection in which the face discrimination device is used,
The process on face detection using the face discrimination device is a face detection process having a pre-stage process for extracting face candidates and a post-stage process for narrowing down the extracted face candidates,
When the predetermined direction is the front and the type of processing is the pre-stage processing, the predetermined face region is a region surrounding all face contours,
When the predetermined direction is the front and the type of processing is the subsequent processing, the predetermined face region is a region surrounding only both eyes, nose and upper lip. Method. An input image is obtained by learning facial features by a machine learning method using a plurality of different learning face images in which the face direction and the vertical direction are the predetermined direction and the vertical direction. In a face discrimination method using a face discrimination device that discriminates whether or not a face image includes a face,
The plurality of learning face images consist only of images of predetermined face areas determined according to the predetermined direction and the type of processing on face detection in which the face discrimination device is used,
The process on face detection using the face discrimination device is a face detection process having a pre-stage process for extracting face candidates and a post-stage process for narrowing down the extracted face candidates,
When the predetermined direction is the front and the type of processing is the pre-stage processing, the predetermined face region is a region surrounding all face contours,
A face discrimination method, wherein the predetermined face area is an area surrounding only both eyes and nose when the predetermined direction is front and the type of processing is the subsequent processing. An input image is obtained by learning facial features by a machine learning method using a plurality of different learning face images in which the face direction and the vertical direction are the predetermined direction and the vertical direction. In a face discriminating apparatus that discriminates whether or not a face image includes a face,
The plurality of learning face images consist only of images of predetermined face areas determined according to the predetermined direction and the type of processing on face detection in which the face discrimination device is used,
The process on face detection using the face discrimination device is a face detection process having a pre-stage process for extracting face candidates and a post-stage process for narrowing down the extracted face candidates,
When the predetermined direction is the front and the type of processing is the pre-stage processing, the predetermined face region is a region surrounding all face contours,
When the predetermined direction is the front and the type of processing is the subsequent processing, the predetermined face region is a region surrounding only both eyes, nose and upper lip. apparatus. An input image is obtained by learning facial features by a machine learning method using a plurality of different learning face images in which the face direction and the vertical direction are the predetermined direction and the vertical direction. In a face discriminating apparatus that discriminates whether or not a face image includes a face,
The plurality of learning face images consist only of images of predetermined face areas determined according to the predetermined direction and the type of processing on face detection in which the face discrimination device is used,
The process on face detection using the face discrimination device is a face detection process having a pre-stage process for extracting face candidates and a post-stage process for narrowing down the extracted face candidates,
When the predetermined direction is the front and the type of processing is the pre-stage processing, the predetermined face region is a region surrounding all face contours,
When the predetermined orientation is the front and the type of processing is the subsequent processing, the predetermined face area is an area surrounding only both eyes and nose. A computer learns the features of a face by a machine learning method using a plurality of different learning face images in which the face direction and the vertical direction are the predetermined direction and the vertical direction. In a program for functioning as a face discriminating device that discriminates whether or not a face image includes a face in a vertical direction,
The plurality of learning face images consist only of images of predetermined face areas determined according to the predetermined direction and the type of processing on face detection in which the face discrimination device is used,
The process on face detection using the face discrimination device is a face detection process having a pre-stage process for extracting face candidates and a post-stage process for narrowing down the extracted face candidates,
When the predetermined direction is the front and the type of processing is the pre-stage processing, the predetermined face region is a region surrounding all face contours,
The program according to claim 1, wherein the predetermined face area is an area surrounding only both eyes, nose and upper lip when the predetermined direction is the front and the processing type is the subsequent process. A computer learns the features of a face by a machine learning method using a plurality of different learning face images in which the face direction and the vertical direction are the predetermined direction and the vertical direction. In a program for functioning as a face discriminating device that discriminates whether or not a face image includes a face in a vertical direction,
The plurality of learning face images consist only of images of predetermined face areas determined according to the predetermined direction and the type of processing on face detection in which the face discrimination device is used,
The process on face detection using the face discrimination device is a face detection process having a pre-stage process for extracting face candidates and a post-stage process for narrowing down the extracted face candidates,
When the predetermined direction is the front and the type of processing is the pre-stage processing, the predetermined face region is a region surrounding all face contours,
The program according to claim 1, wherein the predetermined face area is an area surrounding only both eyes and nose when the predetermined direction is front and the type of processing is the subsequent processing. The predetermined face area is an area surrounding all face contours when the predetermined direction is horizontal and the type of the process is the pre-stage process or the post-stage process. 9. The learning device for a face discrimination device according to 8. The predetermined face area is an area surrounding all face contours when the predetermined direction is horizontal and the type of the process is the pre-stage process or the post-stage process. 10. The program according to 10. 12. The predetermined face area is an area surrounding all face contours when the predetermined direction is horizontal and the type of the process is the pre-stage process or the post-stage process. 12. The face discrimination method according to 12. The predetermined face area is an area surrounding all face contours when the predetermined direction is horizontal and the type of the process is the pre-stage process or the post-stage process. 14. The face discrimination device according to 14. The predetermined face area is an area surrounding all face contours when the predetermined direction is horizontal and the type of the process is the pre-stage process or the post-stage process. 16. The program according to 16.

Images