JP2011113464A - Apparatus and method for attribute identification and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To achieve robust and high-speed attribute recognition in a face direction of an object and accurately identify a subjective age group of the object. <P>SOLUTION: An apparatus 1 for attribute recognition includes: a learning data obtaining portion 11 for obtaining learning data; a learning face region detection portion 12 for outputting face clip image data from the learning data; a face-direction independent attribute identifier generation portion 13 for generating an attribute identifier for each face direction of an object; a face-direction independent attribute identifier storage portion 14 for storing the attribute identifier; a target image data obtaining portion 21 for obtaining target image data; an identification face region detection portion 22 for outputting the face clip image data from the target image data; a face direction estimation portion 23 for outputting an attribute-identification face direction parameter; an identification face region re-detection portion 24 for outputting the face clip image data from the target image data; and an attribute identification portion 25 for selecting an attribute identifier from a plurality of attribute identifiers stored and obtaining an identification result of the target image related to the object. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to an attribute identification device, an attribute identification method, and a program. In particular, the present invention relates to an object attribute identification device, an attribute identification method, and a program in a captured image.

In recent years, techniques for detecting a human face from an image or video and identifying the gender and age group of the person (hereinafter also referred to as “attribute identification” or “attribute estimation”) have been studied. For example, there is a method using a classifier that combines feature extraction by 2DPCA and GMM and SVM (see, for example, Non-Patent Document 1).
In the technique disclosed in Non-Patent Document 1, in the image or video, the direction in which the face of the person who is the object of attribute identification with respect to the direction of the imaging device (lens) faces (hereinafter referred to as “face direction”) is the front. In other words, when the subject of attribute identification is imaged from the front direction, it can be said to be an effective method.

“Examination of multiple classifiers for gender and age group estimation using facial images” Teruhide Hayashida, Kazuya Ueki, Tetsunori Kobayashi Shingaku Technique PRMU 2005-96, October 2005

However, the technique disclosed in Non-Patent Document 1 has a problem that the accuracy of attribute identification is remarkably lowered when the subject of attribute identification is imaged not only from the front direction but also from various directions. This is because a person's face has a three-dimensional structure, and therefore even if the faces of the same person are different in face direction, the brightness pattern on the image varies greatly.
As a simple technique for coping with the above problem, a technique may be considered in which, in the learning stage, in addition to the face image in the front direction, face images in various directions are collected and learned by one classifier. However, when face images in various directions including the front direction are learned, the variation in pattern becomes enormous as compared to the case of learning only the face images in the front direction. Also, depending on the discriminator, there is a problem that the processing time becomes extremely long.

  By the way, among the attribute identification, a technique for identifying an age group (hereinafter also referred to as “age group identification” or “age group estimation”) is a technique for identifying a sex (hereinafter referred to as “sex identification” or “sex estimation”). In general, it is very difficult. This is because the age by the subjectivity of another person (other person) (hereinafter referred to as “subjective age”) is often different from the actual age due to individual differences and makeup. Therefore, when learning a classifier for age group identification, it is considered that a better identification rate can be obtained by using a teacher signal determined based on evaluation by others. In addition, it is considered that there are many cases where it is more practically useful to consider evaluation by others. However, the evaluation by others reflects individual differences in the evaluation of the reviewer. Therefore, when the above evaluations are aggregated, the distribution varies, and it is not always possible to determine a class of one age group. . That is, it is difficult to set an appropriate teacher signal, and there is a problem that the subjective age group cannot be accurately identified in age group identification.

  The present invention has been made in view of such circumstances, and an object of the present invention is to provide a technique for realizing attribute identification that is robust and fast in processing speed with respect to the face direction of a subject. It is another object of the present invention to provide a technology that can accurately identify a subjective age group in identifying an age group of a subject.

  In order to solve the above-described problem, an attribute identification device according to an embodiment of the present invention has face image data captured from various directions as learning data, and the face of the subject of the face image data faces the subject. A learning face direction parameter indicating a direction, a learning data acquisition unit that acquires the attribute data of the subject, and a face area of the subject is detected and cut out from the face image data acquired by the learning data acquisition unit A learning face area detection unit that outputs face cut-out image data, and a plurality of the faces that are the face cut-out image data output by the learning face area detection unit and have the same learning face direction parameter An attribute for identifying the attribute of the subject for each direction in which the face of the subject faces based on the cut-out image data and the attribute data of each of the plurality of face-cut image data A face direction attribute classifier generating unit for generating a classifier, a face direction attribute classifier storage unit for storing the attribute classifier generated by the face direction attribute classifier generating unit, A target image data acquisition unit for acquiring target image data as a target, and a recognition face for detecting a face area of the subject from the target image data acquired by the target image data acquisition unit and outputting face-cut image data Based on the face cut-out image data output by the area detection unit and the recognition face area detection unit, the direction of the face of the subject of the target image is estimated, and the face of the subject faces A face direction estimation unit that outputs a face direction parameter for attribute identification indicating a direction; and a target direction from the target image data based on the face direction parameter for attribute identification output by the face direction estimation unit. A face area re-detection unit for recognition that outputs face-extracted image data that is detected and cut out again by detecting a face area of the body, and the face based on the attribute identification face direction parameter output by the face direction estimation unit One or more attribute classifiers are selected from a plurality of the attribute classifiers stored in the direction-specific attribute classifier storage unit, and the recognition is performed on the selected one or more attribute classifiers. And an attribute identification unit that inputs the face cut-out image data output from the facial area re-detection unit and acquires the identification result relating to the subject of the target image.

  In the above-described attribute identification device, the face direction estimation unit outputs a yaw angle and a pitch angle indicating the direction of the face of the subject of the target image as the attribute identification face direction parameter, and the attribute identification unit includes: When each of the plurality of attribute classifiers stored in the face direction attribute classifier storage unit is arranged in a two-dimensional space composed of the yaw angle and the pitch angle output from the face direction estimation unit One of the nearest attribute classifiers may be selected based on the Euclidean distance, and the identification result relating to the subject of the target image may be acquired.

  In the above-described attribute identification device, the face direction estimation unit outputs a yaw angle and a pitch angle indicating the direction of the face of the subject of the target image as the attribute identification face direction parameter, and the attribute identification unit includes: When each of the plurality of attribute classifiers stored in the face direction attribute classifier storage unit is arranged in a two-dimensional space composed of the yaw angle and the pitch angle output from the face direction estimation unit Two or more neighboring attribute classifiers may be selected based on the Euclidean distance, and the identification result relating to the subject of the target image may be acquired using a weighted average based on the distance.

  In the above-described attribute identification device, the learning data acquisition unit is configured to learn aggregated data obtained by aggregating the ratio of the subjective age of the subject of the facial image data obtained by previously presenting the facial image data to a large number of persons. Further acquiring as data, the face direction attribute classifier generator determines a correct / incorrect answer for each subjective age group indicated by the aggregate data based on a predetermined threshold, and sets a plurality of subjective age groups as correct answers. If it is determined, the internal value weighted according to the rate of evaluation in the subjective age group determined to be correct, or the internal value equivalent to the above rate of evaluation is passed to the attribute classifier as a teacher signal, and the attribute A discriminator may be generated.

  In order to solve the above-described problem, an attribute identification method according to another embodiment of the present invention is directed to learning image data of face image data captured from various directions, and a face of the subject of the face image data. A learning face direction parameter indicating the direction of the subject, learning data acquisition means for acquiring the attribute data of the subject, and detection of the face area of the subject from the face image data acquired by the learning data acquisition means Learning face area detecting means for outputting the face cut-out image data, and the face cut-out image data output by the learning face area detecting means, wherein the learning face direction parameters are the same. Based on the face cut-out image data and the attribute data of each of the plurality of face cut-out image data, the attributes of the subject are identified for each direction in which the face of the subject faces. An attribute classifier generating unit for generating a face direction attribute identifier, a face direction attribute classifier storing unit for storing the attribute classifier generated by the face direction attribute classifier generating unit, and a subject attribute A target image data acquisition unit that acquires target image data that is a target of identification, and a recognition that detects a face area of the subject and outputs face cut-out image data from the target image data acquired by the target image data acquisition unit Based on the face cut image data output by the face area detection means and the recognition face area detection means, the direction of the face of the subject of the target image is estimated, and the face direction of the subject A face direction estimating means for outputting an attribute identifying face direction parameter indicating the direction of the image, and based on the attribute identifying face direction parameter output by the face direction estimating means Recognizing face area re-detecting means for outputting face-cut image data obtained by detecting again the face area of the subject from the target image data, and the attribute identifying face direction parameter output by the face direction estimating means And selecting one or more attribute classifiers from the plurality of attribute classifiers stored in the face direction attribute classifier storage means, and selecting the one or more selected attributes. An attribute identifying unit that inputs the face cut image data output from the recognition face area re-detecting unit to the classifier and obtains the identification result relating to the subject of the target image; To do.

  In the attribute identification method, the face direction estimation means outputs a yaw angle and a pitch angle indicating the direction of the face of the subject of the target image as the attribute identification face direction parameter, and the attribute identification means includes: When each of the plurality of attribute classifiers stored in the face direction attribute classifier storage unit is arranged in a two-dimensional space composed of the yaw angle and pitch angle output from the face direction estimation unit One of the nearest attribute classifiers may be selected based on the Euclidean distance, and the identification result relating to the subject of the target image may be acquired.

  In the attribute identification method, the face direction estimation means outputs a yaw angle and a pitch angle indicating the direction of the face of the subject of the target image as the attribute identification face direction parameter, and the attribute identification means includes: When each of the plurality of attribute classifiers stored in the face direction attribute classifier storage unit is arranged in a two-dimensional space composed of the yaw angle and pitch angle output from the face direction estimation unit Two or more neighboring attribute classifiers may be selected based on the Euclidean distance, and the identification result relating to the subject of the target image may be acquired using a weighted average based on the distance.

  In the above-described attribute identification method, the learning data acquisition means is used for learning aggregated data obtained by aggregating the ratio of the subject's subjective age of the facial image data obtained by previously presenting the facial image data to a large number of persons. Further acquiring as data, the face direction attribute discriminator generating means determines a correct answer / incorrect answer of each subjective age group indicated by the aggregate data based on a predetermined threshold, and sets a plurality of subjective age groups as correct answers. If it is determined, the internal value weighted according to the rate of evaluation in the subjective age group determined to be correct, or the internal value equivalent to the above rate of evaluation is passed to the attribute classifier as a teacher signal, and the attribute A discriminator may be generated.

  In order to solve the above-described problem, a program according to another embodiment of the present invention provides a face image captured from various directions as learning data by a computer that controls an attribute identification device that identifies an attribute of a subject. Data, a learning face direction parameter indicating a direction in which the face of the subject of the face image data faces, a learning data acquisition step of acquiring attribute data of the subject, and the face acquired by the learning data acquisition step A learning face area detection step for outputting face cut image data extracted by detecting a face area of a subject from image data, and the face cut image data output by the learning face area detection step; The plurality of face cut-out image data having the same learning face direction parameter and the attribute data of each of the plurality of face cut-out image data. And generating an attribute classifier for identifying the attribute of the subject for each direction in which the face of the subject faces, and storing the attribute classifier for each face direction stored in a storage unit; A target image data acquisition step for acquiring target image data that is a target of the target, and a recognition area that detects a face area of the subject and outputs face-cut image data from the target image data acquired by the target image data acquisition step Based on the face cut image data output by the face area detection step and the recognition face area detection step, the direction of the face of the subject of the target image is estimated, and the face of the subject faces A face direction estimation step for outputting an attribute identification face direction parameter indicating the current direction, and the attribute identification face direction parameter output by the face direction estimation step. Based on the data, the recognition face area re-detection step for outputting the face cut-out image data extracted again by detecting the face area of the subject from the target image data, and the face direction estimation step Based on the face direction parameter for attribute identification, one or more attribute classifiers are selected from the plurality of attribute classifiers stored in the storage unit, and the selected one or more attributes are selected. An attribute identification step of inputting the face cut-out image data output by the recognition face area redetection step and acquiring an identification result relating to a subject of the target image is executed by the classifier.

  According to the present invention, it is possible to realize attribute identification that is robust and fast in processing speed with respect to the face direction of the subject. Further, the subjective age group can be accurately identified in the age group identification of the subject.

It is a functional block diagram which shows an example of a structure of the attribute identification device 1 by one Embodiment of this invention. It is explanatory drawing explaining operation | movement of the face area redetection part 24 for recognition. It is explanatory drawing explaining age group identification in the attribute identification apparatus. It is explanatory drawing explaining age group identification in the attribute identification apparatus. 5 is a flowchart showing an example of the operation of the attribute identification device 1.

  Hereinafter, an embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing an example of the configuration of an attribute identification device 1 according to an embodiment of the present invention. FIG. 2 is an explanatory diagram for explaining the operation of the recognition face area redetection unit 24.

  As shown in FIG. 1, the attribute identification device 1 includes a learning processing unit 10 and a recognition processing unit 20. The learning processing unit 10 includes a learning data acquisition unit 11, a learning face area detection unit 12, a face direction attribute discriminator generation unit 13, and a face direction attribute discriminator storage unit 14. The recognition processing unit 20 includes a target image data acquisition unit 21, a recognition face region detection unit 22, a face direction estimation unit 23, a recognition face region redetection unit 24, an attribute identification unit 25, and a result output unit 26.

  The learning data acquisition unit 11 acquires, as learning data, face image data captured from various directions, a learning face direction parameter indicating the face direction of the subject of the face image data, and subject attribute data.

  That is, the learning data acquisition unit 11 acquires a plurality of face image data captured from various directions by an imaging device (for example, a digital camera) as learning data. The learning data acquisition unit 11 supplies the face image data to the learning face area detection unit 12.

  The learning data acquisition unit 11 acquires learning face direction parameters (for example, a yaw angle value and a pitch angle value) as learning data. A mode in which the learning data acquisition unit 11 knows which learning face direction parameter is the learning face direction parameter related to which face image data (a mode in which the correspondence between the face image data and the learning face direction parameter is known) Is supplied to the face direction attribute discriminator generating unit 13. For example, the learning data acquisition unit 11 supplies the learning face direction parameter to the face direction attribute classifier generation unit 13 in association with the identification information for identifying the face image data. Note that the learning face direction parameter is obtained by manually inputting the face direction set when each face image data is captured.

  The learning data acquisition unit 11 acquires subject attribute data of each face image data as learning data. The learning data acquisition unit 11 generates an attribute classifier by face direction according to an aspect in which attribute data is attribute data related to which face image data (an aspect in which the correspondence between the face image data and the attribute data is known). To the unit 13. For example, the learning data acquisition unit 11 supplies the attribute data to the face direction-specific attribute classifier generation unit 13 in association with identification information for identifying the face image data.

  When the learning processing unit 10 generates an attribute discriminator for identifying the subjective age group, the learning data acquiring unit 11 obtains the aggregated data related to the subjective age (face image data is presented to many persons in advance). (Aggregated data obtained by totaling the ratios of the subjective ages of the subjects in the face image data) is acquired as learning data. The learning data acquisition unit 11 generates an attribute classifier by face direction according to an aspect (an aspect in which the correspondence between the face image data and the aggregated data is known) that identifies the aggregated data regarding which facial image data. To the unit 13. For example, the learning data acquisition unit 11 supplies the aggregated data to the face direction attribute classifier generation unit 13 in association with identification information for identifying the face image data. A specific example of generating a discriminator for identifying a subjective age group will be described later.

The learning face area detection unit 12 acquires face image data from the learning data acquisition unit 11 and detects a face area from the face image data. For example, the learning face area detection unit 12 may detect a face area at high speed using a statistical method such as stochastic incremental code correlation (for example, see Reference 1).
(Reference 1)
“Probabilistic incremental code correlation suitable for image matching of objects with individual differences” Yuji Mita, Toshimitsu Kaneko, Osamu Hori, IEICE Transactions D-II, Vol. J88-D-II, no. 8, pp. 1614-1623, 2005.

  The learning face area detection unit 12 that has detected the face area uses an image obtained by cutting out the face area from the face image data (hereinafter referred to as “face cut-out image data”) as face cut-out image data related to any face image data. Is supplied to the attribute classifier generating unit 13 according to face direction in a manner that can be understood (a manner in which the correspondence between face image data and face cut-out image data is known). For example, the learning face area detection unit 12 supplies the face cut-out image data to the face direction attribute classifier generation unit 13 in association with identification information for identifying the cut-out face image data. Note that the learning face area detection unit 12 of the learning processing unit 10 is less severe in processing time than the recognition face area detection unit 22 of the recognition processing unit 20. This is because the recognition face area detection unit 22 detects the face area during recognition, but the learning face area detection unit 12 detects the face area during learning of the attribute classifier.

  The face direction attribute classifier generator 13 acquires learning data (learning face direction parameters, attribute data) from the learning data acquisition unit 11. Further, the face direction attribute discriminator generating unit 13 acquires face cut-out image data from the learning face area detecting unit 12. Note that the learning face direction parameter, the attribute data, and the face cut-out image data can all be identified as information relating to which face image data. In other words, the learning face direction parameter, the attribute data, and the face cut-out image data are associated with each other.

  When the learning processing unit 10 generates an attribute classifier that identifies a subjective age group, the face-specific attribute classifier generation unit 13 acquires learning data (aggregated data) from the learning data acquisition unit 11. . It should be noted that each of the aggregate data can identify which face image data is associated with each. In other words, the learning face direction parameter, the attribute data, the total data, and the face cut-out image data are associated with each other.

  The face direction attribute discriminator generation unit 13 that has acquired the learning data acquisition unit 11 and the learning face area detection unit 12 generates an attribute discriminator for each face direction. For example, the face direction attribute discriminator generating unit 13 generates an attribute discriminator such as a gender discriminator that can discriminate between male and female by face direction.

  Specifically, the face direction attribute classifier generation unit 13 is a plurality of face cut images that are face cut image data output by the learning face region detection unit 12 and have the same learning face direction parameter. Based on the data and the attribute data of each of the plurality of face cut-out image data, an attribute discriminator for identifying the attribute of the subject is generated for each face direction of the subject. More specifically, the face direction attribute classifier generator 13 sets the face cut image data (face cut image data with matching face directions) having the same value of the learning face direction parameter as one group, The attribute discriminator is made to learn the attribute in the group (the attribute based on the attribute data corresponding to the face cut image data in each group) as a teacher signal. The face direction attribute classifier generator 13 generates the face direction attribute classifier by performing the process for all face directions. The face direction attribute classifier generator 13 adds a tag indicating information on the face direction (for example, a combination of yaw angle and pitch angle) to each attribute classifier based on the learning face direction parameter.

  The face direction attribute discriminator generation unit 13 stores all the generated attribute discriminators in the face direction attribute discriminator storage unit 14. The face direction attribute classifier generator 13 manages the generation results of the attribute classifiers and determines whether or not all face direction attribute classifiers have been generated. Note that the type of attribute classifier generated by the face direction attribute classifier generator 13 may be anything as long as it can discriminate between two classes. For example, support vector machines and feed-forward neural networks are typical.

  Further, when generating a discriminator for identifying a subjective age group, the face direction attribute classifier generating unit 13 corrects / incorrects each subjective age group (each class) indicated by the aggregate data based on a predetermined threshold. When the correct answer is determined and multiple subjective age groups are determined to be correct, weighting is performed according to the frequency of evaluation in the subjective age group determined to be correct (percentage of others evaluated as belonging to each class determined to be correct) Then, the internal division value or the internal division value equivalent to the evaluation ratio is passed as a teacher signal to the attribute discriminator to generate an attribute discriminator.

  The face direction attribute classifier storage unit 14 is a memory or HDD, and records all the attribute classifiers generated by the face direction attribute classifier generation unit 13. The attribute classifiers stored in the face-direction attribute classifier storage unit 14 are supplied to the attribute classifier 25 in response to a request from the attribute classifier 25.

  The target image data acquisition unit 21 acquires target image data that is a target of attribute identification. The target image data acquisition unit 21 supplies the target image data to the recognition face area detection unit 22 and the recognition face area redetection unit 24.

  The recognition face area detection unit 22 acquires target image data from the target image data acquisition unit 21 and detects a face area from the target image data. For example, the recognition face area detection unit 22 uses the same method as the learning face area detection unit 12 to detect a face area from the target image data. The recognizing face area detecting unit 22 that has detected the face area supplies the face cut-out image data obtained by cutting out the face area from the target image data to the face direction estimating unit 23. Note that the recognition face area detection unit 22 of the recognition processing unit 20 is more restrictive in processing time than the learning face area detection unit 12 of the learning processing unit 10. This is because the learning face area detection unit 12 detects the face area during learning of the attribute classifier, but the recognition face area detection unit 22 detects the face area during recognition. Therefore, it is preferable to detect a face area at high speed using a statistical method such as stochastic incremental code correlation.

  The face direction estimation unit 23 acquires face cut-out image data from the recognition face area detection unit 22. The face direction estimation unit 23 that has acquired the face cut-out image data estimates the face direction of the subject of the target image based on the face cut-out image data output by the recognition face area detection unit 22, and the face direction of the subject The face direction parameter for attribute identification indicating is output. The attribute identification face direction parameter relates to, for example, a yaw angle, a pitch angle, a roll angle, and a scale value. The scale value is a quantitative calculation of the size (for example, the number of dots) occupied by the face with respect to the square frame detected as the face area (for example, it may be expressed as a relative value with respect to a certain reference value). .

More specifically, the face direction estimation unit 23 detects the brightness pattern of the face area from the face cut-out image data, and estimates a face direction parameter for attribute identification based on the detected brightness pattern of the face area. For example, the face direction estimation unit 23 estimates a face direction parameter for attribute identification with high accuracy using a parameter estimation method (for example, see Reference 2) that combines principal component analysis and support vector regression. By using the method described in Reference 2, continuous face direction estimation including a face direction that has not been learned can be performed.
(Reference 2)
“Pose Estimation Method for 3D Objects Using Support Vector Regression” Shingo Ando, Yoshinori Kusachi, Akira Suzuki, Kenichi Arakawa IEICE Transactions D-II, Vol. J89-D No. 8, pp. 1840-1847, 2006.

  The face direction estimation unit 23 that has estimated the face direction supplies the attribute identification face direction parameter related to the roll angle and the scale value to the recognition face region re-detection unit 24, and the attribute identification face direction related to the yaw angle and the pitch angle. The parameter is supplied to the attribute identification unit 25.

  The recognition face area redetection unit 24 acquires target image data from the target image data acquisition unit 21. Further, the recognizing face area re-detecting unit 24 acquires the attribute identifying face direction parameter related to the roll angle and the scale value from the face direction estimating unit 23. The recognizing face area re-detecting unit 24 that has acquired the attribute identifying face direction parameter related to the target image data, the roll angle, and the scale value outputs the attribute identifying face direction parameter (roll angle and scale) output by the face direction estimating unit 23. Value), the face image of the subject is detected again from the target image data and the face cut image data cut out is output. Specifically, as shown in FIG. 2, the recognizing face area re-detecting unit 24 extracts the face area from the target image data so that the roll angle is 0 ° and the scale value is 1. The clipped face image data is output. That is, the recognizing face area re-detecting unit 24 outputs face cut-out image data obtained by cutting out the face area from the target image data again in order to correct a slight rotation or size fluctuation. The recognition face area redetection unit 24 supplies the face cut-out image data to the attribute identification unit 25.

  The attribute identification unit 25 acquires the face direction parameters for attribute identification related to the yaw angle and the pitch angle from the face direction estimation unit 23. Further, the attribute identification unit 25 acquires face cut-out image data from the recognition face area redetection unit 24. The attribute identifying unit 25 having acquired the face direction parameter for attribute identification and the face cut image data related to the yaw angle and the pitch angle, the face direction parameter for attribute identification (yaw angle and pitch angle) output by the face direction estimating unit 23 Based on the above, one or more attribute classifiers are selected from the plurality of attribute classifiers stored in the face direction attribute classifier storage unit 14.

There are various methods for selecting the attribute classifier by the attribute classifier 25. In the present embodiment, the attribute classifier 25 selects the attribute classifier according to either the criterion 1 or the selection criterion 2.
(Selection criteria 1)
Assuming that a plurality of attribute classifiers stored in the face direction attribute classifier storage unit 14 are arranged in a grid on a two-dimensional space with respect to the yaw angle and the pitch angle, the nearest attribute identification based on the Euclidean distance is performed. Select one vessel. In other words, each of the plurality of attribute classifiers stored in the face direction attribute classifier storage unit 14 is arranged in a two-dimensional space composed of the yaw angle and the pitch angle output from the face direction estimation unit 23. Then, one nearest attribute classifier is selected based on the Euclidean distance.
(Selection criteria 2)
Assume that a plurality of attribute classifiers stored in the face direction attribute classifier storage unit 14 are arranged in a grid on a two-dimensional space with respect to the yaw angle and the pitch angle, and classifiers of four neighbors based on the Euclidean distance Select. In other words, each of the plurality of attribute classifiers stored in the face direction attribute classifier storage unit 14 is arranged in a two-dimensional space composed of the yaw angle and the pitch angle output from the face direction estimation unit 23. Two or more nearby attribute classifiers are selected based on the Euclidean distance.

  When one attribute discriminator is selected according to the criterion 1, the attribute discriminating unit 25 inputs the face cut-out image data acquired from the recognition face area re-detecting unit 24 to the selected one attribute discriminator. Get the identification result. Then, the attribute identification unit 25 supplies the identification result to the result output unit 26.

When four attribute classifiers are selected according to the criterion 2, the attribute identification unit 25 inputs the face cut-out image data acquired from the recognition face area redetection unit 24 to each of the selected four attribute classifiers. Then, an identification result is obtained from each. Then, the attribute identification unit 25 calculates a final identification result from the identification results acquired from each of them (for example, calculates a final identification result related to the subject of the target image using a weighted average based on distance), The final identification result is supplied to the result output unit 26.
For example, the support vector machine or the like multiplies the sign function at the end and outputs either 1 or -1. In this embodiment, for the identification result obtained from each, the numerical value before the sign function is multiplied (temporarily N) The identification results are weighted and averaged using a method similar to bilinear interpolation used for enlargement of a digital image. Next, the sign function is multiplied and supplied to the result output unit 26 as a (final) identification result. Note that more advanced methods such as a method using a weighted average of five or more neighborhoods such as bicubic interpolation and spline interpolation can be used, and variations are various.

  The result output unit 26 acquires the identification result from the attribute identification unit 25 and outputs it.

  Hereinafter, a specific example of generating an attribute classifier for identifying a subjective age group in age group identification will be described with reference to FIGS. 3 and 4. 3 and 4 are explanatory diagrams for explaining the concept of the subjective age group. As shown in FIG. 3A, when the subjective age groups of a large number of persons are aggregated for one face image, there is a high possibility that the classes will vary among a plurality of classes (layers). For this reason, first, only a class that exceeds a% (a is a parameter value determined in advance) in the subjective age group is regarded as correct. For example, in the example shown in FIG. 3B, a class of 20 years old to 34 years old and a class of 35 years old to 49 years old are correct, but a class of 19 years old or younger and a class of 50 years old or older are incorrect.

  By the way, there are various ways of dividing the classes when dividing the subjective age group (for example, the method shown in FIG. 3 or the method of dividing every n teens). Set up a way of dividing. When setting four classes of subjective age groups (classes of 19 years old or less, classes of 20 years old to 34 years old, classes of 35 years old to 49 years old, classes of 50 years old or more), for example, as shown in FIG. , An attribute discriminator 1 for identifying 19 years or younger and 20 or older, an attribute discriminator 2 for discriminating 34 years or younger and 35 or older, and an attribute discriminator 2 for identifying 49 or younger and 50 or older. That is, if the identification results of the attribute classifiers 1, 2, and 3 are analyzed, the four classes of subjective age groups can be handled. Each attribute discriminator discriminates two classes of age groups (for example, in the case of the attribute discriminator 2 shown in FIG. 4A, a class of 34 years old or younger and a class of 35 years old or older), and outputs as 1 or -1. It can be configured with the simplest combination of discriminators.

  The problem is to deal with cases where two or more correct answers occur (for example, in the case of a face image that produces a result as shown in FIG. 3 (b)). Therefore, an internal division value calculated according to the following equation (1) may be given as a teacher signal to be passed.

Internal value = (O _S × P _S + O _B × P _B ) ÷ (P _S + P _B ) (1)
However, O output value _{when S} is determined that is smaller age S of the certain attribute identifier X, if O _B is it is judged that age B larger in the attribute identifier X , P _S is the ratio (frequency) of others who are evaluated as belonging to the largest subjective age group included in the age group S, and P _B is evaluated as belonging to the minimum subjective age group included in the age group B Percentage of others (frequency.

Specifically, in the case of the attribute discriminator 2, as shown in FIG. 4A, the output value O _S = −1 when it is determined that it is the smaller age group S (class of 34 years old or less). The output value O _B = 1 when it is determined that it is the larger age group B (class 35 years or older), as shown in FIG. 3A, the age group S (class 34 years or younger) Frequency P _S = 0.6 of others evaluated as belonging to the largest included subjective age group (classes 20 to 34 years old), minimum subjective age group included in age group B (classes 35 years and older) ( The frequency P _B = 0.3 of others evaluated as belonging to the class of 35 to 49 years old. Therefore, the internal value (teacher signal) related to the attribute discriminator 2 is (−1 × 0.6 + 1 × 0.3) ÷ (0...) As shown in FIG. 6 + 0.3) = − 0.333.

Similarly, in the case of the attribute discriminator 1, as shown in FIG. 4 (a), the output value O _S = −1 when the lower age group S (class of 19 years or less) is discriminated is large. The output value O _B = 1 when it is determined that the other age group B (class 20 years or older) is included in the age group S (class 19 years or younger) as shown in FIG. Frequency P _S = 0.1 of others evaluated as belonging to the maximum subjective age group (classes of 19 years old and under), minimum subjective age group (20 to 34) included in age group B (classes of 20 years old and over) The frequency P _B = 0.6 of others evaluated as belonging to the age class). Therefore, the internal value (teacher signal) related to the attribute discriminator 1 is (−1 × 0.1 + 1 × 0.6) ÷ (0...) As shown in FIG. 1 + 0.6) = 0.714.

Similarly, in the case of the attribute discriminator 3, as shown in FIG. 4 (a), the output value O _S = −1 when the lower age group S (class of 49 years old or less) is discriminated is large. Output value O _B = 1 when it is determined that the other age group B (class of 50 years or older) is included in the age group S (class of 49 years or younger) as shown in FIG. Frequency P _S = 0.3 of others evaluated as belonging to the largest subjective age group (35 to 49 years old or lower class), minimum subjective age group (50 included in age group B (class 50 years old or older)) The frequency P _B = 0 of others who are evaluated as belonging to a class aged over). Therefore, the internal value (teacher signal) related to the attribute classifier 3 is (−1 × 0.3 + 1 × 0) ÷ (0.3 + 0) as shown in FIG. = -1.

Further, the internal value may be calculated according to the following formula (2), considering each frequency (ratio) as equivalent. That is, in the above formula (1), P _S = P _B = 0.5 may be set.

Internal value = (O _S × 0.5 + O _B × 0.5) ÷ (0.5 + 0.5) (2)

  According to the above equation (2), for example, the internal value (teacher signal) related to the attribute discriminator 2 is (−1 × 0.5 + 1 × 0.5) ÷ (0.5 + 0.5) = 0. Is calculated.

  In addition, it is known from a basic experiment or the like that if the parameter a is appropriately set, it is rare that two or more correct answers occur without adjacent classes. Therefore, the case where multiple correct classes are not adjacent is ignored. If data such that a plurality of correct classes do not adjoin each other appears, the data is excluded from the learning data. In addition, since it is impossible to deny the possibility that an inconsistent result (for example, a result of being 19 years old or younger and 35 years old or older) is output from the attribute discriminator, a rule when an inconsistent result is output (for example, It is preferable to set in advance a rule such as “always give priority to a class with a lower age”.

  Subsequently, the operation of the pattern recognition method 1 will be described with reference to FIG. The flowchart shown in FIG. 5A is an operation flow of the learning processing unit 10. The flowchart shown in FIG. 5B is an operation flow of the recognition processing unit 20. The flowchart shown in FIG. 5C is an operation flow when the face direction attribute discriminator generation unit 13 creates a teacher signal related to the subjective age.

  In FIG. 5A, the learning data acquisition unit 11 acquires learning data (face image data, learning face direction parameters, attribute data, and total data) (step S11). The learning data acquisition unit 11 supplies the face image data to the learning face area detection unit 12, and supplies the learning face direction parameter and attribute data to the face direction-specific attribute classifier generation unit 13.

  Next, the learning face area detection unit 12 detects a face area from the face image data (step S12). The learning face area detection unit 12 supplies the face cut-out image data to the face direction attribute classifier generation unit 13.

  Next, the face-specific attribute classifier generator 13 generates an attribute classifier (step S13). Specifically, the face direction attribute discriminator generating unit 13 executes the flowchart of FIG. 5C and generates an attribute discriminator using a teacher signal. The face direction attribute discriminator generation unit 13 stores the generated attribute discriminator in the face direction attribute discriminator storage unit 14 (step S14).

  Next, the face direction attribute classifier generator 13 determines whether all face direction attribute classifiers have been generated (step S15). If the face direction attribute classifier generator 13 determines that all face direction attribute classifiers have not been generated (step S15: No), the process returns to step S11. On the other hand, if the face direction attribute classifier generator 13 determines that all face direction attribute classifiers have been generated (step S15: Yes), the flowchart shown in FIG. 5A ends.

  In FIG.5 (b), the target image data acquisition part 21 acquires target image data (step S21). The target image data acquisition unit 21 supplies the target image data to the recognition face area detection unit 22 and the recognition face area redetection unit 24.

  Next, the recognition face area detection unit 22 detects a face area from the target image data (step S22). The recognition face area detection unit 22 supplies the face cut image data to the face direction estimation unit 23.

  Next, the face direction estimation unit 23 estimates the face direction of the target image based on the face cut-out image data output by the recognition face area detection unit 22 (step S23). The face direction estimation unit 23 supplies attribute recognition face direction parameters (roll angle and scale value) indicating the face direction of the subject of the target image to the recognition face region redetection unit 24, and determines the face direction of the subject of the target image. The attribute identification face direction parameters (yaw angle and pitch angle) shown are supplied to the attribute identification unit 25.

  Next, the recognizing face area redetection unit 24 cuts out the face area again from the target image data based on the attribute identification face direction parameters (roll angle and scale value) output by the face direction estimation unit 23 (step S24). ). The recognition face area redetection unit 24 supplies the face cut-out image data to the attribute identification unit 25.

  Next, the attribute identifying unit 25, based on the attribute identifying face direction parameters (yaw angle and pitch angle) output by the face direction estimating unit 23, a plurality of attributes stored in the face direction-specific attribute classifier storage unit 14. One or more attribute classifiers are selected from the attribute classifiers (step S25). Then, the attribute identification unit 25 inputs the face cut-out image data acquired from the recognition face area redetection unit 24 to the selected attribute classifier, and obtains the identification result (step S26). The result output unit 26 acquires and outputs the identification result from the attribute identification unit 25 (step S27). Then, the flowchart shown in FIG. 5B ends.

  In FIG. 5C, the face direction attribute classifier generator 13 acquires learning data (aggregated data) from the learning data acquisition unit 11. Specifically, the face direction attribute classifier generator 13 acquires subjective age group frequency data from the learning data acquisition unit 11, for example, as shown in FIG. 3A (step S31).

  Next, as shown in FIG. 3B, the face direction attribute classifier generator 13 determines the correct / incorrect answer for each class based on a preset threshold value a (step S32). When there is a variation in the subjective age group frequency data for the correct class, the face direction attribute classifier generator 13 calculates a weighted internal value based on the frequency of the adjacent correct classes as described above. The teacher signal is calculated and created (step S33) (see, for example, FIG. 4B).

  FIG. 3B shows the case where the correct answer spans two classes. Even when the correct answer spans three or more classes, the total value of the ratios of the frequency of the correct class corresponding to the teacher signal “−1” Therefore, the internal division value can be easily calculated from the total value of the proportions of the correct class frequencies corresponding to the teacher signal “1”. Further, as described above, the internal division value may be calculated with all the frequency ratios being equivalent. When there is no variation in the subjective age group frequency data for the correct class, 1 or −1 is appropriately output to each attribute classifier as a teacher signal as usual. Also, ignore the class that is incorrect.

  The face direction attribute classifier generator 13 determines whether teacher signals have been created for all persons (step S34). If it is determined that the face direction attribute classifier generator 13 has not created teacher signals for all persons (step S34: No), the process returns to step S31. On the other hand, if the face direction attribute classifier generator 13 determines that all face direction attribute classifiers have been generated (step S34: Yes), the flowchart shown in FIG. 5C ends.

As described above, according to the present embodiment, the yaw angle, the pitch angle, the roll angle, and the scale value indicating the posture of the face after the face detection are estimated, and the face area is cut out again from the input image based on the result. By selecting one or a plurality of appropriate classifiers and integrating the results using a weighted average or the like, it is possible to realize attribute identification that is robust and fast in processing speed with respect to the face direction of the subject.
In addition, since an appropriate teacher signal is calculated and set, the subjective age group can be accurately identified. Specifically, in age group identification, the correct / incorrect answer is determined through a certain threshold based on the frequency distribution of the subjective age group that has been pre-aggregated, and further, an internal value with the ratio of the correct class frequency as a weight (Individual values with weights of the proportion of others evaluated as belonging to each class determined to be correct), or by passing the divided values with all frequency proportions as equivalent to the discriminator as subjective signals, The age group can be accurately identified.

  Note that a program for executing each process of the attribute identification device 1 according to an embodiment of the present invention is recorded on a computer-readable recording medium, and the program recorded on the recording medium is read into a computer system and executed. By doing so, you may perform the various process mentioned above which concerns on each process of the attribute identification device 1 by one Embodiment of this invention. Here, the “computer system” may include an OS and hardware such as peripheral devices. Further, the “computer system” includes a homepage providing environment (or display environment) if a WWW system is used. The “computer-readable recording medium” means a flexible disk, a magneto-optical disk, a ROM, a writable nonvolatile memory such as a flash memory, a portable medium such as a CD-ROM, a hard disk built in a computer system, etc. This is a storage device.

  Further, the “computer-readable recording medium” means a volatile memory (for example, DRAM (Dynamic DRAM) in a computer system that becomes a server or a client when a program is transmitted through a network such as the Internet or a communication line such as a telephone line. Random Access Memory)), etc., which hold programs for a certain period of time. The program may be transmitted from a computer system storing the program in a storage device or the like to another computer system via a transmission medium or by a transmission wave in the transmission medium. Here, the “transmission medium” for transmitting the program refers to a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line. The program may be for realizing a part of the functions described above. Furthermore, what can implement | achieve the function mentioned above in combination with the program already recorded on the computer system, and what is called a difference file (difference program) may be sufficient.

  The embodiment of the present invention has been described in detail with reference to the drawings. However, the specific configuration is not limited to this embodiment, and includes designs and the like that do not depart from the gist of the present invention.

  DESCRIPTION OF SYMBOLS 1 Attribute identification device, 10 Learning processing part, 11 Learning data acquisition part, 12 Learning face area detection part, 13 Face direction attribute discriminator production | generation part, 14 Face direction attribute discriminator memory | storage part, 20 Recognition processing part, 21 target image data acquisition unit, 22 recognition face area detection unit, 23 face direction estimation unit, 24 recognition face area redetection unit, 25 attribute identification unit, 26 result output unit

Claims (9)

Learning data acquisition unit for acquiring face image data captured from various directions, learning face direction parameters indicating the direction of the face of the subject of the face image data, and attribute data of the subject as learning data When,
A learning face area detection unit that outputs face cut-out image data obtained by detecting and cutting out a face area of a subject from the face image data acquired by the learning data acquisition unit;
The face cut image data output by the learning face area detection unit, the plurality of face cut image data having the same learning face direction parameter, and each of the plurality of face cut image data A face direction attribute classifier generator for generating an attribute classifier for identifying the attribute of the subject for each direction in which the face of the subject is facing based on the attribute data;
A face direction attribute classifier storage unit that stores the attribute classifier generated by the face direction attribute classifier generation unit;
A target image data acquisition unit that acquires target image data that is a target for subject identification;
A recognition face area detection unit that detects a face area of a subject and outputs face cut-out image data from the target image data acquired by the target image data acquisition unit;
Based on the face cut-out image data output by the recognition face area detection unit, the direction of the face of the subject of the target image is estimated, and attribute identification indicating the direction of the face of the subject A face direction estimation unit that outputs a face direction parameter for use;
Based on the attribute identification face direction parameter output by the face direction estimation unit, a recognition face area re-output for outputting face cut-out image data obtained by detecting again the face area of the subject from the target image data. A detection unit;
Based on the attribute identification face direction parameter output by the face direction estimation unit, one or more of the attributes from among the plurality of the attribute classifiers stored in the face direction attribute classifier storage unit The discriminator is selected, and the face cut-out image data output from the recognition face area redetection unit is input to the selected one or more attribute discriminators, and the discrimination result relating to the subject of the target image An attribute identification device comprising: an attribute identification unit that acquires The face direction estimation unit
Outputting a yaw angle and a pitch angle indicating the direction of the face of the subject of the target image as the face direction parameter for attribute identification;
The attribute identification unit
When each of the plurality of attribute classifiers stored in the face direction attribute classifier storage unit is arranged in a two-dimensional space composed of the yaw angle and the pitch angle output from the face direction estimation unit The attribute identification device according to claim 1, wherein one of the nearest attribute classifiers is selected based on a Euclidean distance, and an identification result relating to a subject of the target image is acquired. The face direction estimation unit
Outputting a yaw angle and a pitch angle indicating the direction of the face of the subject of the target image as the face direction parameter for attribute identification;
The attribute identification unit
When each of the plurality of attribute classifiers stored in the face direction attribute classifier storage unit is arranged in a two-dimensional space composed of the yaw angle and the pitch angle output from the face direction estimation unit The two or more neighboring attribute classifiers are selected based on the Euclidean distance, and the identification result relating to the subject of the target image is obtained using a weighted average based on the distance. The attribute identification device described in 1. The learning data acquisition unit
Aggregate data obtained by aggregating the ratio of the subjective age of the subject of the face image data obtained by previously presenting the face image data to a large number of persons is further acquired as learning data,
The face direction-specific attribute classifier generator is
Based on a predetermined threshold, the correct answer / incorrect answer of each subjective age group indicated by the aggregated data is determined, and when a plurality of subjective age groups are determined to be correct, the ratio of evaluation in the subjective age group determined to be correct 4. The attribute discriminator is generated by passing the weighted internal value or the internal value equivalent to the evaluation ratio as a teacher signal to the attribute discriminator. The attribute identification device according to any one of the above. Learning data acquisition means for acquiring face image data captured from various directions, learning face direction parameters indicating the direction of the face of the subject of the face image data, and attribute data of the subject as learning data When,
Learning face area detection means for detecting face area of a subject and outputting cut-out face image data from the face image data acquired by the learning data acquisition means;
A plurality of the face cut-out image data output by the learning face area detecting means and having the same learning face direction parameter, and each of the plurality of face cut-out image data An attribute classifier generating unit for each face direction that generates an attribute classifier for identifying the attribute of the subject for each direction in which the face of the subject is facing based on the attribute data;
A face direction attribute classifier storage unit for storing the attribute classifier generated by the face direction attribute classifier generation unit;
Target image data acquisition means for acquiring target image data which is a target for subject identification;
Recognizing face area detecting means for detecting a face area of a subject from the target image data acquired by the target image data acquiring means and outputting face-cut image data;
Based on the face cut-out image data output by the recognition face area detection means, the direction of the subject's face facing the target image is estimated, and attribute identification indicating the direction of the subject's face is directed A face direction estimating means for outputting a face direction parameter for use;
Based on the attribute identification face direction parameter output by the face direction estimating means, a face area for recognition is regenerated to output face cut-out image data obtained by detecting again the face area of the subject from the target image data. Detection means;
Based on the attribute identification face direction parameter output by the face direction estimation unit, one or more of the attributes from among the plurality of the attribute classifiers stored in the face direction attribute classifier storage unit Selecting a discriminator, inputting the face cut-out image data output from the recognition face area re-detecting means to the selected one or more attribute discriminators, and identifying results relating to the subject of the target image An attribute identification method comprising: attribute identification means for acquiring The face direction estimating means includes
Outputting a yaw angle and a pitch angle indicating the direction of the face of the subject of the target image as the face direction parameter for attribute identification;
The attribute identifying means includes
When each of the plurality of attribute classifiers stored in the face direction attribute classifier storage unit is arranged in a two-dimensional space composed of the yaw angle and pitch angle output from the face direction estimation unit The attribute identification method according to claim 5, wherein one of the nearest attribute classifiers is selected based on a Euclidean distance, and an identification result relating to a subject of the target image is acquired. The face direction estimating means includes
Outputting a yaw angle and a pitch angle indicating the direction of the face of the subject of the target image as the face direction parameter for attribute identification;
The attribute identifying means includes
When each of the plurality of attribute classifiers stored in the face direction attribute classifier storage unit is arranged in a two-dimensional space composed of the yaw angle and pitch angle output from the face direction estimation unit 6. The method according to claim 5, wherein two or more neighboring attribute classifiers are selected based on a Euclidean distance, and an identification result relating to a subject of the target image is obtained using a weighted average based on the distance. The attribute identification method described in 1. The learning data acquisition means includes
Aggregate data obtained by aggregating the ratio of the subjective age of the subject of the face image data obtained by previously presenting the face image data to a large number of persons is further acquired as learning data,
The face direction attribute discriminator generating means comprises:
Based on a predetermined threshold, the correct answer / incorrect answer of each subjective age group indicated by the aggregated data is determined, and when a plurality of subjective age groups are determined to be correct, the ratio of evaluation in the subjective age group determined to be correct 8. The attribute discriminator is generated by passing the weighted internal value or the internal value equivalent to the evaluation ratio as a teacher signal to the attribute discriminator. The attribute identification method according to any one of the above. To a computer that controls an attribute identification device that identifies an attribute of a subject,
Learning data acquisition step for acquiring face image data captured from various directions, a learning face direction parameter indicating the direction of the face of the subject of the face image data, and attribute data of the subject as learning data When,
A learning face area detecting step for detecting face area of the subject and extracting the face cut image data extracted from the face image data acquired by the learning data acquisition step;
The face cut-out image data output by the learning face area detection step, the plurality of face cut-out image data having the same learning face direction parameter, and each of the plurality of face cut-out image data Generating an attribute classifier for identifying the attribute of the subject for each direction in which the face of the subject is facing based on the attribute data, and storing the attribute classifier by face direction in a storage unit;
A target image data acquisition step for acquiring target image data which is a target for subject identification;
A recognition face area detecting step of detecting a face area of a subject and outputting face cut-out image data from the target image data acquired by the target image data acquisition step;
Based on the face cut-out image data output by the recognition face area detection step, the direction of the face of the subject of the target image is estimated, and attribute identification indicating the direction of the face of the subject A face direction estimating step for outputting a face direction parameter for use;
Based on the attribute identification face direction parameter output in the face direction estimation step, a recognition face area re-output that outputs face cut-out image data obtained by detecting again the face area of the subject from the target image data. A detection step;
Based on the attribute identification face direction parameter output by the face direction estimation step, select one or more of the attribute classifiers from the plurality of attribute classifiers stored in the storage unit, Attribute identification step of inputting the face cut-out image data output by the recognizing face area re-detection step to the selected one or more attribute classifiers and acquiring an identification result relating to the subject of the target image A program characterized by causing

Images