JP2013254254A - Identification device, control method and program thereof - Google Patents

Abstract

PROBLEM TO BE SOLVED: To increase category identification accuracy.SOLUTION: An individual feature extraction part 102 extracts features from first data. A back ground model generation part 103 generates a back ground model associated with the features extracted from the first data. A noise determination part 104 determines a noise distribution in the back ground model. Also, the individual feature extraction part 102 extracts features from second data. A model generation part 1-6 generates a model associated with the features extracted from the second data on the basis of distribution other than the noise distribution of the back ground model. An identification apparatus generation part 108 generates an identification apparatus for identifying a category on the basis of the model.

Description

  The present invention relates to a technique for performing category identification of data.

  For the task of detecting higher-order features such as “airplane” and “singer” from image data, a recognition method using a statistical method based on a mixed Gaussian distribution (GMM) of SIFT features and MFCC features is known. (See Non-Patent Document 1). In this method, higher-order features are detected by GMM Supervector SVM (GS-SVM). In GS-SVM, GMM of each shot image data is obtained, and learning and identification are performed by SVM using an RBF kernel defined from the distance between GMMs. When generating a GMM for each shot image data, a model called a universal background model (UBM) is first generated by sampling features from the entire shot image data regardless of the category. Next, the parameters of the shot image data are estimated by adaptive to a maximum posterior probability (Maximum A Posteriori; MAP).

Nakanobu Inoue, Yasuhiko Saito, Koichi Shinoda, Sadaaki Furui, "Multimodal higher-order feature detection for large-scale video resources," IEICE Transactions D, vol.J93-D, no.12, pp. 2633-2644, Dec, 2010.

  However, in the technique disclosed in Non-Patent Document 1, a universal background model (UBM) is generated using the entire shot image data regardless of the category, and a MAP-adapted GMM is configured from the UBM for each category. I have to. Therefore, when attention is paid to a certain category, there is a distribution that is not generated from features extracted from the category among the distributions included in the UBM. For this category, the distribution is a noise distribution that does not contribute to recognition. Therefore, in the technique disclosed in Non-Patent Document 1, identification is performed in consideration of noise distribution, and such noise distribution hinders improvement in category identification accuracy.

  Therefore, an object of the present invention is to improve category identification accuracy.

  The identification device of the present invention is extracted by first data input means for inputting first data, first feature extraction means for extracting features from the first data, and the first feature extraction means. Background model generation means for generating a background model for the features, a determination means for determining a noise distribution in the background model, a second data input means for inputting second data, and the second Second feature extraction means for extracting features from the data, and model generation for generating a model for the features extracted by the second feature extraction means based on a distribution other than the noise distribution in the background model And classifier generating means for generating a classifier for identifying a category based on the model.

  According to the present invention, it is possible to improve category identification accuracy.

It is a figure which shows the structure of the identification device which concerns on embodiment of this invention. It is the figure which extracted and showed the structure in connection with a background model production | generation process from the structure of the identification device which concerns on this embodiment shown in FIG. It is a flowchart which shows a background model production | generation process. It is the figure which extracted and showed the structure in connection with the learning process of the discrimination device according to category from the structure of the identification device which concerns on this embodiment shown in FIG. It is a flowchart which shows the learning process of the discrimination device according to category. It is the figure which extracted and showed the structure in connection with the identification process of a category from the structure of the identification apparatus which concerns on this embodiment shown in FIG. It is a flowchart which shows the identification process of a category.

  DESCRIPTION OF EXEMPLARY EMBODIMENTS Hereinafter, preferred embodiments to which the invention is applied will be described in detail with reference to the accompanying drawings.

  FIG. 1 is a diagram illustrating a configuration of an identification device according to an embodiment of the present invention. In FIG. 1, reference numeral 100 denotes an identification device. As illustrated in FIG. 1, the identification device 100 includes a data input unit 101, an individual feature extraction unit 102, a background model generation unit 103, a noise determination unit 104, a storage device 105, a model generation unit 106, and a higher-order feature generation unit 107. The discriminator generating unit 108 and the discriminating unit 109 are configured. Note that the identification device 100 is configured by, for example, a personal computer. That is, the data input unit 101, the individual feature extraction unit 102, the background model generation unit 103, the noise determination unit 104, the model generation unit 106, the higher-order feature generation unit 107, the classifier generation unit 108, and the identification unit 109 described above are: This is a functional configuration realized by the CPU in the identification device 100 loading a necessary program and data from a nonvolatile recording medium such as a ROM into the RAM and executing the program. The storage device 105 has a configuration corresponding to a rewritable recording medium such as a RAM.

  The identification device 100 according to the present embodiment executes (I) background model generation processing, (II) category-specific classifier learning processing, and (III) category identification processing. In the following, in order to make the explanation easy to understand, the processing of the identification device 100 according to the present embodiment includes (I) background model generation processing, (II) category-specific classifier learning processing, and (III) category The description will be divided into identification processing.

  First, (I) background model generation processing will be described. The background model generated here is used in both (II) the learning process of the category classifier and (I) the classification process of the category.

  FIG. 2 is a diagram showing an extracted configuration related to the background model generation process from the configuration of the identification apparatus 100 according to the present embodiment shown in FIG. Hereinafter, a configuration related to the background model generation process among the configurations of the identification device 100 will be described as a background model generation device.

  In FIG. 2, reference numeral 200 denotes a background model generation apparatus. As illustrated in FIG. 2, the background model generation device 200 includes a data input unit 101, an individual feature extraction unit 102, a background model generation unit 103, a noise determination unit 104, and a storage device 105.

  Next, processing of the background model generation apparatus 200 will be described with reference to FIG. In step S301, the data input unit 101 inputs a plurality of image data necessary for generating a background model and annotation data in which category information to which each image data belongs is described. The image data input here is shot image data of several seconds to several tens of seconds. Examples of the category information include scene information such as birthday parties and parades, and subject information such as airplanes and cars that appear in the image data. It is assumed that category information that may be input here is determined in advance. Note that the processing for inputting image data in step S301 is a processing example of the first data input means.

  In step S302, the individual feature extraction unit 102 extracts individual features from each image data input in step S301. Each individual feature is represented by a vector. In the present embodiment, SIFT features that are image features and MFCC features that are acoustic features are extracted as individual features. Then, the individual feature extraction unit 102 links the extracted individual features with the category information that is the annotation data input in step S301. Step S302 is a processing example of the first feature extraction unit.

  In step S303, the background model generation unit 103 generates a background model for each individual feature type, that is, for each of the SIFT feature and the MFCC feature. In this embodiment, a Gaussian mixture distribution (GMM) is generated as a background model. Hereinafter, the GMM serving as the background model is referred to as a universal background model (UBM). When generating the UBM, the background model generation unit 103 stores, in the storage device 105, an average vector and a variance-covariance matrix that are parameters for each distribution constituting the UBM.

  In step S304, the noise determination unit 104 determines the noise distribution in the UBM for each category based on the category information linked to the individual features. Specifically, for example, when determining the noise distribution for the “car” category, the noise determination unit 104 focuses on the individual features linked to the “car” category among the individual features for which the UBM is generated in step S303. To do. Hereinafter, this is referred to as a “car” feature vector. For each distribution constituting the UBM, the noise determination unit 104 calculates the frequency or ratio of the “car” feature vector for all the individual features that make up the distribution. Then, the noise determination unit 104 determines that the distribution with the calculated frequency or ratio is equal to or less than a predetermined threshold is the noise distribution for the “car” category. A plurality of noise distributions may be discriminated, or none may be discriminated. Although the “car” category has been described as an example here, the noise distribution is similarly determined for other categories. The noise determination unit 104 stores the noise distribution thus determined for each category in the storage device 105.

  Next, (II) the learning process of the category discriminator will be described. For category identification of image data, it is necessary to learn a category classifier. Here, in order to make the explanation easy to understand, the learning target is described as the “car” category.

  FIG. 4 is a diagram showing an excerpt of the configuration related to the learning process for the category-specific classifier from the configuration of the identification device 100 according to the present embodiment shown in FIG. Below, the structure in connection with the learning process of the discriminator classified by category among the configurations of the discriminating apparatus 100 will be referred to as a discriminator classified by category.

  In FIG. 4, reference numeral 400 denotes a category discriminator learning device. As shown in FIG. 4, the category classifier learning device 400 includes a data input unit 101, an individual feature extraction unit 102, a model generation unit 106, a higher-order feature generation unit 107, a classifier generation unit 108, and a storage device 105. Is done.

  Next, the processing of the category discriminator learning device 400 will be described with reference to FIG. In step S501, the data input unit 101 inputs a plurality of image data and annotation data in which the category information to which each image data belongs is necessary for the learning process of the category discriminator. Similar to step S301, the image data input here is shot image data of several seconds to several tens of seconds. In step S502, the individual feature extraction unit 102 extracts individual features from the image data input in step S501. Here, the SIFT feature that is an image feature and the MFCC feature that is an acoustic feature are extracted as individual features as in the background model generation process described above. Then, the individual feature extraction unit 102 links the extracted individual features to the category information that is the annotation data input in step S501. Note that the process of inputting image data in step S501 is a process example of the second data input unit. Step S502 is a processing example of the second feature extraction unit.

  In step S503, the model generation unit 106 sets the “car” category model as a positive example model and the “non-vehicle” as a negative example model for each individual feature type, that is, for each SIFT feature and MFCC feature. A model of the category. Here, a Gaussian mixture distribution (GMM) is generated as a model. The probability density function p (x) of GMM is given by the following equation 1.

Here, x∈R ^d is an individual feature of dimension d. K is the number of mixtures (number of distributions). k is a subscript for specifying the distribution. w _k , μ _k, and Σ _k are the weight coefficient, average vector, and variance-covariance matrix of the k-th distribution, respectively. The generation of the GMM here is performed by adapting the parameter from the UBM to Maximum A Posteriori (MAP). Further, the model generation unit 106 assumes that the GMM weighting coefficient w _k and the variance-covariance matrix Σ _k are common to all the image data, and the individual feature extracted from one image data in the “car” category. The average vector of the set is estimated by the following equation 2.

Here, g _k is a Gaussian probability density function having a mean vector μ _k and a variance-covariance matrix Σ _k . τ is a parameter for determining the degree of dependence on the prior distribution. c _ik is a burden factor of the k-th mixing element for the individual feature x _i . Similarly, the model generation unit 106 similarly estimates the average vector of the set Y _F of individual features extracted from the image data of the “non-car” category.

  At this time, the model generation unit 106 reads the noise distribution determined for each category in step 304 from the storage device 105. When generating a GMM by MAP adaptation from the UBM, the model generation unit 106 does not perform MAP adaptation for the noise distribution, but performs MAP adaptation only for a distribution other than the noise distribution. That is, when there are m noise distributions, the number of distributions to which MAP adaptation is performed is K−m.

In step S <b> 504, the high-order feature generation unit 107 generates a high-order feature φ (X _F ) based on the model generated by the model generation unit 106 using the following Equation 3. Similarly, the high-order feature generation unit 107 also generates a high-order feature φ (Y _F ) based on the model generated by the model generation unit 106. Each higher-order feature φ (X _F ), φ (Y _F ) is represented by a vector. X _F = {x _i } ^N _{i = 1} is a set of individual features (F is a type of individual features) extracted from one image data of the “car” category. Y _F = {y _i } ^N _{i = 1} is a set of individual features extracted from one image data of the “non-car” category.

Higher-order features φ (X _F ) or φ (Y _F ) are generated for each of the plurality of image data input in step S501. In step S505, the classifier generation unit 108 generates a category-specific classifier (discriminant function) f _F for the “car” category by learning. Learning of the category classifier is performed for each type of individual feature. The discriminator generation unit 108 learns by SVM that has a plurality of higher-order features φ (X _F ) and a plurality of higher-order features φ (Y _F ) as inputs. The category-specific classifiers f _F generated by the classifier generation unit 108 are stored in the storage device 105.

  Next, (III) category identification processing will be described. Here, in order to make the explanation easy to understand, it is assumed that the image data to be identified is in the “car” category. That is, here, it is identified whether or not a car is shown in the image data to be identified.

  FIG. 6 is a diagram showing a configuration related to category identification processing extracted from the configuration of the identification apparatus 100 according to the present embodiment shown in FIG. Hereinafter, a configuration related to the category identification process among the configurations of the identification device 100 will be referred to as a category identification device 600.

  In FIG. 6, reference numeral 600 denotes a category identification device. As shown in FIG. 6, the category identification device 600 includes a data input unit 101, an individual feature extraction unit 102, a model generation unit 106, a higher-order feature generation unit 107, an identification unit 109, and a storage device 105.

  Next, processing of the category identification device 600 will be described with reference to FIG. In step S701, the data input unit 101 inputs image data to be identified. Similar to step S301, the image data input here is shot image data of several seconds to several tens of seconds. Generally, image data to be identified does not have annotation data. In step S702, the individual feature extraction unit 102 extracts individual features from the image data input in step S701. Here, as in the background model generation process and the category classifier learning process described above, the SIFT feature that is an image feature and the MFCC feature that is an acoustic feature are individually featured from the image data input in step S701. Extracted as Note that the process of inputting image data in step S701 is a process example of the third data input unit.

  In step S703, the model generation unit 106 generates a model for each type of individual feature, that is, for each of the SIFT feature and the MFCC feature. The model is generated by the same method as the learning process for the category classifier. That is, the model generation unit 106 generates a Gaussian mixture distribution (GMM) as a model. The probability density function of GMM is given by Equation 1 above. The generation of the GMM here is performed by adapting the parameter from the UBM to Maximum A Posteriori (MAP). Further, the model generation unit 106 assumes that the GMM weighting coefficient and the variance matrix are common to all the image data, and estimates only the average vector by the above equation 2. Further, the model generation unit 106 reads out the noise distribution for the “car” category determined in step S <b> 304 from the storage device 105. When generating a GMM by MAP adaptation from the UBM, the model generation unit 106 does not perform MAP adaptation for the noise distribution, but performs MAP adaptation only for a distribution other than the noise distribution. That is, when there are m noise distributions, the number of distributions to which MAP adaptation is performed is K−m.

In step S <b> 704, the high-order feature generation unit 107 generates a high-order feature φ (Z _F ) using Equation 3 above. The higher-order feature φ (Z _F ) is expressed by a vector. In step S705, the identification unit 109 identifies whether or not a car is shown in the image data input in step S701. In this identification processing, the identification unit 109 uses a category-specific classifier for the “car” category generated in step S505, and calculates an identification function value for each type of individual feature. Next, the identification unit 109 calculates a weighted sum of the identification function values according to the following expression 4, and calculates a final score.

Here, F is the type of individual feature (here, SIFT feature and MFCC feature). f _F is a classifier for the individual feature F. α _F is a weighting factor. α _F is selected so that Average precision is maximized in validation set.

  In the present embodiment, the category identifier is generated by excluding the noise distribution and the category of the shot image data is identified, so that the category identification accuracy can be improved.

  The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, or the like) of the system or apparatus reads the program. It is a process to be executed.

  DESCRIPTION OF SYMBOLS 100: Identification apparatus, 101: Data input part, 102: Individual feature extraction part, 103: Background model generation part, 104: Noise discrimination | determination part, 105: Memory | storage device, 106: Model generation part, 107: Higher-order feature generation part , 108: classifier generating unit, 109: classifying unit, 200: background model generating unit, 400: classifier classifier learning device, 600: category classifying device

Claims (7)

First data input means for inputting first data;
First feature extraction means for extracting features from the first data;
Background model generation means for generating a background model for the features extracted by the first feature extraction means;
Discriminating means for discriminating a noise distribution in the background model;
Second data input means for inputting second data;
Second feature extraction means for extracting features from the second data;
Model generation means for generating a model for the feature extracted by the second feature extraction means based on a distribution other than the noise distribution in the background model;
A classifier generating unit configured to generate a classifier for identifying a category based on the model;   The said discrimination | determination means discriminate | determines the said noise distribution for every category, The said model production | generation means produces | generates a model for every category based on distribution other than the noise distribution of each category. The identification device described.   The determining means calculates, for each distribution included in the background model, the frequency or ratio of the characteristics of the category to be identified that constitutes the distribution, and calculates the distribution in which the calculated frequency or ratio is equal to or less than a predetermined threshold. The identification device according to claim 2, wherein the identification device discriminates as a noise distribution of a category. Third data input means for inputting third data;
4. The identification device according to claim 1, further comprising: an identification unit that identifies the category of the third data using the classifier. 5.   5. The identification device according to claim 1, wherein the classifier generation unit generates a classifier for each category. A method for controlling an identification device, comprising:
A first data input step for inputting first data;
A first feature extraction step of extracting features from the first data;
A background model generation step for generating a background model for the features extracted by the first feature extraction step;
A determination step of determining a noise distribution in the background model;
A second data input step for inputting second data;
A second feature extraction step for extracting features from the second data;
A model generation step of generating a model for the feature extracted by the second feature extraction step based on a distribution other than the noise distribution of the background model;
And a discriminator generating step for generating a discriminator for identifying a category based on the model. A first data input step for inputting first data;
A first feature extraction step of extracting features from the first data;
A background model generation step for generating a background model for the features extracted by the first feature extraction step;
A determination step of determining a noise distribution in the background model;
A second data input step for inputting second data;
A second feature extraction step for extracting features from the second data;
A model generation step of generating a model for the feature extracted by the second feature extraction step based on a distribution other than the noise distribution of the background model;
A program for causing a computer to execute a discriminator generating step for generating a discriminator for identifying a category based on the model.

Images