JP5063639B2 - Data classification method, apparatus and program - Google Patents

Description

  The present invention relates to a data classification method, apparatus, and program, and more particularly, to a result calculated using a plurality of likelihood calculating means for calculating likelihoods representing likelihood that input data belongs to a plurality of classes, respectively. The present invention relates to a data classification method, apparatus, and program for integrating and classifying data.

  As a method of simultaneously using the outputs of a plurality of likelihood calculating means when classifying input data, log likelihood is obtained by taking the logarithm of the likelihood (the likelihood that the input data belongs to each class) output by each likelihood calculating means. A method is known in which degrees are linearly combined and integrated, and classification is performed based on the integrated log likelihood (see Non-Patent Document 1, for example). If the relative relationship of reliability of each likelihood calculating means is constant, such integration by linear combination is valid and widely used.

  Note that the integration by the above linear combination is, for example, if the likelihood calculation means A is more reliable than the likelihood calculation means B, and integrates by increasing the output weight of the likelihood calculation means A. . That is, the log likelihoods output by the likelihood calculating means are classified by linearly combining them based on the maximum entropy principle and obtaining the posterior probabilities.

  The “likelihood” is “likelihood”, and for example, “probability” is “likelihood”.

Akinori Fujino, Nobuyoshi Ueda, Kazuaki Saito, “Text Classification by Effective Use of Additional Information Based on Maximum Entropy Principle”, IPSJ Journal, Vol. 47, No.10, pp.2929-2937, 2006

  However, as in the above-described prior art, the likelihoods (or logarithmic likelihoods obtained by logarithms) output from the respective likelihood calculating means are linearly combined and integrated, and the integrated likelihood (logarithmic likelihood) is integrated. The method of classification based on the following problems.

  For example, the likelihood calculating means A determines “baseball”, the likelihood calculating means B adopting a classification method different from the classification method by the likelihood calculating means A determines “soccer”, and the truth of the input data If the class is “baseball”, since the likelihoods are linearly combined, there is a possibility of being erroneously determined to be “soccer” by being influenced by an incorrect result by the likelihood calculating means B. That is, there is a problem that a phenomenon in which a correct classification result cannot be obtained as a result often occurs because the advantages of the individual likelihood calculating means are weakened.

  Such a problem is caused by assuming that the reliability of each likelihood calculating means is constant.

  The present invention has been made in view of the above points. A data classification method, apparatus, and program capable of obtaining a classification result with higher accuracy than the case of linearly combining likelihoods and integrating them to obtain a classification result. The purpose is to provide.

  FIG. 1 is a diagram for explaining the principle of the present invention.

The present invention (Claim 1) is a data classification method for classifying predetermined input data into classes,
Each of n (n is 2 or more) likelihood calculating means having different classification methods or constituent elements and features to calculate calculates likelihood that predetermined input data belongs to each of a plurality of classes. A likelihood calculation step (step 1) to be stored in the likelihood storage means;
Acquires the likelihood that is output by the likelihood storage means whether we likelihood calculating means, for the class of interest, the two likelihood in the likelihood of each of the likelihood calculating means is calculated using By using a total of 2n likelihoods by all n likelihood calculating means, a process of calculating a certainty factor that is a certainty indicating that the class of interest is a correct class for the input data is performed. A certainty factor calculating step (step 2) for calculating all the certainty factors of the plurality of classes ;
A data class determining step (step 3) for determining a class to which the input data belongs based on the certainty value.

Further, the present invention (Claim 2), in the probability Sind calculating step (Step 2),
The likelihood of the two classes for each likelihood calculating means is
A likelihood said likelihood calculating means with respect to the class of interest is calculated, the maximum and becomes likelihood class other than the class of interest in the likelihood the likelihood calculating means is calculated.

  FIG. 2 is a principle configuration diagram of the present invention.

The present invention (Claim 3 ) is a data classification device for classifying predetermined input data into classes,
Input means 11 for inputting predetermined input data;
The likelihood that the data belongs to a plurality of classes at a predetermined input is calculated, and stored in the likelihood storage means 17; n classification methods to be calculated, or n elements having different constituent elements and features (n is 2 or more) ) Likelihood calculation means 13;
Acquires the likelihood that is output by the likelihood storage means 17 or we likelihood calculating unit 13, with respect to the class of interest, the two likelihood in the likelihood of each of the likelihood calculating means 13 calculates A process of calculating a certainty factor that is a probability indicating that the class of interest is a correct class for the input data , using a total of 2n likelihoods by all n likelihood calculating means. A certainty factor calculating means 14 for calculating all the certainty factors of the plurality of classes ,
Data class determining means 15 for determining the class to which the input data belongs based on the certainty value.

Further, the present invention (Claim 4), the likelihood likelihood of two classes of each likelihood calculating unit 13 of the confidence factor computing unit 14, where the likelihood calculating unit 13 for the class of interest is calculated, This is the maximum likelihood in a class other than the class of interest among the likelihoods calculated by the likelihood calculating means .

The present invention (Claim 5 ) is a data classification program for causing a computer to function as each means constituting the data classification apparatus according to Claim 3 or 4 .

  The present invention predicts the certainty level indicating that each class is the correct class for the input data from all the likelihoods output by all likelihood calculating means, and classifies the predicted class with a high degree of certainty for the input data. By obtaining as a class, it is possible to obtain a classification result with higher accuracy than when the likelihoods are linearly combined and integrated to obtain a classification result.

It is a figure for demonstrating the principle of this invention. It is a principle block diagram of this invention. It is a block diagram of the data classification device in one embodiment of this invention. It is a flowchart of a process of the data classification device in one embodiment of this invention.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 3 is a configuration diagram of the data classification device according to the embodiment of the present invention.

  The data classification device 10 shown in the figure includes an input unit 11, a likelihood calculation control unit 12, a plurality of likelihood calculation means 13, a certainty factor calculation unit 14, a data class determination unit 15, a memory 16, a likelihood storage unit 17, The calculation parameter storage unit 18 and the certainty factor storage unit 19 are configured, the processing unit storage unit 1 and the keyboard 2 are connected to the input unit 11, and the display 3 is connected to the output unit 15.

  The processing target storage unit 1 is a database in which processing targets such as documents are stored, and is read by the input unit 11.

  The memory 16 stores the processing target input by the input unit 11.

  The likelihood storage unit 17 stores the likelihood calculated by each likelihood calculating means 13.

  The calculation parameter storage unit 18 stores a certainty factor calculation parameter obtained in advance by a model parameter estimation procedure in a logistic regression model described later.

  The certainty factor storage unit 19 stores the certainty factor obtained by the certainty factor calculation unit 14.

  The input unit 11 reads the processing target data from the processing target storage unit 1 and stores it in the memory 16, acquires the number of likelihood calculation means 13 and the number of classes that are classification destinations input from the keyboard 2, and the likelihood It passes to the calculation control unit 12.

  The likelihood calculation control unit 12 determines the likelihood calculating means 13 for inputting the feature of the processing target data from the memory, and calculates the likelihood.

  The likelihood calculating means 13 calculates the likelihood that is the likelihood that the input data belongs to each of the plurality of classes, and stores the likelihood in the likelihood storage unit 17. Each likelihood calculating means 13 is different from each other in the classification method or the constituent elements and features. For example, modeling the joint probability distribution of data feature vectors and class labels, and calculating the class posterior probabilities based on the Bayes rule, generating the class labels of the data, and discriminating approaches directly modeling the class posterior probabilities and so on.

  The certainty factor calculation unit 14 uses the certainty factor calculation parameter stored in the calculation parameter storage unit 18 from all the likelihood values stored in the likelihood storage unit 17, and each class is a correct class for the input data. A certainty factor that is a certainty indicating that it is present is calculated and stored in the certainty factor storage unit 19. The certainty factor is calculated in an integrated manner using the likelihoods output from all likelihood calculating means. Schematically, the likelihood is high in many likelihood calculating means, and the value becomes higher as the likelihood in the opposing class is lower.

  The data class determination unit 15 acquires the certainty factor from the certainty factor storage unit 19 and outputs the class showing the highest certainty factor as the class to which the input data belongs.

  FIG. 4 is a flowchart of processing of the data classification device according to the embodiment of the present invention.

  Step 101) The input unit 11 reads input data to be processed from the processing target storage unit 1 into the memory 16.

  Step 102) The input unit 11 acquires the number n of likelihood calculating means 13 input from the keyboard 2 and passes it to the likelihood calculation control unit 12.

  Step 103) The input unit 11 acquires the class number K that is the classification destination input from the keyboard 2 and passes it to the likelihood calculation control unit 12.

  Step 104) The likelihood calculation control unit 12 initializes the number i of the likelihood calculating means 13 to 1 (i = 1).

  Step 105) The likelihood calculation control unit 12 proceeds to Step 106 if the number i of the likelihood calculation means 13 is i ≦ n, and proceeds to Step 109 otherwise.

Step 106) The likelihood calculation control unit 12 inputs the feature of the input data W stored in the memory 14 to the i-th likelihood calculation means LC _i , and the likelihood calculation means LC _i has the input data as a class. _{C k (k = 1, ...} , K) to calculate the likelihood _P i _{(W│C k)} belonging to. In each class, when the input data is composed of a plurality of features, the product of the probabilities of appearance is determined as the likelihood that the features of the input data W belong to the class. An example of W of the input data is a document, and an example of the characteristics of the input data is “word” or “vector assigned to a word”.

That is, when the words “home run” and “game” are included in a certain input document, the appearance probability of “home run” in the class “baseball” is predetermined as, for example, 1/20, and the appearance probability of “game” is, for example, If it is predetermined that it is 1/5, the product of the probability that the input data appears is
(1/20) × (1/5) = 1/100
It is. The product of the above probabilities is used as the likelihood, and this likelihood indicates “likelihood” belonging to the class.

  In addition, “Osamu Ueda, Kazuaki Saito,“ Probability Model of Multi-Topic Text—The Forefront of Text Model Research (1) ”, Journal of Information Processing Society of Japan Vol. 45, No. 2, pp. 184-190, 2004” and “ Ueda Osamu, Saito Kazuaki, "Probability Model of Multi-Topic Text-The Forefront of Text Model Research (2)", Journal of Information Processing Society of Japan, Vol. 45, No. 3, pp. 282-289, 2004. You may be allowed to use a method called Naive Bayes.

Step 107) The likelihood calculating means LC _i stores the calculated likelihood in the likelihood storage unit 17.

  Step 108) The likelihood calculation control unit 12 sets the number i of the likelihood calculation means 13 to i + 1 and returns to Step 105.

  Step 109) When i> n in Step 105 above, the certainty factor calculation unit 14 initializes the class number k to 1.

Step 110) The certainty factor calculation unit 14 extracts the likelihood from the likelihood storage unit 17, and calculates the certainty factor P _c ^k that the class k is the correct class as follows.

を算出パラメータ記憶部１８から読み込み、

Is read from the calculated parameter storage unit 18,

により確信度Ｐ_c ^kを算出する。ここで、式(２)の第３項のmax_j(j≠k)Ｐⁱ（Ｗ│Ｃ_j）は、クラス番号がｋ以外で最大となる尤度を指す。また、上式が示すように、全ての尤度算出手段１３から出力された尤度を用いて統合的に確信度を算出している。概略的には、多くの尤度算出手段においてクラスｋの尤度Ｐⁱ（Ｗ│Ｃ_j）が高く、クラスｊ（≠ｋ）の尤度Ｐⁱ（Ｗ│Ｃ_ｋ）が低い（つまり対抗するクラスの尤度が低い）ほど確信度が高くなる。この情報は単なる尤度Ｐⁱ（Ｗ│Ｃ_ｋ）を見るだけでは利用できないものであり、この情報を活用することで、分類精度を高めるのが、本発明の核となる考え方である。

Thus, the certainty factor P _c ^k is calculated. Here, max _{j (j ≠ k)} P ⁱ (W | C _j ) in the third term of Equation (2) indicates the likelihood that the class number is the maximum except for k. Further, as indicated by the above equation, the certainty factor is calculated in an integrated manner using the likelihoods output from all likelihood calculating means 13. In general, the likelihood P ⁱ (W | C _j ) of the class k is high and the likelihood P ⁱ (W | C _k ) of the class j (≠ _k ) is low (that is, the countermeasure) in many likelihood calculation means. The lower the likelihood of the class to be This information cannot be used simply by looking at the likelihood P ⁱ (W | C _k ), and the use of this information increases the classification accuracy is the core idea of the present invention.

  The certainty factor calculation parameter stored in the calculation parameter storage unit 18 is calculated in advance. Equations (3) and (4) are logistic regression equations, which can be calculated based on the model parameter estimation procedure in the normal logistic regression model shown below.

The logistic regression model is generally a model for explaining the probability y (result variable) of occurrence of a certain phenomenon by the explanatory variables x = (x ₁ ,..., X _r ) observed for explaining the occurrence of the phenomenon. And using model parameters β = (β ₀ ,..., Β _r )

と表せる。モデルパラメータの推定は、訓練データにより行う。訓練データのサイズをＭとしたとき、第ｍ番目（ｍ＝１，…，Ｍ）の訓練データは、
結果変数： y_m、ｙ_m＝１；ある事象が発生
ｙ_m＝０；ある事象が発生せず
説明変数： ｒ個の変数（ｘ_ｍ１，ｘ_ｍ２，…，ｘ_ｍｒ）
となる。この訓練データを式(３)、(４)に当てはめ、最尤法(最も尤もらしいものを選ぶ方法)等を用いて、適切なモデルパラメータβ＝（β_０，…，β_ｒ)を得る。この考え方を式（１）、（２）に当てはめて考え、入力データのクラスｋが正しいクラスであることを結果変数がｙ＝１であると、誤っていることをｙ＝０と、そのときのクラスｋに属する尤度Ｐⁱ（Ｗ│Ｃ_ｋ）とクラスｋ以外で最大となる尤度max_j(j≠ｋ)Ｐⁱ（Ｗ│Ｃ_j）を説明変数ｘ（＝ｘ_１，…）であるとして、モデルパラメータβに対応する

It can be expressed. Model parameters are estimated from training data. When the size of the training data is M, the m-th training data (m = 1,..., M) is
Result variables: y _m , y _m = 1; some event occurs
y _m = 0; no event occurs Explanation variable: r variables (x _m1 , x _m2 ,..., x _mr )
It becomes. This training data is applied to equations (3) and (4), and an appropriate model parameter β = (β ₀ ,..., Β _r ) is obtained using a maximum likelihood method (a method for selecting the most likely one). This idea is applied to formulas (1) and (2), and the fact that the input variable class k is the correct class and the result variable is y = 1, the error is y = 0, The likelihood P ⁱ (W | C _k ) belonging to class k and the maximum likelihood max _{j (j ≠ k)} P ⁱ (W | C _j ) other than class k are explained as explanatory variables x (= x ₁ ,... ) Corresponding to the model parameter β

を求める。

Ask for.

  Note that the synthetic variable Z and the range of explanatory variables with the range of −∞ to ∞ described in “Tango Toshiro, Yamaoka Kazue, Takagi Haruyoshi,“ Logistic Regression Analysis ”, pp. 3 Asakura Shoten, 1996” A model in which an occurrence probability p (x) having a value of 0, 1) is combined with a logistic function may be used. Also, in equations (1) and (2), different calculation parameters are used for each class k, but they may be common to all classes of parameters.

  The certainty factor calculated as described above is stored in the certainty factor storage unit 19.

  Step 112) The certainty factor calculation unit 14 sets the class number k to k + 1 and returns to Step 110.

Step 113) If k> K in Step 110, the data class determination unit 15 reads the certainty factor P _c ^k (k = 1,..., K) from the certainty factor storage unit 19, and includes The class indicating the maximum value is determined as the data class and output to the display 3.

  It is possible to construct the operation of each component of the data classification apparatus shown in FIG. 3 as a program, install it on a computer used as the data classification apparatus, execute it, or distribute it via a network.

  Further, the constructed program can be stored in a portable storage medium such as a hard disk, a flexible disk, or a CD-ROM, and can be installed or distributed in a computer.

  The present invention is not limited to the above-described embodiments and examples, and various modifications and applications can be made within the scope of the claims.

  The present invention is applicable to the classification of text data such as web pages, blogs, and e-mails.

DESCRIPTION OF SYMBOLS 1 Processing object memory | storage part 2 Keyboard 10 Data classification apparatus 11 Input means, input part 12 Likelihood calculation control part 13 Likelihood calculation means 14 Certainty degree calculation means, Certainty degree calculation part 15 Data class determination means, Data class determination part 16 Memory 17 likelihood storage means, likelihood storage unit 18 calculation parameter storage unit 19 confidence level storage unit

Claims (5)

A data classification method for classifying predetermined input data into classes,
Each of n (n is 2 or more) likelihood calculating means having different classification methods or components and features to calculate calculates the likelihood that the predetermined input data belongs to each of a plurality of classes. A likelihood calculating step for storing in the likelihood storage means;
Wherein acquires the likelihood that is output by the likelihood storage means whether we before Kieu calculating means, for the class of interest, the two likelihood in the likelihood of each of the likelihood calculating means is calculated A process of calculating a certainty factor that is a probability indicating that the class of interest is a correct class for the input data, using a total of 2n likelihoods by all n likelihood calculating means. Doing a certainty factor calculating step for calculating all the certainty factors of the plurality of classes ,
A data class determining step for determining a class to which the input data belongs based on the certainty value;
The data classification method characterized by performing. In the certainty factor calculating step,
The likelihood of the two classes for each likelihood calculating means is
A likelihood said likelihood calculating means with respect to the class of interest is calculated, <br/> claim 1 which is the maximum and becomes likelihood class other than the class of interest in the likelihood the likelihood calculating means is calculated The data classification method described. A data classification device for classifying predetermined input data into classes,
Input means for inputting the predetermined input data;
The likelihood that the data belongs to a plurality of classes at the predetermined input is calculated, and stored in the likelihood storage means. N classification methods to be calculated, or n elements (n is 2 or more) that are different from each other ) Likelihood calculation means,
Wherein acquires the likelihood that is output by the likelihood storage means whether we before Kieu calculating means, for the class of interest, the two likelihood in the likelihood of each of the likelihood calculating means is calculated A process of calculating a certainty factor that is a probability indicating that the class of interest is a correct class for the input data , using a total of 2n likelihoods by all n likelihood calculating means. Doing a certainty factor calculating means for calculating all the certainty factors of the plurality of classes ,
Data class determining means for determining a class to which the input data belongs based on the certainty value;
A data classification apparatus comprising: In the confidence factor computing means,
The likelihood of the two classes for each likelihood calculating means is
A likelihood said likelihood calculating means with respect to the class of interest is calculated, <br/> claim 3 is the likelihood that the maximum class other than the class of interest in the likelihood the likelihood calculating means is calculated The data classification device described. A data classification program for causing a computer to function as each means constituting the data classification device according to claim 3 or 4 .

Images