JP2010170422A - Model selection apparatus, selection method for model selection apparatus, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To implement a fast model selection to a model selection problem with a mixed distribution. <P>SOLUTION: A model selection apparatus includes a mixture number setting means for selecting a candidate value of mixture number that has not been optimized, an initialization means for initializing data with the selected mixture number, an expectation calculation means for calculating an expectation of cluster assignment, an optimization means for calculating an information criterion expectation to a posterior probability of cluster assignment and performing optimization, an information criterion calculation means for calculating an information criterion value, an optimality determination means for repeating the expectation calculation if determining that the optimization is incomplete, a mixture number determination means for determining whether a parameter optimizing the information criterion expectation has been calculated for every candidate value of mixture number in data determined to be optimum, and an optimum distribution selection means for comparing the optimized information criterion to select an optimum mixture number as an optimum model. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a model selection device, a model selection device selection method, and a program for realizing high-speed model selection from an expected value of an information amount standard.

  The mixed distribution is a model that represents the distribution of data by a plurality of distributions, and is an important model for industrial data modeling. Such models include various models such as a mixed normal distribution and a mixed hidden Markov model.

  Generally, when the number of mixtures and the type of each component are specified, it is possible to specify distribution parameters using a known technique such as an EM algorithm (for example, Non-Patent Document 1). .

  In order to estimate the parameters, it is necessary to determine the number of mixtures and the type of each component, and such problems that specify the shape of the model are generally called “model selection problem” or “system identification problem”. This is an extremely important problem for constructing a reliable model, and a technique for that purpose has been proposed.

  In the mixed distribution model selection problem, for a plurality of model candidates such as minimum description length (MDL) (for example, Non-Patent Document 2), Akaike Information Criterion (AIC) (for example, Non-Patent Document 3), cross validation, etc. The policy is to calculate the model selection criteria and select the model for which the criteria are optimal. It is known that a model for optimizing the information criterion has excellent statistical properties such as matching to a true distribution in the case of the MDL criterion and minimum prediction error in the case of the AIC.

  However, with the prior art, it was possible to optimize the number of blends, but it was practically impossible to optimize even the types of components that were blended.

  The problem of computational complexity is explained in the problem of selecting a mixed polynomial curve. The polynomial curve includes a straight line (primary curve), a quadratic curve, a cubic curve, and a plurality of orders.

  When selecting the optimal model by searching the number of mixtures from 1 to Cmax and the order of the curve from 1 to Dmax, in the prior art, there are two straight lines and a quadratic curve (mixing number is 3), and the cubic curve is It is necessary to calculate the information criterion for all model candidates such as three and two quartic curves (the number of mixtures is five). The number of candidates for this model is, for example, approximately 100,000 when Cmax = 10 and Dmax = 10, and tens of billions when Cmax = 20 and Dmax = 20. It increases exponentially with it.

  Therefore, Non-Patent Document 4 proposes a two-stage technique in which the number of mixing is first optimized and then the type of optimization is performed.

Christopher M. Bishop, “Pattern Recognition And Machine Learning”, New edition, Springer-Verlag, August 17, 2006, p. 438-441 Kenji Yamanishi, Tae Hanhan, “Introduction to MDL: From the Viewpoint of Information Theory”, Journal of Artificial Intelligence, May 1992, Vol. 7, No. 3, p. 427-434 Shigehira Hidetoshi and three others, "Model selection Prediction, test, estimation intersection Frontier of statistical science (3)", Iwanami Shoten, December 2004, p. 24-25 Yue Wang, Lan Luo, Matthew T. Freedman, and Sun-Yuan Kung, "Probabilistic Principal Component Subspaces: A Hierarchical Finite Mixture Model for Data Visualization", IEEE TRANSACTIONS ON NEURAL NETWORKS, MAY 2000, VOL.11, NO.3, p.625-636

  However, in the method based on the rule of thumb as described in Non-Patent Document 4, the excellent statistical properties of the information criterion have been lost.

  Therefore, the present invention has been made in view of the above problems, and an object of the present invention is to realize model selection at high speed with respect to the model selection problem of the mixed distribution.

  In order to solve the above-described problem, the model selection apparatus according to the present invention includes a mixture number setting unit that selects a candidate value that is not optimized from a mixture number candidate value, and a mixture number selected by the mixture number setting unit. Using the initialization means for initializing the cluster assignment of the data, the expected value calculation means for calculating the expected value of the cluster assignment in the data initialized by the initialization means, and the expected value calculation means Using the expected value of the cluster assignment, calculate the expected value for the posterior probability of the information-based cluster assignment and calculate the component type and parameters to be optimized for each component. Information criterion calculation means for calculating the information criterion value for the optimized distribution and the information criterion based on the information criterion calculation means If the suitability is judged to be non-optimal, the expectation value calculating means performs the expected value calculation again, and the candidate value of the number of mixtures in the data determined to be optimum by the optimumness judging means For all of the above, it is determined whether a parameter that optimizes the expected value of the information amount criterion is calculated, and if all candidate values are not optimized, a mixture number determination unit that transmits data to the mixture number setting unit; And an optimum distribution selection unit that compares information amount criteria regarding parameters optimized for each number of mixtures and selects the optimum number of mixtures as an optimum model.

  Further, the selection method of the model selection device according to the present invention uses a mixture number setting step for selecting candidate values that are not optimized from the mixture number candidate values, and the number of mixtures selected in the mixture number setting step. An initialization step for initializing the cluster assignment of data, an expected value calculation step for calculating an expected value of cluster assignment in the data initialized in the initialization step, and a cluster assignment calculated in the expected value calculation step Using the expected value of the asset to calculate the expected value related to the posterior probability of the information-based cluster assignment, and to optimize the component type and parameters for each component, and to optimize the optimization step Information criterion calculation step for calculating the information criterion value for the generalized distribution, and information criterion calculation The optimality of the information amount criterion calculated in step is determined, and if it is determined that it is not optimal, the optimality determination step in which the expected value calculation is performed again in the expected value calculation step and the optimality in the optimality determination step. For all candidate values for the number of mixtures in the determined data, it is determined whether a parameter for optimizing the expected value based on the information amount has been calculated. If all candidate values have not been optimized, processing is performed from the mixture number setting step And a step of determining the number of mixtures, and an optimum distribution selection step of comparing an information amount criterion regarding parameters optimized for each number of mixtures and selecting the optimum number of mixtures as an optimum model. And

  Further, the program according to the present invention includes a mixture number setting process for selecting candidate values that are not optimized from the mixture number candidate values, and a cluster assignment of data using the mixture number selected in the mixture number setting process. Initialization processing for initializing, expected value calculation processing for calculating the expected value of cluster assignment in the data initialized by initialization processing, and expected value of cluster assignment calculated by the expected value calculation processing To calculate the expected value for the posterior probability of the information-based cluster assignment, to calculate the type and parameters of the component that is optimal for each component, and to the distribution optimized by the optimization process Determine the optimality of the information criterion based on the information criterion that calculates the information criterion value and the information criterion calculated by the information criterion calculation. If it is determined that the optimal value determination process is performed again in the expected value calculation process, the information criterion is used for all candidate values of the number of mixtures in the data determined to be optimal in the optimality determination process. It is determined whether the parameter that optimizes the expected value is calculated, and if all candidate values are not optimized, the mixture number determination process that repeats the process from the mixture number setting process, and optimization for each mixture number It is characterized in that the computer executes an optimal distribution selection process for comparing the information amount criteria regarding the parameters and selecting an optimal number of mixtures as an optimal model.

  According to the present invention, it is possible to realize model selection at high speed while maintaining the statistical property of the information amount criterion for the model selection problem of the mixed distribution.

It is a block diagram of the model selection apparatus which concerns on embodiment of this invention. It is a flowchart figure of the model selection apparatus which concerns on embodiment of this invention.

  Next, the best mode for carrying out the invention will be described in detail with reference to the drawings.

[First Embodiment]
FIG. 1 is a configuration diagram of a model selection device according to an embodiment of the present invention. The model selection apparatus 100 includes a data input unit 101, a mixture number setting unit 102, a distribution initialization processing unit 103, a cluster assignment expected value calculation processing unit 104, an expected information amount criterion optimization processing unit 105, an information An amount criterion calculation processing unit 106, an optimality determination processing unit 107, a mixture number loop end determination processing unit 108, an optimal distribution selection processing unit 109, and a model selection result output device 110 are provided.

  When the input data 111 is input, the model selection apparatus 100 optimizes the number of mixtures and the type of each component with respect to the input data 111 and outputs the result as a model selection result 112.

  The data input unit 101 is a functional unit for inputting input data 111. Further, the input data 111 includes parameters necessary for model selection, such as the types of components to be mixed and candidate values for the number of mixtures.

  The mixture number setting unit 102 selects and sets the mixture number of the model from the inputted mixture number candidate values. In the following, the set number of mixtures is expressed as K.

  The distribution initialization processing unit 103 performs initialization processing for estimation. Note that initialization may be performed by an arbitrary method.

  The cluster assignment expected value calculation processing unit 104 calculates an expected value of which component of the mixed distribution each input data belongs to (referred to as cluster assignment).

  The expected information criterion optimization processing unit 105 uses the expected value of the calculated cluster assignment to calculate the type of component that optimizes the expected information criterion for each component and the parameters at that time.

  Here, the expected information criterion is an amount obtained by taking the expected value of the information criterion with respect to the posterior probability of cluster assignment. An important point in this processing is that each component can be individually optimized, and there is no need to consider the combination of the entire components. As a result, even if the number of mixed candidates and the number of component types increase and the number of model combinations increases exponentially, model selection is performed in a linear order of the number of mixed number candidates and the number of component types. Things will be possible. It should be noted that any optimization method can be used for the component unit optimization.

  The information amount criterion calculation processing unit 106 calculates the information amount criterion for a model in which the component type and parameters are optimized.

  The optimality determination processing unit 107 compares the information amount reference value calculated in this loop with the information amount reference value calculated in the previous loop to determine whether the optimization processing has converged. If it is not converged, the cluster assignment expected value calculation processing unit 104 is looped.

  The mixture number loop end determination processing unit 108 determines whether or not an optimal information amount reference value has been calculated for all input candidate numbers for the mixture number. If it is determined that it is not optimal, the mixture number setting unit 102 causes a loop.

  The optimum distribution selection processing unit 109 compares the information amount criterion values calculated for all the mixture number candidate values, and selects the mixture number for which the information amount criterion is optimum. Since the type and parameters of each component have already been optimized for this number of mixtures, it is selected as the optimal distribution.

  The model selection result output unit 110 outputs an optimal number of mixtures, component types, parameters, and the like as a model selection result 112.

  FIG. 2 is a flowchart in the embodiment of the present invention. The model selection apparatus 100 according to the present embodiment generally operates as follows.

  First, when the input data 111 is input to the data input unit 101 (step S100), the mixture number setting unit 102 selects a mixture number that has not yet been optimized among the input candidate values for the mixture number. Set (step S101).

  Next, the initialization processing unit 103 initializes the data cluster assignment for the mixture number designated by the mixture number setting unit 102 (step S102).

  Next, the cluster assignment expected value calculation processing unit 104 calculates the expected value of the cluster assignment of the data for the distribution in the middle of the optimization of the designated number of mixtures (step S103). Note that distribution optimization has not started immediately after initialization, and this processing is skipped.

  Next, the expected information criterion optimization processing unit 105 calculates an expected value related to the posterior probability of the information criterion cluster assignment by using the expected value of the cluster assignment calculated for each component. Then, the type and parameter of the component that optimizes it are calculated (step S104).

  Next, the information amount reference calculation processing unit 106 calculates the information amount reference value related to the distribution optimized in step S104 (step S105).

  Next, the optimality determination processing unit 107 compares the information amount reference value calculated in step S105 with the information amount reference value calculated in the previous loop, and determines whether or not the optimization has been completed. Determination is made (step S106).

  If it is determined that it is not optimal, the processing from step S103 to step S106 is repeated. If it is determined to be optimal, the mixture number loop end determination processing unit 108 determines whether or not the optimization of all the mixture number candidate values has been completed and the information amount reference value has been calculated (step) S107). If optimization has not been completed for all candidates, the processing from step S101 to step S107 is repeated.

  When the optimization has been completed for all candidates, the optimum distribution selection processing unit 109 compares the optimized information amount standard values for the number of mixtures, and determines the optimum number of mixtures for that value. Select as a model (step S108). For the selected model, the component types and parameters are optimized in the processing from step S103 to step S106, and a distribution having the optimal number of mixtures and component types is acquired.

  Next, the model selection result output device 110 outputs the model selection result 112 (step S109).

  Hereinafter, models to which the model selection device proposed in the present invention can be applied will be described in detail.

  An example of using the MDL standard as the information amount standard will be described.

  First, the mixture distribution to be learned is expressed by (Expression 1) with respect to the random variable X corresponding to the input data. Here, πk represents a mixing ratio regarding the kth component, ηk represents a distribution parameter regarding the kth component, and θ = [π, η1,..., ΗK]. Here, π = (π1,..., ΠK). The distribution P (X; ηk) of each component is an element of a set S of component candidates. For example, (Equation 1) can be a mixture of a plurality of different distributions such as a normal distribution and an exponential distribution.

  Next, the MDL standard is a standard for selecting, as an optimal model, a model that minimizes the sum of the data description length and the model description length expressed by (Expression 2). When the MDL criterion is used as the information amount criterion, the MDL criterion calculation method is stored in the information amount criterion calculation processing unit 106, and the MDL criterion value of the distribution is calculated by (Equation 2).

Here, l is a description length function, x ^N = (x ₁ ,..., X _N ) is an input data set, and M is a model. x _i represents one point of data, and X is a random variable corresponding to the data. For example, if the distribution of X determined by the model M is P (X; θ) (θ is a distribution parameter), l (x ^N | M) can be calculated as (Equation 3) or (Equation 4). Is possible.

Where log is the logarithm of the base 2 and ln is the natural logarithm. Also, ^ represents that the parameter is a maximum likelihood estimator. I (θ) is a Fisher information matrix. Various description methods have been proposed for the description length functions l (x ^N | M) and l (M) depending on the type of M, and any description method may be used in the embodiment of the present invention. Is possible.

Let z _{i be the} cluster assignment for data x _i and let z ^N = (z ₁ ,..., Z _N ). _{z i = (z i1, ...} , z iK) is a, z _ik is 1 if x _i belongs to the k-th cluster, in the case that do not belong to the k-th cluster, which is a variable that takes a 0. Since z ^N cannot actually be observed, it is often called a hidden variable in learning mixed distributions.

The cluster assignment expectation value calculation processing unit 104, calculation of expected values for the posterior probability of the cluster assignment of in terms of data x ^N is given are described. Note that the posterior probability of cluster assignment varies depending on P (X; θ), and can be calculated by any known method. In the following, Ez [A] represents an expected value related to the posterior probability of the cluster assignment of argument A.

description length of coding both x ^N and z ^N is calculated by equation (5).

^{Here, l (x N, z N} ; M) is, l | can be similarly utilize any description length function and (x ^N M) and l (M). Examples include (Equation 6) and (Equation 7).

  However, Mk is the dimension of ηk, Nk is the number of data belonging to the kth cluster, and can be calculated by (Equation 8). P (zi; πk) represents the probability that the cluster assignment for the i-th data takes 1 or 0.

The expected description length calculated by the expected description length optimization processing unit 105 is an amount obtained by taking an expected value of l (x ^N , z ^N ; M) with respect to the posterior probability of cluster assignment, and Ez [l (x ^{N 1} , z ^N ; M)]. Since parameters and models that optimize the expected description length have no dependency relationship between components, optimization is possible for each component. It should be noted that any known technique can be used for the parameter estimation method of the distribution for each component.

(Multiple mixed distributions with different independence)
When the model selection apparatus according to the present embodiment is used, the number of mixtures and the independence of each component can be optimized at high speed with respect to a mixture distribution of a plurality of distributions having different independence with respect to multidimensional data.

  For example, taking a three-dimensional normal distribution as an example, in a three-dimensional normal distribution, all dimensions are independent, only the first dimension is independent, only the second dimension is independent, only the third dimension is independent, all dimensions are subordinate, It is possible to calculate a mixed distribution of distributions having different kinds of independence.

  When the model selection apparatus according to the present embodiment is used, the present invention can be applied to any multidimensional distribution regardless of the multidimensional normal distribution.

(Multiple mixed distributions of different types)
By using the model selection apparatus in the present embodiment, it is possible to optimize the number of mixtures and the type of distribution of each component with respect to a plurality of mixed distributions.

  For example, when a distribution candidate for mixing a normal distribution and an exponential distribution is used, a mixed distribution in which the number of normal distributions and exponential distributions is optimized can be calculated.

  When the model selection device according to the present embodiment is used, the present invention can be applied to an arbitrary plurality of types of distributions regardless of examples of the normal distribution and the exponential distribution.

(Mixed distribution of different types of stochastic regression functions)
By using the model selection device in the present embodiment, it is possible to optimize the number of mixtures and the regression function for each component regarding the mixture distribution of different types of stochastic regression functions.

  For example, consider a mixed model of regression curves using polynomial curves (or curved surfaces in the case of multi-dimensions). In this case, polynomial curves having different orders can be considered as candidates for each component. By using the model selection apparatus proposed in the present invention, it is possible to optimize the number of mixtures and the degree of the polynomial curve of each component.

  When the model selection device in the present embodiment is used, the present invention can be applied to any mixed model of a plurality of types of regression functions regardless of examples of polynomial curves.

(Mixed distribution of different types of probabilistic discriminant functions)
By using the model selection apparatus according to the present embodiment, it is possible to optimize the number of mixtures and the classification function for each component regarding the mixture distribution of different types of probabilistic classification functions.

  For example, consider a mixed model of classification functions having different orders such as a linear discriminant function and a quadratic discriminant function. In this case, classification functions having different orders can be considered as candidates for each component.

  By using the model selection device in the present embodiment, it is possible to optimize the number of mixtures and the classification function order of each component. Further, the present invention can be applied to a mixed model of arbitrary plural types of classification functions regardless of examples of classification functions having different orders.

(Mixed distribution of hidden Markov models with different numbers of hidden states)
By using the model selection apparatus in the present embodiment, it is possible to optimize the number of mixtures and the number of hidden states for each component regarding the mixture distribution of hidden Markov models having different numbers of hidden states.

(Mixed distribution with different number of mixtures)
By using the model selection device in the present embodiment, it is possible to optimize the number of mixtures and the number of mixtures related to each component with respect to the mixture distribution of mixture distributions having different numbers of mixtures.

  For example, consider a mixture distribution of normal distributions. Using this method, it is possible to optimize the number of mixtures for each component as well as the total number of mixtures.

  When the model selection apparatus according to the present embodiment is used, the present invention can be applied to a mixed model having an arbitrary mixed distribution regardless of an example of a normal distribution.

(Mixed distribution of stochastic dimension compression model with different dimensions)
By using the model selection apparatus in the present embodiment, it is possible to optimize the number of mixtures and the number of compression dimensions for each component regarding the mixture distribution of stochastic dimension compression models having different number of dimensions.

  For example, consider a mixture distribution of probabilistic principal component analysis. By using this method, it is possible to optimize the number of principal components (compression dimension) for each component simultaneously with the number of mixtures.

  When the model selection apparatus according to the present embodiment is used, it can be applied to a mixed model of an arbitrary probabilistic dimension compression model regardless of the example of the probabilistic principal component analysis.

(Mixed distribution of different time series models)
By using the model selection apparatus according to the present embodiment, it is possible to optimize the number of mixtures and the type of time series model for each component regarding the mixture distribution of different time series models.

  For example, a plurality of time series models such as an autoregressive model, a moving average autoregressive model, and a seasonal variation model can be used as candidates, and a model in which the number of model candidates is optimized can be calculated.

  When the model selection apparatus according to the present embodiment is used, it can be applied to any time series model regardless of the above time series model.

(Mixed distribution of time series models with different orders)
When the model selection apparatus according to the present embodiment is used, it is possible to optimize the number of mixtures and the order of the time series model for each component regarding the mixture distribution of time series models having different orders.

  For example, considering the mixture distribution of autoregressive models, it is possible to optimize the order for each autoregressive model as well as the number of mixtures. Further, for example, when considering a state space model, it is possible to optimize the number of state quantities related to each component simultaneously with the number of mixtures.

  When the model selection apparatus according to the present embodiment is used, it can be applied to any time series model regardless of the above time series model.

  Although the embodiments have been described above, various modifications and changes can be made to these embodiments and specific examples without departing from the broad scope and scope of the present invention defined in the claims.

DESCRIPTION OF SYMBOLS 101 Data input part 102 Mixed number setting part 103 Initialization process part 104 Cluster assignment expected value calculation process part 105 Expected information amount reference | standard optimization process part 106 Information amount reference | standard calculation process part 107 Optimality determination process part 108 End of mixture number loop Determination processing unit 109 Optimal distribution selection processing unit 110 Model selection result output unit

Claims (13)

A mixture number setting means for selecting candidate values that are not optimized from the mixture number candidate values;
Initialization means for initializing a cluster assignment of data using the mixture number selected by the mixture number setting means;
Expected value calculation means for calculating an expected value of cluster assignment in the data initialized by the initialization means;
Using the expected value of the cluster assignment calculated by the expected value calculation means, the expected value related to the posterior probability of the information-based cluster assignment is calculated, and the component type and the parameter that are optimal for each component are calculated. Optimization means;
An information criterion calculation means for calculating an information criterion value related to the distribution optimized by the optimization means;
Optimality determining means for determining the optimality of the information amount criterion calculated by the information amount criterion calculating means, and if it is determined that the information amount criterion is not optimal, the optimality determining means for performing expected value calculation again by the expected value calculating means;
For all candidate values of the number of mixtures in the data determined to be optimal by the optimality determination means, it is determined whether parameters for optimizing the information amount-based expected value are calculated, and all candidate values are optimized. If not, a mixture number determining means for transmitting data to the mixture number setting means;
A model selection apparatus comprising: an optimum distribution selection unit that compares information amount criteria regarding parameters optimized for each number of mixtures and selects an optimum number of mixtures as an optimum model.   The model selection apparatus according to claim 1, wherein the information amount criterion is an MDL criterion.   3. The model selection apparatus according to claim 1, wherein the number of mixtures and the independence of each component are optimized with respect to a mixture distribution of a plurality of distributions having different independence with respect to multidimensional data.   The model selection apparatus according to claim 1, wherein the number of mixtures and the kind of distribution of each component are optimized with respect to a mixture distribution of a plurality of different distributions.   3. The model selection apparatus according to claim 1, wherein the number of mixtures and the regression function for each component are optimized with respect to a mixture distribution of different types of stochastic regression functions.   3. The model selection apparatus according to claim 1, wherein the number of mixtures and the classification function for each component are optimized with respect to a mixture distribution of different types of probabilistic classification functions.   3. The model selection apparatus according to claim 1, wherein the number of mixtures and the number of hidden states for each component are optimized with respect to a mixture distribution of hidden Markov models having different numbers of hidden states.   3. The model selection apparatus according to claim 1, wherein the number of mixtures and the number of mixtures for each component are optimized with respect to a mixture distribution of mixture distributions having different numbers of mixtures.   3. The model selection apparatus according to claim 1, wherein the number of mixtures and the number of compression dimensions for each component are optimized with respect to a mixture distribution of stochastic dimension compression models having different dimensions.   3. The model selection apparatus according to claim 1, wherein the number of mixtures and the type of time series model for each component are optimized with respect to the mixture distribution of different time series models.   3. The model selection apparatus according to claim 1, wherein the number of mixtures and the order of the time series model for each component are optimized with respect to the mixture distribution of time series models having different orders. A mixture number setting step of selecting candidate values that are not optimized from the mixture number candidate values;
An initialization step of initializing a cluster assignment of data using the number of mixtures selected in the mixture number setting step;
An expected value calculation step of calculating an expected value of a cluster assignment in the data initialized in the initialization step;
Using the expected value of the cluster assignment calculated in the expected value calculation step, the expected value related to the posterior probability of the information-based cluster assignment is calculated, and the component type and the parameter that are optimal for each component are calculated. An optimization step;
An information criterion calculation step for calculating an information criterion value related to the distribution optimized in the optimization step;
Determining the optimality of the information criterion calculated in the information criterion calculation step, and if it is determined that it is not optimal, the optimality determination step of performing the expected value calculation again in the expected value calculation step;
For all candidate values of the number of mixtures in the data determined to be optimal in the optimality determination step, it is determined whether a parameter for optimizing the information amount-based expected value is calculated, and all candidate values are optimized. If not, a mixture number determination step for repeating the process from the mixture number setting step;
A selection method of a model selection device, comprising: an optimum distribution selection step of comparing information amount criteria regarding parameters optimized for each number of mixtures and selecting the optimum number of mixtures as an optimum model. A mixture number setting process for selecting candidate values that are not optimized from the mixture number candidate values;
An initialization process for initializing a cluster assignment of data using the mixture number selected in the mixture number setting process;
An expected value calculation process for calculating an expected value of a cluster assignment in the data initialized by the initialization process;
Using the expected value of the cluster assignment calculated in the expected value calculation process, the expected value regarding the posterior probability of the information-based cluster assignment is calculated, and the component type and the parameter that are optimal for each component are calculated. Optimization process,
An information amount criterion calculation process for calculating an information amount criterion value related to the distribution optimized in the optimization process;
Determining the optimality of the information criterion calculated in the information criterion calculation process, and if it is determined not to be optimal, the optimality determination process in which the expected value calculation is performed again in the expected value calculation process;
For all candidate values of the number of mixtures in the data determined to be optimal in the optimality determination process, it is determined whether parameters for optimizing the information amount-based expected value are calculated, and all candidate values are optimized. If not, a mixture number determination process for repeating the process from the mixture number setting process;
A program that causes a computer to execute an optimum distribution selection process that compares information amount criteria regarding parameters optimized for each number of mixtures and selects an optimum number of mixtures as an optimum model.

Images