JP2001202358A - Bayesian inference method for mixed model and recording medium with recorded bayesian inference program for mixed model - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a Bayesian inference method for mixed model and a recording medium with recorded program therefor, with which the optimal number of mixtures can be searched from the viewpoint of post-distribution maximization to the number of mixtures concerning the Bayesian inference of the mixed model. SOLUTION: This method is provided with a step for inferring the post- distribution of parameters while using a general Bayesian inference method for an initial parameter value and the initial number of mixtures, step for providing the best merged result, step for providing the best merged/divided result, step for providing the beat divided result, step for selecting the result, with which the lower limit value of a logarithmic ensemble likelihood function becomes maximum, corresponding to each of results by comparing the best merged result, the best merged/divided result and the best divided result provided in the respective steps, and step for repeatedly executing series of steps until the lower limit value of the logarithmic ensemble likelihood function is not increased any more.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a basic technique for Bayes estimation of parametric statistics, such as Bayes estimation of a mixture normal distribution, and more particularly, to Bayes estimation of a mixture model for estimating a posterior distribution of parameters of a mixture model. Regarding the estimation method, in more detail, the probability density function of the mixture model is given parametrically, and the parameter maximizing the lower limit value of the log ensemble likelihood calculated using the probability density function and the observation data is determined by a sequential iterative method. The present invention relates to a Bayesian estimation method for a mixed model to be obtained and a recording medium on which a Bayesian estimation program for a mixed model is recorded.

[0002]

2. Description of the Related Art When the complexity of a model is represented by a mixture number, the class of a parametric probability distribution (probability model) of the mixture model defined by the mixture number m and the model parameter θ is defined as

(Equation 1) And this is called a hypothesis space (·; arbitrary variable).

[0003] Statistical estimation refers to a hypothesis p (D | θ, m) that best approximates a true model in a hypothesis space based on observation data D = {d ₁ , d ₂ ,..., D _n }. You can say "search". "Likelihood" is used as a measure of the goodness of the approximation.

When the observation data D is obtained, the likelihood for the model parameter θ can be calculated.

(Equation 2) In the likelihood estimation, a parameter θ ＾ that maximizes the likelihood is Is set as the best model. The value is called a likelihood estimation value. Since the likelihood generally increases monotonically with the increase of m, the best model is estimated after fixing the model index in advance. That is, the maximum likelihood (M
The L (Maximum Likelihood) estimation method can be said to be a model search in a hypothetical space where m is fixed.

[0005] Bayes (VB: Variational)
Bayes) estimation considers the prior distribution p (θ | m) of the parameter θ in addition to the likelihood. That is, the parameters are also handled as random variables. In Bayesian estimation, instead of obtaining one hypothesis p (D | θ ＾, m) as in the maximum likelihood estimation, unknown data d _{n + 1} are calculated based on θ of unknown data given observation data D. Hypothesis p with posterior distribution p (θ | D, m)
(D _{n + 1} | θ, m) is weighted and averaged to obtain a “post-prediction distribution” p (d _{n + 1} | D, m), and a probabilistic statement is made about d _{n + 1} . That is, it is calculated by the following equation.

[0006]

(Equation 3) In Bayesian estimation, the number of mixtures m can be treated as a random variable. That is, when the distribution P (m) of m is also considered, the expression (2) can be rewritten as the following expression.

[0007]

(Equation 4) For example, a regression model in which d _i = (x _i , y _i ), i = 1,..., N is generated from y = f (x; θ) + ε using a normal noise ε having an average of 0 and a variance of 1 , The unknown input x
Expected expected output for _{n + 1}

(Equation 5) Is calculated by the following equation taking the expected values of both sides of equation (3).

[0008]

(Equation 6) It is difficult to analytically obtain the posterior prediction distribution except for special cases, and some approximation method is used. One such approximation is the Laplace approximation (D. MacKay, “A practical
Bayesian framework for backpropagation networks, ”
Neural Computation, vol. 4, pp. 448-472, 1992). In the Laplace approximation method, a posterior distribution is approximated by a Gaussian function, and the integral is analytically obtained. However, this approximation is an approximation under the assumption that the number of samples is infinite, and there is a problem in the accuracy of approximation in the case of finite data.

A more accurate approximation is the Markov chain Monte Carlo (MCMC) method (D. Gamerman, “Markovc
hain Monte Carlo, "Chapman & Hall, 1997).

(Equation 7) And the sample that follows the distribution of p (x)

(Equation 8) Can be approximated by the following equation.

[0010]

(Equation 9) This is the basic concept of the MCMC method.

A difference from the simple Monte Carlo method is that a finite number of {x _t } approximating p (x) is "generated" in the form of sampling instead of evaluating the entire x space. Metropolis method and Gibbs sampling method are famous as specific sampling methods.

[0012] However, these MCMC methods require a great deal of time for sampling, and the convergence determination is generally not easy. In recent years, approximation accuracy is higher than Laplace approximation and much more efficient than MCMC.
A third approach to Bayesian estimation, Variational Bayesian estimation, was proposed (SRWaterhouse, D. MacKay and A.
J. Robinson, “Bayesian methods for mixture of exper
ts, ”Advances in Neural Information Processing Sys
tems (NIPS8), 1995).

In the case of the mixed model, it is not known from which element model the observation data _xn was generated. in this case,
Introducing a latent variable Z _i ^n, when x _n is generated from the i element model, and Z _i ⁿ = 1, else Z _i ⁿ = 0. Then, a set of latent variables ^{_{Z = {Z i n | i}} = 1, ···, C, n = 1, ···,
N｝. Here, C represents the number of mixtures, and N represents the number of observation data.

In Bayesian estimation, as described above, all unknowns Z, θ, and m are handled as random variables. In the initial Bayesian estimation, m was fixed (constant), but Attias
Extended MacKay's formulation of Bayesian estimation, where m is also a random variable, for model selection in Bayesian estimation (H. Attias,
“Inferrring parameters and structure of Graphical
models by variational Bayes, ”to appear in Advan
ces in Neural Information Processing Systems (NIPS
12)). This will be described in detail below.

Consider an ensemble likelihood of the following equation in which all unknowns are marginalized.

[0016]

(Equation 10) Note that L is a function of observation data D only.

It should be noted that the joint distribution p (D,
Z, θ, m)

(11) p (D, Z, θ, m) = p (D, Z | m) p (θ | ψ, m) p (m | M) (7) The first term on the right side of equation (7) is the likelihood of complete data (D, Z) under the given model index, the second term is the prior distribution of parameter θ when the model index is given, and , The third term respectively correspond to the prior distribution of the model index. ψ and M are hyperparameters (constants) that define the prior distribution.

Here, the following equation is obtained by introducing a new distribution Q and applying Jensen's inequality to the logarithmic function.

[0019]

(Equation 12) Where the notation <f (x)> _{p (x)} is the expected value of f (x) for the distribution p (x) of x:

(Equation 13) Shall be expressed.

F is a functional having Q as a variable function, which is the lower limit of the log ensemble likelihood L. And L
The following relational expression holds between F.

[0021]

[Equation 14] Here, KL (p (x) ‖q (x)) is a Kullback L by the distance between two distributions p (x) and q (x).
It is called ibler information quantity (Sakamoto, Ishiguro, Kitagawa, "Information Information Statistics," Kyoritsu Shuppan, 1991) and is defined by the following equation.

(Equation 15) Note that in equation (9), L is a constant that exists only in D, maximizing F [Q] with respect to Q in order to maximize the lower bound, requires that Q and the true posterior distribution p (· | D) is equivalent to minimizing the amount of KL information. In other words, the distribution Q that maximizes F is the best approximation of the true posterior distribution. Q is called a variational posterior distribution because it is a posterior distribution that approximates the true posterior distribution by variational (Q is a posterior distribution, so it should be written as Q (• m, D), but the notation is simplified. Therefore, D is omitted).

Decomposed for each unknown variable as Q

Q = Q (Z | m) Q (θ | m) Q (m) (10) It is assumed that each distribution has an arbitrary class. In order to estimate the true posterior distribution in the constrained form of equation (10), it generally does not match the true distribution,
It can be said that the approximation accuracy is much higher than the Laplace approximation method in which the simultaneous posterior distribution of all parameters is approximated by a single normal distribution.

Optimum θ for given model index m
The variational posterior distribution Q (θ | m) is obtained by maximizing F [Q] with respect to Q under the constraint ∫Q (θ | m) dθ = 1.

[0024]

[Equation 17] Here, C _θ is a normalized constant for satisfying ∫Q (θ | m) dθ = 1. Similarly,

(Equation 18) As is clear from equations (11) and (12), Q (θ | m) and Q
(Z | m) is mutually dependent and cannot be solved in a closed form, and is determined by a sequential solution method. That is, the estimated values of the posterior distribution at the t-th iteration are respectively ^expressed as Q (θ | m) of Q (Z | m) ^(t ).
^{Assuming (t)} , the estimated value at the ^(t + 1) th iteration is calculated as follows.

[0025]

[Equation 19] By executing equations (13) and (14) until convergence, local optimal posterior distributions Q (Z | m) ^* and Q (θ | m) ^* are obtained.

On the other hand, if Q (Z | m) ^* and Q (θ | m) ^* are obtained, the optimal posterior distribution of the model index m is the Q of F
More analytically than maximization of (m)

(Equation 20) Is obtained. C _m is a normalized constant, at which time

(Equation 21) Is the optimal number of mixtures from the viewpoint of the maximum posterior distribution. This is the conventional Bayesian estimation method for the mixed model.

[0027]

However, as described above, the Bayesian estimation of the mixed model can be executed by using the conventional Bayesian estimation method. However, the conventional Bayesian estimation shown in the equations (13) and (14) can be performed. The method simply converges to a local optimal solution of the lower limit of the log ensemble likelihood function, and does not necessarily maximize the lower limit of the log ensemble likelihood function. Therefore, there still remains a problem in the reliability of the optimum number of mixtures in the equation (15) calculated using the equations (13) and (14). In addition, the conventional method for determining the optimal number of mixtures is a “model-selective” method of selecting the number of mixtures that maximizes Expression (15) from a plurality of candidates.

The present invention has been made in view of the above problems, and solves the problem of the local optimality of the Bayesian estimation for a mixed model. Further, in the Bayesian estimation, the estimation of the model parameters and the estimation of the number of mixtures are the same. An object of the present invention is to provide a Bayesian estimation method for a mixed model and a recording medium on which a Bayesian estimation program for a mixed model is recorded, which can indicate that it can be obtained simultaneously as a problem of maximizing an objective function.

[0029]

In order to achieve the above-mentioned object, according to the first aspect of the present invention, a probability density function of a mixture model is given parametrically, and the probability density function and observation data are obtained. A Bayesian estimation method for a mixture model when the posterior distribution of parameters maximizing the lower limit of the log ensemble likelihood function and the posterior distribution of the number of mixtures calculated using Estimating the posterior distribution of the parameters using a general Bayesian estimation method for the values and the initial number of mixtures, and selecting two element models to obtain the best merging result; After merging as a single element model and estimating the posterior distribution of the parameters of the new element model, the parameters of all the element models are estimated using the Bayesian estimation method. Again estimated posterior distribution of over data, if the lower limit value of the log-ensemble likelihood function by the merging is increased, adopts the estimated value as the best merged result,
If the number does not increase, the process returns to before the merging process, and performs a process of merging with another element model until there is no more than a predetermined finite number of candidates. One element model is selected, these two element models are merged as one new element model, the remaining one is divided as two new element models, and the posterior distribution of the parameters of the new element model is estimated. After that, the Bayesian estimation method re-estimates the posterior distribution of the parameters of all the element models, and when the lower limit of the log ensemble likelihood function increases due to the merged division, the estimated value is adopted as the best merged division result. However, if the number does not increase, the process returns to the step before the merging and splitting process, and the process of performing the merging and splitting with another element model is performed until a predetermined finite number of candidates disappear. Performing, and selecting one element model, dividing the element model as two new models, and estimating the posterior distribution of the parameters of the new element model to obtain the best division result, In the Bayesian estimation method, the posterior distributions of the parameters of all the element models are re-estimated, and when the lower limit of the logarithmic ensemble likelihood function is increased by the division, the estimated value is adopted as the best division result, and when the increase is not increased, Returns to before the division process, performing a process of dividing another element model until there is no more than a predetermined finite number of candidates, the best merged result obtained in each step, the best merged divided result Comparing each of the best segmentation results and selecting a result having a maximum lower limit of the logarithmic ensemble likelihood function corresponding to each of the results, The serial sequence of steps is summarized in that and a step of repeatedly executed until the lower limit value no longer increases the logarithmic ensemble likelihood function.

According to the first aspect of the present invention, it is possible to search for an optimum number of mixtures from the viewpoint of maximizing the posterior distribution of the number of mixtures while avoiding a local optimum solution.

According to a second aspect of the present invention, the probability density function of the mixture model is given parametrically, and the lower limit value of the log ensemble likelihood function calculated using the probability density function and the observation data is maximized. Recording medium on which a Bayesian estimation program of a mixture model for calculating a posterior distribution of parameters to be performed and a posterior distribution of the number of mixtures by an iterative method is used. Estimating the posterior distribution of the parameters using the method and selecting the two element models to obtain the best merge result, merging these two element models as a new one element model, After estimating the posterior distribution of the parameters of the model, re-estimate the posterior distribution of the parameters of all element models by the Bayesian estimation method, When the lower limit of the logarithmic ensemble likelihood function increases due to the merging, the estimated value is adopted as the best merging result. When the lower limit does not increase, the process returns to the merging process, and merging with another element model is performed. Is performed until there is no more than a predetermined finite number of candidates, and three element models are selected to obtain the best merged division result, and these two element models are merged as a new one. , The remaining one is divided as two new models, the posterior distribution of the parameters of the new element model is estimated, and then the posterior distribution of the parameters of all the element models is re-estimated by the Bayesian estimation method. When the lower limit of the logarithmic ensemble likelihood function increases due to the merged division, the estimated value is adopted as the best merged division result. Returning to before the merge division processing, performing a merge division with another element model until there are no more than a predetermined finite number of candidates, and selecting one element model to obtain the best division result After dividing the element model as two new element models, estimating the posterior distribution of the parameters of the new element model, re-estimating the posterior distribution of the parameters of all the element models by the Bayesian estimation method, When the lower limit of the logarithmic ensemble likelihood function increases due to the division, the estimated value is adopted as the best division result, and when the lower limit does not increase, the process returns to before the division processing to perform another element model division. Is performed until there is no more than a predetermined finite number of candidates, and the best merged result, the best merged divided result, and the best divided result obtained in each of the above steps are obtained. Comparing each of them and selecting a result in which the lower limit value of the logarithmic ensemble likelihood function corresponding to each is the largest, and repeating the above series of steps until the lower limit value of the logarithmic ensemble likelihood function does not increase. The gist of the present invention is that a Bayesian estimation program of a mixed model that causes a computer to execute the steps to be executed is recorded on a recording medium.

According to the second aspect of the present invention, since the Bayesian estimation program of the mixed model is recorded as a recording medium, the distribution of the Bayesian estimation program can be enhanced by using the recording medium.

[0033]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS First, the outline of the present invention will be described by taking an embodiment according to the present invention as an example.

[0034] The F [Q], when the functional to varying function of Q, obtained in F [Q], Q contains not when together claim written as F _m the following equation (m).

[0035]

(Equation 22) Here, F _m is given by the following equation.

[0036]

(Equation 23) The first term on the right side of the equation (16) is Q (Z | m), Q
(Θ | m), and the second term depends on Q (m). Therefore, the above-mentioned maximization of F [Q] is actually the following 2
It is equivalent to step maximization.

Step 1: For each m, Q of F _m (Z
| M) and Q (θ | m).

Step 2: For each _m , Q of F _m
Maximization for (m).

Assuming that F _m ^* represents the optimum value of F _m obtained in Step 1, the following equation is obtained from equation (16).

[0040]

(Equation 24) Therefore, the optimal value of Q (m) in Step 2 is obtained by maximizing Equation (18) for Q (m) under Σ _m Q (m) = 1.

(Equation 25) Is obtained. It can easily be confirmed that equation (19) is equivalent to equation (15). Looking carefully at equation (19), the denominator does not depend on m, so maximizing Q (m) with respect to m is

(Equation 26) It is nothing but maximizing. Uniform distribution P a prior distribution of m for simplicity | When (m M) = 1 / M , maximizing Q (m) is the largest of simply F _m.

[0041] This is, the _{F m Q (θ | m)} , Q (Z |
By maximizing not only m) but also m at the same time, it means that the optimal model index m can be obtained at the same time without calculating equation (15).

In other words, maximizing the posterior distribution (Maximum a posteriori Probabili) by simultaneously maximizing Q and m with F _m as the objective function instead of F [Q]
(ty: MAP), an optimal model parameter and an optimal model index are obtained as in the following equation.

[0043]

[Equation 27] If θ _MAP and m _MAP are obtained, the predicted distribution or the expected predicted output for the unknown data d _{n + 1} shown in equations (3) and (4) are respectively

[Equation 28] Approximately.

As described above, it has been shown that the optimum values of θ and m can be simultaneously estimated with the same objective function. Next, a method for simultaneously dealing with the problem of local optimality when finding the optimal value of θ will be described in detail.

In the case of a mixed model, the hypothesis space H _m is a direct sum with respect to m.

(Equation 29) Given by

In this case, most of the local solutions correspond to a situation where an excessive number of element models are assigned to a certain data area and an excessively small number of element models are assigned to a certain data area. In fact, maximization of F _m of Step1 described above, i.e., formula (13), (14) in the successive increase in the, unless you set the appropriate initial value, unbalanced element model such arrangements of (a poor (Local solution).

In order to solve the imbalance of the element model arrangement and realize a better element model arrangement, the author has proposed a merging / dividing operation of the model proposed in the framework of the maximum likelihood estimation method (Japanese Patent Application No. 10-340639). No. “Recording medium that records the maximum likelihood estimation method for mixed models and the maximum likelihood estimation program for mixed models”, and Osamu Ueda and Ryohei Nakano, “EM Algorithm with Merge and Split Operation for Mixed Models” (Transactions of the Institute of Electronics, Information and Communication Engineers) , Vol.J82-D-11, no.5, pp 930-940,1999), Sections 3 and 4) are introduced into Bayesian estimation.

Here, however, the model index m is further extended in that it is optimized at the same time. This will be described in detail below.

When the equation (21) is satisfied, F _m can be written as the following equation.

[0050]

[Equation 30] f _i (Q (Z, θ | m)) means an objective function corresponding to the i-th model when the model index (model complexity) is m. Now, for a certain m, the posterior distribution (local optimal solution) obtained by Expressions (13) and (14) is Q ^* , and F _{m at} that time
Is written as F _m ^* , equation (22) can be further written as the following equation.

[0051]

(Equation 31) However,

(Equation 32) And At this time, the right side of equation (23)

[Equation 33]Focusing on only element model j and element model k
The element model j 'is merged, and the element model l is
By dividing into the prime models k ′ and l ′,F _mValue of
Attempt further increase.

The selection of the element models j, k, l, the initialization of the new element models j ', k', l ', re-estimation, etc. are carried out by the above-mentioned method (N. Ueda, R. Nakano, "Merging for Mixed Models"). EM algorithm with division operation ”(Transactions of the Institute of Electronics, Information and Communication Engineers, vol.J82-D-11, no.5, pp 930-940, 1999).

As described above, in the maximum likelihood estimation, increasing (decreasing) m generally increases (decreases) the likelihood. For example, if only division is performed, the local solution is escaped by division and re-estimation. It becomes difficult to discriminate whether the likelihood has increased after reaching a better solution or simply because m has increased. Therefore, the aforementioned method (Japanese Patent Application No. 10-340639)
In order to fix m, merging and division were performed simultaneously.

On the other hand, it can be performed to optimize the complexity parameter and model using the objective function F _m in the method of the present embodiment. That, F _m values with increasing m is not increased, the maximum value to the optimum value of m.

Therefore, not only the simultaneous merging / dividing operation based on the equation (23) but also a "merging operation only" or a "dividing operation only" is attempted. Obviously, "merge (split) operation only" means increasing (decreasing) m by one.

Therefore, these three types of operations are executed, and F
By increasing _m , the problem of local optimality and the problem of determining the optimal number of mixtures can be solved simultaneously.

Next, an embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing a functional configuration of an apparatus for implementing a Bayesian estimation method for a mixed model according to an embodiment of the present invention.

In the embodiment shown in FIG. 1, arbitrary data such as observation data for training, for example, weather data, is externally distributed via an observation data acquisition unit 1 including storage means such as a database constituting a computer system. Estimation unit 3
The posterior distribution estimating unit 3 estimates the posterior distribution of the parameters and the posterior distribution of the number of mixtures from the observation data using a computer or the like. In addition, the posterior distribution estimating unit 3 performs the following steps S11 to S3.
After the steps 7 are sequentially executed to obtain the posterior distribution, the posterior distribution is output via the posterior distribution output unit 5.

First, in step S11, m is appropriately set, and the optimal posterior distribution is obtained using equations (13) and (14). Next, let Q (θ | m) ^* and Q (Z | m) ^* be the values of the posterior distribution when converging in step S13,

(Equation 34) And

Further, the process proceeds to step S15, in which candidate element models to be merged and divided are sorted based on the optimal posterior distribution.

Next, the following procedures, steps S17, S19 and S21, are executed independently. In FIG. 2, step S17 and step S17 are sequentially performed.
Although step 19 and step S21 are executed, step S17, step S19 and step S21 may be executed simultaneously in parallel.

First, step S17 will be described.
In step S <b ^> 17, a search is performed by using only the merging operation on the C candidate element models to be merged in order until the objective function exceeds F ^* . The value of the objective function at that time is F _m-1
^**

In step S 19, a search is performed for C element model candidates to be subjected to merging and division by a simultaneous merging and division operation until the objective function exceeds F ^* . The value of the objective function at that time is defined as F _m ^** .

In step S21, a search is performed for only C element model candidates in order by division operation only until the objective function exceeds F ^* . The value of the objective function at that time is defined as F _{m + 1} ^** .

Next, in step S23, if there is no candidate exceeding F ^* in steps S17, S19 and S21 described above, the process ends.

On the other hand, if there are more candidates, the process proceeds to step S25,

(Equation 35) In step S27,

[Equation 36] If so (YES), the process proceeds to step S29,
The search result found in step 17 is adopted, and m ← m−1 is returned to step S15.

Here, if in step S31,

(37) If yes (YES), the process proceeds to step S33,
The search result found in step 19 is adopted, and the process returns to step S15.

Further, if in step S35,

(38) If so (YES), the process proceeds to step S37,
The search result found in step 21 is adopted, and m ← m + 1 is set, and the process returns to step S15.

Steps S17, S19, and S21 of the above algorithm perform escape from the local solution of Q (Z, θ | m) and guidance to a better solution while fixing m. And from the viewpoint of optimal model selection,
From the complexity of these three models, the best one is selected in steps S23 to S37. By repeating these, it is possible to search for the optimal model while avoiding local solutions. Steps S17 and S1 above
9 and step S21 are called gree.
Being a dy search, the above algorithm is a search for a better local maximum of F _m and there is no theoretical guarantee that a global maximum will be obtained.

However, the monotonic increase of F ^* is guaranteed, and a search for a better maximum value can be efficiently realized.

FIG. 3 shows experimentally the effectiveness of Bayesian estimation of the mixture model in this embodiment. In the experiment, a two-dimensional mixed normal distribution estimation problem in which the estimation result can be visualized was used. The true number of mixtures is 5, and the group of dashed ellipses in FIG. 3 indicates five true two-dimensional normal distributions, and the point group is data artificially generated from these five normal distributions.

In FIG. 3A, the ten solid ellipse groups correspond to the maximum value (MAP estimation value) of each distribution when the number of mixtures is 10, and the posterior distribution of each parameter is appropriately initialized. 2 shows a normal distribution with parameters FIG. 3B shows a MAP estimation result of the conventional Bayes estimation method. FIG.
(C) is the MAP estimated value of the posterior distribution of the parameter of the optimal number of mixing finally obtained by repeating merging and division in the above procedure from FIG. 3 (b). Despite initializing the number of mixtures to 10, we were able to search for the true number of mixtures,
It can be seen that good results close to the true distribution were obtained.

The value of the objective function is shown in FIG.
(B), (c) -1.37 × 10 ³ , -1.01
× 10 ³ , −0.89 × 10 ³ , and it is clear that a better estimated value is obtained as the objective function increases.

As described above, in the present invention,
Since the Bayesian estimation program of the mixed model is recorded as a recording medium, the distribution of the Bayesian estimation program can be enhanced by using the recording medium.

[0075]

As described above, according to the present invention, the step of estimating the posterior distribution of parameters using a general Bayesian estimation method with respect to the initial parameter values and the initial number of mixtures, and In order to obtain, two element models are selected, these two element models are merged as one new element model, and the posterior distribution of the parameters of the new element model is estimated. Re-estimate the posterior distribution of the parameters of the element model,
When the lower limit of the logarithmic ensemble likelihood function increases due to the merging, the estimated value is adopted as the best merging result, and when the lower limit does not increase, the process returns to before the merging process and merges with another element model. Is performed until there is no more than a predetermined finite number of candidates, and three element models are selected to obtain the best merged division result, and these two element models are merged as a new one. , The remaining one is divided into two new element models, and the posterior distribution of the parameters of the new element model is estimated. Then, the posterior distribution of the parameters of all the element models is re-estimated by the Bayesian estimation method, If the lower limit of the logarithmic ensemble likelihood function is increased by the merged division, the estimated value is adopted as the best merged division result, and if not increased, Returning to before the merged division process, performing a process of performing merged division with another element model until there is no more than a predetermined finite number of candidates, and selecting one element model to obtain the best division result Then, the element model is divided as two new element models, the posterior distribution of the parameters of this new element model is estimated, and then the posterior distribution of the parameters of all the element models is re-estimated by the Bayesian estimation method, When the lower limit of the logarithmic ensemble likelihood function is increased by the division, the estimated value is adopted as the best division result, and when the lower limit is not increased,
Performing a process of backtracking (reverting) before the division process and dividing another element model until there is no more than a predetermined finite number of candidates; Merging division results, comparing each of the best division results, and selecting a result in which the lower limit value of the logarithmic ensemble likelihood function corresponding to each is maximum; and And the step of repeatedly executing until the value does not increase, so that it is possible to search for the optimal number of mixtures from the viewpoint of maximizing the posterior distribution of the number of mixtures while avoiding a local optimal solution for the Bayesian estimation of the mixture model. it can.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a functional configuration of an apparatus for implementing a Bayesian estimation method for a mixed model according to an embodiment of the present invention.

FIG. 2 is a flowchart showing a posterior distribution estimation processing procedure by the Bayes estimation method shown in FIG. 1;

FIG. 3 is a diagram experimentally showing the effectiveness of the present invention.

[Explanation of symbols]

 1 Observation data acquisition section 3 Posterior distribution estimation section 5 Posterior distribution output section

Claims (2)

[Claims] 1. A probability density function of a mixture model defined as a linear sum of a plurality of probability density functions is given parametrically, and a lower limit of a logarithmic ensemble likelihood function calculated using the probability density function and observation data. A Bayesian estimation method for a mixture model when a posterior distribution of a parameter maximizing a value and a posterior distribution of a mixture number are obtained by an iterative method, wherein a general Bayes estimation method is used for an initial parameter value and an initial mixture number. Estimating the posterior distribution of the parameters using, selecting the two component models in the mixture model, and merging these two component models as a new one to obtain the best merged result, After estimating the posterior distribution of the parameters of the new element model, estimating the posterior distribution of the parameters of all the element models by the Bayesian estimation method. And, if the lower limit value of the log-ensemble likelihood function by the merging is increased, adopts the estimated value as the best merged result, if not increase, the process returns before the merging process,
Performing a process of merging with another element model until there is no more than a predetermined finite number of candidates; selecting three element models in order to obtain the best merged division result; After merging as one new element model, dividing the remaining one as two new element models, estimating the posterior distribution of the parameters of this new element model, and using the Bayesian estimation method, If the posterior distribution of the parameters is re-estimated and the lower limit of the logarithmic ensemble likelihood function increases due to the merged division, the estimated value is adopted as the best merged division result. Back to
Performing a process of performing a merged division with another element model until there is no more than a predetermined finite number of candidates; selecting one element model to obtain the best division result, and replacing the element model with a new one After dividing into two element models and estimating the posterior distribution of the parameters of the new element model, the Bayesian estimation method is used to re-estimate the posterior distribution of the parameters of all the element models, and the logarithmic ensemble likelihood is obtained by the division. If the lower bound of the function increases,
Adopting the estimated value as the best division result, and if it does not increase, returning to before the division processing and executing processing of dividing another element model until there is no more than a predetermined finite number of candidates; A step of comparing each of the best merged result, the best merged divided result, and the best divided result obtained in each of the steps, and selecting a result in which the lower limit value of the logarithmic ensemble likelihood function corresponding to each is maximized; Repeatedly executing a series of steps until the lower limit of the logarithmic ensemble likelihood function no longer increases. 2. A probability density function of a mixed model defined as a linear sum of a plurality of probability density functions is given parametrically, and a lower limit of a logarithmic ensemble likelihood function calculated using the probability density function and observation data. A recording medium recording a Bayesian estimation program of a mixture model when the posterior distribution of the parameter maximizing the value and the posterior distribution of the number of mixtures are obtained by an iterative method, and for an initial parameter value and an initial number of mixtures, Estimating the posterior distribution of the parameters using a general Bayesian estimation method, and selecting two element models in the mixture model to obtain the best merged result, and replacing these two element models with a new one element model After estimating the posterior distribution of the parameters of this new element model, the Bayesian estimation method If the lower limit of the log ensemble likelihood function increases due to the merging, the estimated value is adopted as the best merging result, and if it does not increase, the process returns to before the merging process. ,
Performing a process of merging with another element model until there is no more than a predetermined finite number of candidates; selecting three element models in order to obtain the best merged division result; After merging as one new element model, dividing the remaining one as two new element models, estimating the posterior distribution of the parameters of this new element model, and using the Bayesian estimation method, If the posterior distribution of the parameters is re-estimated and the lower limit of the logarithmic ensemble likelihood function increases due to the merged division, the estimated value is adopted as the best merged division result. Back to
Performing a process of performing a merged division with another element model until there is no more than a predetermined finite number of candidates; selecting one element model to obtain the best division result, and replacing the element model with a new one After dividing into two element models and estimating the posterior distribution of the parameters of the new element model, the Bayesian estimation method is used to re-estimate the posterior distribution of the parameters of all the element models, and the logarithmic ensemble likelihood is obtained by the division. If the lower bound of the function increases,
Adopting the estimated value as the best division result, and if it does not increase, returning to before the division processing and executing processing of dividing another element model until there is no more than a predetermined finite number of candidates; A step of comparing each of the best merged result, the best merged divided result, and the best divided result obtained in each of the steps, and selecting a result in which the lower limit value of the logarithmic ensemble likelihood function corresponding to each is maximized; A step of repeatedly executing a series of steps until the lower limit of the logarithmic ensemble likelihood function does not increase any more.