CN104933015A - Relation model determination method and apparatus - Google Patents

Abstract

The present invention discloses a relation model determination method and apparatus and belongs to the technical field of statistics. The method comprises: acquiring a logarithm likelihood, a regular term and a logarithm of variation distribution of each hidden variable, which are determined according to sample data, at least two hidden variables for illustrating sample categories to which the sample data belongs and model parameters, and according to the logarithm likelihood, the regular term and the logarithm of the variation distribution of each hidden variable, determining a target function; and determining the variation distribution of each hidden variable and the model parameters, which enable the target function to be converged, and according to the variation distribution of each hidden variable and the model parameters, which enable the target function to be converged, determining a relation model. According to the present invention, the target function is determined by the logarithm likelihood, the regular term and the logarithm of the variation distribution of each hidden variable, which are determined according to the sample data, the at least two hidden variables for illustrating sample categories to which the sample data belongs and the model parameters, and the relation model is determined according to the variation distribution of each hidden variable and the model parameters, which enable the target function to be converged, so that determining efficiency and accuracy of the relation model are improved; and moreover, the regular item is introduced, so that complexity of the relation model is automatically controlled.

Description

The defining method of relational model and device

Technical field

The present invention relates to statistical technique field, particularly a kind of defining method of relational model and device.

Background technology

Along with the development of statistical technique, the relation information between object is modeled as in order to a hot issue.Wherein, the relation information between object is varied, such as, in respondent group interpersonal contact details, linking relationship information on the Internet between the page and the page etc.Various relation information describes mutual relationship in a class object or the relation between multiclass object, by analyzing relation information, can obtain a lot of valuable information.Also Just because of this, the application kind based on relation information gets more and more, and namely different sample data is carried out relation cluster according to relation information is one wherein.And in the process of relation cluster, usually can use relational model.Such as, if certain film company wants to obtain user to the evaluation of the current a series of films shown, then collect the scoring of a collection of user to a series of films shown, by relational model, user and film are assigned in different sample class, realize carrying out cluster to user, film and film scoring simultaneously, thus carry out film evaluation analysis by cluster result.Therefore, how to determine that relational model becomes the key of current research relation cluster.

In actual applications, relational model by hidden variable variation distribute and model parameter determine.Hidden variable refers to and can not directly be observed, and needs to be derived the variable drawn by sample data, and the variation distribution of hidden variable is for describing sample data by the probability of cluster to corresponding classification; Model parameter is for describing the parameter of submodel under each classification.At present, article Variational Bayesian inference and complexity control forstochastic block models, Latouche et al., Statistical Modelling, gives a kind of variation distribution of the determination hidden variable for network data proposition and the mode of model parameter in 2012.Under which, first, the logarithm of the variation distribution of log-likelihood and the hidden variable determined according to observation data, hidden variable and model parameter is obtained; Secondly, according to the logarithm determination objective function of the variation distribution of log-likelihood and hidden variable, and determine variation distribution and the model parameter of the hidden variable making objective function converges, and this makes the distribution of the variation of the hidden variable of objective function converges and model parameter namely can be used as variation distribution and the model parameter of the hidden variable for determining relational model.

Realizing in process of the present invention, inventor finds that prior art at least exists following problem:

Due to above-mentioned determine the mode of relational model for be network data, and network data is the mutual relationship between a class object, thus can be realized by a hidden variable, cause the range of application of the distribution of the variation of the hidden variable obtained according to aforesaid way and the determined relational model of model parameter to have certain limitation; In addition, because the expectation value of objective function according to the logarithm of the variation distribution of the expectation value of log-likelihood and hidden variable is determined, thus higher by the complexity of this kind of determined relational model of objective function.

Summary of the invention

In order to solve the problem of prior art, embodiments provide a kind of defining method and device of relational model.Described technical scheme is as follows:

First aspect, provides a kind of defining method of relational model, and described method comprises:

Obtain the logarithm of the variation distribution of log-likelihood, regular terms and each hidden variable determined according to sample data, at least two hidden variables and model parameter, each hidden variable is for illustration of sample class belonging to sample data;

According to the logarithm determination objective function that the variation of described log-likelihood, regular terms and each hidden variable distributes;

Determine variation distribution and the model parameter of each hidden variable making described objective function converges, the variation according to each hidden variable making described objective function converges distributes and model parameter determination relational model.

In conjunction with first aspect, in the first possible implementation of first aspect, according to the log-likelihood that described sample data, at least two hidden variables and model parameter are determined be:

$\log p(x^{N_1...N_d},Z^1...,Z^d|\mathrm{\θ});$

Wherein, described logp () represents log-likelihood, and described p represents joint probability density function, described in for sample data, described d is the dimension of sample data, described N ₁be the number of the 1st dimension sample data, described N _dbe the number that d ties up sample data, described Z ¹be the hidden variable of the 1st dimension sample data, described Z ^dbe the hidden variable that d ties up sample data, described θ is the set of model parameter, and described model parameter comprises α ¹..., α ^d, described α ¹be the mixture ratio of the 1st dimension, described α ^dbe the mixture ratio of d dimension, described in represent model parameter.

In conjunction with first aspect, in the implementation that the second of first aspect is possible, according to the regular terms that sample data, at least two hidden variables and model parameter are determined be:

$\underset{i=1}{\overset{d}{\mathrm{\Σ}}}\frac{D_{{\mathrm{\α}}^i}}{2}\log N_i+\underset{p_1=1}{\overset{K_1}{\mathrm{\Σ}}}...\underset{p_d=1}{\overset{K_d}{\mathrm{\Σ}}}\frac{D_{p_1...p_d}}{2}L(\underset{j_1=1}{\overset{N_1}{\mathrm{\Σ}}}...\underset{j_d=1}{\overset{N_d}{\mathrm{\Σ}}}{Z}_{j_1{p}_1}^1...Z_{j_d{p}_d}^d,\underset{j_1=1}{\overset{N_1}{\mathrm{\Σ}}}...\underset{j_d=1}{\overset{N_d}{\mathrm{\Σ}}}{q}^{`}\left(Z_{j_1{p}_1}^1\right)...q^{`}\left(Z_{j_d{p}_d}^d\right));$

Wherein, described d is the dimension of sample data, described N ₁be the number of the 1st dimension sample data, described N _dbe the number that d ties up sample data, described N _ibe the number of the i-th dimension sample data, described K ₁the number of the 1st dimension sample class, described K _dit is the number that d ties up sample class; Described for the approximate value that the variation of hidden variable distributes, described in it is the 1st dimension jth ₁individual sample data place p ₁the hidden variable of individual sample class, described in be that d ties up jth _dindividual sample data place p _dthe hidden variable of individual sample class; Described α ⁱbe the mixture ratio of the i-th dimension, described in for described α ⁱdimension, described in be the 1st dimension p ₁individual sample class ... d ties up p _dthe dimension of the submodel parameter in individual sample class, L (a, b)=logb+ (a-b)/b, described a is described b is

In conjunction with first aspect, in the third possible implementation of first aspect, the logarithm of the variation distribution of each hidden variable determined according to described sample data, at least two hidden variables and model parameter is:

logq(Z ¹)，…logq(Z ^d)；

Wherein, described q (Z ¹) be hidden variable Z ^rvariation distribution, described q (Z ^d) be hidden variable Z ^cvariation distribution.

In conjunction with first aspect to any one the possible implementation in the third possible implementation of first aspect, in the 4th kind of possible implementation of first aspect, the logarithm determination objective function of the described distribution of the variation according to described log-likelihood, regular terms and each hidden variable, comprising:

According to the expectation value determination objective function of the logarithm that the variation of the expectation value of described log-likelihood, the expectation value of described regular terms and each hidden variable described distributes.

In conjunction with the 4th kind of possible implementation of first aspect, in the 5th kind of possible implementation of first aspect, the objective function that the expectation value of the logarithm distributed according to the variation of the expectation value of described log-likelihood, the expectation value of described regular terms and each hidden variable described is determined for:

$\begin{array}{c}\mathrm{\Γ}(q,q^{`},\mathrm{\θ},x^{N_{1...}{N}_d})=\underset{Z^1...Z^d}{\mathrm{\Σ}}q\left(Z^1\right)...q\left(Z^d\right)\{\log p(x^{N_{1...}{N}_d},Z^1,...,Z^d|\mathrm{\θ})-\underset{i=1}{\overset{d}{\mathrm{\Σ}}}\frac{D_{{\mathrm{\α}}_i}}{2}\log N_i\\ -\underset{p_1=1}{\overset{K_1}{\mathrm{\Σ}}}...\underset{p_d=1}{\overset{K_d}{\mathrm{\Σ}}}\frac{D_{p_1...p_d}}{2}L(\underset{j_1=1}{\overset{N_1}{\mathrm{\Σ}}}...\underset{j_d=1}{\overset{N_d}{\mathrm{\Σ}}}{Z}_{j_1{p}_1}^1...Z_{j_d{p}_d}^d,\underset{j_1=1}{\overset{N_1}{\mathrm{\Σ}}}...\underset{j_d=1}{\overset{N_d}{\mathrm{\Σ}}}{q}^{`}\left(Z_{j_d{p}_1}^1\right)...q^{`}\left(Z_{j_d{p}_d}^d\right))-\log q\left(Z^1\right)-...-\log q\left(Z^d\right)\}\end{array}.$

In conjunction with the 5th kind of possible implementation of first aspect, in the 6th kind of possible implementation of first aspect, described variation distribution and the model parameter determining each hidden variable making described objective function converges, comprising:

The variation distribution obtaining each hidden variable upgraded and the model parameter upgraded;

Determine whether described objective function restrains according to the variation distribution of each hidden variable upgraded and the model parameter of renewal, if described objective function is not restrained, the variation distribution then again obtaining each hidden variable of renewal and the model parameter upgraded, until the variation distribution and the model parameter that obtain each hidden variable making described objective function converges.

In conjunction with the 6th kind of possible implementation of first aspect, in the 7th kind of possible implementation of first aspect, the described variation distribution obtaining each hidden variable upgraded and the model parameter upgraded, comprising:

The variation distribution of each hidden variable is alternately upgraded, until obtain the variation distribution of each hidden variable of the renewal restrained according to following formula:

…

Variation according to each hidden variable upgraded distributes according to following formula Renewal model parameter, obtains the model parameter upgraded:

${\mathrm{\α}}_{p_1}^{1\left(t\right)}=\frac{\underset{j_1=1}{\overset{N_1}{\mathrm{\Σ}}}{q}^{\left(t\right)}\left(Z_{j_1{p}_1}^1\right)}{N_1};....{\mathrm{\α}}_{p_d}^{d\left(t\right)}=\frac{\underset{j_d=1}{\overset{N_d}{\mathrm{\Σ}}}{q}^{\left(t\right)}\left(Z_{j_d{p}_d}^d\right)}{N_d};$

Described described t represents current renewal, and described t-1 represents last renewal or initialization.

In conjunction with the 6th kind of possible implementation of first aspect, in the 8th kind of possible implementation of first aspect, the described variation distribution obtaining each hidden variable upgraded and the model parameter upgraded, comprising:

According to following formula Renewal model parameter, obtain the model parameter upgraded:

${\mathrm{\α}}_{p_1}^{1\left(t\right)}=\frac{\underset{j_1=1}{\overset{N_1}{\mathrm{\Σ}}}{q}^{(t-1)}\left(Z_{j_1{p}_1}^1\right)}{N_1};....{\mathrm{\α}}_{p_d}^{d\left(t\right)}=\frac{\underset{j_d=1}{\overset{N_d}{\mathrm{\Σ}}}{q}^{(t-1)}\left(Z_{j_d{p}_d}^d\right)}{N_d};$

Described

Alternately upgrade the variation distribution of each hidden variable according to the model parameter upgraded according to following formula, obtain the variation distribution of each hidden variable of the renewal restrained:

…

Wherein, described t represents current renewal, and described t-1 represents last renewal or initialization.

In conjunction with any one possible implementation in the 6th kind to the 8th kind possible implementation of first aspect, in the 9th kind of possible implementation of first aspect, the variation distribution of described each hidden variable according to upgrading and the model parameter of renewal determine whether described objective function restrains, and comprising:

Whether the distance between the objective function that the objective function relatively determined according to the variation distribution of each hidden variable upgraded and the model parameter that upgrades and last time obtain is less than threshold value, and the variation distribution of each hidden variable that the objective function that the described last time obtains upgraded according to the last time and the model parameter of renewal are determined;

If the distance between the objective function that the objective function determined according to the variation distribution of each hidden variable upgraded and the model parameter of renewal and last time obtain is less than threshold value, then determine described objective function converges.

Second aspect, provides a kind of determining device of relational model, and described device comprises:

Acquisition module, for obtaining the logarithm of the variation distribution of log-likelihood, regular terms and each hidden variable determined according to sample data, at least two hidden variables and model parameter, each hidden variable is for illustration of sample class belonging to sample data;

First determination module, for the logarithm determination objective function distributed according to the variation of described log-likelihood, regular terms and each hidden variable;

Second determination module, for determining variation distribution and the model parameter of each hidden variable making described objective function converges;

3rd determination module, for distributing and model parameter determination relational model according to the variation of each hidden variable making described objective function converges.

In conjunction with second aspect, in the first possible implementation of second aspect, the log-likelihood that described acquisition module gets is:

$\log p(x^{N_1...N_d},Z^1...,Z^d|\mathrm{\θ});$

Wherein, described logp () represents log-likelihood, and described p represents joint probability density function, described in for sample data, described d is the dimension of sample data, described N ₁be the number of the 1st dimension sample data, described N _dbe the number that d ties up sample data, described Z ¹be the hidden variable of the 1st dimension sample data, described Z ^dbe the hidden variable that d ties up sample data, described θ is the set of model parameter, and described model parameter comprises α ¹..., α ^d, described α ¹be the mixture ratio of the 1st dimension, described α ^dbe the mixture ratio of d dimension, described in represent model parameter.

In conjunction with second aspect, in the implementation that the second of second aspect is possible, the regular terms that described acquisition module gets is:

$\underset{i=1}{\overset{d}{\mathrm{\Σ}}}\frac{D_{{\mathrm{\α}}^i}}{2}\log N_i+\underset{p_1=1}{\overset{K_1}{\mathrm{\Σ}}}...\underset{p_d=1}{\overset{K_d}{\mathrm{\Σ}}}\frac{D_{p_1...p_d}}{2}L(\underset{j_1=1}{\overset{N_1}{\mathrm{\Σ}}}...\underset{j_d=1}{\overset{N_d}{\mathrm{\Σ}}}{Z}_{j_1{p}_1}^1...Z_{j_d{p}_d}^d,\underset{j_1=1}{\overset{N_1}{\mathrm{\Σ}}}...\underset{j_d=1}{\overset{N_d}{\mathrm{\Σ}}}{q}^{`}\left(Z_{j_1{p}_1}^1\right)...q^{`}\left(Z_{j_d{p}_d}^d\right));$

Wherein, described d is the dimension of sample data, described N ₁be the number of the 1st dimension sample data, described N _dbe the number that d ties up sample data, described N _ibe the number of the i-th dimension sample data, described K ₁the number of the 1st dimension sample class, described K _dit is the number that d ties up sample class; Described for the approximate value that the variation of hidden variable distributes, described in it is the 1st dimension jth ₁individual sample data place p ₁the hidden variable of individual sample class, described in be that d ties up jth _dindividual sample data place p _dthe hidden variable of individual sample class; Described α ⁱbe the mixture ratio of the i-th dimension, described in for described α ⁱdimension, described in be the 1st dimension p ₁individual sample class ... d ties up p _dthe dimension of the submodel parameter in individual sample class, L (a, b)=logb+ (a-b)/b, described a is described b is

In conjunction with second aspect, in the third possible implementation of second aspect, the logarithm of the variation distribution of each hidden variable that described acquisition module gets is:

logq(Z ¹)，…logq(Z ^d)；

Wherein, described q (Z ¹) be hidden variable Z ^rvariation distribution, described q (Z ^d) be hidden variable Z ^cvariation distribution.

In conjunction with any one possible implementation in second aspect to the third possible implementation of second aspect, in the 4th kind of possible implementation of second aspect, described first determination module, for the expectation value determination objective function of the logarithm that the expectation value of the expectation value according to described log-likelihood, described regular terms and the variation of each hidden variable described distribute.

In conjunction with the 4th kind of possible implementation of second aspect, in the 5th kind of possible implementation of second aspect, the objective function that described first determination module is determined for:

${\mathrm{\α}}_{p_1}^{1\left(t\right)}=\frac{\underset{j_1=1}{\overset{N_1}{\mathrm{\Σ}}}{q}^{\left(t\right)}\left(Z_{j_1{p}_1}^1\right)}{N_1};....{\mathrm{\α}}_{p_d}^{d\left(t\right)}=\frac{\underset{j_d=1}{\overset{N_d}{\mathrm{\Σ}}}{q}^{\left(t\right)}\left(Z_{j_d{p}_d}^d\right)}{N_d};$

In conjunction with the 5th kind of possible implementation of second aspect, in the 6th kind of possible implementation of second aspect, described second determination module, comprising:

Acquiring unit, the variation distribution obtaining each hidden variable upgraded and the model parameter upgraded;

According to the variation distribution of each hidden variable upgraded and the model parameter of renewal, determining unit, for determining whether described objective function restrains;

Described acquiring unit, for when described objective function is not restrained, the variation distribution again obtaining each hidden variable of renewal and the model parameter upgraded, until the variation distribution and the model parameter that obtain each hidden variable making described objective function converges.

In conjunction with the 6th kind of possible implementation of second aspect, in the 7th kind of possible implementation of second aspect, described acquiring unit, comprising:

First upgrades subelement, for alternately upgrading the variation distribution of each hidden variable according to following formula, until obtain the variation distribution of each hidden variable of the renewal restrained:

…

Second upgrades subelement, distributes according to following formula Renewal model parameter, obtain the model parameter upgraded for the variation according to each hidden variable upgraded:

${\mathrm{\α}}_{p_1}^{1\left(t\right)}=\frac{\underset{j_1=1}{\overset{N_1}{\mathrm{\Σ}}}{q}^{\left(t\right)}\left(Z_{j_1{p}_1}^1\right)}{N_1};....{\mathrm{\α}}_{p_d}^{d\left(t\right)}=\frac{\underset{j_d=1}{\overset{N_d}{\mathrm{\Σ}}}{q}^{\left(t\right)}\left(Z_{j_d{p}_d}^d\right)}{N_d};$

Described described t represents current renewal, and described t-1 represents last renewal or initialization.

In conjunction with the 6th kind of possible implementation of second aspect, in the 8th kind of possible implementation of second aspect, described acquiring unit, comprising:

3rd upgrades subelement, for according to following formula Renewal model parameter, obtains the model parameter upgraded:

${\mathrm{\α}}_{p_1}^{1\left(t\right)}=\frac{\underset{j_1=1}{\overset{N_1}{\mathrm{\Σ}}}{q}^{(t-1)}\left(Z_{j_1{p}_1}^1\right)}{N_1};....{\mathrm{\α}}_{p_d}^{d\left(t\right)}=\frac{\underset{j_d=1}{\overset{N_d}{\mathrm{\Σ}}}{q}^{(t-1)}\left(Z_{j_d{p}_d}^d\right)}{N_d};$

Described

4th upgrades subelement, for alternately upgrading the variation distribution of each hidden variable according to following formula according to the model parameter upgraded, obtains the variation distribution of each hidden variable of the renewal restrained:

…

Wherein, described t represents current renewal, and described t-1 represents last renewal or initialization.

In conjunction with any one possible implementation in the 6th kind to the 8th kind possible implementation of second aspect, in the 9th kind of possible implementation of second aspect, described determining unit, comprising:

Relatively subelement, whether be less than threshold value for the distance compared between the objective function that obtains according to the variation distribution of each hidden variable upgraded and the objective function determined of model parameter upgraded and last time, the variation distribution of each hidden variable that the objective function that the described last time obtains upgraded according to the last time and the model parameter of renewal are determined;

Determine subelement, when being less than threshold value for the distance between the objective function that the variation distribution of each hidden variable according to renewal and the model parameter of renewal are determined and the objective function that the last time obtains, determine described objective function converges.

The beneficial effect that the technical scheme that the embodiment of the present invention provides is brought is:

By obtaining the logarithm of the variation distribution of log-likelihood, regular terms and each hidden variable determined according to sample data, at least two hidden variables for illustration of sample class belonging to sample data and model parameter, and according to the logarithm determination objective function that the variation of the log-likelihood got, regular terms and each hidden variable distributes, make the mutual relationship being applicable to analysis two class or the above inter-entity of two classes according to the relational model enabling the distribution of the variation of each hidden variable of objective function converges and model parameter determine, thus extend the range of application of relational model; In addition, by introducing regular terms in objective function, the complexity of the relational model determined can be controlled automatically.

Accompanying drawing explanation

In order to be illustrated more clearly in the technical scheme in the embodiment of the present invention, below the accompanying drawing used required in describing embodiment is briefly described, apparently, accompanying drawing in the following describes is only some embodiments of the present invention, for those of ordinary skill in the art, under the prerequisite not paying creative work, other accompanying drawing can also be obtained according to these accompanying drawings.

Fig. 1 is the process flow diagram of the defining method of the relational model that the embodiment of the present invention one provides;

Fig. 2 is the process flow diagram of the defining method of the relational model that the embodiment of the present invention two provides;

Fig. 3 is the structural representation of the determining device of the relational model that the embodiment of the present invention three provides;

Fig. 4 is the structural representation of the second determination module that the embodiment of the present invention three provides;

Fig. 5 is the structural representation of the first acquiring unit that the embodiment of the present invention three provides;

Fig. 6 is the structural representation of the second acquiring unit that the embodiment of the present invention three provides;

Fig. 7 is the structural representation of the determining unit that the embodiment of the present invention three provides.

Embodiment

For making the object, technical solutions and advantages of the present invention clearly, below in conjunction with accompanying drawing, embodiment of the present invention is described further in detail.

Embodiment one

Embodiments provide a kind of defining method of relational model, see Fig. 1, method flow comprises:

101: the logarithm obtaining the variation distribution of log-likelihood, regular terms and each hidden variable determined according to sample data, at least two hidden variables and model parameter, each hidden variable is for illustration of sample class belonging to sample data;

As a kind of embodiment, according to the log-likelihood that sample data, at least two hidden variables and model parameter are determined be:

$\log p(x^{N_1...N_d},Z^1...,Z^d|\mathrm{\θ});$

Wherein, logp () represents log-likelihood, and p represents joint probability density function, for sample data, d is the dimension of sample data, N ₁be the number of the 1st dimension sample data, N _dbe the number that d ties up sample data, Z ¹be the hidden variable of the 1st dimension sample data, Z ^dbe the hidden variable that d ties up sample data, θ is the set of model parameter, and model parameter comprises α ¹..., α ^d, α ¹be the mixture ratio of the 1st dimension, α ^dbe the mixture ratio of d dimension, represent model parameter.

As a kind of embodiment, according to the regular terms that sample data, at least two hidden variables and model parameter are determined be:

$\underset{i=1}{\overset{d}{\mathrm{\Σ}}}\frac{D_{{\mathrm{\α}}^i}}{2}\log N_i+\underset{p_1=1}{\overset{K_1}{\mathrm{\Σ}}}...\underset{p_d=1}{\overset{K_d}{\mathrm{\Σ}}}\frac{D_{p_1...p_d}}{2}L(\underset{j_1=1}{\overset{N_1}{\mathrm{\Σ}}}...\underset{j_d=1}{\overset{N_d}{\mathrm{\Σ}}}{Z}_{j_1{p}_1}^1...Z_{j_d{p}_d}^d,\underset{j_1=1}{\overset{N_1}{\mathrm{\Σ}}}...\underset{j_d=1}{\overset{N_d}{\mathrm{\Σ}}}{q}^{`}\left(Z_{j_1{p}_1}^1\right)...q^{`}\left(Z_{j_d{p}_d}^d\right));$

Wherein, d is the dimension of sample data, N ₁be the number of the 1st dimension sample data, N _dbe the number that d ties up sample data, N _ibe the number of the i-th dimension sample data, K ₁the number of the 1st dimension sample class, K _dit is the number that d ties up sample class; for the approximate value that the variation of hidden variable distributes, it is the 1st dimension jth ₁individual sample data place p ₁the hidden variable of individual sample class, be that d ties up jth _dindividual sample data place p _dthe hidden variable of individual sample class; α ⁱbe the mixture ratio of the i-th dimension, for α ⁱdimension, be the 1st dimension p ₁individual sample class ... d ties up p _dthe dimension of the submodel parameter in individual sample class, L (a, b)=logb+ (a-b)/b, a is b is

As a kind of embodiment, the logarithm of the variation distribution of each hidden variable determined according to sample data, at least two hidden variables and model parameter is:

logq(Z ¹)，…logq(Z ^d)；

Wherein, q (Z ¹) be hidden variable Z ¹variation distribution, q (Z ^d) be hidden variable Z ^dvariation distribution.

102: the logarithm determination objective function distributed according to the variation of log-likelihood, regular terms and each hidden variable;

As a kind of embodiment, according to the logarithm determination objective function that the variation of log-likelihood, regular terms and each hidden variable distributes, comprising:

According to the expectation value determination objective function of the logarithm that the variation of the expectation value of log-likelihood, the expectation value of regular terms and each hidden variable distributes.

As a kind of embodiment, the objective function that the expectation value of the logarithm distributed according to the variation of the expectation value of log-likelihood, the expectation value of regular terms and each hidden variable is determined for:

$\begin{array}{c}\mathrm{\Γ}(q,q^{`},\mathrm{\θ},x^{N_{1...}{N}_d})=\underset{Z^1...Z^d}{\mathrm{\Σ}}q\left(Z^1\right)...q\left(Z^d\right)\{\log p(x^{N_{1...}{N}_d},Z^1,...,Z^d|\mathrm{\θ})-\underset{i=1}{\overset{d}{\mathrm{\Σ}}}\frac{D_{{\mathrm{\α}}_i}}{2}\log N_i\\ -\underset{p_1=1}{\overset{K_1}{\mathrm{\Σ}}}...\underset{p_d=1}{\overset{K_d}{\mathrm{\Σ}}}\frac{D_{p_1...p_d}}{2}L(\underset{j_1=1}{\overset{N_1}{\mathrm{\Σ}}}...\underset{j_d=1}{\overset{N_d}{\mathrm{\Σ}}}{Z}_{j_1{p}_1}^1...Z_{j_d{p}_d}^d,\underset{j_1=1}{\overset{N_1}{\mathrm{\Σ}}}...\underset{j_d=1}{\overset{N_d}{\mathrm{\Σ}}}{q}^{`}\left(Z_{j_d{p}_1}^1\right)...q^{`}\left(Z_{j_d{p}_d}^d\right))-\log q\left(Z^1\right)-...-\log q\left(Z^d\right)\}\end{array}.$

103: variation distribution and the model parameter of determining each hidden variable making objective function converges, the variation according to each hidden variable making objective function converges distributes and model parameter determination relational model.

As a kind of embodiment, determine variation distribution and the model parameter of each hidden variable making objective function converges, comprising:

The variation distribution obtaining each hidden variable upgraded and the model parameter upgraded;

Whether restrain according to the variation distribution of each hidden variable upgraded and the model parameter determination objective function of renewal, if objective function is not restrained, the variation distribution then again obtaining each hidden variable of renewal and the model parameter upgraded, until the variation distribution and the model parameter that obtain each hidden variable making objective function converges.

As a kind of embodiment, the variation distribution obtaining each hidden variable upgraded and the model parameter upgraded, comprising:

The variation distribution of each hidden variable is alternately upgraded, until obtain the variation distribution of each hidden variable of the renewal restrained according to following formula:

…

Variation according to each hidden variable upgraded distributes according to following formula Renewal model parameter, obtains the model parameter upgraded:

${\mathrm{\α}}_{p_1}^{1\left(t\right)}=\frac{\underset{j_1=1}{\overset{N_1}{\mathrm{\Σ}}}{q}^{\left(t\right)}\left(Z_{j_1{p}_1}^1\right)}{N_1};....{\mathrm{\α}}_{p_d}^{d\left(t\right)}=\frac{\underset{j_d=1}{\overset{N_d}{\mathrm{\Σ}}}{q}^{\left(t\right)}\left(Z_{j_d{p}_d}^d\right)}{N_d};$

t represents current renewal, and t-1 represents last renewal or initialization.

As a kind of embodiment, the variation distribution obtaining each hidden variable upgraded and the model parameter upgraded, comprising:

According to following formula Renewal model parameter, obtain the model parameter upgraded:

${\mathrm{\α}}_{p_1}^{1\left(t\right)}=\frac{\underset{j_1=1}{\overset{N_1}{\mathrm{\Σ}}}{q}^{(t-1)}\left(Z_{j_1{p}_1}^1\right)}{N_1};....{\mathrm{\α}}_{p_d}^{d\left(t\right)}=\frac{\underset{j_d=1}{\overset{N_d}{\mathrm{\Σ}}}{q}^{(t-1)}\left(Z_{j_d{p}_d}^d\right)}{N_d};$

Alternately upgrade the variation distribution of each hidden variable according to the model parameter upgraded according to following formula, obtain the variation distribution of each hidden variable of the renewal restrained:

…

Wherein, t represents current renewal, and t-1 represents last renewal or initialization.

As a kind of embodiment, whether restrain according to the variation distribution of each hidden variable upgraded and the model parameter determination objective function of renewal, comprising:

Whether the distance between the objective function that the objective function relatively determined according to the variation distribution of each hidden variable upgraded and the model parameter that upgrades and last time obtain is less than threshold value, and the variation distribution of each hidden variable that the objective function that the last time obtains upgraded according to the last time and the model parameter of renewal are determined;

If the distance between the objective function that the objective function determined according to the variation distribution of each hidden variable upgraded and the model parameter of renewal and last time obtain is less than threshold value, then determine objective function converges.

The method that the embodiment of the present invention provides, by obtaining according to sample data, the log-likelihood that at least two hidden variables for illustration of sample class belonging to sample data and model parameter are determined, the logarithm of the variation distribution of regular terms and each hidden variable, and according to the log-likelihood got, the logarithm determination objective function of the variation distribution of regular terms and each hidden variable, make the mutual relationship being applicable to analysis two class or the above inter-entity of two classes according to the relational model enabling the distribution of the variation of each hidden variable of objective function converges and model parameter determine, thus the range of application of relational model is extended, by introducing regular terms in objective function, the complexity of the relational model determined can be controlled automatically, and then the efficiency determining relational model can be improved.In addition, due to Existence dependency relationship between hidden variable and model parameter, the distribution of the variation of each hidden variable thus determined and model parameter more accurate.

Embodiment two

Embodiments provide a kind of defining method of relational model, in conjunction with the content of above-described embodiment one, be 2 dimensions for the dimension of sample data, explain explanation in detail to the method that the embodiment of the present invention provides, see Fig. 2, method flow comprises:

201: the logarithm obtaining the variation distribution of log-likelihood, regular terms and each hidden variable determined according to sample data, at least two hidden variables and model parameter, each hidden variable is for illustration of sample class belonging to sample data;

About content and the dimension of sample data, the present embodiment does not do concrete restriction.During concrete enforcement, sample data can be the scoring of multiple user to multiple film, then the dimension of sample data can be 2, namely adds up from user and these two dimensions of film scoring.Certainly, sample data, except foregoing and dimension, can also be other guide and dimension.

For the ease of understanding, be described for sample data as follows.This sample data represents with the form of 5*5 matrix, and the row of matrix represents user 1 to user 5, and matrix column represents film 1 to film 5, the arbitrary element x in matrix _ijrepresent that user i is to the scoring of film j; Wherein 1≤i≤5,1≤j≤5, i and j is integer.

$\left[\begin{array}{ccccc}{x}_{11}& x_{12}& x_{12}& x_{14}& x_{15}\\ x_{21}& x_{22}& x_{23}& x_{24}& x_{25}\\ x_{31}& x_{32}& x_{33}& x_{34}& x_{35}\\ x_{41}& x_{52}& x_{53}& x_{54}& x_{55}\\ x_{51}& x_{52}& x_{53}& x_{54}& x_{55}\end{array}\right]$

Model parameter includes but not limited to the submodel parameter etc. in row mixture ratio, row mixture ratio and each sample class, and the present embodiment does not limit the particular content of model parameter.For the sample data of matrix form, row mixture ratio be the relational model determined each sample class in the line number of matrix account for the ratio of total line number of matrix in the relational model determined, row mixture ratio be the relational model determined each sample class in the columns of matrix account for the ratio of total columns of matrix in the relational model determined, the submodel parameter in each sample class be the relational model determined each sample class in the parameter of Data distribution8.

It should be noted that, hidden variable can be separate with model parameter, all right and model parameter Existence dependency relationship.Due in practical application, hidden variable and model parameter Existence dependency relationship, in order to make the relational model determined more accurate, the embodiment of the present invention is described for hidden variable and model parameter Existence dependency relationship.

In order to obtain the logarithm of the variation distribution of log-likelihood, regular terms and each hidden variable determined according to sample data, at least two hidden variables and model parameter, first the method that the present embodiment provides draws joint probability density function:

Wherein, p represents joint probability density function, for sample data, N _rfor row number of samples, N _cfor row number of samples, Z ^rfor row hidden variable, Z ^cfor row hidden variable, θ is the set of model parameter, model parameter comprise α, β, α, β are respectively row, column mixture ratio, represent the submodel parameter in each sample class, K _rthe number of row sample class, K _cthe number of row sample class, x _ijrepresent the sample data that the i-th row, jth arrange, represent sample data x _ijwhether belong to a kth row sample class, represent sample data xi _jwhether belong to l row sample class, represent the row mixture ratio of a kth row sample class, represent the row mixture ratio of l row sample class.

Above-mentioned joint probability density function determines the probability density distribution of relational model, determine model parameter α in joint probability density function, β, and hidden variable Z ^r, Z ^cthe probability density distribution of relational model can be determined, thus determine relational model.For making joint probability density function separate, take the logarithm respectively to the both members of joint density distribution, obtaining log-likelihood is:

As a kind of embodiment, according to the log-likelihood that sample data, at least two hidden variables and model parameter are determined be:

$\log p(x^{N_1...N_d},Z^1...,Z^d|\mathrm{\θ});$

Wherein, logp () represents log-likelihood, and p represents joint probability density function, for sample data, N _rfor row number of samples, N _cfor row number of samples, Z ^rfor row hidden variable, Z ^cfor row hidden variable, θ is the set of model parameter, model parameter comprise α, β, α, β are respectively row, column mixture ratio, represent the model parameter in each sample class.

Particularly, sample data is worked as when representing by the form of matrix, N _rfor the row number of samples of matrix, N _cfor matrix column number of samples, K _rfor row sample class number, K _cfor row sample class number; Z ^rn _r* K _rvariable blocked matrix; Z ^reach element when time, represent sample data x _ijbelong to a kth row sample class; Z ^cn _c* K _cvariable blocked matrix; Z ^ceach element when time, represent sample data x _ijbelong to l row sample class.Row mixture ratio α represents that the line number in relational model in each row sample class accounts for the ratio of the total line number of sample data matrix, row mixture ratio β represents that the columns in relational model in each row sample class accounts for the ratio of the total columns of sample data matrix, the model parameter in each composition represent the parameter of the distribution that the sample data in relational model in each sample class is obeyed in sample class.Such as, the sample data Gaussian distributed in each sample class, then represent expectation μ and the variance δ of Gaussian distribution; Again such as, the sample data in each sample class obeys Poisson distribution, then represent the expectation and variance λ of Poisson distribution.It should be noted that, except above-mentioned distribution, the sample data of each sample class can also obey other distributions, and the present embodiment does not do concrete restriction to this.

As a kind of embodiment, according to the regular terms that sample data, at least two hidden variables and model parameter are determined be:

$\frac{D_a}{2}\log N_r+\frac{D_{\mathrm{\β}}}{2}\log N_c+\underset{k=1}{\overset{K_r}{\mathrm{\Σ}}}\underset{l=1}{\overset{K_c}{\mathrm{\Σ}}}\frac{D_{\mathrm{kl}}}{2}L(\underset{i=1}{\overset{N_r}{\mathrm{\Σ}}}\underset{j=1}{\overset{N_c}{\mathrm{\Σ}}}{Z}_{\mathrm{ik}}^R{Z}_{\mathrm{jl}}^C,\underset{i=1}{\overset{N_r}{\mathrm{\Σ}}}\underset{j=1}{\overset{N_c}{\mathrm{\Σ}}}{Z}_{\mathrm{ik}}^R{Z}_{\mathrm{jl}}^C,\underset{i=1}{\overset{N_r}{\mathrm{\Σ}}}\underset{j=1}{\overset{N_c}{\mathrm{\Σ}}}\tilde{q}\left(Z_{\mathrm{ik}}^R\right)\tilde{q}\left(Z_{\mathrm{jl}}^C\right));$

Wherein, N _rfor row number of samples, N _cfor row number of samples; K _rthe number of row sample class, K _cit is the number of row sample class; for the approximate value that the variation of hidden variable distributes, be the row hidden variable of i-th row sample data place kth sample class, for the row hidden variable of a jth l sample class in row sample data place; α, β are respectively row, column mixture ratio, D _αfor the dimension of α, D _βfor the dimension of β, D _klfor the dimension of the submodel parameter in the sample class of row k, l row.L (a, b)=lo _gb+ (a-b)/b, a is b is therefore, regular terms can expand into:

$\frac{D_a}{2}\log N_r+\frac{D_{\mathrm{\β}}}{2}\log N_c+\underset{k=1}{\overset{K_r}{\mathrm{\Σ}}}\underset{l=1}{\overset{K_c}{\mathrm{\Σ}}}\frac{D_{\mathrm{kl}}}{2}(\log \underset{i=1}{\overset{N_r}{\mathrm{\Σ}}}\underset{j=1}{\overset{N_c}{\mathrm{\Σ}}}\tilde{q}\left(Z_{\mathrm{ik}}^R\right)\tilde{q}\left(Z_{\mathrm{jl}}^C\right)+(\underset{i=1}{\overset{N_r}{\mathrm{\Σ}}}\underset{j=1}{\overset{N_c}{\mathrm{\Σ}}}{Z}_{\mathrm{ik}}^R{Z}_{\mathrm{jl}}^C-\underset{i=1}{\overset{N_r}{\mathrm{\Σ}}}\underset{j=1}{\overset{N_c}{\mathrm{\Σ}}}\tilde{q}\left(Z_{\mathrm{ik}}^R\right)\tilde{q}\left(Z_{\mathrm{jl}}^C\right))\underset{i=1}{\overset{N_r}{\mathrm{\Σ}}}\underset{j=1}{\overset{N_c}{\mathrm{\Σ}}}\tilde{q}\left(Z_{\mathrm{ik}}^R\right)\tilde{q}\left(Z_{\mathrm{jl}}^C\right));$

Particularly, K is worked as _rrepresent variable blocked matrix Z ^rrow sample class number time, D _α=D (α)=K _r-1; Work as K _crepresent variable blocked matrix Z ^crow sample class number time, D _β=D (β)=K _c-1; When sample data Gaussian distributed in each sample class, because the expectation and variance of Gaussian distribution is respectively μ and δ, namely Gaussian distribution has two parameters, when the sample data in each sample class obeys Poisson distribution, because the expectation and variance of Poisson distribution is λ, namely Poisson distribution has a parameter lambda,

In addition, the present embodiment is not to the approximate value that the variation of hidden variable distributes specifically limit, include but not limited to the value of the variation distribution of the hidden variable of getting the renewal that the last time upgrades or initialization gets.For the ease of understanding, the present embodiment represents the approximate value of the variation distribution of hidden variable with the value of the variation of the hidden variable of the renewal that the last time upgrades or initialization gets distribution for example is described.When determining regular terms first, the value of the variation distribution of the hidden variable of the renewal that the desirable initialization of approximate value of the variation distribution of hidden variable gets; When non-determine regular terms first time, the approximate value desirable last time of the variation distribution of hidden variable upgrades the value of the variation distribution of the hidden variable of renewal got.

It should be noted that, by determining regular terms, making the complexity of the relational model determined can obtain automatic control, improve the determination efficiency of relational model.

As a kind of embodiment, the logarithm of the variation distribution of each hidden variable determined according to sample data, at least two hidden variables and model parameter is:

Logq (Z ^r) and logq (Z ^c);

Wherein, q (Z ^r) be row hidden variable Z ^rvariation distribution, q (Z ^c) be row hidden variable Z ^cvariation distribution.

Particularly, row hidden variable Z ^rvariation distribution can be expressed as:

$q\left(Z^R\right)=\underset{k=1}{\overset{K_r}{\mathrm{\Π}}}q\left(Z_k^R\right)=\underset{k=1}{\overset{K_r}{\mathrm{\Π}}}\underset{i=1}{\overset{N_r}{\mathrm{\Π}}}q{\left(Z_{\mathrm{ik}}^R\right)}^{Z_{\mathrm{ik}}^R};$

The variation distribution of row hidden variable ZC can be expressed as:

$q\left(Z^C\right)=\underset{l=1}{\overset{K_c}{\mathrm{\Π}}}q\left(Z_l^c\right)=\underset{l=1}{\overset{K_c}{\mathrm{\Π}}}\underset{j=1}{\overset{N_c}{\mathrm{\Π}}}q{\left(Z_{\mathrm{jl}}^C\right)}^{Z_{\mathrm{jl}}}.$

202: according to the expectation value determination objective function of the logarithm that the variation of the expectation value of log-likelihood, the expectation value of regular terms and each hidden variable distributes;

In above-mentioned steps 201, log-likelihood represents with the form of factorization, for making log-likelihood separate, using Laplce to be similar to each factor, obtaining the lower bound that compacts of log-likelihood, i.e. FIC(FactorizedInformation Criterion, decomposed information criterion) be:

$\mathrm{FIC}(x^{N_r{N}_c},M)=\underset{q}{\max }\left\{H(q,\stackrel{\‾}{\mathrm{\θ}},x^{N_r{N}_c})\right\};$ Wherein,

$\begin{array}{c}H(q,\stackrel{\‾}{\mathrm{\θ}},x^{N_r{N}_c})=\underset{Z^R{Z}^C}{\mathrm{\Σ}}q\left(Z^R\right)q\left(Z^C\right)\{\log p(x^{N_r{N}_c},Z^R,Z^C|\stackrel{\‾}{\mathrm{\θ}})-\frac{D_{\mathrm{\α}}}{2}\log N_r-\frac{B_{\mathrm{\β}}}{2}\log N_c\\ -\underset{k=1}{\overset{K_r}{\mathrm{\Σ}}}\underset{l=1}{\overset{K_c}{\mathrm{\Σ}}}\frac{D_{\mathrm{kl}}}{2}\log \left(\underset{i=1}{\overset{N_r}{\mathrm{\Σ}}}\underset{j=1}{\overset{N_c}{\mathrm{\Σ}}}{Z}_{\mathrm{ik}}^R{Z}_{\mathrm{jl}}^C\right)-\log q\left(Z^R\right)-\log q\left(Z^C\right)\}\end{array},$

for FIC get maximal value time, the value of θ.

Further, because FIC comprises sample data with hidden variable ZR, ZC, need by EM(Expectation Maximization under normal circumstances, expectation maximization) Algorithm for Solving, but owing to carrying out the determination of relational model with dependent hidden variable, make traditional EM algorithm cannot be applied to solving of FIC.In order to make FIC separate, the present embodiment takes the mode of FIC being carried out to convergent-divergent, and the progressive consistent lower bound obtaining FIC is:

wherein,

$\begin{array}{c}\mathrm{\Γ}(q,\tilde{q},\mathrm{\θ},x^{N_r{N}_c})=\underset{Z^R{Z}^C}{\mathrm{\Σ}}q\left(Z^R\right)q\left(Z^C\right)\{\log p(x^{N_r{N}_c},Z^R,Z^C|\mathrm{\θ})-\frac{D_{\mathrm{\α}}}{2}\log N_r-\frac{B_{\mathrm{\β}}}{2}\log N_c\\ -\underset{k=1}{\overset{K_r}{\mathrm{\Σ}}}\underset{l=1}{\overset{K_c}{\mathrm{\Σ}}}\frac{D_{\mathrm{kl}}}{2}L(\underset{i=1}{\overset{N_r}{\mathrm{\Σ}}}\underset{j=1}{\overset{N_c}{\mathrm{\Σ}}}{Z}_{\mathrm{ik}}^R{Z}_{\mathrm{jl}}^C,\underset{i=1}{\overset{N_r}{\mathrm{\Σ}}}\underset{j=1}{\overset{N_c}{\mathrm{\Σ}}}\tilde{q}\left(Z_{\mathrm{ik}}^R\right)\tilde{q}\left(Z_{\mathrm{jc}}^C\right))-\log q\left(Z^R\right)-\log q\left(Z^C\right)\}\end{array},$

Function L (a, b)=lo _gb+ (a-b)/b.

As a kind of embodiment, the objective function that the expectation value of the logarithm distributed according to the variation of the expectation value of log-likelihood, the expectation value of regular terms and each hidden variable is determined for:

$\begin{array}{c}\mathrm{\Γ}(q,\tilde{q},\mathrm{\θ},x^{N_r{N}_c})=\underset{Z^R{Z}^C}{\mathrm{\Σ}}q\left(Z^R\right)q\left(Z^C\right)\{\log p(x^{N_r{N}_c},Z^R,Z^C|\mathrm{\θ})-\frac{D_{\mathrm{\α}}}{2}\log N_r-\frac{B_{\mathrm{\β}}}{2}\log N_c\\ -\underset{k=1}{\overset{K_r}{\mathrm{\Σ}}}\underset{l=1}{\overset{K_c}{\mathrm{\Σ}}}\frac{D_{\mathrm{kl}}}{2}L(\underset{i=1}{\overset{N_r}{\mathrm{\Σ}}}\underset{j=1}{\overset{N_c}{\mathrm{\Σ}}}{Z}_{\mathrm{ik}}^R{Z}_{\mathrm{jl}}^C,\underset{i=1}{\overset{N_r}{\mathrm{\Σ}}}\underset{j=1}{\overset{N_c}{\mathrm{\Σ}}}\tilde{q}\left(Z_{\mathrm{ik}}^R\right)\tilde{q}\left(Z_{\mathrm{jc}}^C\right))-\log q\left(Z^R\right)-\log q\left(Z^C\right)\}\end{array},$

Further, namely completed the determination of objective function by above-mentioned steps, in order to by objective function determination relational model, the method that the present embodiment provides also comprises subsequent step.

203: the variation distribution obtaining each hidden variable upgraded and the model parameter upgraded;

As a kind of embodiment, the variation distribution obtaining each hidden variable upgraded and the model parameter upgraded, include but not limited to:

The variation distribution of each hidden variable is alternately upgraded, until obtain the variation distribution of each hidden variable of the renewal restrained according to following formula:

Variation according to each hidden variable upgraded distributes according to following formula Renewal model parameter, obtains the model parameter upgraded:

${\mathrm{\α}}_k^{\left(t\right)}=\frac{\underset{i=1}{\overset{N_r}{\mathrm{\Σ}}}{q}^{\left(t\right)}\left(Z_{\mathrm{ik}}^R\right)}{N_r};{\mathrm{\β}}_l^{\left(t\right)}=\frac{\underset{j=1}{\overset{N_c}{\mathrm{\Σ}}}{q}^{\left(t\right)}\left(Z_{\mathrm{jl}}^C\right)}{N_c};$

t represents current renewal, and t-1 represents last renewal or initialization.

Concrete restriction is not done about initialized mode the present embodiment.During specific embodiment, random fashion can be adopted to carry out initialization, namely random initializtion α, β and value.Certainly, except aforesaid way, other modes can also be adopted.

Further, owing to calculating current hidden variable Z ^rvariation distribution time use last hidden variable Z ^cthe result of calculation of variation distribution, then need alternately to upgrade the variation distribution of each hidden variable, by hidden variable Z ^rvariation distribution and hidden variable Z ^cvariation distribution carry out alternately upgrading, until obtain the variation distribution of each hidden variable of the renewal restrained.About the condition of the variation convergence in distribution of each hidden variable, the present embodiment does not do concrete restriction.During concrete enforcement, for row hidden variable Z ^r, the Euclidean distance that the variation distribution that can calculate current line hidden variable distributes with the variation of last row hidden variable, when the Euclidean distance calculated is less than distance threshold, determines the variation convergence in distribution of current line hidden variable.The size of distance threshold can set according to actual conditions, and the present embodiment does not do concrete restriction to this.

Certainly, except above-mentioned determine the mode whether the variation distribute of each hidden variable upgraded restrains except, the variation that the mode arranging update times can also be adopted to obtain each hidden variable restrained distributes.This kind of mode in the specific implementation, when update times reaches the update times threshold value of setting, determines the variation convergence in distribution of each hidden variable, obtains the variation distribution of each hidden variable of the renewal restrained.About the size of the update times threshold value arranged, the present embodiment does not do concrete restriction.

As a kind of embodiment, except the variation distribution that obtains each hidden variable of renewal in the manner described above and the model parameter upgraded, the method that the present embodiment provides also includes but not limited to the mode of the following variation distribution obtaining each hidden variable upgraded and the model parameter upgraded:

According to following formula Renewal model parameter, obtain the model parameter upgraded:

${\mathrm{\α}}_k^{\left(t\right)}=\frac{\underset{i=1}{\overset{N_r}{\mathrm{\Σ}}}{q}^{(t-1)}\left(Z_{\mathrm{ik}}^R\right)}{N_r};{\mathrm{\β}}_l^{\left(t\right)}=\frac{\underset{j=1}{\overset{N_c}{\mathrm{\Σ}}}{q}^{(t-1)}\left(Z_{\mathrm{jl}}^C\right)}{N_c};$

Alternately upgrade the variation distribution of each hidden variable according to the model parameter upgraded according to following formula, obtain the variation distribution of each hidden variable of the renewal restrained:

Wherein, t represents current renewal, and t-1 represents last renewal or initialization.

When the above-mentioned formula in this step 203 be hidden variable for obtaining renewal first variation distribution and upgrade model parameter time, t-1 represents initialization, and the parameter that thus t-1 is corresponding is initialization value.Such as, when this step 203 be the hidden variable obtaining renewal first variation distribution and upgrade model parameter time, in formula represent α _kinitialization value, represent β _linitialization value etc.Do not do concrete restriction about initialized mode the present embodiment, during specific embodiment, the mode of random initializtion can be adopted q (Z ^r) and q (Z ^c) carry out initialization.Certainly, except the mode of random initializtion, other modes can also be adopted.

When the above-mentioned formula in this step 203 is for the non-variation distribution obtaining the hidden variable of renewal first and the model parameter upgraded, t-1 represents last renewal, and the parameter that thus t-1 is corresponding is last updated value.Such as, when this step 203 be the variation distribution obtaining the hidden variable upgraded third time and the model parameter upgraded time, in formula the α obtained when representing the variation distribution obtaining the hidden variable upgraded for the second time and the model parameter upgraded _kvalue, the β obtained when representing the variation distribution obtaining the hidden variable upgraded for the second time and the model parameter upgraded _lvalue etc.

Further, when upgrading the variation distribution of each hidden variable according to the model parameter upgraded, also need hidden variable Z ^rvariation distribution and hidden variable Z ^cvariation distribution carry out alternately upgrading, until obtain the variation distribution of each hidden variable of the renewal restrained.

In addition, when specifically implementing, different row samples can be set and arrange other number K _rwith different row sample component number K _c.Such as, K is set _rminimum value is K _rmin, K _rmaximum occurrences is K _rmax; K is set _cminimum value be K _cmin, K _cmaximum occurrences be K _cmax; At K _rand K _cspan in, for K _rand K _ceach valued combinations, obtain upgrade each hidden variable variation distribution and upgrade model parameter.

It should be noted that, obtain upgrade each hidden variable variation distribution and upgrade model parameter time, first alternately can upgrade the variation distribution of each hidden variable, until obtain the variation distribution of each hidden variable of the renewal restrained, again according to the variation distributed update model parameter of each hidden variable upgraded, obtain the model parameter upgraded; All right first Renewal model parameter, obtain the model parameter upgraded, the variation distribution of each hidden variable is alternately upgraded again according to the model parameter upgraded, obtain the variation distribution of each hidden variable of the renewal restrained, namely the present embodiment does not limit the variation distribution of each hidden variable and the update sequence of model parameter.

204: whether restrain according to the variation distribution of each hidden variable upgraded and the model parameter determination objective function of renewal, if objective function is not restrained, the variation distribution then again obtaining each hidden variable of renewal and the model parameter upgraded, until the variation distribution and the model parameter that obtain each hidden variable making objective function converges;

Particularly, whether restrain according to the variation distribution of each hidden variable upgraded and the model parameter determination objective function of renewal, include but not limited to:

Whether the distance between the objective function that the objective function relatively determined according to the variation distribution of each hidden variable upgraded and the model parameter that upgrades and last time obtain is less than threshold value, and the variation distribution of each hidden variable that the objective function that the last time obtains upgraded according to the last time and the model parameter of renewal are determined;

If the distance between the objective function that the objective function determined according to the variation distribution of each hidden variable upgraded and the model parameter of renewal and last time obtain is less than threshold value, then determine objective function converges.

About the size of threshold value, the present embodiment does not do concrete restriction.During concrete enforcement, different threshold values can be set according to the data volume etc. of sample data.By the model parameter determination objective function that the variation of each hidden variable upgraded distributes and upgrades, objective function is made constantly to approach log-likelihood; When objective function converges, the value of log-likelihood can be approximately the value of objective function, thus unsolvable log-likelihood be converted to the objective function that can separate, achieve the determination of relational model.

It should be noted that, when determining that objective function is not restrained, again obtain each hidden variable of renewal variation distribution and upgrade model parameter time, can step 203 be returned, according to the mode of step 203 again obtain each hidden variable of renewal variation distribution and upgrade model parameter.When first time obtain upgrade each hidden variable variation distribution and upgrade model parameter time, t-1 involved by formula in step 203 represents initial value, but when returning the variation distribution that step 203 obtains each hidden variable of renewal again and the model parameter upgraded, the t-1 involved by the formula in step 203 represent last renewal.Such as, during the model parameter distributed according to the variation of each hidden variable of the formula acquisition renewal in above-mentioned steps 203 first and upgrade, the parameter that t-1 in formula is corresponding uses initial value, the variation distribution of the hidden variable of the renewal got first and the model parameter of renewal.If the variation distribution of the hidden variable of the renewal got first and the model parameter upgraded do not make objective function converges, then using the variation distribution of the hidden variable of renewal got first and the model parameter that the upgrades value as parameter corresponding to t-1 in above-mentioned steps 203, the variation distribution again obtaining the hidden variable of renewal and the model parameter upgraded, and judge whether the variation distribution of the hidden variable of the renewal again got and the model parameter of renewal make the objective function converges determined.Renewal like this, until the variation distribution and the model parameter that obtain the hidden variable making objective function converges.

205: the variation according to each hidden variable making objective function converges distributes and model parameter determination relational model.

For this step, value during objective function converges, close to log-likelihood, distributes and model parameter determination relational model by making the variation of each hidden variable of objective function converges.

Further, different row sample class number K is set _rwith different row sample class number K _c, and for K _rand K _ceach valued combinations, obtain upgrade each hidden variable variation distribution and upgrade model parameter, then on the basis making objective function converges, also can choose the K making objective function value maximum _rand K _c, and by this K _rand K _cthe variation distribution of each hidden variable calculated and model parameter determination relational model.

It should be noted that, the row sample class number K of setting _rwith row sample class number K _cmay be identical with row sample class number with the row sample class number of the relational model determined, also may be different, namely in the process determining relational model, automatically can adjust the structure of relational model.

Namely achieved for 2 dimension sample data determination relational models by above-mentioned steps 201 to step 205, the method that the present embodiment provides also can be used for the determination of the relational model of the sample data more than 2 dimensions.Such as, the dimension of sample data is 3,4,5 etc.

The method provided when the embodiment of the present invention is applied to the relational model of the sample data tieed up more than 2 really regularly, in above-mentioned steps 201, according to the log-likelihood that sample data, at least two hidden variables and model parameter are determined is:

$\log p(x^{N_1...N_d},Z^1...,Z^d|\mathrm{\θ});$

Wherein, logp () represents log-likelihood, and p represents joint probability density function, for sample data, d is the dimension of sample data, N ₁be the number of the 1st dimension sample data, N _dbe the number that d ties up sample data, Z ¹be the hidden variable of the 1st dimension sample data, Z ^dbe the hidden variable that d ties up sample data, θ is the set of model parameter, and model parameter comprises α ¹..., α ^d, α ¹be the mixture ratio of the 1st dimension, α ^dbe the mixture ratio of d dimension, represent model parameter.

According to the regular terms that sample data, at least two hidden variables and model parameter are determined be:

$\underset{i=1}{\overset{d}{\mathrm{\Σ}}}\frac{D_{{\mathrm{\α}}^i}}{2}\log N_i+\underset{p_1=1}{\overset{K_1}{\mathrm{\Σ}}}...\underset{p_d=1}{\overset{K_d}{\mathrm{\Σ}}}\frac{D_{p_1...p_d}}{2}L(\underset{j_1=1}{\overset{N_1}{\mathrm{\Σ}}}...\underset{j_d=1}{\overset{N_d}{\mathrm{\Σ}}}{Z}_{j_1{p}_1}^1...Z_{j_d{p}_d}^d,\underset{j_1=1}{\overset{N_1}{\mathrm{\Σ}}}...\underset{j_d=1}{\overset{N_d}{\mathrm{\Σ}}}{q}^{`}\left(Z_{j_1{p}_1}^1\right)...q^{`}\left(Z_{j_d{p}_d}^d\right));$

Wherein, d is the dimension of sample data, N ₁be the number of the 1st dimension sample data, N _dbe the number that d ties up sample data, N _ibe the number of the i-th dimension sample data, K ₁the number of the 1st dimension sample class, K _dit is the number that d ties up sample class; for the approximate value that the variation of hidden variable distributes, it is the 1st dimension jth ₁individual sample data place p ₁the hidden variable of individual sample class, be that d ties up jth _dindividual sample data place p _dthe hidden variable of individual sample class; α ⁱbe the mixture ratio of the i-th dimension, for α ⁱdimension, be the 1st dimension p ₁individual sample class ... d ties up p _dthe dimension of the submodel parameter in individual sample class, L (a, b)=logb+ (a-b)/b, a is b is

The logarithm of the variation distribution of each hidden variable determined according to sample data, at least two hidden variables and model parameter is:

logq(Z ¹)，…logq(Z ^d)；

Wherein, q (Z ¹) be hidden variable Z ¹variation distribution, q (Z ^d) be hidden variable Z ^dvariation distribution.

In above-mentioned steps 202, according to the expectation value determination objective function of the logarithm that the variation of the expectation value of log-likelihood, the expectation value of regular terms and each hidden variable distributes for:

$\begin{array}{c}\mathrm{\Γ}(q,q^{`},\mathrm{\θ},x^{N_{1...}{N}_d})=\underset{Z^1...Z^d}{\mathrm{\Σ}}q\left(Z^1\right)...q\left(Z^d\right)\{\log p(x^{N_{1...}{N}_d},Z^1,...,Z^d|\mathrm{\θ})-\underset{i=1}{\overset{d}{\mathrm{\Σ}}}\frac{D_{{\mathrm{\α}}_i}}{2}\log N_i\\ -\underset{p_1=1}{\overset{K_1}{\mathrm{\Σ}}}...\underset{p_d=1}{\overset{K_d}{\mathrm{\Σ}}}\frac{D_{p_1...p_d}}{2}L(\underset{j_1=1}{\overset{N_1}{\mathrm{\Σ}}}...\underset{j_d=1}{\overset{N_d}{\mathrm{\Σ}}}{Z}_{j_1{p}_1}^1...Z_{j_d{p}_d}^d,\underset{j_1=1}{\overset{N_1}{\mathrm{\Σ}}}...\underset{j_d=1}{\overset{N_d}{\mathrm{\Σ}}}{q}^{`}\left(Z_{j_d{p}_1}^1\right)...q^{`}\left(Z_{j_d{p}_d}^d\right))-\log q\left(Z^1\right)-...-\log q\left(Z^d\right)\}\end{array}.$

As a kind of embodiment, in above-mentioned steps 203, the variation distribution obtaining each hidden variable upgraded and the model parameter upgraded, include but not limited to:

The variation distribution of each hidden variable is alternately upgraded, until obtain the variation distribution of each hidden variable of the renewal restrained according to following formula:

…

Variation according to each hidden variable upgraded distributes according to following formula Renewal model parameter, obtains the model parameter upgraded:

${\mathrm{\α}}_{p_1}^{1\left(t\right)}=\frac{\underset{j_1=1}{\overset{N_1}{\mathrm{\Σ}}}{q}^{\left(t\right)}\left(Z_{j_1{p}_1}^1\right)}{N_1};....{\mathrm{\α}}_{p_d}^{d\left(t\right)}=\frac{\underset{j_d=1}{\overset{N_d}{\mathrm{\Σ}}}{q}^{\left(t\right)}\left(Z_{j_d{p}_d}^d\right)}{N_d};$

t represents current renewal, and t-1 represents last renewal or initialization.

As a kind of embodiment, in above-mentioned steps 203, the variation distribution obtaining each hidden variable upgraded and the model parameter upgraded, include but not limited to:

According to following formula Renewal model parameter, obtain the model parameter upgraded:

${\mathrm{\α}}_{p_1}^{1\left(t\right)}=\frac{\underset{j_1=1}{\overset{N_1}{\mathrm{\Σ}}}{q}^{(t-1)}\left(Z_{j_1{p}_1}^1\right)}{N_1};....{\mathrm{\α}}_{p_d}^{d\left(t\right)}=\frac{\underset{j_d=1}{\overset{N_d}{\mathrm{\Σ}}}{q}^{(t-1)}\left(Z_{j_d{p}_d}^d\right)}{N_d};$

Alternately upgrade the variation distribution of each hidden variable according to the model parameter upgraded according to following formula, obtain the variation distribution of each hidden variable of the renewal restrained:

…

Wherein, t represents current renewal, and t-1 represents last renewal or initialization.

The implementation of above-mentioned steps 204 and step 205 can directly apply to the sample data more than 2 dimensions, and then determines relational model.

The relational model determined can be used for the cluster of data, also can be used for the classification of data.When the cluster of the relational model determined for data, determine that namely the process of relational model is the process of data clusters; When the classification of the relational model determined for data, the relational model to determining also is needed to be further processed.The result of data clusters and classification is utilized to carry out customer analysis, bioanalysis and geoanalysis etc., a large amount of social value of generation and economic worth.

The method that the embodiment of the present invention provides, by obtaining according to sample data, the log-likelihood that at least two hidden variables for illustration of sample class belonging to sample data and model parameter are determined, the logarithm of the variation distribution of regular terms and each hidden variable, and according to the log-likelihood got, the logarithm determination objective function of the variation distribution of regular terms and each hidden variable, make the mutual relationship being applicable to analysis two class or the above inter-entity of two classes according to the relational model enabling the distribution of the variation of each hidden variable of objective function converges and model parameter determine, thus the range of application of relational model is extended, by introducing regular terms in objective function, the complexity of the relational model determined can be controlled automatically, and then the efficiency determining relational model can be improved.In addition, due to Existence dependency relationship between hidden variable and model parameter, the distribution of the variation of each hidden variable thus determined and model parameter more accurate.

Embodiment three

See Fig. 3, embodiments provide a kind of determining device of relational model, this device comprises:

Acquisition module 301, for obtaining the logarithm of the variation distribution of log-likelihood, regular terms and each hidden variable determined according to sample data, at least two hidden variables and model parameter, each hidden variable is for illustration of sample class belonging to sample data;

First determination module 302, for the logarithm determination objective function distributed according to the variation of log-likelihood, regular terms and each hidden variable;

Second determination module 303, for determining variation distribution and the model parameter of each hidden variable making objective function converges;

3rd determination module 304, for distributing and model parameter determination relational model according to the variation of each hidden variable making objective function converges.

As a kind of embodiment, the log-likelihood that acquisition module 301 gets is:

$\log p(x^{N_1...N_d},Z^1...,Z^d|\mathrm{\θ});$

Wherein, logp () represents log-likelihood, and p represents joint probability density function, for sample data, d is the dimension of sample data, N ₁be the number of the 1st dimension sample data, N _dbe the number that d ties up sample data, Z ¹be the hidden variable of the 1st dimension sample data, Z ^dbe the hidden variable that d ties up sample data, θ is the set of model parameter, and model parameter comprises α ¹..., α ^d, α ¹be the mixture ratio of the 1st dimension, α ^dbe the mixture ratio of d dimension, represent model parameter.

As a kind of embodiment, the regular terms that acquisition module 301 gets is:

$\underset{i=1}{\overset{d}{\mathrm{\Σ}}}\frac{D_{{\mathrm{\α}}^i}}{2}\log N_i+\underset{p_1=1}{\overset{K_1}{\mathrm{\Σ}}}...\underset{p_d=1}{\overset{K_d}{\mathrm{\Σ}}}\frac{D_{p_1...p_d}}{2}L(\underset{j_1=1}{\overset{N_1}{\mathrm{\Σ}}}...\underset{j_d=1}{\overset{N_d}{\mathrm{\Σ}}}{Z}_{j_1{p}_1}^1...Z_{j_d{p}_d}^d,\underset{j_1=1}{\overset{N_1}{\mathrm{\Σ}}}...\underset{j_d=1}{\overset{N_d}{\mathrm{\Σ}}}{q}^{`}\left(Z_{j_1{p}_1}^1\right)...q^{`}\left(Z_{j_d{p}_d}^d\right));$

Wherein, d is the dimension of sample data, N ₁be the number of the 1st dimension sample data, N _dbe the number that d ties up sample data, N _ibe the number of the i-th dimension sample data, K ₁the number of the 1st dimension sample class, K _dit is the number that d ties up sample class; for the approximate value that the variation of hidden variable distributes, be the 1st dimension jth 1 sample data place p ₁the hidden variable of individual sample class, be that d ties up jth _dindividual sample data place p _dthe hidden variable of individual sample class; α ⁱbe the mixture ratio of the i-th dimension, for α ⁱdimension, be the 1st dimension p ₁individual sample class ... d ties up p _dthe dimension of the submodel parameter in individual sample class, L (a, b)=logb+ (a-b)/b, a is b is

As a kind of embodiment, the logarithm of the variation distribution of each hidden variable that acquisition module 301 gets is:

logq(Z ¹)，…logq(Z ^d)；

Wherein, the variation that q (Z1) is hidden variable Z1 distributes, q (Z ^d) be hidden variable Z ^dvariation distribution.

As a kind of embodiment, the first determination module 302, for the expectation value determination objective function of logarithm distributed according to the variation of the expectation value of log-likelihood, the expectation value of regular terms and each hidden variable.

As a kind of embodiment, the objective function that the first determination module 302 is determined for:

$\begin{array}{c}\mathrm{\Γ}(q,q^{`},\mathrm{\θ},x^{N_{1...}{N}_d})=\underset{Z^1...Z^d}{\mathrm{\Σ}}q\left(Z^1\right)...q\left(Z^d\right)\{\log p(x^{N_{1...}{N}_d},Z^1,...,Z^d|\mathrm{\θ})-\underset{i=1}{\overset{d}{\mathrm{\Σ}}}\frac{D_{{\mathrm{\α}}_i}}{2}\log N_i\\ -\underset{p_1=1}{\overset{K_1}{\mathrm{\Σ}}}...\underset{p_d=1}{\overset{K_d}{\mathrm{\Σ}}}\frac{D_{p_1...p_d}}{2}L(\underset{j_1=1}{\overset{N_1}{\mathrm{\Σ}}}...\underset{j_d=1}{\overset{N_d}{\mathrm{\Σ}}}{Z}_{j_1{p}_1}^1...Z_{j_d{p}_d}^d,\underset{j_1=1}{\overset{N_1}{\mathrm{\Σ}}}...\underset{j_d=1}{\overset{N_d}{\mathrm{\Σ}}}{q}^{`}\left(Z_{j_d{p}_1}^1\right)...q^{`}\left(Z_{j_d{p}_d}^d\right))-\log q\left(Z^1\right)-...-\log q\left(Z^d\right)\}\end{array}.$

As a kind of embodiment, see Fig. 4, the second determination module 303, comprising:

Acquiring unit 3031, the variation distribution obtaining each hidden variable upgraded and the model parameter upgraded;

Whether determining unit 3032, for restraining according to the variation distribution of each hidden variable upgraded and the model parameter determination objective function of renewal;

Acquiring unit 3031, for when objective function is not restrained, the variation distribution again obtaining each hidden variable of renewal and the model parameter upgraded, until the variation distribution and the model parameter that obtain each hidden variable making objective function converges.

As a kind of embodiment, see Fig. 5, acquiring unit 3031, comprising:

First upgrades subelement 30311, for alternately upgrading the variation distribution of each hidden variable according to following formula, until obtain the variation distribution of each hidden variable of the renewal restrained:

…

Second upgrades subelement 30312, distributes according to following formula Renewal model parameter, obtain the model parameter upgraded for the variation according to each hidden variable upgraded:

${\mathrm{\α}}_{p_1}^{1\left(t\right)}=\frac{\underset{j_1=1}{\overset{N_1}{\mathrm{\Σ}}}{q}^{\left(t\right)}\left(Z_{j_1{p}_1}^1\right)}{N_1};....{\mathrm{\α}}_{p_d}^{d\left(t\right)}=\frac{\underset{j_d=1}{\overset{N_d}{\mathrm{\Σ}}}{q}^{\left(t\right)}\left(Z_{j_d{p}_d}^d\right)}{N_d};$

t represents current renewal, and t-1 represents last renewal or initialization.

As a kind of embodiment, see Fig. 6, acquiring unit 3031, comprising:

3rd upgrades subelement 30313, for according to following formula Renewal model parameter, obtains the model parameter upgraded:

${\mathrm{\α}}_{p_1}^{1\left(t\right)}=\frac{\underset{j_1=1}{\overset{N_1}{\mathrm{\Σ}}}{q}^{(t-1)}\left(Z_{j_1{p}_1}^1\right)}{N_1};....{\mathrm{\α}}_{p_d}^{d\left(t\right)}=\frac{\underset{j_d=1}{\overset{N_d}{\mathrm{\Σ}}}{q}^{(t-1)}\left(Z_{j_d{p}_d}^d\right)}{N_d};$

4th upgrades subelement 30314, for alternately upgrading the variation distribution of each hidden variable according to following formula according to the model parameter upgraded, obtains the variation distribution of each hidden variable of the renewal restrained:

…

Wherein, t represents current renewal, and t-1 represents last renewal or initialization.

As a kind of embodiment, see Fig. 7, determining unit 3032, comprising:

Relatively subelement 30321, whether be less than threshold value for the distance compared between the objective function that obtains according to the variation distribution of each hidden variable upgraded and the objective function determined of model parameter upgraded and last time, the variation distribution of each hidden variable that the objective function that the last time obtains upgraded according to the last time and the model parameter of renewal are determined;

Determine subelement 30322, when being less than threshold value for the distance between the objective function that the variation distribution of each hidden variable according to renewal and the model parameter of renewal are determined and the objective function that the last time obtains, determine objective function converges.

In sum, the device that the embodiment of the present invention provides, by obtaining according to sample data, the log-likelihood that at least two hidden variables for illustration of sample class belonging to sample data and model parameter are determined, the logarithm of the variation distribution of regular terms and each hidden variable, and according to the log-likelihood got, the logarithm determination objective function of the variation distribution of regular terms and each hidden variable, make the mutual relationship being applicable to analysis two class or the above inter-entity of two classes according to the relational model enabling the distribution of the variation of each hidden variable of objective function converges and model parameter determine, thus the range of application of relational model is extended, by introducing regular terms in objective function, the complexity of the relational model determined can be controlled automatically, and then the efficiency determining relational model can be improved.In addition, due to Existence dependency relationship between hidden variable and model parameter, the distribution of the variation of each hidden variable thus determined and model parameter more accurate.

It should be noted that: the determining device of the relational model that above-described embodiment provides is when determining relational model, only be illustrated with the division of above-mentioned each functional module, in practical application, can distribute as required and by above-mentioned functions and be completed by different functional modules, inner structure by device is divided into different functional modules, to complete all or part of function described above.In addition, the determining device of the relational model that above-described embodiment provides and the determination embodiment of the method for relational model belong to same design, and its specific implementation process refers to embodiment of the method, repeats no more here.

The invention described above embodiment sequence number, just to describing, does not represent the quality of embodiment.

One of ordinary skill in the art will appreciate that all or part of step realizing above-described embodiment can have been come by hardware, the hardware that also can carry out instruction relevant by program completes, described program can be stored in a kind of computer-readable recording medium, the above-mentioned storage medium mentioned can be ROM (read-only memory), disk or CD etc.

The foregoing is only preferred embodiment of the present invention, not in order to limit the present invention, within the spirit and principles in the present invention all, any amendment done, equivalent replacement, improvement etc., all should be included within protection scope of the present invention.

Claims (20)

1. a defining method for relational model, is characterized in that, described method comprises:

Obtain the logarithm of the variation distribution of log-likelihood, regular terms and each hidden variable determined according to sample data, at least two hidden variables and model parameter, each hidden variable is for illustration of sample class belonging to sample data;

According to the logarithm determination objective function that the variation of described log-likelihood, regular terms and each hidden variable distributes;

Determine variation distribution and the model parameter of each hidden variable making described objective function converges, the variation according to each hidden variable making described objective function converges distributes and model parameter determination relational model.

2. method according to claim 1, is characterized in that, according to the log-likelihood that described sample data, at least two hidden variables and model parameter are determined is:

$\log p(x^{N_1...N_d},Z^1...,Z^d|\mathrm{\θ});$

Wherein, described logp () represents log-likelihood, and described p represents joint probability density function, described in for sample data, described d is the dimension of sample data, described N ₁be the number of the 1st dimension sample data, described N _dbe the number that d ties up sample data, described Z ¹be the hidden variable of the 1st dimension sample data, described Z ^dbe the hidden variable that d ties up sample data, described θ is the set of model parameter, and described model parameter comprises α ¹..., α ^d, described α ¹be the mixture ratio of the 1st dimension, described α ^dbe the mixture ratio of d dimension, described in represent model parameter.

3. method according to claim 1, is characterized in that, according to the regular terms that sample data, at least two hidden variables and model parameter are determined is:

$\underset{i=1}{\overset{d}{\mathrm{\Σ}}}\frac{D_{{\mathrm{\α}}^i}}{2}\log N_i+\underset{p_1=1}{\overset{K_1}{\mathrm{\Σ}}}...\underset{p_d=1}{\overset{K_d}{\mathrm{\Σ}}}\frac{D_{p_1...p_d}}{2}L(\underset{j_1=1}{\overset{N_1}{\mathrm{\Σ}}}...\underset{j_d=1}{\overset{N_d}{\mathrm{\Σ}}}{Z}_{j_1{p}_1}^1...Z_{j_d{p}_d}^d,\underset{j_1=1}{\overset{N_1}{\mathrm{\Σ}}}...\underset{j_d=1}{\overset{N_d}{\mathrm{\Σ}}}{q}^{`}\left(Z_{j_1{p}_1}^1\right)...q^{`}\left(Z_{j_d{p}_d}^d\right));$

Wherein, described d is the dimension of sample data, described N ₁be the number of the 1st dimension sample data, described N _dbe the number that d ties up sample data, described N _ibe the number of the i-th dimension sample data, described K ₁the number of the 1st dimension sample class, described K _dit is the number that d ties up sample class; Described for the approximate value that the variation of hidden variable distributes, described in it is the 1st dimension jth ₁individual sample data place p ₁the hidden variable of individual sample class, described in be that d ties up jth _dindividual sample data place p _dthe hidden variable of individual sample class; Described α ⁱbe the mixture ratio of the i-th dimension, described in for described α ⁱdimension, described in be the 1st dimension p ₁individual sample class ... d ties up p _dthe dimension of the submodel parameter in individual sample class, L (a, b)=logb+ (a-b)/b, described a is described b is

4. method according to claim 1, is characterized in that, the logarithm of the variation distribution of each hidden variable determined according to described sample data, at least two hidden variables and model parameter is:

logq(Z ¹)，…logq(Z ^d)；

Wherein, described q (Z ¹) be hidden variable Z ¹variation distribution, described q (Z ^d) be hidden variable Z ^dvariation distribution.

5. the method according to claim arbitrary in Claims 1-4, is characterized in that, the logarithm determination objective function of the described distribution of the variation according to described log-likelihood, regular terms and each hidden variable, comprising:

According to the expectation value determination objective function of the logarithm that the variation of the expectation value of described log-likelihood, the expectation value of described regular terms and each hidden variable described distributes.

6. method according to claim 5, is characterized in that, the objective function that the expectation value of the logarithm distributed according to the variation of the expectation value of described log-likelihood, the expectation value of described regular terms and each hidden variable described is determined for:

$\begin{array}{c}\mathrm{\Γ}(q,q^{`},\mathrm{\θ},x^{N_{1...}{N}_d})=\underset{Z^1...Z^d}{\mathrm{\Σ}}q\left(Z^1\right)...q\left(Z^d\right)\{\log p(x^{N_{1...}{N}_d},Z^1,...,Z^d|\mathrm{\θ})-\underset{i=1}{\overset{d}{\mathrm{\Σ}}}\frac{D_{{\mathrm{\α}}_i}}{2}\log N_i\\ -\underset{p_1=1}{\overset{K_1}{\mathrm{\Σ}}}...\underset{p_d=1}{\overset{K_d}{\mathrm{\Σ}}}\frac{D_{p_1...p_d}}{2}L(\underset{j_1=1}{\overset{N_1}{\mathrm{\Σ}}}...\underset{j_d=1}{\overset{N_d}{\mathrm{\Σ}}}{Z}_{j_1{p}_1}^1...Z_{j_d{p}_d}^d,\underset{j_1=1}{\overset{N_1}{\mathrm{\Σ}}}...\underset{j_d=1}{\overset{N_d}{\mathrm{\Σ}}}{q}^{`}\left(Z_{j_d{p}_1}^1\right)...q^{`}\left(Z_{j_d{p}_d}^d\right))-\log q\left(Z^1\right)-...-\log q\left(Z^d\right)\}\end{array}.$

7. method according to claim 6, is characterized in that, described variation distribution and the model parameter determining each hidden variable making described objective function converges, comprising:

The variation distribution obtaining each hidden variable upgraded and the model parameter upgraded;

Determine whether described objective function restrains according to the variation distribution of each hidden variable upgraded and the model parameter of renewal, if described objective function is not restrained, the variation distribution then again obtaining each hidden variable of renewal and the model parameter upgraded, until the variation distribution and the model parameter that obtain each hidden variable making described objective function converges.

8. method according to claim 7, is characterized in that, the described variation distribution obtaining each hidden variable upgraded and the model parameter upgraded, comprising:

The variation distribution of each hidden variable is alternately upgraded, until obtain the variation distribution of each hidden variable of the renewal restrained according to following formula:

…

Variation according to each hidden variable upgraded distributes according to following formula Renewal model parameter, obtains the model parameter upgraded:

${\mathrm{\α}}_{p_1}^{1\left(t\right)}=\frac{\underset{j_1=1}{\overset{N_1}{\mathrm{\Σ}}}{q}^{\left(t\right)}\left(Z_{j_1{p}_1}^1\right)}{N_1};....{\mathrm{\α}}_{p_d}^{d\left(t\right)}=\frac{\underset{j_d=1}{\overset{N_d}{\mathrm{\Σ}}}{q}^{\left(t\right)}\left(Z_{j_d{p}_d}^d\right)}{N_d};$

Described described t represents current renewal, and described t-1 represents last renewal or initialization.

9. method according to claim 7, is characterized in that, the described variation distribution obtaining each hidden variable upgraded and the model parameter upgraded, comprising:

According to following formula Renewal model parameter, obtain the model parameter upgraded:

${\mathrm{\α}}_{p_1}^{1\left(t\right)}=\frac{\underset{j_1=1}{\overset{N_1}{\mathrm{\Σ}}}{q}^{(t-1)}\left(Z_{j_1{p}_1}^1\right)}{N_1};....{\mathrm{\α}}_{p_d}^{d\left(t\right)}=\frac{\underset{j_d=1}{\overset{N_d}{\mathrm{\Σ}}}{q}^{(t-1)}\left(Z_{j_d{p}_d}^d\right)}{N_d};$

Described

Alternately upgrade the variation distribution of each hidden variable according to the model parameter upgraded according to following formula, obtain the variation distribution of each hidden variable of the renewal restrained:

…

Wherein, described t represents current renewal, and described t-1 represents last renewal or initialization.

10. the method according to claim arbitrary in claim 7 to 9, is characterized in that, the variation distribution of described each hidden variable according to upgrading and the model parameter of renewal determine whether described objective function restrains, and comprising:

Whether the distance between the objective function that the objective function relatively determined according to the variation distribution of each hidden variable upgraded and the model parameter that upgrades and last time obtain is less than threshold value, and the variation distribution of each hidden variable that the objective function that the described last time obtains upgraded according to the last time and the model parameter of renewal are determined;

If the distance between the objective function that the objective function determined according to the variation distribution of each hidden variable upgraded and the model parameter of renewal and last time obtain is less than threshold value, then determine described objective function converges.

The determining device of 11. 1 kinds of relational models, is characterized in that, described device comprises:

Acquisition module, for obtaining the logarithm of the variation distribution of log-likelihood, regular terms and each hidden variable determined according to sample data, at least two hidden variables and model parameter, each hidden variable is for illustration of sample class belonging to sample data;

First determination module, for the logarithm determination objective function distributed according to the variation of described log-likelihood, regular terms and each hidden variable;

Second determination module, for determining variation distribution and the model parameter of each hidden variable making described objective function converges;

3rd determination module, for distributing and model parameter determination relational model according to the variation of each hidden variable making described objective function converges.

12. devices according to claim 11, is characterized in that, the log-likelihood that described acquisition module gets is:

$\log p(x^{N_1...N_d},Z^1...,Z^d|\mathrm{\θ});$

Wherein, described logp () represents log-likelihood, and described p represents joint probability density function, described in for sample data, described d is the dimension of sample data, described N _dbe the number that d ties up sample data, described N _ibe the number of the i-th dimension sample data, described Z ¹be the hidden variable of the 1st dimension sample data, described Z ^dbe the hidden variable that d ties up sample data, described θ is the set of model parameter, and described model parameter comprises α ¹..., α ^d, described α ¹be the mixture ratio of the 1st dimension, described α ^dbe the mixture ratio of d dimension, described in represent model parameter.

13. devices according to claim 11, is characterized in that, the regular terms that described acquisition module gets is:

$\underset{i=1}{\overset{d}{\mathrm{\Σ}}}\frac{D_{{\mathrm{\α}}^i}}{2}\log N_i+\underset{p_1=1}{\overset{K_1}{\mathrm{\Σ}}}...\underset{p_d=1}{\overset{K_d}{\mathrm{\Σ}}}\frac{D_{p_1...p_d}}{2}L(\underset{j_1=1}{\overset{N_1}{\mathrm{\Σ}}}...\underset{j_d=1}{\overset{N_d}{\mathrm{\Σ}}}{Z}_{j_1{p}_1}^1...Z_{j_d{p}_d}^d,\underset{j_1=1}{\overset{N_1}{\mathrm{\Σ}}}...\underset{j_d=1}{\overset{N_d}{\mathrm{\Σ}}}{q}^{`}\left(Z_{j_1{p}_1}^1\right)...q^{`}\left(Z_{j_d{p}_d}^d\right));$

Wherein, described d is the dimension of sample data, described N ₁be the number of the 1st dimension sample data, described N _dbe the number that d ties up sample data, described N _ibe the number of the i-th dimension sample data, described K ₁the number of the 1st dimension sample class, described K _dit is the number that d ties up sample class; Described for the approximate value that the variation of hidden variable distributes, described in it is the 1st dimension jth ₁individual sample data place p ₁the hidden variable of individual sample class, described in be that d ties up jth _dindividual sample data place p _dthe hidden variable of individual sample class; Described α ⁱbe the mixture ratio of the i-th dimension, described in for described α ⁱdimension, described in be the 1st dimension p ₁individual sample class ... d ties up p _dthe dimension of the submodel parameter in individual sample class, L (a, b)=logb+ (a-b)/b, described a is described b is

14. devices according to claim 11, is characterized in that, the logarithm of the variation distribution of each hidden variable that described acquisition module gets is:

logq(Z ¹)，…logq(Z ^d)；

Wherein, described q (Z ¹) distribute for the variation of hidden variable Z1, described q (Z ^d) be hidden variable Z ^dvariation distribution.

15. according to claim 11 to the device described in arbitrary claim in 14, it is characterized in that, described first determination module, for the expectation value determination objective function of the logarithm that the expectation value of the expectation value according to described log-likelihood, described regular terms and the variation of each hidden variable described distribute.

16. devices according to claim 15, is characterized in that, the objective function that described first determination module is determined for:

$\begin{array}{c}\mathrm{\Γ}(q,q^{`},\mathrm{\θ},x^{N_{1...}{N}_d})=\underset{Z^1...Z^d}{\mathrm{\Σ}}q\left(Z^1\right)...q\left(Z^d\right)\{\log p(x^{N_{1...}{N}_d},Z^1,...,Z^d|\mathrm{\θ})-\underset{i=1}{\overset{d}{\mathrm{\Σ}}}\frac{D_{{\mathrm{\α}}_i}}{2}\log N_i\\ -\underset{p_1=1}{\overset{K_1}{\mathrm{\Σ}}}...\underset{p_d=1}{\overset{K_d}{\mathrm{\Σ}}}\frac{D_{p_1...p_d}}{2}L(\underset{j_1=1}{\overset{N_1}{\mathrm{\Σ}}}...\underset{j_d=1}{\overset{N_d}{\mathrm{\Σ}}}{Z}_{j_1{p}_1}^1...Z_{j_d{p}_d}^d,\underset{j_1=1}{\overset{N_1}{\mathrm{\Σ}}}...\underset{j_d=1}{\overset{N_d}{\mathrm{\Σ}}}{q}^{`}\left(Z_{j_d{p}_1}^1\right)...q^{`}\left(Z_{j_d{p}_d}^d\right))-\log q\left(Z^1\right)-...-\log q\left(Z^d\right)\}\end{array}.$

17. devices according to claim 16, is characterized in that, described second determination module, comprising:

Acquiring unit, the variation distribution obtaining each hidden variable upgraded and the model parameter upgraded;

According to the variation distribution of each hidden variable upgraded and the model parameter of renewal, determining unit, for determining whether described objective function restrains;

Described acquiring unit, for when described objective function is not restrained, the variation distribution again obtaining each hidden variable of renewal and the model parameter upgraded, until the variation distribution and the model parameter that obtain each hidden variable making described objective function converges.

18. devices according to claim 17, is characterized in that, described acquiring unit, comprising:

First upgrades subelement, for alternately upgrading the variation distribution of each hidden variable according to following formula, until obtain the variation distribution of each hidden variable of the renewal restrained:

…

Second upgrades subelement, distributes according to following formula Renewal model parameter, obtain the model parameter upgraded for the variation according to each hidden variable upgraded:

${\mathrm{\α}}_{p_1}^{1\left(t\right)}=\frac{\underset{j_1=1}{\overset{N_1}{\mathrm{\Σ}}}{q}^{\left(t\right)}\left(Z_{j_1{p}_1}^1\right)}{N_1};....{\mathrm{\α}}_{p_d}^{d\left(t\right)}=\frac{\underset{j_d=1}{\overset{N_d}{\mathrm{\Σ}}}{q}^{\left(t\right)}\left(Z_{j_d{p}_d}^d\right)}{N_d};$

Described described t represents current renewal, and described t-1 represents last renewal or initialization.

19. devices according to claim 17, is characterized in that, described acquiring unit, comprising:

3rd upgrades subelement, according to following formula Renewal model parameter, obtains the model parameter upgraded:

${\mathrm{\α}}_{p_1}^{1\left(t\right)}=\frac{\underset{j_1=1}{\overset{N_1}{\mathrm{\Σ}}}{q}^{(t-1)}\left(Z_{j_1{p}_1}^1\right)}{N_1};....{\mathrm{\α}}_{p_d}^{d\left(t\right)}=\frac{\underset{j_d=1}{\overset{N_d}{\mathrm{\Σ}}}{q}^{(t-1)}\left(Z_{j_d{p}_d}^d\right)}{N_d};$

Described

4th upgrades subelement, for alternately upgrading the variation distribution of each hidden variable according to following formula according to the model parameter upgraded, obtains the variation distribution of each hidden variable of the renewal restrained:

…

Wherein, described t represents current renewal, and described t-1 represents last renewal or initialization.

20. according to claim 17 to the device described in arbitrary claim in 19, and it is characterized in that, described determining unit, comprising:

Relatively subelement, whether be less than threshold value for the distance compared between the objective function that obtains according to the variation distribution of each hidden variable upgraded and the objective function determined of model parameter upgraded and last time, the variation distribution of each hidden variable that the objective function that the described last time obtains upgraded according to the last time and the model parameter of renewal are determined;

Determine subelement, when being less than threshold value for the distance between the objective function that the variation distribution of each hidden variable according to renewal and the model parameter of renewal are determined and the objective function that the last time obtains, determine described objective function converges.