CN101551789A - Interval propagation reasoning method of Ising graphical model - Google Patents

Abstract

The present invention discloses an interval propagation reasoning method based on Ising graphical model, wherein the method mainly comprises the following steps: firstly establishing an Ising mean field lopping computation tree based on Ising graphical model; and using a mean field interval propagation algorithm based on the Ising mean field lopping computation tree for computing the expected boundary of root node variate. The reasoning method of the invention has lower computing complexity and has an effective reasoning precision through the expected boundary of variate.

Description

The interval propagation reasoning method of Ising graph model

Technical field

The present invention relates to the approximate variation inference method on approximate probability inference method, especially the Ising graph model on the graph model.

Background technology

1.Ising graph model

Ising graph model (Ising graphical model) originates from statistical physics, is based on the Markov random field model of two-value random vector, for fields such as graphical analysis and natural language processing provide important modeling method.It is to be based upon graph structure G=(wherein set of node V is corresponding to Bernoulli random vector x={x for V, the E) probability model on ₁..., x _n∈ 0,1} ⁿ, limit collection E concerns corresponding to the condition independence between variable.(x θ) is the exponential family probability density distribution p of Ising graph model

$\left\{\begin{array}{c}p(x;\mathrm{\θ})=\exp \{\mathrm{\ψ}(x;\mathrm{\θ})-A\left(\mathrm{\θ}\right)\},\\ A\left(\mathrm{\θ}\right)=\log \underset{x}{\mathrm{\Σ}}\exp \left\{\mathrm{\ψ}(x;\mathrm{\θ})\right\},\\ \mathrm{\ψ}(x;\mathrm{\θ})=\underset{i\∈V}{\mathrm{\Σ}}{\mathrm{\θ}}_i{x}_i+\underset{(i,j)\∈E}{\mathrm{\Σ}}{\mathrm{\θ}}_{\mathrm{ij}}{x}_i{x}_j.\end{array}\right.$

Wherein, θ _i, θ _IjThe representation model parameter, and $\∀(i,j)\∉E,$ θ _Ij=0; The logarithm partition function of A (θ) representation model.

On the Ising graph model, the key issue of probability inference is to calculate logarithm partition function A (θ) and marginal probability distribution p (x _i), for general Ising graph model, accurately inference method calculates computation complexity exponential increase with the model scale of A (θ), thus developed various approximate resoning methods, as the method for sampling, variational method etc.

2.Ising average field

Ising average field (Ising mean field) is a kind of basic variation reasoning (varitionalinference) method on the Ising graph model, its basic thought is by the variation conversion probability inference problem to be converted into the functional extreme value problem, and calculates partition function lower bound and variable expectation approximate value by finding the solution functional extreme value.This method has simple and clear variational form, propinquity effect preferably, is the important tool of handling the large-scale complex data.

The variation reasoning is by minimizing the KL distance between free distribution q (x) and the former distribution p (x), the probability inference problem is converted into the functional extreme value problem, and calculates logarithm partition function A (θ) and variable expectation by finding the solution functional extreme value.KL distance between distribution q (x) and the p (x) is

$\mathrm{KL}\left(q\left(x\right)\right|\left|p\left(x\right)\right)=\underset{x}{\mathrm{\Σ}}q\left(x\right)\log \frac{q\left(x\right)}{p\left(x\right)}.$

Carry out variation and change by minimizing KL distance

$A\left(\mathrm{\θ}\right)=\underset{q\left(x\right)}{\max }\{\underset{x}{\mathrm{\Σ}}q\left(x\right)\mathrm{\ψ}(x;\mathrm{\θ})+H\left(q\left(x\right)\right)\},---\left(1\right)$

Wherein, entropy function $H\left(q\left(x\right)\right)=-\underset{x}{\mathrm{\Σ}}q\left(x\right)\log q\left(x\right).$

Because it is higher accurately to find the solution the computational complexity of variation formula (1), the constraint subclass can be handled freely distributing in Ising average field $M_{\mathrm{tract}}\⊆M$ On, calculate lower bound and the variable of A (θ) and expect approximate value.M _TractOn free distribution q (x) be defined in non-intersect variable bunch { c ₁..., c _mOn complete decomposed form $q\left(x\right)={\mathrm{\Π}}_{\mathrm{\α}=1}^m{q}_{\mathrm{\α}}\left(c_{\mathrm{\α}}\right),$ Promptly

$M_{\mathrm{tract}}=\left\{q\left(x\right)\right|q\left(x\right)=\underset{\mathrm{\α}=1}{\overset{m}{\mathrm{\Π}}}{q}_{\mathrm{\α}}\left(c_{\mathrm{\α}}\right)\},$

Then Ising average field variation formula is

$A\left(\mathrm{\θ}\right)\≥\underset{q\left(x\right)\∈M_{\mathrm{tract}}}{\max }\{\underset{x}{\mathrm{\Σ}}q\left(x\right)\mathrm{\ψ}(x;\mathrm{\θ})+H\left(q\left(x\right)\right)\}.---\left(2\right)$

M the average field that can obtain variation formula (2) according to Eulerian equation is iterative, wherein corresponding to variable bunch c _αThe average field iteratively be

$q_{\mathrm{\α}}\left(c_{\mathrm{\α}}\right)\∝\exp \{\underset{i\∈V_{v_{\mathrm{\α}}}}{\mathrm{\Σ}}{\widehat{\mathrm{\θ}}}_i{x}_i+\underset{(i,j)\∈E_{c_{\mathrm{\α}}}}{\mathrm{\Σ}}{\mathrm{\θ}}_{\mathrm{ij}}{x}_i{x}_j\},---\left(3\right)$

Wherein,

1.

Expression variable bunch c _αCorresponding set of node.

2. Expression variable bunch c _αOn the limit collection.

3.

The expression distribution parameter ${\widehat{\mathrm{\θ}}}_i={\mathrm{\θ}}_i+\underset{t\∈N(c_{\mathrm{\α}},i)}{\mathrm{\Σ}}{\mathrm{\θ}}_{\mathrm{it}}{\mathrm{\μ}}_t,$

And $N(c_{\mathrm{\α}},i)=\left\{t\right|(t,i)\∈E,i\∈V_{c_{\mathrm{\α}}},t\∉V_{c_{\mathrm{\α}}}\},$

The variable expectation ${\mathrm{\μ}}_t=\underset{c_{\mathrm{\β}}}{\mathrm{\Σ}}{q}_{\mathrm{\β}}\left(c_{\mathrm{\β}}\right)x_t,$ X wherein _t∈ c _β

Can calculate variable expectation approximate value according to iterative (3), and variable is expected that bringing variation formula (2) into can calculate logarithm partition function lower bound.

Finding the solution functional extreme value is the important step of variation reasoning, also is the calculating core of reasoning process.Directly utilize the iterative functional convergency value that calculates of functional, but its intactly iterative process make whole model information depth intersection, not only computational complexity is higher, and is unfavorable for increasing fresh information, so developed the approximate variation inference method under the incomplete iteration.

Existing approximate variation inference method comprises based on the local training method (local training method) of conviction propagation with based on conviction propagates the BP-SAW method.Local training method utilization conviction message iteration tentatively several times comes the computation model partition function, can under local message, train separately model parameter, reduced computational complexity, also made things convenient for the increase fresh information, but local training method is difficult to measure the computational accuracy that approximate conviction is propagated.The BP-SAW method is calculated tree at graph model SAW (self-avoiding walk) and is gone up the propagation of execution limited number of time conviction, calculate the marginal probability APPROXIMATE DISTRIBUTION, and the APPROXIMATE DISTRIBUTION error bound have been provided based on the message error concept, but this method needs to carry out message propagation on whole model, and computation complexity is higher.

Existing these approximate variation inference methods are mainly propagated based on conviction and are carried out approximate treatment research, and seldom consider other variation inference methods, as average field inference method; Simultaneously computational accuracy is the important indicator of approximate variation reasoning research, and these methods or be difficult to the analytical calculation precision, as local training method, or computation complexity is higher, as the BP-SAW algorithm.

Summary of the invention

The objective of the invention is to overcome the above-mentioned deficiency of prior art, provide a kind of can under low computational complexity, provide variable expectation circle in the inference method of Ising graph model.The technical solution used in the present invention is:

A kind of interval propagation reasoning method based on the Ising graph model calculates tree by definition Ising average field, and realize expectation interval propagation reasoning process on the calculating tree, comprises the following steps:

(1) tree is calculated in definition Ising average field: under the field reasoning of Ising average, and variable bunch c _γThe calculating tree-model be a four-tuple T (D _γ, R, M, Q), wherein:

1) D _γ: with c _γVariable cluster knot point set D for root node _γ={ c _γ∪ Ch (c _γ) ∪ Ch (Ch (c _γ)) ∪ ..., wherein, Ch (c _γ) expression c _γThe child node collection, Ch (c _γ)={ c _β| x _i∈ c _γ, x _j∈ c _β, (i, j) ∈ E, γ ≠ β }, Ch (Ch (c _γ)) expression variables set Ch (c _γ) child node set: $\mathrm{Ch}\left(\mathrm{Ch}\left(c_{\mathrm{\γ}}\right)\right)={\∪}_{c_{\mathrm{\β}}\∈\mathrm{Ch}\left(c_{\mathrm{\γ}}\right)}\mathrm{Ch}\left(c_{\mathrm{\β}}\right),$

2) R: set of relations R={＜c _α, c _β| c _α∈ D _γ, c _β∈ Ch (c _α), wherein, relation＜c _α, c _βExpression c _βBe c _αChild node,

3) M: calculate the bottom-up one way propagation of message on the tree, note $M_{\mathrm{\α}-\mathrm{out}}=\left\{{\mathrm{\μ}}_i\right|i\∈V_{c_{\mathrm{\α}}}\}$ Expression c _αThe output message collection,

$M_{\mathrm{\α}-\mathrm{in}}=U_{c_{\mathrm{\β}}\∈\mathrm{Ch}\left(c_{\mathrm{\α}}\right)}{M}_{\mathrm{\β}-\mathrm{out}}$ Expression c _αInput message set, then M={M _α-out| c _α∈ D _γ,

4) Q: probability distribution collection Q={q _α(c _αθ ' _α) | c _α∈ D _γ, and ${\mathrm{\θ}}_{\mathrm{ij}}{x}_i{x}_j\∝\exp \{\underset{i\∈V_{c_{\mathrm{\α}}}}{\mathrm{\Σ}}{{\mathrm{\θ}}^{\′}}_i{x}_i+\underset{(i,j)\∈E_{c_{\mathrm{\α}}}}{\mathrm{\Σ}}{\mathrm{\θ}}_{\mathrm{ij}}{x}_i{x}_j\},$

Wherein, ${{\mathrm{\θ}}^{\′}}_i={\mathrm{\θ}}_i+\underset{{\mathrm{\μ}}_t\∈M_{\mathrm{\α}-\mathrm{in}}}{\mathrm{\Σ}}{\mathrm{\θ}}_{\mathrm{it}}{\mathrm{\μ}}_t,$

(2) definition Ising average field lopping computation tree: under the field reasoning of Ising average, variable bunch c _γBeta pruning to calculate tree-model be a four-tuple T _c(D _γ, R, M, Q), wherein:

1) D _γ: with c _γVariable cluster knot point set D for root node _γ={ c _γ∪ CCh (c _γ) ∪ CCh (CCh (c _γ)) ∪ ..., wherein, CCh (c _γ) expression c _γThe child node collection, CCh (c _γ)=Ch (c _γ) An (c _γ), CCh (CCh (c _γ)) expression variables set CCh (c _γ) lopping child node collection: $\mathrm{CCh}\left(\mathrm{CCh}\left(c_{\mathrm{\γ}}\right)\right)={\∪}_{c_{\mathrm{\β}}\∈\mathrm{CCh}\left(c_{\mathrm{\γ}}\right)}\mathrm{CCh}\left(c_{\mathrm{\β}}\right),$

2) R: set of relations R={＜c _α, c _β| c _α∈ D _γ, c _β∈ CCh (c _α), wherein, relation＜c _α, c _βExpression c _βBe c _αChild node,

3) M: calculate the bottom-up one way propagation of message on the tree, note $M_{\mathrm{\α}-\mathrm{out}}=\left\{{\mathrm{\μ}}_i\right|i\∈V_{c_{\mathrm{\α}}}\}$ Expression c _αThe output message collection,

$M_{\mathrm{\α}-\mathrm{in}}={\∪}_{c_{\mathrm{\β}}\∈\mathrm{Ch}\left(c_{\mathrm{\α}}\right)}{M}_{\mathrm{\β}-\mathrm{out}}$ Expression c _αInput message set, then M={M _α-out| c _α∈ D _γ,

4) Q: probability distribution collection Q={q _α(c _αθ ' _α) | c _α∈ D _γ, and ${\mathrm{\θ}}_{\mathrm{ij}}{x}_i{x}_j\∝\exp \{\underset{i\∈V_{c_{\mathrm{\α}}}}{\mathrm{\Σ}}{{\mathrm{\θ}}^{\′}}_i{x}_i+\underset{(i,j)\∈E_{c_{\mathrm{\α}}}}{\mathrm{\Σ}}{\mathrm{\θ}}_{\mathrm{ij}}{x}_i{x}_j\},$

Wherein, ${{\mathrm{\θ}}^{\′}}_i={\mathrm{\θ}}_i+\underset{{\mathrm{\μ}}_t\∈M_{\mathrm{\α}-\mathrm{in}}}{\mathrm{\Σ}}{\mathrm{\θ}}_{\mathrm{it}}{\mathrm{\μ}}_t;$

(3) on lopping computation tree, according to c _αThe input message area between calculate this variable bunch the probability distribution interval, make M _α-in ^Int.Expression variable bunch c _αSet between the input message area, promptly $M_{\mathrm{\α}-\mathrm{in}}^{\mathrm{int}.}=\left\{\right[{\mathrm{\μ}}_t^l,{\mathrm{\μ}}_t^u\left]\right|t\∈V_{c_{\mathrm{\β}}},c_{\mathrm{\β}}\∈\mathrm{Ch}\left(c_{\mathrm{\α}}\right)\},$ Variable bunch c _αBetween the parameter region of probability distribution be

${{\mathrm{\θ}}_i}^{\′l}=\min \{{\mathrm{\θ}}_i+\underset{{\mathrm{\μ}}_t\∈M_{\mathrm{\α}-\mathrm{in}}}{\mathrm{\Σ}}{\mathrm{\θ}}_{\mathrm{it}}{\mathrm{\μ}}_t|{\mathrm{\μ}}_t^l\≤{\mathrm{\μ}}_t\≤{\mathrm{\μ}}_t^u\},$

${{\mathrm{\θ}}_i}^{\′u}=\max \{{\mathrm{\θ}}_i+\underset{{\mathrm{\μ}}_t\∈M_{\mathrm{\α}-\mathrm{in}}}{\mathrm{\Σ}}{\mathrm{\θ}}_{\mathrm{it}}{\mathrm{\μ}}_t|{\mathrm{\μ}}_t^l\≤{\mathrm{\μ}}_t\≤{\mathrm{\μ}}_t^u\},$

${{\mathrm{\θ}}_{\mathrm{\α}}}^{\′l}=\{{{\mathrm{\θ}}_i}^{\′l},{\mathrm{\θ}}_{\mathrm{ij}}|i,j\∈V_{c_{\mathrm{\α}}},(i,j)\∈E_{c_{\mathrm{\α}}}\},$

${{\mathrm{\θ}}_{\mathrm{\α}}}^{\′u}=\{{{\mathrm{\θ}}_i}^{\′u},{\mathrm{\θ}}_{\mathrm{ij}}|i,j\∈V_{c_{\mathrm{\α}}},(i,j)\∈E_{c_{\mathrm{\α}}}\}.$

Make A={a ₁, a ₂..., B={b ₁, b ₂... the equipotentiality set one to one of expression element, and A≤B={a _i≤ b _i| i=1,2 ..., variable cluster knot point c then _αThe probability distribution interval be $\left\{q_{\mathrm{\α}}(c_{\mathrm{\α}};{{\mathrm{\θ}}^{\′}}_{\mathrm{\α}})\right|{{\mathrm{\θ}}_{\mathrm{\α}}}^{\′l}\≤{{\mathrm{\θ}}^{\′}}_{\mathrm{\α}}\≤{{\mathrm{\θ}}_{\mathrm{\α}}}^{\′u}\};$

(4), utilize sum-product algorithm to calculate variable bunch c based on the probability distribution interval of variable bunch _αThe output message interval: at probability distribution q _α(c _αθ ' _α) go up and move sum-product algorithm calculating variable x _i∈ c _αExpectation μ _i, i.e. μ _i=Sum-Prod (q _α(c _αθ ' _α)), as given parameter θ ' _iDuring interval, μ _iExtreme value be taken at end points place between parameter region, that is:

${\mathrm{\μ}}_i^l=\min \{\mathrm{Sum}-\mathrm{Prod}\left(q_{\mathrm{\α}}(c_{\mathrm{\α}};{{\mathrm{\θ}}^{\′}}_{\mathrm{\α}})\right)|({{\mathrm{\θ}}^{\′}}_1,{{\mathrm{\θ}}^{\′}}_2,\·\·\·)\∈\{{{\mathrm{\θ}}_1}^{\′l},{{\mathrm{\θ}}_1}^{\′u}\}\×\{{{\mathrm{\θ}}_2}^{\′l},{{\mathrm{\θ}}_2}^{\′u}\}\×\·\·\·\},$

${\mathrm{\μ}}_i^u=\max \{\mathrm{Sum}-\mathrm{Prod}\left(q_{\mathrm{\α}}(c_{\mathrm{\α}};{{\mathrm{\θ}}^{\′}}_{\mathrm{\α}})\right)|({{\mathrm{\θ}}^{\′}}_1,{{\mathrm{\θ}}^{\′}}_2,\·\·\·)\∈\{{{\mathrm{\θ}}_1}^{\′l},{{\mathrm{\θ}}_1}^{\′u}\}\×\{{{\mathrm{\θ}}_2}^{\′l},{{\mathrm{\θ}}_2}^{\′u}\}\×\·\·\·\}$

Variable cluster knot point c then _αThe interval set of output message is $M_{\mathrm{\α}-\mathrm{out}}^{\mathrm{int}.}=\left\{\right[{\mathrm{\μ}}_i^l,{\mathrm{\μ}}_i^u\left]\right|i\∈V_{c_{\mathrm{\α}}}\},$ During calculating, according to the interval of bottom variable cluster knot point input message, the bottom-up message interval propagation that successively carries out calculates root variable bunch variable expectation interval.

Interval propagation reasoning method based on the Ising graph model of the present invention, calculate tree by definition Ising average field, according to the interval [0 1] of bottom variable cluster knot point input message, the bottom-up message interval propagation that successively carries out calculates root variable bunch variable expectation circle.The present invention can calculate root variable bunch variable expectation circle by using propagation algorithm between the average place on the lopping computation tree of Ising average field.This inference method has lower computational complexity, and has provided effective reasoning precision by variable expectation circle.

Description of drawings

Fig. 1: tree is calculated in Ising graph model and 3 layers of average field thereof;

Fig. 1 (a): 3 * 3 two-dimentional trellis Ising graph model;

Fig. 1 (b): the free distributed architecture in average field of 3 * 1 piecemeals;

Fig. 1 (c): based on Fig. 1 (b) structure, with variable bunch c ₁Calculate tree-model for the k=3 layer average field of root, reach the bottom-up communication process of message;

Fig. 2: on 3 * 3 two dimensional graph models,, be the k=3 layer calculating tree of root with node 1 based on 3 * 3 free distributed architectures;

Fig. 3: on 3 * 3 two dimensional graph models,, be the lopping computation tree of the k=3 layer of root with node 1 based on 3 * 3 free distributed architectures;

Fig. 4: the free distributed architecture based on 3 * 1, the variable expectation circle that the MFIP algorithm provides on k (k=2,3,4) the layer lopping computation tree relatively.Wherein μ represents the expectation of variable, and the solid line at variable place is represented k=2 from left to right successively, 3,4 o'clock variable expectation circle, some expression variable expectation exact value;

Fig. 5: gravitation Ising graph model G ₂Last variable expectation circle compares, and wherein, μ represents the variable expectation value, and solid line is represented the variable expectation circle that the MFIP algorithm provides on 2 layers of lopping computation tree, and dotted line is represented the variable expectation circle that the BP-SAW algorithm provides, and some expression variable is expected exact value;

Fig. 6: repulsion Ising graph model G ₃Last variable expectation circle relatively, all same Fig. 5 of meaning of representing of μ, solid line, dotted line, point wherein.

Embodiment

At first inference method of the present invention is described in detail below.

1.Ising tree is calculated in the average field

It is to represent average field iterative computation process on the Ising graph model with tree-building version that tree is calculated in Ising average field.

Under the field reasoning of definition 1:Ising average, variable bunch c _γThe calculating tree-model be a four-tuple T (D _γ, R, M, Q).

Wherein:

5) D _γ: with c _γVariable cluster knot point set D for root node _γ={ c _γ∪ Ch (c _γ) ∪ Ch (Ch (c _γ)) ∪ ...

Wherein, Ch (c _γ) expression c _γThe child node collection, Ch (c _γ)={ c _β| x _i∈ c _γ, x _j∈ c _β, (i, j) ∈ E, γ ≠ β }, Ch (Ch (c _γ)) expression variables set Ch (c _γ) child node set: $\mathrm{Ch}\left(\mathrm{Ch}\left(c_{\mathrm{\γ}}\right)\right)={\∪}_{c_{\mathrm{\β}}\∈\mathrm{Ch}\left(c_{\mathrm{\γ}}\right)}\mathrm{Ch}\left(c_{\mathrm{\β}}\right).$

6) R: set of relations R={＜c _α, c _β| c _α∈ D _γ, c _β∈ Ch (c _α).

Relation＜c wherein _α, c _βExpression c _βBe c _αChild node.

7) M: calculate the bottom-up one way propagation of message on the tree, note $M_{\mathrm{\α}-\mathrm{out}}=\left\{{\mathrm{\μ}}_i\right|i\∈V_{c_{\mathrm{\α}}}\}$ Expression c _αThe output message collection,

$M_{\mathrm{\α}-\mathrm{in}}={\∪}_{c_{\mathrm{\β}}\∈\mathrm{Ch}\left(c_{\mathrm{\α}}\right)}{M}_{\mathrm{\β}-\mathrm{out}}$ Expression c _αInput message set, then M={M _α-out| c _α∈ D _γ.

8) Q: probability distribution collection Q={q _α(c _αθ ' _α) | c _α∈ D _γ, and ${\mathrm{\θ}}_{\mathrm{ij}}{x}_i{x}_j\∝\exp \{\underset{i\∈V_{c_{\mathrm{\α}}}}{\mathrm{\Σ}}{{\mathrm{\θ}}^{\′}}_i{x}_i+\underset{(i,j)\∈E_{c_{\mathrm{\α}}}}{\mathrm{\Σ}}{\mathrm{\θ}}_{\mathrm{ij}}{x}_i{x}_j\},$

Wherein ${{\mathrm{\θ}}^{\′}}_i={\mathrm{\θ}}_i+\underset{{\mathrm{\μ}}_t\∈M_{\mathrm{\α}-\mathrm{in}}}{\mathrm{\Σ}}{\mathrm{\θ}}_{\mathrm{it}}{\mathrm{\μ}}_t.$

Ising average field lopping computation tree is a kind of special shape that Ising calculates tree, and it carries out the lopping gained by the node of recalling that the average field is calculated on the tree.

Under the field reasoning of definition 2:Ising average, variable bunch c _γBeta pruning to calculate tree-model be a four-tuple T _c(D _γ, R, M, Q).

Wherein:

5) D _γ: with c _γVariable cluster knot point set D for root node _γ={ c _γ∪ CCh (c _γ) ∪ CCh (CCh (c _γ)) ∪ ...

Wherein, CCh (c _γ) expression c _γThe child node collection, CCh (c _γ)=Ch (c _γ) An (c _γ),

CCh (CCh (c _γ)) expression variables set CCh (c _γ) lopping child node collection:

$\mathrm{CCh}\left(\mathrm{CCh}\left(c_{\mathrm{\γ}}\right)\right)={\∪}_{c_{\mathrm{\β}}\∈\mathrm{CCh}\left(c_{\mathrm{\γ}}\right)}\mathrm{CCh}\left(c_{\mathrm{\β}}\right).$

6) R: set of relations R={＜c _α, c _β| c _α∈ D _γ, c _β∈ CCh (c _α).

Relation＜c wherein _α, c _βExpression c _βBe c _αChild node.

7) M: calculate the bottom-up one way propagation of message on the tree, note $M_{\mathrm{\α}-\mathrm{out}}=\left\{{\mathrm{\μ}}_i\right|i\∈V_{c_{\mathrm{\α}}}\}$ Expression c _αThe output message collection,

$M_{\mathrm{\α}-\mathrm{in}}={\∪}_{c_{\mathrm{\β}}\∈\mathrm{Ch}\left(c_{\mathrm{\α}}\right)}{M}_{\mathrm{\β}-\mathrm{out}}$ Expression c _αInput message set, then M={M _α-out| c _α∈ D _γ.

8) Q: probability distribution collection Q={q _α(c _αθ ' _α) | c _α∈ D _γ, and ${\mathrm{\θ}}_{\mathrm{ij}}{x}_i{x}_j\∝\exp \{\underset{i\∈V_{c_{\mathrm{\α}}}}{\mathrm{\Σ}}{{\mathrm{\θ}}^{\′}}_i{x}_i+\underset{(i,j)\∈E_{c_{\mathrm{\α}}}}{\mathrm{\Σ}}{\mathrm{\θ}}_{\mathrm{ij}}{x}_i{x}_j\},$

Wherein ${{\mathrm{\θ}}^{\′}}_i={\mathrm{\θ}}_i+\underset{{\mathrm{\μ}}_t\∈M_{\mathrm{\α}-\mathrm{in}}}{\mathrm{\Σ}}{\mathrm{\θ}}_{\mathrm{it}}{\mathrm{\μ}}_t.$

Theorem 1: under the field reasoning of Ising average, tree T (D is calculated in k layer average field _γ, R, M Q) goes up root node c _γProbability distribution, be equivalent to c under k average field iteration _γProbability distribution.

2. propagation algorithm between the average place

Calculate tree design MFIP algorithm based on Ising.

The MFIP basic idea is that on Ising average field calculating tree T, according to the interval [01] of bottom variable cluster knot point input message, the bottom-up message interval propagation that successively carries out calculates root variable bunch variable expectation interval.The basic calculating unit of MFIP algorithm is a communication process between message area on the variable bunch.For the variable cluster knot point c that calculates on the tree _α, make q _α(c _αθ ' _α) expression node probability distribution, [μ _i ^l, μ _i ^u] expression variable x _i∈ c _αThe expectation interval, [θ ' _i ^l, θ ' _i ^u] expression probability distribution relevant parameter interval.Variable cluster knot point c _αOn the interval propagation process comprise between message area output procedure between input and message area.

In the 1st step, (message interval input MII), is according to c to input process between message area _αThe input message area between calculate this variable bunch the probability distribution interval.Make M _α-in ^Int.Expression variable bunch c _αSet between the input message area, promptly

$M_{\mathrm{\α}-\mathrm{in}}^{\mathrm{int}.}=\left\{\right[{\mathrm{\μ}}_t^l,{\mathrm{\μ}}_t^u\left]\right|t\∈V_{c_{\mathrm{\β}}},c_{\mathrm{\β}}\∈\mathrm{Ch}\left(c_{\mathrm{\α}}\right)\}.$

According to the probability distribution of calculating the last variable cluster knot point of tree as can be known, variable bunch c _αBetween the parameter region of probability distribution be

${{\mathrm{\θ}}_i}^{\′l}=\min \{{\mathrm{\θ}}_i+\underset{{\mathrm{\μ}}_t\∈M_{\mathrm{\α}-\mathrm{in}}}{\mathrm{\Σ}}{\mathrm{\θ}}_{\mathrm{it}}{\mathrm{\μ}}_t|{\mathrm{\μ}}_t^l\≤{\mathrm{\μ}}_t\≤{\mathrm{\μ}}_t^u\},$

${{\mathrm{\θ}}_i}^{\′u}=\max \{{\mathrm{\θ}}_i+\underset{{\mathrm{\μ}}_t\∈M_{\mathrm{\α}-\mathrm{in}}}{\mathrm{\Σ}}{\mathrm{\θ}}_{\mathrm{it}}{\mathrm{\μ}}_t|{\mathrm{\μ}}_t^l\≤{\mathrm{\μ}}_t\≤{\mathrm{\μ}}_t^u\},$

${{\mathrm{\θ}}_{\mathrm{\α}}}^{\′l}=\{{{\mathrm{\θ}}_i}^{\′l},{\mathrm{\θ}}_{\mathrm{ij}}|i,j\∈V_{c_{\mathrm{\α}}},(i,j)\∈E_{c_{\mathrm{\α}}}\},$

${{\mathrm{\θ}}_{\mathrm{\α}}}^{\′u}=\{{{\mathrm{\θ}}_i}^{\′u},{\mathrm{\θ}}_{\mathrm{ij}}|i,j\∈V_{c_{\mathrm{\α}}},(i,j)\∈E_{c_{\mathrm{\α}}}\}.$

Make A={a ₁, a ₂..., B={b ₁, b ₂... the equipotentiality set one to one of expression element, and A≤B={a _i≤ b _i| i=1,2 ..., variable cluster knot point c then _αThe probability distribution interval be $\left\{q_{\mathrm{\α}}(c_{\mathrm{\α}};{{\mathrm{\θ}}^{\′}}_{\mathrm{\α}})\right|{{\mathrm{\θ}}_{\mathrm{\α}}}^{\′l}\≤{{\mathrm{\θ}}^{\′}}_{\mathrm{\α}}\≤{{\mathrm{\θ}}_{\mathrm{\α}}}^{\′u}\}.$

In the 2nd step, (message interval output MIO), based on the probability distribution interval of variable bunch, utilizes sum-product algorithm to calculate variable bunch c to output procedure between message area _αThe output message interval.At probability distribution q _α(c _αθ ' _α) go up and move sum-product algorithm calculating variable x _i∈ c _αExpectation μ _i, i.e. μ _i=Sum-Prod (q _α(c _αθ ' _α)).According to Ising graph model character μ as can be known _iAbout coefficient θ ' ₁, θ ' ₂... have strictly monotone, so as given parameter θ ' _iDuring interval, μ _iExtreme value be taken at end points place between parameter region, promptly

${\mathrm{\μ}}_i^l=\min \{\mathrm{Sum}-\mathrm{Prod}\left(q_{\mathrm{\α}}(c_{\mathrm{\α}};{{\mathrm{\θ}}^{\′}}_{\mathrm{\α}})\right)|({{\mathrm{\θ}}^{\′}}_1,{{\mathrm{\θ}}^{\′}}_2,\·\·\·)\∈\{{{\mathrm{\θ}}_1}^{\′l},{{\mathrm{\θ}}_1}^{\′u}\}\×\{{{\mathrm{\θ}}_2}^{\′l},{{\mathrm{\θ}}_2}^{\′u}\}\×\·\·\·\},$

${\mathrm{\μ}}_i^u=\max \{\mathrm{Sum}-\mathrm{Prod}\left(q_{\mathrm{\α}}(c_{\mathrm{\α}};{{\mathrm{\θ}}^{\′}}_{\mathrm{\α}})\right)|({{\mathrm{\θ}}^{\′}}_1,{{\mathrm{\θ}}^{\′}}_2,\·\·\·)\∈\{{{\mathrm{\θ}}_1}^{\′l},{{\mathrm{\θ}}_1}^{\′u}\}\×\{{{\mathrm{\θ}}_2}^{\′l},{{\mathrm{\θ}}_2}^{\′u}\}\×\·\·\·\}.$

Variable cluster knot point c then _αThe interval set of output message is $M_{\mathrm{\α}-\mathrm{out}}^{\mathrm{int}.}=\left\{\right[{\mathrm{\μ}}_i^l,{\mathrm{\μ}}_i^u\left]\right|i\∈V_{c_{\mathrm{\α}}}\}.$

Calculate tree T at k layer Ising average field, the bottom-up message interval propagation that successively carries out, it is interval to calculate bunch variable expectation of root variable.K layer Ising calculates on the tree T, and the formalized description of MFIP algorithm is as follows:

Data：T(D _γ，R，M，Q)，k

Result： $\left\{\right[{\mathrm{\μ}}_s^l,{\mathrm{\μ}}_s^u\left]\right|s\∈V_{c_{\mathrm{\γ}}}\}$

Begin

For?layer?l←k?to?1?do

For?all?c _α?of?layer?l?do

If?l≡k?then $M_{\mathrm{\α}-\mathrm{in}}^{\mathrm{int}.}\←\left\{\right[\mathrm{0,1}],[\mathrm{0,1}],\·\·\·\}$

Else $M_{\mathrm{\α}-\mathrm{in}}^{\mathrm{int}.}\←{\∪}_{c_{\mathrm{\β}}\∈\mathrm{Ch}\left(c_{\mathrm{\α}}\right)}{M}_{\mathrm{\β}-\mathrm{out}}^{\mathrm{int}.}$

End

$({{\mathrm{\θ}}_{\mathrm{\α}}}^{\′l},{{\mathrm{\θ}}_{\mathrm{\α}}}^{\′u})\←\mathrm{MII}\left(M_{\mathrm{\α}-\mathrm{in}}^{\mathrm{int}.}\right)$

$\left\{q_{\mathrm{\α}}(c_{\mathrm{\α}};{{\mathrm{\θ}}^{\′}}_{\mathrm{\α}})\right|{{\mathrm{\θ}}_{\mathrm{\α}}}^{\′l}\≤{{\mathrm{\θ}}^{\′}}_{\mathrm{\α}}\≤{{\mathrm{\θ}}_{\mathrm{\α}}}^{\′u}\}$

$M_{\mathrm{\α}-\mathrm{out}}^{\mathrm{int}.}\←\mathrm{MIO}\left(\right\{q_{\mathrm{\α}}(c_{\mathrm{\α}};{{\mathrm{\θ}}^{\′}}_{\mathrm{\α}})|{{\mathrm{\θ}}_{\mathrm{\α}}}^{\′l}\≤{{\mathrm{\θ}}^{\′}}_{\mathrm{\α}}\≤{{\mathrm{\θ}}_{\mathrm{\α}}}^{\′u}\})$

End

End

$\left\{q_{\mathrm{\γ}}(c_{\mathrm{\γ}};{{\mathrm{\θ}}^{\′}}_{\mathrm{\γ}})\right|{{\mathrm{\θ}}_{\mathrm{\γ}}}^{\′l}\≤{{\mathrm{\θ}}^{\′}}_{\mathrm{\γ}}\≤{{\mathrm{\θ}}_{\mathrm{\γ}}}^{\′u}\}$

$\left\{\right[{\mathrm{\μ}}_s^l,{\mathrm{\μ}}_s^u\left]\right|s\∈V_{c_{\mathrm{\γ}}}\}$

End

3. expect the boundary based on the variable of lopping computation tree MFIP algorithm

Theorem 2: order $\left\{\right[{\mathrm{\μ}}_s^l,{\mathrm{\μ}}_s^u\left]\right|s\∈V_{c_{\mathrm{\γ}}}\}$ The interval set of variable expectation that expression MFIP algorithm provides, μ _s ^*Variable x on the expression Ising graph model _sThe exact value of expectation.Then at average field lopping computation tree T _c(D _γ, R, M, Q) on, the expectation of variable that the MFIP algorithm provides is interval expects the boundary for variable, promptly ${\mathrm{\μ}}_s^l\≤{\mathrm{\μ}}_s^{*}\≤{\mathrm{\μ}}_s^u,$ $\∀x_s\∈c_{\mathrm{\γ}}.$

Below in conjunction with embodiment, the present invention will be further described.

1. set up Ising average field lopping computation tree based on the Ising graph model

Two-dimentional trellis Ising graph model to 3 * 3 is shown in Fig. 1 (a).

The designated model parameter generates general Ising graph model G at random ₁(θ _i∈ (0.25,0.25), θ _Ij∈ (2,2)), gravitation Ising graph model G ₂(θ _i∈ (0.25,0.25), θ _Ij∈ (0,2)) and repulsion Ising graph model G ₃(θ _i∈ (0.25,0.25), θ _Ij∈ (2,0)).For graph model G ₁, set up number of plies k=2 respectively based on freely distributing of 3 * 1 piecemeals, 3,4 lopping computation tree (k=3 is as Fig. 1 (c)).For graph model G ₂, G ₃, set up 2 layers of lopping computation tree-model based on freely distributing of 3 * 1 piecemeals respectively.

2. calculate root variable bunch variable expectation circle

(the mean field interval propagation of propagation algorithm between the average place, MFIP) basic thought is on the lopping computation tree of Ising average field, interval [0 1] according to bottom variable cluster knot point input message, the bottom-up message interval propagation that successively carries out calculates root variable bunch variable expectation circle.

For different graph models, relatively MFIP algorithm, associating tree algorithm (junction tree, JT) and BP-SAW algorithm (self-avoiding walk).The compactness on the efficient of comparison algorithm and variable expectation circle.

For G ₁, operation MFIP algorithm computation variable expectation circle on lopping computation tree; At G ₁Last operation associating tree algorithm is calculated the exact value of variable expectation, and the result as shown in Figure 4.To interpretation as can be known, the interval convergence rapidly of variable expectation that provides with the increase MFIP algorithm of lopping computation tree number of plies k, and the MFIP algorithm provides variable and expects the boundary when k=2.

For G ₂, G ₃, operation MFIP algorithm computation variable expectation circle on lopping computation tree; At G ₂, G ₃On move BP-SAW algorithm computation variable expectation circle respectively; Then at G ₂, G ₃On move the exact value of JT algorithm computation variable expectation respectively, the result is respectively shown in Fig. 5, Fig. 6 and table 1.To interpretation as can be known, for gravitation graph model G ₂, compact than expectation circle that BP-SAW algorithm provides in the variable expectation circle that the MFIP algorithm provides on 2 layers of lopping computation tree; For repulsion graph model G ₃, in 9 groups of data of Ising graph model, based on the MFIP algorithm of 2 layers of lopping computation tree than BP-SAW algorithm expectation circle tight have 5 groups, pine have 2 groups, other 2 groups of data presentation BP-SAW algorithms do not provide the expectation circle.

At last, as shown in Table 1, compare with the BP-SAW algorithm, the MFIP algorithm has higher efficient.Be that MFIP is a kind of efficiently approximate easily variation inference method; The variable expectation circle that the MFIP algorithm provides on 2 layers of lopping computation tree has higher compactness.

Table 1: variable expectation circle compares, and wherein, μ represents the variable expectation, and t represents the average operating time of algorithm.

Claims (1)

1. the interval propagation reasoning method of an Ising graph model calculates tree by definition Ising average field, and realize expectation interval propagation reasoning process on the calculating tree, it is characterized in that, comprises the following steps:

(1) tree is calculated in definition Ising average field: under the field reasoning of Ising average, and variable bunch c _γThe calculating tree-model be a four-tuple T (D _γ, R, M, Q), wherein:

1) D _γ: with c _γVariable cluster knot point set D for root node _γ={ c _γ∪ Ch (c _γ) ∪ Ch (Ch (c _γ)) ∪ ..., wherein, Ch (c _γ) expression c _γThe child node collection, Ch (c _γ)={ c _β| x _i∈ c _γ, x _j∈ c _β, (i, j) ∈ E, γ ≠ β }, Ch (Ch (c _γ)) expression variables set Ch (c _γ) child node set: $\mathrm{Ch}\left(\mathrm{Ch}\left(c_{\mathrm{\γ}}\right)\right)={\∪}_{c_{\mathrm{\β}}\∈\mathrm{Ch}\left(c_{\mathrm{\γ}}\right)}\mathrm{Ch}\left(c_{\mathrm{\β}}\right),$

2) R: set of relations R={＜c _α, c _β| c _α∈ D _γ, c _β∈ Ch (c _α), wherein, relation＜c _α, c _βExpression c _βBe c _αChild node,

3) M: calculate the bottom-up one way propagation of message on the tree, note $M_{\mathrm{\α}-\mathrm{out}}=\left\{{\mathrm{\μ}}_i\right|i\∈V_{c_{\mathrm{\α}}}\}$ Expression c _αThe output message collection,

$M_{\mathrm{\α}-\mathrm{in}}={\∪}_{c_{\mathrm{\β}}\∈\mathrm{Ch}\left(c_{\mathrm{\α}}\right)}{M}_{\mathrm{\β}-\mathrm{out}}$ Expression c _αInput message set, then M={M _α-out| c _α∈ D _γ,

4) Q: probability distribution collection Q={q _α(c _αθ ' _α) | c _α∈ D _γ, and ${\mathrm{\θ}}_{\mathrm{ij}}{x}_i{x}_j\∝\exp \{\underset{i\∈V_{c_{\mathrm{\α}}}}{\mathrm{\Σ}}{{\mathrm{\θ}}^{\′}}_i{x}_i+\underset{(i,j)\∈E_{c_{\mathrm{\α}}}}{\mathrm{\Σ}}{\mathrm{\θ}}_{\mathrm{ij}}{x}_i{x}_j\},$

Wherein, ${{\mathrm{\θ}}^{\′}}_i={\mathrm{\θ}}_i+\underset{{\mathrm{\μ}}_t\∈M_{\mathrm{\α}-\mathrm{in}}}{\mathrm{\Σ}}{\mathrm{\θ}}_{\mathrm{it}}{\mathrm{\μ}}_t,$

(2) definition Ising average field lopping computation tree: under the field reasoning of Ising average, variable bunch c _γBeta pruning to calculate tree-model be a four-tuple T _c(D _γ, R, M, Q), wherein:

1) D _γ: with c _γVariable cluster knot point set D for root node _γ={ c _γ∪ CCh (c _γ) ∪ CCh (CCh (c _γ)) ∪ ..., wherein, CCh (c _γ) expression c _γThe child node collection, CCh (c _γ)=Ch (c _γ) An (c _γ), CCh (CCh (c _γ)) expression variables set CCh (c _γ) lopping child node collection: $\mathrm{CCh}\left(\mathrm{CCh}\left(c_{\mathrm{\γ}}\right)\right)={\∪}_{c_{\mathrm{\β}}\∈\mathrm{CCh}\left(c_{\mathrm{\γ}}\right)}\mathrm{CCh}\left(c_{\mathrm{\β}}\right),$

2) R: set of relations R={＜c _α, c _β| c _α∈ D _γ, c _β∈ CCh (c _α), wherein, relation＜c _α, c _βExpression c _βBe c _αChild node,

3) M: calculate the bottom-up one way propagation of message on the tree, note $M_{\mathrm{\α}-\mathrm{out}}=\left\{{\mathrm{\μ}}_i\right|i\∈V_{c_{\mathrm{\α}}}\}$ Expression c _αThe output message collection,

$M_{\mathrm{\α}-\mathrm{in}}={\∪}_{c_{\mathrm{\β}}\∈\mathrm{Ch}\left(c_{\mathrm{\α}}\right)}{M}_{\mathrm{\β}-\mathrm{out}}$ Expression c _αInput message set, then M={M _α-out| c _α∈ D _γ,

4) Q: probability distribution collection Q={q _α(c _αθ ' _α) | c _α∈ D _γ, and ${\mathrm{\θ}}_{\mathrm{ij}}{x}_i{x}_j\∝\exp \{\underset{i\∈V_{c_{\mathrm{\α}}}}{\mathrm{\Σ}}{{\mathrm{\θ}}^{\′}}_i{x}_i+\underset{(i,j)\∈E_{c_{\mathrm{\α}}}}{\mathrm{\Σ}}{\mathrm{\θ}}_{\mathrm{ij}}{x}_i{x}_j\},$ Wherein, ${{\mathrm{\θ}}^{\′}}_i={\mathrm{\θ}}_i+\underset{{\mathrm{\μ}}_t\∈M_{\mathrm{\α}-\mathrm{in}}}{\mathrm{\Σ}}{\mathrm{\θ}}_{\mathrm{it}}{\mathrm{\μ}}_t;$

(3) on lopping computation tree, according to c _αThe input message area between calculate this variable bunch the probability distribution interval, make M _α-in ^Int.Expression variable bunch c _αSet between the input message area, promptly $M_{\mathrm{\α}-\mathrm{in}}^{\mathrm{int}.}=\left\{\right[{\mathrm{\μ}}_t^l,{\mathrm{\μ}}_t^u\left]\right|t\∈V_{c_{\mathrm{\β}}},$ c _β∈ Ch (c _α), variable bunch c _αBetween the parameter region of probability distribution be

${{\mathrm{\θ}}_i}^{\′l}=\min \{{\mathrm{\θ}}_i+\underset{{\mathrm{\μ}}_t\∈M_{\mathrm{\α}-\mathrm{in}}}{\mathrm{\Σ}}{\mathrm{\θ}}_{\mathrm{it}}{\mathrm{\μ}}_t|{\mathrm{\μ}}_t^l\≤{\mathrm{\μ}}_t\≤{\mathrm{\μ}}_t^u\},$

${{\mathrm{\θ}}_i}^{\′u}=\max \{{\mathrm{\θ}}_i+\underset{{\mathrm{\μ}}_t\∈M_{\mathrm{\α}-\mathrm{in}}}{\mathrm{\Σ}}{\mathrm{\θ}}_{\mathrm{it}}{\mathrm{\μ}}_t|{\mathrm{\μ}}_t^l\≤{\mathrm{\μ}}_t\≤{\mathrm{\μ}}_t^u\},$

${{\mathrm{\θ}}_{\mathrm{\α}}}^{\′l}=\{{{\mathrm{\θ}}_i}^{\′l},{\mathrm{\θ}}_{\mathrm{ij}}|i,j\∈V_{c_{\mathrm{\α}}},(i,j)\∈E_{c_{\mathrm{\α}}}\},$

${{\mathrm{\θ}}_{\mathrm{\α}}}^{\′u}=\{{{\mathrm{\θ}}_i}^{\′u},{\mathrm{\θ}}_{\mathrm{ij}}|i,j\∈V_{c_{\mathrm{\α}}},(i,j)\∈E_{c_{\mathrm{\α}}}\}.$

Make A={a ₁, a ₂..., B={b ₁, b ₂... the equipotentiality set one to one of expression element, and A≤B={a _i≤ b _i| i=1,2 ..., variable cluster knot point c then _αThe probability distribution interval be

$\left\{q_{\mathrm{\α}}(c_{\mathrm{\α}};{{\mathrm{\θ}}^{\′}}_{\mathrm{\α}})\right|{{\mathrm{\θ}}_{\mathrm{\α}}}^{\′l}\≤{{\mathrm{\θ}}^{\′}}_{\mathrm{\α}}\≤{{\mathrm{\θ}}_{\mathrm{\α}}}^{\′u}\};$

(4), utilize sum-product algorithm to calculate variable bunch c based on the probability distribution interval of variable bunch _αThe output message interval: at probability distribution q _α(c _αθ ' _α) go up and move sum-product algorithm calculating variable x _i∈ c _αExpectation μ _i, i.e. μ _i=Sum-Prod (q _α(c _αθ ' _α)), as given parameter θ ' _iDuring interval, μ _iExtreme value be taken at end points place between parameter region, that is:

${\mathrm{\μ}}_i^l=\min \{\mathrm{Sum}-\mathrm{Prod}\left(q_{\mathrm{\α}}(c_{\mathrm{\α}};{{\mathrm{\θ}}^{\′}}_{\mathrm{\α}})\right)|({{\mathrm{\θ}}^{\′}}_1,{{\mathrm{\θ}}^{\′}}_2,\·\·\·)\∈\{{{\mathrm{\θ}}_1}^{\′l},{{\mathrm{\θ}}_1}^{\′u}\}\×\{{{\mathrm{\θ}}_2}^{\′l},{{\mathrm{\θ}}_2}^{\′u}\}\×\·\·\·\},$

${\mathrm{\μ}}_i^u=\max \{\mathrm{Sum}-\mathrm{Prod}\left(q_{\mathrm{\α}}(c_{\mathrm{\α}};{{\mathrm{\θ}}^{\′}}_{\mathrm{\α}})\right)|({{\mathrm{\θ}}^{\′}}_1,{{\mathrm{\θ}}^{\′}}_2,\·\·\·)\∈\{{{\mathrm{\θ}}_1}^{\′l},{{\mathrm{\θ}}_1}^{\′u}\}\×\{{{\mathrm{\θ}}_2}^{\′l},{{\mathrm{\θ}}_2}^{\′u}\}\×\·\·\·\},$

Variable cluster knot point c then _αThe interval set of output message is $M_{\mathrm{\α}-\mathrm{out}}^{\mathrm{int}.}=\left\{\right[{\mathrm{\μ}}_i^l,{\mathrm{\μ}}_i^u]i\∈V_{c_{\mathrm{\α}}}\},$ During calculating, according to the interval of bottom variable cluster knot point input message, the bottom-up message interval propagation that successively carries out calculates root variable bunch variable expectation interval.

Images