KR102153161B1 - Method and system for learning structure of probabilistic graphical model for ordinal data - Google Patents

Abstract

Disclosed is a method and system for learning association of sequence data based on a probability graph. A method for learning association of sequence data performed by an association learning system according to an embodiment includes: predicting a correlation between respective variables in sequence data; And providing a predicted correlation between the respective variables as a graph.

Description

Sequence data association learning method and system based on probability graph {METHOD AND SYSTEM FOR LEARNING STRUCTURE OF PROBABILISTIC GRAPHICAL MODEL FOR ORDINAL DATA}

The following description relates to a method and system for learning association of sequence data based on a probability graph.

An undirected graphic model called Markov Random Field (MRF) models a multivariate random variable, which uses a promiscuous graph to model a conditional independent structure between the variables. These conditional independence structures provide useful insights into how different variables interact with each other. As a result, MRF is widely used in various fields such as natural language processing, biology and medicine.

Korean Patent Laid-Open Publication No. 10-2013-0052432 relates to a pattern recognition method based on a Markov chain concealment conditional random field model, extracting a feature vector from a training input signal measured for a specific pattern, and combining the entire covariance Gaussian distribution. The hidden conditional random field model to which the parameter is applied receives a plurality of combinations of feature vectors and labels indicating the specific pattern to obtain the parameters of the hidden conditional random field model, and the hidden conditional random field model to which the parameter is applied is actually Disclosed is a configuration in which a label indicating an actual pattern is inferred by receiving a feature vector extracted from a test input signal measured for a pattern.

Probability graph-based sequence data association learning method and system can be provided.

A method for learning association of sequence data performed by an association learning system includes: predicting a correlation between respective variables in sequence data; And providing a predicted correlation between the respective variables as a graph.

The step of predicting the correlation between each variable in the sequence data includes designating a node conditional distribution through a univariate sequence distribution for the variable, and searching for a binding distribution by performing analysis on the designated node conditional distribution. It may include steps.

를 포함하는 p차원의 확률 벡터

이고, 각각의 확률 변수

에 대응하는 노드를 갖는 그래프를

라고 하면, 확률 벡터의 모든 노드 조건부 분포가 수학식 1(

)의 단변량 누적 비율 모델에 적용될 경우, 각 노드

에 대하여, 위치 파라미터

가 나머지 변수의 임의 함수일 수 있다. Predicting the correlation between each variable in the sequence data, the domain

P-dimensional probability vector containing

And each random variable

A graph with nodes corresponding to

If, then, the conditional distribution of all nodes of the probability vector is Equation 1 (

), when applied to the univariate cumulative ratio model, each node

Regarding, the location parameter

Can be any function of the remaining variables.

를 포함하는 p 차원의 확률 벡터

에서, 각각의 확률 변수

에 대응하는 노드를 갖는 그래프를

라고 하면, 확률 벡터의 모든 노드 조건부 분포가 수학식 2(

)의 단변수 연속 비율 모델에 적용될 경우, 각 노드

에 대하여, 위치 파라미터

가 나머지 변수의 임의 함수이고,

에 대하여, 특정 노드 조건부 분포가 확률 벡터 Y를 통한 임의의 결합 분포에 대한 마르코프와 일치하지 않는

실수값 파라미터가 존재할 수 있다. Predicting the correlation between each variable in the sequence data, the domain

P-dimensional probability vector containing

In, each random variable

A graph with nodes corresponding to

If, then, the conditional distribution of all nodes of the probability vector is Equation 2 (

), when applied to a univariate continuous ratio model, each node

Regarding, the location parameter

Is an arbitrary function of the remaining variables,

For, a certain node conditional distribution does not match Markov's for any joint distribution through the probability vector Y

There may be real value parameters.

에 대한 연속 비율 모델은 충분한 통계

:

를 갖는 지수족에 포함되며,

이고, 서열 확률 벡터

에 대하여 노드 조건부 분포를 지정하기 위하여 단변량 서열 분포를 사용할 경우, 각 노드

에 대해 수학식 3(

)과 같이 표현하고,

에 대해

이고, 위치 파라미터

가 나머지 변수의 임의 함수일 수 있다. Predicting the correlation between each variable in the sequence data, the sequence random variable

The continuous ratio model for

:

It is included in the exponential family having

Is, the sequence random vector

When using a univariate sequence distribution to specify a node conditional distribution for each node

For Equation 3(

), and

About

And the positional parameter

Can be any function of the remaining variables.

와 관련하여, 마르코프인 결합 분포와 일치하며 크기가 가장 큰 2개의 요소를 갖는 쌍으로 된 경우, 수학식 4(

)와 같이 표현되고, 상기 연속 비율 모델의 파라미터를 추정하기 위하여, 각 노드

에서 정규화된 노드 조건부 로그 우드 최대화 문제를 해결할 수 있다. Predicting the correlation between each variable in the sequence data includes the step of matching the node conditional distribution with the binding distribution, the graph

In relation to, if the paired with the two elements of the largest size coinciding with the Markovin bond distribution, Equation 4 (

), and to estimate the parameters of the continuous ratio model, each node

The node conditional logwood maximization problem normalized in can be solved.

는 잠재 다변량 가우시안 확률 벡터

에 의해 생성되고,

와

일 때, 각

가

의 이산화를 통해 획득될 수 있다. The step of predicting the correlation between each variable in the sequence data is, when the multivariate latent probability vector in the multivariate quantization sequence distribution is multivariate Gaussian, it is called a multivariate probit model, and the dependency is on the potential probability vector through a Gaussian distribution. And in the multivariate probit model, a sequence random vector

Is a latent multivariate Gaussian random vector

Created by,

Wow

When, each

end

Can be obtained through the discretization of

, iff

일 때,

는 임계값이

,

으로 설정되며, Y의 밀도 함수,

가 수학식 5(

)와 같이 제안되고,

와

가

에 의하여 정의된 하이퍼큐브일 수 있다. Predicting the correlation between each variable in the sequence data,

, iff

when,

Is the threshold

,

Is set to the density function of Y,

Equation 5(

) Is proposed as,

Wow

end

It may be a hypercube defined by.

가 파라미터

를 포함하는 프로빗 모델로부터 유도된 확률 벡터 Y로부터 실현될 경우,

로부터 파라미터

를 학습하는

-정규화된 최대 우드(ML) 추정기가 수학식 6(

)과 같은 형식으로 표시되고,

가 diagonal entries를 제외한 항목별

표준일 수 있다. Predicting the correlation between each variable in the sequence data,

Parameter

When realized from a probability vector Y derived from a probit model containing

Parameters from

To learn

-Normalized maximum Wood (ML) estimator is Equation 6 (

), and

Is by item excluding diagonal entries

It can be standard.

를 추정하고, 이변량 주변 분포로부터 polychoric 상관 관계

를 추정할 수 있다. The step of predicting the correlation between each variable in the sequence data includes a threshold value around a univariate in order to estimate an unknown parameter in the probit graph model distribution.

And the polychoric correlation from the distribution around the bivariate

Can be estimated.

를 추정하기 위하여, 이변량 주변 우도로부터 원시 추정치

를 계산하고, sparse 잠재 그래프와 평활화된 추정치

를 추정하기 위하여 예측된 공분산 행렬

을 그래픽 lasso 추정기로 플러그인할 수 있다. Predicting the correlation between each variable in the sequence data, polychoric correlation from the distribution around the bivariate

To estimate, the raw estimate from the likelihood around the bivariate

And the sparse latent graph and smoothed estimate

Covariance matrix predicted to estimate

Can be plugged into a graphical lasso estimator.

에 대해

를 추정하면,

의 결합 분포는 확률

를 갖는 다항식이고, 확률 변수

,

의 확률 분포가 평균 [0, 0]과 공분산

를 갖는 이변량 정규 분포이며,

, 이변량 주변 로그 우도 함수를 최대화함으로써 파라미터

를 수학식 7 (

)을 통해 추정하고,

,

일 수 있다. Predicting the correlation between each variable in the sequence data,

About

If you estimate

The combined distribution of the probability

Is a polynomial with

,

The probability distribution of is covariates with the mean [0, 0]

Is a bivariate normal distribution with

, Parameters by maximizing the log likelihood function around the bivariate

Equation 7 (

) Through estimation,

,

Can be

를 추정하기 위하여

를 추정기

로 대체하고, 수학식 8(

)과 같이 로그 우도를 최대화하고,

은

의 도메인이며, (-1, 1)일 수 있다. Predicting the correlation between each variable in the sequence data, the

To estimate

Estimator

And Equation 8 (

) To maximize the log likelihood,

silver

Domain of, and may be (-1, 1).

를 플러그인할 수 있다. The step of predicting the correlation between each variable in the sequence data may be performed by using a parametric Gaussian graph model estimator to obtain the structure of the graph and the final covariance.

Can be plugged in.

The association learning system includes: a prediction unit that predicts a correlation between each variable in sequence data; And a providing unit that provides a predicted correlation between the respective variables in a graph.

The association learning system according to an embodiment enables identification of the relationship through analysis of sequence data.

The association learning system according to an embodiment may increase an understanding of the generation and structure of data by grasping the association between variables in sequence data analysis.

The association learning system according to an embodiment may recommend specific information based on the association between respective variables.

1 shows a comparison of various estimates when data is generated from a probit model having a chain graph structure in an embodiment.
2 and 3 show data sampled from a Consec model with a 2D grid structure (10 x 5 grid) in one embodiment.
FIG. 4 shows a potential tentative graph structure corresponding to SmokeNow and socio-demographic indicators in an embodiment.
5 is a block diagram illustrating a configuration of a relevance learning system according to an embodiment.
6 is a flowchart illustrating a method of performing association learning in the association learning system according to an embodiment.

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings.

5 is a block diagram illustrating a configuration of a association learning system according to an embodiment, and FIG. 6 is a flowchart illustrating a method of performing association learning in a association learning system according to an embodiment.

The association learning system 100 is for learning the association of sequence data based on a probability graph, and may include a prediction unit 510 and a providing unit 520. Components of the association learning system may control the association learning system 100 to perform steps 610 to 620 included in the method for performing association learning of FIG. 6.

In step 610, the prediction unit 510 may predict the correlation between each variable in the sequence data, and in step 620, the provider 520 graphs the predicted correlation between the respective variables. Can be provided as Hereinafter, in the following description, learning of sequence data association based on a probability graph will be described in detail.

A model based on a multivariate probability ratio will be described. First, it is possible to designate a node conditional distribution through a conventional univariate sequence distribution, and search for the corresponding binding distribution through Hammersley-Clifford-esque analysis.

-MRF through Univariate Latent Quantified Ordinal Models

로 표시되는 실수값 잠재 확률 변수

가 있다고 가정하자. 여기서,

는 분포의 위치 파라미터이다. 서열 확률 변수

는 어떤 일부 위치(또는 컷 포인트) 파라미터

,

에 대해

,

와 같은 실수값 변수 Z의 이산화된 버전으로 작성될 수 있다.CDF

Real-valued latent random variable represented by

Suppose there is. here,

Is the positional parameter of the distribution. Sequence random variable

Is some position (or cut point) parameter

,

About

,

Can be written as a discrete version of the real-valued variable Z, such as

The probability mass function of the sequence variable Y can be expressed as follows.

Equation 1:

는 즉 위의 함수

가 로지스틱 함수

가 되도록 한다. 이 경우, 앞서 설명한 분포는 로그-오즈비의 측면에서도 보다 간결한 형식으로 표현될 수 있다. The popular distribution for the real latent variable Z is the univariate logistic distribution, where

Is the above function

Logistic function

Let it be. In this case, the distribution described above can be expressed in a more concise form in terms of log-ods ratio.

Hence, the rank of the sequence distribution is also called the cumulative proportion model.

The univariate sequence distribution of Equation 1 can be used to specify a node conditional distribution and derive a consistent binding distribution.

를 p차원의 서열 확률 벡터라고 하자. 표기법을 단순화하기 위하여, 후속에서 확률 변수

의 도메인은 동일하고,

와 같다고 가정한다.

를 각각의 확률 변수에 대응하는 노드로 나타낸 그래프라고 하자.

Let be the p-dimensional sequence probability vector. To simplify the notation, random variables in the subsequent

The domain of is the same,

Is assumed to be equal to

Let be a graph represented by nodes corresponding to each random variable.

에 대하여, 수학식 2와 같이 표현할 수 있다. bracket

With respect to, it can be expressed as in Equation 2.

Equation 2:

는 나머지 변수들의 임의의 함수이고,

는 로지스틱 함수이다. 노드 조건부 분포가 일관된 결합 분포로 이어지지 않는 것을 증명하는 다음의 정리를 제시한다. Where, the position parameter

Is an arbitrary function of the remaining variables,

Is a logistic function. We present the following theorem, which proves that the node conditional distribution does not lead to a consistent joint distribution.

를 포함하는 p차원의 확률 벡터

고려하자. 그리고, 각각의 확률 변수

에 대응하는 노드를 가진 그래프를

라 하자. 확률 벡터의 모든 노드 조건부 분포가 수학식 2의 단변량 누적 비율 모델을 따른다고 가정하면, 각 노드

에 대하여, 위치 파라미터

는 나머지 변수의 임의 함수이다. Theorem 1: domain

P-dimensional probability vector containing

Let's consider. And, each random variable

A graph with nodes corresponding to

Let's do it. Assuming that the conditional distribution of all nodes of the probability vector follows the univariate cumulative proportion model of Equation 2, each node

Regarding, the location parameter

Is an arbitrary function of the remaining variables.

에 대하여, 특정 노드-조건부 분포가 크기가 최대 2인 그래프 G에 대한 Markov인 Y에 대한 임의의 결합 분포와 일치하지 않는 실수값 파라미터

가 존재한다. then,

For, a real-valued parameter in which a particular node-conditional distribution does not match any joint distribution for Y, which is Markov for a graph G with a size of up to 2

Exists.

-MRFs via Continuation Ratio Models

A modification of the log-odd ratio, which is closely related to the cumulative ratio model, is considered.

를 나타낼 때, 확률 변수 Y의 확률 질량 함수(RMF)는 다음과 같이 유도될 수 있다.The univariate probability distribution class is also referred to as a continuous ratio model. From the log-odd ratio above

When represents, the probability mass function RMF of the random variable Y can be derived as follows.

Equation 3:

에 대해,

About,

는 다음과 같이 수정될 수 있다. then,

Can be modified as follows.

, PMF의 합계는 1이 된다.

, The sum of PMF becomes 1.

에 대해 수학식 4를 가지고 있다고 가정하기로 하자.Specifically, each node

Suppose we have Equation 4 for

Equation 4:

과 위치 파라미터

는 나머지 변수의 임의 함수이다. 다음 정리는 이러한 노드 조건부 분포가 일관된 결합 분포로 나타나지 않는 것을 증명한다. here,

And position parameters

Is an arbitrary function of the remaining variables. The following theorem proves that this node conditional distribution does not appear as a coherent joint distribution.

를 포함하는 p 차원의 확률 벡터

를 고려한다.

는 각각의 확률 변수

에 대응하는 노드를 갖는 그래프라고 가정하자. 확률 벡터의 모든 노드 조건부 분포가 수학식 4의 단변수 연속 비율 모델을 따른다고 가정하면, 각 노드

에 대하여, 위치 파라미터

는 나머지 변수의 임의 함수이다.Theorem 2: domain

P-dimensional probability vector containing

Consider.

Is each random variable

Suppose it is a graph with nodes corresponding to. Assuming that the conditional distribution of all nodes of the probability vector follows the univariate continuous ratio model of Equation 4, each node

Regarding, the location parameter

Is an arbitrary function of the remaining variables.

에 대하여, 특정 노드 조건부 분포가 Y를 통한 임의의 결합 분포, 즉, 무방향성 그래프 G에 대한 마르코프와 일치하지 않는

실수값 파라미터 가 존재한다.then,

For, a certain node conditional distribution does not coincide with an arbitrary joint distribution through Y, that is, Markov for an undirected graph G

There are real-valued parameters.

-MRF through MRFs via a Consecutive Ratio model

The univariate cumulative proportion model and the continuous proportion model are not included in the exponential family, and in particular, they do not have a regularity that allows a coherent coupling to exist in the node conditionals belonging to this distribution. In other words, each node conditional distribution has irregularities in the univariate cumulative ratio model and the continuous ratio model.

Consider a third class of univariate sequence distribution called a continuous ratio model, defined as:

As can be seen below, the sequence distribution is included in the exponential family, unlike the sequence distribution described above.

에 대한 연속 비율 모델은 충분한 통계

:

를 갖는 지수족에 속하며,

이다.Proposition 1: Sequence random variable

The continuous ratio model for

:

Belongs to the exponential family with

to be.

에 대해 노드 조건부 분포를 지정하기 위하여 단변량 서열 분포를 사용한다고 가정하자. 특히, 각 노드

에 대해, 수학식 5와 같이 표현할 수 있다.Sequence random vector

Suppose we use a univariate sequence distribution to specify a node conditional distribution for. Specifically, each node

For can be expressed as in Equation 5.

Equation 5:

에 대해

이고, 위치 파라미터

는 나머지 변수의 임의 함수이다. 노드 조건부 분포는 단변량 지수족에 속하기 때문에 명제 1을 적용하면 다음 정리를 산출할 수 있다. here,

About

And the positional parameter

Is an arbitrary function of the remaining variables. Since the node conditional distribution belongs to the univariate exponential family, applying the proposition 1 can yield the following theorem.

Theorem 3: In Equation 5, the node conditional distribution matches the joint distribution.

와 관련하여, 마르코프인 결합 분포와 일치하며, 크기가 가장 큰 2개의 요소를 갖는 쌍으로 된 경우에는 다음과 같은 형식을 취할 수 있다. Undirected graph

Regarding, in the case of a pair with two elements of the largest size, consistent with the Markovin bond distribution, the following form can be taken:

In Theorem 3, the distribution can be rewritten in the same manner as in Equation 6.

Equation 6:

를 통하여 Y의 순서를 쌍으로 기재한다.

The order of Y is described in pairs.

에서 정규화된 노드 조건부 로그 우드 최대화 문제를 해결한다. In order to estimate the parameters of the continuous ratio model of Equation 6, each node

Solve the problem of maximizing the node conditional logwood normalized in.

,

,

은 트레이닝 샘플이고,

이다.here,

Is the training sample,

to be.

Existing results of statistical guarantees for estimators of exponential graph models lead to continuous ratio models.

Discrete vs. Contrast/Nominal Graph Model: Contrast the continuous ratio model of Equation 6 with the classic discrete nominal graph model, which treats random variables as nominal variables at each node. For the probability vector Y, a discrete graph model such as Equation 7 is considered.

Equation 7:

,

의 다른 값에 대해 공통 엣지 파라미터

를 가지지 않는다. 범주형 모델의 각각의 엣지는 M² 변수를 사용하여 파라미터화 된다. 결과적으로 이산 그래프 모델은 Y의 순서를 사용하지 않고, 연속 비율 모델과 비교했을 때 더 복잡하다. 이 파라미터화는 연속 비율 모델 파라미터화를 포함하는 반면, 주요 단점은 명목 그래프 모델이 더 많은 파라미터를 가지므로 샘플 복잡성이 더 크다는 것이다.Unlike the continuous ratio model, the discrete graph model

,

Common edge parameter for different values of

Does not have Each edge of the categorical model is parameterized using an M ² variable. Consequently, the discrete graph model does not use the order of Y and is more complex compared to the continuous ratio model. While this parameterization involves the continuous ratio model parameterization, the main drawback is that the nominal graph model has more parameters and therefore the sample complexity is greater.

-Multivariate Latent Quantized Models

Consider constructing a multivariate sequence graph model directly from a univariate sequence distribution. We reconsider the classical and most popular class of univariate sequence distributions based on quantization of real-valued latent variables. The natural class of a multivariate distribution can be obtained by taking a multivariate latent probability vector and quantizing to obtain a multivariate sequence probability vector.

-Probit Graphical Model

The most common example of a multivariate quantization sequence distribution is when the multivariate latent probability vector is multivariate Gaussian, which is also known as a multivariate probit model. Thus, the dependency can be expressed by a potential probability vector through a Gaussian distribution.

는 잠재 다변량 가우시안 확률 벡터

에 의해 생성되는 것으로 가정하고,

와

이다. 각

는 다음과 같이

의 이산화를 통해 획득될 수 있다.In the probit model, the sequence random vector

Is a latent multivariate Gaussian random vector

Is assumed to be generated by

Wow

to be. bracket

Is as follows

Can be obtained through the discretization of

, iff

일 때,

는 임계값이

,

으로 설정된다. 그러면, Y의 밀도 함수,

는 수학식 8과 같이 주어진다.

, iff

when,

Is the threshold

,

Is set to Then, the density function of Y,

Is given as in Equation 8.

Equation 8:

와

는

에 의하여 정의된 하이퍼큐브이다. here,

Wow

Is

It is a hypercube defined by.

는 파라미터

를 포함하는 프로빗 모델로부터 유도된 확률 벡터 Y로부터 실현된다고 하자. 그러면,

로부터 파라미터

를 학습하는

-정규화된 최대 우드(ML) 추정기가 수학식 9와 같은 형식을 취한다.

Is the parameter

Suppose that it is realized from a probability vector Y derived from a probit model including. then,

Parameters from

To learn

-The normalized maximum Wood (ML) estimator takes the same form as in Equation 9.

Equation 9:

는 diagonal entries를 제외한 항목별

표준이다. 목적이 비볼록하고 일반적으로 최적화하기가 어렵다는 것을 알 수 있다. 모델 파라미터를 학습하기 위하여 근사 EM 기반 접근법이 제안되었지만, 여전히 상대적으로 계산적으로 요구되고 있으며, 실제 정규화된 MLE 솔루션에 대하여 강력한 통계 보증을 제공하지 않는다.

Is for each item excluding diagonal entries

It is standard. It can be seen that the purpose is non-convex and is generally difficult to optimize. Although an approximate EM-based approach has been proposed to learn model parameters, it is still relatively computationally required and does not provide strong statistical guarantees for actual normalized MLE solutions.

-A Direct Estimation Method

를 추정하고, 두 번째 단계에서는 이변량 주변 분포로부터 polychoric상관관계

를 추정한다. In Equation 8, we propose an alternative procedure for estimating unknown parameters in the probit graph model distribution. As a two-step procedure, in the first step, the threshold value around the univariate

Is estimated, and in the second step, polychoric correlation from the distribution around the bivariate

Estimate

-ESTIMATION OF THRESHOLDS

의 추정량,

를 다음과 같이 정의한다.

The estimator of,

Is defined as follows.

는 표준 정규 분포의 CDF이고,

은 지시 함수,

는 벡터

의

번째 좌표이다.

는 일관되게

를 추정한다는 것을 알 수 있다.

Is the CDF of the standard normal distribution,

Is an indication function,

The vector

of

Is the second coordinate.

Is consistently

It can be seen that it estimates

-Estimation of correlation and latent graph structure

의 추정을 위한 두 단계 접근법을 제시한다. 첫 번째 단계에서, 이변량 주변 우도로부터 원시 추정치

를 계산한다. 두 번째 단계에서, sparse 잠재 그래프와 평활화된 추정치

를 추정하기 위하여 추정된 공분산 행렬

을 그래픽 lasso 추정기로 플러그인 한다.

We present a two-step approach for the estimation of In the first step, the raw estimate from the likelihood around the bivariate

Calculate In the second step, the sparse latent graph and the smoothed estimate

The estimated covariance matrix to estimate

Plug in the graphic lasso estimator.

의 각 항목을 추정하기 위하여 독립적인 최적화 문제를 해결한다.

에 대해

를 추정한다고 가정하자.

의 결합 분포는 확률

를 갖는 다항식이다. 여기서, 확률 변수

,

의 확률 분포는 평균 [0, 0]과 공분산

를 갖는 이변량 정규 분포이다. Step 1:

Independent optimization problems are solved to estimate each item of.

About

Suppose you estimate

The combined distribution of the probability

Is a polynomial with Where, random variable

,

The probability distribution of is covariance with the mean [0, 0]

Is a bivariate normal distribution with

이 알려져 있고, 이변량 주변 로그 우도 함수를 최대화함으로써 미지의 파라미터

를 추정할 수 있고, 다음과 같이 나타낼 수 있다.if,

Is known, and the unknown parameter by maximizing the log likelihood function around the bivariate

Can be estimated, and can be expressed as

Equation 10:

이고,

이다. 그러나, 임계값

이 알려져 있지 않다.

를 추정하기 위하여

를 추정기

로 대체하고, 다음의 로그 우도를 최대화한다.

ego,

to be. However, the threshold

This is not known.

To estimate

Estimator

And maximize the following log likelihood.

은

의 도메인이며, 공분산에 대한 추가적인 제한이 설정되지 않는다면 (-1, 1)이다. 일차원 최적화 문제로, 목표의 매끄러움과 같이 특정 규칙 하에서는

에서 미세한 그리드를 통해 목표를 단순히 평가하고 최적의 그리드 포인트를 선택함으로써 시간

에서 오류

내에서 해결할 수 있다.

silver

Is the domain of, and is (-1, 1) if no additional restrictions on covariance are set. It is a one-dimensional optimization problem, under certain rules, such as smoothness of the target

Time by simply evaluating the goal through a fine grid and selecting the optimal grid point

Error in

Can be solved within

를 플러그인한다. 일관된 파라 메트릭 가우시안 추정기 (예컨대, graphical lasso estimator, CLIME, graphical Dantzig selector 등)을 사용하여 잠재 그래프 구조를 추정하는데 사용될 수 있지만, 본 발명에서는 graphical lasso estimator에 기반하여 설명하기로 한다. 다음은 최적화 문제를 해결할 수 있다. Step 2: Use the parametric Gaussian graph model estimator to obtain the graph structure and final covariance.

Plug in. Although it can be used to estimate the latent graph structure using a consistent parametric Gaussian estimator (eg, graphical lasso estimator, CLIME, graphical Dantzig selector, etc.), in the present invention, it will be described based on the graphical lasso estimator. The following can solve the optimization problem.

Equation 11:

Here, <<A, B>> represents the trace inner product of A and P.

-Theoretical Properties Theoretical Properties

에 대한

로 향하는 것을 제공하고, 그래픽 모델 구조 복구와 관련하여 희소성을 보여준다. 단순화하기 위하여,

가 주어진 것으로 가정하자. 그러나,

가 알려지지 않은 경우의 확장은 매우 간단해야 한다. The direct estimation method described above is not only simple, but also provides strong statistical guarantees. Specifically, inverse covariance

for

It provides a heading to and shows scarcity in relation to the recovery of the graphic model structure. To simplify,

Suppose is given. But,

If is unknown, the extension should be very simple.

라고 하자. 이때,

는 크로네커 매트릭스 곱을 나타내고,

에서 평가된 –log det(A)의 헤시안(Hessian )을 나타낸다. S를

에 모든 0이 아닌 항목에 해당하는 인덱스 집합이라고 하고, S^c를 S의 보수이다. 또한,

는 최대 절대 행 합계를 나타내는 표기 단순성을 위해

를 정의한다. d를 잠재 그래프에서 최대 노드 차수라고 하자.

는 수학식 10에서 정의된 샘플 손실의 모집단 버전이다. 아래에서는 가정을 밝힌다. First, let's introduce the notation.

Let's say. At this time,

Denotes the Kronecker matrix product,

It represents the Hessian of -log det(A) evaluated at. S

Is called the set of indices corresponding to all nonzero items, and S ^c is the complement of S. Also,

Represents the maximum absolute row sum, for notation simplicity

Defines Let d be the maximum node order in the latent graph.

Is the population version of the sample loss defined in Equation 10. The assumptions are shown below.

인

이 존재한다.(c-1)

sign

Exists.

인

인 상수가 있다. 더욱이, 우도 함수

는

와 같이 양의

를 갖는다. (c-2)

sign

There is a constant phosphorus. Moreover, likelihood function

Is

As positive

Has.

는 L1, L2, L3,

에 의하여 각각 상한값을 갖는다. 더욱이

가

에서 퇴행성 임계점을 갖지 않는 온화한 규칙 성질이 성립한다. (c-3) Absolute values of the first, second and third derivatives

L1, L2, L3,

Each has an upper limit. Furthermore

end

A mild regular property that does not have a degenerative threshold is established.

(c-1) is the standard inconsistency assumption made for the guarantee of the glasso estimator.

(c-2) It is a mild condition that ensures that the two potential variables are not collinear and that all categories of sequence variables have a non-zero probability.

를 갖는 잠재 가우시안 모델을 해결하기 위하여 수학식 11을 추정치를 고려한다. c-1, c-3 조건이 만족된다고 가정하자. 그러면, L1, L2, L3, M,

,

,

,

,

에 따라 알려진 C1, C2, C3이 존재하므로,

과 n이

만큼 경계가 낮으면,

, 역추적

는 다음의 경계를 만족시킨다.Theory 4: parameters

Equation 11 is considered an estimate to solve the latent Gaussian model with. Assume that the conditions c-1 and c-3 are satisfied. Then, L1, L2, L3, M,

,

,

,

,

Since there are known C1, C2, C3 according to,

And n

If the boundary is as low as,

, Backtracking

Satisfies the following boundary.

Equation 12:

확률

만큼

로 인코딩된 잠재적인 가우시안 그래프 구조는 지속적으로

에 의해 복원될 수 있다.At least

percentage

as much as

The potential Gaussian graph structure encoded with

Can be restored by

가 1단계를 만족한다. 높은 확률과 함께,

Estimate

Satisfies step 1. With high probability,

가 높은 확률로 수학식 12를 만족한다는 것을 보여준다.Estimates from step 2 using glasso's consistency property

It shows that Equation 12 is satisfied with a high probability.

하기 위하여, 비볼록 경험적 위험 최소화 문제의 정점의 속성을 연구한다.

To do this, we study the attributes of the peaks of the non-convex empirical risk minimization problem.

= -0.3,

= -0.9에 해당된다. 왼쪽의 두개의 열은 n=50, 100에 대한 ROC 곡선을 나타낸다. 오른쪽 세개의 열은 log likelihood, 프로베니우스, 엔트로피 손실에 대한 성능을 나타낸다. 1 shows a comparison of various estimates when data is generated from a probit model with a chain graph structure. The top row and bottom row respectively

= -0.3,

= -0.9. The two columns on the left show the ROC curve for n=50, 100. The three columns on the right show the performance for log likelihood, probenius, and entropy loss.

Rating Scale: The ROC curve calculated by changing the normalization parameters can be used to compare the performance of the estimating tool with the baseline for the recovery of the graph structure. When generating data from the Probit model, you can use Frobenius Loss and Entropy Loss to compare the parameter prediction performance of Oracle, ProbitEM, ProbitEMApprox and ProbitDirect.

Provenius loss:

Entropy loss:

공분산 행렬이고,

는 추정된 공분산 행렬이다.here,

Is the covariance matrix,

Is the estimated covariance matrix.

Finally, you can compare ProbitEM, ProbitEMApprox, and ProbitDirect for log probability calculated from 500 test samples. To compare these three metrics, you can use cross-validation to select the optimal tuning parameters for each method. For example, the number of nodes in the graph is fixed at 50, and the number of categories for each ordinal variable is set to 5. In order to reduce the variance, more than 10 results can be obtained on average.

First, sequence data can be generated from a probit model, and data can be simulated from a chain graph. The inverse covariance matrix of the latent variable may be selected as shown in Equation 13 below.

Equation 13:

를 선택하고, 노드 j에서 임계값

을 다음과 같이 설정할 수 있다.At this time,

And the threshold at node j

Can be set as follows:

= -0.3,

= -0.9를 사용하여 획득된 결과를 나타낸다. ProbitDirect와 ProbitEM은 비슷한 성능을 보이나, ProbitDirect는 ProbitEM보다 1-2 배 더 빠르고, ProbitEMApprox는 특히 낮은 샘플 복잡성 설정에서 성능이 매우 낮음을 알 수 있다. At this time, the covariance matrix can be scaled so that all variables are 1. 1 is

= -0.3,

= -0.9 is used to represent the results obtained. ProbitDirect and ProbitEM perform similarly, but ProbitDirect is 1-2 times faster than ProbitEM, and ProbitEMApprox has very low performance, especially at low sample complexity settings.

)는 [-1, 1]로부터 균등하게 샘플링될 수 있다. 쌍방향 상호작용 항(

)은 모든 수평 모서리에 대하여 0.1로 설정되고, 모든 수직 모서리에 대하여 -0.1로 설정될 수 있다. Figure 2 is data sampled from a Consec model with a 2D grid structure (10 x 5 grid). Node specific parameters (

) Can be evenly sampled from [-1, 1]. Interactive interaction term (

) Is set to 0.1 for all horizontal edges, and can be set to -0.1 for all vertical edges.

)는 [-1, 1]로부터 균등하게 샘플링될 수 있다. 쌍방향 상호 작용 항 (

)은 모든 수평 모서리에 대하여 0.3으로 설정되고 모든 수직 모서리에 대하여 -0.3으로 설정될 수 있다.Figure 3 is data sampled from a Consec model with a 2D grid structure (10 x 5 grid). Node specific parameters (

) Can be evenly sampled from [-1, 1]. Two-way interaction term (

) Can be set to 0.3 for all horizontal edges and -0.3 for all vertical edges.

In an embodiment, data of the Consec model may be sampled. Figures 2 and 3 present the results in a grid graph with details of the exact parameters used. Referring to FIG. 2, since the interaction between variables is low, it can be seen that the Consec model exhibits similar performance to other estimates, and referring to FIG. 3, it can be determined that the performance decreases when the interaction is high. It can be suggested that the node conditional likelihood-based estimator for the continuous ratio model is not efficient or that a tentative graphic model such as the probit model is a better model than the continuous model.

Figure 4 shows a potential tentative graph structure corresponding to SmokeNow and socio-demographic indicators. The graph can be generated from the marginal distribution of the corresponding variable. The green and red edges represent positive and negative partial correlations, respectively, and the edge thickness is proportional to the magnitude of the partial correlation.

For example, in a nationally conducted survey conducted by the National Cancer Institute (NCI), the Health Information National Trends Survey (HINTS) treats each question in the survey as a node of a graph and an individual's response to the question as a sample extracted from the graph. I can. We can select some questions from the data set relevant to the analysis, apply the probit model using ProbitDirect to the selected questions, and use 10-fold cross-validation to select the optimal tuning parameters. Thereafter, a 95% confidence interval may be obtained for the edge strength of the latent graph through the jack knife resampling technique. At this time, corners can be placed on the graph only when the confidence interval does not intersect with [-0.1, 0.1].

Figure 4 shows how various variables related to socio-demographic indicators relate to smoking behavior in humans. In particular, SmokeNow indicates that there is a very important connection with education, which indicates that if a person is well educated and conditional on all other variables, that person is less likely to smoke. SmokeNow and FewCigarettesHarmHealth have a positive partial correlation, indicating that smokers perceive that they are less harmful than the rest of the variables, those who smoke, and those who do not smoke. According to one embodiment, these insights can help to design efficient strategies to inform the public about smoking related health information.

The apparatus described above may be implemented as a hardware component, a software component, and/or a combination of a hardware component and a software component. For example, the devices and components described in the embodiments are, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA). , A programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions, such as one or more general purpose computers or special purpose computers. The processing device may execute an operating system (OS) and one or more software applications executed on the operating system. In addition, the processing device may access, store, manipulate, process, and generate data in response to the execution of software. For the convenience of understanding, although it is sometimes described that one processing device is used, one of ordinary skill in the art, the processing device is a plurality of processing elements and/or a plurality of types of processing elements. It can be seen that it may include. For example, the processing device may include a plurality of processors or one processor and one controller. In addition, other processing configurations are possible, such as a parallel processor.

The software may include a computer program, code, instructions, or a combination of one or more of these, configuring the processing unit to behave as desired or processed independently or collectively. You can command the device. Software and/or data may be interpreted by a processing device or to provide instructions or data to a processing device, of any type of machine, component, physical device, virtual equipment, computer storage medium or device. Can be embodyed in The software may be distributed over networked computer systems and stored or executed in a distributed manner. Software and data may be stored on one or more computer-readable recording media.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, and the like alone or in combination. The program instructions recorded on the medium may be specially designed and configured for the embodiment, or may be known and usable to those skilled in computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. -A hardware device specially configured to store and execute program instructions such as magneto-optical media, and ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine language codes such as those produced by a compiler but also high-level language codes that can be executed by a computer using an interpreter or the like.

Although the embodiments have been described by the limited embodiments and drawings as described above, various modifications and variations are possible from the above description to those of ordinary skill in the art. For example, the described techniques are performed in an order different from the described method, and/or components such as a system, structure, device, circuit, etc. described are combined or combined in a form different from the described method, or other components Alternatively, even if substituted or substituted by an equivalent, an appropriate result can be achieved.

Therefore, other implementations, other embodiments, and those equivalents to the claims also fall within the scope of the claims to be described later.

Claims (15)

In the association learning method of sequence data performed by a computer-implemented association learning system,
The association learning system implemented by the computer,
A memory for storing instructions in a computer-readable recording medium; And
A plurality of processors or one processor configured to store and execute instructions stored in the memory
Including,
The method of learning the association of sequence data,
In the association learning system, predicting a correlation between each variable based on a probability graph model from sequence data; And
In the association learning system, providing a predicted correlation between the respective variables as a graph
Including,
Predicting a correlation between each variable based on a probability graph model from the sequence data,
domain

P-dimensional probability vector containing

And each random variable

A graph with nodes corresponding to

If you say,
If all node conditional distributions in the probability vector follow the univariate cumulative proportion model, then the specific node conditional distribution does not match the Markov joint distribution.
How to learn associations. The method of claim 1,
Predicting a correlation between each variable based on a probability graph model from the sequence data,
Designating a node conditional distribution through a univariate sequence distribution for the variable, and searching for a binding distribution by performing analysis on the designated node conditional distribution
Association learning method comprising a. delete In the association learning method of sequence data performed by a computer-implemented association learning system,
The association learning system implemented by the computer,
A memory for storing instructions in a computer-readable recording medium; And
A plurality of processors or one processor configured to store and execute instructions stored in the memory
Including,
A method of learning the association of sequence data performed by a computer-implemented association learning system,
In the association learning system, predicting a correlation between each variable based on a probability graph model from sequence data; And
In the association learning system, providing a predicted correlation between the respective variables as a graph
Including,
Predicting a correlation between each variable based on a probability graph model from the sequence data,
domain

P-dimensional probability vector containing

In, each random variable

A graph with nodes corresponding to

If you say,
If all node conditional distributions in the probability vector follow the univariate continuous ratio model, then the specific node conditional distribution does not match the Markov joint distribution.
Association learning method, characterized in that. In the association learning method of sequence data performed by a computer-implemented association learning system,
The association learning system implemented by the computer,
A memory for storing instructions in a computer-readable recording medium; And
A plurality of processors or one processor configured to store and execute instructions stored in the memory
Including,
A method of learning the association of sequence data performed by a computer-implemented association learning system,
In the association learning system, predicting a correlation between each variable based on a probability graph model from sequence data; And
In the association learning system, providing a predicted correlation between the respective variables as a graph
Including,
Predicting a correlation between each variable based on a probability graph model in the sequence data,
Sequence random variable

The continuous ratio model for is included in the exponential family, and the sequence random vector

When a univariate sequence distribution is used to specify a node conditional distribution for, since the node conditional distribution belongs to the univariate exponential family,
Association learning method, characterized in that. delete In the association learning method of sequence data performed by a computer-implemented association learning system,
The association learning system implemented by the computer,
A memory for storing instructions in a computer-readable recording medium; And
A plurality of processors or one processor configured to store and execute instructions stored in the memory
Including,
A method of learning the association of sequence data performed by a computer-implemented association learning system,
In the association learning system, predicting a correlation between each variable based on a probability graph model from sequence data; And
In the association learning system, providing a predicted correlation between the respective variables as a graph
Including,
Predicting a correlation between each variable based on a probability graph model from the sequence data,
In a multivariate quantization sequence distribution, when the multivariate latent probability vector is a multivariate Gaussian, it is called a multivariate probit model, and the dependency is expressed by the potential probability vector through a Gaussian distribution.
Association learning method, characterized in that. delete delete The method of claim 7,
Predicting a correlation between each variable based on a probability graph model from the sequence data,
In order to estimate an unknown parameter in the multivariate probit model, a critical value is estimated around a univariate and a polychoric correlation is estimated from a distribution around a bivariate.
Association learning method, characterized in that. The method of claim 10,
Predicting a correlation between each variable based on a probability graph model from the sequence data,
In order to estimate the polychoric correlation from the bivariate marginal distribution, a raw estimate is calculated from the likelihood around the bivariate, and the predicted covariance matrix is plugged into a graphic lasso estimator to estimate a sparse latent graph and a smoothed estimate.
Association learning method, characterized in that. delete delete The method of claim 11,
Predicting a correlation between each variable based on a probability graph model from the sequence data,
Plugging the predicted covariance matrix into a graphical lasso estimator among parametric Gaussian graph model estimators to obtain the graph structure and final covariance
Association learning method, characterized in that. In a computer-implemented association learning system,
A memory for storing instructions in a computer-readable recording medium; And
A plurality of processors or one processor configured to store and execute instructions stored in the memory
Including,
The plurality of processors or one processor,
A prediction unit predicting a correlation between each variable based on a probability graph model in sequence data; And
Providing unit that provides the predicted correlation between the respective variables in a graph
Including,
The prediction unit,
domain

P-dimensional probability vector containing

And each random variable

A graph with nodes corresponding to

If you say,
If all node conditional distributions in the probability vector follow the univariate cumulative proportion model, then the specific node conditional distribution does not match the Markov joint distribution.
Association learning system.

Images