KR20200026351A - Device and method for topic analysis using an enhanced latent dirichlet allocation model - Google Patents

Abstract

The present invention relates to a topic analysis method using an enhanced latent Dirichlet allocation (LDA) model. A topic analysis method comprises: a pre-processing step of pre-processing text data included in a document; and a step of analyzing the topic from the pre-processed text data using an enhanced LDA model which adds a hierarchical Dirichlet Process (DP) structure to an LDA model structure. Sampling is executed using partially collapsed Gibbs samplers (PCG) for the enhanced LDA model.

Description

Device and method for topic analysis using an enhanced latent dirichlet allocation model}

The present invention relates to a topical analysis apparatus and method using an improved latent Dirichlet allocation model.

As the amount of web documents is rapidly increased due to the development of the Internet, a topic analysis technology for classifying a large amount of large documents generated by the Internet by topic has recently been actively researched.

In particular, the latent Dirichlet Allocation (LDA) model is the most widely used topic analysis technique.

1 is a structure of a conventional latent Dirichlet allocation model.

Referring to FIG. 1, the conventional latent Dirichlet allocation model should specify the α and β values called hyperparameters in advance. Since the α and β values may vary from document to document, the incorrect assignment of α and β values There is a problem that causes false analysis.

In addition, the conventional latent Dirichlet allocation model has a problem in that it treats text data of heterogeneous documents as homogeneous text data and shows limitations in analyzing topics of heterogeneous documents.

Korea Patent Registration No. 10-1616544 (Announcement on 2016.04.28)

An object of the present invention is to provide an apparatus and method for topic analysis using an improved latent Dirichlet allocation model that can automatically find hyperparameter values.

Another technical problem to be solved by the present invention is to provide an apparatus and method for topic analysis using an improved latent Dirichlet allocation model that can analyze topics of heterogeneous documents.

In order to solve the above technical problem, according to a preferred aspect of the present invention, a topic analysis method comprising: a preprocessing step of preprocessing text data included in a document; And analyzing the topic in the preprocessed text data using an enhanced latent Dirichlet allocation model that adds a hierarchical Dirichlet Process (DP) structure to a latent Dirichlet allocation (LDA) model structure. Include, but provide a topic analysis method using the enhanced latent Dirichlet allocation model to perform sampling using the Partially Collapsed Gibbs Samplers (PCG) for the enhanced latent Dirichlet allocation model.

Here, α and β, which are hyperparameters of the enhanced latent Dirichlet allocation model, may not be a predetermined constant.

Here, the hyperparameters α and β of the improved latent Dirichlet allocation model can be estimated automatically.

Here, α and β, which are hyperparameters of the improved latent Dirichlet allocation model, may be different dimensional parameters of the Dirichlet distribution.

Here, the improved latent Dirichlet allocation model clusters text data of the heterogeneous document and allocates hyperparameters to each cluster for analysis of the heterogeneous document.

According to another preferred aspect of the invention, the pre-processing unit for pre-processing the text data contained in the document; A topic analyzer configured to analyze the topic using the enhanced latent Dirichlet allocation model in which the preprocessed text data is added to a hierarchical Dirichlet allocation model structure and a hierarchical Dirichlet process structure; And a sampling unit configured to perform sampling using the partially collapsed Gibbs Samplers (PCG) with respect to the enhanced latent Dirichlet allocation model.

According to another preferred aspect of the present invention, a computer-readable recording medium having a computer program recorded thereon for performing the topic analysis method using the improved latent Dirichlet allocation model can be provided.

The present invention has the effect of automatically finding the optimal hyperparameter value through the topic analysis apparatus and method using the improved latent Dirichlet allocation model.

In addition, the present invention has an effect that can accurately analyze the topics of heterogeneous documents through the topic analysis apparatus and method using the improved latent Dirichlet allocation model.

에 대한 평균 제곱 오차(MSE)의 성능을 비교한 그래프이다.
도 6은 본 발명의 일 실시예에 따른 향상된 잠재 디리클레 할당 모델을 이용한 토픽 분석 방법과 종래 다양한 방법의 로그-가능도(log-likelihood) 및 퍼플렉서티(perplexity)와 관련한 시뮬레이션 성능을 비교한 그래프이다.
도 7은 본 발명의 일 실시예에 따른 향상된 잠재 디리클레 할당 모델을 이용한 토픽 분석 방법과 종래 다양한 방법의 실제 기사에 대한 로그-가능도 및 퍼플렉서티와 관련한 시뮬레이션 성능을 비교한 그래프이다.
도 8은 본 발명의 일 실시예에 따른 향상된 잠재 디리클레 할당 모델을 이용한 토픽 분석 방법과 종래 다양한 방법의 실제 기사에 대한 하이퍼파라미터 α_d의 클러스터링과 관련한 시뮬레이션 성능을 비교한 그래프이다.
도 9는 본 발명의 다른 실시예에 따른 향상된 잠재 디리클레 할당 모델을 이용한 토픽 분석 장치의 구성도이다.1 is a structure of a conventional latent Dirichlet allocation model.
2 is a structure of an improved latent Dirichlet allocation model in a topic analysis method using an improved latent Dirichlet allocation model according to an embodiment of the present invention.
3 is a flowchart of a topic analysis method using an improved latent Dirichlet allocation model according to an embodiment of the present invention.
4 is a diagram comparing the post-distribution of the topic contribution to the document of the topic analysis method and the conventional method using the improved latent Dirichlet allocation model according to an embodiment of the present invention.
5 is a θ ^(d) and the topic analysis method using the improved latent Dirichlet allocation model according to an embodiment of the present invention and various conventional methods

This is a graph comparing the performance of the mean square error (MSE) for.
FIG. 6 is a graph comparing simulation performance with respect to log-likelihood and complexity of a topic analysis method using an improved latent Dirichlet allocation model according to an embodiment of the present invention and various conventional methods. to be.
FIG. 7 is a graph comparing topic performance using an improved latent Dirichlet allocation model according to an embodiment of the present invention and simulation performance with respect to log-likelihood and transparency of actual articles of various conventional methods.
FIG. 8 is a graph comparing simulation performances related to clustering of hyperparameters α _d for actual articles of various conventional methods and a topic analysis method using an improved latent Dirichlet allocation model according to an embodiment of the present invention.
9 is a block diagram of a topic analysis apparatus using an improved latent Dirichlet allocation model according to another embodiment of the present invention.

As the inventive concept allows for various changes and numerous embodiments, particular embodiments will be illustrated in the drawings and described in detail in the written description. However, this is not intended to limit the present invention to specific embodiments, it should be understood to include all changes, equivalents, and substitutes included in the spirit and scope of the present invention.

Terms including ordinal numbers such as first and second may be used to describe various components, but the components are not limited by the terms. These terms are only used to distinguish one component from another.

When a component is said to be 'connected' or 'connected' to another component, it may be directly connected to or connected to that other component, but other components may be present in between. It should be understood that. On the other hand, when a component is said to be 'directly connected' or 'directly connected' to another component, it should be understood that there is no other component in between.

The terminology used herein is for the purpose of describing particular example embodiments only and is not intended to be limiting of the present invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In the present application, the term 'comprises' or 'having' is intended to indicate that there is a feature, number, step, operation, component, part, or combination thereof described in the specification, and one or more other features. It is to be understood that the present invention does not exclude the possibility of the presence or the addition of numbers, steps, operations, components, components, or a combination thereof.

2 is a structure of an improved latent Dirichlet allocation model in a topic analysis method using an improved latent Dirichlet allocation model according to an embodiment of the present invention.

Referring to FIG. 2, the structure of the enhanced latent Dirichlet model 200 is a hierarchical Dirichlet process (DP) structure 220 added to the latent Dirichlet allocation model structure 210.

는 n번째 d문서의 단어,

는 문서 d의 n 번째 단어에 대한 토픽 색인, θ^(d)는 매개변수 α_d를 가진 디리클레 분포에 뒤따르는 T차원 무작위 변수로 d문서에서 토픽의 확률,

는 T X M 차원 행렬로 토픽 t에서 단어의 확률, α_d에서 α는 하이퍼파라미터 중 하나인 디리클레 분포의 T차원 파라미터로 사전에 지정된 상수가 아닌 θ^(d)의 디리클레 분포의 스카라 파라미터, β_t의 β는 하이퍼파라미터 중 하나인 디리클레 분포의 M차원 파라미터로 사전에 지정된 상수가 아닌

의 디리클레 분포의 스카라 파라미터, G 및 P는 α 와 β 각각의 농도 파라미터(concentration parameters), G₀ 및 P₀는 α 와 β의 기본 분포(base measure),

및

는 디리클레 프로세스 이전 각각의 α 와 β 이며, a₀ = 1, b₀ = 1이다.Where N is the number of words in the nth document, D is the total number of documents in the corpus,

Is the word of the nth d document,

Is the topic index for the nth word of document d, θ ^(d) is the T-dimensional random variable following the Dirichlet distribution with parameter α _d, and the probability of the topic in document d,

Is the probability, α _d of the words in the topic t to TXM-dimensional matrix α is a T-dimensional parameter of the Dirichlet distribution, one of the hyperparameters, and is a scalar parameter of the Dirichlet distribution of θ ^(d) , and β of _t is the M-dimensional parameter of the Dirichlet distribution, one of the hyperparameters. Not a specified constant

SCARA parameters of the Dirichlet distribution, G and P of α And the concentration parameters, G ₀ and P ₀ , respectively α Base measure of and β,

And

Α of each prior to the Dirichlet process And β, a ₀ = 1, b ₀ = 1.

및 λ의 밑변 분포 G₀를 갖는 디리클레 프로세스로부터 유도되며, β는 독립적이고 미지의 사전 분포 P에 따라 동일하게 분포되며, P는 정밀도 매개 변수

및 μ의 밑변 분포 P₀를 갖는 디리클레 프로세스로부터 유도되어, α 및 β는 자동으로 추정된다.α is independent and equally distributed according to the unknown prior distribution G, where G is a precision parameter

And a Dirichlet process with a base distribution G ₀ of λ, β is independent and equally distributed according to an unknown prior distribution P, where P is a precision parameter

And a Dirichlet process with base distribution P ₀ of μ, α and β are automatically estimated.

The enhanced latent Dirichlet assignment model 200 clusters text data of heterogeneous documents with different document types, and assigns hyperparameters to each cluster to analyze heterogeneous documents, in which hyperparameters α and β are predefined. This is because it is derived from a mixture of Dirichlet processes of prior dirichlet distribution, not constant.

, α^*, β^*,

,

|W)이며, 목표 분포를 기반으로 θ,

는 Z에 대한 조건부 분포에서 부분적으로 붕괴 될 수 있기 때문에 부분 붕괴된 깁스 샘플러(Partially Collapsed Gibbs Samplers, PCG)를 실행해 θ,

를 붕괴시켜 샘플링 한다. 여기서, 부분 붕괴된 깁스 샘플러는 공지된 기술이므로 자세한 설명은 생략하도록 한다.The target distribution of the enhanced latent Dirichlet allocation model 200 is p (Z, S, U, θ,

, α ^* , β ^* ,

,

W), based on the target distribution θ,

Since Partially Collapsed in the conditional distribution for Z, we run Partially Collapsed Gibbs Samplers (PCG).

Collapse and sample. Here, the partially collapsed Gibbs sampler is a well-known technique, so a detailed description thereof will be omitted.

Sampling of a partially decayed Gibbs sampler consists of six steps for parameter extraction.

|Z_-(n,d), S, U, α^*, β^*,

,

, W)로부터

를 추출하는데, 추출식은 아래 식 1과 같다.Process 1 is a probability separated p (

Z- _{(n, d)} , S, U, α ^* , β ^* ,

,

From W)

To extract, the extraction formula is shown in Equation 1.

Equation 1

=t^*|Z_-(n,d), S, U, α^*, β^*,

,

, W) ∝

, t^* = 1, ... , Tp (

= t ^* Z- _{(n, d)} , S, U, α ^* , β ^* ,

,

, W) ∝

, t ^* = 1, ..., T

}는 문서 d의 n 번째 단어를 제외한 토픽 색인의 집합을 나타내며,

는 단어 w가 Z_-(n,d)에서 토픽 t에 할당 된 횟수를 나타내고,

는 문서 d 내의 단어가 토픽 t에 할당 된 횟수를 나타낸다. W는 말뭉치, Z는 토픽 t에서 단어 n이 나올 확률, S = {

},

=

,

=

,

는 α_d가 문서 클러스터 i에 속하는 것을,

는 β_t가 토픽 클러스터 t에 속하는 것을 나타내며, U = {

},

=

,

=

, M은 고유 단어, T는 잠재적인 토픽이다.Where Z- _{(n, d)} = Z ＼ {

} Represents a set of topic indices excluding the nth word of document d,

Denotes the number of times the word w has been assigned to the topic t in Z- _{(n, d)} ,

Denotes the number of times a word in document d has been assigned to topic t. W is the corpus, Z is the probability of the word n on topic t, S = {

},

=

,

=

,

That α _d belongs to document cluster i,

Indicates that β _t belongs to topic cluster t, U = {

},

=

,

=

, M is a unique word, and T is a potential topic.

, θ|Z, S, U, α^*, β^*,

,

, W)로부터 (θ,

)를 추출하는데, 추출식은 아래 식 2 및 식 3과 같다. 식 2는 독립 T차원 디리클레 분포, 식 3은 T 독립 M 차원 디리클레 분포에 대한 것이다.Process 2 is p (

, θ | Z, S, U, α ^* , β ^* ,

,

, W) from (θ,

), And the extraction equations are as shown in Equations 2 and 3 below. Equation 2 is for independent T-dimensional Dirichlet distribution, and Equation 3 is for T independent M-dimensional Dirichlet distribution.

Equation 2

,

, W)

Dirichlet

, d = 1, ... , D θ ^(d) | (Z, S, U, α ^* , β ^* ,

,

, W)

Dirichlet

, d = 1, ..., D

Expression 3

|(Z, S, U, α^*, β^*,

,

, W)

Dirichlet

, t = 1, ... , T

(Z, S, U, α ^* , β ^* ,

,

, W)

Dirichlet

, t = 1, ..., T

는 문서 d 내의 단어가 토픽 t에 할당 된 횟수를 나타내며,

는 단어 집합 w가 토픽 색인 집합 Z에서 토픽 t에 할당되는 횟수를 나타낸다.here,

Denotes the number of times a word in document d has been assigned to topic t,

Denotes the number of times word set w is assigned to topic t in topic index set Z.

, θ, α^*, β^*,

,

, W)로부터 S를 추출하는데, 추출식은 아래 식 4 및 식 5와 같다. 식 4는 독립 이산 분포 D, 식 5는 독립 이산 분포 T에 관한 것이다.Process 3 is p (S | Z, U,

, θ, α ^* , β ^* ,

,

, S) is extracted, and the extraction equation is as shown in equations 4 and 5. Equation 4 relates to the independent discrete distribution D, and Equation 5 relates to the independent discrete distribution T.

Equation 4

= i|Z, U,

, θ, α^*, β^*,

,

, W) ∝

, i = 1, ... , I p (

= i | Z, U,

, θ, α ^* , β ^* ,

,

, W) ∝

, i = 1, ..., I

Equation 5

= j|Z, U,

, θ, α^*, β^*,

,

, W) ∝

, j = 1, ... ,Jp (

= j | Z, U,

, θ, α ^* , β ^* ,

,

, W) ∝

, j = 1, ..., J

, θ, α^*, β^*,

,

, W)로부터 U를 추출하는데, 추출식은 아래 식 6 및 식 7과 같다. 식 6은 독립 베타 분포 I, 식 7은 독립 베타 분포 J에 관한 것이다.Process 4 is p (U | Z, S,

, θ, α ^* , β ^* ,

,

U is extracted from W), and the extraction equations are as shown in Equations 6 and 7 below. Equation 6 relates to the independent beta distribution I and Equation 7 relates to the independent beta distribution J.

Equation 6

|(Z, S,

, θ, α^*, β^*,

,

, W)

Beta

, i = 1, ... , I-1

(Z, S,

, θ, α ^* , β ^* ,

,

, W)

Beta

, i = 1, ..., I-1

Equation 7

|(Z, S,

, θ, α^*, β^*,

,

, W)

Beta

, j = 1, ... , J-1

(Z, S,

, θ, α ^* , β ^* ,

,

, W)

Beta

, j = 1, ..., J-1

= 1,

= 1 이다.here,

= 1,

= 1

, θ,

,

, W)로부터 (α^*, β^*)를 추출하는데, 추출식은 아래 식 8과 같다.Process 5 is defined by p (α ^* , β ^* | Z, S, U,

, θ,

,

, W) is extracted from (α ^* , β ^* ).

Equation 8

, θ,

,

, W) =

이며,p (α ^* , β ^* | Z, S, U,

, θ,

,

, W) =

Is,

는 식 9로 추출된다.

Is extracted by equation 9.

Equation 9

는 식 10으로 추출된다.

Is extracted by equation (10).

Equation 10

이다.here,

to be.

,

|Z, S, U,

, θ, α^*, β^*, W)로부터 (

,

)를 추출하는데, 추출식은 아래 식 11과 식 12와 같다.Process 6 extracts the product of the independent gamma distribution p (

,

Z, S, U,

, θ, from α ^* , β ^* , W) (

,

) Is extracted, as shown in Equations 11 and 12 below.

Equation 11

|(Z, S, U,

, θ, α^*, β^*, W) ~ Gamma

(Z, S, U,

, θ, α ^* , β ^* , W) to Gamma

Equation 12

|(Z, S, U,

, θ, α^*, β^*, W) ~ Gamma

(Z, S, U,

, θ, α ^* , β ^* , W) to Gamma

3 is a flowchart of a topic analysis method using an improved latent Dirichlet allocation model according to an embodiment of the present invention.

Referring to FIG. 3, in operation S310, text data included in a document is preprocessed. Since the data contained in the document is text data, the document must be preprocessed to analyze the topic.

Specifically, the sentences may be separated from the text data included in the document and tagged for each morpheme. Morphological means the smallest word with a meaning. In addition, only nouns may be extracted from the tagged results of each morpheme, and stopwords may be removed from the extracted nouns.

In step S320, the topic is analyzed on the preprocessed text data using an improved latent Dirichlet allocation model. That is, the preprocessed text data can be represented as a set of words by analyzing the topic using an improved latent Dirichlet allocation model.

At this point, sampling is performed using a partially decayed Gibbs sampler for the enhanced latent Dirichlet allocation model.

The topic analysis method using the improved latent Dirichlet allocation model according to an embodiment of the present invention may be implemented in the form of program instructions that may be implemented as an application or executed through various computer components, and recorded in a computer-readable recording medium. .

The computer readable recording medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded on the computer-readable recording medium are those specially designed and constructed for the present invention, and may be known and available to those skilled in the computer software arts.

Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tape, optical recording media such as CD-ROMs, DVDs, and magneto-optical media such as floptical disks. media) and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like.

Examples of program instructions include not only machine code generated by a compiler, but also high-level language code that can be executed by a computer using an interpreter or the like. The hardware device may be configured to operate as one or more software modules to perform the processing according to the present invention, and vice versa.

4 is a diagram comparing the post-distribution of the topic contribution to the document of the topic analysis method and the conventional method using the improved latent Dirichlet allocation model according to an embodiment of the present invention. Figure 4 (a) is the actual distribution of topic contribution to the document, Figure 4 (b) is the post-distribution of topic contribution based on the topic analysis method using the improved latent Dirichlet allocation model, Figures 4 (c-1) to 4 ( c-9) is a posterior distribution diagram of the topic contribution of the topic analysis method using a conventional latent Dirichlet allocation model.

4 is a simulation result of a corpus composed of 100 documents. Each document includes word sequences generated from an average of 300 Poisson distributions, and has 100 unique words and 3 topics. Solid points represent topic contributions expected for all documents in the corpus.

Referring to FIG. 4, the actual distribution of FIG. 4 (a) appears to be very complex, which is well predicted by the topic analysis method using the improved latent Dirichlet allocation model of FIG. 4 (b).

On the other hand, Figures 4 (c-1) to 4 (c-9) is a topic analysis method using a conventional latent Dirichlet allocation model using the hyperparameters α and β as a predetermined constant, while the decayed Gibbs sampler ( Because of using Collapsed Gibbs Samplers (CG), only Figure 4 (c-4) with α = 0.5 and β = 0.001 yields an estimate with no apparent bias in the topic contribution post-distribution and the other cases ununiformally. You can see what appeared.

This means that in the case of a topic analysis method using a conventional latent Dirichlet allocation model, the multi-mode topic contribution distribution for topic analysis of heterogeneous documents may not be properly mapped. That is, the performance of the decayed Gibbs sampler depends on the choice of fixed hyperparameter variables, which can lead to large deflections.

In contrast, the topic analysis method using the improved latent Dirichlet allocation model of FIG. 4 (b) flexibly models the post-distribution of topic contributions, gives flexibility to the distribution of word contributions, and automatically calculates the hyperparameter values. By estimating it can produce powerful results.

에 대한 평균 제곱 오차(MSE)의 성능을 비교한 그래프이다.5 is a θ ^(d) and the topic analysis method using the improved latent Dirichlet allocation model according to an embodiment of the present invention and various conventional methods

This is a graph comparing the performance of the mean square error (MSE) for.

FIG. 6 is a graph comparing simulation performance with respect to log-likelihood and complexity of a topic analysis method using an improved latent Dirichlet allocation model according to an embodiment of the present invention and various conventional methods. to be.

5 and 6, the topic analysis method (PCG) using the enhanced latent Dirichlet allocation model is average squared error (MSE) and purple compared to the topic analysis method (CG) using VB, CVB and the conventional latent Dirichlet allocation model. It can be seen that the performance is relatively high because of the low complexity and high log-likelihood.

5 and 6 are simulation results of a corpus consisting of 390 documents, each document containing word sequences generated from an average of 1000 Poisson distributions, and having 1200 unique words and 10 topics. .

In addition, 390 documents were divided into 300 training materials and 90 test documents, and the simulation was repeated 100 times.

The VB and CVB methods were run until convergence, VB used the Newton-Raphson method to calculate hyperparameter values, and CVB used α = 0.05, β = 0.005.

The topic analysis method (CG) using the conventional latent Dirichlet allocation model and the topic analysis method (PCG) using the improved latent Dirichlet allocation model performed 2000 iterations with 500 burn-in iterations.

의 하이퍼파라미터에 대해 사전 디리클레 분포의 디리클레 프로세스의 혼합물로 유도되었고, 기본 분포 G₀ 및 P₀는 각각 1과 10의 지수 분포로 설정했다.In addition, the topic analysis method using the conventional latent Dirichlet allocation model sets α = 0.01, 0.05, 0.2 and 0.5 expressed in CG (0.01), CG (0.05), CG (0.2), CG (0.5) to β = 0.005. And the improved latent Dirichlet allocation models are θ ^(d) and

The hyperparameters of were derived into a mixture of dirichlet processes of prior dirichlet distribution, and the base distributions G ₀ and P ₀ were set to exponential distributions of 1 and 10, respectively.

FIG. 7 is a graph comparing topic performance using an improved latent Dirichlet allocation model according to an embodiment of the present invention and simulation performance with respect to log-likelihood and transparency of actual articles of various conventional methods.

FIG. 8 is a graph comparing simulation performances related to clustering of hyperparameters α _d for actual articles of various conventional methods and a topic analysis method using an improved latent Dirichlet allocation model according to an embodiment of the present invention.

Here, the actual articles are the 2000-2002 Neuronal Information Processing System (NIPS) conference data of 388 articles of different topics and sizes, and the 2004-2005 sports and BBC articles, in order to obtain various spectra in terms of topics and sizes of documents. We analyzed 912 articles in the technology section.

Referring to FIG. 7, it can be seen that the topic analysis method using the improved latent Dirichlet allocation model shows the highest log-likelihood compared to other methods.

In addition, although the topic analysis method using the improved latent Dirichlet allocation model provided the smallest transparency compared with other methods, it can be seen that as the number of topics increases, the difference with other methods decreases.

In particular, when the number of topics is large, the topic analysis method using the conventional latent Dirichlet allocation model with α = 0.05 or 0.2 shows similar performance to the topic analysis method with the improved latent Dirichlet allocation model, indicating that α = 0.05 and 0.2 are near optimal. It can be seen that the topic analysis method using the improved latent Dirichlet allocation model automatically shows that the hyperparameter α selected is close to the optimum.

Referring to FIG. 8, hyperparameter clustering shows that the α _d atoms sampled in the pre-Dillikle mixture are multiples with similar cluster sizes, ie ^, the estimated distribution of θ ^(d) tends to have several modes.

Since the weighted average of the sampled α _d is between 0.1 and 0.2, the optimal α of the topic analysis method using the conventional latent Dirichlet allocation model is close to the weighted average of the topic analysis method using the enhanced latent Dirichlet allocation model, and coincident with each other to improve the latent Dirichlet allocation. We can see that the topic analysis method using the model automatically estimates the optimal hyperparameters from the data.

9 is a block diagram of a topic analysis apparatus using an improved latent Dirichlet allocation model according to another embodiment of the present invention.

Referring to FIG. 9, the topic analysis apparatus 900 using the improved latent Dirichlet allocation model includes a preprocessor 910, a topic analyzer 920, and a sampling unit 930.

The preprocessor 910 preprocesses the text data included in the document. Since the data contained in the document is text data, the document must be preprocessed to analyze the topic. In other words, the preprocessor 910 preprocesses the text data to analyze the topic.

In detail, the preprocessor 910 may separate the sentences from the text data included in the document and tag them by morpheme. Morphological means the smallest word with a meaning. In addition, only nouns may be extracted from the tagged results of each morpheme, and stopwords may be removed from the extracted nouns.

The topic analyzer 920 receives the preprocessed text data through the preprocessor 910 and analyzes the topic using the improved latent Dirichlet allocation model. That is, the topic analyzer 920 may express the preprocessed text data as a set of words by analyzing the topic using the improved latent Dirichlet allocation model.

In this case, the sampling unit 930 performs sampling using the partially decayed Gibbs sampler for the improved latent Dirichlet allocation model.

Although embodiments according to the present invention have been described above, these are merely exemplary, and those skilled in the art will understand that various modifications and equivalent embodiments of the present invention are possible therefrom. . Therefore, the true technical protection scope of the present invention will be defined by the following claims.

910: preprocessor 920: topic analysis unit
930: sampling unit

Claims (11)

In the topic analysis method,
A preprocessing step of preprocessing the text data included in the document; And
Analyzing the topic in the preprocessed text data using an enhanced latent Dirichlet allocation model that adds a hierarchical Dirichlet Process (DP) structure to a latent Dirichlet allocation (LDA) model structure; But
Performing sampling using the Partially Collapsed Gibbs Samplers (PCG) on the enhanced latent Dirichlet allocation model.
Topic analysis method using an improved latent Dirichlet allocation model characterized in that.
The method of claim 1,
The hyperparameters α and β of the enhanced latent Dirichlet allocation model are not predetermined constants
Topic analysis method using an improved latent Dirichlet allocation model characterized in that.
The method of claim 1,
The hyperparameters α and β of the improved latent Dirichlet allocation model are automatically estimated
Topic analysis method using an improved latent Dirichlet allocation model characterized in that.
The method of claim 1,
Hyperparameters α and β of the improved latent Dirichlet allocation model are different dimensional parameters of the Dirichlet distribution
Topic analysis method using an improved latent Dirichlet allocation model, characterized in that.
The method of claim 1,
The improved latent Dirichlet allocation model clusters text data of heterogeneous documents and assigns hyperparameters to each cluster for analysis of heterogeneous documents.
Topic analysis method using an improved latent Dirichlet allocation model characterized in that.
A preprocessor for preprocessing the text data contained in the document;
A topic analyzer configured to analyze the topic using the enhanced latent Dirichlet allocation model in which the preprocessed text data is added to a hierarchical Dirichlet allocation model structure and a hierarchical Dirichlet process structure; And
And a sampling unit configured to perform sampling using the partially collapsed Gibbs Samplers (PCG) on the improved latent Dirichlet allocation model.
Topic analysis device using an improved latent Dirichlet allocation model, characterized in that.
The method of claim 6,
Hyperparameters α and β of the enhanced latent Dirichlet allocation model are not predetermined constants
Topic analysis device using an improved latent Dirichlet allocation model, characterized in that.
The method of claim 6,
The hyperparameters α and β of the improved latent Dirichlet allocation model are automatically estimated
Topic analysis device using an improved latent Dirichlet allocation model, characterized in that.
The method of claim 6,
Hyperparameters α and β of the improved latent Dirichlet allocation model are different dimensional parameters of the Dirichlet distribution
Topic analysis device using the improved latent Dirichlet allocation model, characterized in that.
The method of claim 6,
The improved latent Dirichlet allocation model clusters text data of heterogeneous documents and assigns hyperparameters to each cluster for analysis of heterogeneous documents.
Topic analysis device using an improved latent Dirichlet allocation model, characterized in that. A computer-readable recording medium having recorded thereon a computer program for performing a topical analysis method using the improved latent Dirichlet allocation model according to any one of claims 1 to 5.

Images