JP2013069141A - Document analysis device and document analysis method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To optimize a mixture ratio when generating prior distribution by mixing present topics and past topics.SOLUTION: Topic distribution generation means 301 generates word distribution which is polynomial distribution of a frequency of appearance of each word included in a second document set obtained after a first document set on the basis of first prior distribution which is polynomial distribution of a frequency of appearance of each word included in the first document set obtained in the past. Mixture ratio calculation means 302 calculates a mixture ratio of the first prior distribution and the word distribution on the basis of the first prior distribution. Prior distribution generation means 303 generates second prior distribution by mixing the first prior distribution and the word distribution at the mixture ratio. Prior distribution accumulation means 304 accumulates the second prior distribution. When a third document is obtained after the second document set, the topic distribution generation means 301 generates word distribution which is polynomial distribution of a frequency of appearance of each word included in a third document set on the basis of the second prior distribution.

Description

  The present invention relates to a technique for extracting a topic from a document.

Massive documents are generated every day on the Internet, such as text posted on microblogs such as Twitter (registered trademark), articles provided on news special sites, comments exchanged on SNS (social networking service), etc. .
Buzztter is known as one of the techniques for extracting matters that have become a hot topic on the Internet. Buzztter extracts words that appear frequently from text posted on Twitter in real time and distributes them to the user's terminal.

As one of topic models for extracting a topic, which is a potential factor that governs the appearance frequency of words on a document, from documents, there is Latent Dirichlet Allocation (hereinafter referred to as “LDA”, for example, Non-Patent Document 1). . FIG. 1 is a diagram showing a graphical model of LDA. Table 1 is a table showing the meaning of each symbol in FIG.

  In LDA, there are a plurality of topics in each document, each topic appears in each document with a certain probability, a plurality of words corresponding to each topic exist, and each word appears in each topic with a certain probability. Under the assumption, each topic is characterized by a multinomial distribution of the number of occurrences of each word (hereinafter referred to as “word distribution”), and a multinomial distribution of the number of occurrences of each topic (hereinafter referred to as “topic distribution”). This is a technique for characterizing each document. For example, if one sentence posted on Twitter (hereinafter referred to as “tweet”) is captured as a document, immediately after the earthquake occurs, many documents containing many topics related to earthquakes and tsunamis will be generated. The topic about is characterized by a multinomial distribution of the number of occurrences of each word (including word combinations) such as “earthquake”, “seismic intensity”, “earthquake breaking news”, and the topics about tsunami are “tsunami”, “height ”,“ Arrival time ”and the like, which are characterized by a multinomial distribution of the number of appearances of each word.

The conventional LDA is configured to extract topics from a specific document set given in advance. Therefore, topic changes cannot be extracted from a document set whose contents change over time. On the other hand, Dynamic Topic Model (hereinafter referred to as “DTM”, for example, Non-Patent Document 2) is a technique that extends LDA so that it can be applied to analysis of a document set whose contents change over time. is there. FIG. 2 is a diagram showing a graphical model of DTM. t is the time when the document set is acquired. A document set D ^(t) is input for each acquisition time t, and a word distribution β ^(t) and topic distribution θ ^(t) for each acquisition time t are generated. The feature of DTM is that the word distribution β ^(t−1) at the acquisition time ^t−1 is a prior distribution of the word distribution β ^{(t) at} the acquisition time t, in other words, the current topic from the past topic. It is a point to extract. Therefore, by comparing β ^(t) with β ^(t-1) , it is possible to observe changes in the topic over time. In addition, since the influence of past topics is reflected on the current topic, a result that does not overlearn the current document set can be obtained as compared with the case of extracting a topic from a document set only at a specific acquisition time. As a result, topics included in the document set at the next acquisition time t + 1 can be predicted with high accuracy.

D. M. Blei, A. N. Ng, and M. I. Jordan. Latent dirichlet allocation. In Journal of Machine Learning Research archive., 2003. D. M. Blei and J. D. Lafferty. Dynamic Topic Models. In Proc. Of the 23rd international conference on Machine learning., 2006. G. Kitagawa.Monte carlo filter and smoother for non-Gaussian nonlinear state space models.In Journal of computationaland graphical statistics., 1996.

JP 2010-244264 A

By the way, in DTM, since the prior distribution of the topic at the next acquisition time is generated from the current topic distribution, the topic distribution at the next acquisition time may overlearn the current topic distribution.
To solve this problem, a method of preventing overlearning by mixing and smoothing the current topic distribution and the past topic distribution can be considered. For example, Patent Document 1 proposes a technique for generating a prior distribution by mixing past topics. However, no proposal has been made regarding the optimal distribution of past topics and current topics.
The present invention has been made in view of the above-described background, and an object of the present invention is to optimize a mixing ratio when a prior distribution is generated by mixing a current topic and a past topic.

  The document analysis apparatus according to claim 1 is an acquisition unit that acquires a document set from an external device, and a multinomial distribution of the number of occurrences of each word included in the first document set acquired by the acquisition unit in the past. Generating a word distribution that is a multinomial distribution of the number of occurrences of each word included in the second document set acquired by the acquisition unit after the first document set based on the prior distribution of the first document set; And a topic distribution generation means for generating a topic distribution which is a multinomial distribution of the number of occurrences of each topic characterized by the following: a mixing ratio between the first prior distribution and the word distribution based on the first prior distribution; Mixing ratio calculating means for calculating, prior distribution generating means for generating a second prior distribution by mixing the first prior distribution and the word distribution at the mixing ratio, and the second prior distribution Accumulate A prior distribution storage unit, and the topic distribution generation unit includes each of the third document sets included in the third document set when the third document is acquired by the acquisition unit after the second document set. A word distribution, which is a multinomial distribution of word appearances, is generated based on the second prior distribution.

  The document analysis apparatus according to claim 2 is the document analysis apparatus according to claim 1, wherein the mixture ratio calculation unit includes a first particle group corresponding to a plurality of different mixture ratios that are candidates for the mixture ratio; Assuming a first likelihood group consisting of likelihoods associated with each particle included in the first particle group, the first particle group is dispersed as particles in a particle filter, and the first A second likelihood group is calculated by re-spreading the first particle group based on the likelihood group, and a weighted average of the first particle group using the second likelihood group as a weight The mixing ratio is calculated by the following.

  A document analysis method according to a third aspect of the present invention is based on a first prior distribution that is a multinomial distribution of the number of appearances of each word included in a first document set acquired in the past, after the first document set. A topic that generates a word distribution that is a multinomial distribution of the number of appearances of each word included in the acquired second document set, and generates a topic distribution that is a multinomial distribution of the number of appearances of each topic characterized by the word distribution A distribution generation step; a mixing ratio calculating step for calculating a mixing ratio between the first prior distribution and the word distribution based on the first prior distribution; and the first prior distribution and the word distribution. In the topic distribution generation step, a prior distribution generation step of generating a second prior distribution by mixing at the mixing ratio, and a prior distribution storage step of storing the second prior distribution, When a third document is acquired after the second document set, a word distribution that is a multinomial distribution of the number of appearances of each word included in the third document set is based on the second prior distribution. It is characterized by generating.

  The program according to claim 4 is a multinomial distribution of the number of appearances of each word included in the first document set acquired by the acquisition means for acquiring a document set from an external device and the acquisition means in the past. Based on the first prior distribution, a word distribution which is a multinomial distribution of the number of appearances of each word included in the second document set acquired by the acquisition unit after the first document set is generated, and the word A topic distribution generating means for generating a topic distribution which is a multinomial distribution of the number of appearances of each topic characterized by the distribution, and a mixing ratio between the first prior distribution and the word distribution based on the first prior distribution. A mixture ratio calculating means for calculating the second prior distribution by mixing the first prior distribution and the word distribution at the mixture ratio; and the second distribution A program for functioning as a prior distribution accumulating unit for accumulating a pre-distribution, wherein the topic distribution generating unit is configured such that when a third document is acquired by the acquiring unit after the second document set, A word distribution which is a multinomial distribution of the number of appearances of each word included in the third document set is generated based on the second prior distribution.

  According to the present invention, it is possible to optimize the mixing ratio when the prior distribution is generated by mixing the current topic and the past topic.

It is a figure which shows the graphical model of LDA. It is a figure which shows the graphical model of DTM. 1 is a diagram illustrating a configuration of a communication system 1. FIG. 2 is a block diagram showing a hardware configuration of a document analysis device 30. FIG. 3 is a block diagram showing a functional configuration of a document analysis device 30. FIG. It is a graphical model of this embodiment. It is a figure which shows the estimation method of the mixture ratio using a particle filter.

An embodiment of the present invention will be described.
(1) Configuration of Embodiment FIG. 3 is a diagram showing a configuration of the communication system 1 according to the embodiment of the present invention. The communication system 1 includes a mobile communication network 10, a mobile communication device 20, an Internet 50 connected to the mobile communication network 10 via a gateway device 60, a document analysis device 30 connected to the gateway device 60, and an Internet 50. And a plurality of web server devices 40 connected to each other.

  The mobile communication device 20 is a communicable computer such as a cellular phone, for example, and includes a control unit including an arithmetic device such as a CPU (Central Processing Unit) and a storage device such as a ROM (Read Only Memory) and a RAM (Random Access Memory). And a storage unit such as an EEPROM (Electronically Erasable and Programmable ROM) or SRAM (Static Random Access Memory) with a backup power source, a wireless communication unit including an antenna and a wireless communication circuit, a speaker, a microphone, and a sound processing circuit A voice input / output unit, an operation unit having a plurality of operators such as keys and a touch screen, and a display unit including a liquid crystal panel and a liquid crystal driving circuit are provided. In this mobile communication device 20, the control unit accesses the web server device 40 via the mobile communication network 10 and the Internet 50 by controlling the communication unit in accordance with a user operation received by the operation unit, and the web server. Information stored in the device 40 is acquired and displayed on the display unit. Thereby, the user can access and browse various information on the Internet 50.

  The mobile communication network 10 is a network that provides a communication service to the mobile communication device 20. The mobile communication network 10 includes a base station that performs wireless communication with a mobile communication device 20 that exists in a wireless cell of the mobile station, an exchange that performs routing of data transmitted in the network, and the mobile communication device 20. Are provided with various nodes such as a control station for performing location registration, and a communication line for interconnecting these nodes.

  The web server device 40 is a computer, and includes a control unit including a calculation device such as a CPU and a storage device such as a ROM and a RAM, a storage unit such as a hard disk device, and a communication unit connected to the Internet 50. . The web server device 40 has a function of performing data communication with the mobile communication device 20 via the Internet 50 and the mobile communication network 10. The web server device 40 is a server that provides a microblog service. When a user registers in the web server device 40 as a user of this service, the user is allowed to post to the microblog. When a user posts a text to the microblog using the mobile communication device 20, the web server device 40 sends the text back to the mobile communication device 20, whereby the text is displayed on the display unit of the mobile communication device 20. Is done. In addition, when the user registers another user, the text posted by the other user is also displayed on the display unit. In this way, it is possible to communicate among a plurality of users.

(2) Configuration of Document Analysis Device FIG. 4 is a block diagram showing a hardware configuration of the document analysis device 30. The document analysis apparatus 30 is a computer and includes a control unit 31, a communication unit 32, and a storage unit 33. The control unit 31 includes an arithmetic device such as a CPU and a storage device such as a ROM and a RAM. The CPU controls the operation of each unit of the document analysis apparatus 30 by executing a program group stored in the ROM or the storage unit 33 using the RAM as a work area.

The communication unit 32 includes a communication interface and is connected to the gateway device 60. The communication unit 32 acquires a document set distributed from the web server device 40 to the mobile communication device 20 relayed by the gateway device 60 from the gateway device 60 and inputs the document set to the document analysis device 30. That is, the communication unit 32 is an example of an acquisition unit that acquires a document set from an external device.
The storage unit 33 is a writable nonvolatile storage unit, for example, a hard disk device. The storage unit 33 stores a program group in which a procedure of processing executed by the control unit 31 is described. In addition, the storage unit 33 has a storage area for storing information acquired from the gateway device 60 and distributed to the mobile communication device 20 by the web server device 40.

FIG. 5 is a block diagram showing a functional configuration of the document analysis apparatus 30 according to the present embodiment. Table 2 shows symbols used in the description of the embodiments and their meanings.

The document analysis apparatus 30 includes a topic distribution generation unit 301, a mixture ratio calculation unit 302, a prior distribution generation unit 303, and a prior distribution storage unit 304. There are mainly four processes executed by the document analysis device 30: topic distribution generation, mixture ratio calculation, prior distribution generation, and prior distribution accumulation.
Here, terms used in the following description will be described.
In this embodiment, a sentence (text data) posted on a microblog is an object of analysis, and a sentence transmitted from the mobile communication device 20 to the web server device 40 in one posting is called a document. A document set is a set of one or more documents.
A word is a word constituting a document and corresponds to a morpheme in Japanese. The word can be any part of speech.

The acquisition time t is a time when the document analysis device 30 acquires the document set distributed from the web server device 40 to the mobile communication device 20 via the gateway device 60. The document analysis apparatus 30 executes a series of processes described below, triggered by acquisition of a document set. For example, the acquisition time t may be set at regular intervals such as every 24 hours or every 6 hours, or the administrator of the document analysis device 30 or the user of the mobile communication device 20 may set the intervals at any time without setting the intervals. The document analysis apparatus 30 may be instructed to acquire a document set.
The acquisition time t is represented by an integer. That is, if the time when the document set being processed by the document analysis apparatus 30 is acquired is t, the acquisition time of the document set that was the object of the previous processing is t-1.

The documents that make up the document set may be selected in any way. For example, 100,000 postings may be acquired retroactively from the latest posting date, or 100,000 postings may be randomly extracted from postings for the past 24 hours.
A new word is a word that is not included in a vocabulary set generated in the past. A known word is a word included in a vocabulary set generated in the past.
The prior distribution is a prior probability distribution in Bayesian estimation, and a posterior distribution is generated by multiplying the prior distribution by a likelihood function.

Next, the contents of each process will be described.
(2.1) Generation of topic distribution

(2.2) Calculation of mixing ratio

粒子フィルタの利点は、経時的に変化する値を推定するにあたり、前時刻における状態と現在時刻における入力値との間に状態方程式を必要としない点である。そのため、前時刻における状態からの現在時刻における入力値の予測が困難である今回の問題には最適な方式と言える。

The advantage of the particle filter is that no state equation is required between the state at the previous time and the input value at the current time in estimating the value that changes over time. Therefore, it can be said that this method is optimal for the current problem in which it is difficult to predict the input value at the current time from the state at the previous time.

FIG. 7 is a diagram illustrating a method for estimating a mixing ratio using a particle filter. Here, it is assumed that N candidates ρ _i ^(t−1) (where i = 1, 2,..., N) corresponding to different mixing ratios as the mixing ratio at the acquisition time t−1. Each is dispersed as particles in a particle filter. Each particle is associated with the likelihood w _i ^(t−1) for the optimum estimated value ρ _i ^(t) of each particle at the acquisition time t.
At time t, the mixture ratio calculation unit 302 first respreads each particle based on the corresponding likelihood. Specifically, each particle is dispersed N times while allowing overlap with the likelihood as the probability. This means that particles with a high likelihood may be scattered several times and particles with a low likelihood may disappear without being scattered.

Next, the mixing ratio calculation unit 302 causes each particle to walk randomly according to a specific distance. The distance is generated by a normal distribution with a standard value of a variance value σ, which is a fixed value given in advance. Here, since ρ _i ^(t) is 0 or more and 1 or less, when a distance exceeding the range is generated, the distance is generated again by the same method. Each particle obtained here represents the state of the particle at the acquisition time t.
Next, a new likelihood w _i ^(t) is calculated for each particle. The likelihood calculation method will be described later. Ρ _i ^{(t) at} time t is calculated from the weighted average of each particle with the likelihood as a weight, as follows:

In short, the mixture ratio calculation unit 302 is an example of a unit that calculates a mixture ratio between the first prior distribution and the word distribution based on the first prior distribution.
In addition, the mixture ratio calculation unit 302 calculates a first particle group corresponding to a plurality of different mixture ratios that are candidates for the mixture ratio, and a likelihood associated with each particle included in the first particle group. Assuming that the first likelihood group is, the first particle group is dispersed as particles in a particle filter, and the first particle group is redispersed based on the first likelihood group. The second likelihood group may be calculated, and the mixture ratio may be specified by a weighted average of the first particle group using the second likelihood group as a weight.

(2.3) Generation of Prior Distribution Generation of the prior distribution is executed by the prior distribution generation unit 303.

また、トピック分布生成手段３０１は、前記第２の文書集合の後に前記取得手段によって第３の文書が取得された場合に、前記第３の文書集合に含まれる各単語の出現回数の多項分布である単語分布を、前記第２の事前分布に基づいて生成する。 (2.4) Accumulation of prior distribution

The topic distribution generation unit 301 is a multinomial distribution of the number of occurrences of each word included in the third document set when the third document is acquired by the acquisition unit after the second document set. A certain word distribution is generated based on the second prior distribution.

As described above, according to the present embodiment, it is possible to generate a topic distribution with higher accuracy than the conventional DTM by smoothing the prior distribution. Good accuracy here means that the document is well modeled. This can be quantitatively evaluated by perplexity (Non-Patent Document 2). The perplexity is an index indicating the accuracy of the model for the test document. The smaller the value, the better the model. Here, it is considered to evaluate how the topic distribution at the acquisition time t−1 models the document acquired at the acquisition time t. In this case, perplexity is expressed by the following equation.

  Table 3 is a table showing the results of actually implementing the system of the present embodiment and evaluating the perplexity. This evaluation result was collected from Twitter posts from February to March 2011 at a rate of 200,000 a day, and for that data, regular LDA, DTM, which does not take into account changes in topics over time, The perplexity when the three methods are applied is shown. The DTM and the present embodiment are operated from February 1, 2011 to March 31, 2011, and the perplexity is derived using the topic distribution on March 30 and the document on March 31. For ordinary LDA, the topic distribution was derived only from the March 30 document, and the perplexity was derived using the March 31 document. Further, since the performance of the present embodiment varies depending on the distance σ of the random walk of the particle filter, the perplexity is calculated with four values of σ = 0, 0.05, 0.1, and 0.15.

From this evaluation result, it can be seen that the performance of the DTM is much improved compared to the normal LDA, but the performance of the system of this embodiment is further improved than the DTM. In addition, it can be seen that the data set used for this evaluation has the best performance when σ = 0.1.

(3) Modifications The above embodiment may be modified as follows. Moreover, you may implement combining the following modifications.
(3.1) Modification 1
In the embodiment, an example of analyzing a document set posted on a microblog has been shown. However, other types of document sets may be analyzed.
For example, when the web server device 40 is a server that distributes news articles, articles distributed during a specific period may be acquired as a document set, and the same processing as in the embodiment may be performed on the document set. Good. Further, when the web server device 40 is a server that manages SNS, a comment exchanged on the SNS for a specific period is acquired as a document set, and the same processing as that of the embodiment is performed on this document set. Also good.

(3.2) Modification 2
In the embodiment, the example in which the document delivery destination is the mobile communication device has been described, but the document delivery destination may be any device. For example, a stationary computer connected to the Internet may be used.
In the embodiment, an example in which the control unit 31 of the document analysis apparatus 30 executes processing by executing a program has been described, but the same function may be implemented by hardware. The program is provided by being recorded on a computer-readable recording medium such as an optical recording medium or a semiconductor memory, and the program is read from the recording medium and stored in the storage unit 33 of the document analysis apparatus 30. Also good. Further, this program may be provided via a telecommunication line.

DESCRIPTION OF SYMBOLS 1 ... Communication system, 10 ... Mobile communication network, 20 ... Mobile communication apparatus, 30 ... Document analysis apparatus, 40 ... Web server apparatus, 50 ... Internet, 60 ... Gateway apparatus, 31 ... Control part, 32 ... Communication part, 33 ... Storage unit 301... Topic distribution generation unit 302... Mixing ratio calculation unit 303... Prior distribution generation unit 304.

Claims (4)

An acquisition means for acquiring a document set from an external device;
Based on a first prior distribution that is a multinomial distribution of the number of occurrences of each word included in the first document set acquired by the acquisition unit in the past, the acquisition unit acquires the first document set after the first document set. Generating a word distribution that is a multinomial distribution of the number of occurrences of each word included in the second document set, and generating a topic distribution that is a multinomial distribution of the number of occurrences of each topic characterized by the word distribution Means,
A mixing ratio calculating means for calculating a mixing ratio between the first prior distribution and the word distribution based on the first prior distribution;
A prior distribution generating means for generating a second prior distribution by mixing the first prior distribution and the word distribution at the mixing ratio;
A prior distribution accumulation means for accumulating the second prior distribution;
The topic distribution generation unit is a word that is a multinomial distribution of the number of occurrences of each word included in the third document set when the acquisition unit acquires a third document after the second document set. A document analysis apparatus characterized in that a distribution is generated based on the second prior distribution. The mixture ratio calculation means includes a first particle group corresponding to a plurality of different mixture ratios that are candidates for the mixture ratio, and a likelihood associated with each particle included in the first particle group. Assuming one likelihood group, the first particle group is dispersed as particles in a particle filter, and the first particle group is re-dispersed based on the first likelihood group. 2. The document analysis apparatus according to claim 1, wherein the likelihood ratio is calculated, and the mixture ratio is calculated by a weighted average of the first particle group using the second likelihood group as a weight. . Included in the second document set acquired after the first document set based on the first prior distribution which is a multinomial distribution of the number of occurrences of each word included in the first document set acquired in the past Generating a word distribution that is a multinomial distribution of the number of occurrences of each word, and generating a topic distribution that is a multinomial distribution of the number of occurrences of each topic characterized by the word distribution;
A mixing ratio calculating step for calculating a mixing ratio between the first prior distribution and the word distribution based on the first prior distribution;
A prior distribution generation step of generating a second prior distribution by mixing the first prior distribution and the word distribution at the mixing ratio;
A prior distribution accumulation step for accumulating the second prior distribution;
In the topic distribution generation step, when a third document is acquired after the second document set, a word distribution that is a multinomial distribution of the number of occurrences of each word included in the third document set is A document analysis method, wherein the document analysis method is generated based on the second prior distribution. Computer
An acquisition means for acquiring a document set from an external device;
Based on a first prior distribution that is a multinomial distribution of the number of occurrences of each word included in the first document set acquired by the acquisition unit in the past, the acquisition unit acquires the first document set after the first document set. Generating a word distribution that is a multinomial distribution of the number of occurrences of each word included in the second document set, and generating a topic distribution that is a multinomial distribution of the number of occurrences of each topic characterized by the word distribution Means,
A mixing ratio calculating means for calculating a mixing ratio between the first prior distribution and the word distribution based on the first prior distribution;
A prior distribution generating means for generating a second prior distribution by mixing the first prior distribution and the word distribution at the mixing ratio;
A program for functioning as a prior distribution accumulation means for accumulating the second prior distribution,
The topic distribution generation unit is a word that is a multinomial distribution of the number of occurrences of each word included in the third document set when the acquisition unit acquires a third document after the second document set. A program for generating a distribution based on the second prior distribution.

Images