JP6204261B2 - Topic modeling apparatus, topic modeling method, and topic modeling program - Google Patents

Description

  The present invention relates to a topic modeling technique in time-series text data.

  With the spread of microblogging and the like, it has become important to create a topic model that can extract topics from time-series text data with high real-time properties and capture the topic of the world, especially in the field of marketing. Here, the topic means information on a specific topic. The topic model represents a model (function, formula) describing the relationship between a topic and a character string such as a word included in text data.

  Prior art of topic modeling that captures topics in time-series text data includes an extension of LDA (Lentative Analysis) (Non-Patent Document 1) and an extension of NMF (Non-Negative Matrix Factorization) (Non-Patent Documents). 2) has been proposed. In NMF, a feature matrix of a document and a word is subjected to matrix decomposition under non-negative constraints and dimension compression is performed to obtain a model and estimate a topic. In Non-Patent Document 2, NMF is given a constraint that takes into account the amount of change in the topic over time, and a topic that is rising is estimated. Below, the outline | summary of the method in NMF and a nonpatent literature 2 is described.

<NMF>
NMF expresses a document by two matrices obtained by decomposing a document and word feature matrix X under non-negative constraints. One matrix is a document topic matrix W representing the degree of relationship between topics corresponding to columns in a document corresponding to a row. The other matrix is a topic word matrix H representing the degree of relationship of words corresponding to columns in the topic corresponding to the row. In NMF, the feature matrix X is decomposed into a document topic matrix W and a topic word matrix H as shown in the following equation.

  In the above equation, i and j represent matrix indices. In order to decompose the matrix X as described above, for example, the matrix W and the matrix H are calculated based on the square error as in the following equation.

In the above formula, ‖X-WH‖ _F is the Frobenius norm of "X-WH".

<Method of Non-Patent Document 2>
In the method of Non-Patent Document 2, a topic that is excited in time-series text data is obtained by extending the NMF. Specifically, W and H are calculated based on the following equation.

In the above equation, w _i is the i-th column vector of W. S is a matrix for calculating the sum of w _{i in} the same time zone for topic i. ^Wem is a submatrix corresponding to the topic from which the excitement is extracted. μ is a hyperparameter. L (SW _i ) is a function that gives a large penalty when the topic variation at each time is small. With this penalty, it is possible to extract topics that are exciting in time.

Diao, Q., Jiang, J., Zhu, F., Lim, E.-P .: Finding bursty topics from microblogs, In Proc. Of ACL'12, 2012, pp.536-544. Saha, A. and Sindhwani, V .: Learning evolving and emerging topics in social media: a dynamic nmf approach with temporal regularization, In Proc. Of WSDM'12, 2012, pp.693-702. Mei, Q., Cai, D., Zhang, D., Zhai, C .: Topic modeling with network regularization, In Proc. Of WWW’08, 2008, pp.101-110. Jun Suzuki and Hideki Amagasaki, “Learning Conditional Random Fields Based on Learning Error Minimization: Application to Language Analysis”, Proc. Of the 12th Annual Conference of the Language Processing Society, 2006, pp.548-551.

  However, the topic model described in Non-Patent Document 2 does not consider geographical information. For example, when an event such as music guerrilla live is held suddenly in Yokosuka, a message with location information acquired by many users using GPS (Global Positioning System) is posted on social media, or the place name is Post a message that includes it. Social media information includes a lot of local topics such as events. As described in Non-Patent Document 2, it is difficult to detect the swelled topic only by considering the swell of time.

  Non-Patent Document 3 discloses a technique for extracting a topic specific to a region by LDA in consideration of position information given to text. However, since this method does not take into account both position information and time information, topics with contents that are frequently posted in a certain area are extracted. That is, there is a problem that a topic that is locally rising in a certain time and area cannot be extracted, and a topic related to an event or the like that suddenly occurs in a specific area as in the above example cannot be extracted.

  The present invention has been made in view of the above-described problems of the prior art, and an object of the present invention is to provide a technique capable of generating a topic model that can obtain topic information that is exciting geographically and temporally.

Therefore, the topic modeling device of the present invention is a topic modeling device that creates a topic model that depends on position information and time information, the position information extracting means for extracting position information from time series text, and the extracted A position information dependency matrix calculating means for calculating a position information dependency matrix indicating a degree of similarity according to position information between the time series texts based on the position information, and a document feature matrix of the time series text as a temporal topic change amount. A document topic matrix and a topic indicating the degree of relationship between the text and the topic by performing matrix decomposition and dimension compression under the topic extraction constraint based on the location information and the topic extraction constraint based on the position information dependence matrix A topic word matrix indicating the degree of relationship with words is calculated over time, and this document topic matrix and topic word lines are calculated. The latest document topic matrix and topic word matrix temporal change amount is less than the specified value of, as the topic models, and a model calculation means for determining.

The topic modeling method of the present invention is a topic modeling method executed by a topic modeling device that creates a topic model depending on position information and time information, the step of extracting position information from time series text, and the extracted and calculating a position information dependent matrix indicating a degree of similarity due to positional information between the time series text based on the location information, a document feature matrix of the time series text, topic extraction based on the amount of change in temporal topic Document topic matrix indicating degree of relation between text and topic and degree of relation between topic and word by matrix decomposition and dimension compression under restriction and topic extraction restriction based on position information dependency matrix The topic word matrix indicating the time is calculated over time, and this document topic matrix and topic word matrix The latest document topic matrix and topic word matrix temporal change amount is less than the specified value, as the topic models, and a step of determining.

  Note that the present invention may be in the form of a program that causes a computer to function as each unit of the apparatus or a program that causes a computer to execute the steps of the method.

  ADVANTAGE OF THE INVENTION According to this invention, the topic model which can obtain the topic information which excites geographically and time can be provided.

The block block diagram of the topic modeling apparatus in embodiment of this invention. The flowchart of the topic modeling process in the embodiment. An example of a document feature matrix. The flowchart of the extraction process of a positional information. An example of a positional information dependence matrix. The flowchart of the calculation process of a positional information dependence matrix. The flowchart of a model calculation process.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. However, the present invention is not limited to these embodiments.

[Overview]
The topic modeling apparatus 1 according to the present embodiment shown in FIG. 1 adds a constraint for extracting a geographical excitement in the conventional NMF that extracts a temporal excitement, thereby providing a geographical and temporal topic. Generate a topic model that takes into account excitement.

[Device configuration]
As shown in FIG. 1, the topic modeling apparatus 1 includes an input unit 10, a word feature amount calculation unit 20, a document feature matrix calculation unit 30, a position information extraction unit 40, a position information dependency matrix calculation unit 50, and a model calculation unit. 60 and an output unit 70.

  The functional units 10 to 70 are realized by computer hardware resources. That is, the topic modeling device 1 includes hardware resources related to a computer such as at least a computing device (CPU), a storage device (memory, a hard disk device, etc.), a communication interface, and the like. Then, the functional units 10 to 70 are implemented by cooperation of these hardware resources with software resources (OS, applications, etc.). Moreover, you may make it each implement | achieve the function parts 10-70 in each computer.

  The input unit 10 is a means for receiving input of time-series text data, and includes, for example, a keyboard, a mouse, a disk drive device, and the like. In the time series text data, a character string and time information are associated with each other. Furthermore, when there is a geotag representing position information measured by GPS, the geotag is also associated.

  The word feature amount calculation unit 20 calculates the word feature amount of the character string of each text in the time series text data received from the input unit 10.

  The document feature matrix calculation unit 30 calculates a document feature matrix of the text based on the word feature amount in the calculated text document.

  The position information extraction unit 40 extracts position information of each text in the time series text data and calculates a position information vector of each text.

  The position information dependency matrix calculation unit 50 calculates a position information dependency matrix indicating the similarity according to the position information between the texts based on the calculated position information vector.

  The model calculation unit 60 is based on the position information dependency matrix and the document feature matrix of the time series text, and as a topic model depending on position information and time information, a document topic matrix indicating the degree of relationship between text and topic, Then, a topic word matrix indicating the degree of relationship between topics and words is calculated.

  More specifically, the model calculation unit 60 converts the document feature matrix into a topic extraction constraint based on a temporal topic change amount and a topic extraction constraint based on the position information dependency matrix. The document topic matrix and the topic word matrix are calculated over time by performing matrix decomposition and dimension compression. Then, the latest document topic matrix and topic word matrix in which the amount of change with time of the document topic matrix and the topic word matrix is equal to or less than a predetermined value are determined as the topic model.

  The output unit 70 outputs the determined document topic matrix and topic word matrix as a topic model depending on position and time.

[Explanation of topic modeling process]
The topic modeling process will be described below with reference to FIGS.

  S1: The input unit 10 receives time-series text data as input data. The received time series text data is sent to the word feature quantity calculation unit 20 and the position information extraction unit 40.

  S2: The word feature amount calculation unit 20 calculates the word feature amount of the character string of each text in the time-series text data received from the input unit 10.

In this step, after the character string of each text is divided into word units such as nouns, verbs, and adjectives by a morphological analyzer, f _di representing the characteristic degree of the word w in the text document d _i based on the appearing word information. _{, w} is calculated. Specific examples of the calculation method include TF-IDF calculated by the following equation.

In the above equation, TF (d _i , w) is the number of occurrences of word w in document d _i , DF (w) is the number of documents in which word w appears in the data set, and N is the total number of documents in the data set. .

  The calculated word feature is sent to the document feature matrix calculator 30.

S3: document characteristic matrix calculation unit 30, f _di representative of the characteristics of the word w in document d _i, which is the _calculated, based on _w, calculate the document feature matrix. If the feature vector representing the feature of each word in the document d _i is f _di , the document feature matrix X is defined as follows.

  FIG. 3 shows an example of the document feature matrix X. In this example, since the feature amounts of “Yokosuka”, “Curry”, and “Niigata” of document 1 are 2, 2, 0, the elements corresponding to the row corresponding to document 1 and the column corresponding to each word are as follows. 2, 2, 0.

  The calculated document feature matrix X is provided to step S6 by the model calculation unit 60.

  S4: The position information extraction unit 40 extracts position information of each text in the time series text data received from the input unit 10, and calculates a position information vector p. The position information may be extracted using not only the geotag attached to the time series text but also the place name (region name) in the character string of the time series text.

  FIG. 4 shows a flowchart of a specific position information extraction process example in this step. The following processing is performed for each text document dεD in the time-series text data.

S401: It is determined whether a geotag is attached to the text document d _i .

S402: If a geotag is assigned to the text document d _i , the latitude / longitude information of the geotag is substituted into the position information vector p _i . P _i represents the i-th element of the vector p.

S403: If no geotag is assigned to the text document d _i , it is determined whether or not a word representing the place name exists in the character string of the document d _i . A known technique such as a dependency analysis method based on a conditional random field (Non-Patent Document 4) can be used as a method for determining a word representing a place name.

S404: If a word representing a place name exists in the character string of the text document d _i , the place name is converted into latitude and longitude information using a geocoder. When there are a plurality of place names, for example, one is selected at random using random numbers. Then, the latitude / longitude information obtained by this conversion is provided to the above-described step S402 and substituted into the position information vector p _i .

S405: A string can document d _i text no word representing a place name, substitutes null in the position information vector p _i.

  The above S401 to S405 are executed for all time series texts, and the obtained position information vector p is sent to the position information dependency matrix calculation unit 50.

  S5: The position information dependency matrix calculation unit 50 calculates the position information dependency matrix G based on the calculated position information vector p. The position information dependence matrix G is an m × m matrix that represents the similarity based on the position information of each document, where m is the total number of documents.

  FIG. 5 shows an example of the position information dependence matrix G. Since the similarity between the document 1 and the document 2 based on the position information is 0.02, the corresponding element is 0.02.

  A flowchart of the calculation process of the position information dependency matrix G is shown in FIG. All text pairs are processed as follows:

S501: 0 is substituted into the position information dependence matrix G _{i, j} . Note that G _{i, j} represents the element in the i-th row and j-th row of the matrix G.

S502: It is determined whether i = j or position information vector p _i = null or position information vector p _j = null. If the determination is YES, the process proceeds to the next loop process.

S503: If the determination in step S502 is No, the distance between the position information vectors p _i and p _j is calculated. The distance dist _{i, j} between the position information vectors p _i and p _j can be calculated by, for example, the following equation as the Hubeni distance.

  In the above equation, M is a meridian radius of curvature, dP is a latitude difference between two points, N is a Shaanxi radius of curvature, P is an average latitude between two points, and dR is a longitude difference between two points.

S504: calculating the distance dist i, and subjected to calculation of the following equation _j location dependent matrix G between the position information calculated above vectors p _i, p _j.

In the above equation, σ ² is a constant, and the position information dependency matrix G _{i, j} represents the similarity based on the position information of the document d _i and the document d _j . As σ ^2, a variance value calculated in advance using the distance dist _{i, j} of each document can be used.

  The above S501 to S504 are executed between all the texts, and the obtained position information dependence matrix G is provided to step S6 by the model calculation unit 60.

  S6: The model calculation unit 60 learns the topic model using the document feature matrix X calculated in step S3 and the position information dependency matrix G calculated in step S5.

  Since the topic model learned in this step detects a topic that is geographically active, the document topic matrix W is constrained so that the value of the following equation becomes small.

  In the above formula, L is a matrix representing a graph structure based on the distance of each text, and can be calculated as the following Laplacian matrix, for example.

  In the above equation, D is an m × m matrix where the total number of texts is m, the diagonal component has the sum of the elements of the corresponding row vector in the position information dependence matrix G, and the other components Has 0. This restriction has the property that the closer the position information of each text is, the more similar topic each has. This makes it possible to extract geographically popular topics. In consideration of time-dependent constraints and position-dependent constraints, the model calculation unit 60 calculates a document topic matrix W and a topic word matrix H by the following equations.

In the above equation (1), λ _t and λ _g are hyperparameters that determine the strength of time-dependent constraints and query-dependent constraints, respectively.

  As a specific flow of calculation processing, a flowchart of a process of calculating the document topic matrix W and the topic word matrix H by a method of alternately optimizing the document topic matrix W and the topic word matrix H is shown in FIG.

S601: a t = 0, the initial value H ⁽⁰⁾ of the initial value W ⁽⁰⁾ and the topic word matrix H of the document topic matrix W and determining the. The initial value may be any value, for example, it is initialized with a random value from 0 to 1.

S602: The topic word matrix H ^(t) is used in equation ( ¹⁾ to calculate the document topic matrix W ^{(t + 1)} . As a calculation method, a known technique such as a steepest descent method or a Newton method can be used.

S603: The topic word matrix H ^{(t + 1)} is calculated by using the document topic matrix W ^{(t + 1)} for the equation (1). As a calculation method, a known technique such as a steepest descent method or a Newton method can be used.

  S604: It is determined whether the end condition is satisfied. The termination conditions are illustrated below.

[Exit condition example 1]
Document topic matrix W ^(t) and topic word matrix H ^(t) obtained in the t-th iteration, document topic matrix W ^{(t + 1)} and topic word matrix H ^{(t + 1 )} obtained in the t + 1 iteration ^The end condition is that the amount of change in ⁾ is less than or equal to the specified value. For example, the document topic matrix W ^(t) and the topic word matrix H ^(t) obtained in the t-th iteration, the document topic matrix W ^{(t + 1)} and the topic word matrix H ^{(t in the t + 1} iteration ^The end condition is that the sum of the square error of each element in ⁺¹⁾ is not more than a specified value.

[Exit condition example 2]
An end condition is that t has reached a predetermined number of iterations.

[Exit condition example 3]
Satisfying both the end condition examples 1 and 2 is the end condition.

  If it is determined that the above illustrated termination condition is not satisfied, t + 1 is set as a new t, and the process proceeds to step S602.

On the other hand, if it is determined that the termination condition is satisfied, the process of step S6 is terminated, and the calculated document topic matrix W ^{(t + 1)} and topic word matrix H ^{(t + 1)} are respectively converted into the document topic matrix. W is provided to the output unit 70 as a topic word matrix H.

  S7: The output unit 70 outputs the document topic matrix W and the topic word matrix H calculated in step S6 as position-dependent and time-dependent topic models. This topic model is output and displayed on a monitor or the like.

[Effect of this embodiment]
As described above, according to the topic modeling device 1 of the present embodiment, the position information extraction unit 40 extracts position information from the time series text. In addition, the position information dependency matrix calculation unit 50 calculates a position information dependency matrix G indicating the degree of similarity depending on the position information between the time series texts based on the extracted position information. Then, the model calculation unit 60 uses the position information dependency matrix G and the document feature matrix X of the time series text as a topic model depending on the position information and the time information, and shows the degree of relationship between the text and the topic. A topic matrix W and a topic word matrix H indicating the degree of relationship between topics and words are calculated. As described above, the topic model depending on the position information and the time information is generated, so that topics that are exciting geographically and temporally can be extracted.

  In addition, when position information is not included in the time series text, the position information is extracted based on the area name included in the text. Therefore, even when position information is not included in the time series text, the position information can be extracted from the text.

  Further, in calculating the position information dependency matrix G, the similarity is calculated based on the distance between the position information of each time series text, so that the position information dependency matrix G with a clearer geographical relationship is obtained.

  In step S6, the determination in step S604 is executed, so that the topic model depending on the position information and time information can be acquired arbitrarily and accurately.

[Other Embodiments of the Present Invention]
The present invention can be realized by configuring a program that causes a computer to function as a part or all of the functional units 10 to 70 constituting the topic modeling apparatus 1 and causing the computer to execute the program. Alternatively, the above-described processes S1 to S7, S401 to S405, S501 to S504, and S601 to S604 executed by the apparatus 1 are configured by a program that causes a computer to execute and realized by causing the computer to execute the process. it can. The program can be provided by being stored in a known recording medium (for example, a hard disk, a flexible disk, a CD-ROM, etc.) that can be read by the computer. Alternatively, the program can be provided via the network via the Internet or e-mail.

  The present invention is not limited to the above-described embodiment, and various modifications and applications can be made within the scope of the claims.

DESCRIPTION OF SYMBOLS 1 ... Topic modeling apparatus 20 ... Word feature-value calculation part 30 ... Document feature matrix calculation part 40 ... Position information extraction part 50 ... Position information dependence matrix calculation part 60 ... Model calculation part

Claims (6)

A topic modeling device for creating a topic model depending on position information and time information,
Position information extraction means for extracting position information from time series text;
A position information dependency matrix calculating means for calculating a position information dependency matrix indicating a degree of similarity depending on position information between each time series text based on the extracted position information;
The document feature matrix of the time series text is subjected to matrix decomposition and dimension compression under the restriction of topic extraction based on a temporal topic change amount and the restriction of topic extraction based on the position information dependency matrix. To calculate the document topic matrix indicating the degree of relationship between the text and the topic and the topic word matrix indicating the degree of the relationship between the topic and the word over time, and the amount of change over time of the document topic matrix and the topic word matrix is a predetermined value. A topic modeling apparatus , comprising: model calculation means for determining, as the topic model, the latest document topic matrix and topic word matrix as described below . The model calculation unit, according to claim 1 in which the calculation of the document topic matrix and topic word matrix document topic matrix and topic word matrix upon reaching a predetermined number of times, as the topic models, and determines Topic modeling equipment. 3. The topic modeling according to claim 1, wherein, when the time series text does not include position information, the position information extraction unit extracts the position information based on an area name included in the text. 4. apparatus. The position information dependent matrix calculation means, topic modeling device according to any one of claims 1 to 3, and calculates the similarity based on the distance between the position information of each time series text. A topic modeling method executed by a topic modeling device that creates a topic model depending on position information and time information,
Extracting location information from time series text ;
Calculating a position information dependence matrix indicating similarity based on position information between time series texts based on the extracted position information ;
The document feature matrix of the time series text is subjected to matrix decomposition and dimension compression under the restriction of topic extraction based on a temporal topic change amount and the restriction of topic extraction based on the position information dependency matrix. To calculate the document topic matrix indicating the degree of relationship between the text and the topic and the topic word matrix indicating the degree of the relationship between the topic and the word over time, and the amount of change over time of the document topic matrix and the topic word matrix is a predetermined value. And determining the latest document topic matrix and topic word matrix as the topic model . A topic modeling program for causing a computer to function as each means constituting the topic modeling device according to any one of claims 1 to 4 .

Images