JP5605730B2 - Extraction apparatus, extraction method and extraction program - Google Patents

Description

  The present invention relates to an extraction apparatus, an extraction method, and an extraction program.

  Currently, coined words created by combining existing words are used as names for new products. Whether the coined word is prevalent or not depends on the combination of words that make up the coined word, but there are many word combinations that can be used in the world. I do not understand. Also, it is difficult to verify whether coined words are popular for all word combinations.

  To deal with this problem, in Non-Patent Document 1, the combination evaluation system estimates the novelty of a combination of words and the possibility of being accepted by the public from the number of appearances of the keyword on the WEB page, and thereby the effectiveness of the combination It is shown to determine.

Yoko Nishihara, Wataru Sunayama, Masahiko Taniuchi “Creative Activity Support by Discovering Effective Combinations”, IEICE Transactions DI, Vol. J87-D-I, no. 10, pp. 939-949, October 2004

  However, although the combination evaluation system in Non-Patent Document 1 can extract a keyword that is listed as a type in a text such as a WEB page, it is not listed as a type in the text, but part of the text Or, there is a problem that it is impossible to extract a concept that can be captured from the whole, and it is impossible to provide an unexpected combination of concepts.

  Therefore, the present invention has been made in view of the above problems, and an object thereof is to provide an extraction apparatus, an extraction method, and an extraction program that can provide a combination of unexpected concepts.

In order to solve the above problem, an extraction device according to an aspect of the present invention relates to information indicating a word and information indicating an affiliation degree to which the word belongs to a cluster. Cluster storage unit in which information indicating and information indicating the position of the word are stored in association with each other, and information on the position of the word associated with information indicating a word having an affiliation degree equal to or greater than a predetermined value from the cluster storage unit 3 or more clusters, and based on the information on the position of the word, the degree of indirect association between the two clusters via the third cluster other than the two target clusters, and the two clusters by multiplying the possible surprising degree combining, reading and finding index calculation unit that calculates a discovery index information indicating the appertaining to each of the clusters from the cluster storage section, read the Target relevance indicating the relevance between the two target clusters and the third cluster based on the information indicating the degree of affiliation and the information indicating the characteristics of the target input from the outside of the device. A target relevance index calculating unit for calculating an index; and a cluster set extracting unit for extracting a combination of the clusters based on the calculated heuristic index and the target relevance index. .

  The extraction device includes, for each predetermined period, information indicating the word and information indicating the importance of the word in association with each other, and an importance storage unit that stores the associated information from the importance storage unit for each predetermined period. Information indicating the degree of importance of the word, reading information indicating the degree of belonging for each cluster from the cluster storage unit, and based on information indicating the degree of importance of the word and information indicating the degree of belonging An activation prediction unit that predicts activation of each cluster for each period of time, and the cluster set extraction unit predicts activation of at least one cluster of the cluster combinations by prediction by the activation prediction unit If so, the combination of the clusters may be extracted based on the heuristic index and the target relevance index.

  The activation prediction unit of the extraction apparatus includes, for each predetermined period, information indicating the degree of affiliation that a word belongs to a predetermined cluster, and the word in the period read from the importance storage unit. An activity level calculation unit that calculates the activity level of the cluster based on the information indicating the importance level, and an activity level increase that is an expected increase in the activity level of each cluster based on the calculated activity level An activity increase expected value calculation unit for calculating an expected value, and predicting activation of the cluster based on the calculated activity and the calculated activity increase expected value It may be.

The heuristic index of the extraction device becomes higher as the indirect relevance and the unexpectedness are higher, and the cluster set extraction unit is based on a weighted sum of the heuristic index and the target relevance index, A combination of the clusters may be extracted.

In the extraction method according to one aspect of the present invention, information indicating a word is associated with information indicating a degree of belonging to which the word belongs to a cluster, and the information indicating the word and the position of the word are indicated A position of a word associated with information indicating a word having an affiliation degree equal to or greater than a predetermined value from the cluster storage unit, which is an extraction method executed by an extraction apparatus including a cluster storage unit stored in association with information Information for three or more clusters, and based on the information on the position of the word, the degree of indirect association between the two clusters via a third cluster other than the two target clusters, and the two by multiplying the surprising degree of combining clusters, reading and finding index calculation step of calculating a discovery index information indicating the appertaining to each of the clusters from the cluster storage section Based on the information indicating the degree of belonging that has been read and information indicating the characteristics of the target input from the outside of the device, the relevance between the two target clusters and the third cluster and the target And a target relevance index calculation procedure for calculating a target relevance index indicating a cluster combination extraction procedure for extracting a combination of the clusters based on the calculated heuristic index and the target relevance index. It is characterized by that.

An extraction program according to an aspect of the present invention relates to information indicating a word and information indicating an affiliation degree to which the word belongs to a cluster, and indicates the information indicating the word and the position of the word Information on the position of a word associated with information indicating a word having an affiliation degree equal to or greater than a predetermined value from the cluster storage unit is stored in a computer of an extraction apparatus including a cluster storage unit stored in association with information. Based on the information of the above-mentioned cluster reading and the position of the word, the indirect association degree between the two clusters via the third cluster other than the two target clusters can be combined with the two clusters. by multiplying the surprising degree, reading and finding index calculation step of calculating a discovery index information indicating the appertaining to each of the clusters from the cluster storage section Based on the information indicating the degree of belonging that has been read and information indicating the characteristics of the target input from the outside of the device, the relevance between the two target clusters and the third cluster and the target A target relevance index calculating step for calculating a target relevance index indicating a cluster, and a cluster set extraction step for extracting a combination of the clusters based on the calculated heuristic index and the target relevance index This is an extraction program.

  According to the present invention, an unexpected combination of concepts can be provided.

It is a block block diagram of the extraction apparatus in embodiment of this invention. It is an example of the word vector table T1 memorize | stored in the importance memory | storage part. It is a figure for demonstrating the process by a cluster production | generation part. It is a figure for demonstrating the calculation method of activity. It is the flowchart which showed the flow of the process which the extraction apparatus of this embodiment produces | generates a cluster. It is the flowchart which showed the flow of the process which the extraction device of this embodiment extracts the combination of a cluster.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. First, the outline | summary of the extraction apparatus 100 in embodiment of this invention is demonstrated. The extraction device 100 is related to the characteristics of the target (person) that is the target of providing the concept, which is compatible with both the fact that it is on the trend which is an important element of the buzzword and that there is a new surprise. Generated by combining some concepts. Thereby, the extraction apparatus 100 can present a concept (hit concept) that is popular in the world and is surprising to the target according to the characteristics of the target.

Here, the concept is expressed as a set of words appearing in the data. As a special case, there is a one word concept.
As a concept that becomes a combination element, there is a concept Cn that serves to connect the two concepts C1 and C2. The extraction device 100 extracts the concepts of C1, C2, and Cn by measuring a measure that measures a feature defined as a trend factor (hit factor) from time series data on a newspaper or the web.

  The extraction apparatus 100 extracts a combination in which the degree of direct association between the concepts C1 and C2 is low but the degree of indirect association of C1-Cn-C2 via the concept Cn is high. For example, if the target is a reader of a predetermined magazine that is related to the golf club (C1), the extraction device 100 has a golf club (C1), lipstick (C2), and Cn (Cn () as combinations of the concepts C1, Cn, and C2. Present). At first glance, the degree of association between the golf club and the lipstick is low, but the degree of indirect association between the two increases via the present concept Cn, so it is highly worth extracting the combination of the golf club (C1) and the lipstick (C2).

  Furthermore, the extraction apparatus 100 also sets the conditions for the concept extraction that the concepts tend to be activated during the target period and that at least one of the concepts is related to the characteristics of the target. For example, during the Christmas season, the concept of presents tends to be activated, and the relevance of a golf club is high for readers of a predetermined magazine as a target.

  The extraction apparatus 100 outputs the combination of the concepts C1, C2, and Cn as information indicating a novel concept for the target. Thereby, the extraction apparatus 100 can provide the target with information indicating the concept (for example, the concept C1) related to the target and the concept C2 related through the connection concept Cn. Thus, for example, a lipstick (concept) for a present (concept Cn) as a trendy concept (hit concept) at Christmas time for a reader of a predetermined magazine related to the target golf club (concept C1) C2) Special features can be provided.

FIG. 1 is a block configuration diagram of an extraction device 100 according to an embodiment of the present invention. The extraction apparatus 100 includes an importance calculation unit 101, an importance storage unit 102, a cluster generation unit 103, a cluster storage unit 104, a heuristic index calculation unit 110, a target relevance index calculation unit 114, and an activation. A prediction unit 120 and a cluster set extraction unit 130 are provided.
The heuristic index calculation unit 110 includes an indirect relevance calculation unit 111, an unexpectedness calculation unit 112, and an integration unit 113. The activation prediction unit 120 includes an activity calculation unit 121 and a relative force index calculation unit (activity increase expected value calculation unit) 122.

  The importance calculation unit 101 receives an article set D input from the outside of its own device. Here, the input article set D is time-series data of a document representing a social aspect such as a newspaper or a document representing a market characteristic such as a magazine. Then, the importance calculation unit 101 generates a document in which a predetermined period of documents is separated from the article set D and is arranged in chronological order. Here, a single segment document is referred to as one document, and a document for the entire period is referred to as an all document.

  The importance calculation unit 101 calculates information indicating the importance of words in each period. Specifically, for example, the importance calculation unit 101 divides the frequency tf at which the attention word appears in the document for each period by the total number of words in the document, so that the tf-idf of each word in each period. Calculate the value. Here, the tf-idf value is an index generally used as the importance of a word in information retrieval.

  The importance calculation unit 101 calculates a word vector in which the tf-idf values are arranged in a predetermined word order for each predetermined period. This word vector is a list of tf-idf values for each word, and represents the characteristics of that period. The importance calculation unit 101 associates information indicating the calculated word vector with the word and stores it in the word vector table T1 of the importance storage unit 102 for each period.

FIG. 2 is an example of the word vector table T1 stored in the importance storage unit 102. In the figure, the predetermined period is defined as one day, and the tf-idf values of the words for each day are shown in a predetermined word order. Each column represents a word vector (W_1, W_2, W_3,..., W_30).
In this way, by arranging the word vectors in time series order, the feature of the article for each predetermined period is shown in time order.

Returning to FIG. 1, the importance calculation unit 101 outputs a set of word vector information (hereinafter referred to as a word vector set) to the cluster generation unit 103.
The cluster generation unit 103 uses the word vector set input from the importance calculation unit 101 to classify the words into clusters that are a predetermined group, and gives a label to each cluster.

  In the present embodiment, it is assumed that the concept is represented by a set of similar words due to some commonality or relationship. The set mentioned here is a fuzzy set that represents the affiliation of an element by an arbitrary value between 0 and 1 even in the case of a normal set in which the affiliation of whether or not it is an element of the set is 0 or 1. In the case of both.

  Therefore, the cluster generation unit 103 clusters words that appear in the article set D according to a predetermined clustering method. Usually, one cluster includes tens of thousands of words, and each word has an affiliation level Mem C (w) that is a value belonging to the cluster. Here, the affiliation degree Mem C (w) represents a value to which the word w belongs to the cluster C. This value means the degree to which the cluster belongs to the corresponding concept.

Various methods have already been proposed for clustering, but the cluster generation unit 103
As an example, words appearing in the article set D are clustered by the k-means method. Specifically, the cluster generation unit 103 calculates a cluster that minimizes the evaluation value represented by the following formula (1). Here, k is assumed to be given in advance.

  However, the following conditional expression (2) is satisfied.

Here, x _i is the i-th word data (i is an integer from 1 to I), x _i = (x _i1 , x _i2 ), K is the number of clusters, and v _k is the centroid of the k-th cluster (k Is an integer from 1 to K), and v _k = (v _k1 , v _k2 ), g _ik is the degree of affiliation of the i-th data to the k-th cluster.

  The cluster generation unit 103 uses the k-means method, but is not limited thereto, and may use the fuzzy c-means method. In that case, specifically, the cluster generation unit 103 calculates a cluster that minimizes the evaluation value represented by the following formula (3). Here, k is assumed to be given in advance.

  However, the following conditional expression (4) is satisfied.

Here, x _i is the i-th word data (i is an integer from 1 to I), x _i = (x _i1 , x _i2 ), K is the number of clusters, and v _k is the centroid of the k-th cluster (k Is an integer from 1 to K), and v _i = (v _i1 , v _i2 ), and g _ik is the degree of affiliation of the i-th data to the k-th cluster.
As described above, the cluster generation unit 103 calculates the degree of affiliation belonging to the cluster for each element, regardless of which of the k-means method and the fuzzy c-means method.

  The cluster generation unit 103 assigns a label to assign one concept to each obtained cluster. Specifically, the cluster generation unit 103 uses the word closest to the cluster centroid as a representative of the cluster and uses it as the label of the cluster. Note that the cluster generation unit 103 may use a word having the highest degree of affiliation in the cluster as a representative of the cluster and use it as the label of the cluster.

  FIG. 3 is a diagram for explaining processing by the cluster generation unit 103. FIG. 3A is a diagram for explaining a cluster generated by the cluster generation unit 103. In the figure, an article set D is shown on the left side. On the right side, an example of a cluster is shown on the two-dimensional plane of xy.

  On the two-dimensional plane, each word that is each element of the cluster is indicated by a cross. Three clusters C_1, C_2, and C_3 are shown, and each cluster includes a word indicated by a cross in the circle. The cluster C_1 is a cluster to which the label of the agricultural product is given, and its elements include processor and orange. On the other hand, the cluster C_2 is a cluster to which a computer label is assigned, and the elements include processor and memory. That is, the processor belongs to the cluster C_1 in the sense of a food processing machine (food processor), and belongs to the cluster C_2 in the sense of a computer processor.

  The cluster C_3 is a cluster to which a brain label is assigned, and the element includes memory. In other words, memory belongs to cluster C_2 in the sense of computer memory, and belongs to cluster C_3 in the sense of brain memory.

  FIG. 3B is an example of the concept table T <b> 2 stored in the cluster storage unit 104. In the concept table T2, identification information C_i (i is a positive integer) for identifying the cluster shown in FIG. 3A and information indicating a label assigned to each cluster shown in FIG. And are associated.

  FIG. 3C is an example of the affiliation degree table T 3 stored in the cluster storage unit 104. In the affiliation degree table T3, information indicating the word shown in FIG. 3A and information indicating the affiliation degree to which the word belongs to the cluster are identified for each identification information C_i. Associated with.

  FIG. 3D is an example of a coordinate table T4 stored in the cluster storage unit 104. In the coordinate table T4, information indicating the word shown in FIG. 3A is associated with information indicating coordinates, which is information indicating the position of the word.

  Returning to FIG. 1, the cluster generation unit 103 obtains cluster identification information C_i (hereinafter, i is a positive integer from 1 to n indicating the index of the cluster) and information indicating a label assigned to each cluster. The data are stored in the cluster storage unit 104 in association with each other. Further, the cluster generation unit 103 associates the information indicating the word and the information indicating the degree of affiliation to which the word belongs to the cluster in the cluster storage unit 104 in association with the identification information C_i for identifying the cluster. Remember. Further, the cluster generation unit 103 causes the cluster storage unit 104 to store information indicating a word and information indicating the position of the word in association with each other.

Further, as shown in FIG. 3B, the cluster storage unit 104 includes identification information C_i for identifying a cluster and information indicating a label assigned to each cluster as a result of the processing by the cluster generation unit 103. Associated and stored.
Further, as shown in FIG. 3C, the cluster storage unit 104 shows information indicating the word and the degree of affiliation to which the word belongs to the cluster as a result of the processing by the cluster generation unit 103. Information is stored in association with each cluster.

  As a result of processing by the cluster generation unit 103, the cluster storage unit 104 stores information indicating a word and information indicating the position of the word in association with each other as shown in FIG. Here, for example, when the position of each word is allocated on the two-dimensional plane by clustering by the cluster generation unit 103, the information indicating the position of each word is information indicating the coordinates on the two-dimensional plane. .

The heuristic index calculation unit 110 reads information indicating the degree of affiliation associated with different clusters from the cluster storage unit 104 for a predetermined number (for example, three) of clusters, and information indicating the read degree of affiliation Based on the above, a heuristic index that reflects the degree of association between the two clusters via the third cluster other than the two target clusters and the unexpectedness of combining the two clusters is calculated.
Here, the heuristic index increases as the direct relationship between the two clusters decreases, and increases as the relationship between the two clusters and the remaining third cluster increases.

  The indirect degree-of-association calculation unit 111 reads out information on the position of the word associated with information indicating a word having an affiliation degree of a predetermined value or more from the cluster storage unit 104 for three or more clusters, and uses the position information of the word Based on this, the degree of indirect association between the two clusters via the third cluster other than the two target clusters is calculated. As an example, the indirect relevance calculation unit 111 calculates the maximum indirect relevance MIR among the relevance levels between the clusters that have passed through the third cluster other than the two target clusters.

  Specifically, for example, the indirect relevance calculation unit 111 associates a cluster C_i and a cluster C_j (hereinafter, j is an integer from 1 to n indicating a cluster index) via the connection cluster CN. The indirect relevance is calculated while changing the connection cluster CN from C_1 to C_n, and the maximum indirect relevance MIR that is the maximum among the calculated n indirect relevance It calculates using Formula (5). Here, the connection cluster CN can take clusters from C_1 to C_N.

MIR (C_i, C_j) = MAX _CN {A (C_i, CN) × A (CN, C_j)} (5)

Here, MAX _CN is a function that extracts the connection cluster CN that maximizes the indirect relevance of the right side that is an argument, and outputs the value of the argument at that time. A is the first argument and the second argument. It is a function for calculating relevance.
The indirect relevance calculating unit 111 may calculate the maximum indirect relevance MIR indicating the degree to which the clusters C_i and C_j are related via the connection cluster CN using the following equation (6). Good.

MIR (C_i, C_j) = MAX _CN {A (C_i, CN) + A (CN, C_j)} (6)

  The indirect relevance calculation unit 111 calculates the relevance A in Expression (5) or Expression (6) using the cosine similarity.

As an example, a method in which the indirect relevance calculation unit 111 calculates the relevance A using the cosine similarity will be described.
The vector x is a vector from the origin to the center of gravity of the cluster C_i, and the vector y is a vector from the origin to the center of gravity of the cluster C_j. For example, the indirect relevance calculation unit 111 calculates the relevance A according to the following formula (7).

  A (C_i, C_j) = x · y / (| x | × | y |) (7)

  Here, x · y is an inner product of vectors x and y, and is represented by (x1 × y1 + x2 × y2 +... + Xm × ym) (m is a positive integer). | X | is the norm of the vector x = √ (x · x). The right side of Equation (7) represents the cosine cos θ of the angle θ formed by the vectors x and y, which is called cosine similarity and represents the closeness similarity in the direction of the vector.

The indirect relevance calculation unit 111 may calculate the relevance A in Expression (5) or Expression (6) using a method such as a Jacquard coefficient or mutual information.
When Jacquard coefficients are used, the indirect relevance calculation unit 111 calculates the relevance A based on the number of appearances of words that appear in one of the two clusters C_i and C_j when C_i and C_j are normal clusters. . Specifically, the indirect association degree calculation unit 111 calculates the association degree A according to the following equation (8).

Here, | C | is the number of elements (words) included in cluster C. The greater the degree of association A, the higher the similarity between the two clusters.
When the cluster C_i and the cluster C_j are fuzzy sets calculated by the fuzzy c-means method, the indirect relevance calculating unit 111 sets the _p p elements of the word vector x of the cluster C_i (p is 1 to P). (Integer), y _q is the q-th element of word vector y of cluster C_j (q is an integer from 1 to Q), and the relevance of cluster C_i and cluster C_j is calculated by the following equation (9).

  On the other hand, when the mutual information amount is used, the indirect association degree calculation unit 111 calculates the mutual information amount MI (C_i, C_j) of the cluster C_i and the cluster C_j as the association degree A according to the following equation (10). Here, the mutual information amount is an index of relevance obtained by a ratio in which two certain words co-occur.

Where x _p is the p th element of the word vector x of C_i, y _q is the q th element of the word vector y of C_j, P (x _p , y _q ) is the probability of simultaneous occurrence of x _p and y _q , _{P (x p), P (} y q) is a peripheral probability of occurrence of each _x p, _{y q.}

The indirect relevance calculating unit 111 calculates the maximum indirect relevance MIR (C_i, CN_ (i, j), C_j) for all combinations of the cluster C_i and the cluster C_j. Here, CN_ (i, j) is a cluster that is selected when the degree of indirect association between the cluster C_i and the cluster C_j is maximized, and the middle of the clusters C_1 to C_N for each combination of the cluster C_i and the cluster C_j. Is a cluster selected from
The indirect relevance calculating unit 111 includes information indicating all the calculated maximum indirect relevance MIR (C_i, CN_ (i, j), C_j), and the cluster C_i used when calculating each maximum indirect relevance MIR. , CN_ (i, j), and information indicating the combination of C_j are output to integrating section 113.

  The unexpectedness degree calculation unit 112 reads three or more clusters of information indicating words having a degree of membership of a predetermined value or more from the cluster storage unit 104, and based on the information indicating the positions of the read words, the combination of clusters is read out. An unexpected degree U is calculated. Specifically, for example, the unexpectedness degree calculation unit 112 uses the reciprocal of the relevance degree expression of Expression (7) as the unexpectedness degree, and according to the following expression, the unexpectedness degree U (C_i,) between the cluster C_i and the cluster C_j C_j) is calculated.

  U (C_i, C_j) = (| x | × | y |) / x · y (11)

  Here, the vector x is a vector from the origin to the center of gravity of the cluster C_i, and the vector y is a vector from the origin to the center of gravity of the cluster C_j.

  Note that the unexpectedness degree calculation unit 112 may calculate the unexpectedness degree using the reciprocal number of the Jackard coefficient (the reciprocal number on the right side of Expression (7)). In that case, specifically, the unexpectedness degree calculation unit 112 calculates the unexpectedness degree U (C_i, C_j) between the cluster C_i and the cluster C_j according to the following equation (12).

Here, the lower the relationship between the cluster C_i and the cluster C_j, the higher the unexpectedness U (C_i, C_j), reflecting that the combination of both clusters is unexpected.
Moreover, the unexpected degree calculation unit 112 may calculate the unexpected degree using the reciprocal number of the mutual information amount MI ((the reciprocal number on the right side of Expression (10))). In that case, specifically, the unexpectedness degree calculation unit 112 calculates the unexpectedness degree U (C_i, C_j) between the cluster C_i and the cluster C_j according to the following equation (13).

  U (C_i, C_j) = 1 / MI (C_i, C_j) (13)

The unexpectedness degree calculation unit 112 calculates the unexpectedness degree U (C_i, C_j) for all combinations of the cluster C_i and the cluster C_j.
The unexpectedness degree calculation unit 112 includes information indicating all the calculated unexpected degrees U (C_i, C_j), identification information of the cluster C_i used when each unexpected degree U (C_i, C_j) is calculated, and the cluster The identification information of C_j is output to the integrating unit 113.

  Subsequently, the integrating unit 113 calculates a discoverability index based on the maximum indirect association degree MIR and the unexpectedness degree U. Specifically, the integrating unit 113 calculates the degree of association between the two clusters (C_i, C_j) via the third cluster CN other than the two target clusters (C_i, C_j), and the two clusters. A cluster heuristic index S that reflects the unexpectedness of combining (C_i, C_j) is calculated according to the following equation (14).

  S (C_i, C_j) = MIR (C_i, C_j) × U (C_i, C_j) (14)

  The heuristic index S is used to make it necessary to have a relationship between the cluster C_i and the cluster C_j via the cluster CN and at the same time a combination of the cluster C_i and the cluster C_j needs a new surprise. It is an indicator. That is, the heuristic index S increases as the direct relationship between the two clusters (C_i, C_j) decreases, and the two clusters are related to the remaining third cluster (CN_ (i, j)). The higher the is, the higher it is.

  The accumulating unit 113 calculates the heuristic index S for all combinations of the clusters C_i and C_j, and outputs information indicating the calculated heuristic index S to the cluster set extraction unit 130. Further, the integrating unit 113 outputs information indicating the cluster C_i, information indicating the cluster C_j, and information indicating the connected cluster CN_ (i, j) to the target relevance index calculating unit 114.

  The target relevance index calculation unit 114 accepts information indicating target characteristics (for example, target social characteristics, market, personal characteristics) T input from the outside of the device itself. Further, the target relevance index calculating unit 114 receives the information indicating the cluster C_i, the information indicating the cluster C_j, and the information indicating the connected cluster CN_ (i, j) input from the accumulating unit 113.

  The target relevance index calculation unit 114 reads information indicating the degree of affiliation for each cluster (C_i, C_j, CN_ (i, j)) from the cluster storage unit 104, Based on the information T indicating the characteristics of the target input from the outside of the apparatus, a target relevance index N indicating the relevance between the three different clusters (C_i, C_j, CN_ (i, j)) and the target is obtained. calculate.

  Specifically, for example, the target relevance index calculation unit 114 calculates the target relevance index N according to the following equation (15).

  N (C_i, C_j, CN_ (i, j), T) = min (A (C_i, T), A (C_j, T), A (CN_ (i, j), T)) (15)

  The target relevance index calculation unit 114 outputs information indicating the calculated target relevance index N to the cluster set extraction unit 130.

The activity calculation unit 121 reads information indicating the word vector of each period from the importance storage unit 102, and determines the activity of each cluster in each period based on the read information indicating the word vector of each period. calculate.
Specifically, for example, when the activity of the i-th cluster C_i is R (C_i, k) in the k-th period, the activity calculation unit 121 calculates the activity according to the following equation (16). .

  R (C_i, k) = sim (Y_i, W_k) (16)

Here, Y_i is an affiliation vector composed of affiliations of words belonging to the cluster C_i, and W_k is a word vector of a document in the k-th period (k is a positive integer).
In the above equation (15), the activity calculation unit 121 obtains the similarity between the word vector W_k of the document in the kth period and the membership vector Y_i representing the cluster C_i as the activity of the cluster C_i as it is. Is.
The function sim is a function representing the degree of similarity, and is represented by the following formula (17) using the cosine similarity.

  sim (Y_i, W_k) = Y_i · W_k / (| Y_i | × | W_k |) (17)

  FIG. 4 is a diagram for explaining a method of calculating the activity. In the figure, each element of the degree-of-affiliation vector 401 indicates the degree of belonging of the words (Word 1 to Word M) belonging to the cluster (M is a positive integer). Further, each element of the word vector 402 of the document in the kth period indicates the tf-idf value of the word (Word 1 to Word M) belonging to the cluster in the document in the kth period.

  The activity calculation unit 121 may use a jacquard coefficient as the function sim. Further, the activity calculation unit 121 may calculate the activity R (C_i) of the cluster C_i according to the following equation (18).

Here, mem C_i (y _q ) is the degree of affiliation of the word y _{q to} the cluster C_i. MI (x _p , y _q ) is a mutual information amount between the word x _p and the word y _q . tfidf (x) is the tfidf value word _{x p} in word vectors.

  Instead of calculating using all the words included in each concept, the activity calculation unit 121 calculates a certain number of upper words having a high tf-idf value or tf of a word having a tf-idf value exceeding a predetermined value. The degree of activity may be calculated based on a word vector composed of -idf values. As a result, the activity level calculation unit 121 can reduce the number of calculations, thereby reducing the time required for the calculation.

The activity calculation unit 121 outputs information indicating the calculated activity R (C_i, k) of the cluster C_i for each period to the relative force index calculation unit 122.
Based on the activity R (C_i, k) of the cluster C_i of each period input from the activity calculation unit 121, the relative force index calculation unit 122 pays attention to the temporal change in the activity of each cluster, and The degree to which the activity of each cluster is expected to increase (activity increase expectation value) is calculated for the general, target market, and individual.

Specifically, for example, the relative force index calculation unit 122 calculates a relative force index RSI (C_i) as an example of an activity increase expected value. Here, the relative force index (RSI) is a ratio of an increase width with respect to the movement of the past value, and it is generally said that when the RSI value falls below 30, it tends to increase. When calculating the relative force index (RSI), the relative force index calculating unit 122 provides a sampling period of a predetermined length such as one month or one day, and the activity increase value within the sampling period A relative force index (RSI) is calculated from the descending value.
For example, the relative force index calculation unit 122 calculates a relative force index (RSI) according to the following equation (19).

    RSI = u / (u + d) × 100 (19)

Here, u is the sum of the increase values of the activity during the predetermined sampling period, and d is the sum of the decrease values of the activity during the predetermined sampling period.
Although the relative force index calculation unit 122 uses the relative force index RSI as the activity increase expected value, the present invention is not limited to this, and other economic indicators may be used.

Then, the activation prediction unit 120 predicts the activation of the cluster based on the calculated activity and the calculated activity increase expected value.
Specifically, the activation predicting unit 120 generalizes the value of 30 as the threshold L, and sets the following two conditions for predicting the increase. The first is that (i) the relative force index (RSI) may have fallen below a threshold L during a certain period in the past, and the second is that (ii) the current activity value R is an upper limit Ru and a lower limit. It is between RL. The activation predicting unit 120 predicts the future activation of the cluster when these two conditions are satisfied, and otherwise predicts that the cluster will not be activated from now on.

The activation prediction unit 120 outputs information indicating the prediction result to the cluster set extraction unit 130.
The cluster set extraction unit 130 receives information indicating the heuristic index S from the integration unit 113, information indicating the target relevance index N from the target relevance index calculation unit 114, and information indicating the prediction result from the activation prediction unit 120. receive.

When the activation prediction unit 120 predicts the activation of at least one cluster among the cluster combinations, the cluster set extraction unit 130 determines the cluster based on the heuristic index S and the target relevance index N. Extract combinations.
Specifically, the cluster set extraction unit 130 extracts a combination of clusters (C_i, C_j, CN (i, j)) based on the following three conditions.

(1) As a condition of the new heuristic index S, the heuristic index S of the cluster set C_i, C_j, CN (i, j) is greater than or equal to a predetermined value;
(2) As a condition for the activation prediction, the relative force index (RSI) and the activity R of any one of the cluster C_i, the cluster C_j, and the cluster CN (i, j) are respectively set as the activation prediction condition (i ) And (ii)
(3) As a condition of the target relevance index N, any one of the cluster sets C_i, C_j, and CN (i, j) has a relevance greater than a predetermined value with the target characteristic T.

  For example, the cluster set extraction unit 130 calculates an optimum cluster combination (C_i, C_j, CN (i, j)) for the characteristic T from the following equation (20) when the characteristic T of a certain target exists. To do.

  arg max {aS (C_i, C_j, CN (i, j)) + bN (C_i, C_j, CN (i, j), T)} (20)

  Here, a and b are coefficients representing weights for S and N, and arg max is a function for obtaining a value that maximizes the argument. From this equation (18), the cluster set extraction unit 130 can extract the combination of clusters having the maximum argument value. However, the relative force index (RSI) and the activity R of any of C_i, C_j, and CN (i, j) satisfy the cluster activation prediction conditions (i) and (ii), respectively. To do.

  In the present embodiment, the cluster set extraction unit 130 extracts, as an example, one cluster combination that maximizes the argument of Expression (20). However, the present invention is not limited to this. The cluster set extraction unit 130 may extract all the combinations of one or more clusters in which the value of the argument in Expression (20) is a predetermined value or more. In addition, the cluster set extraction unit 130 may extract all combinations of clusters of the top M (M is a positive integer) in descending order of the argument value of Expression (20).

Then, the cluster set extraction unit 130 outputs the information indicating the cluster C_i, the information indicating the cluster C_j, and the information indicating the cluster CN_ (i, j) constituting the extracted cluster combination to the outside of the own apparatus.
The cluster set extraction unit 130 reads the labels associated with the clusters constituting the extracted cluster combination from the table T2 of the cluster storage unit 104, and indicates the read concept information indicating the hit concept combination. Information may be output outside the device itself.

  FIG. 5 is a flowchart showing a flow of processing in which the extraction apparatus 100 of the present embodiment generates a cluster. First, the importance calculation unit 101 calculates the tf-idf value of each word posted in a document separated by a predetermined period (step S101). Next, the importance calculation unit 101 calculates a word vector in which tf-idf values of the respective words are arranged in a predetermined word order every predetermined period (step S102).

  The importance calculation unit 101 determines whether the word vector has been calculated for the document for the entire period (step S103). If the word vector is not calculated for the document for the entire period (NO in step S103), the importance calculation unit 101 returns to the process of step S101. On the other hand, when the importance calculation unit 101 calculates a word vector for the document for the entire period (YES in step S103), the cluster generation unit 103 generates a cluster (step S104).

  Next, the cluster generation unit 103 calculates the degree of belonging to the cluster for each word (step S105). Next, the cluster generation unit 103 extracts a cluster label for each cluster (step S106). Next, the cluster generation unit 103 associates the cluster identification information with the information indicating the cluster label, and stores them in the cluster storage unit 104 (step S107). Next, the cluster generation unit 103 stores the information indicating the word and the information indicating the degree of affiliation with each cluster in the cluster storage unit 104 in association with each cluster (step S108). Above, the process of this flowchart is complete | finished.

  As described above, the extraction apparatus 100 can calculate the importance of each word posted in a document separated by a predetermined period from the article set D. Further, the extraction apparatus 100 can generate a cluster from the article set D.

  FIG. 6 is a flowchart showing a flow of processing in which the extraction apparatus 100 according to the present embodiment extracts a combination of clusters. First, the indirect association degree calculation unit 111 calculates the maximum indirect association degree MIR (step S201). Next, the indirect association degree calculation unit 111 determines whether or not the maximum indirect association degree MIR has been calculated for all combinations of clusters (step S202). If the indirect relevance calculation unit 111 has not calculated the maximum indirect relevance MIR for all combinations of clusters (NO in step S202), the process returns to step S201.

  On the other hand, when the indirect association degree calculation unit 111 calculates the maximum indirect association degree MIR for all combinations of clusters (YES in step S202), the unexpected degree calculation unit 112 calculates the unexpected degree U (step S203). Next, the unexpectedness degree calculation unit 112 determines whether or not the unexpectedness degree U has been calculated for all combinations of clusters (step S204). If the unexpectedness degree calculation unit 112 has not calculated the unexpectedness degree U for all combinations of clusters (NO in step S204), the process returns to the process in step S203.

  On the other hand, when the unexpectedness degree calculation unit 112 calculates the unexpectedness degree U for all the combinations of clusters (YES in step S204), the integrating unit 113 calculates a discoverability index (step S205). Next, the integrating unit 113 determines whether or not the heuristic index is calculated for the document for the entire period (step S206). The accumulation unit 113 returns to the process of step S201 when the discoverability index is not calculated for the document of the entire period (NO in step S206).

  On the other hand, when the integrating unit 113 calculates the discoverability index for the document for the entire period (step S206 YES), the target relevance index calculating unit 114 calculates the target relevance index (step S207).

  In parallel with the processing from step S201 to step S207, the extraction apparatus 100 performs the processing from step S208 to step S215. At that time, the extraction apparatus 100 first initializes i, j, and k. Next, the processing activity calculation unit 121 calculates the activity of the i-th cluster C_i in the k-th period (step S208). Next, the activity level calculation unit 121 determines whether the activity levels of all the clusters have been calculated (step S209). If the activity calculation unit 121 has not calculated the activity of all the clusters (NO in step S209), i is increased by 1 (step S210), and the process returns to step S208.

On the other hand, when the activity calculation unit 121 calculates the activity of all the clusters (YES in step S209), the activity calculation unit 121 determines whether the activity is calculated for the document for the entire period (step S211). . If the activity level is not calculated for the document for the entire period (NO in step S211), the activity level calculation unit 121 increases k by 1 (step S212), and the process returns to step S208.
On the other hand, when the activity calculation unit 121 calculates the activity of the document for the entire period (YES in step S211), the relative force index calculation unit 122 calculates the relative force index (RSI) of the jth cluster C_j (step S211). S213).

Next, the relative force index calculation unit 122 determines whether or not the relative force index (RSI) of all the clusters has been calculated (step S214). If the relative force index calculation unit 122 has not calculated the relative force index (RSI) of all clusters (NO in step S214), j is increased by 1 (step S215), and the process returns to step S213.
On the other hand, when the relative force index calculation unit 122 calculates the relative force index (RSI) of all the clusters (step S214 YES), the extraction device 100 proceeds to the process of step S216.

  Next, in step S216, the cluster set extraction unit 130 extracts a combination of clusters that maximizes the evaluation value based on the new discovery index and the target relevance index under the activation prediction condition (step S216). S216). Above, the process of this flowchart is complete | finished.

  As described above, the extraction apparatus 100 according to the present embodiment is that at least one of the three extracted clusters is activated, the combination of the two extracted clusters is surprising, and the two clusters The combination is not directly related, but it is connected via the other extracted cluster (third cluster), extracted from the characteristics of the target for which the combination of clusters is provided A combination of clusters can be provided under the condition that at least one of the clusters is relevant. Since each cluster corresponds to one concept, the extraction device 100 is a combination of concepts that are surprising to the target for a given period and are linked via a third concept corresponding to the third cluster. Can be provided.

  In addition, by recording a program for executing each process of the extraction device 100 of the present embodiment on a computer-readable recording medium, causing the computer system to read and execute the program recorded on the recording medium, The above-described various processes related to the extraction apparatus 100 may be performed.

  Here, the “computer system” may include an OS and hardware such as peripheral devices. Further, the “computer system” includes a homepage providing environment (or display environment) if a WWW system is used. The “computer-readable recording medium” means a flexible disk, a magneto-optical disk, a ROM, a writable nonvolatile memory such as a flash memory, a portable medium such as a CD-ROM, a hard disk built in a computer system, etc. This is a storage device.

    Further, the “computer-readable recording medium” refers to a volatile memory (for example, DRAM (Dynamic) in a computer system serving as a server or a client when a program is transmitted via a network such as the Internet or a communication line such as a telephone line. Random Access Memory)) that holds a program for a certain period of time is also included. The program may be transmitted from a computer system storing the program in a storage device or the like to another computer system via a transmission medium or by a transmission wave in the transmission medium. Here, the “transmission medium” for transmitting the program refers to a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line. The program may be for realizing a part of the functions described above. Furthermore, what can implement | achieve the function mentioned above in combination with the program already recorded on the computer system, and what is called a difference file (difference program) may be sufficient.

  As mentioned above, although embodiment of this invention was explained in full detail with reference to drawings, the concrete structure is not restricted to this embodiment, The design etc. of the range which does not deviate from the summary of this invention are included.

100 Extraction Device 101 Importance Calculation Unit 102 Importance Storage Unit 103 Cluster Generation Unit 104 Cluster Storage Unit 110 Heuristic Index Calculation Unit 111 Indirect Relevance Calculation Unit 112 Unexpectedness Calculation Unit 113 Integration Unit 114 Target Relevance Index Calculation Unit 120 Activity Prediction unit 121 activity calculation unit 122 relative force index calculation unit (activity increase expected value calculation unit)
130 Cluster group extraction unit

Claims (6)

A cluster in which information indicating a word is associated with information indicating the degree of affiliation to which the word belongs to the cluster, and the information indicating the word and the information indicating the position of the word are stored in association with each other A storage unit;
The information on the position of the word associated with the information indicating the word whose degree of belonging is a predetermined value or more is read from the cluster storage unit for three or more clusters, and based on the information on the position of the word, A heuristic index calculator that calculates a heuristic index by multiplying an indirect association degree between the two clusters via a third cluster other than the cluster and an unexpected degree of combining the two clusters;
Read the information indicating the degree of affiliation for each cluster from the cluster storage unit, and based on the information indicating the degree of affiliation read out and the information indicating the characteristics of the target input from the outside of the own device A target relevance index calculating unit for calculating a target relevance index indicating relevance between the two clusters and the third cluster and the target;
A cluster set extraction unit that extracts a combination of the clusters based on the calculated discoverability index and the target relevance index ;
An extraction device comprising: An importance storage unit that stores information indicating the word and information indicating the importance of the word in association with each other for each predetermined period;
Information indicating the importance of the word is read from the importance storage unit for each predetermined period, information indicating the affiliation for each cluster is read from the cluster storage unit, and information indicating the importance of the word and the affiliation And an activation prediction unit that predicts activation of each cluster every predetermined period based on the information indicating the degree,
When the activation of at least one cluster of the cluster combinations is predicted by the prediction by the activation prediction unit, the cluster set extraction unit is configured to use the cluster based on the heuristic index and the target relevance index. The combination according to claim 1 is extracted. The activation prediction unit
For each predetermined period, based on information indicating the degree of affiliation of a word belonging to a predetermined cluster and information indicating the importance of the word in the period read from the importance storage unit, An activity calculation unit for calculating the activity of the cluster;
Based on the calculated activity, an activity increase expectation value calculation unit that calculates an activity increase expectation value that is a degree in which an increase in activity of each cluster is expected;
With
The extraction apparatus according to claim 2 , wherein the activation of the cluster is predicted based on the calculated activity and the calculated activity increase expected value. The heuristic index increases as the indirect relevance and the unexpectedness increase.
The cluster set extraction unit, the said discovery index based on the weighted sum of the target-related index, any one of claims 1 to 3, characterized in that extracts a combination of the cluster The extraction device described in 1. A cluster in which information indicating a word is associated with information indicating the degree of affiliation to which the word belongs to the cluster, and the information indicating the word and the information indicating the position of the word are stored in association with each other An extraction method executed by an extraction device including a storage unit,
The information on the position of the word associated with the information indicating the word whose degree of belonging is a predetermined value or more is read from the cluster storage unit for three or more clusters, and based on the information on the position of the word, A heuristic index calculation procedure for calculating a heuristic index by multiplying an indirect association degree between the two clusters via a third cluster other than the cluster and an unexpected degree of combining the two clusters;
Read the information indicating the degree of affiliation for each cluster from the cluster storage unit, and based on the information indicating the degree of affiliation read out and the information indicating the characteristics of the target input from the outside of the own device A target relevance index calculating procedure for calculating a target relevance index indicating relevance between the two clusters and the third cluster and the target;
A cluster set extraction procedure for extracting a combination of the clusters based on the calculated discoverability index and the target relevance index ;
The extraction method characterized by having. A cluster in which information indicating a word is associated with information indicating the degree of affiliation to which the word belongs to the cluster, and the information indicating the word and the information indicating the position of the word are stored in association with each other In the computer of the extraction device comprising a storage unit,
The information on the position of the word associated with the information indicating the word whose degree of belonging is a predetermined value or more is read from the cluster storage unit for three or more clusters, and based on the information on the position of the word, A discoverability index calculating step of calculating a discoverability index by multiplying the indirect association degree between the two clusters via a third cluster other than the cluster and the unexpectedness of combining the two clusters;
Read the information indicating the degree of affiliation for each cluster from the cluster storage unit, and based on the information indicating the degree of affiliation read out and the information indicating the characteristics of the target input from the outside of the own device A target relevance index calculating step for calculating a target relevance index indicating relevance between the two clusters and the third cluster and the target;
A cluster set extraction step of extracting a combination of the clusters based on the calculated discoverability index and the target relevance index ;
Extraction program to execute.

Images