JP2013117866A - Keyword place name pair extraction device, method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To avoid that a pair cannot be extracted by a fact that a co-occurrence frequency is low, and to appropriately extract the pair without causing noise even when adding the co-occurrence frequencies together.SOLUTION: A keyword place name pair extraction device refers to a geographic database on the basis of an input keyword and a place name, acquires an aggregate of partial areas belonging to the place name, refers to a frequency database on the basis of each area of the partial area aggregate, acquires a whole frequency corresponding to the area and a co-occurrence frequency with the keyword of the area, and when determining that the keyword is widely distributed in the area in which the key word represents the place name, on the basis of the whole frequency and the co-occurrence frequency with the keyword, adds the co-occurrence frequencies of the partial areas together to update the frequency database, and when an aggregate of a pair of the keyword and the place name is input, searches the frequency database on the basis of the aggregate to extract a pair of a keyword and a place name other than the input pair.

Description

  The present invention relates to a keyword place name pair extraction apparatus, method, and program, and more particularly, to a keyword place name pair extraction apparatus, method, and program for extracting a specified keyword and place name pair. For example, the present invention relates to a keyword place name pair extraction apparatus, method, and program for applying to a technique of extracting “Ehime” corresponding to the keyword and displaying a map of the place name when “Mikan” is input as a keyword.

  When various things are specified as keywords, it is convenient to know the location related to the keywords. For example, when “Gyoza” is specified as a keyword, a map of “Utsunomiya” famous for gyoza is displayed as a place related to “Gyoza”, and information on stores providing dumplings in Utsunomiya is displayed on the map. Is beneficial. In order to do so, it is necessary to store a pair of a keyword and its associated place name as data.

  As a technique for extracting a pair of a keyword and a place name, a technique for extracting a pair having a certain relationship such as a name and an occupation has been proposed. A small number of pairs are prepared, and a large number of pairs are extracted by repeating extraction of patterns from the pairs and extraction of pairs from the patterns. By applying this method to keywords and place names, it is possible to extract a large number of pairs from a small amount of pairs consisting of keywords and place names (see, for example, Non-Patent Document 1).

Pantel, P., Pennacchiotti, M., Espresso: leveraging generic patterns for automatically harvesting semantic relations. COLING-ACL 2006.

  However, the method of Non-Patent Document 1 has a problem in that it is not extracted as a pair unless the co-occurrence frequency of the keyword and the place name is high. For this reason, when the size of the corpus used for calculating the co-occurrence frequency is small, the co-occurrence frequency is low and a pair cannot be extracted. In order to solve this problem, it is conceivable to add the co-occurrence frequencies in the partial areas (for example, wards belonging to the city, etc.) belonging to the area indicated by the place name to obtain the co-occurrence frequency in the place name. However, when the frequency of co-occurrence in some partial areas is high, adding them will cause noise when extracting pairs, and inappropriate pairs will be extracted. .

  The present invention has been made in view of the above problems, and determines whether or not a keyword is widely distributed in an area indicated by a place name. If the keyword is widely distributed, the co-occurrence frequency in a partial area is determined. Adding together the place name and keyword co-occurrence frequency avoids the inability to extract a pair due to low co-occurrence frequency, and also adds the keyword and place name pair appropriately without adding noise. It is an object of the present invention to provide a keyword place name pair extraction apparatus, method, and program capable of extracting a name.

In order to solve the above problems, the present invention (Claim 1) is a keyword place name pair extraction device that extracts a pair of a keyword and a place name corresponding to a set of input keyword and place name pairs,
A geographic database storing a place name and a set of partial areas to which the area represented by the place name belongs;
A frequency database storing the overall frequency in the entire document of partial keyword distribution and the frequency of co-occurrence with a plurality of keywords in the region;
A partial area acquisition means for referring to the geographic database based on the input place name and acquiring a set of partial areas belonging to the place name;
Referring to the frequency database based on each region of the partial region set, a frequency acquisition means for acquiring a total frequency corresponding to the region and a co-occurrence frequency with the keyword of the region;
Based on the co-occurrence frequency of the overall frequency and the keyword, if it is determined that the keyword is widely distributed in the area representing the place name, the frequency database is obtained by adding the co-occurrence frequencies of partial areas. Frequency integration means to update
And a place name pair extraction unit that searches the frequency database based on the set of the input keyword and place name pairs and extracts a keyword and place name pair.

Further, the present invention (Claim 2) is configured to calculate a distribution similarity score representing a similarity between a distribution obtained from the overall frequency of the frequency database and a distribution obtained from a co-occurrence frequency of keywords. Having sex score calculation means,
In the frequency integration means,
It is determined whether to integrate frequencies based on the distribution similarity score, and in the case of integration, the co-occurrence frequencies for each region are integrated and the frequency database is updated.

In the present invention (Claim 3), in the distribution similarity score calculation means,
Using KL divergence, the value becomes smaller as the distribution is similar,
In the frequency integration means,
When the distribution similarity score is smaller than a predetermined value, it is determined that the co-occurrence frequencies for each region are integrated.

  According to the present invention, it is determined whether or not a keyword is widely distributed in the area represented by the place name. If the keyword is widely distributed, the co-occurrence of the place name and the keyword is added by adding the co-occurrence frequencies in the partial areas. By setting the frequency, it is possible to avoid that the pair cannot be extracted due to the low co-occurrence frequency, and it is possible to appropriately extract the pair without adding noise even if they are added together.

It is a block diagram of the keyword place name pair extraction apparatus in one embodiment of this invention. It is an example of the geographic database in one embodiment of this invention. It is an example of the frequency database in one embodiment of this invention. It is a flowchart of the operation | movement in one embodiment of this invention. It is an example (1) of the distribution similarity score in one embodiment of the present invention. It is an example (2) of the distribution similarity score in one embodiment of the present invention. It is an example (3) of the distribution similarity score in one embodiment of the present invention.

  Embodiments of the present invention will be described below with reference to the drawings.

  FIG. 1 is a diagram showing a configuration of a keyword place name pair extraction device 100 according to an embodiment of the present invention. A keyword place name pair extraction device 100 shown in FIG. 1 includes a partial area acquisition unit 1, a frequency acquisition unit 2, a distribution identity score calculation unit 3, a frequency integration unit 4, a keyword place name pair extraction unit 5, and a geographic database. 6 and a frequency database 7.

  FIG. 2 shows an example of the geographic database 6. As shown in the figure, the geographic database 6 stores a place name and a set of partial areas to which the area represented by the place name belongs.

  FIG. 3 shows an example of the frequency database 7. The frequency database 7 shown in the figure stores, for each partial area, the overall frequency in which the area exists in the document and the co-occurrence frequency for each keyword related to the area. It is assumed that the co-occurrence frequency for each keyword is stored for the number of keywords K (n).

  The partial area acquisition unit 1 refers to the geographic database 6 for a place name, and acquires a set of partial areas to which the area represented by the place name belongs.

  The frequency acquisition unit 2 refers to the frequency database 7 for each partial area, and acquires the overall frequency and the co-occurrence frequency with the keyword.

  The distribution similarity score calculation unit 3 calculates a distribution similarity score representing the similarity between the distribution obtained from the overall frequency and the distribution obtained from the co-occurrence frequency of the keywords.

  The frequency integration unit 4 determines whether frequencies should be integrated based on the distribution similarity score. If it is determined that the frequencies should be integrated, the co-occurrence frequencies for each region are integrated to share the name of the place name input. The occurrence frequency is set, and the frequency database 7 is updated.

  The keyword place name pair extraction unit 5 extracts a pair of a keyword and place name other than the input pair by referring to the frequency database 7 when a set of search target keyword and place name pairs is inputted at the time of search.

  Next, the operation of the keyword place name pair extraction apparatus 100 will be described more specifically.

  FIG. 4 is a flowchart of the operation in one embodiment of the present invention.

  In the following process, steps 1 to 5 are processes for updating the frequency database, and step 6 is a search process for a set of input keywords and place names.

  Step 1) When a set of place names is input, the partial area acquisition unit 1 refers to the geographic database 6 for each place name and obtains a set of partial areas to which the area represented by the place name belongs. Here, it is assumed that “A” is input as the place name. An example of the geographic database 6 is shown in FIG. As a partial region corresponding to “A”, “a1, a2, a3, a4, a5, a6” are obtained. The method of acquiring a partial area is not limited to this, and an area having an address that coincides with the address representing the place name “A” may be used as a partial area.

  Step 2) The frequency acquisition unit 2 refers to the frequency database 7 for each of the partial areas, and the total frequency that the partial area exists in the document and the co-occurrence frequency with the keywords related to the area. To get. In the example of FIG. 3, when the total frequency and the co-occurrence frequency with the keyword K1 are acquired from the frequency database 7 for the partial area “a1”, they become 8000 and 8, respectively. Here, the document frequency is used as the overall frequency to be used. The total frequency used may be any as long as the distribution in each region is reflected, and the total frequency of the target keywords in that region may be used.

  The above-mentioned overall frequency refers to the frequency related to the entire data to be handled with respect to the frequency for each keyword. In the present embodiment, since the document frequency is used, if there are 1000 documents related to each region (for example, a document including the place name in the text) among the documents being handled, the total frequency is 1000.

  Step 3) The distribution similarity score calculation unit 3 calculates a distribution similarity score representing the similarity between the distribution obtained from the overall frequency and the distribution obtained from the co-occurrence frequency of the keywords. Here, KL divergence is used as a distribution similarity score representing the similarity between the distribution P obtained from the overall frequency and the distribution Q obtained from the co-occurrence frequency of keywords.

利用する分布類似性スコアは、全体頻度から得られる分布とキーワードとの共起頻度から得られる分布との間の類似性を表していればどのようなものでもよく、JSダイバージェンスを用いたりしても構わない。

The distribution similarity score to be used may be anything as long as it represents the similarity between the distribution obtained from the overall frequency and the distribution obtained from the co-occurrence frequency of the keyword, such as using JS divergence. It doesn't matter.

5 to 7 show examples of distribution similarity scores determined based on KL divergence. KL divergence is calculated for the keyword K1. When the document distribution P (a1) is calculated from the overall frequency of the frequency database 7,
P (a1) = 8000 / (8000 + 4000 + 2000 + 1000 + 500 + 500)
It becomes. The same applies to a2. When the keyword distribution Q (a1) is calculated,
Q (a1) = 8 / (8 + 6 + 4 + 4 + 4 + 4)
It becomes. The same applies to a2. As a result, P (a1) log (P (a1) / Q (a1)) = 0.314. The same applies to a2. The final distribution similarity score is 0.224. When the distribution similarity score is similarly calculated for the keywords K2 and K3, they are 0.520 and 0.044, respectively.

  Step 4) The frequency integration unit 4 determines whether or not the frequencies should be integrated based on the distribution similarity score. If it is determined that the frequencies should be integrated, the process proceeds to step 5; Move on to step 6. The KL divergence value becomes smaller as the distributions are similar, and becomes 0 at the minimum when the distributions are exactly the same, so it is determined here that distribution similarity scores of 0.3 or less should be integrated I will decide. The determination method of whether to integrate is not restricted to this, A different threshold value or a ratio may be used. Since it is determined that the keywords K1 and K3 should be integrated, the co-occurrence frequencies of K1 and K3 from a1 to a6 are integrated into the co-occurrence frequencies of the keyword K1 and K3 of the place name A, and the frequency database 7 is updated. No integration is performed for keyword K2.

  Step 5) If it is determined in step 4 that the frequency integration unit 4 should be integrated, the frequency occurrence unit 7 updates the frequency database 7 by integrating the co-occurrence frequencies of the respective areas into the co-occurrence frequency of the place name input.

  Step 6) When the keyword place name pair extraction unit 5 receives a set of keyword and place name pairs, the keyword place name pair extraction unit 5 refers to the frequency database 7 and extracts a pair of keyword and place name. A method such as Espresso (registered trademark) can be applied as a method for extracting a pair of a keyword and a place name.

In particular,
<Udon, Kagawa>
<Beef tongue, Sendai>
<Mandarin orange, caress>
<Takoyaki, Osaka> other than that input
<Hitsumabushi, Nagoya>
...
A large number of keyword / place name pairs can be acquired.

  Thus, if it is widely distributed in the area indicated by the place name such as keywords K1 and K3, the frequency database 7 is updated as the co-occurrence frequency of the place name and the keyword by adding the co-occurrence frequencies in the partial areas. Therefore, it can be used to extract a pair for a set of keyword and place name pairs input at the time of search. Further, since the integration is not performed when the keyword K2 co-occurs in a specific partial area, a pair can be appropriately extracted without generating noise.

  In addition, a series of operations of the constituent elements of the keyword place name pair extraction device shown in FIG. 1 is constructed as a program, installed and executed on a computer used as the keyword place name pair extraction device, or distributed through a network. It is possible to make it.

  The present invention is not limited to the above-described embodiments, and various modifications and applications are possible within the scope of the claims.

DESCRIPTION OF SYMBOLS 1 Partial area acquisition part 2 Frequency acquisition part 3 Distribution similarity score calculation part 4 Frequency integration part 5 Keyword place name pair extraction part 6 Geographic database 7 Frequency database

Claims (7)

A keyword place name pair extraction device that extracts a keyword and place name pair corresponding to a set of input keyword and place name pairs,
A geographic database storing a place name and a set of partial areas to which the area represented by the place name belongs;
A frequency database storing the overall frequency in the entire document of partial keyword distribution and the frequency of co-occurrence with a plurality of keywords in the region;
A partial area acquisition means for referring to the geographic database based on the input place name and acquiring a set of partial areas belonging to the place name;
Referring to the frequency database based on each region of the partial region set, a frequency acquisition means for acquiring a total frequency corresponding to the region and a co-occurrence frequency with the keyword of the region;
Based on the co-occurrence frequency of the overall frequency and the keyword, if it is determined that the keyword is widely distributed in the area representing the place name, the frequency database is obtained by adding the co-occurrence frequencies of partial areas. Frequency integration means to update
Based on the set of input keyword / place name pairs, the frequency database is searched, and place name pair extraction means for extracting a keyword / place name pair;
A keyword place name pair extraction device characterized by comprising: A distribution similarity score calculating means for calculating a distribution similarity score representing a similarity between a distribution obtained from the overall frequency of the frequency database and a distribution obtained from a co-occurrence frequency of keywords;
The frequency integration means includes
Determining whether to integrate frequencies based on the distribution similarity score, if integrating, integrate the co-occurrence frequency for each region, and update the frequency database,
The keyword place name pair extraction device according to claim 1. The distribution similarity score calculation means includes:
Using KL (Kullback Leibler) divergence, the value becomes smaller as the distribution is similar,
The frequency integration means includes
The keyword place name pair extraction device according to claim 2, wherein when the distribution similarity score is smaller than a predetermined value, it is determined that the co-occurrence frequencies for each region are integrated. A keyword place name pair extraction method for extracting a keyword and place name pair corresponding to a set of input keyword and place name pairs,
A geographic database storing a place name and a set of partial areas to which the area represented by the place name belongs;
A frequency database storing the overall frequency in the entire document of partial keyword distribution and the frequency of co-occurrence with a plurality of keywords in the region;
In a device having
A partial area acquisition unit refers to the geographic database based on the input place name, and acquires a partial area set belonging to the place name;
A frequency acquisition unit that refers to the frequency database based on each region of the partial region set and acquires a total frequency corresponding to the region and a co-occurrence frequency with the keyword of the region;
If the frequency integration means determines that the keyword is widely distributed in the area representing the place name based on the co-occurrence frequency of the overall frequency and the keyword, the co-occurrence frequency of the partial area is added. A frequency integration step of updating the frequency database together;
A place name pair extraction unit searches the frequency database based on the set of the input keyword and place name pairs, and extracts a place name pair of keywords and place names; and
The keyword place name pair extraction method characterized by performing. Distribution similarity score calculation means for calculating a distribution similarity score representing a similarity between a distribution obtained from the overall frequency of the frequency database and a distribution obtained from a co-occurrence frequency of keywords. Do the steps,
In the frequency integration step,
Determining whether to integrate frequencies based on the distribution similarity score, if integrating, integrate the co-occurrence frequency for each region, and update the frequency database,
The keyword place name pair extraction method according to claim 4. In the distribution similarity score calculation step,
Using KL divergence, the value becomes smaller as the distribution is similar,
In the frequency integration step,
The keyword place name pair extraction method according to claim 5, wherein when the distribution similarity score is smaller than a predetermined value, it is determined that the co-occurrence frequencies for each region are integrated. Computer
A keyword place name pair extraction program that functions as each means of the keyword place name pair extraction apparatus according to any one of claims 1 to 3.

Images