JP2012164242A - Related word extraction device, related word extraction method, related word extraction program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a related word extraction device which extracts various related words with less data analysis and presents an extraction result to a user.SOLUTION: Burst detection means 4 analyzes a query log of a retrieval log DB 3 by optional date units to detect retrieval words for which retrieval frequencies have rapidly increased. A burst information DB 5 stores detection results. Related word extraction means 6 refers to stored data in the DB 5 to acquire retrieval words for each detection date, and extracts related words of retrieval words from the query log. A burst related word information DB 7 stores the extracted related words. Related word output means 8 receives a retrieval word which a user has input as a request from a retrieval engine 2, and acquires related words for the received retrieval word from the DB 7, then replies the acquired related words to the retrieval engine 2.

Description

  The present invention relates to a technical field of a search engine, and more particularly to a technique for extracting related words of a search word (keyword) from a search log and recommending the extracted related word to a user as an additional search word.

  As is well known, a search engine responds with a list of documents on the Internet according to a character string (query) of a search term input by a user, and the user can obtain necessary information from a vast amount of information on the Internet. Is used. Here, nouns such as “people, things, and events” that the user wants to know are often input as search terms that the user inputs to the search engine.

  However, there are cases where a search word has a plurality of meanings, or a user has a search request to know something specific about the search word. As an example of the former, for the search term “Yakult (registered trademark)”, there is a meaning of a company or a baseball team in addition to “Yakult” as a beverage. In addition, as the latter example, there are a user who wants to know “autumn leaves” of “Kyoto” and a user who wants to know “cherry-blossom viewing”, “souvenir” and the like for the search term “Kyoto”.

  Therefore, search engines such as “goo (registered trademark)” and “Google (registered trademark)” additionally search related words taking into account the current trend for user search assistance as shown in FIG. It has a guide function that recommends and presents words, improving user convenience. As for this related word, as shown in FIG. 12, a method of extracting from the co-occurrence of the search word in the query log and the commonality of the click destination URL between the search words, or the electronic ranked in the top of the search result by the search word A method of Patent Document 1 is known that extracts from words appearing around a search word in a document (Web page).

JP 2006-139484 A

  However, although the space on the Web page for presenting additional search terms to the user is limited, when related terms are displayed in the order of relevance, related terms having the same meaning (for example, “Cherry”, “Hanami”, “ Sakura "etc.) may be placed at the top.

  In addition, since there are users with various intentions, it is preferable to recommend and present not only related words that are considered to have the same intention but also related words that have different intentions in order to improve the convenience of search. .

  However, in order to semantically cluster a set of related words extracted by a conventional method such as Patent Document 1, it is necessary to perform calculation after retaining the semantic information of each word. In this case, since there are a large number of types of search words, it is difficult to hold the semantic information of all the words.

  The present invention has been made in order to solve the above-described problems of the prior art, and it is an object of the present invention to extract various related words by analyzing less data and present them to the user. .

  Therefore, the present invention pays attention to the fact that search terms have a trend, and the term in which the search terms have increased rapidly and the search terms are grouped into one group (burst information), and the related terms (for example, the search terms are searched at the same time). For example). A group of related terms is extracted for each group in response to a request from the search engine and presented to the user.

  The related word extraction apparatus according to the present invention analyzes a search log in an arbitrary unit, detects a search word in which the number of searches increases at a rate equal to or greater than a threshold for each analysis unit, and detection by the detection unit A related word extracting means for extracting a related word group related to the search word from the search log, grouping the extracted related word group for each analysis unit, and storing the group group together with the search word in a database; And related word output means for sequentially selecting related words of search words input by the user from the database in response to a request from the search engine, and outputting the selected related word group to the search engine.

  The related word extraction method according to the present invention includes a detection step of analyzing a search log in an arbitrary unit, and detecting a search word in which the number of searches is increasing at a rate equal to or greater than a threshold for each analysis unit, and detecting in the detection step Extracting a related word group related to the search term from the search log, grouping the extracted related word group for each analysis unit, and storing the group group together with the search word in a database; and A related word output step of sequentially selecting related words of the search words input by the user from the database in response to a request of the search engine, and outputting the selected related word group to the search engine.

  In each of the above embodiments, the groups stored in the database or related word groups in the group can be clustered. For clustering, the similarity of related words in a group, information on a click destination, and the like are used.

  In addition, this invention is good also as an aspect of the program which makes a computer function as said extraction apparatus. This program can be provided through a network or a recording medium.

  According to the present invention, various related words can be extracted by analyzing less data and presented to the user.

The lineblock diagram of the related term extraction device concerning a 1st embodiment of the present invention. Explanatory drawing of the related word output means. Explanatory drawing which extracts a related word for every burst detection day. The image figure of the same FIG. Output data example of the related wording means (keyword = Yakult) Output data example of the relevant wording means (keyword = Disney) Output data example of the related terming means (keyword = roll cake) The figure which shows the other process example of the related word extraction means. The block diagram of the related word extraction apparatus which concerns on 2nd Embodiment of this invention. Explanatory drawing of the related word output means. Explanatory drawing of a search engine guide function. The figure which shows the conventional related word extraction method.

<< First Embodiment >>
A related word extraction apparatus according to the first embodiment of the present invention will be described with reference to FIG. The extraction device 1 extracts related words related to the search word from the search log DB 3 of the search engine 2, and outputs the extracted related word in response to a request from the search engine 2. Specifically, the extraction device 1 is configured by a computer, and hardware resources of a normal computer such as a CPU. Memory (RAM). A hard disk drive device is provided.

  As a result of the cooperation between the hardware resource and the software resource, the extraction device 1 uses the burst detection means 4. Burst information DB5. Related term extraction means6. Burst related word information DB7. The related word output means 8 is implemented. Each DB 3.5.7 is assumed to be built in a storage device such as a memory (RAM) or a hard disk drive device. Here, a search log obtained from the search window of the search engine 2 is recorded in the DB 3. Here, the search log includes a query log of a search term used for the search and a click log of click destination information (such as URL and page information) for the search result of the search term.

  The detection means 4 analyzes query log data recorded in the DB 3 in arbitrary date and time units. Here, as an example, the data of the query log is analyzed on a daily basis, and a search word whose search frequency has increased rapidly is detected (burst detection step). A threshold is used to determine whether or not the number of searches has increased rapidly, that is, whether or not a burst has occurred. For example, when the number of searches increases by 3σ or more (σ is a standard deviation) with respect to the moving average value for the past several days (arbitrary period), it may be determined as abnormal and may be detected as a burst. At this time, the detection means 4 stores in the DB 5 burst information in which a burst detection date and a burst search word are paired.

  The extraction means 6 refers to the data stored in the DB 5 to acquire a search term for each detection date, analyzes the data in the query log, and extracts a related term group of search terms (related term extraction step). That is, only the query log data narrowed down by the search date and the search word from the stored data of the DB 5 is analyzed, and the related word group of the search word is extracted. When extracting related terms, the number of co-occurrence of words input simultaneously with the search term at the time of search or an index such as “tf · idf” obtained by dividing the number of co-occurrence by the number of searches of the search term is used. Alternatively, as in Patent Document 1, a related word may be extracted by analyzing a high-order Web page of the search result of the detection date.

  The related word group extracted here is grouped into related word groups for each burst detection date, and the related word group group is stored in the DB 7 together with the search word. Specifically, the DB 7 includes “search word. Related word group (detection date:“ word | score, word | score, word | score) ”, related word group (detection date:“ word | score, word | score, word | score),... Here, “score” recorded in pairs with each related word (word) indicates the degree of relevance of the related word and score information such as “tf · idf”.

  The output means 8 receives a user input as a request from the front end of the search engine 2 when a user inputs a search word into the search window of the search engine 2 through a terminal (not shown) (for example, a PC or a mobile phone). Receive search terms. The received search word record is acquired from the DB 7, and as shown in FIG. 2, related words are selected from the related word group of the acquired records without duplication in a round robin format.

  Any number of related terms selected here are returned to the front end of the search engine 2 and used for the guide function of the search engine 2. That is, the related word group received by the search engine 2 is displayed as a list in the additional word presentation space such as a search window as a recommended word to be added to the search word input by the user.

  As a result, related word groups can be extracted in consideration of the search word trend (burst) and presented to the user as search recommended words. That is, the number of searches for a certain search term is not constant, and may increase rapidly (burst). Typical reasons for this include being taken up by media such as TV and newspapers, and being talked about in certain Web communities (such as bulletin boards and social media).

  On the day of burst detection when the number of searches for such search terms has increased temporarily, a certain search term is biased toward a specific search intention (a high percentage of specific search intentions), so the relevant relationship according to that intention Words can be extracted from the query log. For example, among the users who use the search term “Yakult” in the steady state instead of the burst detection date, the percentage of users who intend to use the information (“beverage”, “health food”, “Yakult baseball team”) Assume that they are respectively (“40%”, “30%”, “30%”). However, if Yakult (Yakult Swallows) as a team wins on November 3, and the number of searches for “Yakult” has increased rapidly (when bursting), the percentage of search intentions (“5%”, “ 5% "," 90% "). The related word group extracted from the query log at this time includes many related words of the Yakult team.

  Therefore, if a related word group is created for each burst detection date, related words with various search intentions can be extracted and presented to the user, and the convenience of search can be improved. If it demonstrates based on FIG. 3, since the related word group of the burst 1 contains many related words of the intention B about the same search word, while the related word of the intention B is easy to be extracted, Since many related words of the intention A are included in the word group, the related words of the intention A are easily extracted. That is, as shown in FIG. 4, the related word group of burst 1.2 has words a. The ratio of b is increased by “0.3”. At this time, the word b extracted from the burst 1 is highly likely to be due to the intention B, and the word a extracted from the burst 2 is likely to be due to the intention A. Therefore, by presenting the related words extracted from each related word group to the user, the intention A.A. B's recommended words can be presented.

  In FIG. 5 to FIG. 7, “keyword” indicates a search word detected by the detection unit 4, “whole” indicates a related word group presented by the conventional guide function, and “burst consideration” indicates the extraction The related word group presented by the guide function using the apparatus 1 is shown, the underlined part shows the related word group not presented by the conventional guide function, and if the extracting apparatus 1 is used, the related word group having diversity is shown. It can be seen that it can be presented to the user.

  At this time, according to the extraction apparatus 1, since related words are extracted using only the query log data of the burst detection date, a variety of related words can be extracted by analyzing less data, thereby improving processing efficiency. Can also contribute. In addition, since a related word group is created for each burst and related words are presented as recommended words, the user can easily search for past fashions and events, and in this respect can also support the user's search activities.

  FIG. 8 shows another processing example of the extraction means 6. Here, the effect is further expanded by calculating the difference between the score information of the related words between the burst detection date and the steady state, that is, the difference of the “score” value of “word” stored in the DB 7.

  Specifically, an arbitrary period (in this case, one week) before the burst detection date is set as the steady state, and the degree of association of each related word in the steady state and the score average such as “tf · idf” are calculated as the burst detection date. Is subtracted from the score information of the same related word in, ie, the value of “score” stored in the DB 7. Thereby, the score value of the related word which has obtained a high score value in the steady state is decreased, and the related word having a relatively high score value is extracted as a higher rank only on the burst detection date.

  For example, the related word “Tokyo Disney Resort” in FIG. 8 obtains the maximum value “0.15” of “tf · idf” in the steady state, but on the burst detection date of the search word “Disney (registered trademark)”. Since the minimum value is “0.05”, it is extracted in the lower order in the related word group. On the other hand, although the related word “Disney room lamp” has the lowest value “0.02” of “tf · idf” in the steady state, the second value “0.14” is displayed on the burst date of the search word “Disney”. Therefore, it is extracted in the second place.

<< Second Embodiment >>
FIG. 9 shows a related word extraction apparatus according to the second embodiment of the present invention. The extraction device 11 is provided with clustering means 9 for clustering the data stored in the DB 7. Here, the related word group for the burst search word is regrouped.

  That is, there may be a related term group including similar search terms with the same intention in the related term group of search terms stored in the DB 7. For example, the search term “Kyoto” has a burst (trend) every spring, and “sakura”, “cherry blossom viewing”, “Yoshino”, etc. are extracted in the same way as related terms on the burst detection date. When these related words are extracted as round loops and related words are extracted, the number of related words having the same intention increases.

  At this time, since the space for presenting additional words such as a search window is limited, there is a possibility that related words having the same meaning are arranged at the top and various related words cannot be presented to the user. Therefore, clustering means 9 is provided in the extraction device 1 in order to cluster related word groups stored in the DB 7 and group similar groups into one group.

  The clustering means 9 acquires the records of the DB 7, and clusters related word groups in the acquired records based on the similarity of related words included in the group (clustering step). In this related word similarity determination, the type of the related word and the score information of “score” recorded in the DB 7 as a pair with the related word are used. As a clustering method, any means such as a typical Ward method or a K-average method may be used, and the number of clusters created at the time of clustering may be arbitrarily designated.

  For example, in a burst detected on April xx 20XX in the search term “Kyoto”, “Sakura” and “Hanami” are stored as related word groups, and “Sakura” and “Hanami” are also related words on April yy, 200Y. If they are stored as a group, they are considered related terms with the same intention. At that time, the types of related words “sakura” and “cherry-blossom viewing” and the degree of relevance (tf / idf etc.) are used as score information, and each related word group is clustered according to its distribution tendency.

  As described above, if the Yakult Swallows wins and the search term “Yakult” bursts for the first time, if related word groups of the same intention continue after the burst detection date, Cluster word groups.

  Then, related word groups of records included in each cluster are integrated, and a list of related words is created for each cluster in descending order of relevance. If the related terms in the related term group to be integrated at this time are duplicated, the total value of the score information may be used as the relevance level, the maximum value, the median value, or the average value may be used. It may be used.

  If it demonstrates based on FIG. 10, G1-G4 has shown the record of the said DB7, ie, the related word group of a certain search term. Here, G1. Since each related word in G2 has a search intention A (yy / mm / dd), the score distribution is similar, and as a result of clustering, G1. G2 is integrated.

  Then, as a result of clustering, the related word group group is reconstructed for each cluster, so that the record in the DB 7 is updated and stored. The output means 8 from the DB 7 updated here selects related words in order without duplication in a round robin format, and returns an arbitrary number of related words to the front end of the search engine 2.

  Since related word groups having the same intention are integrated and reconstructed in this way, it is possible to prevent an excessive number of related word groups created for each burst detection date or a large number of related word groups having the same intention. Therefore, the related word group can be effectively displayed in the additional word presentation space such as the search window, and can be easily used for display in the search engine.

  In addition, related word groups that are reconstructed as a result of clustering contain related words with the same intention, so if you display related words as recommended words for each related word group in the search window, etc. It is possible to present recommended words intended for a search to a user. At this time, considering that the space of the search window is limited, if the number of recommended words exceeds the threshold (arbitrary number), the representative related words may be displayed as recommended words in related word group units (cluster units). Good.

(1) Other processing example 1
The clustering means 9 can also perform clustering by related word unit instead of the related word group unit stored in the DB 7. For this clustering, click logs stored in the DB 3 are used.

  That is, the clustering means 9 extracts each record (search word and related word group group) from the DB 7. Information on the click destination clicked from both the search word and the related word for each extracted record is acquired from the click log of the DB 3, and the related word group is clustered by the acquired click destination information. As the click destination information, for example, the host name or path name of the click destination URL, or page information of the click destination URL (words with high appearance frequency, etc.) can be used. If these pieces of information are common, related words are arranged in the same cluster, and a list of related words is created for each cluster in the order of the score value of “score”.

  As a result of the clustering, the related word group is reconstructed for each cluster, the record of the DB 7 is updated for the search word and each cluster, and the same effect as described above can be obtained. Here, when the number of recommended words exceeds a threshold value (arbitrary number), the representative related words can be displayed as recommended words in units of related word groups.

(2) Other processing example 2
The clustering means 9 can also cluster the related word group groups in the DB 7 not by the similarity of the related words but by the click destination information of each related word group.

  Here, information on the click destination clicked from each related word in the related word group is acquired from the click log of the DB 3. The related word group group may be clustered with all the acquired click destination information, or the click destination information representing each related word group may be clustered. As a representative click destination, for example, a click destination with the highest number of clicks (above a predetermined rank) can be selected.

  At this time, as the click destination information, the host name or path name of the click destination URL, or page information of the click destination URL (for example, a word having a high appearance frequency) can be used as in the processing example 1. Related word groups having common information are integrated, and a list of related words is created for each cluster in descending order of the degree of relevance.

  As a result of the clustering, the related word group group is reconstructed for each cluster, the record of the DB 7 is updated to the reconstructed related word group (cluster), and the same effect as described above can be obtained. Again, if the number of recommended words exceeds a threshold (arbitrary number), the representative related words can be displayed as recommended words in units of related word groups.

≪Programs≫
The present invention can also be configured as a related word extraction program that causes a computer to function as a part or all of the means 4 to 9 of the extraction device 1.11. According to this related word extraction program, a part or all of the steps can be executed by a computer.

  The program can be provided through a network such as a website or e-mail. The program is stored in a recording medium such as a CD-ROM, DVD-ROM, CD-R, CD-RW, DVD-R, DVD-RW, MO, HDD, BD-ROM, BD-R, or BD-RE. It is also possible to record, save and distribute. This recording medium is read using a recording medium driving device, and the program code itself realizes the processing of the above embodiment, so that the recording medium also constitutes the present invention.

1.11 ... Related term extraction device 2 ... Search engine 3 ... Search log DB (database)
4 ... Burst detection means (detection means)
5 ... Burst information DB (database)
6 ... Related word extraction means 7 ... Burst related word information DB (database)
8 ... Related word output means 9 ... Clustering means

Claims (9)

A related word extraction device that extracts related words related to a search word based on a search log of a search engine in advance and returns a related word of a user input search word in response to a request from the search engine,
A detection means for analyzing a search log in an arbitrary unit, and detecting a search term in which the number of searches increases at a rate equal to or greater than a threshold for each analysis unit;
A related word group related to the search word detected by the detecting means is extracted from the search log, the extracted related word group is grouped for each analysis unit, and the related word group is stored in the database together with the search word. Extraction means;
A related word output means for sequentially selecting related words of user input search words from the database in response to a request of the search engine, and outputting the selected related word group to the search engine;
A related word extraction device comprising: A related word extraction device that extracts related words related to a search word based on a search log of a search engine in advance and returns a related word of a user input search word in response to a request from the search engine,
A detection means for analyzing a search log in an arbitrary unit, and detecting a search term in which the number of searches increases at a rate equal to or greater than a threshold for each analysis unit;
A related word group related to the search word detected by the detecting means is extracted from the search log, the extracted related word group is grouped for each analysis unit, and the related word group is stored in the database together with the search word. Extraction means;
Clustering means for clustering groups stored in the database with the similarity of each related word in each group, integrating each group, and updating the database,
Related word output means for sequentially selecting related words of user input search words from the database updated by the clustering means in response to a request from the search engine, and outputting the selected related word group to the search engine;
A related word extraction device comprising: A related word extraction device that extracts related words related to a search word based on a search log of a search engine in advance and returns a related word of a user input search word in response to a request from the search engine,
A detection means for analyzing a search log in an arbitrary unit, and detecting a search term in which the number of searches increases at a rate equal to or greater than a threshold for each analysis unit;
A related word group related to the search word detected by the detecting means is extracted from the search log, the extracted related word group is grouped for each analysis unit, and the related word group is stored in the database together with the search word. Extraction means;
Clicked information from the search term and related terms in the database is extracted from the click log in the search log, and the related terms in the database are clustered according to the extracted clicked information, and the database is updated. Clustering means to
Related word output means for sequentially selecting related words of user input search words from the database updated by the clustering means in response to a request from the search engine, and outputting the selected related word group to the search engine;
A related word extraction device comprising: A related word extraction device that extracts related words related to a search word based on a search log of a search engine in advance and returns a related word of a user input search word in response to a request from the search engine,
A detection means for analyzing a search log in an arbitrary unit, and detecting a search term in which the number of searches increases at a rate equal to or greater than a threshold for each analysis unit;
A related word group related to the search word detected by the detecting means is extracted from the search log, the extracted related word group is grouped for each analysis unit, and the related word group is stored in the database together with the search word. Extraction means;
Click destination information clicked from each related term in the group saved in the database is extracted from the click log in the search log, and the group of groups is clustered and integrated with the extracted click destination information, and the database record A clustering means for updating
Related word output means for sequentially selecting related words of user input search words from the database updated by the clustering means in response to a request from the search engine, and outputting the selected related word group to the search engine;
A related word extraction device comprising: A related word extraction method executed by a device that extracts a related word related to a search word based on a search log of a search engine in advance and returns a related word of a user input search word in response to a request from the search engine,
A detection step of analyzing a search log in an arbitrary unit, and detecting a search term in which the number of searches increases at a rate equal to or greater than a threshold for each analysis unit;
A related word group extracted from the search log related word groups related to the search word detected in the detecting step, the extracted related word groups are grouped for each analysis unit, and the group words are stored in the database together with the search words. An extraction step;
A related word output step of selecting related words of a search word input by a user from a database in order according to a request of the search engine without duplication, and outputting the selected related word group to the search engine;
The related word extraction method characterized by having. A related word extraction method executed by a device that extracts a related word related to a search word based on a search log of a search engine in advance and returns a related word of a user input search word in response to a request from the search engine,
A detection step of analyzing a search log in an arbitrary unit, and detecting a search term in which the number of searches increases at a rate equal to or greater than a threshold for each analysis unit;
A related word group extracted from the search log related word groups related to the search word detected in the detecting step, the extracted related word groups are grouped for each analysis unit, and the group words are stored in the database together with the search words. An extraction step;
A clustering step of clustering groups stored in the database according to the similarity of each related word in each group, integrating each group, and updating the database;
A related word output step of sequentially selecting related words of the search word input by the user from the database updated in the clustering step according to a request of the search engine, and outputting the selected related word group to the search engine;
The related word extraction method characterized by having. A related word extraction method executed by a device that extracts a related word related to a search word based on a search log of a search engine in advance and returns a related word of a user input search word in response to a request from the search engine,
A detection step of analyzing a search log in an arbitrary unit, and detecting a search term in which the number of searches increases at a rate equal to or greater than a threshold for each analysis unit;
A related word group extracted from the search log related word groups related to the search word detected in the detecting step, the extracted related word groups are grouped for each analysis unit, and the group words are stored in the database together with the search words. An extraction step;
Clicked information from the search term and related terms in the database is extracted from the click log in the search log, and the related terms in the database are clustered according to the extracted clicked information, and the database is updated. A clustering step to
A related word output step of sequentially selecting related words of the search word input by the user from the database updated in the clustering step according to a request of the search engine, and outputting the selected related word group to the search engine;
The related word extraction method characterized by having. A related word extraction method executed by a device that extracts a related word related to a search word based on a search log of a search engine in advance and returns a related word of a user input search word in response to a request from the search engine,
A detection step of analyzing a search log in an arbitrary unit, and detecting a search term in which the number of searches increases at a rate equal to or greater than a threshold for each analysis unit;
A related word group extracted from the search log related word groups related to the search word detected in the detecting step, the extracted related word groups are grouped for each analysis unit, and the group words are stored in the database together with the search words. An extraction step;
Click destination information clicked from each related term in the group saved in the database is extracted from the click log in the search log, and the group of groups is clustered and integrated with the extracted click destination information, and the database record The related words that select the related words of the user input search words in order without duplication from the database updated in the clustering step according to the request of the search engine, and output the selected related words to the search engine An output step;
The related word extraction method characterized by having.   The related word extraction program which makes a computer function as each means of the related word extraction apparatus of Claims 1-4.

Images