KR20080026948A - Method for related keyword group extraction - Google Patents

Abstract

A method for extracting a related keyword group is provided to enhance importance of keywords existing at plural documents of the same site group, to analyze how many site group related keyword candidates appear at, and to analyze relationship between a specific keyword and related keyword candidates. A method for extracting a related keyword group comprises the following several steps. A site group is formed by grouping sites searched via a specific keyword(S111,S112). Important keywords are extracted from the site group and related keyword candidates are generated by using modified TFIDF(Term Frequency Inverse Document Frequency) which enhances the importance of the keyword appearing at documents of the same site group and reduces the importance of the keyword appearing at various site groups(S113-S118). Relationship between the related keyword candidates and the specific keyword is analyzed by using a modified relation rule to which concentration rate and reliability are applied wherein the concentration rate means how frequently the related keyword candidates appear at the same site group and the reliability means how frequently the related keyword candidates appear at various site groups(S119). Relationship between the related keyword candidates and the specific keyword is analyzed by using Cosine similarity for analyzing content similarity among site groups(S120). Related keyword groups are extracted by intersecting the related keyword candidates analyzed via the modified relation rule with those analyzed via the Cosine similarity(S121,S122).

Description

{Method for Related Keyword Group Extraction}

1 is a flowchart of a method for extracting a related keyword group according to the present invention;

2 illustrates an embodiment of a method for extracting a related keyword group according to the present invention;

3 is another embodiment of a method for extracting a related keyword group according to the present invention;

4 is another embodiment of a method for extracting a related keyword group according to the present invention;

The present invention relates to a method for extracting a related keyword group, and more particularly, by using a term frequency inverse document frequency (hereinafter referred to as 'TFIDF') and a modified association rule in consideration of a modified inverse document frequency. A group of related keywords that can increase the importance of keywords from multiple documents in the group, identify how many site groups of related keyword candidates appear, and efficiently analyze the association between specific keywords and related keyword candidates to extract groups of related keywords. It relates to an extraction method.

As the use of the Internet becomes more common, the size of the advertising market using the Internet is growing rapidly. Representative Internet advertising methods include banner advertisement, pop up advertisement, e-mail advertisement, and video advertisement. It is a way to show the core contents that can interest the user and curious users to click on the advertisement and connect to the advertisement site. These methods must induce the user's attention in order to show the actual advertisement, which requires more than a certain time of exposure. And since it is made to a predetermined size, it cannot provide much content. The CTR of banner ads has recently fallen below 0.5%, and pop-up ads, e-mail ads, and video ads have been dropping even before the contents are reviewed. Accordingly, various efforts have been made to increase the effectiveness of Internet advertising, and the most recently used method is keyword advertising using a search engine.

Keyword advertising means that when a user searches a keyword in a search engine to find the information he / she wants, most of the search engines operate keyword ads as if the search result shows the advertiser's site as if it were a search result. In keyword advertising, the purchase cost is differentiated according to the number of impressions and the position of exposure. The keyword that users search for is priced higher, and the higher the position of exposure, the higher the price. This is because the more keywords a user searches, the more likely his ad is exposed, and the higher the user is, the more likely the user is to visit the site. Therefore, when an advertiser wants to purchase a keyword, there is a problem such that it cannot be purchased because the sale is completed or cannot be purchased due to a price burden. Therefore, it is necessary to create a keyword group similar or related to a specific keyword and to use it for product recommendation in a keyword shop. Currently, most search engine sites provide related keywords at the administrator's discretion regarding important keywords. In addition, some provide related keywords based on the association rules by using the keyword search order of users, but they are not used in the keyword shop.

The prior art TFIDF is a formula that calculates the importance of each keyword in a text document. The TFIDF can be representative of each document to calculate the similarity between the query input by the user and each document or the similarity between the document and the document. This method is used to extract important keywords.

The basic idea behind TFIDF is to consider keywords that appear more in one document but less in another. That is, inversely proportional to the frequency df (d _i) for priority v _i d _i of the keyword in the particular document is T d _i is proportional to the frequency tf (d _i) and the appearance in the T keyword is emerged in the other document. However, since the frequency of the keyword tends to be proportional to the length of the document, the importance of the keyword increases when the document length is long.

Equation 1 is a calculation formula of a normalized TFIDF.

[Equation 1]

T: any document d _i : Keywords that appeared in T

V _i : importance of d _i in T tf (d _i ): frequency of d _i in T

tf _max : The largest value of tf in T df (d _i ): Frequency of documents in which d _i appears

However, since the conventional TFIDF does not have a concept of a site group, a keyword that appears a lot in one site and a few appear in another site is regarded as an important keyword, and the importance of a keyword appearing evenly in the same site group decreases. Therefore, it is not suitable as a method of extracting important keywords representing a group of sites consisting of several documents (sites).

The association rule, which is a conventional technology, is used in fields such as shopping cart analysis, cross-selling strategy, and user access pattern analysis as a technique of analyzing a transaction composed of a set of items to determine the association between each item. This method finds the relationship of items (itemsets) that frequently appear together in several transactions, and calculates which items (itemsets) appear frequently at the same time in a transaction in which a specific items (itemsets) appear. That is, in a transaction in which item X appears, if item Y frequently appears, item Y is regarded as highly related to item X. In this case, item X is called all items and item Y is called after items, and all items and after items can be composed of a subset of item sets. However, item X may not be highly associated with item Y because, in contrast, item X may not be significant in a transaction in which item Y appears.

Support, confidence and improvement are used as a measure of validity of association rules. Generally, a rule is generated only when the minimum support and minimum reliability are satisfied, and the improvement is used as an item for evaluating the generated rule.

Equation 2 is a definition of support, which means the probability that an entire transaction includes X and Y simultaneously, and is a measure of how many times X and Y appear simultaneously in the entire transaction.

[Equation 2]

Equation 3 is a definition of reliability, which means the probability that a transaction including X simultaneously includes X and Y, and is a measure of how many times X and Y appear simultaneously in a transaction including X.

[Equation 3]

Equation 4 is a definition of the improvement degree, and compares the probability that the entire transaction includes the Y item and the probability that the transaction including the X includes the X and Y simultaneously.

[Equation 4]

Conventional association rules have a problem in that it is not appropriate to determine the association with the relevant keyword candidate in the site group for a particular keyword.

Accordingly, an object of the present invention is to solve the problems of the prior art, and by using the modified TFIDF, an object of the present invention is to increase the importance of keywords appearing in various documents of the same site group.

In addition, there is another purpose of determining how many site groups related keyword candidates appear by using the modified association rule, and efficiently analyzing the association between a specific keyword and the related keyword candidate to generate a related keyword group.

An object of the present invention is a first step of forming a site group by binding a site searched by a specific keyword; Extract key keywords from the site groups by using the terminology calculation method that takes into account the revised inverse document frequency, which increases the importance of keywords appearing in documents of the same site group and reduces the importance of keywords appearing in multiple site groups. Generating a keyword candidate; Relevance of the related keyword candidate to the specific keyword using a modified association rule applying a concentration that calculates that the related keyword candidates appear intensively in the same site group and a reliability that calculates the appearance of documents in multiple site groups. A third step of analyzing; A fourth step of analyzing a correlation between the related keyword candidate and the specific keyword by using cosine similarity for analyzing content similarity between each site group; And a fifth step of extracting a related keyword group by intersecting relevant keyword candidates analyzed by the modified association rule and the cosine similarity, respectively.

Another object of the present invention is a first step of forming a site group by binding a site searched by a specific keyword; Extract key keywords from the site groups by using the terminology calculation method that takes into account the revised inverse document frequency, which increases the importance of keywords appearing in documents of the same site group and reduces the importance of keywords appearing in multiple site groups. Generating a keyword candidate; Relevance of the related keyword candidate to the specific keyword using a modified association rule applying a concentration that calculates that the related keyword candidates appear intensively in the same site group and a reliability that calculates the appearance of documents in multiple site groups. A third step of analyzing; A fourth step of analyzing a correlation between the related keyword candidate and the specific keyword by using cosine similarity for analyzing content similarity between each site group; And a fifth step of extracting a related keyword group by merging related keyword candidates analyzed by the modified association rule and the cosine similarity, respectively.

Still another object of the present invention is a first step of forming a site group by binding a site searched by a specific keyword; Extract key keywords from the site groups by using the terminology calculation method that takes into account the revised inverse document frequency, which increases the importance of keywords appearing in documents of the same site group and reduces the importance of keywords appearing in multiple site groups. Generating a keyword candidate; Relevance of the related keyword candidate to the specific keyword using a modified association rule applying a concentration that calculates that the related keyword candidates appear intensively in the same site group and a reliability that calculates the appearance of documents in multiple site groups. A third step of analyzing; And extracting a related keyword group by using the related keyword candidate analyzed by the modified related rule.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. Prior to this, terms or words used in the specification and claims should not be construed as having a conventional or dictionary meaning, and the inventors should properly explain the concept of terms in order to best explain their own invention. Based on the principle that can be defined, it should be interpreted as meaning and concept corresponding to the technical idea of the present invention.

Therefore, the embodiments described in the specification and the drawings shown in the drawings are only the most preferred embodiment of the present invention and do not represent all of the technical idea of the present invention, various modifications that can be replaced at the time of the present application It should be understood that there may be equivalents and variations.

1 is a flowchart of a method for extracting a related keyword group according to the present invention.

First, a site group is formed by grouping sites searched by a specific keyword (S111) (S112).

If the number of sites in the site group is too small, it is difficult to say that there is a correlation between keywords. If the number of sites in the site group is smaller than the minimum number of sites (N ₁ ) entered by the user (S113), a site group is created. Do not (S114).

An important keyword representing each generated site group is extracted (S115). Important keywords representing each site group are related keyword candidates, and representative keywords constituting the site group are also related keyword candidates.

An important keyword in the related keyword group is a keyword that appears a lot in a site and appears in several documents in the same site group, while a small number of site groups in which the keyword appears. However, since the conventional TFIDF does not have a concept of a site group, a keyword that appears a lot in one site and a few keywords in another site is regarded as an important keyword, and thus, the importance of the keyword evenly appearing in the same site group is reduced.

Therefore, in order to overcome this problem, an important keyword which is a related keyword candidate is extracted using the modified TFIDF devised in the present invention.

Equation 5 is a modified TFIDF equation.

[Equation 5]

T: random site tf (d _i ): frequency of d _i in T

G: the site group to which T belongs df _G (d _i ): the frequency of the appearance of d _i in G

d _i : keyword from T n _G : number of sites in G

v _i The importance of v _i in T gf (d _i ): The number of site groups in which d _i appeared.

tf _max : the largest tf value in T N _G : total number of site groups

Equation 6 is a formula for calculating df in the modified TFIDF, Equation 7 is to calculate and reflect the frequency of group appearance.

[Equation 6]

[Equation 7]

In Equation 6, n _G represents the number of sites belonging to a site group G to which any site T belongs, and df _G (d _i ) is the number of sites where a specific keyword d _i appears in the site group G. Therefore, as the value of df _G (d _i ) increases, the value of Equation 6 increases, thereby increasing the importance of keywords appearing in multiple documents of the same site group.

Also, in Equation 7, gf (d _i ) is the number of site groups in which a specific keyword appears and N _G is the number of total site groups. Therefore, as the value of gf (d _i ) increases, the value of Equation 7 decreases, thereby reducing the importance of keywords appearing in various site groups.

In this case, since the purpose of generating the related keyword group is for sale in the market, it is necessary to determine the sale possibility. That is, even though the correlation between the keywords is high, if the keywords are not frequently searched by the general users, since there is little opportunity for the advertisement to be exposed, the related keyword candidates are filtered using the number of exposures (N ₂ ) in the search engine (S116). .

Keywords smaller than the number of exposures are deleted from the related keyword candidates (S117) and the related keyword candidates are generated from the remaining important keywords (S118).

Then, using the modified association rules devised in the present invention, the correlation between the generated related keyword candidate and the specific keyword is analyzed (S119). In the association analysis based on the modified association rule, the user constructs a list of important keywords of each site belonging to a site group by a specific keyword for which the user wants to create a related keyword group in one transaction. In order to reduce the exploration time, unlike the conventional association rule that analyzes the association between subsets of a set of items, all items and all items are set as a single item, but all items are specific keywords and all items are candidates for related keywords. .

According to the modified association rule according to the present invention, when a related keyword candidate appears evenly in various documents of a site group for a specific keyword, and a small number of site groups in which the related keyword candidate appears, the association between the specific keyword and the related keyword candidate is high. Consider. Therefore, we use the reliability of conventional association rule exploration as a measure of how many sites in a group of sites a particular keyword candidate appears for a particular keyword, and as a measure of how many site groups the relevant keyword candidate appears in. We introduce and use a new concept of). In addition, the degree of association is used as a measure of the association between a specific keyword and an associated keyword candidate, which is obtained by multiplying the reliability and the concentration.

Equations 8 and 9 are calculations of reliability and concentration.

[Equation 8]

df _G (r): number of sites where r appears in G d: specific keyword

n _G : The number of sites belonging to G r: associated keyword candidate

[Equation 9]

df _G (r): number of sites in which r appeared in G

df _all (r): number of sites in which r appeared in all sites

r: site group for associated keyword candidate G: d

d: the number of site groups where the specific keyword gf (r): r appears

N _G : total number of site groups

In Equation 8, n _G represents the number of sites belonging to the site group G for the specific keyword d, and df _G (r) is the number of sites where the associated keyword candidate r appears in the site group G. Therefore, as the value of df _G (r) increases, the reliability value of R: d → r increases, which means that the related keyword candidate r appears in various documents of the site group G.

In addition, in Equation 9, df _All (r) is the number of sites in which r appeared and gf (r) is the number of site groups in which r appeared. And N _G represents the total number of site groups. Therefore, as df _All (r) decreases and gf (r) decreases, the value of the concentration of R: d → r increases, which means that the related keyword candidate r appears intensively in the site group G.

The value of df _G (r) is always less than or equal to n _{G and} less than or equal to df _All (r). Also, gf (r) is always less than or equal to N _G. Therefore, the reliability and the concentration always have a value between 0 and 1, and the association, which is the product of the two, also has a value between 0 and 1.

Correlation analysis based on the modified association rule extracts keywords that appear in common from sites searched by specific keywords. Therefore, keywords that do not appear in the site group do not belong to the related keyword group. Keyword B cannot be an associated keyword for keyword A unless keyword B appears or appears less in the site group for keyword A. However, if the contents of the sites that make up the site group for each of the keywords A and B are similar, the two keywords should be related keywords.

Therefore, the present invention analyzes the association of the contents of the sites constituting each site group by using the cosine similarity (S120). In other words, if the correlation analysis based on the modified association rule is to analyze the association between a specific keyword and keywords frequently found in the sites searched by the keyword, the association analysis based on cosine similarity analyzes the content similarity between each site group. It is a way.

The final association keyword group is generated by combining the association keyword candidate generated by the modified association rule with the association keyword candidate based on cosine similarity (S121). The method of combining the related keyword groups includes a method of using an intersection of the related keyword candidate by the modified association rule and the related keyword candidate by the cosine similarity, a method using the union, and a method using only the related keyword candidate by the modified association rule. .

The following is an embodiment of implementing a method of extracting a related keyword group according to the present invention. Pentium-4 was implemented using JAVA on IBM compatible Window platform with 1.7GHz CPU and 512Mb memory. The experimental data refers to the sales status of keywords purchased for a month from the search engine of the search site 'Naver'. The specific keywords were marriage, loan, moving, travel agency and start-up. The minimum exposure count was 100, the exposure count was 0.7, and the sales count was 0.3.

2 is an embodiment of a method for extracting a related keyword group according to the present invention, which is a related keyword group generated using an intersection of two methods. The method by the modified association rule and the method by cosine similarity analyze the association of different meanings. Therefore, it is most meaningful to use the intersection between the candidate candidates of related keywords by the two methods, but in this case, there are not many keywords belonging to the intersection.

3 is another embodiment of a method for extracting a related keyword group according to the present invention, which is a related keyword group generated using a union of two methods. Since the associated keyword group generated by the intersection has few keywords, it is used to obtain many keywords.

4 is another embodiment of a method for extracting a related keyword group according to the present invention, which is a related keyword group generated using only a modified association rule. Correlation analysis based on cosine similarity may take a very long time depending on the number of specific keywords to generate a group of related keywords. Therefore, when the number of related keyword groups to be generated is large, it provides a method of generating related keyword groups by only analyzing the association by the modified association rule, not analyzing the association by cosine similarity.

Although the present invention has been shown and described with reference to the preferred embodiments as described above, it is not limited to the above embodiments and those skilled in the art without departing from the spirit of the present invention. Various changes and modifications will be possible.

Therefore, the related keyword group extraction method of the present invention can increase the importance of keywords appearing in several documents of the same site group by using the modified TFIDF.

Also, by using the modified association rule, it is possible to identify how many site groups of related keyword candidates appear, and thus, it is a significant and advantageous way to efficiently analyze the association between specific keywords and related keyword candidates for the generation of related keyword groups. It works.

Claims (8)

A first step of forming a site group by binding sites searched by a specific keyword; Extract key keywords from the site groups by using the terminology calculation method that takes into account the revised inverse document frequency, which increases the importance of keywords appearing in documents of the same site group and reduces the importance of keywords appearing in multiple site groups. Generating a keyword candidate; Relevance of the related keyword candidate to the specific keyword using a modified association rule applying a concentration that calculates that the related keyword candidates appear intensively in the same site group and a reliability that calculates the appearance of documents in multiple site groups. A third step of analyzing; A fourth step of analyzing a correlation between the related keyword candidate and the specific keyword by using cosine similarity for analyzing content similarity between each site group; And A fifth step of extracting a related keyword group by intersecting relevant keyword candidates analyzed based on the modified association rule and the cosine similarity; Association keyword group extraction method comprising a. A first step of grouping sites searched by a specific keyword to form a site group; Extract key keywords from the site groups by using the terminology calculation method that takes into account the revised inverse document frequency, which increases the importance of keywords appearing in documents of the same site group and reduces the importance of keywords appearing in multiple site groups. Generating a keyword candidate; Relevance of the related keyword candidate to the specific keyword using a modified association rule applying a concentration that calculates that the related keyword candidates appear intensively in the same site group and a reliability that calculates the appearance of documents in multiple site groups. A third step of analyzing; A fourth step of analyzing a correlation between the related keyword candidate and the specific keyword by using cosine similarity for analyzing content similarity between each site group; And A fifth step of extracting a related keyword group by combining related keyword candidates analyzed based on the modified association rule and the cosine similarity; Association keyword group extraction method comprising a. A first step of forming a site group by binding sites searched by a specific keyword; Extract key keywords from the site group by using the term frequency calculation method with modified inverse document frequency that increases the importance of keywords appearing in the documents of the same site group and decreases the importance of keywords appearing in the various site groups. Generating a keyword candidate; Relevance of the related keyword candidate to the specific keyword using a modified association rule applying a concentration that calculates that the related keyword candidates appear intensively in the same site group and a reliability that calculates the appearance of documents in multiple site groups. A third step of analyzing; And A fourth step of extracting a related keyword group by using the related keyword candidate analyzed by the modified related rule; Association keyword group extraction method comprising a. The method of claim 1, wherein the first step comprises: The associated keyword group extraction method, characterized in that the site group is not formed when the number of searched sites is small. The method according to any one of claims 1 to 3, wherein the second step, The related keyword group extraction method, characterized in that if the extracted keyword is exposed less than a predetermined number of times, the corresponding keyword is deleted. The method according to any one of claims 1 to 3, wherein in the second step, The term frequency calculation method taking into account the modified inverse document frequency, An associated keyword group extraction method calculated using the following equation. [Equation]

[T: any site, tf (d _i): frequency of one in the T d _i the occurrence, G: Site T belongs to Group, df _G (d _i): in G d _i, the frequency of occurrence sites, d _i : Keyword from T, n _G : number of sites in G, v _i : importance of v _i in T, gf (d _i ): number of site groups where d _i appears, tf _max : most in T Large tf value, N _g : number of entire site groups] The method according to any one of claims 1 to 3, wherein in the third step, The modified association rule uses the concept of concentration and the calculation of the concentration, An associated keyword group extraction method calculated using the following equation. [Equation]

[df _G (r): number of sites in which r appeared in G, df _all (r): number of sites in which r appeared in total sites, r: candidate candidates for related keywords, site groups for G: d, d: Specific keyword, gf (r): number of site groups where r appeared, N _G : number of whole site groups] The method according to any one of claims 1 to 3, wherein in the third step, The modified association rule uses the concept of reliability and the calculation of the reliability, An associated keyword group extraction method calculated using the following equation. [Equation]

[df _G (r): number of sites where r appears in _G , n _G : number of sites belonging to G]

Images