KR101404644B1 - Apparatus and method for expanding knowledge base - Google Patents

Abstract

The present invention relates to a related document collecting unit for collecting data related to the category in each category of a hierarchical subject classification directory and generating related documents; A verb extraction unit for extracting and refining candidate verbs from the related document; A relevance evaluator for selecting one or more verbs having high relevance to the category among the extracted and refined candidate verbs; And an extension unit for integrating the selected verbs into the subject classification directory data.

Description

[0001] APPARATUS AND METHOD FOR EXPANDING KNOWLEDGE BASE [0002]

The present invention relates to a knowledge base extension apparatus and method.

Recently, a personalized service that provides a customized service for each user has become popular. Since personalized information service provides appropriate service for each user who uses service even under the same situation, user satisfaction and service quality are higher than general service without personalization.

For example, in the case of a personalized news service, a service is provided to increase user satisfaction by recommending news that the user may be interested in before requesting the user. As another example, a personalized search engine returns different results to each user when two users of different interests request the same query.

In most cases, user queries are often short and ambiguous, making it difficult to accurately represent users' intentions. For example, the question 'Café' means 'online community' and 'coffee and beverages'. Therefore, what is most important in personalized information services is to infer user intentions.

In order to infer the user 's intention, a method of expressing the user' s interests in a layered category, such as Open Directory Project (ODP), has been studied. The open source ODP is a hierarchical web directory designed to organize web sites into thematic categories, managing over one million directories and over 4.9 million web sites in a tree format. ODP is a useful source of user interest in terms of size and quality.

The user's intention is expressed in the ODP category, which means that the user's intention is expressed in the form of a noun. This is a suitable method for expressing a user's intention by subject, but is not sufficient for expressing a user's intention more clearly. For example, if the intention of a user is represented by the ODP category "cafe", which is a noun form, the personalized search engine deduces that the intention of the user is not an 'online community' but a place to 'sell coffee and drinks' You can return the right results, but what you want is to find the right cafe to set the appointment place, to search for the appropriate franchise brand to start a business, There is a limitation that it can not be inferred whether or not it is for searching.

Therefore, it is necessary to expand knowledge base such as ODP to provide personalization service more effectively.

Korean Patent No. 10-0341396 ("3-D Clustering Generation System and Method Therefor ", relating to personalization service) discloses a configuration combining a hierarchical word concept with a similarity-based clustering technique.

Korean Patent Laid-Open No. 10-2011-0052821 ("a topic classification module and a contextual advertisement system using the same") discloses a configuration in which a context advertisement is provided using an ODP.

The object of the present invention is to provide a knowledge base extension apparatus and method for expanding a subject classification directory so as to express a user's interest more effectively.

According to a first aspect of the present invention, there is provided a knowledge base expanding apparatus comprising: a related document collecting unit for collecting data related to the category for each category of a hierarchical subject classification directory and generating a related document; A verb extraction unit for extracting and refining candidate verbs from the related document; A relevance evaluator for selecting one or more verbs having high relevance to the category among the extracted and refined candidate verbs; And an extension unit for integrating the selected verb into the subject classification directory data.

According to a second aspect of the present invention, there is provided a knowledge base expanding method comprising the steps of: (a) collecting data related to the category for each category of a hierarchical subject classification directory to generate a related document; (b) extracting and refining the candidate verb from the related document; (c) selecting one or more verbs having high relevance to the category among the extracted and refined candidate verbs; And (d) incorporating the selected verb into the subject classification directory data, wherein the subject classification directory is an Open Directory Project (ODP).

The present invention obtains the effect of expanding a subject classification directory to more effectively express a user's interest in a knowledge base extension apparatus and method.

Specifically, a user's interest can be represented by a combination of a noun and an associated verb.

Therefore, since the user interest inference performance is improved, a further improved personalization service can be provided.

In addition, it extracts the related verbs efficiently by using various external sources, so that the accuracy of the integrated verbs extracted is high.

FIG. 1 shows a structure of a knowledge base extending apparatus according to the present invention.
Figure 2 shows an embodiment of ODP.
Figure 3 illustrates an embodiment of an extended ODP according to the present invention.
FIG. 4 shows a flow of a knowledge base expansion method according to the present invention.
5 shows the flow of the related document collection step according to the present invention.
FIG. 6 shows a flow of a candidate verb extraction step according to the present invention.
FIG. 7 shows a flow of a highly related verb selection step according to the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings, which will be readily apparent to those skilled in the art. The present invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. In order to clearly illustrate the present invention, parts not related to the description are omitted, and similar parts are denoted by like reference characters throughout the specification.

Throughout the specification, when a part is referred to as being "connected" to another part, it includes not only "directly connected" but also "electrically connected" with another part in between . Also, when an element is referred to as "comprising ", it means that it can include other elements as well, without departing from the other elements unless specifically stated otherwise.

1 is a block diagram showing a knowledge base expansion apparatus 10 according to the present invention.

The knowledge base extension device 10 according to the present invention extends the knowledge base to provide enhanced personalization services as described above. The knowledge base is classified according to the topic of interest of each user, and is expressed in the form of a subject classification directory having a hierarchical structure. It is an object of the knowledge base expanding apparatus 10 according to the present invention that a knowledge base expressed in the form of a noun is extended to include an associated verb to more effectively express a user's interest.

To this end, the knowledge base extension apparatus 10 according to the present invention includes a related document collection unit 100 for collecting data related to each category of a subject classification directory and generating related documents, a verb An associating unit 300 for selecting one or more verbs having high relevance to the category among the extracted candidate verbs and an expanding unit 400 for incorporating the selected verbs into the subject classification directory data .

Also, in one embodiment, the knowledge base extension apparatus 10 may further include an application unit (not shown) for providing a personalized service using the extended subject classification directory. Or the knowledge base extension device may be included and used in the user's intention reasoning module of the personalized service device.

In a preferred embodiment, the subject classification directory uses Open Directory Project (ODP). ODP is an open directory where users voluntarily participate and collect links of world wide web sites in a hierarchical ontology form. It is also called Dmoz because the original domain name was directory.mozilla.org. The current ODP site address is http://www.dmoz.org.

An embodiment of the ODP is shown in Fig.

In the present embodiment, the user can select Computer> Company> Apple Inc. The ODP directory was searched in order. The query 'apple' could be interpreted as an 'IT company named Apple' or it could mean apple fruit. Figure 2 shows how this user's interest is expressed in a subject classification directory such as ODP. In this embodiment, when the user selects Computer> Company> Apple Inc. Another user browses the ODP directory in the order of Shopping> Food> Fruit> Apple.

At this time, as described above, the personalized search engine uses the subject classification directory, and even if the user simply inputs the query 'apple', the information on the company, Apple Company, , And another user can return a search result with information about the apples that are fruits. The similarity calculation is used for the user's intention reasoning. For example, the similarity score for the user shown in FIG. Is higher than 0.2 on Shopping> Food> Fruit> Apple side at 0.86, so Apple Inc. will return the search results shown above.

Figure 3 illustrates an embodiment of an extended ODP according to the present invention.

 2, the user entered the query 'apple' because he or she wanted to download content such as iTunes, to purchase a product such as an iPhone, or to the CEO You might want to find information about Steve Jobs. For each case, you can extract 'download', 'buy', and 'search' as an associative verb for 'Apple Inc.'.

Extending the computation of similarity to an associative verb can provide a more personalized service by more accurately inferring a user's intention. In the case of the user shown in FIG. 3, the similarity score of (apple, download) is 0.67, the similarity score of (apple, buy) is 0.78, and the similarity score of (apple, search) is 0.23. So this user is interested in purchasing Apple's products. Therefore, the personalized search engine can display the information about the product among the information about the Apple.

FIG. 4 shows a flow of a knowledge base extension method according to the present invention.

The related data is collected for each category of the subject classification directory of the hierarchical structure to generate a related document (S510). The candidate verbs are extracted and refined in the generated related document (S520). One or more verbs having high relevance to the category among the extracted and refined candidate verbs are selected (S530), and the selected verbs are classified into the subject classification directory data (S540).

Let us examine the relevant document generation step (S510) through FIG. 5 in more detail.

The knowledge base extension method according to the present invention generates a document including a large number of verbs related to categories by using various external sources. That is, the data associated with a given category from various sources is collected and integrated into one document. It is important to look for data that may contain verbs that have a high relevance to the category, so it uses a variety of sources such as search results pages, web pages classified in the ODP, and search logs.

A first example of an information source utilized by the knowledge base extension method according to the present invention to generate a related document is title and description data of web pages classified in the ODP (S610). Because Web pages classified in the ODP are pages associated with each category, they are a good source of information because they contain many related verbs. At this time, the web pages classified into the subcategories may be utilized together in case that the number of the web pages classified into the lower level is small.

A second example of an information source utilized by the knowledge base extension method according to the present invention to generate a related document is the title and summary data of the web pages included in the search result page returned by the search engine when the category is used as a query (S620). The search result page is a list of the web pages returned by the search engine for the query, and returns appropriate and large amount of data for any query, so if you use the category as a query, you can collect web page information containing many related verbs .

A third example of the information source utilized by the knowledge base extension method according to the present invention to generate a related document is the title and summary data of web pages included in the search log (S630). The search log is a record of usage patterns of web users such as a query used by a user, a clicked URL, a clicked time, and the like. The search log includes information on web pages that are frequently clicked by users. Search logs are a useful external source because they contain many verbs.

Specifically, in order to effectively extract information from a search log, a knowledge base extension method according to the present invention generates temporary documents using the title, URL, and summary information of clicked web pages on queries having a high degree of similarity with a directory Then, the degree of similarity with the provisional document is measured for a given ODP category, the related query is ranked, and the provisional document for the ranked queries is integrated to generate one document.

When the information collection is completed, the collected information is integrated into one related document (S640). At this time, the order of performing the information collecting steps (S610, S620, S630) is not important, and the information source utilized by the knowledge base extension method according to the present invention to generate the related document is limited to the three examples It does not.

FIG. 6 shows a step (S520) of extracting candidate verbs from the generated related document.

First, a sentence including the category name is extracted from the generated related document (S710), and the sentence is divided into tokens of word units (S720). For example, if there is a sentence called 'Apple designs and creates iPod and iTunes' that includes the category name 'apple' in the related document, then 'Apple', 'designs', 'and' ',' iTunes' tokens are extracted.

Next, each token is tagged, and only the token tagged with the verb is left (S730). As a tool for this, the knowledge base extension method according to the present invention can use the Stanford Natural Language Processing Tool. For example, in the above example, only the 'designs' and 'creates' tokens remain.

Next, the same verbs of various forms are recognized as one verb, leaving only the root of each token (S740). As a tool for this, the knowledge base extension method according to the present invention can use a potter stemmer. For example, 'designs' and 'designing' leave only the root 'design'.

Next, in step S750, the previously-designated abbreviations are removed from the extracted verbs. An abbreviation is a verb which appears frequently irrespective of category, and means a verb that is not meaningful in extracting an association verb such as 'is' or 'are'.

Let us examine the more relevant verb selection step (S530) through FIG.

In this step, the relevance score is calculated to select the verb to be used in the knowledge base expansion of the candidate verbs. The relevance score means how much the candidate verb is related to a given category. The higher the frequency of occurrence of the verb in the document, the higher the score is given as the distance between the verb and the category name in the sentence is shorter.

Accordingly, the average distance between the candidate verbs and the category name is calculated (S810), the normalized occurrence frequency in the document of the candidate verb is calculated (S820), and the relevance of the candidate verb is calculated based on the two information S830).

Specifically, the relevance score between the candidate verb and the category is calculated using the following equations.

Where vi is the i-th verb, and c is the category name. nvf (vi) is the normalized occurrence frequency of the candidate verb in the document, and avd (vi, c) is the average distance in the sentence between the candidate verb and the category name.

vf (vi) means the occurrence frequency of the verb, and V means the entire set of verbs extracted from the document. That is, nvf (vi) means a value obtained by normalizing the occurrence frequency of the verb to the sum of occurrence frequencies of all verbs occurring in the document.

Loc (vi) represents the location set of the verb vi. This is a function to obtain each position value because the same verb appears multiple times in the document. dist (l, c) is a function to obtain the distance value between the verb and the directory name.

When all the candidate verbs are calculated, a certain number of candidate verbs having a high relevance score are selected and designated as an associated verb of the category (S840).

It will be understood by those skilled in the art that the foregoing description of the present invention is for illustrative purposes only and that those of ordinary skill in the art can readily understand that various changes and modifications may be made without departing from the spirit or essential characteristics of the present invention. will be. It is therefore to be understood that the above-described embodiments are illustrative in all aspects and not restrictive. For example, each component described as a single entity may be distributed and implemented, and components described as being distributed may also be implemented in a combined form.

It is intended that the present invention covers the modifications and variations of this invention provided they come within the scope of the appended claims and their equivalents. .

10: Knowledge base extension device
100: Related document collection section
200: verb extraction unit
300: Association evaluation unit
400: Extension part

Claims (11)

A knowledge base extension apparatus comprising:
A related document collecting unit for collecting data related to the category for each category of the hierarchical structure classification directory to generate a related document;
A verb extraction unit for extracting and refining candidate verbs from the related document;
A relevance evaluator for selecting one or more verbs higher than the candidate verbs having a score of relevance to the category among the extracted and refined candidate verbs; And
And an extension for integrating the selected verb into the subject classification directory data. The method according to claim 1,
Wherein the knowledge base extension apparatus further comprises an application unit for providing a personalization service using the extended subject classification directory. The method according to claim 1,
Wherein the subject classification directory is open directory data (ODP). The method according to claim 1,
The related document collection unit collects the title and summary data of web pages in the search result page returned by the search engine for the user's query, collects title and summary data of the web pages in the search log, A knowledge base extension device for collecting title and description data of classified web pages. The method according to claim 1,
Wherein the verb extractor extracts only a verb among words constituting a sentence including the name of the category in the related document and then extracts the candidate verb by purifying the verb to be divided only into a root, A knowledge base extension device. The method according to claim 1,
The relevance evaluating unit may determine that the higher the occurrence frequency of the candidate verb in the related document, or the higher the relevance score of the candidate verb is, the shorter the distance between the candidate verb and the category is in a sentence including the category name Knowledge base extension device. The method according to claim 1,
The association evaluating unit
Using the position value of the verb in the document, we derive the mean distance between the candidate verb and the category name,
After deriving the normalized frequency of occurrences of the candidate verb in the document using the distance value between the verb and the category name and the occurrence frequency of the verb,
And calculates the association between the candidate verb and the name of the category based on the derived average distance and the occurrence frequency. In a knowledge base extension method,
(a) collecting data associated with the category for each category of a hierarchical subject classification directory and generating a related document;
(b) extracting and refining the candidate verb from the related document;
(c) selecting one or more verbs higher than the candidate verbs having a score of relevance to the category among the extracted and refined candidate verbs; And
(d) incorporating the selected verb into the subject classification directory data,
Wherein the subject classification directory is open directory data (ODP). 9. The method of claim 8,
The step (a)
Collecting title and description data of web pages classified in the published directory data;
Collecting title and summary data of web pages on a search result page returned by the search engine for the user's query; And
And collecting title and summary data of web pages in a search log. 9. The method of claim 8,
The step (b)
Extracting only the verb among the words constituting the sentence including the name of the category in the related document, extracting the candidate verb such that the verb is divided into only the root and separating the verb from the candidate verb, Base extension method. 9. The method of claim 8,
The step (c)
Using the position value of the verb in the document, we derive the mean distance between the candidate verb and the category name,
After deriving the normalized frequency of occurrences of the candidate verb in the document using the distance value between the verb and the category name and the occurrence frequency of the verb,
Wherein the association between the candidate verb and the name of the category is calculated based on the derived average distance and the occurrence frequency.

Images