TWI442249B - Domain Knowledge Network Construction Method and Its System - Google Patents

Description

Domain knowledge network construction method and system thereof

The invention relates to a network resource integration technology, in particular to a domain knowledge network (DKN) construction method and a system thereof.

With the rapid development and popularization of network technology, the amount of information on the Internet has exploded, and the information on the Internet is all-encompassing, like a huge decentralized database. When users search or browse on the Internet, there is often no specific and clear idea. In the absence of such specific concepts, it is of course impossible to obtain specific ideas or ideas through the use of the Internet.

Generally speaking, there are many relatedities between concepts, and the semantic meaning of concepts is often related to the domain. There is often the problem of "multiple meanings" or "consent words". For example, the term "Bridge" stands for different meanings in the "architectural area" and "network area"; for this problem, it is usually necessary to refer to the context of the content; therefore, the correlation between concepts is very suitable for Express your meaning clearly.

A conventional method of constructing a knowledge network, such as G. Marchionini et al., "Toward a statistical knowledge network," In Proceedings of the National Conference on Digital Government Research, pages 27-32, Boston, 2003. National Science Foundation. It is revealed that a large number of manual interventions, operations with specific user interfaces, and processing of programs are required to complete the construction of the knowledge network. The present invention seeks to find a method and system for automatically constructing a domain knowledge network by automatically extracting concepts from domain-related web pages and utilizing the association between concepts.

Accordingly, it is an object of the present invention to provide a domain knowledge network construction method.

Therefore, the domain knowledge network construction method of the present invention is suitable for execution by a system, and the method comprises the following steps:

A) obtaining a plurality of web pages related to a field;

B) dividing each web page into plural blocks;

C) identifying those belonging to an interpretation data type from the blocks of each web page;

D) performing a word class tag processing on the webpage text corresponding to the block belonging to the type of the interpretation data, assigning a word class tag on the word part of the word in the webpage text, and dividing the webpage text into plural sentences;

E) performing named entity identification on the sentence processed and segmented by the word class tag of step D);

F) performing an inter-object correlation process according to the processing results of steps D) and E) to obtain an active guest structure for each sentence;

G) According to the processing result of step F), a domain knowledge network corresponding to the field is generated.

Another object of the present invention is to provide a domain knowledge network construction system.

Therefore, the domain knowledge network construction system of the present invention comprises a webpage dividing unit, a block type identifying unit, a word class tag processing unit, a named entity identifying unit, an inter-object correlation processing unit, and a domain knowledge network. Generate a unit.

The webpage dividing unit is configured to divide each of the plurality of webpages related to a domain into a plurality of webpages. The block type identifying unit is configured to identify, from the blocks of each web page, those belonging to an interpretation data type. The part-of-speech tag processing unit is configured to perform word class tag processing on the webpage text corresponding to the block belonging to the interpretation material type, to assign a word class tag on the part of the word in the webpage text, and divide the webpage text into plural sentences. The named entity identification unit is configured to perform named entity identification on the sentence processed and segmented by the word class tag processing unit. The inter-object correlation processing unit is configured to perform an inter-object correlation process according to the processing result of the word class tag processing unit and the named entity identification unit to obtain an active guest structure of each sentence. The domain knowledge network generating unit is configured to generate a domain knowledge network corresponding to the domain according to the active guest structure of each sentence.

The above and other technical contents, features and advantages of the present invention will be apparent from the following detailed description of the preferred embodiments.

Referring to FIG. 1, a preferred embodiment of the domain knowledge network construction system 1 of the present invention comprises a webpage obtaining unit 11, a database 12 connected to the webpage obtaining unit 11, and a webpage partitioning connected to the database 12. a partitioning unit 13, a block type identifying unit 14 connected to the database 12, a word-of-speech (POS) processing unit 15 connected to the database 12, and the partitioning unit A Name Entity Recognition (NER) unit 16 connected to the database 12, an Object-Object Relation processing unit 17 connected to the database 12, and a connection with the database 12 The domain knowledge network generating unit 18 and a user interface unit 19 connected to the domain knowledge network generating unit 18.

The webpage obtaining unit 11 is configured to obtain a plurality of webpages related to a domain, and store the webpages in the database 12. The webpage dividing unit 13 is configured to divide each of the webpages related to the domain into a plurality of chunks, and store the partitioning result in the database 12. The block type identifying unit 14 is configured to identify a metadata type from the blocks and store the recognition result in the database 12. The part-of-speech tag processing unit 15 is configured to perform a part-of-speech tag process on a webpage text (text) corresponding to the block belonging to the interpret data type, to assign a word class tag (POS tags) to the word in the webpage text. The webpage text is divided into plural sentences, and the tagged results are stored in the database 12. The named entity identification unit 16 is configured to perform named entity identification on the sentence processed and segmented by the part-of-speech tag processing unit 15, and store the processing result in the database 12. The inter-object correlation processing unit 17 is configured to perform an inter-object correlation process according to the processing result of the part-of-speech tag processing unit 15 and the named entity recognizing unit 16 to obtain an active guest (subject-verb-object) of each sentence. , Subject-Verb-Object (SVO) structure, and the processing result is stored in the database 12. The domain knowledge network generating unit 18 is configured to generate a domain knowledge network corresponding to the domain according to the active guest structure of each sentence, and provide the user to the user through the user interface unit 19.

Referring to FIG. 1 and FIG. 2, a preferred embodiment of the domain knowledge network construction method of the present invention is implemented by the domain knowledge network construction system 1, which includes the following steps.

In step 301, the webpage obtaining unit 11 searches the website 2 according to the keyword input by the user to obtain a webpage related to the domain; wherein the keyword input by the user is related to the domain. In the preferred embodiment, the web page obtaining unit 11 is implemented by an existing metasearch engine, such as a Web Crawler.

In step 302, the web page dividing unit 13 divides each of the web pages obtained in step 301 into blocks. In the preferred embodiment, the webpage dividing unit 13 uses the webpage division disclosed in the master's thesis published by Dr. Lin Xuanhua, Zhu Guangbo, "Web site automatic generation system based on web block identification and hyperlink analysis". The details of the technology are not described here.

In step 303, the block type identifying unit 14 identifies the file type that belongs to the interpretation data type according to the webpage text corresponding to each of the blocks divided in step 302; wherein the block type identification unit 14 is based on each The attribute-value pairs of the corresponding webpage texts of a block are identified, and the attribute having the highest frequency of occurrence in the webpage text corresponding to the block of each webpage belongs to the interpretation data type.

In step 304, the part-of-speech tag processing unit 15 performs word class tag processing on the webpage text corresponding to the block belonging to the interpretation material type, to assign a word class tag on the part of the word in the webpage text, and then the word class tag processing unit 15 Find the boundary of the sentence in the webpage text based on the word assigned to the word class to segment the webpage text into sentences.

In the preferred embodiment, a word class tag is used as an example for a web page text of a Chinese language system. For example, a web page text "the Yankees first takes the first win of a three-game battle" to the word class tag processing unit 15 is input. The output is "Yanke (Nb (speech)" team (Na) takes the lead (D) to obtain (VC) three (Neu) Lien Chan (Nb) (DE) first (Neu) win (VH)"; The tag processing unit 15 is implemented by the Chinese Term Segmentation (CTS) system of the Chinese Knowledge and Information Processing (CKIP). For details, refer to http://godel.iis. Sinica.edu.tw/CKIP/; however, the word class tag processing unit 15 may also use other existing word class tagging tools according to the language of the webpage text, and is not limited to the preferred embodiment.

In step 305, the named entity identification unit 16 first performs a dictionary comparison on the sentence processed and segmented by the part-of-speech tag processing in step 304 based on a pre-established dictionary, and identifies the corresponding naming word from the word class as a noun. Then, for the naming fails to identify the corresponding naming by the dictionary, based on a Conditional Random Fields (CRFs) model for identification, this step is mainly used to identify and correspond to words whose words are marked as nouns. To appropriate proper names, such as "person", "organization", and "location". For details on the implementation of the conditional random domain model, see Chen et al., "Chinese Named Entity Recognition with Conditional Random Fields," Proceedings of the Fifth SIGHAN Workshop on Chinese Language Processing, pp. 118-121, 2006. Therefore, it is not described here.

It is worth mentioning that, in addition to the features used in the paper published by Chen et al., the present invention additionally adds two enhanced features to establish the conditional random domain model; wherein the enhanced The features are word boundary features and char features, respectively. For word boundary features, some specific nouns are used as the boundary of a noun phrase. For example, "title for a person", like "president" is often used As the boundary of a person's name, it helps to identify a noun phrase like "Obama President"; and "organization name", like "group" is often used As the boundary of a certain corporate group, it helps to identify a noun phrase like "Hon Hai Group". For character features, this feature is particularly useful for certain naming rules with a specific word count. For example, "person name", like Chinese first name and first name, generally character length limit Within a certain range.

In step 306, the inter-object correlation processing unit 17 performs an inter-object correlation process according to the processing result of the part-of-speech tag processing unit 15 and the named entity recognizing unit 16, to obtain an active guest structure of each sentence; If there is a name that belongs to the same compound noun, the inter-object correlation processing unit 17 first finds a compound noun to form a compound noun, and then performs an association between the objects to obtain the initiative of the sentence. Guest structure. For example, a sentence "Lee Tenghui (Nb) visit (VC) Japan (Nc) Tokyo (Nc) Yasukuni (Nb) Shrine (Nc), its structure is: Nb + VC + Nc + Nc + Nb + Nc, in which "Japan (Nc) Tokyo (Nc) Yasukuni (Nb) Shrine (Nc)" belongs to the same compound noun, so the structure of the sentence can be simplified to: Nb + VC + compound noun naming; and Lee Teng-hui (subject The relevance of the Japanese Yasukuni Shrine (object) is the visit (verb). By this step, the naming is used as the basic concept of the domain-related web page, and the verb or verb-like between the naming is used as the association between the naming.

In step 307, the domain knowledge network generating unit 18 generates a domain knowledge network corresponding to the domain according to the processing result of step 306. For example, a news page corresponding to a certain date (for example, 2007/01/01) is searched for by "Lee Tenghui", and a sentence in the webpage text corresponding to the block of the interpretation data type in the news webpage " Lee Teng-hui visited Japan's Tokyo Yasukuni Shrine, which can be used to construct and generate a domain knowledge network similar to Figure 3. The nodes in Figure 3 are named, and the connections between nodes are the association between naming. It is worth mentioning that each node can also be extracted from the domain by an important website on the Internet (for example, search engine or Wikipedia (http://www.wikipedia.org/)). Additional information to enhance the integrity of the domain knowledge network as shown in Figure 3.

In step 308, the user interface unit 19 uses the W3C (World Wide Web Consortium) Scalable Vector Graphics (SVG) specification standard to establish a graphical use of the domain knowledge network. Interface to provide and present to the user.

In summary, the domain knowledge network construction system 1 and method of the present invention can automatically extract appropriate naming as a concept from a domain-related web page, and construct a domain knowledge network by using the association between concepts. The object of the invention can be achieved.

The above is only the preferred embodiment of the present invention, and the scope of the invention is not limited thereto, that is, the simple equivalent changes and modifications made by the scope of the invention and the description of the invention are All remain within the scope of the invention patent.

1. . . Domain knowledge network construction system

11. . . Web page acquisition unit

12. . . database

13. . . Web page division unit

14. . . Block type identification unit

15. . . Word class tag processing unit

16. . . Named entity identification unit

17. . . Inter-object correlation processing unit

18. . . Domain knowledge network generation unit

19. . . User interface unit

2. . . website

301~308. . . step

1 is a block diagram showing a preferred embodiment of the domain knowledge network construction system of the present invention;

2 is a flow chart showing a preferred embodiment of a method for constructing a domain knowledge network of the present invention; and

Figure 3 is a schematic diagram showing an example of a domain knowledge network constructed in accordance with the present invention.

1. . . Domain knowledge network construction system

11. . . Web page acquisition unit

12. . . database

13. . . Web page division unit

14. . . Block type identification unit

15. . . Word class tag processing unit

16. . . Named entity identification unit

17. . . Inter-object correlation processing unit

18. . . Domain knowledge network generation unit

19. . . User interface unit

2. . . website

Claims (10)

A domain knowledge network construction method is adapted to be executed by a system, the method comprising the steps of: A) obtaining a plurality of web pages related to a field; B) dividing each web page into a plurality of blocks; C) according to each Identifying attribute-value pairs of individual webpage texts of a block to identify those belonging to an interpretation data type from the blocks of each webpage, wherein the interpretation data type is corresponding to the block of each webpage The highest frequency of occurrence of the attribute in the webpage text; D) performing word class tag processing on the webpage text corresponding to the block belonging to the interpretation data type, assigning a word class tag on the word in the webpage text, and dividing the webpage text into a plurality of sentences; E) performing a named entity identification on the sentence processed and segmented by the word class tag of step D); F) performing an inter-object correlation process according to the processing results of steps D) and E) to obtain one of each sentence actively The guest structure; and G) according to the processing result of step F), generate a domain knowledge network corresponding to the domain. According to the domain knowledge network construction method described in claim 1, wherein the step E) comprises the following sub-steps: e-1) performing dictionary comparison on the sentences based on a pre-established dictionary, and marking the words from the word class as The noun word identifies the corresponding naming; and e-2) identifies the corresponding naminger that was not identified by sub-step e-1) based on a conditional random domain model. The domain knowledge network construction method according to claim 2, wherein the feature used in the conditional random domain model includes a word boundary feature. The domain knowledge network construction method according to claim 2, wherein the feature used in the conditional random domain model comprises a character feature. According to the domain knowledge network construction method described in claim 2, in step F), the naming of the same compound noun in each sentence is also found, and then the inter-object correlation processing is performed to obtain each The active guest structure of a sentence. A domain knowledge network construction system, comprising: a webpage dividing unit, configured to divide each of a plurality of webpages related to a domain into a plurality of webpages; and a block type identifying unit for each block according to each block Identifying the attribute-value pairs of the respective corresponding webpage texts to identify those belonging to an interpretation data type from the blocks of each webpage, wherein the interpretation data type is the webpage text corresponding to the block of each webpage The word attribute occurrence frequency is the highest; a word class tag processing unit is configured to perform word class tag processing on the webpage text corresponding to the block belonging to the interpretation data type, to assign a word class tag on the word in the webpage text, and the webpage The character is divided into a plurality of sentences; a named entity identification unit is configured to perform named entity identification on the sentence processed and segmented by the word class tag processing unit; An inter-object correlation processing unit is configured to perform an inter-object correlation process according to the processing result of the word class tag processing unit and the named entity identification unit to obtain an active guest structure of each sentence; and a domain knowledge network generation The unit is configured to generate a domain knowledge network corresponding to the domain according to the active guest structure of each sentence. According to the domain knowledge network construction system described in claim 6, wherein the named entity identification unit first compares the sentences based on a pre-established dictionary, and identifies the words from the word class as a noun. The corresponding naming is then based on a conditional random domain model for identifying the corresponding naming person. The domain knowledge network construction system according to claim 7, wherein the features used in the conditional random domain model include word boundary features. The domain knowledge network construction system according to claim 7, wherein the features used in the conditional random domain model include character features. According to the domain knowledge network construction system described in claim 7, wherein the inter-object correlation processing unit further finds the naming of the same compound noun in each sentence, and then performs the correlation processing between the objects to obtain The active guest structure of each sentence.