TW201131388A - Domain metadata retrieval method and its system - Google Patents

Abstract

A domain metadata retrieval method, which includes the following steps: (A) receiving a plurality of domain-related homepages related to a domain; (B) analyzing the homepage semantics of the domain-related homepages to obtain a plurality of first metadata attributes; (C) obtaining a plurality of keyphrases from those domain-related homepages based on a statistical language model, and using the keyphrases as a plurality of second metadata attributes; (D) according to the result of the first and second metadata attributes corresponding to a set of Dublin Core Field, retrieving a plurality of domain-related metadata from the domain-related homepages; and (E) outputting the domain-related metadata to a carrier.

Description

201131388 VI. Description of the Invention: [Technical Field of the Invention] The present invention relates to a related art for data mining, in particular, a field full metadata (d〇main metadata) Method and system. [Prior Art] With the rapid development and popularization of the network technology, the information on the Internet is exploding, and the information on the Internet is all-encompassing. It is like a huge distributed database, how to be fast and correct. Finding the information you need on the Internet is a very important research topic in the current network knowledge engineering. When the user wants to search for relevant resources in a certain field or topic

δΤΙα , mouth J precedes the search site (eg, G〇ogle, Yah〇〇, Bing, 〇penfind) = enters the domain or topic related keyword (k-), and through its search engines Send keywords to search. To get > search results for webpages related to keywords. Then, the user clicks on the retrieved large webpage data to make a review and determine whether it meets his or her needs. If the retrieved webpage data does not meet the demand, the user repeatedly inserts the keyword and (4) the search result. . In general, users find the information that suits their needs. It often takes many times to do the above. It is time-consuming, inefficient, and boring. /Tian = the user's keyword is not enough. When the woman is in the search result, there is a lack of information that is actually available. 'Although the user can accumulate relevant domain knowledge in the continuous preparation process, in order to improve the situation. The second time (4) the accuracy of the keywords entered in the 201131388 domain or topic, but when the user wants to search for another new domain or topic, it still takes a lot of time to build the relevant domain knowledge. Moreover, in order to serve different users, the back-end often manages the structured data through the database system, and dynamically renders according to different information, for example, CGI (n Gateway Interface) Program; the back-end database of this kind of website is not a general-purpose search engine through the hyperpedink: the data 'ince', there are many useful hidden network information, called concealment. = Hdden Web or deep network (printed web), with existing search engines, may not be available for analysis. In view of this, the present invention is directed to a self-network (including a hidden network and a deep domain interpretation material to provide a user with a domain-related network), and extracts the corresponding collar' Provided to the network knowledge engineering phase [invention content] is to provide a field of interpretation data, therefore, the object of the present invention. Thus, the present invention is directed to a method for full data manipulation, which is adapted to include a coefficient by a method comprising: A) receiving a plurality of money related web pages related to a field; and B) analyzing web page grammar of related web pages in the fields Obtaining a plurality of first release material attributes; 〇 based on - statistical language: Second, obtaining a plurality of key pieces from the relevant web pages of the fields, and using the one-key piece -. As a plurality of second interpretation data attributes; 〇) according to the first and second (four) data attributes corresponding to the group - Dublin core set block results 201131388, from the S Xuan and other fields related to the field to extract the multiple pen field related release Information, and E) output of relevant release data in such fields to a carrier. Another object of the present invention is to provide a field interpretation data capture system. Thus, the field of data interpretation system of the present invention comprises a memory unit and a field interpretation data acquisition unit. The memory unit stores a plurality of domain related web pages associated with a field. The field interpretation data acquisition unit is connected to the memory unit; the domain interpretation data acquisition unit comprises a grammar knife analysis module, a key phrase statistics module, and an interpretation data capture module; the syntax analysis module Group of web pages used to analyze relevant web pages in these fields

The law 'to obtain a plurality of the first-(four) data attributes; the key piece-of-speech statistical model = used to be based on the -statistical language model, obtained from the relevant web pages of the fields, several key n and with the key words as the plural The second sentence fully explains the data attributes, and the data extraction module is used to calculate the results of the first and second interpretation data corresponding to the group of Dublin core set fields, and the related network 1 of the fields Take out the full-release data related to the field of the plural pen. The effect of the invention is that by making the first and second release information pairs: after searching for the relevant network in the core group of the group, the search for the related network can not only obtain hidden network or deep layer. The network: Beixun can also improve the efficiency of the search and the accuracy of the results, and the _ data found can be used to construct domain knowledge. [Embodiment] Other technical contents, features, and effects will be described in conjunction with the reference drawings, and in the detailed description of a preferred embodiment, it will be clearly shown in 201131388. (4) Drawings ^ (4) Fields (4) Data (4) System 丨 Comparative Example Include - User Interface Sheet ", with the User Interface Unit: - Domain Data Collection Unit 12, and the User Interface Unit = Domain Data Collection Unit 12 A data-item has ten 厍13, and is connected with the user's face soap 7L Π and the database 13 - 7-member domain release data acquisition unit ^ i3c m ....~J Gan Bei 8 9 Guangzhao 1U a collection result selection interface 112, - Dublin core t (Dublin^ no interface H3' and - domain (four) f material retrieval result display interface, the keyword input interface ηι, the collection result selection interface (1), ς all ^ wood Core 4 indicates interface 113, and the field _ _ _ take the result ^ No, | face m implementation of the interactive „, for the squatter to fall in the field (four) poor (four) material, system! And the pure line results hide The domain data collection unit 12 includes a webpage acquisition module (2) and a _king module 22. The database 13 is a implementation aspect of the memory unit. & 14 includes a fascination eight:: Wu group (4), a key phrase statistics module 142, and Interpreting the data capture module 143. Referring to Figures 1 and 2' corresponding to the above-mentioned field interpretation data acquisition system! The preferred embodiment of the invention [4] f material extraction method is based on the s full release data #贞, The system includes the following steps: _ When the backup user wants to search and summarize the data of a certain field, the keyword input interface ηι of the user interface 11 of the user can input at least one of the fields corresponding to the 201131388 field. For example, when the user wants to search for information related to the tourism field, the user can input "tour" as the keyword through the keyword input interface 111. In step S301, the domain data collecting unit 12 The webpage obtaining unit 121 searches for the keyword according to the keyword, and obtains the webpages related to the keyword in the website 2, and stores the webpages in the database 13. In the preferred embodiment, the webpage The obtaining unit 121 is an existing meta search engine, such as 'WebCrawler, from which the website 2 searches for and obtains the hypertext markup language associated with the keyword. Hypertext Markup Language (hereinafter referred to as HTML) webpage. Since the technology for searching and acquiring webpages by keywords is a well-known technology and is not the focus of the present invention, the implementation details are not described herein. In step S302, the domain interpreting materials The grammar analysis module 141 of the capture unit 14 analyzes the web page grammar of the web pages to obtain a Document Object Model (DOM) tree corresponding to each web page, and uses a conventional vocabulary analysis (lexical). The analysis and indexing methods establish a token and an inverted index corresponding to each web page, and store the token and the inverse index in the database 13. For details of the vocabulary analysis and indexing method, refer to "Indexing by latent Semantic Analysis, J. Amer. Soc. Info. Sci., vol. 41, pp. 391-407, 1990." by S. Deerwester et al. And "A Vector Space Model for Automatic Indexing/5 Communications of the ACM, vol. 18, no. 11, pp. 613-620, 1975." by G. Salton et al. 201131388 In step S303, the inductive disc sorting module m of the domain data collecting unit i2 first summarizes the webpages obtained in step S3G] to classify the webpages into three hierarchical granularities (3_丨evel g_laHties). Concept objects (C〇nCeptUal(10)) are conceptual objects of the "Website hierarchy", conceptual objects of the directory hierarchy, and conceptual objects of the "Website hierarchy", and then the induction and sequencing module 122 then The "Website Hierarchy", "Table of Content", and (4) objects of the "Page Level" are sorted to form the corresponding domain-domain data collection results. Among them, Step Magic 3 is further described below. First, the induction and ordering module 122 is summarized according to the following definitions. The concept of “Website Hierarchy” refers to the Uniform Resource Locator (URL) of the website homepage (h〇mepage), which is registered to the Internet Domain Name Server (Domain Name Server, below) For example, the host name indicated by DNS), for example, "http://travel.network.com.tw/" is a concept object of "website hierarchy". The concept object of "directory hierarchy" refers to a webpage that contains a plurality of domain-related webpages. In general, a website can be subdivided into several domain-related catalogues. These catalogues are conceptual objects belonging to the "directory hierarchy". For example, Children's "http://travel.network.com.tw/tourguide/" is a concept object of the "directory hierarchy". The concept of "web class" refers to a single-domain related web page. 'This single-area related web page usually contains domain-related information' or contains a web link that links to more relevant information. Then, the summary and sorting module 122 sorts the conceptual objects of the "website hierarchy", "directory hierarchy", and "webpage hierarchy" into 201131388 according to the following parameters: - cluster size, a vocabulary frequency _ Erm Frequency-Inverse Document Frequency (hereinafter referred to as TF-IDF) similarity' and an Authority and Divergence value. In the preferred embodiment, for the "Website Class" , the "information hierarchy" and the "page hierarchy" concept object, the induction and sorting module 122 is to multiply the corresponding material set, the π·similarity, and the authority and the divergence value respectively Each of them is weighted, and then summed up as a sorting score. The greater the sorting score, the earlier the order of appearance. The shai cluster size refers to the number of web pages included in the concept object; the IDF similarity is calculated by using a conventional statistical technique and matching the user-defined keywords and the mark of the web page to evaluate a certain For the details of the vocabulary for a file set or a corpus (c〇rpus), please refer to "http://en-wikipedia.org/wiki/Tf-idf" for details. Use a well-known super-chain guide to search for topics (

Hyperlink-Induced Topic Search, hereinafter referred to as HITS) algorithm, can be found in the details of r http://en_wikipedia 〇rg/wiki/mTS_ algorithm; and in this step s3〇3. -————--- The weights used in the table are as shown in Table 1. The website hierarchy directory level page level cluster size weight = 1 weight = 1 N / A authority and divergence value weight = 2 weight = 2 weight = 1 TF- IDF similarity weight = 3 weight = 3 weight = 2 In step S304, in order to make the subsequent field interpretation data acquisition result 201131388 more accurate and has a real stand-up, the user interface unit set result selection interface 112 】1 straw can be seen by the user through the receipt hm, continue to be used in the field of interpretation / to screen out after hx release section ~ multiple fields related pages, and other related web pages j 』 并将 并将 并将 并将 该 该 该 该 该 该 该 该 该 该 该 该 该 该 该 该 该 该 该 该 该 该 该 该 该 该 该 该 该 该 该 该 该 该 该 该 该 该 该 该'Winning the above steps S301~S3〇3 imaginary results The results are summarized in the way of sorting and sorting, and the data is collected in such fields. Xiao can be easy and quick. It’s worth mentioning, though,

Mmw -Γ4·, ,,,',,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, The data collection agency ("the concept of the hierarchy of the website hierarchy") is the corresponding webpage of the "category class" or "the concept of the "page hierarchy"), directly connected to you & Web page to continue the follow-up process. & π 14 get the in the step S 3 0 5, the ancient woman's first rape a birthday, this τ 4 field § full release of the poor material extraction unit

Related field pages. In step S3〇6, the domain certificate interpretation data manipulation unit 14 ten-note analysis module (4) also analyzes the webpage words of the related webpages in the fields::: Obtain a plurality of first-documentation data attributes. _ In general, because the area related to the search service in the weekly station is the following & ((10) (four) (four)) in the HTML (four) with the clause, in the preferred embodiment, the grammatical analysis loose group 141 analyzes the same The attribute_value pair in the HTML<hrm> clause of the domain-related web page to obtain the first interpreter 10 201131388 material attribute. Wherein, the relationship between the attribute and the value may be 1 to 1 (l-to-l mapping), N to 1 (N-to-1 mapping) ' or 1 to N (1-to-N mapping), the parsing module The group 141 uses only the attribute-value pairs whose relationship is 1-to-1 mapping and 1-to-N mapping to obtain the first interpretation material attributes.

In step S307, the key phrase statistics module 142 of the domain interpretation data capture unit 14 cooperates with the relevant webpages of the domain according to the tags of the related webpages established in step S302, and based on a statistical language model. The mark is obtained by obtaining a plurality of key phrases from the relevant web pages of the fields, and using the key words as the plurality of second full release data attributes. In the preferred embodiment, the n-gram language model is used for statistics. For details, refer to "N-Gram-Based Text Categorization," by WB Cavnar et al., ''Proc. 3rd Symp. On Document Analysis and Information Retrieval, pp. 161-175, 1994.", and "PAT-Tree-Based Keyword Extraction for Chinese Information Retrieval" by LF Chien, "Proc· 24th ACM SIGIR Infl Conf. Research and Development in Information Retrieval, pp. 50-58, 1997.". In step S308, the user interface unit 11 uses the Dublin Core Set Interface 113 to allow the user to map the first and second interpretation data attributes to a set of Dublin Core Fields. Among them, the Dublin Core Collection is an interpretation data format widely used in Internet resources, government publications, library collections, and museum collections. The first and second interpretation data attributes are mapped to the Dublin Core Set. The purpose of 201131388 is to enable the interpreting of the format of all types of f-interpreted data to enhance the interoperability of data between different websites; the group of Dublin core set includes 15 fields, respectively Yes · UUe, subject ' descripti〇n, coffee, _, - such as 'creator, contbutor, f01.mat, id, source, language, relation, coverage, and rights. Similarly, in the field of tourism, and the Dublin Core Set Interface 113 is shown in Figure 3, through the Dublin Core. The corresponding results after the labeling of the privacy interface 113 are shown in Table 2.矣0

12 201131388 Accommodation, restaurants, countries

It is worth mentioning that when the first website in the same field is marked by steps, other websites in the same field can be judged according to the accumulated corresponding funds; the judgment method is: doing sequence comparison or phase by word unit The number (ldentlty score) is calculated. In the library field, for example, the fake and fully-released bedding attribute is “f/title”, which corresponds to the Dublin core set, which is titled “Title”. For other websites in the same library field, it will be released to the poor. The material attribute is compared with the previously established corresponding result, and the sequence comparison or the same word score (identitysc〇re) is calculated in units of words to "automatically: the group of Dublin core set sheds. On the second day of the second day, the corresponding results are also more complete, and in the process, the human health correction can be performed in the nit condition to improve the accuracy of the corresponding money. In the steps S3G9 to S31G, the collar county data extraction unit 14 The amount of material handling 141 is firstly based on the corresponding information in the Dublin Core Collection page, which is automatically related to the domain of the field. , sex/value pairs); then, the 1 full release data is output to the carrier, eg, stored in the database 13 for subsequent construction of the field, the field is fully interpreted, and the results are not interfaced. 114 is presented to the user. No, ... According to the present invention, after the first-level 丨 corresponds to the set of Dublin core sets, the content of the third-party 13 201131388 is collided to hide the network, but the search service of each website itself can be obtained. In order to obtain the accuracy of the results, and:!! can also improve the efficiency of the search to construct a specialization ... and the comparative information obtained by the technology can be used for the domain knowledge of = so it can achieve the purpose of the present invention. The present invention is only a preferred embodiment of the present invention, and the scope of the invention is not limited to the scope of the invention, that is, the simple equivalent of the scope of the invention and the description of the invention. ;; is within the scope of the patent of the present invention. > '% still [schematic description of the drawings] i diagram ... block diagram, illustrating the field of the invention" - preferred embodiment; Flowchart' illustrates the present invention; and the method of extracting data from the field of the moon is shown in FIG. 3 as a schematic diagram illustrating a use of the _ core set designation interface. , the face of the early morning - Dublin

14 201131388 [Explanation of main component symbols] I..............Field interpretation data 121 ........... Capture system 122........ ... II .............user interface single component 13............. III ........... Word input interface 14............. Face. Capture unit 112 ........... Collection result selection 141 ........... Interface 142.............. 113 ........... Dublin Core Set Module Labeling Interface 143 ........... 114 ...... ..... Field Interpretation Data Module Capture Results Display Interface 2 .............. 12.............Field Data Collection S301-S310 · 〇〇—* Early Web page acquisition module induction and sorting model database field interpretation data grammar analysis module key phrase statistical interpretation data retrieval site steps

15

Claims (1)

201131388 VII. Patent application scope: L A field interpretation data extraction method suitable for execution by a system, the method comprises the following steps: A) receiving a plurality of domain related web pages related to a field; B) analyzing the fields The webpage grammar of the relevant webpage is obtained to obtain a plurality of first proof data attributes; θ is based on a statistical language model, and a plurality of key phrases are paid from related webpages in the fields, and the key words are used as plural numbers A full-text data attribute; D) extracting a plurality of interpret-related interpretation materials from relevant web pages in accordance with the results of the first and second interpretation data attributes corresponding to a set of Dublin Core Set fields; and E Exporting relevant interpretation data from these fields to a carrier. 2. According to the field interpretation data described in item i of the scope of application for patents: take the method 'where' in step B) +, analyze the complex attribute-value pairs corresponding to the relevant web pages in these fields to obtain the first interpretation Data attribute. 3. According to the field mentioned in the patent scope (4) data manipulation method 'where' in step C), the statistics are based on the n_gram language model to obtain the key words. (The method of extracting the data from the field mentioned in item i of the patent scope) also includes the following steps before step A): F) searching for lights in a plurality of websites according to at least the keyword related to the field, To obtain a plurality of web pages related to the keyword; G) to group each web page into a conceptual object of a website hierarchy, a concept object of a 16 201131388 directory hierarchy, and one of the conceptual objects of a web page hierarchy; Sorting the concept objects of the website hierarchy, the directory hierarchy, and the webpage hierarchy; and I) obtaining the related webpages of the domain according to the sorting result of the step Η), wherein the %•domain related webpage belongs to the webpages. 5. In accordance with the method of extracting data in the field of claim 4, wherein in step Η), any of the concept elements of the website hierarchy, the directory hierarchy, and the web page hierarchy are based on Corresponding to one of the cluster size, a vocabulary frequency-anti-file frequency similarity, and an authority and divergence value for sorting. According to the method for extracting data in the field of claim 5, wherein in the step Η), the concept object of the website hierarchy, the directory hierarchy and the webpage hierarchy is corresponding to The cluster size, the vocabulary frequency-anti-file frequency similarity, and the authority and the divergence value are respectively multiplied by the respective weights, and then a sorted integral is synthesized and sorted by the sorted integral. - an area interpretation data extraction system, comprising: » a hidden unit, storing a plurality of domain related web pages related to a field; and a mouth data interpretation unit = domain release data acquisition unit including _ grammar analysis module Group, one-off-statistical module' and an interpretation data capture module, the grammar analysis refers to the webpage grammar used to analyze relevant webpages in these fields, to obtain the first interpretation data attribute of 201131388, the key The phrase statistics module is configured to obtain a plurality of key phrases based on the statistical language model from the relevant webpages of the fields, and use the key monthly language as a plurality of second interpretation data attributes, and the interpretation data acquisition module According to the results of the first and second interpretation data attributes corresponding to a set of Dublin core set blocks, the relevant fields of the fields are extracted from the related pages of the fields. 8. 9. 10. 11. According to the field mentioned in item 7 of the application patent (4) (4), please take the system 'where' the grammar analysis module analyzes the complex attribute-value pairs corresponding to the relevant web pages in these fields to obtain such The first - interpretation of data attributes. ~ According to the scope of patent application, the full-text data extraction rate is described in the following section: 'The key documentary statistical module is based on the n_gram language model to obtain these key phrases. According to the data of the scope of the patent application, the data is also included in the data unit and the data interpretation unit of the field: the two are combined with the user interface unit. - Dublin Core Set 'No Interface' The Dublin Core Set interface user uses the first and second interpretation data attributes to correspond to the set field. As for the group of Dublin cores, according to the scope of the application for patents, the full range of data in the first paragraph of the scope of the patent application & fruit display - two users, one side also includes - field transfer information, take the picture ·丨面,, used to relate users to these fields. The full release of poverty is presented to the field of information in accordance with paragraph 1 of the scope of the patent application. It also contains a data collection unit with the memory unit and the data interpretation unit of the field: 18 12. 201131388. The data collection unit of the domain includes a webpage obtaining module, and an inductive and sorting module, wherein the webpage obtaining module is configured to perform a search on a plurality of websites according to at least one keyword related to the domain to obtain a plurality of webpages associated with the keyword, the inductive and sorting module is used to group each webpage into a conceptual object of a website hierarchy, a conceptual object of a directory hierarchy, and a conceptual object of a webpage hierarchy, and The hierarchical objects of the website, the directory hierarchy, and the conceptual objects of the web page are sorted to form a data collection result in one of the corresponding keywords. η• According to the towel, please (4) encircle the field mentioned in 12 items (4) data manipulation system, in which the concept of the website class, the directory class, and the page level is the towel--the sorting mode_ The ranking of the conceptual objects of the website hierarchy, the directory hierarchy, and the webpage hierarchy is performed according to one of its corresponding cluster size, vocabulary frequency, anti-file frequency similarity, and an authoritative and divergent value. 14. According to the application for patents _ 13 of the full release data extraction system, in which the site hierarchy, the directory hierarchy, and the webpage concept object - the 'induction and sorting module system corresponding to it The cluster size, the vocabulary frequency _ anti-file frequency similarity, and (d) multiply the divergence values by respective weights, then total __ = 'and the site hierarchy' directory hierarchy, and the page hierarchy: Other sorting points are sorted. ~ 15. According to the scope of the patent, please refer to the field of information in the 12th paragraph of the patent scope, where the user interface unit also includes the keyword wheeling interface, 19 201131388 16. For the user to enter and The domain is related to this keyword. According to the field of patent application 胄12, the field of the data is provided, wherein the user interface unit further includes a collection result selection interface for the user to select the fields from the data collection result in the field: The related webpages such as Yanhai belong to the webpages, and the user interface unit also stores related webpages in the domain in the quotation unit. 20