TWI232390B - Business search engine - Google Patents

Abstract

The information need of a business may be specific. How to find this specific information on the Internet is a new and important task that can be effectively done by a business search engine (BSE) coined in this invention. Principles for constructing such a proprietary search engine for business are suggested, including system architecture of the BSE, construction of concept databases, comparison of search intentions with contents of Web pages using an adequate method and supports for managing search results. Based on these principles, we develop a workable system employing a rigorous methodology. Since a single keyword or a logical combination of keywords cannot represent the information need of a business adequately and completely, we represent it with the help of a knowledge frame. Accordingly, we develop an adequate match algorithm using the Feature Analysis Method suggested by the applicant, to evaluate the relevance of a Web page for the business information need. After matching, relevant pages are ranked for user browsing. Interesting (relevant) Web stations are organized for handy reference. The system has a feedback mechanism: relevant pages will first be verified by human experts and, then, analyzed by computers to abstract new concepts to be added to the concept database. Assessment of the search results reveals that the system achieves high retrieval precision. Thus, it is convinced to be capable of carrying out retrieval tasks to find useful information for supporting business decision-making. The construction of a knowledge frame and the design of a match program realizing the FAM are characteristic of the system. Another feature of this invention is automatic fetch of interesting Web pages for the business.

Description

! 23239〇 V. Description of the invention (1) One ------- Field of industrial use The present invention discloses a novel comparison of web content and query concept, which sets the search accuracy rate of two enterprise-specific search engines ( precisi〇n), complete the difficult data search task delivered by the enterprise. 5 Background Generally, the General-Purpose Search Engine provides a user interface, which can input search concepts into the system to query the required data. The implementation techniques of the known search engines can be divided into four categories: (1) Directoiy-based search engines such as Yahoo and sweet potato vines.

(2) Index-based search engines such as Infoseek, AltaVista and other systems.

Is (3) Metasearchers such as MetaCrawler, SawySearch and other systems. (4) Intelligent Searchers, such as Gouge

Page 5 1232390 5. Description of the invention (2) (Google) system or other personalized search engine. The above system is probably suitable for immediate search (Ad hoc Search), but after data search, users often face the dilemma of not finding suitable data or too much data, which cannot form useful information, especially for those who want to use Searching for information as a decision support company is often in a dilemma. Know-how. Although many common search engines exist today, these search engines can also support ad hoc search for general data search, but often cannot effectively support special information needs. There are three main design methods currently used by search engines: 15 (1) keyword-based design method; (2) directory-based design method; 1232390

In addition, Wei ’s time and time again, he has watched a thorough technical axis, the present invention is different. The patent is intended to describe a method suitable for simultaneous search by multiple search engines on the Internet, which can be classified as a metasearcher. The data collection of the present invention is all done independently, without using other artificial search engines. This is the difference between the two. Object of the Invention The present invention discloses a novel bse 10 Business Search Engine. The system includes a web crawler (Web

Crawler), Match Program, Concept Database, and related websites / website databases (Database 0f Interesting Web Sites / Pages) and other components The combination of words or keywords is used to make up for sporadic fragments, and it is impossible to fully express the company's data search intentions. The general search engine comparison program is based on processing keywords (and

1232390 V. Description of the invention (4) Its combination), so it often has the disadvantages of not finding information or finding too much information. Therefore, one of the objectives of the present invention is to use a "knowledge framework" to represent the concept of corporate data collection. It is more complete and organized than conventional keywords or combinations of keywords, and can properly express the information needs of the enterprise. 5 Another object of the present invention is to disclose a "Feature Analysis Method" which is an ideal method for comparing the content of web pages with the concept of query. Those skilled in the art will undoubtedly know the objects and advantages disclosed by the present invention after reading the following detailed description of the preferred embodiment shown in the drawings. Detailed Description of the Invention 15 The business search engine (BSE) system of the present invention includes a web crawler, a match program, an enterprise-specific Concept Database, i ^ bi mm Page 8 1232390 V. Description of the Invention (5) -------- and four main components such as Database of Interesting Web Sites / pages. Among them, the enterprise-specific search engine 先, the first web crawler (from Crawler), and the matching program (Match Program) are the basic components common to all search engine, and the enterprise-specific concept database 5 is based on the "knowledge framework ( Knowledge Frame) "expresses the information demand of the enterprise. The comparison program of this system is constructed by the Feature Analysis Method according to the information recommended by the enterprise. The enterprise-specific search engine (BSE) system of the present invention is shown in Figure 1. The details of the 10-item components according to the drawings are as follows: ： InformationNeed; the special information needs of the enterprise will be abstracted after automatic analysis ( 11) Become a query concept. The method adopted by the present invention for automation is Clustering, which finds several Semantic Fields, which constitute several important facets of the demand, represented by the Hypemym. (2) Search Concepts; —A kind of information demand will be expressed by the above-mentioned knowledge frame (Frame), and each of the above words will form this frame. Page 9 1232390 V. Description of the invention (6)

Shelf attributes. The information needs represented by the knowledge framework form the query concept (also called query basis) of the present invention. (3) Internet / Web pages (Internet, Web Pages); The web page fetcher according to the universal resource locator (Uniform Resource Locator, URL) from the site in order to hold one by one web pages. ⑷Text Processor; perform preliminary processing on each * .html building's webpage, and remove tags. (5) Match Program; The comparison program scores 10 (Support) on the support of web pages and query concepts according to the "ffeaturc Analysis Method". The search and comparison of web pages will be automatically performed continuously in the background. Retrieved Pages; each page will be sorted according to its support for a particular search concept, and report results will be submitted regularly for reference by the producer. () Interesting Web sites / data; related web pages will be stored in the relevant web pages / website database. i ^ an ^ a Page 10 1232390 V. Description of the invention (7) '------ (8): Base for Search)' · Relevant web pages can be obtained after automatic analysis of each The new concept is added to the enterprise-specific concept database, so the system is endless, and the company will gain knowledge. The operation method of this system is described in detail as follows: The concept of enterprise data search in the present invention is first formed, and some related webpage templates can be provided by the company. The secretary will take the concept words (rm) contained in the cap and use the cluster method together. It is classified into several categories, and each 10-category το prime can be regarded as the meaning word Gaya below the series. Each element (concept word) has its own view, which can be determined by its appearance. Each category represents a facet of the company's data needs (Hypemym). Parents with the same meaning will also be given a weight index. The weight index can be given manually or automatically assigned by the system. The weight given automatically can be determined by the system according to the proportion of the number of members of each synonym, or by other mathematical formulas (such as Analytie Hi_hy p_s, Zhan)

1232390 V. Description of the invention (8) Seeking 'is the basis of the feature analysis method. The new feature of the present invention is to uncover a system that can collect data according to the special requirements of the enterprise. The interface provided by the system allows companies to turn their special information needs into data collection commands. Then, the collection of information becomes a regular task without manual intervention. The invention has two special features: one is that the system's query basis is from some fairly complete and organized enterprise-specific concept databases, and the query concept comparison 10 is represented by a "knowledge frame"; the second is the comparison of the invention The program is designed according to the "Feature Analysis Method" disclosed by the applicant. This query method is very different from the general search engine's use of sporadic keywords or search engines based on keywords. 15 The advantages of the present invention can be learned from the test results. The search precision of the company's exclusive search engine is quite high, which can complete the high chicken delivered by the company.

Page 12 1232390

Degree data search task. The problems to be solved by the present invention are as follows: (1) Ren-enterprise, firm, and even a certain professional researcher, the 5 degree of demand for information has its specificity, specialized Pm, and-other types of general resources δ Uncle is not considered by 4. For example, in order to protect its rights and interests, an information media company is thinking about whether its publications have been pirated on the Internet. However, using a general search engine cannot achieve the desired results; even if the search using the publication name is extremely laborious and time-consuming, it can barely find one or two. 10 (2) At present, people use ordinary search engines, which are often performed by clicking on data items or inputting words during search. However, it is difficult to accurately describe profitable keywords-some special information needs, because keywords The search concept represented by (and its logical combination) is often sporadic and incomplete. For example, 15 interested in finding websites that sell pirated optical discs on the Internet, using keywords to search, may be disappointed and lead to nothing. Therefore, it is necessary to design an appropriate expression of knowledge. 1232390 Description of the invention (10) (3) Not only does the concept of weaving search have its own special expression, the comparison of webpage content should also have a special method. Therefore, design-face Yingji Song is necessary. And in the webpage In summary, the present invention relates to a method for comparing novel webpage content and query concepts 5; its technical scope includes programming, knowledge expression, capacity evaluation methods, and the like. The information needs of enterprises have their particularities. If you want to obtain such information from the Internet, you also need a special collection method. The data collection in this application is automatic and does not require user operation. The input of the system is the knowledge in the enterprise-specific concept database, and the output is the relevant webpage with the most reference value. The working method of the present invention is as follows:

Re_ce Locator (url) retrieves a web page from the site, and the knowledge of the company ’s exclusive search engine (BSE) reading library is used to analyze the correlation between the web page and the 15 enterprise information needs, and give-weight. After recording the weight, delete the webpage, and then retrieve the other webpages of the site in sequence. The method of analyzing the content of this page is based on the "Special Page 14 1232390" disclosed by the applicant.

Feature Analysis Method ". The search and comparison of web pages will be performed continuously in the background, and the results reports will be submitted regularly. In the result report, the system will sort each web page by weight to provide corporate reference. The web pages deemed relevant by the user will be stored in the relevant webpage database 5. After the content of related webpages is automatically analyzed, new concepts may be obtained and added to the company's proprietary concept database. The system is therefore endless and the company will gain knowledge. The practicality of the present invention is that it can solve the special information needs of enterprises. Example 10: For example, a body media company wants to find online pirated publications to protect its rights. It is not easy to accomplish this task with a common search engine; even if you can barely find one or two, it is extremely laborious and time consuming. Conversely, using the system designed by the present invention, most of the work can be done almost completely automatically. 15 The "Business Search Engine (BSE)" design of the present invention. It can have the following characteristics:

Ms Page 15 1232390 V. Description of the invention (12) (1) Assist the enterprise to describe its special information needs as long as possible, complete and sometimes; (2) Provide a method to analyze the availability of data; (3) Assist the enterprise management office Collect useful information to form the company's wisdom 5 assets; record the relevant website URLs as completely as possible to facilitate subsequent search activities, and analyze the obtained data to form new information needs as the basis for subsequent searches. Technology Intrinsic, Features and Functions 10 Enterprise-specific concept database, which provides search engines with professional databases required by the company's special information needs. The present invention will describe this information requirement in the following knowledge framework:

1232390 V. Description of the invention (13)-The "alist〇ftenns (a series of concept words)" referred to in this knowledge frame forms a semantic field, with its meaning as an attribute. The 个-(concept) concept will be divided into two layers of the semantic word and the semantic field and give weight. The 5th layer of weight represents the ratio of the significance of the semantic word to the query concept. The sum of the weights of all concept words. (2) The web page extractor downloads the web pages on the website one by one according to the URL specified by the user, and adds them to the comparison program for weight evaluation. The comparison type of the present invention is designed according to the "Feature Analysis Method" disclosed by the applicant, and with the above 15 knowledge frame structure, the evaluation of a webpage for the query concept webpage can be evaluated by The cumulative weight of the webpage is measured. Cumulative weight is defined as the weight of the synonym multiplied by the sum of the weights of the conceptual words in its semantic field. Cumulative weight is at a certain gate iraHl ^ n Page 17 1232390

Above the root value, Crane Butterfly (Relevant). Algorithm for accumulating weights Suppose a query concept contains m semantic fields, and the weights of the corresponding semantics are a (m). Assuming that each semantic field is composed of n (m) concept words, the relevance of a webpage to this query concept can be calculated from its cumulative weight: (# 1) cumulative weight = 0; / * initial value * / (# 2) for i = 1 to m (# 3) semantic field W weight = 0; / * initial value * / (# 4) f〇rj = l to n (m) / * calculate the weight of the ith semantic field * / (# 5) Semantic field W weight = semantic field ⑴ weight + semantic field (i) the first j concept word weight * the number of occurrences of the jth concept word in the web page; (# 6) nextj (# 7) cumulative weight = Cumulative weight + a (i) * semantic field (i) weight; 1232390 V. Description of the invention (15) (# 8) next i; 〇 (# 9) Output cumulative weight ⑶ The relevance and URL of a web page will be recorded . After the comparison is complete, the page is deleted. The web crawler will fetch the next web page and continue to compare. Web page capture and comparison can be performed in the background, and search results are reported regularly. Users don't have to wait online. You can even start the system before work in the evening and check the results the next day at work. 15 (4) The address of a webpage judged to be relevant by the system will be automatically recorded in # 料 库 for subsequent reference. For the website to which the webpage belongs, its URL will also be recorded in the database as a basis for subsequent data search. The content of a web page that has been identified as a reference value may be analyzed to arrive at a new search concept. This search concept will be added to the corpus to form a new search basis. In this way, the system can evolve continuously and endlessly. Page 19 1232390 V. Description of the invention (16) Examples The following examples are used to explain the preliminary tests of the enterprise-specific search engine system using the present invention and other general search engines after comparison. Assume that the search target is a page that sells pirated discs. A template webpage provided by (criminal bureau) 5 experts. After analysis, a total of 879 concept words are listed. The ten most frequently used concept words and their weight values are listed. The weights in parentheses are listed in order: CD ( 10 · 19), Featured (5 · 80), Pack (5 · 79), One (3,46), Selected (3.05), Chinese (2.25), Album (2 · 15), Selected ( 2.03), VCD (2.02), optical disc (1.93) 0 Using the above ten conceptual words and their logical combinations, find a web page for "selling pirated discs" on a general search engine based on the conceptual word 1 ° As a result, the system responded to dozens of CDs, companies, CDs, or music websites, but it had nothing to do with "selling pirated CDs" because the concepts of "CD, select, film" and so on took extremely high weight.

1232390 V. Description of the invention (17) In a test set of about a webpage containing rhymes, search for the "sale of pirated discs" photo_ status difficult scale bribes. The branches are described as follows: (Group A): Using this 879 concept words and their weights, the search is based on the output rate. When the cumulative weight is 1474, the detection rate is 58 views, and the accuracy rate is α76%. In other words, each search page contains a page related to "Selling Pirated CDs". The accuracy is too low for practical use. (Group B) ·· Using the enterprise-specific search engine system of the present invention to conduct a search, the "enterprise-exclusive search secretary" Zhen Zhen will touch the 879 companies into four semantic fields. Link it. The concept words and their weight values are arranged in order in the brackets as follows: (1) · Media technology: VCD (2.02) Burn (U), Crack (0 · 25) CD (^ A ^ (2J5) CD ( 1.93), movie (U8) original (0 · 26), music (0 · 20) song (0 · 32), ··· page 21 1232390 V. invention description (18) (2). Transaction ·. Registration (ί.5), phone (0.58), subscription (0.34), order (0.33), payment (0.29), postal code (¢ 0.25), special price (0.22), website (0.19), price (0.15), transfer (0.03), • • ^ ...... a .....--'... (3) · Advertising Marketing: Featured (5.80), 丨 Selected (3.05), Chinese (2.25), Featured · 03),: ^ (1 ^^ official version (1.71), gold piece (0.79), latest (0.67), treasure (0.54), instant noodles (0.51), patch (G.10), cheap ( (0.03), ··· (4). Quantity: Pack (5.79), one (3.46), one two (1.72), one set (0.27), full set (0J3), • * * The following are the two sets of experiments Data: (Group A) Without any expert knowledge, each of the four semantic fields is given an average weight of 25%. Age 0Φ "to analyze the web page. The result is that the cumulative weight is 12 · 86 ## to the detection rate of 4 ^ 2.37 ^^ and the accuracy rate is 8 72%, that is, about every search ιμηι ηκβ -———— 1232390 "- --------— _ V. Description of the invention (19) —----- There are 12 pages containing a page about "Selling Pirated CDs" (Group B) If you enter the system again The 30 new template webpages will adjust the four semantic field weights even if they are learned through feedback: media technology 5 (22.53%), transactions (28 · 28%), advertising marketing (3083%), The number is (18 36). As a result, when the cumulative weight is 13.83, the detection rate is 47.54%, and the accuracy rate is 13/72%. "Relevant web pages. It must be noted that the weight of each concept word has changed slightly during the feedback learning process. 10 For such a difficult and intelligent search task, this accuracy rate can effectively support practical use. In summary, The invention is original, novel, and progressive. Although the invention is disclosed above with some preferred embodiments, it is not intended to limit the invention Inventions, anyone skilled in the art without departing from the spirit and scope of the invention

In addition, it can be used as a slight modification and retouch, so the scope of protection of the present invention

Page 23 1232390 V. Description of the invention (20) The scope of the attached patent application shall prevail.

Page 24 1232390 Schematic description Schematic description: Figure 1. System architecture of enterprise-specific search engine 5 Description of drawing numbers: 1 ...... • Information Need 2 ...... • Query Concept (Search Concepts) 3 ...... • Internet / Web Pages (Internet, Web Pages) ίο 4 ...... • File Processor (Text Processor) 5 ...... • Comparison Program (Match Program) 6 ...... • Retrieved Pages 7 ...... • Interesting Web Sites / Data 8 ...... • Data Collection Knowledge Base for Search 15 11 ... Abstraction 13 ... Web pages retrieved by network data extractor 15 ... Sorted after calculating cumulative weights Page 25 1232390 Illustration 16 …… · Webpage relevance test 17 …… · Extract keywords to form a new query concept 5 10

Page 26

Claims (1)

10 15 1232390 VI. Patent application scope L An enterprise-specific search engine system, which includes a Web Crawler, Match Program, Enterprise-specific Concept Database, and related websites / Database of Interesting Web 5 Sites / Pages and other four components; the characteristic of which is that the enterprise-specific concept database uses "KnowledgeFrame" to express the company's information needs; and the comparison program is based on characteristics Constructed by the Feature kiysk Method. 2. If the enterprise-specific search engine system of item 1 of the scope of patent application, the knowledge frame is the information requirement described as follows: Object_frame = {attribute 1: a list of terms; attributej: alist of terms; attribute N: a list of terms; The attribute in this knowledge framework is the facet of the query, which is a superordinate word; each superordinate word has a list of subordinate words. Page 27 1232390 VI. Scope of patent application 3. If the company's exclusive search engine system for item 1 of the patent scope is applied, the Feature Analysis Method is the basis of the design comparison program; it will be based on the webpage for query Scoring support for the concept. 5 4 · If the company's exclusive search engine system for the second item of the patent application, the facet (Clustering) method is used to find several semantic fields, which constitutes the demand. Several important aspects (Facet) are represented by the synonym (Hypernym). ίο 5. If the company's exclusive search engine system for item 3 of the patent application scope, the weight of each synonym (concept word) in the feature analysis method can be given manually; or automatically assigned by the system (such as determined by the frequency of occurrence) ); The weight of each synonym can be determined by the ratio of the number of members of the subordinate to which it belongs, or calculated by other mathematical formulas.