KR100296500B1 - An Intelligent Goods Comparison And Search Engine For Internet Shpping Mall - Google Patents

Abstract

The present invention does not require separate processing from shopping malls existing on the Internet, and automatically extracts and classifies product information from documents provided in a web form and provides an intelligent Internet shopping mall to users as a directory type service. It's about a product comparison search engine. To this end, on the Internet, the web searching means 110 for receiving document information periodically from a plurality of web containing information about the product; Information analysis means (130) for extracting commodity information and location information around the commodity information from the received document information, and comparing word types and types and types of information from the extracted information; A classification analysis means (150) for classifying the classification of the web page itself and the classification of the product information in the web page by comparing the classification with a built-in product classification database and storing the classification in the database; Intelligent search means 170 for searching the database of the classification analysis means 15 by using at least one search term input; And a web server 190 to which a plurality of users can access and input / output data on the Internet.

Description

An Intelligent Goods Comparison And Search Engine For Internet Shpping Mall}

The present invention relates to a search engine for searching for shopping goods scattered on the Internet, and more particularly, product information from documents provided in a web form without requiring a separate processing operation from shopping malls existing on the Internet. Is related to an intelligent internet shopping mall product comparison search engine that automatically extracts, classifies, and provides users as a directory-type service. Conventionally, a product search engine on the Internet simply compares and matches words existing in documents provided on the web. It was a level to judge. Therefore, when information about a product name, a price, a manufacturer, and the like are gathered together to form product information, the product search engine cannot properly determine and classify such information, thereby providing a best search service. In addition, these conventional search engines also require a considerable amount of manual work in the registration and processing of the shopping mall, product search, or have not been able to provide information smoothly due to too low recognition rate and errors.

Therefore, the present invention was derived to solve the above problems, the first object of the present invention is to search and classify the product information scattered on the Internet, and then intelligently provide a systematic shopping mall product comparison search A second object of the present invention is to provide an intelligent Internet shopping mall product comparison search engine that has a high recognition rate for product information and does not require a separate manual operation by an administrator. The web search means 110 for receiving document information periodically from a plurality of web containing information on the product on the Internet; Information analysis means for extracting the product information and the information located on the upper, lower, left, right of the product information from the received document information, and performs a word search and attribute search using the extracted product information and location information ( 130); A product classification database including a classification of the web page and a classification of the product information based on the web page information connected by the web search means 110 and the product information analyzed by the information analysis means 130; Classification analysis means 150 for updating the database after comparing and searching; And having at least one keyword, the representative word and the similar word for the search word are searched from the similar word dictionary, and the database of the classification analysis means 150 is searched based on the representative word and the similar word. Search means 170; And it can be achieved by an intelligent Internet shopping mall product comparison search engine, characterized in that it comprises a web server 190 that can be connected to a plurality of users on the Internet to input and output data. The web search means 110 Is, an HTTP client 113 capable of receiving HTML files and / or CGIs, a URL parsing unit 115 for grasping other web pages linked with the received web page, and a URL cache unit 117 for temporarily storing the URLs. Meta robot 111 consisting of; And a first mode 119 for grasping and storing the structure of the web page and the expression method of the product information from the unspecified web page, and changing the contents of the web page from the structure and the expression method of the web page stored by the first mode. A search robot 112 operated in a restart mode 123 for identifying the structure of the web page and the product information from a specific web page in which the high speed mode 121 for determining and updating the product information exists; In addition, the URL cache unit 117 of the meta-robot 111, the URL field 125, the absolute path of the URL can be stored; 'NOT' in the unsearched state, 'END' in the end of the search, and contents in which one state value of 'DATA' is stored when the product information exists and is transmitted to the search robot 112 ( contents field 129; And a search_time field 132 for storing the last searched time so that the search can be performed periodically. The search robot 112 may be an absolute structure of a URL. A URL field 133 in which a path is stored; A content field 134 for storing whether or not the web page includes product information for use in the fast mode; 'NOT' in the unsearched state, 'END' when the search is completed, and when the product information exists and the product information is transmitted to the search robot 112, the state value of one of the 'DATA' is stored ( state) field 135; And a time field 136 for storing the last searched time so that the search can be performed periodically. Further, the information analyzing means 130 may further include a database. In order to analyze the web page, it is preferable to determine whether the web page is a vertical relationship or a parallel relationship based on the HTML tag of the web page. The similar word dictionary of the search means 170 is representative. And a plurality of similar words corresponding to the representative words. Other objects, obvious advantages, and novel features of the present invention can be made from the preferred embodiments according to the following detailed description and the accompanying drawings. It will be clearer.

1 is a schematic configuration diagram of an intelligent internet shopping mall product comparison search engine according to an embodiment of the present invention;

FIG. 2 is a schematic configuration diagram of the web search system shown in FIG. 1;

3 is a data structure diagram of a URL cache unit shown in FIG. 2;

4 is a structural diagram of a database of the search robot shown in FIG.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a schematic diagram of an intelligent Internet shopping mall product comparison search engine according to an embodiment of the present invention. The overall system configuration as shown in FIG. 1 may be represented by a search engine 100 and a plurality of shopping malls 200, and manages, maintains, and maintains a plurality of users 10 and systems accessible to the search engine 100. There is an administrator 20. The search engine 100 includes a web search system 110, an information analysis system 130, a classification analysis system 150, an intelligent search system 170, and a web server 190. It consists of. Hereinafter, the configuration, function, and effect of each of the components constituting the inside of the search engine 100 will be described sequentially.

The web search system 100 automatically imports data on the web having product information, periodically updates the data and grasps new information. When the structure of the webpage changes or the structure of the homepage itself changes, it automatically checks for this and updates the data periodically, and includes a function to reduce the load by setting the update cycle automatically.

FIG. 2 is a schematic configuration diagram of the web search system shown in FIG. 1. As shown in FIG. 2, the web search system 110 is composed of two systems, a meta robot 111 and a search robot 112 using robot technology.

The meta robot 111 performs a random search of the entire web. It constructs its own DB for the entire website, and distinguishes between searched and unsearched websites. In addition, it does not have a basic web site DB initially, and simply designates a specific web site as a starting site, and continuously searches using a web-like structure of the entire web.

Because it operates as a multi-process itself, distributed processing is possible and its multi performance can be determined. While searching a web site, it determines whether or not specific information exists in connection with an information analysis system, and delivers data to a search robot when specific information exists. If the keyword corresponding to the specific information is clearly present, it may be higher than that of the entire web through the filtering function using existing search engines at home and abroad.

Hereinafter, the configuration of the meta-robot 111 will be described in detail. The meta-robot 111 is a part for parsing a URL and an HTTP client 113 capable of importing an HTML file or CGI. 115) and the portion 117 that caches the URL. The HTTP client 113 solves through a typical network programming method using C language or Java. In order to operate normally in the virtual web hosting site, the header itself is controlled and the robots.txt file, which is a protocol for excluding the web robot, is processed. Performed as an argument. In other words, the main purpose is to get an absolute URL based on the link information on the HTML TAG, save the current URL in 'current_url' and save the current relative path as 'reference_url' to make them both urls. For example, if an incorrect link is made from a specific website, it is regarded as an error. Basically, it is implemented in the same way that the absolute url is obtained from the web browser. Determining the relationship between absolute paths of two urls is implemented by using '/' to separate physical paths. This will be described in detail later in the section called URL parsers. The URL cache unit 117 prevents the robot from infinitely circulating a plurality of websites without re-searching the same website again. . The searched site is distinguished from the site which has not been searched yet, and the site in which the specific information exists is transmitted to the search robot for searching. As shown in FIG. 3, the URL 125, It consists of three fields: Contents (129) and Search_Time (132). The URL field 125 stores the absolute path, the Contents field 129 contains the content, and the Search_Time field 132 indicates the searched time. The Contents field 129 indicates' NOT ',' END ',' DATA 'can have three values,' NOT 'indicates the state that has not been searched yet,' END 'indicates that the search has been completed, and' DATA 'indicates the state where data has been sent to the search robot because certain information exists. Indicates. The Search_Time field 132 stores the last searched time and provides a search for the web site periodically. Hereinafter, the search robot 112, which is another robot existing in the web search system 110, may be used. Let's explain.

The search robot 112 searches only the site where specific information exists among the websites determined by the meta robot 111. The first mode (first mode) 119 to grasp the expression method of the specific information and the structure of the website, and the fast mode 121 to search at a high speed based on the determined structure, the structure of the website It is composed of three types of restart mode (Restart mode) 123 to identify when changed or added / deleted.

The first mode 119 is a mode used when searching for a specific site for the first time. The first mode 119 determines a structure of a website and a method of expressing specific information. Accurately separate and register only pages with specific information within the website. In addition, when there is a large amount of unnecessary information such as a bulletin board or a guest book, it is determined and excluded from the search to improve the search speed. By default it will search the entire page of your website. In addition, in order to prevent the search robot from expanding a search object outside of a specific website, a boundary is set by itself. This boundary prevents the search target from being changed to another site.

The high speed mode 121 searches based on only the information grasped in the first mode 119, so that the data can be updated at a high speed. That is, the search is performed only on the page in which the information exists, and the structure thereof is also compared based on the data previously created in the first mode 119. When the data is newly added or deleted / changed, all of them can be detected, and the difference with the existing data can be detected, so the information is perfectly imported.

There is a separate server program that periodically operates the high speed mode 121 by itself, and since it includes its own scheduling function, an interval to be periodically searched can be set. Even if you initially set the frequency as the default, search through it several times to see how often the information on your website changes. Thus, the load on the other web server can be reduced to a minimum.

After importing the data of the web page where the information exists, only the target data is accurately imported in conjunction with the information analysis system 130, which will be described later. In parallel with this process, the web server of the other side is automatically approached at intervals. . The fast mode 121 cannot detect this when the structure of the website is changed. In this case, the fast mode 121 solves this by calling the restart mode 123, which is the third mode through error processing.

The restart mode 123 may be referred to as a hybrid of the first mode 119 and the fast mode 121. By importing information only on pages where specific information exists, the entire structure of the website is identified to identify the new structure. The main difference from the first mode is that the administrator does not update the data on the information manually changed, but reflects the result of manual work. In addition, data can be updated through the high speed used in the high speed mode 121.

The search robot 112 refers to two DB tables, the structure of which will be described first. As shown in Fig. 4, in order for the search robot 112 to operate, two pieces of information need to be stored, that is, a page in which the information exists in a website and a method of storing the information. The page in which the information is present and the presentation method can be changed in the website, which can be detected in restart mode (123). However, in order to update information at a high speed in the high speed mode 121, it is necessary to store it in a cache form. This table consists of four fields: 'URL' 133, 'C0NTENT' 134, 'STATE' 135, and 'TIME' 136. The URL field 133 is an absolute path. And the CONTENT field 134 stores what information is contained. That is, it is possible to search only the page containing the specific information in the fast mode by determining whether the specific information is included or not. The STATE field 135 indicates the current state. The STATE field 135 stores whether the current search is completed, whether the search is over, or has not been searched yet, so that the search is not repeated. The TIME field 136 stores the latest time when the search is completed, and is configured to allow re-search at a specific period.

Storing the expression method is to identify and store the expression format of specific information in the information analysis system, which will be explained in the information analysis system.

Hereinafter, an operation configuration of the first mode 119 of the search robot 112 will be described in detail. That is, the first mode consists of the following steps.

(1) Target website search-Finds a website containing specific information to be searched in the first mode based on the information transmitted from the metarobot 111.

(2) Check the current state-Stores the state that the search robot 112 is currently searching for a specific website. It works in a multi-process way, avoiding duplicate searches.

(3) Deleting Existing Material-When operating in the first mode 119, there is no existing material, but the same site has a different URL or abnormally terminated in the existing first mode 119. If so, delete any information that may remain.

(4) Change STATE-Adjust STATE so that all URLs corresponding to the website can be searched. This function is not necessary when the website is first searched, but may be abnormally terminated and retried in the existing first mode 119, and thus, STATE is changed to 'NOT'.

(5) Determining URLs to Search-Recursively call unsearched URLs one by one. Whenever we search for a page, we check that the search is over, so if we simply call them one by one, we will actually search all the pages in order.

(6) Web page import-Use the HTTP client 113 to retrieve the page of the URL.

(7) Determination of the existence of information-Determine whether the imported web page contains specific information. If specific information exists, it is classified as a page with information. Otherwise, it is stored as a page without information.

(8) Information Storage-In the case of a page in which specific information exists, the specific information is extracted and stored in association with the information analysis system 130 to be described later. Instead of entering the DB directly, a temporary DB is used, which is used to recover data in case of error. After searching the entire website, copy the contents of the temporary DB to the main DB.

(9) URL parser operation-Using a program that parses URLs, it grasps all the links to other pages in the imported web page. In this way, you will search the entire website. The implementation of the URL parser will be described in detail later.

(10) URL checking-As a result of parsing, the URL is deleted except for those belonging to the boundary of the website.

(11) Check the current status-save the end of the search and change the last searched time.

(12) DB Update-Moves data from the temporary DB that was used temporarily while searching to the actual DB. This completes the registration process.

Hereinafter, the operation configuration of the high speed mode 121 of the search robot 112 will be described in detail. That is, the high speed mode 121 is composed of the following steps.

(1) Target website search-finds a website to be searched in the fast mode 121. The latest search time and the current time are compared to determine the search time.

(2) Check the current state-Stores the state that the search robot 112 is currently searching for a specific website. It works in a multi-process way, avoiding duplicate searches.

(3) Copy existing information-Since there is a function to save the changes based on the existing information, the contents are copied from the main DB to the temporary DB without deleting the existing information.

(4) Change STATE-Adjust STATE so that all URLs corresponding to the website can be searched.

(5) Determining URLs to Search-Recursively call unsearched URLs one by one. Whenever we search for a page, we check that the search is over, so if we simply call them one by one, we will actually search all the pages in order. Since the fast mode 121, the target URL is searched only for pages in which information exists in the existing URL DB.

(6) Get web page-Get the page of the URL using HTTP client.

(7) Determination of the existence of information-Determine whether the imported web page contains specific information. In the case of the fast mode, only the page in which the information exists is searched for, but the information may not exist when the structure of the website is changed. In this case, it is determined that the structure of the website is changed and the restart mode 123 is performed. The search robot 112 is operated.

(8) Information Storage-In the case of a page in which specific information exists, the specific information is extracted and stored in association with the information analysis system 13 to be described later. Instead of entering the DB directly, a temporary DB is used, which is used to recover data in case of error. After searching the entire website, copy the contents of the temporary DB to the main DB. If the same information already exists, only the change is identified and stored. In addition, when comparison with existing information is required, the DB is configured to store the changes in a separate field.

(9) Check the current status-Save the end of the search and change the last searched time.

(10) DB Update-Moves data from the temporary DB that was used temporarily while searching to the actual DB. This completes the registration process.

(11) Period setting-If the existing information and the newly searched information are very different, the search period is further reduced. Otherwise, if the existing information and the newly searched information match, the search period is further increased. As a result, it is possible to reduce the load on the web server by identifying the change cycle of the web site.

Hereinafter, the operation configuration of the restart mode 123 of the search robot 112 will be described in detail. That is, the restart mode 123 is composed of the following steps.

(1) Target website search-Based on the information delivered in the high speed mode, it determines the website to perform the restart mode. There are two cases in which the restart mode is performed, either when a change in the structure of the website is detected in the high speed mode, or when the time period for performing the restart mode has passed. The time period in which the restart mode must be performed is automatically set.

(2) Check the current state-Stores the state that the search robot 112 is currently searching for a specific website. It works in a multi-process way, avoiding duplicate searches.

(3) Copy existing information-Since there is a function to save the changes based on the existing information, the contents are copied from the main DB to the temporary DB without deleting the existing information.

(4) Change STATE-Adjust STATE so that all URLs corresponding to the website can be searched. This function is not necessary when searching the website for the first time, but it may be abnormally terminated and retried in the existing first mode 119, so all the STATE is changed to 'NOT'.

(5) Determining URLs to Search-Recursively call unsearched URLs one by one. Whenever we search for a page, we check that the search is over, so if we simply call them one by one, we will actually search all the pages in order.

(6) Web page import-Use the HTTP client 113 to retrieve the page of the URL.

(7) Identifying the structure of the web page-If there is a change in the information expression method of the web page previously identified, it is reconstructed and stored.

(8) Determining the existence of information-Determine whether the imported web page contains specific information. If specific information exists, it is classified as a page with information. Otherwise, it is stored as a page without information.

(9) Information Storage-In the case of a page in which specific information exists, the specific information is extracted and stored in association with the information analysis system 130 to be described later. Instead of entering the DB directly, a temporary DB is used, which is used to recover data in case of error. After searching the entire website, copy the contents of the temporary DB to the main DB.

(10) URL Parser Operation-Using a program that parses URLs, it grasps all the links to other pages in the imported web page. In this way, you will search the entire website. The implementation of the URL parser will be described in detail later.

(11) URL checking-As a result of parsing, the URL is deleted except for those belonging to the boundary of the website.

(12) Check the current status-save the end of the search and change the last searched time.

(13) DB Update-Moves data from the temporary DB that was used temporarily while searching to the actual DB. This completes the registration process.

(14) Period setting-If the existing web site structure and the newly identified web site structure are different, the restart search period is reduced. If the existing web site structure and the newly identified web site structure are the same, the restart search period is increased. As a rule of thumb, it can reduce the load of the web server by identifying the change cycle of the web site.

Hereinafter, the URL parser will be described in detail.

The URL parser parses based on three arguments: the current url, the website name, and the HTML file name. Based on this, the work is done in the following order.

(1) Link Tag Determination-Finds all the tags that represent links to other pages among HTML tags. This is mainly the case using the '<a href' tag and Java Script technology.

(2) Determination of link string-If there is a link in a text string, identify it. This is not necessary in the web search system, but when linked with the classification analysis system, the string itself may have a great meaning.

(3) Determining the Current URL-Determining the current URL. This is to determine the depth of the current directory, mainly around the '/' character that divides the physical structure.

(4) Link URL Analysis-Analyze link URL. The link URL may be an absolute path, may indicate a relative path, or may indicate only a file name. In case of absolute path, it can be used as it is. In case of relative path or file name, a new URL should be created by combining with the current URL.

Hereinafter, the information analysis system 130 of the search engine 100 shown in FIG. 2 will be described.

The information analysis system 130 reads external document information, analyzes the contents contained therein, and forms a consistent information system. In the current system, the information analysis system 130 is used to read documents in HTML format from external sites existing on the Internet, and to extract product-related information existing in the documents.

The information analysis system 130 reads external information and introduces human-centered thinking in processing it. That is, the information that humans most depend on when analyzing information is image information, and the feature that distinguishes image information from other information is that location information exists. By developing and applying a visual-based information recognition method that distinguishes people from each other, we can achieve similar results as humans analyze information. This concept is commonly applied to the whole system, and the parts that discriminate the structure of information by themselves or identify the structure of similar information are based on such human-centered thinking.

The information analysis system 130 has four main characteristics as a whole. These features are properties that are hard to find in a conventional analysis system, and may be the biggest factors defining the information analysis system 130. The main functions and descriptions present in the information analysis system 130 are as follows.

(1) Vision-based Information Recognition

Currently, the entire system has collected information of various contents and formats from various sites based on the Internet environment. Therefore, the information gathered from the whole system is very diverse in its kind and format, making it difficult to find any consistent composition or concept. The current computer program technology has a common problem that all steps are logically operated, and thus batch data processing is impossible for illogical or overly diverse data types. Therefore, a more special type of technology is required to solve these problems. do.

The information analysis system 130 employs a vision-based information recognition method to solve this problem. Visually-based information recognition method analyzes the information of a given document format and reconstructs it into a form that includes visual information such as up, down, left, and right together. To analyze information in such a way as to When this technique is applied, it helps to find exactly the desired information based on a wider variety of information by extending the kind of attributes that the information has from simple text-based information to visual information.

In case of using the visual-based information recognition method, it is possible to extract various information in addition to the extraction of product information like the one used in the entire system, and to grasp information on the priority and hierarchy of the information. It can have various advantages such as being applicable as an enemy.

(2) Automatic extraction of key information

One of the most important problems in dealing with a variety of information that you don't define yourself is to find out what you want. In particular, if you want to clearly extract thousands or tens of thousands of similar information based on simple information such as the whole system, the importance becomes more important. Knowing exactly what is the information you want is helping to reduce the errors that can occur in information extraction. However, to this end, up to now, it usually uses simple dictionaries or relies entirely on human help. In this case, the proportion of manual labor is greatly increased, and thus the desired level of automation cannot be achieved.

However, the information analysis system 130 has an automatic extraction function for the core information, it is possible to extract the desired information without the help of people. To this end, the information analysis system 130 utilizes a separate analyzer that analyzes according to the type and format of the information in addition to the analysis function through comparison of word units, and analyzes the information. Therefore, much wider information can be placed on the analysis target than the conventional method, and the depth and accuracy of the information is also very high. And by automating the information acquisition process, it is possible to automate the entire stage of information analysis and greatly reduce the management area of the manager.

(3) Intelligent Information Analysis

Even if you narrow down the scope of information you want, there is a significant difference from knowing exactly what you want. Even if the structural form that can represent the information is analyzed, it is still difficult to analyze all the similar information on the basis of fragmentary analysis information. In order to solve this problem, conventionally, a method of automatically extracting information only for a few cases of perfect matches is not available, so that all information cannot be obtained forever. From the perspective of search engines targeting an infinite number of pieces of information on the Internet, these fundamental constraints are a huge obstacle and must be overcome.

In order to solve such a problem, the information analysis system 130 has developed an intelligent information analysis capability of inferring even though the structure or contents of the information are not completely identical, and determining that the information is the same but different. The whole information can never exist consistently in one structure, and such forms of assumptions should not be assumed by default. The information analysis system 130 makes no assumption that any form may exist from the beginning, and is conceptually completely separated from the method of considering the number of cases. The information analysis system 130 is generally considered because it only knows the basic rules on what information is key information and how such information interacts, and does not have any kind of database related to the type of shopping mall. It is possible to analyze the information even if it is out of the situation, and to solve the problem on its own for the unpredictable situation that may occur in the future, even if the information does not exist, the judgment that the information does not exist. It can even get down, and can handle even if no information exists. Therefore, the information analysis system 130 has fundamentally solved problems that may occur in the acquisition of related information through the intelligent analysis of the information, and secured the permanence that can be inferred in the same way forever.

(4) Convenient management structure

Although the information analysis system 130 was developed independently of the management of the person with the aim of fully automatic, this does not mean that the management by the human being is not unnecessary at all. Even with intelligent software, it is natural to be more accurate with human help, and considering the possibility of unpredictable errors, human management must be fully considered and prepared.

In the information analysis system 130, it is possible to intervene management by people only at the beginning and end of information acquisition. The actual process of acquiring information completely excludes the participation of people, but only allows checking before and after the information. In this case, the manager only needs to help in advance the information related to the acquisition of the information in advance, and confirms it briefly afterwards, so that the manager is completely free from all kinds of excessive work occurring in the acquisition of the information. Through this, it is possible to drastically reduce the management cost by drawing down the number of personnel required for the management of the information analysis system 130 to one person level, and there is little administrator intervention, and the error occurrence caused by mistakes is extremely reduced.

Hereinafter, the classification analysis system 150 shown in FIG. 2 will be described.

The classification analysis system 150 is a system for automatically analyzing and grasping a classification corresponding to external data when inputting external data into its own DB including classification information. Therefore, it can be usefully used in a system that automatically inputs external data into its own DB.

When the classification analysis system 150 is applied to a website, the classification analysis system 150 includes a function of determining a classification of the web page itself and a function of determining a separate classification for each information in the page. It also includes the ability to match external and self-classifications when the category is comprehensive or detailed rather than self-classification when other web sites are present.

The classification analysis system 150 is composed of two functions: a function of determining a classification of an information group including a plurality of information, and a function of determining a corresponding classification for each piece of information. In addition, classification judgment using these two functions is possible. Each function will be described in detail as follows.

(1) Classification judgment function for information group

It is a function to judge the whole group of information as one classification. In this case, the case where the information group is included in a plurality of classifications is excluded because it is implemented as a complex function to be described later.

Before judging the classification of the information group, the correlation between the multiple information groups is identified. Rather than using a simple method of matching information and classification 1: 1 by reorganizing the relationship between information groups into logical and non-physical correlations, linking information and classification within each classification is called recursive. The approach is implemented regardless of the depth of the classification.

(3) complex classification judgment function

This is a function for judging by using two kinds of classification judgment functions described above in (1) and (2). If the classification of external information is more comprehensive than its own classification, or if the information group itself contains information corresponding to multiple classifications, this function is used to determine the classification.

The classification described in (1) is first determined, and it is determined as the upper category including the lower category without matching the classification of the lowest category. The classification is determined for each piece of information using the dictionary DB mentioned in (2). An error that does not include the classification of each information in the classification of the currently determined information group does not occur because the dictionary DB is recursively configured to apply to the information group.

Hereinafter, an implementation method of the classification analysis system 150 will be described.

The interior of the classification analysis system 150 is largely divided into two parts: (1) classification judgment for information groups and (2) classification judgment for each information. Since the matching method between the classification determined for the external information and the self classification is performed individually in each case, it is not composed of separate parts.

Each implementation method will be described based on the form of a shopping mall comparison search engine which is the current overall system. Therefore, information refers to a product being sold in a shopping mall, and the information group refers to a web page in which a plurality of products exist.

(1) Classification judgment function for information group

If there is already a classification of products for sale in a shopping mall, the program should determine the logical classification system by identifying these correlations. In this case, the characteristics of the HTML tag itself are understood and analyzed using the characteristic of web page. The classification determination function by the information group can be roughly classified into iv) a function of judging each other's relations and ii) a function of identifying a parallel structure.

Iii) the ability to determine the relationship between

In order to determine the relationship between web pages, it is basically based on HTML tags that can be linked. However, in this case, it is important to identify the web page as a top / bottom relationship because the web page does not have a physical structure in the upper / lower structure and is connected like a spider web. In order to grasp this structure, it is necessary to determine the core link among the multiple links in one page unit. The first link and the previous link are stored and reorganized. In this case, the exclusions are those connected in parallel. For example, a product page is divided into several parts and composed of 'previous' and 'next'. This parallel structure will be described again below.

In logically changing the structure of web pages connected like spider webs, we explained that it is necessary to find the core link among multiple links, but it is possible to determine whether the information exists, and based on this, only the page on which the information exists By reconfiguring the link, it is possible to find only the key link.

In the case of a link, it finds all the information deemed necessary and finds the core link based on it, which is processed recursively. That is, assuming that the classification of the upper stage is standing, and that the form is matched with its own classification structure, the classification corresponding to one product page is found.

The information needed in the connection of a general web page is a string to be connected, which is composed by comparing a separate dictionary. Called Link_dictionary, this dictionary is configured to determine the classification corresponding to the connection string, which basically has a 1: 1 match to the product classification, and has similar words as those for more detailed items. have. A separate similar dictionary is applied, which is identical to the similar dictionary used in a general search.

Even if there is no concatenated string, there is no problem in the classification judgment, because the implementation method based on the string is one of several structural identification methods.

Even if there is no information on one product page, and only a product list can be obtained, the product classification can be grasped, because there is a function to determine the classification of each product. By using this function, even if it is a 'hit product' page that includes products of several categories, the classification of products can be found accurately.

Ii) the ability to identify parallel structures

In general, when pages with product information exist in parallel, they are configured as page links. This general information can be determined. However, even if it is not in the form of such a page link, it is possible to determine the classification of a page through the function of judging the products contained in each page, so that if there are consecutive pages that are determined to be the same classification, this is a parallel structure To judge. In addition, even if it is configured in a manner that is not substantially parallel structure, since it is determined based on the information itself regardless of the physical connection, it appears as a parallel structure in the reconstructed form.

(2) Classification judgment function for each information

In order to determine the classification corresponding to each product information, a separate ITEM_D dictionary is required. This dictionary is automatically generated based on the product information that you have. If a product with the same brand name exists in different classifications, it may cause an error. This problem is not composed of the ITEM_D DB structure that matches the product name and the classification information in a one-to-one manner, but includes a self-classification. It is possible to solve this problem.

Hereinafter, the intelligent search system 170 shown in FIG. 2 will be described.

The intelligent search system 170 refers to a system for effectively searching for data in a DB having a classification structure. It is a system that analyzes the meaning of search terms and finds correlations with classifications to find only the results that users want from a large number of data.

To this end, the intelligent search system 170 is divided into three categories: (1) one word search, (2) two word search, and (3) complex word search. In each case, it also has a function to grasp the correlation with the classification structure.

(1) single word search

In the simplest case, a user searches for a word, it is divided into two categories. By referring to the dictionary of synonyms, both the representative word and the similar word for the search term are found. If there is no similar word, the search word itself becomes a representative word, and the search word itself is treated as a single similar word.

Based on the classification information, identify the classification corresponding to the representative word. In this case, the classification is divided into a case where there is a classification and a case where there is no classification. When a classification does not exist, a general similar word search is implemented. If the classification exists, the information in the classification is displayed. If the classification is a top category instead of the lowest category, the sub category is displayed, so that a more detailed classification can be selected when a comprehensive search term is entered. do.

(2) two-word search

In the case of two word search, the correlation between two words is determined. First, find representative words and similar words for each of the two words. Similar to single word search, the corresponding classification is searched based on the found representative word. The classification may or may not exist. Unlike single word search, each search word may or may not have a classification. It is divided into three rather than four because the order between two words is not important. In general, even if a word appears in detail after a comprehensive word, the correlation itself is judged again, so there is no meaning.

Iii) If no classification exists for both search terms

If both search terms do not have a classification, a general AND and OR search is performed. Since the previous work has already determined the representative words and similar words, it automatically includes the function for searching for similar words.

Ii) If only one search term exists

If only one search term exists, a classification is automatically searched. In the corresponding category, information corresponding to the other search term is found. In this case, the similarity function determined above is included.

Iii) If a classification exists for both search terms

If the classification exists in both search terms, the upper and lower relations between the two classifications are determined. Therefore, it can be divided into several cases according to the relationship between the two classifications. The first is the case where the inclusion relation between the two categories is established, and the information corresponding to the subcategory is shown.

Second, there is no inclusion relationship between the two classifications. In this case, an intra-category search corresponding to the second search term is performed in the first category, and an intra-category search corresponding to the first search term is output in the second category.

(3) compound word search

Compound word means three or more words. In the case of inputting a search word of three words or more, the search is basically performed in the same manner as the two word search. However, since it cannot be divided into three cases like two words, it is necessary to recursively reduce the search word. That is, when two or more search terms corresponding to the classification exist among the search terms and an inclusion relationship is established between the two, the search terms corresponding to the upper classification are removed. If the search term becomes less than 2 words through this process, the search function implemented in (A) and (B) is executed. If more than 3 words appear after the search term is reduced, the word 'AND' and 'OR' are searched. Output the result.

Hereinafter, before describing the method of implementing the intelligent search system 17, the synonym dictionary DB necessary for building the intelligent search system will be introduced. The implementation of a compound word of three or more words is the same as the search for two words. There is a difference between using a recursive implementation internally, so I will not mention it separately. In addition, each implementation method will be described based on the shape of a shopping mall comparison search engine which is the current overall system. Thus, classification refers to a classification that includes a product, and information refers to product information.

Since the analogy dictionary uses only a search word, its structure can be very simple. It is composed of a list of representative words and similar words corresponding to the representative word. In the case of a multiword word, since each word is judged separately, it is assumed that there is no representative word for the multiword word. Therefore, this process is performed by creating a separate function to remove spaces or other special characters.

Because the dictionary of synonyms is not fixed, but can change dynamically, you need a tool to register, modify, or delete it. In addition, in order to effectively register the similar words, the user can easily register the similar words for the words frequently searched by the user by analyzing the log of the search word searched by the user. Hereinafter, the intelligent search system 170 implements (1) Word search, (2) two word search will be described.

(1) single word search

For a single word search, the first thing to do is to find a representative word and a similar word. This process uses the synonym dictionary DB mentioned above, which can be easily solved using the SQL select statement. The processing of multiple similar words is implemented using a list tree structure.

Secondly, the product classification corresponding to the representative word is found. The reason for using only the representative word without using the similar word is that the similar word matching the product classification is registered as the representative word in the process of registering several similar words.

If the product classification exists, the product in the upper classification is shown as a result irrespective of the search word. Even if a string corresponding to the search word does not exist in the product name, the search is possible. The search order processed internally is as follows.

search for 'mouse'-> determine 'mouse' as a representative-> find 'mouse' as a synonym-> search for product categories corresponding to 'mouse'-> print products in category 'mouse'-> Search is possible even if 'mouse' does not exist.

(2) two-word search

In the case of searching for two words, the classification corresponding to each search word needs to be found.

The entire search term is separated into tokens using space as the token separator. The separated token removes the space again because there is no space in the synonym dictionary DB itself. The existence of a space already implies that there is more than one word in itself, so there is no space in the dictionary of words.

Each token that is separated is searched for a representative word and a similar word through a dictionary of similar words. This process is the same as that of the one word described in (1) above.

In both words, the representative word must exist, because the search word itself is used as the representative word for the search word that is not registered in the similar dictionary.

The representative words are used to find the corresponding classification in the classification DB. In the classification dictionary, the classification name is connected with the representative words of the similar dictionary. This is because when a new dictionary of similar words is registered, a similar word corresponding to a classification among the similar words is registered as a representative word.

When the classification is found, it is divided into three cases.

(Iii) no classification exists for both search terms

If both search terms do not have a classification, a general AND and OR search is performed. In this case, the similarity function is included. The following is implemented using SQL Query. DICTIONARY and 'keyword field' mean a field corresponding to a table of similar words and a representative word, respectively.

select * from [DICTIONARY] where [keywordField] = 'keyword'

SQL Query uses regular expressions by default, because there are many similar words. Multiple synonyms are all ORed, and many can be implemented recursively. Therefore, the entire query is implemented in the following form. DBName and model indicate a field containing a randomly designated DB table name and a brand name.

select * from [DBName] where mode regexp [prescript1 OR synonym1 OR synonym2 OR ...] AND [prescriptor1 'OR synonym1' OR synonym2 'OR ...]

In addition, in case of OR search between two search terms, SQL Query is implemented as follows.

select * from [DBName] where mode regexp [prescript1 OR synonym1 OR synonym2 OR ...] OR [prescript1 'OR synonym1' OR synonym2 'OR ...]

(Ii) if only one search term exists

If only one of the two search terms has a classification, the search is performed within the classification and includes a similar word search function by itself.

The DB containing the classification information has a classification code. By using this classification code, a SQL Query is constructed as follows.

select * from [DBName] where [classification code Field] = 'classification code' AND mode l regexp [keyword1 OR synonym1 OR synonym2 OR ...]

Each product contains information on the corresponding classification, which is stored in the "Category Code Field" above. In addition, the "classification code" means a classification code extracted from the search word.

For example, this is the case of '15 inch monitor '. Since there is a classification corresponding to' monitor ', the string 15 inches is searched for in the monitor classification. In addition, since 15 inches and 15 'exist in the synonym dictionary, a product such as a' 15 'plane' can be searched.

(Iii) a classification exists for both search terms

If a category exists in both search terms, it is divided again by the inclusion relationship between the two categories. The first case is when two classifications have inclusion relations with each other, and the second case is no inclusion relation with each other.

In the case of a search term such as 'computer mouse', a classification corresponding to 'computer' exists, a classification corresponding to 'mouse' exists, and the two classifications have an inclusion relationship with each other. In this case, the product in the 'mouse' category will be shown. Therefore, even if the string 'computer mouse' is not found in the product information, it is possible to search. Since products with completely different product information such as 'Logitech Wheel' appear in the search results, intelligent search is possible.

In the second case, there is a classification corresponding to 'computer', such as 'computer desk', and the classification corresponding to 'desk' exists in the 'furniture' category. In this case, a compound AND search is performed, which searches for 'desk' in the 'computer' category and at the same time searches for a product 'computer' in the 'desk' category. If there is a 'desk' in the 'computer' category, and a 'computer desk' in the 'desk' category, both can be shown as search results.

A simple SQL query looks like this:

select * from [DBName] where ([Category code Field] = 'Category code 1' AND mode l regexp [Representative 1 OR Synonym 1 OR Synonym 2 OR ...]) OR ([Category code Field] = 'Category code 2 'AND mode regexp [keyword1' OR synonym1 'OR synonym2' OR ...])

Compared with the example presented above, the classification code 1 corresponds to the 'computer' and the representative word 1 corresponds to the 'desk'. Classification code 2 is a classification corresponding to 'desk', and the representative word 1 is 'computer'. In this case, the similar words can be searched through the similar word dictionary.

Almost all information on the Internet is randomly registered, unorganized or unrelated, which has already exceeded the amount of information an individual can digest, making it impossible to have meaning as a collection of valuable information. In order to overcome this situation, high-level software is required to understand the meaning of information, analyze it, and provide an appropriate level of information when necessary. The related technologies used in the software will mark important milestones in the future development of Internet-related intelligent agent software by automating the three phases of the recognition, analysis and provision of such ambiguous information. Although described in connection with the examples, it is possible to make various modifications or variations without departing from the spirit and scope of the invention. Accordingly, the appended claims will cover such modifications and variations as fall within the spirit of the invention.

Claims (6)

A web search means (110) for receiving document information periodically from a plurality of webs containing information on goods on the Internet; Information analysis means for extracting the product information and the information located in the upper, lower, left, right of the product information from the received document information, and performs a word search and attribute search using the extracted product information and location information ( 130); A product classification database including a classification of the web page and a classification of the product information based on the web page information connected by the web search means 110 and the product information analyzed by the information analysis means 130; Classification analysis means 150 for updating the database after comparing and searching; And having at least one keyword, the representative word and the similar word for the search word are searched from the similar word dictionary, and the database of the classification analysis means 150 is searched based on the representative word and the similar word. Search means 170; And Intelligent Internet shopping mall product comparison search engine comprising a web server (190) that can be connected to a plurality of users on the Internet to input and output data. The method of claim 1, wherein the web search means 110, An HTTP client 113 capable of receiving HTML files and / or CGIs, a URL parsing unit 115 for grasping other web pages linked with the received web page, and a URL cache unit 117 for temporarily storing the URLs. A configured meta robot 111; And A first mode 119 for grasping and storing the structure of the web page and the expression method of the product information from an unspecified web page, and determining the change of the content of the web page from the structure and the expression method of the web page stored by the first mode A search robot 112 operated in a restart mode 123 for identifying the structure of the web page and the product information from a specific web page in which the fast mode 121 and product information exist to be updated; Intelligent Internet shopping mall product comparison search engine comprising a. The method of claim 2, The URL cache unit 117 of the meta robot 111, A URL field 125 in which an absolute path of the URL may be stored; 'NOT' in the unsearched state, 'END' in the end of the search, and contents in which one state value of 'DATA' is stored when the product information exists and is transmitted to the search robot 112 ( contents field 129; And An intelligent Internet shopping mall product comparison search engine, comprising: a database having a structure including a search time (search_time) field 132 storing a last searched time to perform a search periodically. The method of claim 2, wherein the search robot 112, A URL field 133 in which an absolute path of the URL is stored; A content field 134 for storing whether or not the web page includes product information for use in the fast mode; 'NOT' in the unsearched state, 'END' when the search is completed, and when the product information exists and the product information is transmitted to the search robot 112, the state value of one of the 'DATA' is stored ( state) field 135; And And a database having a structure including a time field (136) for storing a last searched time to perform a search periodically. The method of claim 1, The information analysis means 130, the intelligent internet shopping mall product comparison search engine, characterized in that for determining whether the web page is a vertical relationship or a parallel relationship based on the HTML tag of the web page for analysis. The method of claim 1, The similar word dictionary of the search means 170, Intelligent Internet shopping mall product comparison search engine comprising a representative word and a plurality of similar words corresponding to the representative word.