KR20010105967A - Method of web data mining using on-line web data acquisition and analysis and consulting system using the same - Google Patents

Abstract

A process for the preparation of a metal salt of clavulanic acid which comprises the reaction between an organic amine salt of clavulanic acid and a metal salt precursor compound, the reaction taking place in a liquid medium which comprises a liquid fluorinated and/or chlorinated hydrocarbon.

Description

Web data mining method using real-time web data collection and analysis and consulting system using the result {METHOD OF WEB DATA MINING USING ON-LINE WEB DATA ACQUISITION AND ANALYSIS AND CONSULTING SYSTEM USING THE SAME}

The present invention relates to a web data mining method and a method of manufacturing a web data mining solution using the results, in particular, real-time web surfing log data, server connection web log and transaction data of internet users using internet survey, web pointer software, The present invention relates to the construction of a data warehousing system and the development of a web data mining solution using real-time collection and analysis of real-time web customer evaluation data.

Existing Internet consulting firms and Internet research firms are conducting consulting and research based on materials that have not proved their objectivity and credibility, and developing data mining solutions based on only one or two simple data. Only narrow information providing services lacking objectivity and reliability are provided. In addition, the provision of real-time data using web tracking so far has a significant limitation in reflecting the propensity of all netizens in Korea in the composition of the panel, which can objectively reflect the propensity of netizens changing in real time. The system is incomplete. The existing web consulting industry, the web log analysis industry, provides a very narrow part of consulting services by constructing a data warehouse, a simple data refining system. In addition, most data mining solution providers in Korea adopt data mining algorithms derived from foreign data that do not correspond to Korea's situation, and are not based on internet data. It suggests a development method that is very remote and shows no objectivity and reliability for the data used to derive the algorithm, which is the core technology of data mining. The most vulnerable part of the existing data mining solution company is that one solution is applied to all companies, and the fact that one solution is applied to all industries without neglecting the characteristics of each industry is the objectivity of the solution. It's a way of drastically reducing clarity.

The object of the present invention is to develop a web data mining method of real-time web data and a web data mining solution using the results in order to compensate for the shortcomings of Internet business development based on the data mining solution having the limitations of the prior art as described above. And a consulting system that uses web data mining solutions. To this end, the web survey system, web tracking system, server analytics system, industry and individual product trends are used to collect and process web data in real time to provide objective and reliable web data mining solutions.

1 is a diagram illustrating an internet business consulting system according to an embodiment of the present invention.

Figure 2 is a detailed view of the Internet survey system.

3 is a diagram showing the configuration of a server computer and a client computer system among web tracking systems.

4 is a detailed diagram illustrating an implementation principle of a client system in a web tracking system.

5 is a view showing in detail the structure of the server system of the web tracking system.

Figure 6 illustrates in detail the principle of implementation of the server analytics system.

7 is a diagram showing the structure of a real-time Web customer evaluation system for identifying industry and individual product trends.

8 is a diagram illustrating the entire process of a web pointer program for implementing a web tracking system.

9 illustrates in detail the one-step process of the web pointer program.

10 is a detailed diagram of a two-step process of the web pointer program.

11 is a detailed diagram illustrating a three step process of the web pointer program.

12 illustrates in detail the four-step process of the web pointer program.

13 is a view showing the overall process of the business method of the consulting system according to an embodiment of the present invention.

14 is a detailed view of the process of the Internet survey system, which is a one-step process of the business method of FIG.

FIG. 15 illustrates in detail a method of selecting panels to be applied to the business method of FIG.

16 is a diagram showing the configuration of a web tracking system which is a two-step process of the business method of FIG.

FIG. 17 illustrates in detail the one-step process of the web tracking system. FIG.

18 illustrates in detail the two-step process of the web tracking system.

19 is a diagram showing in detail the configuration of a server analytics system that is a three-step process of the business method of FIG.

20 is a diagram showing in detail the configuration of the real-time web customer evaluation system for identifying the industry and individual product trends.

FIG. 21 is a diagram showing in detail the configuration of integrating the data converted from the above steps into a single data warehousing system through conversion; FIG.

FIG. 22 illustrates in detail the process of a web data mining solution (method) for extracting data conforming to the format of each web data mining algorithm from a data warehousing system.

FIG. 23 illustrates a process of implementing a personalized web surfing system using the web data mining solution obtained in the process of FIG. 22. FIG.

Explanation of symbols on the main parts of the drawings

100: customer system

110: individual user

120 a: Internet Survey Panel

120 b: Internet Survey Panel

120 c: Internet Survey Panel

130 a: Web tracking panel

130 b: Web Tracking Panel

130 c: Web tracking panel

140 a: Web testimonial panel

140 b: Web testimonial panel

140 c: Web testimonial panel

150: Corporate customer system

200: database system

211 a: Internet Survey Panel Profile Database

211 b: WebTracking Panel Profile Database

211 c: Web testimonials panel profile database

211 d: Database for each individual site member profile

212 a: Internet survey results database

212 b: Internet Survey Results Database

212 c: Internet Survey Results Database

212 d: General Survey Response Database

212 e: Telephone survey response database

212 f: Internet Census Database

213: Real-time Internet Usage Database

214: Website Traffic Database

215: Panel Surfing Log Database

216: server log database

217: Individual Site Surfing Database

218: Member Profile Database

219: Overall Industry Trends Database

220: transactional database

221: Purchasing Information Database

222: Site evaluation database

223 data warehouse database

300: Internet Survey System

310: Internet survey panel selection system

320: Internet survey response system

330: general survey response system

340: telephone question answering system

400: Web Tracking System

410: Web tracking panel selection system

420: Real-time Panel Internet Surfing Log Information System

430 Website Traffic Information System

500: Server Weblog System

510: Log Information System

520: Transaction Information System

600: industry and individual product trend system

610: Site Evaluation Information System

620: customer satisfaction measurement system

630: Purchase and service usage information system

640: Total Industry Information System

700: Data Conversion System

710: Internet Survey Data Conversion System

720: Web tracking log file conversion system

730: Server Weblog Conversion System

740: Individual Industry Survey Data Conversion System

750: Total DB Conversion System

760: Data Automatic Selection System

800: Web data mining system

810: Web data mining algorithm application system

900: report automatic extraction system

910: OLAP system

920: Data Mining OLAP System

930: Consulting report automatic extraction system

The web data mining solution manufacturing method according to the present invention uses a web data mining method using a collection and analysis of real-time web data, and a method of constructing a data warehousing system. A consulting system utilizing the web data mining solution developed according to the method is also provided according to the present invention.

Data mining is the process of finding useful information that is not easily revealed in huge amounts of data. For example, it may sound natural to say that homes with many windows have many thieves, but no one would have thought this could apply to insurance rates. Indeed, an insurance company in the UK uses this fact to apply differentiated insurance rates, resulting in a more effective policy, which contributes significantly to the company's profit growth.

Data Mining can be divided into definitions by computer science, MIS, and statistics. firstComputer Science PerspectiveIs defined as a series of processes that use statistical and mathematical analysis methods, as well as pattern recognition techniques, to find a variety of valuable information, such as new relationships, inclinations, and patterns, that are beneficial and interesting to us from huge data stored. MIS PerspectiveData Mining is defined throughout the development of a decision support system that enables users to use valuable information without expert knowledge, as well as a process of leaking useful information from huge databases or data.Statistics perspectiveIs defined as Data Analysis and Model Selection to support good decision making.

Finding the relationship between a window and a thief, this is Data Mining. Hidden knowledge, unexpected trends, or new knowledge based on all available source data, including, but not limited to, daily transaction data, customer data, product data, or customer response data from various marketing activities held by the company. Data Mining is about discovering rules and using them as useful information for actual business decisions. Data Mining is a conceptual methodology that searches and analyzes the source data using various methods to find unexpected information as well as information expected from it.

-Features of Data Mining

The characteristics of Data Mining can be summarized into five categories. First, it deals with large amounts of data collected unintentionally from past data accumulated in the operating system. Second, it has been put to practical use using the powerful processing power of computers. Third, most data mining techniques are empirically developed, not mathematically proven and advanced. Fourth, Data Mining's main interest is in generalizing prediction models rather than statistical inferences and tests. Fifth, it is used to apply to various decision-making activities of companies. Sixth, Data Mining began to be developed in the fields of statistics, computer science, artificial intelligence, and engineering. In practice, however, the experts who use them come from the fields of management, economics and information technology.

-Data Mining Process

When talking about data mining, it is often misunderstood that certain techniques (for example, Neural Networks, Case Based Reasoning, Decision Tree, etc.) focus on what is mining. However, Data Mining is not a specific technique such as Neural Networks or Decision Trees, but a conceptual method of information extraction and a series of related processes. Looking at the process that Data Mining is applied in, it can be divided into the following steps.

First is the problem definition phase. It determines the business problem definitions and goals you want to apply. In other words, it is necessary to fully recognize the need for data mining, to understand the current business problems, and to establish the purpose. Without understanding the exact problem, successful data mining cannot be performed. In addition, consideration should be given to the linkage with actual work on how to use the information obtained as a result of data mining.

Second is the database construction phase. It prepares the data by selecting the required data according to the defined business problem and building a data warehouse (or data mart).

Third, data mining process. In other words, the prepared data is sampled, searched through pre-analysis, transformed, and then found and evaluated for information patterns using appropriate data mining techniques. In other words, it is a task to prepare data held in proper state for data mining. Although it may vary depending on the tasks to be applied, in practice, data mining requires more than 80% of the time and effort of connecting the necessary tables and preparing the high-quality data when the data is not properly prepared. Once the target data is ready, basic information about the data can be obtained, understood and outlined through simple SQL analysis, OLAP, or some kind of pre-analysis using a variety of graphical or statistical methods. I can catch you. Based on the information in this search process, appropriate data mining methods including statistical methods are applied.

Fourth, business report preparation stage. It is a stage that is re-expressed according to the problem and purpose of the business so that users can easily understand the result obtained during the data mining process. The advanced information obtained through this process is interpreted and evaluated for its meaning and degree to determine whether it is suitable for the purpose of actual work. It will create a user environment that includes the delivery of results so that it can be re-presented in a form that can be easily understood by the actual information user or decision maker.

Fifth, it is the decision making stage. This step is used in actual work through strategy or decision-making established based on information from Data Mining.

Sixth, it is a feedback stage. It is the stage of returning to the early stage of data mining to obtain the improved information based on the result or effect after actual application. The information obtained through data mining is fed back through evaluation in actual situation, reflected in data mining, and re-analyzed to increase the reliability and accuracy of the result.

Data mining must be parallel with a business intelligence system for meaning and purpose. In other words, data mining solutions are not limited to a specific task or are simply a collection of techniques or methodologies for applying data mining. It should be possible to provide easy access to various existing source data and provide various techniques that can be usefully applied. This is because a specific technique or rule is not set to deal with a problem, but various techniques should be applicable depending on the data or the nature of the problem to be dealt with. In addition, a guideline should be provided to implement the actual work with specific mining methodology, and it should be a more comprehensive solution including the part that makes the result of Data Mining easy to understand and use. And, with the recent development of technology, various data mining techniques have been introduced, but the role of human factor in data mining is very important. In other words, judgment by people is very important in the mining process, and it is only possible for people to compare and evaluate the mined results and determine how to use them in actual work.

On the other hand, web data mining used in the present specification is to find a meaningful connection, etc. from a number of Internet-related materials (for example, surfing and Internet shopping mall purchase materials, etc.) and to softwareize the material that is embodied as such It is called web data mining solution. After all, a web data mining solution simply means a computer software package. In other words, each individual company makes a web data mining software that suits its characteristics.

Hereinafter, the present invention will be described in detail with reference to the accompanying drawings.

1 and 13, the types of real-time web data collected and used by the method or system of the present invention are (1) general Internet user information through general surveys and Internet surveys, and (2) computers of panels. (3) Real-time web surfing log data and website traffic data on internet users' trends collected using on-board web tracking software, (3) Real-time server access web log and transaction information data of specific individual internet site users, (4 ) Real-time web customer evaluation data through internet survey targeting actual internet service users of a specific internet site through information on customers and companies in real time on the website. Detailed description of each data and the process of acquiring it is as follows.

(1) Referring to Figs. 2 and 14, the Internet survey system sends a guide mail for the questionnaire to the clients so that the clients can access the Internet survey server, and when they connect and respond to the server, the response information is in the form of a file. It has a structure that is stored like a DB of panels. These server programs can run on top of the Windows NT operating system. Information from Internet survey panels is a collection of data that is stored in the form of records in the database. Use unique file names to avoid duplicate file names for database records. In addition, general surveys and telephone survey responses that do not use the Internet are also stored in the database of the server in the form of a database through a question-and-answer coding process.

3 to 5 and 16 to 18, the web tracking system is divided into a client system and a server system. First, the client system tracks a web address input in an address input window of a user web browser. Log it and send it to the server. As shown in the figure below, the server program has a structure in which the information is stored in the form of a file when messages are sent from clients. These server programs can run on top of the Linux operating system. The information from the client is in a streamable form and uses a unique file name to store this data in a file so that the file name is not duplicated. In the present invention, a web pointer software (Web Pointer Software) for implementing the web tracking system (a client program mounted on the client system for implementing the web tracking system, which is referred to as a web pointer program or a web pointer software in the present specification and drawings). Is used). 8 to 12, Web Pointer Software is responsible for collecting the real-time web surfing URL log files and sending the log files to the server regarding the web surfing trend of Internet users. Same as

The first step for the execution of the program is for users to download the web pointer software from the homepage site of the server system and install it on each panel's computer. By automatically registering the startup program to reduce the inconvenience of having to run the program every time the user accesses the Internet, the program can be exited if desired. The second step is to catch the URL value starting with http: // by hooking the window message of the browser's URL window when the user connects to the Internet using a web browser. The third step is to save the captured URL value as a log file on the user's computer. If the old log file exists on the user's computer, the existing log file continues to accumulate. If the old log file does not exist This step saves the URL value by creating a new log file. The fourth step is to transfer the log files stored in each user's computer to the server system. If the size of the log file stored in the user's computer exceeds 1 k bytes, the stored log files are transferred to the server system. Deleting files.

Referring to FIG. 15, in order to overcome the problem of representativeness of the panels used in the present business method, first, a panel of 10,000 people who fit the demographic variable ratio of the Korean people based on the statistics of the Korea National Statistical Office was selected through a telephone survey, and among them, the Internet was selected. Make sure you know the percentage of demographic variables that netizens use. Next, based on the ratio of demographic variables of these netizens, 2,000 web tracking panels were selected separately and the web tracking software was installed on each panel's personal computer with the panel's agreement. Overcome

(3) Referring to Figs. 6 and 19, server analytics includes a log of the panel's movement paths and sites along with a database of panels that were recorded before Internet users access a particular server. In addition, all the transaction information databases that have been traded are integrated and stored in the database. This web server log analysis step is to grasp the real-time server access web log files and actual transactions of specific site Internet users and store the information in a database. If the users who access the server of each Internet site are the panel of the site, the panel profile database created when joining the panel is used. If the panel is not the panel, the server is registered after using the panel form that meets the member requirements of each company. Surfing and transactions take place after accessing. The surf data is stored in each server in the form of log files, and the log files are saved in the database through a log cleaning process. In addition, the actual transaction details are stored in the transaction history DB. These server programs depend on the operating system of each server system and are usually run on Linux and Windows NT operating systems.

(4) Referring to FIG. 20, the industrial and individual product trend system sends a site-based questionnaire to the site users to measure the satisfaction with the visited site and the satisfaction with the purchased product, and store the information in a database. to be. When a user purchases a product over the Internet, a questionnaire is sent to the customer to evaluate the site where the purchase was made immediately after the purchase of the product, and the result of the survey is received via the Internet and stored in a corresponding database. When the customer reaches the customer and uses the product, information about the customer's satisfaction and importance of the product is collected in a separate questionnaire response database, and log data about the user's purchase and service usage information DB for purchase and service usage information In this step, the information on each site is clustered by industry, and the information on each industry trend is stored in the database. When customers use the products and services of the site, consumers respond to the evaluation of the site and the purchased products through the Internet survey, and this response is stored in the database of the service company's server in the form of an e-mail. . These server programs run on the Windows NT operating system. The information conveyed from the Internet Survey Panels is in a continuous format and uses a unique file name to store this data in a file so that the file name is not duplicated.

The next step is to perform data conversion so that all real-time Internet information collected through these steps can be integrated and stored in the dataware database. Referring to FIG. 21, in this step, an basic analytical processing (OLAP) function using basic frequency analysis is performed.

Referring to FIG. 22, in the final step, each web data mining algorithm passes through a data selection system that extracts only the data required by each data mining algorithm from the real-time Internet information stored in the dataware database through data conversion. This step is to produce web data mining solution. Through this step, as shown in Fig. 23, a personalized web surfing system comes out.

Meanwhile, the data model used in this business method is as follows.

Database 1

(a) Name: Panel Profile Database (b) Stored Information: Panel ID, Password, Name, Age, Gender, Social Security Number, Address 1, Address 2, Phone Number, Mail Address, Education, Occupation, Marital Status, Income level, interests, internet usage time (c) other: information is stored in a database on the server computer for each individual

Database 2

(a) Name: Survey Results Database (b) Stored Information; Panel ID, panel demographic information, survey response (c) Others: Panel survey responses are stored in the server's database via email.

Database 3

(a) name: panel surfing log database (b) storage information; Panel ID, access host name, access page, access time, browser used (c) Others: Save panel surfing URL information to log server database

Database 4

(a) Name: Server log database (b) Stored information: Site name, site type (shopping mall, auction, etc.) (c) Other: Stores site name and information about the site type

Database 5

(a) Name: Member Profile Database (b) Stored Information: Member ID, Password, Name, Age, Gender, Social Security Number, Address 1, Address 2, Phone Number, Mail Address, Education Level, Occupation, Marital Status, Income levels, interests, time spent on the Internet (c) Others: Information is stored in a database on the server computer for each individual.

Database 6

(a) name: transaction database (b) storage information; Member ID, Member Address, Member Contact, Member Email, Purchase Item, Purchase Quantity, Transaction Date, Transaction Amount, Payment Method (c) Other: Depends on the transaction database structure of each site

Database 7

(a) Name: log information database (b) Stored information; Member ID, Intra-Site Surfing Path Log File (c) Other: Depends on the log information database structure of each site

Database 8

(a) Name: Site Assessment Database (b) Stored Information; Membership ID, Evaluation Questionnaire Response File (c) Others: Survey response database that was sent to the panel divided into 1st and 2nd

Database 9

(a) Name: Dataware Housing Database (b) Stored Information: Member ID, Member's Age, Address, Education Level, Occupation, Marital Status, Income Distribution, Interests, Purchase Items, Purchase Date, Purchase Quantity, Payment Method, (C) A database that has been converted to a numerical form, which can be entered into other data mining algorithms. Data warehouse database in which data mining algorithms can be applied

Although the present invention has been described with reference to the above embodiments, the present invention is not limited thereto. Those skilled in the art will appreciate that modifications, changes, and the like may be made without departing from the spirit and scope of the present invention, and such modifications and changes also belong to the present invention.

According to the present invention, a web data mining solution using real-time web data can develop a unique web data mining solution for a corresponding company by using real-time web data for each industry. Therefore, by applying additional web data mining algorithms to meet the rapidly changing Internet industry trend as a single solution development, the company develops a practical web data mining solution for each industry in Korea and provides consulting services. To provide domestic companies with global competitiveness.

Claims (1)

Obtaining full Internet user information through general surveys and Internet surveys; Acquiring real-time web surfing log data and website traffic data relating to internet usage trends of Internet users collected using panel-mounted web tracking software; Obtaining real-time server access web logs and transaction information data of specific individual internet site users; Acquiring real-time web customer evaluation data through internet survey targeting actual Internet service users of a specific internet site through information on customers and companies in real time on the website, Converting the obtained data to be combined and stored in a data warehouse database; A data selection step of extracting only data required by the data mining step among the stored data; A method of obtaining a web data mining solution comprising data mining the data.