KR20040026178A - Data Mining Method and Data Mining System - Google Patents

Abstract

PURPOSE: A data mining system and a method thereof are provided to quickly obtain a frequent item-set or a frequent sequential pattern by analyzing a non-restricted data set comprising transactions continuously generated according to the flow of time through an opened data mining method. CONSTITUTION: A data server module(10) manages the transaction data continuously generated from an application domain by a database system or a file system. A data mining kernel(20) finding a mining result by analyzing a data set comprises a transaction data read module(21) reading the data of the database server module and a data mining module(22) finding out the mining result by analyzing the frequency information of the items appeared in the transaction. A mining result present module(30) presents the result get from the data mining kernel to the user. The mining result is stored in a storage(40) and the stored data mining result is provided to the user from the data mining present module.

Description

Data Mining Method and Data Mining System

The present invention relates to a data mining method for analyzing feature sets in a large data set to find feature information, a data mining system implemented using the method, and a method thereof. In more detail, new data can be continuously generated over time, and the correlation between the data contained in the non-limiting data set continuously including the newly generated data can be analyzed in real time at each change point of the data set. The present invention relates to a data mining method and a method of searching for frequent itemset and frequent sequential patterns in real time, a data mining system implemented by applying the same, and a method thereof.

In general, in a data set that is the subject of data mining, all unit information that appears in an application field is defined as an item, and a collection of unit information having semantic concurrency (that is, semantically occurring with each other) in the application field. Is defined as a transaction. A transaction has information of unit items with semantic concurrency, and the data set to be analyzed for data mining is defined as a set of transactions generated in the corresponding application field.

In the data mining method, the conventional technology is basically designed to define a data set that is a mining target before starting a mining task and to be effectively performed when basic preprocessing for the data set is possible. That is, the conventional method has a purpose to define a fixed data set and extract the knowledge existing in the fixed data set at the time of data mining analysis. These conventional techniques can extract only the knowledge generated at a specific time since the data set is fixedly defined. However, as time passes, the knowledge implied in newly created transactions can change, and the mining results of recently generated datasets also cause problems of validity in the near future if new data is continuously generated. In order to obtain accurate mining results including the newly generated data set, the mining operation of the entire data set including the previous mining target set and the new transactions occurring thereafter must be performed again. In general, as data sets grow larger, mining operations that use traditional mining methods require more work time and computer processing power, and thus cannot provide results in real time.

In some conventional studies or inventions, the term item set is used to express a bundle composed of a plurality of unit information, and one unit information is expressed as an item. In other words, instead of the term frequent items, it is expressed as a frequent item set. However, in the present invention, a bundle of a plurality of unit informations is expressed by simply shortening the item as an item set and individual unit information is called a unit item. This is explained in detail in the definition of mining target data set in Section 2.1. Data mining techniques are classified into frequent itemset search, frequent sequential pattern search, and clustering method. Analyzing the dataset, defining a collection of items that appear simultaneously in more transactions than a predefined threshold of support is called the set of frequent itemset, and finding the frequent item set in the dataset is such a frequent item set. Finding a job. Here, "support" is a value representing the ratio of the number of occurrences of an item in the entire data set, and is generally defined as the ratio of the transaction in which the corresponding item appears to the total number of transactions constituting the data set. In the related art, the frequent item search task is largely divided into two methods. One is the basic and most widely used method of analyzing all transactions in a data set at the same time, and the other is analyzing each transaction one by one. A representative method for analyzing all transactions simultaneously is "R. Agrawal, T. Imielinski, and A. Swani. Mining Association rules between sets of items inlarge database. In Proc. Of the ACM SIGMOD Int'l Conference on Management of Data, pages 207-216, May 1993. " And the Apriori method proposed in "R. Agrawal, and R. Srikant. Fast algorithms for mining association rules. In Proc. Of the 20th Int'l Conference on Very Large Databases, Sept. 1994." This method is an algorithm that searches a data set several times and finds frequent items for each unit item, and finds frequent items consisting of two unit items based on the result. This process is repeated until there are no more frequent items, increasing the size of the frequent items by one. In addition, a method of dividing and searching a data set has been proposed. However, when considering memory usage in a processing process, at least one data set searching is required.

See "C. Hidber. Online Association Rule Mining. In Proc. Of the ACM SIGMOD Int'l Conference on Management of Data, pages 145-156, May 1999." There is a Carma method offered. This method searches the dataset up to two times. In the first search, the items of interest to obtain the analysis results are identified and their frequency information is managed in memory using a lattice structure. In this case, the item may be an item of interest only when information on all subitems of the item is managed in the memory. In general, in order for one item to be a frequent item, all sub items must be frequent items. Therefore, only the items in which all subitems are managed in memory can be regarded as the interest items that can be frequent items. In general, other methods of managing the frequency of appearance of all items rapidly increase the memory usage during the mining process as the data set grows. However, by managing only those items that are expected to be frequent items, the number of items managed can be reduced, thereby reducing the memory usage required during the mining process. In the second search, items that cannot be frequent items are omniscient and the exact support of the frequent items is identified.

Existing frequent search methods including the above two approaches require that all transactions that make up the dataset must be defined statically and the dataset be searched at least once before mining starts. Has the disadvantage of not being able to analyze the change in real time.

The frequent item sequential pattern search is a mining method for extracting frequently occurring order type information about frequently occurring items from a data set consisting of transactions representing occurrence order among items. In other words, frequent item sequential pattern search is to find the sequential pattern with occurrence frequency greater than the threshold by analyzing occurrence sequence patterns among different items that occur together. This is a mining method that analyzes the frequency of frequent occurrences in the transactions that make up the data set. In other words, in the sequential pattern, an analysis task of finding frequent items by performing the frequent item search operation described above and adding an analysis job of identifying the order relations between them is added. As a mining method for exploring sequential patterns of frequent items in datasets, see "R. Agrawal and R. Srikant. Mining Sequential Patterns. In Proc. Of the Int'l Confernce on Data Engineering, pages 3-14, Mar. 1995." In sequential pattern search method is proposed. This method consists of a frequent item search step for the data set and a sequential pattern search step using the information obtained therefrom. In addition, the data set is multi-searched according to the number of frequent items constituting the sequential pattern. That is, to search for a sequential pattern composed of n items, the data set must be searched n times. Recently, as a method for reducing the number of candidate items generated in the intermediate stage while reducing the target data set in an iterative search for the data set, J. Pei, J. Han, B. Mortazavi-Asi, H. Pinto, Q. Chen, U. Dayal and M.-C. Hsu.PrefixSpan: Mining Sequential Patterns Efficiently by Prefix-Projected Pattern Growth.In Proc. Of the Int'l Conference on Data Engineering, pages 215-224, April 2001 PrefixSpan method is proposed. In the PrefixSpan method, a frequent 1-item is searched from the entire data set to search for a sequential pattern with an initial length of 1, and a separate data set is formed by extracting patterns that can be sequentially connected to each of these items. It consists of a small amount of data compared to the entire data set. Next, search for frequent items that follow each 1-item in the sequential pattern. This process is repeated until a longer sequential pattern does not occur and extracts all sequential patterns in the data set for a given extraction condition.

[ references ]

R. Agrawal, T. Imielinski, and A. Swani. Mining Association rulesbetween sets of items in large database. In Proc. of the ACM SIGMOD Int'l Conference on Management of Data, pages 207-216, May 1993.

"R. Agrawal, and R. Srikant. Fast algorithms for mining association rules. In Proc. Of the 20th Int'l Conference on Very Large Databases, Sept. 1994.

Christian Hidber. Online Association Rule Mining. In Proc. of the ACM SIGMOD Int'l Conference on Management of Data, pages 145-156, May 1999.

R. Agrawal and R. Srikant. Mining Sequential Patterns. In Proc. of the Int'l Confernce on Data Engineering, pages 3-14, Mar. 1995.

S. Brin, R. Motwani, J. D. Ullman, and S. Tsur. Dynamic itemset counting and implication rules for market basket data. In Proc. of the ACM SIGMOD Int'l Conference on Management of Data, pages 255-264, Tucson, AZ, May 1997

M. J. Zaki, Generating Non-Redundant Association Rules. In Proc. of the 6th ACM SIGKDD Int'l Conference on Knowledge Discovery and Data mining, pages 34-43, Boston, MA, Sept. 2000.

J. Pei, J. Han, B. Mortazavi-Asi, H. Pinto, Q. Chen, U. Dayal and M.-C. Hsu. PrefixSpan: Mining Sequential Patterns Efficiently by Prefix-Projected Pattern Growth. In Proc. of the Int'l Conference on Data Engineering, pages 215-224, Heidelberg, Germany, April 2001.

As data mining methods have proved to be effective for information analysis by a number of conventional studies, mining techniques are used to obtain necessary knowledge information through data set analysis in various application fields. However, the prior art has the following technical limitations.

Limitations in Basic Mining Methods

-The prior art is designed to predefine data sets before data mining and obtain results efficiently when basic statistical preprocessing analysis is possible for them. However, in an environment where the unit items constituting the data set can be changed and the data set is continuously increasing, it is impossible to define the unit items constituting the data set and their transactions in a limited manner, and thus, a basic statistical preprocessing analysis of the data set is required. It is also impossible.

In the conventional mining system, an object of the present invention is to provide analysis information about a fixed data set. Therefore, due to the change of the data set due to the addition of new data, the change of analysis information cannot be provided to the user.

In a conventional mining system, reference thresholds for support, confidence, correlation, interest, etc., which are indicators for determining the usefulness of items obtained as a result of mining, are fixedly defined. This is common and does not support the identification of changes in these values for addition of new data.

◈ Limitation of mining processing time and limitation of real time processing

The prior art requires a great deal of processing time to obtain analytical results that include information that has recently occurred in a constantly growing set of data. In other words, if a dataset is newly expanded in an environment where new transactions continue to occur, the results of the previous analysis become historical information, which reduces the value as the latest information including the information of the entire dataset that occurred recently. In order to obtain analytical results involving the dataset, you must re-mining some or all of the previous dataset and all newly occurring transactions. Therefore, the mining operation execution time is long.

The prior art has limitations in obtaining mining results in real time. Real-time processing of mining tasks is the ability to obtain analysis results in a limited amount of time. In the prior art, there is a limit in supporting fast processing time in consideration of accurate information analysis of a data set to be analyzed. In particular, each transaction that makes up a dataset must be read repeatedly, so in an environment where the dataset is constantly growing, all previous transactions must be stored separately, and the time required to obtain mining results that include information from the latest dataset. This lengthens, so there is a limit to getting results in real time. That is, when a new transaction occurs, the mining result of adding a transaction can be obtained through analysis of the entire data set. Therefore, the mining result of adding new transactions cannot be provided in real time.

The prior art in sequential pattern search is divided into a process of searching for frequent item sets for a data set and a process of searching for a sequential pattern based on the frequent item sets found through the data set. As in the frequent item search, in the sequential pattern search process, the data set is searched more than once according to the number of frequent word items constituting the pattern, and thus the same problems described above are inherent. Therefore, the prior art has a long search time for a data set in sequential pattern search and does not support real time processing.

◈ Low adaptability to changing information under differentiation of importance of information

The prior art does not differentiate the importance of information change in newly generated transactions over time. In other words, the information generated in the past or the recent in the analysis has the same importance. Therefore, it is impossible to distinguish between information that does not occur recently in the past and information that does not occur at all in the past, but recently generated. Therefore, a mining result based on a data set that occurs recently at the present time cannot be obtained.

The prior art has low adaptability to information changes. In other words, it is not possible to analyze the rate of change of new information appearing in a continuously increasing data set or information that increases or decreases at a certain time. In other words, in order for the latest information changes appearing in the data set to be reflected in the analysis results of the entire data set, the new information changes must continuously occur for a long time.

Limitations in the Analysis of Changes in Usability Indicators

The conventional technique requires a lot of processing time in one mining operation, and thus, there is a limit in analyzing a temporal change of knowledge in a data set that is continuously generated according to time change. That is, the conventional technology analyzes only static information because it extracts the ratio of generated information to the total number of defined transactions in analyzing information such as support and reliability for the data set to be analyzed. However, information having the same value may be interpreted as different values according to the change state of the corresponding information with respect to the time interval. That is, it can be interpreted as different knowledge according to the state that the corresponding value is increasing or decreasing for the same usefulness index value such as support or reliability, but in the prior art, this cannot be distinguished, and in the prior art, the change of knowledge with time change In order to analyze the data, separate and independent mining tasks should be performed for each time period to compare and analyze the difference of these results separately.

In order to solve the problems of the prior art and to satisfy application environment and requirements changes, the present invention aims to develop the following techniques for frequent item search and sequential pattern search, which are typical data mining techniques. .

◈ Improvement of basic structure and method for mining

In this paper, we propose a basic structure for mining non-limiting data sets in applications where new transactions are continuously occurring. As the data set is increased by the addition of continuously occurring transactions, the analyst freely analyzes the current point in time, overcoming the limitations of the prior art, which takes too much time to perform mining or continuously increases the required working memory space. The basic structure is defined to support the real-time processing by defining the scope of the data set and continuously destroying the data that is out of the scope of analysis.

◈ Support mining processing time and real time processing

Design a mining method that enables real-time processing by reducing the processing time for the entire data set including the increasing data set. Real-time processing requires first reducing the data set to be analyzed. In other words, the entire data set can be flexibly defined at a certain point in time, thereby minimizing the number of transactions to be analyzed while minimizing the number of transactions to be analyzed while ensuring the qualitative aspect of the analysis results for the initial range of transactions required by the user. Devise ways to minimize and maintain quality of results.

◈ Support change analysis over time of usability indicator

-An indicator of the usefulness of a set of items obtained from data mining is constantly changing in the data set that changes with the increase of the analysis transaction. The present invention devises a method for efficiently grasping the usefulness determination indexes of items obtained as a result of mining in such an environment.

In addition, the present invention devises a method for analyzing the change over time of the availability indicators that change frequently as the data set changes. In other words, we propose a method to provide additional knowledge about mining results by analyzing the rate of change and acceleration of the usability indicator.

-Supports the dynamic change of the usability indicator constraints for the mining result set in a mining environment where the data set is constantly growing. That is, the minimum support, minimum confidence, minimum correlation, and minimum interest for each item constituting the result set are dynamically changed based on the mining result at each time point.

◈ Increased adaptability to information differentiation and information change

-It suggests ways to differentiate the importance of information within the scope of analysis based on when the information occurred. That is, in order to differentiate the importance of information according to the time of occurrence of information, to improve the adaptability to the change of information in the data set generated by the change of time, and to efficiently detect the change of information in recent data according to the time of occurrence. We devise information differentiation method to give different weights.

-In the new mining method that enables real-time processing and differentiation of the importance of information according to the time of occurrence, it minimizes the degree of information differentiation (that is, all transactions occurring at each point in time have the same importance as each other). Devise ways to get results. This shows that the present invention can cover the conventional method.

◈ Optimize the elements and result set necessary for mining

-Devise a method to reduce the memory usage required for mining operations on a constantly increasing data set. Unlike the conventional method of managing the frequency information of all items appearing in a transaction, the number of items to be managed is reduced by predicting and managing the frequency information of items that are likely to be frequent items. Devise ways to reduce it.

Mining for a constantly growing set of data can cause errors in the mining results to minimize the space and time required to perform the task. In other words, if you want to keep the mining results accurately, the work space required for mining increases and the working time takes longer. Therefore, the present invention devises an optimized method that can adapt the accuracy of mining results and the space and time required for mining according to various needs of the user.

◈ Integrated system

The present invention devises an integrated data mining system based on the mining method devised in the present invention so that frequent items and frequent item sequential patterns can be searched for a data set continuously increasing with time.

1 is a flowchart illustrating an operation of applying an open data mining method in a data mining environment based on a continuously occurring transaction set.

2 illustrates various approaches of an open data mining method applicable to application domains.

3 is a flowchart of a method for searching a simple accumulated frequent items

4 is a flowchart illustrating operations of updating a frequency of occurrence of a set of items and adding a new item in a simple accumulation frequent item searching method;

5 is a flowchart illustrating the operation of updating the frequency of occurrence and pruning in the simple accumulation frequent items searching method.

6 is a flowchart illustrating an operation of adding a delay in a simple accumulation frequent item searching method;

7 is a flowchart illustrating a step of determining whether an item set can be added in the simple accumulation frequent item searching method;

8 is a flow chart of a forced battery operation step in a simple accumulation frequent items search method

9 is a flowchart illustrating a frequent item selection step in the simple accumulation frequent item search method;

10 is a flowchart illustrating a method for searching a simple accumulated item sequence pattern.

11 is a curve of weight change of a transaction according to an occurrence time in the information discrimination method

12 is a flowchart illustrating a method of searching for frequent items by the window method

13 is a flowchart illustrating the operation of updating the frequency of occurrence and adding a new item in the frequent item searching method using the window method;

14 is a flowchart illustrating a transaction window update step in the frequent item searching method using the window method;

15 is a flowchart illustrating a method of searching for frequent items by the attenuation rate method;

16 is a flowchart illustrating an operation of updating a frequency of occurrence in a frequent item searching method using a damping rate method;

17 is a flowchart illustrating a forced cell operation step in the frequent item searching method using the attenuation rate method;

18 is a flowchart illustrating a frequent item selection step in the frequent item search method using the attenuation rate method;

19 is a flowchart illustrating a method of searching for an item sequential pattern by the window method

20 is an example of differentiating the frequency of appearance of items according to the rate of change of support

21 is a flowchart illustrating operations of updating the frequency of appearance of items and adding new items for maintaining the mining result information separately;

22 is a flowchart of a modified transaction read step for processing concurrent transaction processing at once

23 is a flowchart illustrating operations of updating the frequency of appearance of items and adding new items for analyzing the change over time of the usability indicator.

24 is a flowchart illustrating a frequent item selection step for analyzing a change in a usability indicator over time.

25 is a flowchart illustrating a simple accumulation frequent item searching method for transaction purification through characteristic indicator value analysis

Figure 26 is a flowchart of operations in the transaction purification step

The present invention includes a newly added transaction in a data set in which a new transaction is continuously generated, and a mining method and system for extracting a mining result applying a frequent item and frequent item sequential pattern searching method in real time. A data mining system for analyzing a set to obtain a mining result can be divided into three parts as shown in FIG. 1. First, the data server module 10 that stores and manages a transaction data set continuously generated in a field to which the data mining method and system of the present invention is applied is a database system for continuously generating transaction data generated in an application domain. Or file system. Second, the data mining kernel 20 analyzing the data set to obtain the mining result analyzes the transaction data read module 21 for reading data of the database server module and the occurrence frequency information of items appearing in the transaction to obtain the mining result. It consists of a data mining module 22. Third, the mining result presentation module 30 presents the result obtained in the data mining kernel to the user in a satisfactory manner. Meanwhile, the mining result obtained in the data mining kernel may be maintained in a separate storage device 40 and the stored mining result may be provided to the user in the mining result presentation module 30. The mining method and system devised in the present invention are for the data mining kernel part among the configuration modules for mining shown in FIG. 1 and are composed of seven parts as follows.

1. Open Data Mining Basic Concepts

Conventional data mining methods that target a limited data set that is clearly defined before performing a mining task have many limitations in obtaining a mining result for a data set that is continuously increased with time. That is, the conventional mining method is designed to extract only the knowledge inherent in the data set because the data set to be analyzed is fixed before the mining start. In the present invention, a mining method of analyzing a predefined limited data set is defined as closed data mining. In contrast, the mining method devised in the present invention defines a data range as a method of obtaining a mining result in real time for a data set of an analysis range defined by an analyst based on a current point of time, based on a data set in which a new transaction is continuously generated. The current point in time is always defined as the point at which a new transaction occurs, and the analyst always gets the latest analysis. The data mining method that analyzes data sets that occur continuously is defined as open data mining.

Open data mining has the following characteristics: First, since the data set to be analyzed is defined indefinitely, the type and number of unit items constituting the data set may be continuously changed. Second, it extracts the result reflecting the information of the data set generated within the specified analysis range at the present time in a short time. In order to obtain information at the present time in an environment in which transactions continuously occur as time changes, real-time mining ability that can generate results within a required time through fast mining of data sets is required. Third, recent information is more important than the past in new transaction data that is continuously generated over time in the application field. Therefore, a method is needed to evaluate the value of the latest information higher than the information in the past, and the range of past transactions to be analyzed should be flexibly adjusted. Fourth, the mining analysis requirements for the application field can be changed. At this time, the minimum support for determining the frequent items must be dynamically changed to satisfy the changed requirements. Fifth, if there are a large number of data sets to be analyzed, allow the error within the mining result set within a certain range to reduce the memory usage and processing time during the mining process. This is a good way to trade off.

FIG. 2 illustrates various approaches that can be applied according to the degree of information differentiation according to the time of occurrence of each data in a newly generated data set over time in an open data mining environment. The accumulation method 200 is divided into a window method 300 that analyzes only a certain range of information generated recently, and attenuation rate method 400 that differentiates data generated according to time change according to a predetermined ratio.

2. Open Data Mining Method by Simple Accumulation

A method of obtaining a mining result by simply accumulating the number of occurrences is regarded as a data mining method 200 by a simple accumulation method, considering that each information generated at different points in open data mining has the same importance. The simple accumulation method is divided into a frequent item search method 210 and a simple accumulation sequential pattern search method 220 by simple accumulation according to a mining technique to be applied. The analysis target of the simple accumulation method is defined as the entire data set including all transactions that have occurred up to the present time. That is, since the information generated at each time point has the same weight, all the information from the first occurrence point to the time point for obtaining the mining result are analyzed.

2.1 Finding frequent items

Similar to searching for frequent items in closed data mining, a method of searching for frequent items by simply accumulating the information of all generated transactions is considered to have the same importance without considering the difference in transaction origin in open data mining. It is referred to as a simple accumulation frequent item search method 210. In the present invention, an adjacency lattice structure based on a prefix tree is used as a basic structure for managing the frequency information for finding frequent items in an open data mining environment. Mining using the prefix-tree based neighboring lattice structure is suitable for use in open data mining environments that require real-time mining because each transaction is read only once. In representing the items as potential-tree based adjacent lattice, each item is represented by a potential path and the frequency of occurrence is stored at the last node of the potential path. Therefore, each node of the lattice structure based on the prefix-tree has unit item information representing each unit information on the potential path and frequency of occurrence of an item composed of unit items existing from the top node to the node. When a new transaction is added, the frequency information of items appearing in the transaction is applied to each corresponding node on the lattice and updated. In the present invention, a lattice structure for managing the frequency of occurrence information of items appearing in a transaction is called a monitoring lattice.

In open data mining, the data set for frequent items exploration is

Is defined.

The transaction identifier TID may be defined as a transaction occurrence time and may be regarded as generating only one transaction at the same time by subdividing the time unit into very small units.

The basic concept defined in the conventional closed data mining method is redefined as shown in Table 1 in the open data mining method, and the number of transactions is not limited and continuously increases with time, so the mining results at each time point are analyzed. Mining results for. Of the concepts presented in the conventional method, the basic concept not defined in the following table is the same as the basic concept in the conventional mining.

Table 1. Basic concepts in open data mining

3 is a flowchart illustrating a method of searching for frequent items 210 in data mining by a simple accumulation method. In the transaction reading step 211, the transaction data newly generated in an application field is read (100) and the mining basic information updating step ( 212) updates basic information required for mining, such as the total number of transactions, the addition of items, and the threshold for pruning. The addition of items and the battery operation will be described in detail in the corresponding steps. In the update frequency count and add item step 213, the newly read transaction is analyzed to update the frequency count of the items managed on the current monitoring lattice or to determine whether to manage the newly appeared items and add them to the monitoring lattice. . At this time, it is determined whether the forced battery operation (214), if necessary, performs a forced battery operation on the items that do not need to be managed among the items managed in the forced battery operation step 215. In the frequent item selection step 216, the monitoring lattice is searched to determine the frequent item if the degree of support of the item existing in the lattice is equal to or greater than the minimum degree of support for determining the frequent item.

In the simple accumulation method, the information generated at each point in time has the same weight, so when one new transaction is added, the total number of transactions at the present point in the mining basic information update step ( ) Is the total number of transactions ( )from Obtain as In the conventional mining method, since the entire data set is fixed, the minimum support for adding and expressing items expressed as a support value and determining the frequent items and the minimum support for determining frequent items always have a constant value when converted into the frequency of appearance. However, in open data mining, new transactions can be added continuously, so the total number of transactions continues to increase, and the number of occurrences of items showing constant support continues to increase. Therefore, the threshold for adding items to or removing items from the monitoring lattice and the minimum number of occurrences for determining the empty item should be newly calculated in consideration of the increase in transactions. In this case, the value expressed as support is the total number of transactions ( ), 'Number of occurrences' = 'edge map' × The relationship is applied to the frequency of occurrence. Based on this change relationship, the threshold value and the minimum support are transformed into the frequency information by considering the total number of transactions at each time point.

Basically, in open data mining, the mining result is obtained by analyzing the item information appearing in all the transactions occurring up to that point in time. Therefore, it is necessary to manage the item information included in all the transactions that occur continuously. At this time, due to the nature of open data mining, which continuously increases the data set, the information that needs to be monitored may increase rapidly during execution. This results in increased memory usage during the mining process, increases the mining execution time, and increases the number of transactions when transactions continue to increase. The amount of information that monitoring lattice needs to manage can be increased to an unacceptable level in the system. In open data mining, items that appear in a transaction have very low support when they first appear. In other words, the total number of transactions in a continually increasing data set generally has a fairly large value. However, the number of occurrences of the first item is 1, and if there are many transactions up to now, the first item has very low support. That is, at the time of the first appearance of the item has a very low support even compared to the minimum support for the determination of frequent items, even if these information is not monitored at the first appearance does not affect the result of the frequent item determination. In the present invention, when the frequency of occurrence of items is predicted to have a value large enough to affect the determination of frequent items, it is added to the monitoring lattice to minimize the number of items to be managed during the open data mining process. . In this way, monitoring lattice is not continuously increased, which reduces memory usage and mining time during mining. The method of adding only the items having a sufficiently high frequency to the monitoring lattice is called a delay adding method, and a detailed process will be described in detail in the delay adding step 234 of FIG. At this time, the threshold frequency of occurrence as a criterion for adding delay is called a delay addition threshold value. The delay addition threshold is defined as a value less than or equal to the minimum support for the data set.

In the conventional closed data mining process, the former job of removing an item that can be sure that it cannot be a frequent item in the entire data set is performed. In open data mining, the data set is constantly growing, so there is no one to be sure that it is not a frequent item in the entire data set. However, the number of occurrences of items that do not appear for a certain period of time does not change, but the total number of transactions continues to increase, which reduces support. Thus, items having a lower support than a certain threshold can be removed from the lattice by a decrease in the support with time. As described above, the operation of removing the item having the support smaller than the predetermined threshold value from the monitoring lattice is called the item battery operation, and the detailed process will be described in detail in the frequency update and the battery operation step 233 of FIG. 5. In this case, the threshold frequency of occurrence is referred to as a battery threshold. Represented by Pruning can reduce memory usage during mining by reducing the number of items managed in the monitoring lattice. The battery threshold is defined as a value below the delay addition threshold for the data set. If the battery threshold is defined to be higher than the delay addition threshold, the monitoring lattice to manage the items may occur, or the unnecessary delay addition and the battery operation may occur repeatedly.

As shown in FIG. 4, the frequency update and the item addition step 213 first read items one by one in the newly added transaction in the item read step 231. The read-out item is divided into two main steps through the determination step 232 of whether it exists in the monitoring lattice. If an item appearing in the newly added transaction exists in the monitoring lattice, update the frequency of occurrence or pruning, and perform step 233. Among the items appearing in the new transaction, an item not present in the monitoring lattice is delayed adding step 234. Is added to the lattice if necessary. In step 235, if there is an item that is not analyzed, the above process is repeated.

5 shows update frequency count and battery operation step 233, where the frequency of occurrence of the target item at the present time ( ) Is the number of occurrences ( Considering new advent information) And update item frequency step 241, and the item battery step 243 to remove the item from the monitoring lattice when the updated frequency is less than the battery threshold of the item. In the simple accumulation method, since the information generated at each time point is equally important to each other, 1 is added to the frequency of occurrences up to the previous time point. The reason for adding 1 is that in the frequency update and the item addition step 233, the frequency update operation is performed only for the items that appear in the new transaction. Item cell step 243 removes the item from the monitoring lattice if the frequency of occurrence is less than or equal to the battery threshold. At this time, the 1-item keeps it even without removing it from the lattice even if it has a frequency of occurrence below the battery threshold. Because 1-item does not have subitems, it is impossible to estimate the frequency of occurrence of the item from other items. Therefore, the actual value must be maintained at all times, and when the value once calculated is added to the lattice, information about the frequency of occurrence before the battery is ignored and an error occurs in the frequency of occurrence of the corresponding 1-item. On the other hand, in one 1-item, if the item does not appear in transactions that have occurred for a significant period of time, the total number of transactions is greatly increased compared to the frequency of occurrence of the item, and thus has very low support. In the basic cell method, even in this case, the 1-item is not celled. However, if you have a very small number of occurrences relative to the total number of transactions, even if it is a 1-item, that item can be passed from Lattice. In this case, a separate threshold value is defined, and if the frequency of occurrence of 1-item is smaller than this, it is transferred from the monitoring lattice. In this case, the threshold which is a reference is referred to as a 1-item battery threshold. The 1-item cell threshold is defined as a degree of support that is less than the n-item (n ≧ 2) cell threshold and can be ignored in the entire dataset. If the value is defined too high even at the 1-item cell threshold, the accuracy of the information generated in the entire dataset may be low.

FIG. 6 shows a process of performing the delay adding step 234 of adding items not present in the monitoring lattice among items appearing in a new transaction to the monitoring lattice, and immediately adding or appearing in consideration of the length of the item to be added as follows. Add it through the frequency estimation process. In the one-item addition step 251, all new one-items appearing in the transaction are added immediately without undergoing an estimation process for the frequency of appearance. That is, the frequency of appearance at the time of the first addition becomes one. An n-item (n≥2) consisting of two or more unit items is not added to the monitoring lattice immediately upon appearance, and all subitems of that item are present in the monitoring lattice, and the frequency of occurrence estimates for that item set the delay add threshold. Can be added when exceeded. In this way, the n-item is added to the lattice immediately after the first occurrence, and it is determined whether the delay is added, and the frequency of occurrence of the delayed addition is the number of occurrences of the subitems of the item at step 253. Is estimated from In this case, it is determined whether the appearance frequency estimation value can be added beyond the delay addition threshold defined by the user (254), and if applicable, the item is added to the monitoring lattice. When adding items by delayed addition, the frequency of occurrence is estimated, and the delayed addition process can reduce the number of items managed by monitoring lattice. That is, the memory capacity required in the actual mining process can be reduced.

7 shows an addability determination step 253 for determining whether the item can be added. When there is an item for which you want to estimate the frequency of occurrences, first search for the necessary subitems of the item in the monitoring lattice. To determine whether the frequency of occurrence can be estimated, the subitem of the item currently managed by the monitoring lattice is searched for in the monitoring lattice. If more than one subitem is not present in the monitoring lattice, the item does not estimate the frequency of occurrences. It is clear that if some of the sub-items are not managed by the current monitoring lattice, the item's frequency count estimate may not have a value above the threshold for delayed addition. It can be determined that it does not need to be added to this monitoring lattice. If all subitems are present in the monitoring lattice, a possible frequency estimate step 253c is used to estimate the possible maximum and minimum values of the frequency of occurrence. If the maximum value of the frequency of occurrence obtained is greater than or equal to the threshold for delay addition, the corresponding item is added to the monitoring lattice to start monitoring (253f). If some subitems do not exist in the lattice or the maximum value of the frequency count estimate is less than the delay addition threshold, it is determined that the item cannot be added (253e).

When the item for which the frequency of occurrence is to be estimated as e, the possible maximum and minimum values of the frequency of occurrence are estimated as follows. In order to estimate the frequency of occurrence, first, a set of subitems of e, a set of m-subitems, and a set of frequency of m-subitems are defined as follows.

Meanwhile, two items and Merge is given And common items Define as

Iii. Merged item Is or This item consists of all the unit items belonging to.

Ii. Common item Is and An item consisting of all the unit items belonging to all.

In this case, for one item, each subitem has at least as many occurrences as the frequency of appearance of the corresponding item. For example, if all the unit items constituting the item always appear at the same time, the item has the same frequency of occurrence as the subitems of the item. Therefore, the frequency of appearance of the item is affected by how frequently the unit items constituting the item appear together. Taking into account the results of these analyses, two distinct types of distribution that determine the number of occurrences of an item are defined as follows. First, for any two items that appear in a data set consisting of multiple transactions, the case where these two items appear together in as many transactions as possible is called the minimum exclusive distribution. Next, the case where these two items appear as exclusively as possible is called the maximum exclusive distribution. For example, two items in a dataset consisting of 10 transactions and If the number of transactions that appeared is 6 and 7, respectively, and they appear in the form of merge items in all possible transactions (i.e., if the number of occurrences of merge items is 6), these two items are the minimum exclusive distribution. On the other hand, if a merge item appeared only in three transactions, then these two items are the maximum exclusive distribution.

Two items And Merge when is the minimum and maximum exclusive distribution The frequency of occurrence of can be found as follows. First, let D be a data set composed of transactions. In this case, | D | represents the total number of transactions included in the data set. In addition, the TS (e) represents the set of all transactions in which item e appears in the data set, and C (e) represents the total number of transactions belonging to TS (e). Two items And For the following equation, the following equation is established by the definition of the merge item.

If the two items are at least exclusive, TS ( ) ⊆TS ( ) Or TS (e1) ⊇TS ( ) The relationship is established. So TS ( ) Is TS ( ) Or TS ( ) Is the same set. So C ( ) Is the number of occurrences of two items, C ( ) And C ( ) Will have the same value as the minimum value. That is, the following relationship holds.

Min (V) is a function for obtaining a minimum value among the values included in the set with respect to the set V of the numeric values. Where two items are common If exists, the following relationship holds.

At this time, considering the process of obtaining the number of elements in the set it can be seen that the following equation is established.

if If this does not exist, the number of occurrences of the merge item of the two items can be obtained by considering the total number of transactions constituting the data set. TS ( ) And TS ( ) Is a subset of dataset D, so the following relationship holds:

At this time, considering the number of elements in the set it can be seen that the following equation holds.

If the frequency of occurrence of two items is C ( ) And C ( ) Is the total number of transactions in the entire data set | D | If less than C ( ) May have a value less than zero. However, C ( ) Must have a value greater than or equal to zero since it is the number of occurrences of the item. When the two items are at the maximum exclusive distribution, the merged item consisting of the two items takes the smallest value and can be obtained as follows.

Here, max (V) is a function for obtaining a maximum value among the values included in the set with respect to the set V of the numeric values.

When two items have a minimum exclusive distribution, the number of occurrences of the merged item has a maximum value. On the other hand, when two items are at the maximum exclusive distribution, the frequency of occurrence of the merged item has the minimum value. That is, the number of occurrences of the merged item has a value in a range larger than the number of occurrences in the case of the maximum exclusive distribution and less than the number of occurrences in the case of the minimum exclusive distribution.

As described above, the frequency of occurrence of items to be delayed may be obtained from the frequency of subitems of the item. At this time, both the minimum number of occurrences and the maximum number of occurrences are obtained. Minimum possible frequency of occurrences in the following parts of the invention And maximum possible frequency Denotes the possible minimum frequency and maximum frequency of item e. The maximum possible frequency of items is determined by assuming that all subitems have a minimum exclusive distribution. In other words, it is assumed that sub-items appear in as many transactions as possible at the same time. The two subitems of item e in item e. And When is the minimum exclusive distribution, the number of occurrences of these merged items (i.e., equal to item e) is calculated as min (C ( ), C ( Is obtained as Therefore, the maximum possible frequency of occurrence of item e when all subitems of that item are the minimum exclusive distribution Can be defined as the minimum value among the frequency counts of all subitems. However, of all the subitems, the subitem of the maximum length has the smallest frequency of occurrence. therefore, Can be obtained by considering only the frequency of appearance of (n-1) -sub-items. That is, (n-1) -subitem for item e Is not an empty set, the maximum number of possible occurrences Is obtained as follows.

Minimum number of possible occurrences of item e Can also be obtained from the number of occurrences of all subitem sets or from the number of occurrences of (n-1) -subitems. However, this is taken into account when the subitems are at the maximum exclusive distribution. Two different (n-1) -subparts And The minimum number of possible occurrences when two subitems are at the maximum exclusive distribution for each combination of Can be obtained as described above. Here, the maximum value among the minimum possible frequency counts obtained in all combinations may be the minimum possible frequency count of item e. That is, (n-1) -subitem set Are not empty sets and they are the smallest possible frequency of occurrence when they are at the maximum exclusive distribution Is obtained as follows.

On the other hand, in order for an item to be added to a lattice, all subitems of the item must exist in the lattice. If one or more of the subitems of the item do not exist in the lattice, the maximum possible frequency of occurrence of the item Is min ( ), So the value is calculated as zero. Therefore, it can be determined that the item is not added without calculating the frequency of appearance. That is, only when all subitems of an item exist in a lattice, it is necessary to determine whether the item is added. If one or more subitems do not exist in the lattice, the item is not added to the lattice.

About item e, Is specified as the actual number of occurrences of the item. But, Since is a value obtained by estimation, the number of occurrences of each item may include an error according to the estimation. In this case, the difference between the maximum possible frequency and the minimum possible frequency is referred to as the estimated error ε (e) of the corresponding item and represents the maximum error that can be generated by estimating the frequency of appearance. However, the number of occurrences, which is regarded as an error in estimating the frequency of occurrence, is fixed as the value at the time of estimating the frequency of occurrence of the item, but as the number of new transactions is added, the total number of transactions increases, so the difference in support due to the estimation error It is very small. Therefore, the frequency of appearance of items added by delayed addition has negligible error after a certain time. In other words, you get reliable mining results.

After updating the frequency of occurrence of items and additional work on newly appeared items, the frequent items are searched. Here, the item frequency update and battery operation are performed on newly appeared items. Thus, items that do not reappear after appearing long ago in the past have very little support but can be maintained without being removed from the monitoring lattice. In other words, since unnecessary items that can be removed exist in the lattice, memory may be wasted during the mining process and thus removed. In addition, to minimize this waste, all items present in the lattice can be searched to remove the batteryable items from the lattice. This operation is called forced cell operation. Lattice has the advantage of minimizing memory usage because Lattice retains only items above the battery threshold. Forced pruning, however, is relatively time consuming because it requires the entire monitoring lattice to be explored. Therefore, it is performed periodically at regular time intervals or only when there is an insufficient memory situation. 8 illustrates a step 215 of performing a forced battery operation. First read a single item from Lattice and count the frequency of occurrence of that item (262). If the comparison results in that the number of occurrences of the item is less than the battery threshold, the item is removed from the lattice in the item removal step 263. If no items are left in Lattice, continue the forced battery process.

The frequent item selection step 216 described in FIG. 9 searches for all items present in the monitoring lattice and determines the item as a frequent item when the number of occurrences of the item is greater than or equal to the minimum support for determining the frequent item. If there is an item that has not been searched in the lattice, the frequent item selection step is repeated and ends when all items have been searched. The frequent item selection step is the same as the frequent item selection process in the conventional mining method based on the lattice structure, and searches the entire lattice like the forced cell work. Therefore, it is possible to reduce work execution time by executing at the point where frequent items are to be obtained rather than executing each time a new transaction is added. In order to solve the problem of long execution time by searching the entire monitoring lattice for the selection of frequent items, a method of keeping the frequent items obtained as a result of mining separately may be considered. This is described in detail in Chapter 5.

The error in the number of occurrences due to delayed addition is reduced to a negligible value as new transactions are added. In other words, the error caused by the estimation is always maintained at the same value due to the difference in the frequency of appearance at the time of adding the item to the lattice, but as the total number of transactions increases, it is attenuated based on the support. However, when there is an item with a large difference in support due to an estimation error in obtaining a mining result at a certain point in time, the result may be different from that of mining without applying the delayed addition method. That is, compared to the case where the delayed addition method is not applied, the support of a specific item may be large or the number of frequent items obtained as a result of the marning may be large. In order to compensate for this problem, the monitoring lattice manages the appearance frequency information of each item together with the error frequency estimation error information at the time of delay addition, so that the current degree of support and the degree of error of the frequent items can be accurately identified. Highly reliable mining results can be obtained. In general, the higher the delay addition threshold, the lower the number of items managed by Lattice, but the frequent items obtained as a result of mining may have a large error. This is because the estimation error is judged to be a frequent item before the negligible decrease.

Table 2. Change of Procedure for Error Information Management

Change position Estimation error not managed Estimated Error Management Add item delay (234) 1-item Node only information on lattice contains frequency count information Estimation error = 0 when the node on the lattice has the frequency of occurrence and estimated error information n-item Node only information on lattice contains frequency count information When nodes on the lattice have frequency and estimated error information, the estimation error differs between the maximum and minimum values of the frequency estimate. Item Battery Operation (233, 215) Consider only frequency of occurrence Consider only frequency of occurrence Frequently Selected Items (216) Frequent items when the frequency of occurrence is above the minimum support Frequently, the number of occurrences is greater than the minimum support and the estimated error is less than the tolerance.

To provide this error information to the analyst, each item in the monitoring lattice must manage the estimated error information as well as the frequency of occurrence of the item. Therefore, the process of estimating the frequency of appearance when the item delay is added in FIG. 6 is the same, but the estimation error as well as the frequency of appearance of the items are kept together in the monitoring lattice. In the case of 1-item, the estimation error is zero because the frequency of appearance is not an estimate but an actual value. Error information is not taken into account in pruning. Because the frequency information of the items to be battery is considered the maximum possible value in the estimation step for delay addition, when the frequency of occurrence is less than or equal to the battery threshold, it becomes less than or equal to the battery threshold. The battery threshold is determined by the user and must be specified within a range not exceeding the minimum map. In this case, as the threshold is higher, the number of items to be removed increases, but the accuracy of the mining result may be lowered. When selecting the frequent items, the estimation error as well as the number of occurrences are considered. In other words, items with a frequency above the minimum number of maps may be frequent items, and the estimated error of these items shows a value in which the error when the item is classified as frequent items is reduced to the present time point. It is an error that you can have. Therefore, the frequent items may be limited to cases in which the estimated error is less than or equal to a predetermined threshold value defined by the user. Represented by Tolerances are determined by the user within the minimum map. When the tolerance is the minimum support, the result is that the frequent items are selected only by the frequency information. Mining methods using error information management can be summarized as shown in Table 2, compared with methods that do not manage error information.

2.2 Unit Item Sequential Pattern Search

The data set for unit item sequential pattern search is expressed as a set of unit items in which a transaction has ordered information as follows so that an occurrence order between unit items can be analyzed. That is, in the data set for frequent item search, duplication is not allowed between unit items that constitute one transaction, but in the transaction for unit item sequential pattern duplication, unit items are allowed to be duplicated. As it has meaning, it is defined as an ordered set of unit items as follows. In the next transaction, → is used to express the semantic meaning, not in the actual transaction set.

Unit Item Sequential Pattern Transaction = {100 → 200 → 100 → 300 → 400 → 500}

Like the sequential pattern search method in closed data mining, all accumulated transaction information is regarded as having the same importance without considering the difference in transaction occurrence time in open data mining. The method of searching for a sequential pattern for the second method is called a simple accumulation unit item sequential pattern search method 230. The simple accumulation unit item sequential pattern search method 230 is the same as the simple accumulation frequent item search method described in Section 2.1, the following steps of reading a transaction, updating basic mining information, updating the frequency of occurrence of the sequential pattern, and adding a new sequential pattern. It consists of a forced battery operation step and a unit item sequential pattern selection step. However, in the unit item sequential pattern search, the overlap between the unit items constituting a transaction is allowed, and thus the process of estimating the frequency of occurrence of the n-sequential pattern (n ≧ 2) has a different process from that of the frequent item search method.

As in the frequent item search, the one-sequential pattern consisting of one unit item does not estimate the frequency of occurrence and manages the frequency of occurrence by adding all items to the lattice at the first appearance. On the other hand, the number of occurrences of the n-sequential pattern (n≥2) is estimated from the partial sequence patterns, as in the method of estimating the frequency of occurrence in the frequent item search process described in Section 2.1. In this case, unlike the frequent item search method that analyzes only the simultaneous appearance regardless of the order between the items, the overlapping of items appearing in the transaction is allowed in the unit item sequential pattern search and the order information of the appearance unit items is analyzed. As described above, in the method of unit pattern sequential pattern detection, which allows the duplication of unit items and analyzes the sequence information, a new n-sequence pattern (n≥2) is introduced and the order information is considered in the process of estimating the frequency of occurrence. Is performed together. For example, consider a sequential pattern "abc" consisting of three unit items a, b and c. In the frequent item search method described in Section 2.1, when considering sub-items, the order information is not taken into account but only simultaneous occurrence information is considered from the number of occurrences of three sub-items such as "ab", "bc", and "ac". Estimate the number of occurrences of the item. However, in the unit item sequential pattern search method, not only considering concurrency but also considering the sequence information in the partial sequential patterns used for estimating the frequency of appearance, the unit sequential pattern search method should be selected in consideration of the start position of the partial sequential pattern. In the unit item sequential pattern search method, when the frequency of occurrence of the sequential pattern "abc" as mentioned above is estimated, the "ab" and "ac" partial sequential patterns have the same frequency of occurrence information "abc" even if the sequence information is considered. It is used to estimate the frequency of occurrence of patterns. However, in the "bc" partial sequential pattern, unlike the frequent item search method, the frequency of occurrence divided by the start position of the sequential pattern is used to estimate the frequency of occurrence of the "abc" sequential pattern in consideration of the order information. In other words, the "bc" sequence pattern starts with the first unit of a transaction consisting of a series of units and the second after the transaction, depending on where the sequence pattern appears in a transaction with sequence information. It can be divided into two cases starting from the unit of. At this time, the partial sequential pattern to be considered in estimating the frequency of occurrence of the sequential pattern "abc" is a case where the "bc" partial sequential pattern appears in the second and subsequent unit items of the transaction. Therefore, only the frequency of occurrence of the case other than the case where "bc" starts at the beginning of a transaction is used to estimate the frequency of occurrence of the sequential pattern "abc". In order to apply the frequency of occurrence according to the appearance position differently, in the unit item sequential pattern search method, each sequential pattern maintains two kinds of frequency information. In other words, the number of occurrences calculated for all cases in which the corresponding sequential pattern appears and the number of occurrences calculated only when the sequential pattern starts from the second and subsequent unit items in the transaction must be managed separately. This can improve the accuracy of the frequency count estimate in the unit item sequential pattern search method.

On the other hand, in the unit item sequential pattern search method, immediately after the sequential pattern is added to the monitoring lattice in the n-sequential pattern (n≥2), the number of occurrence frequencies of the sequential pattern is obtained by estimation. Therefore, it is impossible to distinguish the number of occurrences of the sequential pattern according to the start position in the transaction. In this case, the number of occurrences of the sequential pattern managed by irrespective of the starting position in the number of occurrences of the sequential pattern and the number of occurrences of the sequential pattern starting with the second and subsequent unit items in the transaction are obtained by the same estimation. Specify by value. After that, additionally generated transactions are managed by distinguishing the number of occurrences of the corresponding sequential pattern. That is, each sequence pattern present in the monitoring lattice for unit item sequence pattern search has two occurrence frequency information.

As in the frequent item search process, in the unit item sequential pattern search process, it is determined whether or not the corresponding sequential pattern is added in consideration of whether or not the partial sequential pattern exists in the lattice. That is, in order for one sequence pattern to be added to the lattice, all partial sequence patterns of the sequence pattern must exist in the lattice. If at least one of the partial sequence patterns of the sequence pattern does not exist in the lattice, the maximum possible frequency of occurrence of the sequence pattern is calculated as zero. Therefore, it can be determined that the addition is not necessary even if the frequency of occurrence of the sequential pattern is not calculated. That is, only when all subitems of the sequence pattern exist in the lattice, it is determined whether to add the sequence pattern. If at least one partial sequence pattern does not exist in the lattice, the sequence pattern is not added to the lattice.

2.3 Item Sequential Pattern Search

In the sequential pattern search method, in which one transaction is expressed only with sequential information between unit items in the sequential pattern search, in the sequential pattern search method, as follows: It has the order information of items consisting of the unit items that appeared. In this case, there is no overlapping unit item in the association information expression unit that appears at the same time, but there may be duplicate items between items indicating ordered information.

Item Sequential Transaction = {(100, 200, 300) → (100, 200) → (300, 400, 500)}

In open data mining, it is possible to search for sequential patterns of frequent occurrences among frequent items by simply accumulating the information appearing in the transactions defined as an analysis target without considering the difference in time of transaction occurrence. The method is referred to as a simple accumulation item sequential pattern search method 220. FIG. 10 illustrates a simple accumulation frequent item sequential pattern searching method 220, which is composed of a frequent item searching step 221a and a sequential pattern searching step 222. As in FIG. 10, two steps are performed continuously. First, the frequency of occurrence of items appearing in the transaction is updated in the frequent item search step, and new items or frequent items are obtained. Next, we search for frequent item sequential patterns by analyzing the corresponding transactions with only these frequent items. At this time, to analyze the sequential information on the transaction, the transaction is maintained in memory until the end of the sequential information analysis.

The frequent item search step has the same process as the frequent item search step by the simple accumulation method described in Section 2.1, and an item for sequential pattern composition is selected as a result of the frequent item search. An item constituting the sequential pattern must be a frequent item for the same data set first, and an item that does not satisfy this cannot form a sequential pattern. Therefore, in the frequent item search phase, all the frequent items satisfying the given minimum support are first searched for in the items that occur in the transaction. The frequent items obtained are basic items for sequential pattern search and become one sequential pattern unit item in the sequential pattern search step. The other steps 222b to 222f are identical to the frequent item search step except that the sequential pattern search step 222 includes the unit item mapping step 222a and the unit item mapping analysis step 222g for the sequence pattern. In the unit item mapping step, the items for the sequential pattern obtained in the frequent item search step are mapped to separate identifiers so that the sequential pattern unit items are represented by one identifier. The items obtained in the frequent item search step are composed of a plurality of unit items, and these items become a sequential pattern unit item in the sequential pattern search step. Therefore, in order to efficiently manage the sequence pattern unit item information in the sequential pattern search step, the item obtained in the frequent item step is represented by a separate identifier. In the unit item mapping information analysis step, each of the sequential pattern unit items in the sequential pattern analysis step is analyzed to express the unit items constituting the original data set. Meanwhile, in the frequent item search step and the sequential pattern search step, separate lattices are maintained, and the sequential pattern is searched in consideration of the association-lattice and the frequent item analysis results based on the associations of items appearing in the transaction. Is divided into sequential-lattices. By maintaining such a double lattice structure, it is possible to minimize the data set under consideration in the sequential pattern step of analyzing the ordered information on the frequent items obtained in the frequent item search step. Associative-lattices have the same structure as the monitoring lattice structure for the frequent item search described in Section 2.1, and sequential-lattices basically have the same structure as the monitoring lattice for the unit-item sequential pattern search described in Section 2.2.

The item mapping and interpretation process is as follows. First, in order to maintain a mapping relationship between the items created in the frequent item search step and the sequential pattern unit items in the sequential pattern search step, a separate connection relationship table is maintained. The sequential pattern unit items in the sequential pattern search step are each items generated in the frequent item search step, and are represented by a separate integer identifier, and become sequential pattern unit items in the sequential pattern search step. On the other hand, in the sequential pattern search step, the items that do not appear in a plurality of newly generated transactions may be deleted from lattice. In this case, when one sequential pattern unit item is deleted in the sequential pattern search step, no sequential pattern including the sequential pattern unit item exists on the sequential-lattice. That is, the sequential pattern unit item no longer exists in Lattice. Therefore, the sequential pattern unit item deletes the corresponding mapping information in the association table for representing the association relationship between association-lattice and sequential-lattice. That is, the identifier representing the sequential pattern unit item that is no longer utilized in the sequential pattern search step is recycled when a new item is added in the frequent item search to map the sequential pattern unit item identifier. Basically, when a new item is generated in the frequent item search step and an identifier for mapping to the sequential pattern unit item is assigned, the identifier is increased by 1 from the identifier given to the previously generated sequential pattern unit item. However, if there is a sequential pattern unit item deleted in the sequential pattern search step, it is used again. That is, when there are sequential pattern unit items deleted in the sequential pattern search step, the smallest value among all available identifiers including these identifiers is given as the sequential pattern unit item identifier for the newly created item in the frequent item search step.

The process of adding the newly generated item sequential pattern to the lattice in the sequential pattern search step is the same as the process of adding the unit item sequential pattern in the method of searching the unit item sequential pattern described in Section 2.2. In addition, the process of estimating the number of occurrences of the item sequential pattern required in this process is the same as that of the unit item sequential pattern search method. That is, in considering the partial sequence pattern of the item sequential pattern for which the frequency of occurrence is to be considered, the starting position in the transaction of the partial sequential pattern should be considered and the redundancy of the items constituting the sequential pattern in determining the number of partial sequential patterns Consider whether or not.

Meanwhile, in the sequential pattern search step 222, the mining basic information update operation is not performed. That is, since the update has already been performed once in the frequent item search step 221a, no additional update operation is performed in the sequential pattern search step.

3. Open data mining method by information differentiation

The importance of the information to be analyzed by applying the mining method is expressed as the frequency of occurrence in the data set to be analyzed, and data mining analyzes the information of high importance (hereinafter referred to as 'significant information') and applies the same information. It is used to predict or judge data set generated in the field. Meaningful information about the entire dataset in the dataset that occurs continuously with time will generally be repeated in proportion to the importance of the information over time unless significant changes occur in the application. That is, meaningful information in the entire dataset will be generated repeatedly in recent datasets. In addition, if information that has not occurred in the past increases in frequency in recent transactions, it can also be referred to as meaningful information (or information change) in the data set. Therefore, in open data mining, the information generated from all transactions up to now can be differentiated according to the time of occurrence in the analysis of the entire data set. That is, an information differentiation method that considers the importance of recently generated information as compared to the information generated in the past may be considered. This information differentiation method should be maintained in the mining process by keeping the size of the dataset being analyzed as a range of datasets that occurred previously from the current point of view when focusing on recent changes in the continuous dataset. At the same time, there is an advantage that the analysis can be performed by focusing on the latest information. The scope of analysis can be defined variably in the entire data set according to the degree of setting mining parameters that define the scope of information to be analyzed. Different weights can be assigned according to the time of occurrence or the order of occurrence so that higher weights can be analyzed for the recently generated information. As the weight differentiation method according to the time of occurrence or the order of occurrence, the following methods may be considered.

In the information differentiation method, the degree of information differentiation can be defined based on the time point of the transaction constituting the data set, or the degree of differentiation can be defined by the generation order of the generated transactions. In the description of the present invention, each information differentiation method will be described based on the TID. At this time, when the TID is defined as the occurrence time of the transaction, the degree of information differentiation is defined by the occurrence time, and when the TID indicates the order of occurrence, it is defined as the number of transactions.

3.1 Search for frequent items

In the information differentiation method, the frequent item search method is similar to the frequent item search process by the simple accumulation method described in Section 2.1, but has the following differences. First, the data set to be analyzed is defined in a certain range from the current point in time. In other words, unlike the simple accumulation method that always maintains analysis information of the entire data set in memory and obtains frequent items from each point in time, the data set of the previous range is regarded as the target of analysis based on the current time point. Keep only important information in memory. In this case, the method is divided into a window method 300 and attenuation rate method 400 according to a method of determining a range of a data set to be analyzed. In each method, the information of the transaction is differentiated in different ways according to the TID of the transactions that make up the data set.

11 shows a weight differentiation method of each transaction according to an occurrence time point. The weight curve (a) represents the transaction weight in the simple accumulation method described above, and all transactions have the same importance regardless of the time of occurrence. The weight curve (b) shows a change in the weight of a transaction in the window method in which only a transaction occurring within a certain time range is analyzed and considered as an analysis result for an application field. The weight curve (c) shows a change in the transaction weight in the decay rate method in which the weight of the transaction occurring at the present time is regarded as 1, and the weight of the previously generated transaction is considered to be decayed at a predetermined rate based on the current transaction weight.

a. Frequent Item Search by Window Method

In the frequent item search using the window method, a data set consisting of transactions occurring within a certain range from the present time point is analyzed. In this case, a certain range that defines the data set to be analyzed is called a transaction window and is represented by TW. The size of the window is defined by the user and is defined by the transactions that occurred in the latest schedule at the time of the transaction. Based on the count, it can be defined as a certain number of recent transactions. The following description defines the window size to be analyzed based on the TID. If the TID is defined as the time of occurrence of a transaction, the window size is defined by the time of occurrence to define the most recent maximum time each transaction has meaning, and when the TID indicates the order of occurrence, the window size is defined as the number of transactions. The latest maximum number of transactions with is defined. On the other hand, in the Windows method, the set of target transactions is changed over time. That is, only recent transactions as large as the window size are analyzed based on the current transaction. At this time, the frequency of appearance should be reduced for items appearing in transactions excluded from analysis over time. On the other hand, in order to manage the change of the transaction window over time, the window method must maintain log information on transactions as many as the window size based on the current time.

12 shows a process of performing the frequent item search method 310 by the window method, and reads a transaction 311, updates basic mining information 312, updates a frequency of occurrence and adds an item 313, and updates a transaction window. Step 314 and frequent item selection step 315. The steps for reading transactions and selecting frequent items are the same as described in the simple accumulation method in Section 2.1. The window method, on the other hand, has a lattice structure as a basic structure, as in the simple accumulation method.

In the mining basic information update step 312, the minimum frequency of occurrence for frequent item selection is updated in consideration of the minimum support defined by the user and the total number of transactions. On the other hand, the total number of transactions and the minimum number of occurrences are defined in two ways depending on the transaction identifier TID at the present time. If the window size is smaller than the current analysis size, it is limited to the initial stage of analysis. The total number of transactions at the present time is added to the total number of transactions at the previous point in time, and the minimum frequency of occurrence is determined by considering the total number of transactions. Update On the other hand, if the number of transactions to be analyzed is larger than the defined window size, the total number and minimum frequency of transactions have the same value as the previous time.

FIG. 13 shows an update of the frequency of occurrence and the addition of items 313 for items that appear in a transaction. In an item reading step 331, an item generated in a transaction is read. At this time, if an item exists in a lattice, the frequency of occurrence of the item is increased and updated (333), and an item not present in the lattice is added to the lattice (334). In this case, unlike the simple accumulation method, the item is added to the monitoring lattice without an extra frequency estimation process. This process is repeated while there are items in the transaction that are not processed by this process.

14 shows a transaction window update step 314, which shows a transaction window change and an appearance frequency update process. First, in step 314, the transaction occurring at the present time is added to the transaction window. At this time, the information update work on the items that occurred in the added transaction was performed in the previous step, so it is not considered separately. Next, update the transaction window information as the transaction increases. First, in the initial stage, if the total number of transactions that have occurred so far is less than or equal to the transaction window size (when the transaction window is defined as the time of occurrence, when the current transaction occurs before the base time that defines the transaction window) End the update without any additional work for updating the window. That is, if the current transaction identifier TID is smaller than the window size, the transaction window update process is terminated. Next, if the total number of transactions that have occurred so far is larger than the transaction window size (if the transaction window is defined by the time of occurrence, then the occurrence of the current transaction is later than the base time that defines the transaction window), that is, the TID is If it is larger than the window size, the following process is performed to remove transactions that occurred before the window size time from the current transaction occurrence and to reduce the frequency of occurrence of items appearing in the transaction. First, an item that appears for one of the transactions to be removed in the previous transaction window is read (343) and the number of occurrences of the corresponding item is reduced by one. In this case, when the reduced frequency of occurrence becomes zero (that is, the corresponding item does not appear at the point of time of the latest window size), it is removed from the lattice (346). If an unprocessed item exists in the transaction to be removed, steps 344 to 347 are repeated. If the processing is completed for all items, the transaction is removed from the transaction window. If another transaction to be removed still exists in the transaction window, steps 343 to 348 are repeated. Therefore, the monitoring lattice always manages the occurrence frequency information of items for transactions occurring in the window.

b. Frequently searched items by damping rate method

In the open data mining environment, the attenuation rate method can be considered as a method that can reduce the importance of each time point by a certain ratio among the information differentiation methods that differentiate each transaction constituting the data set according to occurrence time in order to obtain frequent items.

The attenuation rate is a value representing the degree of attenuation of the weight according to a certain amount of change in the information differentiation method that differentiates the importance of the past information and the recent information in the data set generated by the time change. In this case, the change amount may be defined based on the time point of the occurrence of the transaction or may be defined as the number of generated transactions. The following description is based on the TID. When the TID is defined as the occurrence time of a transaction, the decay rate is defined by the occurrence time, and when the TID indicates the order of occurrence, the decay rate is defined as the number of transactions. On the other hand, the damping rate is defined by the damping default and the damping fundamental period. Herein, a predetermined time defined by the user is called attenuation unit time. The attenuation default is the default value that defines the attenuation rate, denoted by b. The damping fundamental period is the weight of the current transaction in the data set (= 1) The number of unit hours that elapse until this point is represented by h. When the decay default is defined as b, the maximum possible decay rate for one unit time change is 1 when the decay fundamental period is 1. Obtained by On the other hand, given the attenuation default value b and the attenuation fundamental period h, the attenuation ratio d, which represents the degree of attenuation of the weight according to one unit time change, is defined as follows. When the decay rate d is defined, the actual degree of attenuation represented by one time change is obtained as (1-d).

On the other hand, in the data set to be analyzed in open data mining, the degree of attenuation is defined by considering the amount of time change when the transaction occurrence time (that is, the TID) is not a multiple of the attenuation unit time. For example, if the weight of the current transaction is 1 in a dataset with attenuation rate d defined as 1 hour of decay time (that is, if the time variance is 30 minutes), then the transaction is weighted. Is attenuated by. Hereinafter, in the present invention, the time difference is defined as a value obtained by dividing the actual time change amount by the attenuation unit time (that is, the change amount based on the damping unit time). In general, given a single damping default, as the damping fundamental period increases, the damping rate rapidly approaches the maximum value of 1. Especially when the damping default is small, as the damping fundamental period increases, the damping rate gets closer to the maximum value of 1.

The damping rate method has an overall process similar to the simple accumulation method described in Section 2.1. However, the information such as the frequency of each occurrence and the total number of transactions are attenuated over time by the attenuation rate, which must be taken into account at each stage of execution.

15 shows the frequent item search method 410 by the attenuation rate method. The overall configuration from the transaction data reading step 411 to the frequent item selection step 416 is the same as the frequent item search process by the simple accumulation method. That is, each step is identical to the corresponding step in the frequent item search method by the simple accumulation method described in Section 2.1. However, the attenuation rate method that differentiates the importance of transactions occurring at each time point by the attenuation rate has a difference from the simple accumulation method in the following process.

First, in the mining basic information update step 412, since the information generated at each time point has a weight that is differentiated according to the time of occurrence, the total number of transactions at the current time point ( ) Takes into account the total number of transactions and the decay rate Obtained as Here, Δt represents the time difference between the occurrence of the newly generated transaction and the occurrence of the transaction occurring at the most recent previous time, in multiples of the attenuation unit time. As in the simple accumulation method, the attenuation rate method newly calculates the critical frequency count for adding items and battery operations and the minimum frequency count for determining frequent items in consideration of the change in the number of transactions. At this time, the value expressed as the support is converted into the frequency of appearance in consideration of the total number of transactions.

16 shows frequency update and cell work step 413 in the damping rate method. In the appearance frequency update step 431, the frequency of appearance at the current time of the item ( ) Is the number of occurrences ( ) And taking into account the attenuation factor Is updated to At this time, Δt is for the item appeared in the newly generated transaction, so the difference between the current time and the most recent time that the item occurred is expressed as a multiple of the attenuation unit time. Item cell step 433 removes the item from the monitoring lattice if the updated frequency of occurrence is less than the cell threshold of the item. At this time, the 1-item is basically not carried out. However, as with the one-item cell work in the process of searching for frequent items in the simple accumulation method described in Section 2.1, the number of occurrences of an item if one item does not appear in transactions that have occurred over a significant period of time. Compared to this, the total number of transactions is greatly increased, resulting in very low support. In the basic cell method, even in this case, the 1-item is not celled. However, if you have a very small number of occurrences relative to the total number of transactions, even if it is a 1-item, that item can be passed from Lattice. In this case, a separate threshold value is defined, and if the frequency of occurrence of 1-item is smaller than this, it is transferred from the monitoring lattice. In this case, the threshold which is a reference is referred to as a 1-item battery threshold. The 1-item cell threshold is defined as a degree of support that is less than the n-item (n ≧ 2) cell threshold and can be ignored in the entire dataset.

On the other hand, in the frequent item search by the attenuation rate method, a threshold value that does not affect the mining result at all in setting up delay addition and battery threshold may be defined. That is, for any item, if the item does not appear in a transaction that has occurred for a period of time, the item has very low support. In other words, as time passes or the transaction continues to increase, the number of occurrences of the item decreases to a very low value. When a large amount of time is increased or a large number of additional transactions occur, the reduced support of the item will be small enough that the item will not affect later determining whether the item is frequent. Therefore, if the delay addition and battery threshold are defined as small enough to not affect the frequent item determination, there is no error in the mining result set even if the delay addition and battery operation are performed. However, the values that are small enough to not affect the frequent item search results are actually very small values, and thus the number of items managed by the monitoring lattice will not be significantly reduced even when delayed addition and pruning are performed.

17 and 18 illustrate a forced battery operation performing step 415 and a frequent item selecting step 416 in the frequent item search by the attenuation rate method. The overall process of performing each step is the same as the method of finding frequent items in the simple accumulation method described in Section 2.1. However, in order to determine whether the item is batteryable and the frequent item, the frequency of occurrence of the item should be updated with the latest information by applying the attenuation rate.

3.2 Unit Sequential Pattern Search

In the information differentiation method, the unit sequential pattern search method analyzes the same data set as the unit sequential pattern search method in the simple accumulation method described in Section 2.2, and searches the unit data sequential pattern included in the data set. According to the method of differentiating the importance of composing transaction information, it is divided into two ways as follows.

a. Unit pattern sequential pattern search by window method

The unit item sequential pattern search by the window method is a method of searching the unit item sequential pattern based on a range of transactions that occurred recently based on the current point in time as in the frequent item search in the window method described in Section 3.1. At this time, a certain range defined based on the current time point is called a transaction window. The unit item sequential pattern search method 330 by the window method undergoes the same processing as the frequent item search method by the window method described in Section 3.1. However, in the unit item sequential pattern search, the overlap between the unit items constituting a transaction is allowed, and thus the same unit item may be duplicated in one sequential pattern expressed on the monitoring lattice. In addition, since adding a newly appeared unit item to the monitoring lattice does not go through the process of adding delay, etc., if the occurrence frequency estimation process is not performed, each sequential pattern existing on the monitoring lattice has one occurrence frequency without distinction. Manage only numbers.

b. Unit pattern sequential pattern search by damping rate method

The unit item sequential pattern search by the damping rate method is similar to that of the frequent item search by the damping rate method described in Section 3.1. This is a method for searching a sequential pattern. The unit item sequential pattern search method 430 by the attenuation rate method goes through the same steps as the method for searching a simple accumulated unit item sequential pattern described in Section 2.2. However, there is a difference in that the importance of each information is differentiated by the attenuation rate according to the time of occurrence. That is, the frequency of appearance of the items is updated by applying the attenuation rate in the step of updating the frequency of appearance and adding the sequential pattern, the forced battery operation step, and the step of selecting the unit item sequential pattern among the detailed steps constituting the unit item sequential pattern search method. The detailed attenuation rate application process is the same as for the frequent item search by the attenuation rate method described in Section 3.1.

3.3 Item Sequential Pattern Search

In the information differentiation method, the item sequential pattern search method analyzes the same data set as the method of item sequential pattern search in the simple accumulation method described in Section 2.3, searches for the item sequential pattern included in the data set, and constructs the data set. There are two main ways to differentiate the importance of information:

a. Item Sequential Pattern Search by Window Method

The item sequential pattern search by the window method is a method of searching the item sequential pattern based on a certain range of information that occurred recently from the current time point, similar to the frequent item search method in the window method described in Section 3.1. At this time, a certain range defined based on the current time point is called a transaction window. The item sequential pattern search method 320 by the window method is largely divided into a frequent item search step 321a and a sequential pattern search step 322 as shown in FIG. The frequent item searching step is the same as the frequent item searching method 310 by the window method described in Section 3.1. Similar to the item sequential pattern search in the simple accumulation method, the frequent item set obtained in this step becomes a sequential pattern unit item in each transaction in the next step, the sequential pattern search step. In the sequential pattern search step 322, an item mapping step 322a is performed to map the items obtained in the frequent item search step to the sequential pattern unit items for the sequential pattern search before starting the sequential pattern search. This process is the same as the mapping process in the simple accumulation item sequential pattern search method described in Section 2.3. The transaction reading step 322b for sequential pattern analysis, the update of the frequency of occurrence, the addition of the newly appeared sequential pattern 322c, and the transaction window updating step 322d are the same as the frequent item search process in the window method. In the item sequential pattern selection step 322e, an item sequential pattern having a support value equal to or greater than a threshold value is selected. In the item mapping information analysis step 322f, the item constituting the item sequential pattern is reinterpreted into unit item information constituting the actual data set of the application domain with reference to the mapping information in the item mapping step 322a. Meanwhile, in the sequential pattern search step 322, the mining basic information update operation is not performed. That is, since the update has already been performed once in the frequent item search step 321a, no update operation is performed in the sequential pattern search step. The item sequential pattern search in the window method is composed of a double lattice structure with associative-lattice and sequential-latty as in the simple accumulated item sequential pattern searching method described in Section 2.3. That is, each item in the frequent item search step is maintained in the association-lattice and each sequence pattern for the sequence pattern search is managed in the sequence-lattice. However, in the process of adding the sequential pattern for sequential pattern search to the sequential-lattice, the actual frequency of occurrence is managed without estimating the number of occurrences of the sequential pattern. Rather than managing the number of occurrences of the two types according to the starting position, only one number of occurrences is managed. On the other hand, the method of updating the frequency of appearance of the items and the sequential patterns and the transaction window management method according to the additional time change or transaction are the same as in the frequent item search method in the window method described in Section 3.1.

When one transaction to be processed is generated, the frequent item search step and the sequential pattern search step described above are continuously performed as shown in FIG. 19. That is, first, the frequency of occurrence of items appearing in the transaction in the frequent item search step is updated, and new items and frequent items newly added to the lattice are obtained in this process. Next, we search for frequent sequential patterns by rereading the transaction and analyzing sequential patterns consisting of only these frequent items. At this time, to analyze the sequential information on the transaction, the transaction is maintained in memory until the end of the sequential information analysis. The process after the analysis for sequential information search for a transaction is the same as for the frequent item search method using the window method described in Section 3.1.

b. Item Sequential Pattern Search by Attenuation Ratio Method

Item sequential pattern search by decay rate method is similar to that of frequent item search by decay rate method described in Section 3.1. How to navigate. The item sequential pattern search by the decay rate method goes through the same performance steps as the item sequential pattern search by the simple accumulation method described in Section 2.3. That is, basically, as in the item sequential pattern search by the simple accumulation method, the process shown in FIG. 10 is performed, and is divided into a frequent item search step and a sequential pattern search step. The frequent item search step has the same process as the frequent item search method 410 by the attenuation rate method described in Section 3.1. The frequent item set obtained in this step becomes a sequential pattern unit item defining each transaction in the next step, the sequential pattern search step.

The sequential pattern search step is the same as the sequential pattern search step in the simple accumulated item sequential pattern search method shown in FIG. That is, all processes from the item mapping step for constructing the sequential pattern to the step of interpreting the item mapping information are the same as in the simple accumulated item sequential pattern searching method. However, there is a difference in that the importance of each information is differentiated by the attenuation rate according to the time of occurrence. That is, among the detailed steps constituting the sequential pattern search step, the frequency of appearance of the item is updated by applying the attenuation rate in the update of the frequency of appearance, the addition of the sequential pattern, the forced battery operation, and the selection of the item sequential pattern. The detailed attenuation rate application process is the same as in the frequent item search method by the attenuation rate method described in Section 3.1. In the sequential pattern search method using the decay rate method, the importance of information is reduced by one decay rate in the frequent item search step and the sequential pattern search step according to the passage of time or the addition of a new transaction. That is, the importance of information is differentiated by one decay rate defined by the same decay unit time, decay default value, and decay fundamental period in the frequent item search step and the sequential pattern search step.

4. Open data mining to keep mining result sets separate

In each of the methods described above, a method for selecting a frequent item set or a frequent sequential pattern set should determine whether there are frequent items by analyzing all the information managed by the monitoring lattice at each time point. In this method, the minimum amount of memory required can be minimized by keeping the minimum information necessary to obtain the frequent item set in the open data mining environment. However, since the entire monitoring lattice must be explored at each point in time to obtain frequent items, the time required to obtain the frequent items can be greatly increased if the data set increases over time and the information managed by the monitoring lattice increases. . In this chapter, in order to reduce such time waste, it is necessary to obtain frequent item set by separately tracking frequent item set information at each time point without searching entire monitoring lattice to obtain frequent item set at the present time. This section describes how to save time.

At any time t = k, the information managed by the monitoring lattice The frequent item set obtained at this point Let's do it. Information managed by the monitoring lattice when the viewpoint changes over time and changes to t = k + 1 And frequent item sets May be defined as follows.

Meanwhile, Is the monitoring lattice information consisting of the items from the first transaction. Represents a set of frequent items among items that appear in the first transaction. In addition, ΔMS indicates items appearing in a newly generated transaction at t = k + 1, and updates the information by adding the item change shown in one transaction to the existing item information managed in the monitoring lattice to t = k + 1. Monitoring Lattice manages the information of all transactions that occurred. ΔLS refers to a change in the frequent item set generated by items included in a transaction occurring at t = k + 1. There are two main processes for processing a change included in ΔLS. The first process is the processing of the items included in the newly generated transaction, which is performed simultaneously with the update of the information managed by the monitoring lattice. In other words, Based on the item information that appears in the newly occurred transaction If the corresponding item is identified as a frequent item in the process of configuring a, update the frequent item set as follows. If an item that doesn't exist in your account is newly identified as frequent, add it. If the item exists in, the new frequent item set is updated by updating the frequency of occurrence of the item. Configure The second process is to process items that do not appear in the newly generated transaction among items included in LSk. These items do not change the frequency of appearance, but rather the change in the total number of transactions in the data set. Therefore, in consideration of the minimum support for the data set, it is determined whether these items are frequent items, and it is determined that the items are not frequent items. By removing from Configure

If mining result information is kept separately, the mining process is similar to the mining methods described in Chapters 2 and 3. However, when a new transaction occurs, a process for updating the frequency of occurrence of the item and the updating of the mining result set managed separately at the step of adding a new item should be added. FIG. 21 illustrates a step 213 of updating the frequency of appearance and adding a new item in the method of maintaining a mining result set separately in the simple accumulation frequent item search method. If an item from the newly generated transaction exists in the mining result set, the frequency of occurrence of the item is updated (236b) .If the update of the frequency of one transaction is finished, the mining result set is passed through steps 236c to 236e. Remove items from the mining result set that are no longer frequent items due to transaction growth. On the other hand, if the mining result set is maintained separately, the frequent item selection step 216 is changed to a very simple process. That is, as shown in FIG. 9, support for all items existing in the monitoring lattice is not compared, and all items present in the LS are selected as frequent items without a separate judgment process. In this case, if the frequent items are selected by searching the entire monitoring lattice, the time required for the lattice search is very large but the relatively small size LS is searched, and the execution time is drastically reduced. On the other hand, the structure for managing the LS information can be managed by using a potential-tree lattice structure or as an array structure as in the monitoring lattice. Other methods devised in the present invention can perform data mining to minimize the mining result set by maintaining the mining result set separately by performing the same operation in the frequency update, the addition of new items, and the frequent item selection step. .

In order to minimize the execution time in obtaining the frequent item set at each time point, in order to utilize the frequent item set obtained at the previous time point, in addition to managing information generated from each data set in the monitoring lattice, frequent items obtained at the present time point The set must be managed separately. Maintaining a separate set of frequent items increases memory usage for mining. However, since the execution time for obtaining the frequent item set is reduced, the mining operation can be performed efficiently.

Meanwhile, in the above method, each time a new transaction occurs in the frequent item selection, the frequent item set is updated. If there is no need to obtain the frequent item set every time a new transaction occurs, update the frequent item set only at a point in time when the frequent item set is to be obtained instead of updating the LS for each transaction. Can be done. At this time, the change information of items appearing in the transaction from the last time the update operation on the frequent item set to the current time should be maintained separately. This requires additional memory space and may increase the time required to update the frequent item set if the amount of separately managed information is increased in the process of updating the frequent item set including separately managed information for selecting frequent items. have. Therefore, it is possible to select and apply the LS update every time each transaction occurs and the frequent item set update only when necessary according to the actual application environment. On the other hand, the mining method that maintains the mining result set separately can be applied not only to the frequent item search described in this chapter, but also to the frequent unit item sequential pattern search method and the frequent item sequential pattern search method. It can be applied regardless of the method.

5. Open data mining that processes multiple transactions simultaneously

The open data mining methods devised in Chapters 2, 3, and 4 update the frequency information of items in each transaction unit and select frequent items from each transaction when one transaction occurs at a time. However, in general applications, multiple transactions can occur simultaneously in one application domain. In this case, basic analysis of multiple transactions that occurred at the same time is performed to obtain information on items and transactions that can update the frequency of occurrence information at the same time. It can be processed relatively fast compared to processing one by one independently.

In order to process multiple transactions simultaneously, the transaction reading step of each of the data mining methods of the present invention described in Chapters 2 and 3 and shown in FIGS. 3, 10, 12, 15, and 19 is performed at the same time. It is changed to include an item and a transaction duplication analysis step for a number of transactions that occurred. That is, instead of simply reading one transaction and updating the frequency of occurrence of the item, the number of occurrences of the item appearing in multiple transactions is handled at once. In this case, the processing is partially changed in other steps in the mining process. First, in the mining basic information change step, the methods described previously were incremented by 1 because they are processed by one transaction increase. However, if multiple transactions are processed at the same time, the total number of transactions is increased. That is, the total number of transactions in the changed data set when k transactions are processed at once Is the total number of transactions from the previous point in time from Obtained as Next, even when updating the frequency of occurrence of an item and adding a newly appeared item, in the previously described methods, if the frequency of the item whose frequency of occurrence information is to be changed is changed by 1, but the number of transactions is to be processed simultaneously, The number of occurrences of the item is updated in consideration of the number of times the item actually appears in a plurality of concurrent transactions. This method requires additional space for analyzing the redundancy of items or transactions occurring in transactions to be processed at the same time, but can reduce processing time compared to analyzing each transaction independently.

6. Analysis of changes over time of usability indicators

In the conventional closed data mining method, support and reliability indicating the usefulness of each item represent static values appearing in the data set to be analyzed at the time of mining. In open data mining, however, it is possible to analyze changes in information over time, and more accurately predict changes in future transactions by analyzing changes in the availability indicators in the previously occurring transaction set based on the current point in time. That is, it may be determined whether the usefulness index is increased or decreased with time. The usefulness index is a value representing the usefulness of the items obtained from mining results. , Confidence, conviction, interest, interests, surprisingness, variance, correlation, and similarity.

Curve (a) in Figure 20 is an item over time And the curve (b) shows the change over time. At the time t = k and Indicates the same degree of support for each other. However Support is on the rise Support is on the decline. If judged only by support , Are considered to be of equal importance to each other, but in subsequent transactions over time Support is likely to increase from the current value It can be predicted that the degree of support for is likely to decrease.

If the primary purpose of data mining is to extract feature information from the analysis of historical data, the second purpose is to use the results extracted by the analysis as a basis for predicting or judging future data sets. can do. In the example of FIG. 20 presented above , Determines that they have the same importance, they meet the primary purpose of data mining, but this Have higher importance. In other words, in the prediction of the future, if only the support value at the present time is considered, a correct analysis result cannot be presented. In this case, the degree of support change at each point of time can be supported, and in particular, when changing patterns such as increase or decrease compared to the previous support change rate can be supported, it is possible to support more accurate prediction of changes in the future data set. In order to implement such an analysis in the conventional closed data mining, a separate mining operation must be performed on various data sets for which changes are to be identified, and the differences must be analyzed. However, in open data mining, it is easy to identify the change trend of the mining result set because the mining result that is changed according to the transaction increase can be obtained in real time.

6.1 Velocity and Acceleration in the Simple Accumulation Method

As an indicator for predicting the change over time of the usability indicator, the speed and acceleration of the usability indicator may be considered. The support of the item, which is one of the usefulness indicators, is described, and the following descriptions apply equally to other usefulness indicators such as reliability.

a. speed

If support for one item e changes over time, the speed of support S of item e is defined as the amount of change in support per unit time, and the speed at t = k Represented by Here, the unit time for defining the rate is defined as the rate base time, and may be defined separately from the minimum unit time at which the transaction can occur, but in the present invention, the rate base time is defined as a multiple of the minimum time at which the transaction can occur. . For example, suppose the rate base time is defined as m times the minimum time that a transaction can occur. In this case, the speed of support means the change in support during the change of time by m times the minimum time of the transaction, and is defined as the sum of the change in support during the time. At any point in time t = k , Support velocity at that time Is defined as If k≤m Cannot return a value Is assumed to be zero.

On the other hand, the velocity at an arbitrary time point t = k + 1 Is the velocity at t = k Can be obtained recursively as Meanwhile, silver Is defined as

b. acceleration

For one item e, the acceleration of the support S is defined as the change in velocity base hourly velocity and the acceleration at t = k Represented by As in the velocity definition, assuming that the velocity base time is defined as m times the minimum amount of time that a transaction can occur, the acceleration of the support is defined as the sum of the changes in velocity over that time. At any point in time t = k , The map acceleration at that time Is defined as If k≤m Cannot return a value Is assumed to be zero.

On the other hand, the acceleration at an arbitrary time point t = k + 1 as in the support velocity Is the acceleration at t = k Can be obtained recursively as silver Is defined as

6.2 Velocity and Acceleration in Information Discrimination Methods

Velocity (or acceleration) is defined as a simple arithmetic sum of support (or velocity) changes in a simple accumulation method that defines equal weights regardless of when transactions occur. In the window method, as in the simple accumulation method, it is simply defined as an arithmetic sum. However, if the unit time defining the speed and acceleration is larger than the window size, the speed and acceleration variation may be very large. In the window method, the definition method for the change rate and acceleration of the support is the same as in the simple accumulation method, and thus a separate description is omitted. The damping rate method is defined as follows by using a separate damping rate for defining the velocity in calculating the velocity and acceleration.

a. speed

The rate of change in support in the damping rate method is basically defined as the rate-based hourly change in support as in the simple accumulation method. In this case, the support change amount is defined as the change amount considering the attenuation rate. However, this method is a waste of maintaining a change in support during the speed base time. As a method for preventing this, in the present invention, a separate attenuation rate for defining the speed is defined, and the speed of change of the support is defined based on this. In this case, the speed decay rate dv to define the speed is defined as a separate decay default value, decay unit time, and decay fundamental period, which are different from the basic decay rate for the data set. do. Velocity decay rate is defined as more important information of the recent short intervals, so the change in support calculated for the entire interval by the speed decay rate is very close to the change in support in these intervals. At any point in time t = k When the rate of decay rate is applied, the rate of change of support at that time Is defined as

On the other hand, the velocity at an arbitrary time point t = k + 1 Is the velocity at t = k Can be defined recursively as silver Is defined as

b. acceleration

In the damping rate method, the acceleration of the change in support is basically defined as the speed-based speed change per hour, as in the simple accumulation method. In this case, the speed change amount is defined as the change amount considering the attenuation rate. As with speed, however, this method is a waste of maintaining a separate speed change during the speed base time. As a method for preventing this, in the present invention, the acceleration of the change in the support is defined based on this using a separate attenuation rate for defining the speed. At this time, the damping rate for defining the speed is the same as the damping rate used in the speed definition. At any point in time t = k When the speed decay rate is applied, the acceleration of support change at that time Is defined as

On the other hand, the velocity at any time point t = k + 1 as in the support velocity Is the velocity at t = k Can be defined recursively as silver Is defined as

6.3 Data Mining by Analyzing Changes in Indicators of Usefulness Determination

This section describes mining methods that support the more useful mining results by analyzing the rate of change and the acceleration of the usability indicators. The mining method by change analysis described in this section can be applied to each of the mining methods described in Chapters 2 and 3 and the modified mining methods based on them. This paper describes the data mining method by analyzing the change in the usefulness judgment indicator that analyzes the rate of change of the support of the mining items in the search method. The contents described below may be equally applied to the other methods devised in the present invention, and the same applies to not only the support but also other usefulness determination indicators, and the data mining method by the acceleration analysis is also performed by the speed analysis described in this section. It may be implemented in the same manner as the mining method.

In order to analyze the support rate of items in the simple accumulation frequent items searching method, two items are added or changed as follows in the frequent items searching method 210 by the simple accumulation method shown in FIG. 3. First, update the frequency of occurrence and update the speed of the item with respect to the newly added or existing items on the monitoring lattice in step 213. At this time, the speed of the item can be obtained recursively from the speed at the previous time by the method given in Section 6.2. FIG. 23 shows the frequency update module 237 of adding an item and updating a frequency of adding an item. 24 shows a detailed process of step 273 for analyzing the rate of change of support. Step 273 is a detailed step of the frequent item selection step, in which the item is determined to be a frequent item because the degree of support of the item is greater than or equal to the minimum degree of support for the data set. In Figure 24, the support change rate analysis is largely divided into two methods. One is the method of refining the frequent items by the average rate of time, and the other is the method of refining the frequent items by the average rate of the total frequent items. In the method of resolving frequent items by the average time rate, the ratio of the change rate of the support rate of each item in the situation where the current transaction is processed to the average value of the change rate of the support rate of the corresponding item according to the time change or transaction increase is defined in advance by the user. If it is above a certain threshold, it is a method of selecting it as a refined frequent item in consideration of the change rate of support. According to the method of resolving frequent items by the average rate of total frequent items, the ratio of the rate of change of the support rate of each item to the average value of the rate of change of each item included in the set of frequent items determined by the support only in the current transaction processing state is determined by the user in advance. If it is more than the defined threshold value, it is a method of selecting it as a refined frequent item in consideration of the change rate of support. On the other hand, without a relative comparison between the support rate change of each item, if the support change rate of the item is more than a predetermined threshold value predefined by the user may be selected as a refined frequent items in consideration of the change rate of support.

7. Data Set Preprocessing Techniques for Open Data Mining

In the conventional closed data mining, it is very important to define a data set that best reflects the characteristics of the application domain among the various data sets generated in the application domain as an analysis target. At this time, by preprocessing the data set, the transactions that deviate significantly from the characteristics of the application domain are excluded from the analysis target data set, and the characteristics of the application domain are included by including the transactions that can reflect the characteristics of the application domain well. The mining results can be obtained more clearly. Similarly, open data mining performs preprocessing for transactions that occur continuously to include only transactions that reflect the characteristics of the application domain in the continuously increasing transactions.

In open data mining, transaction information constituting the data set to be analyzed can be variously changed. At this time, by shaping the transaction set by a certain criterion, the reliability of the result obtained by the mining can be increased. For example, the length of a new transaction (the length of a transaction refers to the number of units constituting one transaction) is statistically analyzed by analyzing the length of a transaction in the current data set to be mined. By excluding transactions from the mining analysis target data set, you can obtain mining results that better reflect the common characteristics of the application domain that you are analyzing. In open data mining, the mining results of adding new transactions must be analyzed in real time, so the specificity of comparing newly generated transactions with past transaction sets must be determined before adding the transaction information to the monitoring lattice. In conventional closed data mining, this process is performed in the pre-processing stage before performing the mining operation. In open data mining, this process must be performed at the time of adding each transaction. On the other hand, the following characteristics analysis index and data set characterization method using the same is a method that can be efficiently applied in open data mining. However, these indicators can be effectively used to characterize data sets even in conventional closed data mining. In order to analyze the characteristics of the data set to be analyzed, the following indicators are used in the present invention.

7.1 Dataset Characterization Indicators

a. Average Length

The length of a transaction at any point in time t = k The value indicates the number of unit items that constitute a transaction. In this case, the length of each transaction constituting the entire data set is obtained and statistically analyzed to analyze the value representing the distribution degree of the length such as the average of the transaction length, the standard deviation or the variance, and so on. Analyze common characteristics for

In a constantly changing dataset, if the transactions occurring at each point in time have the same weights, the average length of the transactions in the entire dataset up to a point in time t = k Is the average length of transactions across the entire dataset at t = k + 1. Can be obtained as

In this case, in the decay rate method in which the transactions occurring at each time point have different weights, the average length of the transaction at t = k + 1 may be obtained by considering the decay rate d as follows. Is the same as in the simple accumulation method.

b. Average Coverage

In one transaction, reflectance is a value that indicates the degree to which the transaction reflects the characteristics of the application domain to be analyzed. That is, it is expressed as the ratio of the items included in the frequent item set among the items constituting the transaction. Transactions with high reflectance can be considered to reflect the characteristics of the application domain where the transaction occurred. Conversely, transactions with low reflectance can be regarded as not reflecting the characteristics of the domain. Meanwhile, the reflectance rate is expressed as 1-item reflectance rate, 2-item reflectance rate, ..., n-item reflectance rate, etc. according to the length of the item which is a standard for obtaining the reflectance rate. That is, when the reflectance is calculated based on the n-item whose length is n, it is defined as the n-item reflectance rate. transaction N-item reflectance of May be defined as follows.

The average reflectance is an average reflectance value of each transaction constituting the data set to be analyzed, and may indicate how much the entire data set to be analyzed reflects the characteristics of the application domain. If the transactions occurring at each time point in the entire data set have the same weights, then the average reflectance up to any time point t = k Is the average reflectance at t = k + 1. Can be obtained as here, Of the transaction Indicates the reflectance.

In this case, in the attenuation rate method in which transactions occurring at each time point have different weights, the average reflectance at t = k + 1 may be obtained as follows in consideration of the attenuation rate d. Is the same as in the simple accumulation method.

c. Average Length of Frequent Itemset

Among the items appearing in a transaction, the average length of the frequent items that satisfy the minimum map for the data set is called the empty bill average length. Lower minimum support in one data set increases the number of frequent items and the average length. Frequent average length for Is defined as

LS: frequent item set, | LS | : Number of elements in the frequent item set

The average length of frequent items for the entire data set can be obtained from the average length of frequent items generated in each transaction, and the total data up to any point in time t = k if the transactions occurring at each point in time in the entire data set have the same weights. The average length of frequent items for a set Is the average length of frequent items at t = k + 1. Can be obtained as Here, TLk + 1 represents the average length of frequent items of Tk + 1 of the transaction.

In this case, in the decay rate method in which transactions occurring at each time point have different weights, the average length of a transaction at t = k + 1 may be obtained as follows in consideration of the decay rate d. Is the same as in the simple accumulation method.

d. Average Support of Frequent Itemset

Among the items appearing in a transaction, the average support of frequent items that satisfy the minimum map for the data set is called the empty bill average support. transaction Frequent average support for Is defined as

The average support for the frequent items for the entire data set can be obtained from the average support for the frequent items that occurred in each transaction, and the total data up to any point in time t = k if the transactions occurring at each time point in the data set have the same weight. The average support for frequent items for a set , Mean support for frequent items at t = k + 1 Can be obtained as here Of the transaction Shows the average support for frequent items.

In this case, in the decay rate method in which the transactions occurring at each time point have different weights, the average support of the transaction at t = k + 1 may be obtained by considering the decay rate d as follows. Is the same as in the simple accumulation method.

e. Average Rule-Point of Frequent Itemset

Among the items appearing in a transaction, the average length and the average support were examined as values representing the characteristics of the frequent items that satisfy the minimum map for the data set. In order to represent these two indicators as a composite indicator, the frequent item rule score is considered. In the present invention, a transaction Frequent rule score for Is defined as follows to consider the length and support of frequent items simultaneously.

The average rule score for the frequent items for the entire data set can be obtained from the rule scores for the frequent items that occurred in each transaction, and the total up to any point t = k if the transactions that occur at each point in the data set have the same weights. Frequently averaged rule scores for a dataset The average rule score for frequent items at t = k + 1. Can be obtained as Here, TRPk + 1 represents the average rule score of the frequent items of transaction Tk + 1.

In this case, in the decay rate method in which the transactions occurring at each time point have different weights, the average rule score of the transaction at t = k + 1 may be obtained by considering the decay rate d as follows. Is the same as in the simple accumulation method.

f. Percentage of Reflection at a Interval in a set of Frequent Itemset

When each item is classified by a certain interval based on the support in the frequent item set obtained as a result of mining the transaction that occurred up to a certain point, the value that represents the number of items that occurred in the current transaction among the frequent items corresponding to each interval. It is referred to as the distribution ratio of the support section of the frequent items. The distribution ratio for each support interval can be analyzed by classifying the support interval by not only the number or support of frequent items generated in each transaction, but also the degree to which each transaction reflects the characteristics of a set of frequent items. One transaction Any support interval u defined by a value greater than the minimum map for And maximum support Distribution interval by frequency of support for frequent items) Is defined as

The distribution ratio of the support interval of the frequent items for the entire data set can be obtained from the distribution ratio of the frequent support intervals in each transaction, and the random support interval u (minimum map) defined as a value larger than the minimum map. And maximum support If the transactions occur at each time point in the entire data set have the same weights, then the distribution ratio of frequent items support intervals up to a point in time t = k Where t = k + 1 Can be obtained as here Is a transaction The ratio of the frequent item support distribution in the interval u of.

In this case, in the attenuation rate method in which transactions occurring at each time point have different weights, the distribution ratio for each support interval of the frequent items at t = k + 1 may be calculated as follows by considering the attenuation rate d. Is the same as in the simple accumulation method.

7.2 Use of Characteristic Indicators

The dataset characterization indicators defined in Section 7.1 can be used in the actual mining process as a method of purifying transactions by analyzing the degree of formality and setting the minimum requirements as mining parameters as follows. Can be. This section describes a method for supporting open data mining using characteristic analysis indicators in the simple livestock frequency search method described in Section 2.1. The information described in this section is equally applicable to other data mining methods devised in the present invention.

a. Transaction refinement by analyzing the degree of formality of characteristic analysis indicators

Among the data set characteristic analysis indicators, the degree of uniformity can be defined based on the distribution of the values obtained from the above-described formulas in the mean reflectance rate, the average length of frequent items, the average support of frequent items, and the rule score of frequent items. In other words, by calculating these index values in each transaction unit and calculating the average and variance (or standard deviation) between the transactions, it is possible to know where each transaction constituting the entire data set exists in the corresponding index value. In this way, by considering the position in the distribution of the entire transaction set, it is possible to determine the degree of reflection of the characteristics of the entire transaction set. Based on this, transaction cleansing to decide whether to include the transaction in the data set to be analyzed may be performed.

FIG. 25 illustrates a simple accumulation frequent item searching method for refining transactions to be included in a data set to be analyzed by analyzing characteristic indicator values. Unlike the simple accumulating data frequent items search method, which does not purify transactions in Section 2.1, instead of analyzing all transactions that occur, only the transactions that satisfy certain conditions are included in the analysis target data set through characteristic index analysis. At this time, each transaction obtains a characteristic analysis index value based on a mining result set obtained as a result of processing up to the previous transaction. This process is referred to as transaction purification step 217 by characterization analysis. The mining basic information updating step 212 and updating the frequency of appearance of the item and adding a newly appearing item 213 are the same as the simple accumulation frequent item searching method of Section 2.1. Next, in step 218, the degree of accuracy of the characteristic analysis indicator values is obtained from the entire data set. The result obtained here becomes a standard for refining the transaction when a new next transaction is entered. Subsequent forced cell operations and frequent item selection steps are the same as the existing data mining methods.

b. Mining method by setting minimum value of characteristic analysis indicator

The data set characteristic analysis indicators devised in the present invention do not exist independently of each other, but one indicator value is mutually influenced from other indicator values. For example, in order to increase the reflectance in one data set, it is necessary to lower the minimum map of the data set, which causes changes in the average length of frequent items and the average support for frequent items. In this way, by setting a request value for one index value in the mining operation for the interactive characteristic analysis indexes, other index values can be adapted to this. That is, in the present invention, most of the mining methods define a minimum map for defining frequent items in a data set as a mining parameter and search for frequent items based on the mining parameter. However, in the mining methods described in the present invention, mining is performed by defining reference threshold values for other characteristic analysis indicators, such as an average reflectance rate, an average length of frequent items, an average support of frequent items, and a rule score of frequent items, as a mining parameter. Can be obtained.

8. Mining system capable of searching frequent items and frequent patterns

The open data mining system built on the open data mining concepts and methods described above supports obtaining the characteristic information appearing in the data set to be analyzed from a continuously increasing data set within a limited working time.

Open data mining systems support the dynamic change of usability indicator constraints for mining result sets. That is, the minimum support, minimum confidence, minimum correlation, and minimum interest for each item constituting the result set are dynamically changed based on the mining result at each time point. This dynamic change in usability indicators allows for more efficient analysis of the nature of application domains that change over time.

On the other hand, in the simple accumulation method and the decay rate application method, it is basically determined whether or not the information that appears in the transaction is added to the frequency update operation and monitoring lattice. In addition, pruning from monitoring lattice is only possible for items that appear in a transaction. This can unnecessarily increase the number of items present in the monitoring lattice. Therefore, each method is designed to perform forced cell work. However, each time you perform a forced cell operation at each point in time, it can be a waste of time, especially if you are forced to perform the same cell operation by the same processor that performs the mining operation. have. In open data mining systems, a separate processor is designed to remove batteryable items from those present on the monitoring lattice. In this way, a separate processor that removes unnecessary items present in the monitoring lattice is defined as an idle item collector. The idle item collector is driven separately from the mining task processor and is repeatedly executed at regular time intervals. This can reduce memory usage by eliminating unnecessary entries in the monitoring lattice without increasing the time required for actual mining.

Through the open data mining method devised in the present invention, frequent items or frequent sequential patterns can be quickly obtained by analyzing a non-limiting data set composed of transactions continuously generated over time. In other words, the mining operation time is reduced for data sets whose analysis data sets are not clearly defined before the start of mining, and are not limited and continuously changed. This shortening of mining operation time for non-limiting datasets enables the analysis of datasets that occur continuously in the application domain in real time. In particular, the time required for frequent item sequential pattern search is significantly reduced. Such a mining method can minimize the analysis time required to obtain mining results not only in open data mining where the data set is continuously increased, but also in a conventional closed data mining environment in which the data set to be analyzed is clearly defined in advance.

In general, when the data set to be analyzed is continuously increasing, the memory usage required during the mining operation is rapidly increased. The delay adding method of the present invention reduces the amount of memory required for mining by reducing the amount of information managed in the memory during the mining process, and supports the same mining result as in the conventional mining method. In addition, by estimating the number of occurrences of the item to be delayed added from the subitems of the item, the number of occurrences of the item added to the delayed item can be approximated to the actual value.

On the other hand, the information differentiation method devised in the present invention efficiently differentiates the importance of each information from the data set continuously generated over time according to the time of occurrence, while effectively highlighting the importance of recent information and efficiently sharing common features of the entire data set. Support for extraction. This method has the effect of reducing the size of the data set to be analyzed, and can also increase the adaptability to the information changes in the data set when the data set to be analyzed is greatly increased.

In the present invention, the usefulness indicators indicating the usefulness of items appearing in the data set continuously increasing over time are analyzed as well as the static values at a specific time point for obtaining a mining result as well as the change in the usability indicator over time. . By analyzing the changes over time for each usability indicator, it is possible to predict that the items with the same value at a certain time will change the usability indicator values in later transactions if the values change over time. This allows more detailed analysis of the items that make up the data set. In addition, in the present invention, the transaction data generated in the application domain selects only the transaction data that satisfies a predetermined condition based on the characteristic value of the transaction as the analysis target. Through this, mining can be performed on only data that reflects the characteristics of the application domain, thereby increasing the reliability of the mining results.

Claims (113)

As a new transaction is created for a non-limiting dataset consisting of transactions that occur continuously in the application domain, the transaction is read only once to perform data mining to obtain information embedded in the increased dataset that contains the transaction. Data mining system department to analyze a) a transaction data reading detail module unit for reading the newly generated analysis target transaction data and analyzing the item information included in the transaction; b) a frequency count management detailed module unit for gradually managing the frequency information of items generated in a newly created transaction using a monitoring lattice; c) a mining result selection module for analyzing the frequency of occurrence of items managed in a monitoring lattice to find frequent items or frequent patterns; d) A data mining system comprising a detailed module for presenting mining results for presenting frequent items or frequent sequential patterns to a user. The method of claim 1, Data mining system characterized in that each detailed module is performed to analyze the information inherent in the data set, considering that transactions to be analyzed are not always different from each other in importance due to time change or transaction increase. The method of claim 2, A data mining system, characterized in that each detailed module unit is performed to analyze the frequency count information of items to search for an item whose support degree is equal to or greater than a predetermined threshold defined by the user. The method of claim 3, In order to provide a mining result in a short time when time changes or new transactions are increased, it maintains the frequency information of all items that appeared in all transactions that occurred before, without re-analyzing the transactions included in the entire data set. A data mining system, characterized in that the appearance frequency count management detail module unit and the mining result selection detail module unit are performed so as to obtain a mining result by analyzing only the added transaction. The method of claim 4, wherein A) The number of items to be managed increases rapidly as the number of items to be managed rapidly increases as transactions increase rapidly, so managing the frequency information of items occurring in all transactions increases the limit of available memory space and increases the mining processing time. It is impossible due to physical limitations such as, etc. When items appearing in a transaction occur repeatedly and have high support enough to affect the determination of frequent items, memory items can be used by adding the item to the monitoring lattice to manage the frequency of appearance. Delay with additional module to reduce the b) a frequency count estimating module unit for estimating the frequency of occurrence of delayed addition items from the frequency of occurrence of subitems of the corresponding item that appeared in a previously generated transaction. The method of claim 4, wherein The frequency count management detail module unit removes the item from the monitoring lattice when the support frequency is low enough because the item does not appear in a transaction that occurred for a certain period of time among the items whose frequency count is managed in the monitoring lattice. Data mining system comprising a battery operation module unit for reducing the memory space usage required for the mining process by performing. The method of claim 4, The delay adding step of the frequency count management detailed module of the frequency count management detailed module of the item frequency management structure and the estimation error of the items for separately managing the estimated errors generated in the process of estimating the frequency of occurrence for the delay addition as well as the frequency information of the items And the mining result selection step in consideration of estimated error information, thereby limiting the error included in the frequent items within a predetermined range defined by the user, thereby increasing the reliability of the mining result. The method of claim 2, And analyzing each occurrence order information between the unit items, and performing each detailed module unit to search for a sequential pattern of unit items appearing above a user defined threshold value. The method of claim 8, In order to provide mining results in a short time when time changes or new transactions are increased, the frequency counts of all the sequential patterns that appeared in all previously occurring transactions are maintained, without reanalyzing the transactions included in the entire data set. A data mining system, characterized in that the occurrence frequency count management detailed module unit and the mining result selection detailed module unit are performed to obtain a mining result by analyzing only a newly added transaction. The method of claim 9, A) The number of sequential patterns to be managed rapidly increases as the number of transactions is rapidly increased, so managing information on the number of sequential patterns of sequential patterns occurring in all transactions is limited in available memory space and mining processing time. Since it is impossible due to physical limitations such as the increase of the number of times, when the sequence patterns appearing in the transaction occur repeatedly and have a large enough support to affect the determination of the unit item sequence pattern, the sequence pattern is added to the monitoring lattice to manage the frequency of appearance. Adds a delay module to reduce memory space usage by starting b) a data mining system comprising a frequency count estimating module unit for estimating the frequency of occurrence of delayed addition of sequential patterns from the frequency of occurrence of partial sequential patterns of a corresponding sequential pattern that appeared in a previously generated transaction The method of claim 9, The frequency count management detailed module removes the sequential pattern from the monitoring lattice when the support pattern has a sufficiently low value because the sequential pattern does not appear in a transaction occurring for a long period of time among the sequential patterns managed by the monitoring lattice. A data mining system comprising a battery operation module unit for reducing the amount of memory space required for the mining process by performing a battery operation. The method of claim 9, When using the partial sequence pattern to estimate the frequency of occurrence of the sequence pattern, to select the partial sequence pattern in consideration of the sequential information. Items include a data mining system comprising a dual frequency management structure that manages two frequency information according to the start position of the corresponding sequential pattern. The method of claim 9, Adding delay in the detail module for managing the frequency of appearance pattern management and estimation error for managing sequential patterns separately for each sequential pattern. Data mining system characterized in that the reliability of the mining results can be improved by limiting the errors included in the sequential patterns of the unit item within a predetermined range defined by the user in consideration of the estimated error information in the step of selecting the mining result. . The method of claim 2, And analyzing each occurrence order information among items to perform a detailed module unit to search for an item sequential pattern appearing above a user-defined threshold value. The method of claim 14, In order to provide mining results in a short time when time changes or new transactions are increased, the frequency counts of all the sequential patterns that appeared in all previously occurring transactions are maintained, without reanalyzing the transactions included in the entire data set. A data mining system, characterized in that the occurrence frequency count management detailed module unit and the mining result selection detailed module unit are performed to obtain a mining result by analyzing only a newly added transaction. The method of claim 14, One lattice manages information on the frequency of occurrence of items for the search for frequent items with a frequency higher than a certain threshold, and on the other, analyzes the ordered information of items obtained as frequent items and analyzes the frequency of occurrence higher than the corresponding threshold. A data mining system comprising: a dual lattice structure for managing the frequency of appearance of the sequential pattern in order to search for a sequential pattern having a number; The method of claim 16, In order to efficiently use items with higher support than the predetermined threshold in the occurrence frequency count management module, a separate sequential pattern unit item identifier is newly assigned when the item is created and the sequential pattern is searched in the sequential pattern search step. When the unit item is removed, it terminates the connection relationship between the item and the sequential pattern unit item identifier, thereby including a sequential pattern unit item identifier mapping module unit for supporting the items appearing in the new transaction to be mapped to the sequential pattern unit item. Data mining system characterized by. The method of claim 15, A) The number of sequential patterns to be managed increases rapidly as the number of transactions is rapidly increased, so managing the frequency information of sequential patterns occurring in all transactions is limited in the available memory space and mining processing time. Since it is impossible due to physical limitations such as the increase of number, when the sequential patterns appearing in the transaction occur repeatedly and have big support enough to affect the item sequential pattern determination, the sequential pattern is added to the monitoring lattice to manage the frequency of occurrence. A delay addition module section for reducing memory space usage by starting b) a frequency count estimating module for estimating the number of occurrences of delayed addition of the sequential patterns from the number of occurrences of partial sequential patterns of the corresponding sequential patterns that appeared in a previously generated transaction. Data mining system comprising a. The method of claim 15, The frequency count management detailed module removes the sequential pattern from the monitoring lattice when the support pattern has a sufficiently low value because the sequential pattern does not appear in a transaction occurring for a long period of time among the sequential patterns managed by the monitoring lattice. A data mining system comprising a battery operation module unit for reducing the amount of memory space required for the mining process by performing a battery operation. The method of claim 15, When using the partial sequence pattern to estimate the frequency of occurrence of the sequential pattern, in order to select the partial sequential pattern in consideration of the sequential information, the frequency of each occurrence in the monitoring lattice for managing the frequency of occurrence of the sequential pattern in the detailed module The sequence pattern items are data mining systems comprising an appearance frequency double management structure that manages two occurrence frequency information according to the start position of the sequence pattern. The method of claim 15, Adding delays to the appearance frequency management submodule for the sequential pattern number management structure and estimation error that separately manages the estimation errors generated in the process of estimating the frequency of occurrence to add delay as well as the frequency information of sequential patterns A data mining system characterized in that the reliability of the mining result can be increased by limiting the error included in the item sequential pattern within a predetermined range defined by the user by considering the estimated error information in the step of selecting the mining result. The method of claim 1, The data mining system part by the window method redefines the transaction window defining the search range based on the present time and changes the window among the transactions included in the window from the previous time when the current time is changed due to a time change or an additional transaction occurs. It includes a transaction window update module that updates the frequency information of the items managed in the monitoring lattice for items appearing in transactions that are not included in the transaction, and differentiates the importance of each transaction according to time changes or transaction growth. A data mining system characterized by analyzing information inherent in a data set within a specified range by performing each detailed module with respect to a transaction or a certain number of transactions occurring at a predetermined time defined by a user. The method of claim 22, A data mining system, characterized in that each detailed module unit is performed to analyze the frequency count information of items to search for an item whose support degree is equal to or greater than a predetermined threshold defined by the user. The method of claim 23, wherein In order to provide a mining result in a short time when time changes or new transactions are increased, it maintains the frequency information of all items that appeared in all transactions that occurred before, without re-analyzing the transactions included in the entire data set. A data mining system, characterized in that the appearance frequency count management detail module unit and the mining result selection detail module unit are performed so as to obtain a mining result by analyzing only the added transaction. The method of claim 22, A data mining system, characterized in that each sub-module is performed to analyze the sequence information between the unit items and to search for the sequential pattern of the unit items appearing above a user defined threshold. The method of claim 25, Re-analyze the transactions included in the entire data set by maintaining the frequency count of all unit sequential patterns from all previously occurring transactions to provide mining results in a short time when time changes or new transactions increase. The data mining system, characterized in that the appearance frequency number management detailed module unit and the mining result selection detailed module unit are performed to obtain a mining result by analyzing only a newly added transaction. The method of claim 22, A data mining system, characterized in that each detailed module is performed to analyze occurrence order information between items to search for an item sequential pattern that appears above a user-defined threshold. The method of claim 27, One lattice manages the frequency information of items for searching for items with a frequency higher than a certain threshold value, and the other, analyzes the order information of items obtained as frequent items and analyzes the frequency of occurrences higher than the threshold value. The data mining system, characterized in that it comprises a dual lattice structure for managing the frequency of appearance of the sequence pattern management unit to search for a sequential pattern having a sequence pattern. The method of claim 27, In order to efficiently utilize frequent items with higher support than a certain threshold in the sequential pattern search process, the frequency module management detail module newly assigns a separate sequential pattern unit item identifier at the time of item creation and applies the sequential pattern in the sequential pattern search step. When the unit item is removed, it terminates the connection relationship between the item and the sequential pattern unit item identifier, thereby including a sequential pattern unit item identifier mapping module unit for supporting the items appearing in the new transaction to be mapped to the sequential pattern unit item. Data mining system characterized by. The method of claim 27, In order to provide mining results in a short time when time changes or new transactions are increased, the frequency counts of all the sequential patterns that appeared in all previously occurring transactions are maintained, without reanalyzing the transactions included in the entire data set. A data mining system, characterized in that the occurrence frequency count management detailed module unit and the mining result selection detailed module unit are performed to obtain a mining result by analyzing only a newly added transaction. The method of claim 1, The data mining system part by the reduction rate method: a) attenuation rate, which indicates the degree of decrease in importance of a transaction that occurs whenever a unit time or a number of unit transactions elapses based on a decay unit time defined by a user, a decay default value, and a decay fundamental period. Attenuation factor definition module for defining b) In the damping rate method of updating the frequency of occurrence of previously generated items by applying the attenuation rate when the current time is changed due to a time change or the occurrence of additional transactions, including an item frequency update module. A data mining system characterized by analyzing the information inherent in a data set within a specified range by performing each detailed module by differentiating the importance of each transaction according to time change or transaction increase by attenuation rate based on a predetermined unit. The method of claim 31, wherein A data mining system, characterized in that each detailed module is performed to analyze the frequency count information of items so as to search for an item whose degree of support is equal to or greater than a predetermined threshold defined by the user. The method of claim 32, In order to provide a mining result in a short time when time changes or new transactions are increased, it maintains the frequency information of all items that appeared in all transactions that occurred before, without re-analyzing the transactions included in the entire data set. A data mining system, characterized in that the appearance frequency count management detail module unit and the mining result selection detail module unit are performed so as to obtain a mining result by analyzing only the added transaction. The method of claim 33, A) The number of items to be managed increases rapidly as the number of items to be managed rapidly increases as transactions increase rapidly, so managing the frequency information of items occurring in all transactions increases the limit of available memory space and increases the mining processing time. It is impossible due to physical limitations such as, etc. When items appearing in a transaction occur repeatedly and have high support enough to affect the determination of frequent items, memory items can be used by adding the item to the monitoring lattice to manage the frequency of appearance. Delay with additional module to reduce the b) a frequency count estimating module unit for estimating the frequency of occurrence of delayed addition items from the frequency of occurrence of subitems of the corresponding item that appeared in a previously generated transaction. The method of claim 33, The frequency count management detail module unit removes the item from the monitoring lattice when the support frequency is low enough because the item does not appear in a transaction that occurred for a certain period of time among the items whose frequency count is managed in the monitoring lattice. Data mining system comprising a battery operation module for reducing the memory space usage required for the mining process. The method of claim 33, Delay addition step of the frequency count management detailed module for the frequency count management structure and the estimation error of the items for separately managing the estimated errors generated in the process of estimating the frequency of occurrence for adding delay as well as the frequency count information of the items. And the mining result selection step in consideration of estimated error information, thereby limiting the error included in the frequent items within a predetermined range defined by the user, thereby increasing the reliability of the mining result. The method of claim 31, wherein Data mining system characterized in that each module module is executed to analyze the order of occurrence between the unit items and search for the sequential pattern of unit items that appeared above the user-defined threshold. The method of claim 37, wherein In order to provide mining results in a short time when time changes or new transactions are increased, the frequency counts of all the sequential patterns that appeared in all previously occurring transactions are maintained, without reanalyzing the transactions included in the entire data set. A data mining system, characterized in that the occurrence frequency count management detailed module unit and the mining result selection detailed module unit are performed to obtain a mining result by analyzing only a newly added transaction. The method of claim 38, A) The number of sequential patterns to be managed rapidly increases as the number of transactions is rapidly increased, so managing information on the number of sequential patterns of sequential patterns occurring in all transactions is limited in available memory space and mining processing time. Since it is impossible due to physical limitations such as the increase of the number of times, when the sequence patterns appearing in the transaction occur repeatedly and have a large enough support to affect the determination of the unit item sequence pattern, the sequence pattern is added to the monitoring lattice to manage the frequency of appearance. Adds a delay module to reduce memory space usage by starting b) a frequency count estimating module unit for estimating the frequency of occurrence of delayed addition of the sequence patterns from the frequency of occurrence of partial sequence patterns of the sequence pattern that appeared in a previously generated transaction. The method of claim 38, The frequency count management detailed module removes the sequential pattern from the monitoring lattice when the support pattern has a sufficiently low value because the sequential pattern does not appear in a transaction occurring for a long period of time among the sequential patterns managed by the monitoring lattice. A data mining system comprising a battery operation module unit for reducing the amount of memory space required for the mining process by performing a battery operation. The method of claim 38, In order to select the partial sequential pattern in consideration of the sequential information in utilizing the partial sequential pattern to estimate the frequency of occurrence of the sequential pattern, the sequential pattern in the frequency count management detailed module of each module in the monitoring lattice The pattern mining data mining system includes an appearance frequency double management structure that manages two occurrence frequency information divided according to a start position of a corresponding sequential pattern. The method of claim 38, Select the mining result and manage the sequential pattern frequency count management structure and estimation error, which separately manages the estimated errors generated in the process of estimating the frequency of appearance for delay addition as well as the frequency count information of the sequential patterns Data mining system, characterized in that the reliability of the mining results can be increased by limiting the error included in the item sequential pattern within a predetermined range defined by the user in consideration of the estimated error information in the step. The method of claim 31, wherein A data mining system, characterized in that each detailed module unit performs an analysis of order information between items to search for an item sequential pattern that appears above a user-defined threshold. The method of claim 43, In order to provide mining results in a short time when time changes or new transactions are increased, the frequency counts of all the sequential patterns that appeared in all previously occurring transactions are maintained, without reanalyzing the transactions included in the entire data set. A data mining system, characterized in that the occurrence frequency count management detailed module unit and the mining result selection detailed module unit are performed to obtain a mining result by analyzing only a newly added transaction. The method of claim 44, One lattice manages the frequency information of items for searching for items with a frequency higher than a certain threshold value, and the other, analyzes the order information of items obtained as frequent items and analyzes the frequency of occurrences higher than the threshold value. In order to search for a sequential pattern having a frequency count management detailed module unit comprises a dual lattice structure for managing the frequency of appearance of the sequence pattern mining system. The method of claim 44, In order to efficiently utilize items having higher support than the predetermined threshold in the sequential pattern search process, the occurrence frequency count management detailed module allocates a new sequential pattern unit item identifier at the time of item creation and in the sequential pattern search step When the item is removed, the connection pattern between the item and the sequential pattern unit item identifier is terminated, thereby including a sequential pattern unit item identifier mapping module for supporting the items appearing in a new transaction to be mapped to the sequential pattern unit item. Data mining system. The method of claim 44, A) The number of sequential patterns to be managed rapidly increases as the number of transactions is rapidly increased, so managing information on the number of sequential patterns of sequential patterns occurring in all transactions is limited in available memory space and mining processing time. Since it is impossible due to physical limitations such as the increase of number, when the sequential patterns appearing in the transaction occur repeatedly and have big support enough to affect the item sequential pattern determination, the sequential pattern is added to the monitoring lattice to manage the frequency of occurrence. A delay addition module section for reducing memory space usage by starting b) a frequency count estimating module for estimating the number of occurrences of delayed addition of the sequential patterns from the number of occurrences of partial sequential patterns of the corresponding sequential patterns that appeared in a previously generated transaction. Data mining system comprising a. The method of claim 44, The frequency count management detailed module removes the sequential pattern from the monitoring lattice when the support pattern has a sufficiently low value because the sequential pattern does not appear in a transaction occurring for a long period of time among the sequential patterns managed by the monitoring lattice. A data mining system comprising a battery operation module unit for reducing the amount of memory space required for the mining process by performing a battery operation. The method of claim 44, In order to select the partial sequential pattern in consideration of the sequential information in using the partial sequential pattern to estimate the frequency of occurrence of the sequential pattern, the frequency of each occurrence in the monitoring lattice for managing the sequential pattern frequency of the submodule The sequence pattern items include a frequency count dual management structure that manages the frequency number information having the frequency number information divided into two types according to the start position of the corresponding sequence pattern. The method of claim 44, Adding delays to the appearance frequency management submodule for the sequential pattern number management structure and estimation error that separately manages the estimation errors generated in the process of estimating the frequency of occurrence to add delay as well as the frequency information of sequential patterns A data mining system characterized in that the reliability of the mining result can be increased by limiting the error included in the item sequential pattern within a predetermined range defined by the user by considering the estimated error information in the step of selecting the mining result. The method of claim 1, Step-by-Step Information Separately Managed Data Mining System maintains a separate mining result set obtained by performing mining to a previous time point or a previous transaction, so that items appearing in newly added transactions without searching the entire monitoring lattice to obtain a mining result set. Data mining system, characterized in that the mining result set selection detail module obtains the mining result by maintaining the mining result set separately so that the mining result of the entire data set can be obtained by analyzing only the appearance information of the data and reflecting the result in the mining result. The method of claim 1, Open data mining system part by processing multiple transactions simultaneously Performs mining to obtain mining results by performing basic analysis on multiple transactions occurring at the same time and updating the frequency of occurrence of items appearing in the transactions at once in monitoring lattice Data mining system characterized by reducing the time The method of claim 1, The data mining system unit according to the analysis of useful indicator changes includes a) selecting the mining result in consideration of the change rate of the current support of the item or the sequential pattern. Analyzes the rate of change of support indicators, reliability, persuasion, interest, severity, distribution, correlation, and similarity to indicate the usefulness of the item in the data set as well as the future data set. Data mining system, characterized in that the mining result set is obtained by analyzing the rate of change of the usefulness judgment indicator to support the predictable change of the indicator The method of claim 1, a) Supporting, reliability, persuasion, indexes indicating the usefulness of items obtained by mining results, including the mining result selection module through the support change acceleration analysis that selects mining results in consideration of the current support change acceleration of an item or sequential pattern. Analyze the acceleration of changes in interest, hardness, distribution, correlation, and similarity to predict the change in the rate of change of the corresponding usefulness indicator of the item in the future data set, as well as analysis of information on the data set up to now. Data mining system, characterized in that the mining result set is obtained by supporting the analysis of the usability indicator indicator change acceleration The method of claim 1, Data mining system part based on data collection characteristic analysis is used to refine the data set to be mined by using data set characteristic analysis index values, so that the data mining result reflects the characteristics of the application domain better. Data mining system comprising a characterization and purification module unit. The method of claim 1, And a mining result set selection detail module unit to obtain a mining result in consideration of a minimum reference value for a data set characteristic analysis indicator value defined by a user. The method of claim 1, The integrated system unit including the idle item collector reuses the memory space occupied by the corresponding item or the sequential pattern for the item or the sequential pattern that is no longer needed to manage the frequency of occurrence of the item or the sequential pattern during the mining process. Data mining system comprising a pause item collection module for allocating as possible space. As a new transaction is created for a non-limiting dataset consisting of transactions that occur continuously in the application domain, the transaction is read only once to perform data mining to obtain information embedded in the increased dataset that contains the transaction. How to analyze data mining a) transaction data reading method that reads newly created transaction target transaction data and analyzes item information included in transaction; b) a frequency count management method for gradually managing frequency of occurrence information of items generated in a newly created transaction using a monitoring lattice; c) a method of selecting mining results by analyzing the frequency of occurrence of items managed in a monitoring lattice to find frequent items or frequent patterns; d) a data mining method comprising a method for presenting a mining result for presenting a frequent item or a frequent sequential pattern to a user. The method of claim 58, Data mining method that analyzes the information inherent in the data set by performing each sub-step, considering that transactions that are the analysis target by the simple accumulation method are always the same without being different in importance due to time change or transaction increase. The method of claim 59, A method for mining data, characterized in that each sub-module is performed to search for items whose degree of support is greater than or equal to a predetermined threshold defined by the user by performing each step for analyzing the frequency information of the items. The method of claim 60, In order to provide a mining result in a short time when time changes or new transactions are increased, it maintains the frequency information of all items that appeared in all transactions that occurred before, without re-analyzing the transactions included in the entire data set. A data mining method, characterized in that the method of managing the frequency of occurrence and the method of selecting a mining result are performed to obtain a mining result by analyzing only the added transaction. 62. The method of claim 61, In the method of managing the frequency of occurrence, a) a data mining method comprising a method of estimating the frequency of occurrence of estimating the frequency of occurrence of items to be delayed from the frequency of occurrence of all sub-items of the corresponding item that appeared in a previously generated transaction. or b) a method of estimating the frequency of occurrence of estimating the frequency of occurrence of items to be delayed by only the frequency of occurrence of the (n-1) -sub-items of the item that appeared in a previously generated transaction; c) As the number of items to be managed rapidly increases as transactions increase rapidly, managing information about the frequency of occurrence of items that occur in all transactions is due to physical limitations such as available memory space limitations and increased mining processing time. If the items appearing in the transaction are repeatedly generated and have high support enough to affect the determination of frequent items, the delayed addition method that adds the items to the monitoring lattice and starts to manage the frequency of occurrences will reduce the memory space usage. Data mining method comprising a. 62. The method of claim 61, In the method of managing the frequency of occurrence, the task of removing the item from the monitoring lattice when the frequency of support is small enough because the item does not appear in the transaction that occurred for a considerable period of time among the items managed by the monitoring lattice. And a battery operation method for reducing memory space usage required for the mining process. 62. The method of claim 61, In addition to the information on the frequency of occurrence of the items, the error in the estimation of the frequency of occurrence for adding delays is managed separately from the delay adding step for each item, and the error included in the frequent items is selected by considering the estimation error information in the mining result selection step. Data mining method characterized in that the confidence in the mining results can be increased by limiting within a predetermined range defined by the user. The method of claim 59, A method of mining data, characterized in that each detailed method is performed to analyze the order of occurrence between unit items and to search for a sequential pattern of unit items that appears above a user-defined threshold. 66. The method of claim 65, In order to provide mining results in a short time when time changes or new transactions are increased, the frequency counts of all the sequential patterns that appeared in all previously occurring transactions are maintained, without reanalyzing the transactions included in the entire data set. A data mining method, characterized in that a method of managing the frequency of occurrence and a method of selecting a mining result are performed to analyze only newly added transactions and provide a mining result. The method of claim 66, In the method of managing the frequency of appearance, a) a method of estimating the frequency of occurrence of estimating the frequency of occurrence of delayed addition of sequential patterns from the frequency of occurrence of all partial sequential patterns of the corresponding sequential patterns that appeared in a previously generated transaction; Data mining methods or b) Data mining including a method of estimating the frequency of occurrence of estimating the number of occurrences of delayed addition of sequence patterns by only the number of occurrences of (n-1) -partial sequence patterns of the corresponding sequence pattern that appeared in a previously generated transaction. How to c) As the number of sequential patterns managed to increase rapidly as transactions increase rapidly, managing information on the number of occurrences of sequential patterns occurring in all transactions is limited by physical limitations such as available memory space and increase in mining processing time. Since the sequential patterns appearing in the transaction are repeatedly generated and have a large enough support to affect the determination of the unit sequential pattern, the sequential patterns are added to the monitoring lattice to start managing the frequency of occurrence of memory space. A data mining method comprising the method of adding delay to reduce. The method of claim 66, In the method of managing the frequency of occurrence, the sequence is removed from the monitoring lattice when the support has a value that is small enough because the sequence does not appear in the transaction that occurred for a long period of time in the monitoring lattice. And a battery operation method for reducing the memory space usage required for the mining process by performing the battery operation. The method of claim 66, In addition to the information on the number of occurrences of sequential patterns, the estimation errors generated in the process of estimating the frequency of occurrence for delay addition are managed separately from the delay addition step for each sequential pattern. A data mining method for a simple accumulated unit item sequential pattern search considering a maximum tolerance, wherein the reliability of the mining result can be increased by limiting the included error within a predetermined range defined by a user. The method of claim 59, Based on the dual lattice structure, data mining for simple accumulation item sequential pattern searching is performed by analyzing each item occurrence order information to search for an item sequential pattern that appears above a user-defined threshold. Way. The method of claim 70, In order to provide mining results in a short time when time changes or new transactions are increased, the frequency counts of all the sequential patterns that appeared in all previously occurring transactions are maintained, without reanalyzing the transactions included in the entire data set. A data mining method comprising a simple accumulated real-time item sequential pattern searching method, characterized in that a frequency count management method and a mining result selection method are performed to obtain a mining result by analyzing only a newly added transaction. The method of claim 71, wherein In order to efficiently utilize items with higher support than a certain threshold in the method of managing the frequency of occurrence, a new sequential pattern unit item identifier is newly assigned when creating an item, and the sequential pattern unit is used in the sequential pattern search step. When the item is removed, the method includes mapping the sequence pattern unit item identifier mapping method to terminate the connection relationship between the item and the sequence pattern unit item identifier so that items appearing in a new transaction can be used when the items are mapped to the sequence pattern unit item. Data mining method. The method of claim 71, wherein In the method of managing the frequency of appearance, a) a method of estimating the frequency of occurrence of estimating the frequency of occurrence of delayed addition of sequential patterns from the frequency of occurrence of all partial sequential patterns of the corresponding sequential patterns that appeared in a previously generated transaction; Data mining methods or b) Data mining including a method of estimating the number of occurrences of estimating the number of occurrences of delayed addition of the sequential patterns only by the number of occurrences of the (n-1) -partial sequential patterns of the corresponding sequential patterns that appeared in a previously generated transaction. How to c) As the number of sequential patterns managed to increase rapidly as transactions increase rapidly, managing information on the number of occurrences of sequential patterns occurring in all transactions is limited by physical limitations such as available memory space and increase in mining processing time. Since the sequential patterns appearing in the transaction are repeatedly generated and have a large enough support to affect the determination of the sequential patterns, the memory space usage is reduced by adding the sequential patterns to the monitoring lattice to start managing the frequency of occurrence. Data mining method comprising a delay adding method The method of claim 71, wherein In the method of managing the frequency of occurrence, the sequence is removed from the monitoring lattice when the support has a value that is small enough because the sequence does not appear in the transaction that occurred for a long period of time in the monitoring lattice. And a battery operation method for reducing the memory space usage required for the mining process by performing the battery operation. The method of claim 71, wherein In addition to the information on the number of occurrences of sequential patterns, the estimation errors generated in the process of estimating the frequency of occurrence for delay addition are managed separately from the delay addition step for each sequential pattern, and included in the item sequential pattern considering the estimation error information in the mining result selection step. Data mining method characterized in that to increase the reliability of the mining results by limiting the error within a certain range defined by the user The method of claim 58, In the window method step: a) time-transactions or transactions, including methods of differentiating the importance of transactions at the time of occurrence, ignoring information appearing in previously occurring transactions if only a transaction or a certain number of transactions that occurred in recent time have significance. Differentiate the importance of each transaction according to the increase, and perform the steps for the transaction or the number of transactions that occurred at the latest fixed time defined by the user, and analyze the information inherent in the data set within the specified range. Data mining method. 77. The method of claim 76, Performing each step to analyze the frequency of occurrence of items information, each sub-module is performed to search for items whose degree of support is above a certain threshold defined by the user 78. The method of claim 77 wherein If the current point of view changes due to a time change or an additional transaction occurs, the transaction window defining the search range is redefined based on the present, and it appears in the transactions not included in the changed window among the transactions included in the window at the previous point in time. All previous occurrences to provide mining results in a short time when time changes or new transactions increase, including a transaction window update method that updates the frequency information of the items managed in the monitoring lattice for one item. Frequency count management method and mining to maintain mining frequency information of all items appearing in a transaction so as to provide mining results by analyzing only newly added transactions without re-analyzing the transactions included in the entire data set Data mining method, characterized in that the selected method is performed 77. The method of claim 76, A method for mining data, characterized in that each detailed module is performed to analyze the order of occurrence of unit items and search for a sequential pattern that is expressed above a user-defined threshold. The method of claim 79, If the current point of view changes due to a time change or an additional transaction occurs, the transaction window defining the search range is redefined based on the present, and it appears in the transactions not included in the changed window among the transactions included in the window at the previous point in time. All previous occurrences to provide mining results in a short time when time changes or new transactions increase, including a transaction window update method that updates the frequency information of the items managed in the monitoring lattice for one item. By maintaining the frequency information of all the sequential patterns that appeared in the transaction, the frequency of occurrence frequency management method and the method to obtain the mining result by analyzing only the newly added transaction without re-analyzing the transactions included in the entire data set. Method for data mining results real-time sequential pattern unit entry Discover the window characterized in that the selection method is performed 77. The method of claim 76, Based on the double lattice structure, each step is performed to analyze the sequential information between the items and search for the sequential pattern that appears above the user-defined threshold. Data mining method. 82. The method of claim 81 wherein In order for the frequency management method to efficiently use items with higher support than a certain threshold in the sequence pattern search process, a separate sequence pattern unit item identifier is newly assigned when the item is created, and the sequence pattern unit item in the sequence pattern search step is used. If this is removed, the method includes mapping a sequence pattern unit item identifier mapping method to terminate the connection relationship between the item and the sequence pattern unit item identifier so that items appearing in a new transaction can be utilized when the items are mapped to the sequence pattern unit item. Data mining methods. 82. The method of claim 81 wherein If the current point of view changes due to a time change or an additional transaction occurs, the transaction window defining the search range is redefined based on the present, and it appears in the transactions not included in the changed window among the transactions included in the window at the previous point in time. All previous occurrences to provide mining results in a short time when time changes or new transactions increase, including a transaction window update method that updates the frequency information of the items managed in the monitoring lattice for one item. By maintaining the frequency information of all the sequential patterns that appeared in the transaction, the frequency of occurrence frequency management method and the method to obtain the mining result by analyzing only the newly added transaction without re-analyzing the transactions included in the entire data set. Method for real-time data mining sequential patterns entry search result by the window characterized in that the selection method is performed. The method of claim 58, a) The number of unit time or unit transactions has elapsed using the decay rate, which indicates the degree of decrease in importance according to the unit time or unit number change, based on the decay unit time, the decay default, and the decay fundamental period defined by the user in the decay rate method step. How to differentiate the importance of transactions, b) If the current point of time changes due to a change in time or the occurrence of additional transactions, the change in time, including a method of updating the number of occurrences of an item, by updating the number of occurrences of the item by applying the attenuation rate to the number of occurrences of previously generated items Differentiating the importance of each transaction so that the importance of each transaction is attenuated by the decay rate based on a certain unit based on a certain unit, and performing each step to analyze the information inherent in the data set within a specified range. . 85. The method of claim 84, Each step is performed to analyze information about the frequency of occurrence of items so that each step is performed to search for items whose degree of support is above a certain threshold defined by the user. 86. The method of claim 85, In order to provide a mining result in a short time when time changes or new transactions are increased, it maintains the frequency information of all items that appeared in all transactions that occurred before, without re-analyzing the transactions included in the entire data set. A data mining method, characterized in that the method of managing the frequency of occurrence and the method of selecting a mining result are performed to obtain a mining result by analyzing only the added transaction. 87. The method of claim 86, In the method of managing the frequency of occurrence, a) a data mining method comprising a method of estimating the frequency of occurrence of estimating the frequency of occurrence of items to be delayed from the frequency of occurrence of all sub-items of the corresponding item that appeared in a previously generated transaction. or b) a data mining method including a method of estimating the number of occurrences of estimating the number of occurrences of delayed-added items only by the number of occurrences of (n-1) -subitems of the corresponding item appearing in a previously generated transaction; And c) As the number of items to be managed rapidly increases as transactions increase rapidly, managing information about the frequency of occurrence of items that occur in all transactions is due to physical limitations such as available memory space limitations and increased mining processing time. If the items appearing in the transaction are repeatedly generated and have high support enough to affect the determination of frequent items, the delayed addition method that adds the items to the monitoring lattice and starts to manage the frequency of occurrences will reduce the memory space usage. Data mining method comprising a. 87. The method of claim 86, The frequency count management method removes the item from the monitoring lattice when it has a value that is small enough because the item does not appear in a transaction that occurred for a considerable period of time among the items whose frequency count is managed in the monitoring lattice. Data mining method comprising a battery operation method for reducing the memory space required for the mining process by performing 87. The method of claim 86, In addition to the information on the frequency of occurrence of the items, the error in the estimation of the frequency of occurrence for adding delays is managed separately from the delay adding step for each item, and the error included in the frequent items is selected by considering the estimation error information in the mining result selection step. Data mining method characterized in that the confidence in the mining results can be increased by limiting within a predetermined range defined by the user. 85. The method of claim 84, Data mining method characterized in that each detailed step is performed in order to analyze the order of occurrence of the unit items to search the sequential pattern of the unit items appeared above the user-defined threshold value 91. The method of claim 90, In order to provide mining results in a short time when time changes or new transactions are increased, the frequency counts of all the sequential patterns that appeared in all previously occurring transactions are maintained, without reanalyzing the transactions included in the entire data set. A data mining method, characterized in that the method of managing the frequency of occurrence and the method of selecting a mining result are performed to obtain a mining result by analyzing only a newly added transaction. 92. The method of claim 91 wherein In the method of managing the frequency of appearance, a) a method of estimating the frequency of occurrence of estimating the frequency of occurrence of delayed addition of sequential patterns from the frequency of occurrence of all partial sequential patterns of the corresponding sequential patterns that appeared in a previously generated transaction; Data mining methods or b) Data mining including a method of estimating the number of occurrences of estimating the number of occurrences of delayed addition of the sequence patterns by the number of occurrences of the (n-1) -partial sequence patterns of the corresponding sequence pattern that appeared in a previously generated transaction. How to c) As the number of sequential patterns managed to increase rapidly as transactions increase rapidly, managing information on the number of occurrences of sequential patterns occurring in all transactions is limited by physical limitations such as available memory space and increase in mining processing time. Since the sequential patterns appearing in the transaction are repeatedly generated and have a large enough support to affect the determination of the unit sequential pattern, the sequential patterns are added to the monitoring lattice to start managing the frequency of occurrence of memory space. A data mining method comprising the method of adding delay to reduce. 92. The method of claim 91 wherein The occurrence frequency management method removes the sequential pattern from the monitoring lattice when the support frequency has a sufficiently small value because the sequential pattern does not appear in a transaction occurring for a long period of time among the sequential patterns managed by the monitoring lattice. And a battery operation method for reducing memory space usage required for the mining process by performing the battery operation. 92. The method of claim 91 wherein In addition to the information on the number of occurrences of sequential patterns, the estimation errors generated in the process of estimating the frequency of occurrence for delay addition are managed separately from the delay addition step for each sequential pattern. Data mining method characterized in that to increase the confidence in the mining results by limiting the included error within a certain range defined by the user. 85. The method of claim 84, Data mining characterized in that each detailed step is performed in order to search for the item sequential pattern by the attenuation rate method that analyzes the occurrence order information of the items based on the double lattice structure and searches for the sequential pattern that appears above the user-defined threshold. Way. 96. The method of claim 95, In order to provide mining results in a short time when time changes or new transactions are increased, the frequency counts of all the sequential patterns that appeared in all previously occurring transactions are maintained, without reanalyzing the transactions included in the entire data set. A data mining method, characterized in that the method of managing the frequency of occurrence and the method of selecting a mining result are performed to obtain a mining result by analyzing only a newly added transaction. 97. The method of claim 96, In order for the frequency management method to efficiently use items with higher support than a certain threshold in the sequence pattern search process, a separate sequence pattern unit item identifier is newly assigned when the item is created, and the sequence pattern unit item in the sequence pattern search step is used. If this is removed, the method includes mapping a sequence pattern unit item identifier mapping method to terminate the connection relationship between the item and the sequence pattern unit item identifier so that items appearing in a new transaction can be utilized when the items are mapped to the sequence pattern unit item. Data mining methods 97. The method of claim 96, In the method of managing the frequency of appearance, a) a method of estimating the frequency of occurrence of estimating the frequency of occurrence of delayed addition of sequential patterns from the frequency of occurrence of all partial sequential patterns of the corresponding sequential patterns that appeared in a previously generated transaction; Data mining methods, or b) Data mining including a method of estimating the number of occurrences of estimating the number of occurrences of delayed addition of the sequence patterns by the number of occurrences of the (n-1) -partial sequence patterns of the corresponding sequence pattern that appeared in a previously generated transaction. Method, c) As the number of sequential patterns managed to increase rapidly as transactions increase rapidly, managing information on the number of occurrences of sequential patterns occurring in all transactions is limited by physical limitations such as available memory space and increase in mining processing time. Since the sequential patterns appearing in the transaction are repeatedly generated and have a large enough support to affect the determination of the unit sequential pattern, the sequential patterns are added to the monitoring lattice to start managing the frequency of occurrence of memory space. Data mining method comprising the method of adding delay to reduce 97. The method of claim 96, In the method of managing the frequency of occurrence, the sequence is removed from the monitoring lattice when the support has a value that is small enough because the sequence does not appear in the transaction that occurred for a long period of time in the monitoring lattice. And a battery operation method for reducing the memory space usage required for the mining process by performing the battery operation. 97. The method of claim 96, In addition to the information on the number of occurrences of sequential patterns, the estimation errors generated in the process of estimating the frequency of occurrence for delay addition are managed separately from the delay addition step for each sequential pattern, and included in the item sequential pattern considering the estimation error information in the mining result selection step. Data mining method, characterized in that the reliability of the mining results can be increased by limiting the error within a predetermined range defined by the user. The method of claim 58, Step-by-Step Information In order to obtain the mining results by maintaining the mining results obtained by mining the previous time point or the previous transaction in a separate management step, only the appearance information of items appearing in the newly added transaction is analyzed without searching the entire monitoring lattice. A data mining method characterized by obtaining mining results by separately managing the mining result information in the mining result set selection method so that the mining result for the entire data set can be obtained by reflecting the result in the mining results up to the previous. The method of claim 58, In the step of open data mining by simultaneous processing of multiple transactions, a) analyzing the redundancy of items or units of transactions in multiple transactions at the same time, and updating the frequency of occurrences of items in multiple transactions at once. A method of data mining, comprising: performing a basic analysis on a plurality of transactions that occur and updating the frequency of occurrences of items appearing in the transactions to a monitoring lattice at a time to reduce mining execution time. The method of claim 58, In the analysis of usefulness indicator change step, a) define a certain unit time, define the change of usefulness indicator index during the unit time as the speed of the indicator, and in the damping rate method, define the rate of change of the usefulness judgment indicator that defines the amount of change in consideration of the decay rate. and, b) including a method of selecting a mining result by analyzing a rate of change of support for selecting a mining result in consideration of a current rate of change of support of an item or a sequential pattern, Analyzes the rate of change of support, reliability, persuasion, interest, hardness, distribution, correlation, and similarity, which are indicators of the usefulness of the items obtained as a result of mining, and analyzes the data set up to now, as well as the data to be generated later. A data mining method comprising mining results in consideration of the rate of change of the usefulness judgment index for predicting the change in the corresponding usefulness judgment index of the corresponding item in the set. The method of claim 58, a) Define a certain unit time, define the change rate of the change rate of the usefulness judgment indicator during the unit time as the acceleration of the indicator, and define the acceleration rate of change of the usefulness judgment index that defines the change rate of change rate in consideration of the attenuation rate. How, b) a method of selecting a mining result through a support change acceleration analysis in which a mining result is selected in consideration of the acceleration of the current support change of an item or a sequential pattern; Analyze the acceleration of changes in support, reliability, persuasion, interest, hardness, distribution, correlation, and similarity, which are indicators of the usefulness of the items obtained as a result of mining, and analyze data on the data set up to now, as well as data to be generated later. A data mining method comprising mining results in consideration of an acceleration of a change in the usability judgment indicator for predicting a change in a change rate of a corresponding usability determination indicator of a corresponding item in a set. The method of claim 58, In the dataset feature segmentation related step, the dataset feature analysis and refinement method is used to refine the dataset to be mined by using the dataset feature analysis indicator value. Data mining method. The method of claim 58, Data mining method characterized in that to obtain a mining result set that satisfies this by setting a minimum reference value for the data set characteristic analysis indicator value 107. The method of claim 105 or 106, And a method for analyzing and purifying the characteristics of one transaction by obtaining an average length of a transaction to be analyzed and obtaining an average transaction length for the entire data set. 107. The method of claim 105 or 106, A method of mining data comprising analyzing a reflection rate of an item appearing in a transaction to be analyzed and analyzing and purifying the characteristics of one transaction and obtaining an average reflection rate of the item for the entire data set. 107. The method of claim 105 or 106, The data mining method includes analyzing and refining the characteristics of one transaction by analyzing the average length of the frequent items among the items appearing in the transaction to be analyzed, and obtaining the average length of the frequent items for the entire data set. Way. 107. The method of claim 105 or 106, The data mining method includes analyzing and refining the characteristics of one transaction by analyzing the average map of the frequent items among the items appearing in the transaction to be analyzed, and obtaining the average map of the frequent items for the entire data set. Way. 107. The method of claim 105 or 106, Among the items appearing in the transaction to be analyzed, the average rule scores of the frequent items are analyzed and the characteristics of one transaction are analyzed and refined, and a method for obtaining the frequent rule average rule scores for the entire data set is also included. Data mining method. 107. The method of claim 105 or 106, Among the items appearing in the transaction to be analyzed, the distribution ratio of the support items of the frequent items is analyzed and refined by analyzing the characteristics of one transaction, and the distribution ratio of the support items of the support items for the frequent items is also obtained for the entire data set. Data mining method, characterized in that. The method of claim 58, It is possible to reuse the memory space occupied by an item or a sequence pattern for an item or a sequence pattern that is no longer needed to manage the frequency of occurrence of the item or sequence pattern in the mining process at the idle item collector stage in the integrated system. A data mining method comprising a method of collecting idle items allocated to space.