KR101801091B1 - System of Multi-Dimensional Hierarchical Data Cube and Evaluation Method thereof - Google Patents

Abstract

The present invention relates to a multidimensional hierarchical data cube system and a processing method thereof which build a hyper data cube which processes hierarchical data based on continuous queries and enables each cuboid to compute the smallest computational cost cuboid from previously computed data cuboids. The present invention can process complex queries at high speed by arranging a composition including: (a) a step of expressing multidimensional hierarchical data in structured query language (SQL); (b) a step of generating a data cube based on continuous queries about multidimensional hierarchical data expressed in the step (a); (c) a step of monitoring the number of generated tuples which is the cost of each cuboid about the data cube generated in the step (b); and (d) a step of computing the minimum cost parent cuboid according to the monitoring in the step (c) and making a child cuboid compute the minimum cost according to the computed result.

Description

[0001] The present invention relates to a multidimensional hierarchical data cube system and a multidimensional hierarchical data cube system,

The present invention relates to a multidimensional hierarchical data cube system and a processing method thereof, and more particularly, to a hyperdata cube processing hierarchical data on a continuous query basis, and each of the queue voids has a smallest computational cost of the previous aggregated data queue void Dimensional hierarchical data cube system and a processing method thereof.

The advent of global communications networks such as the Internet has continued to exchange tremendous amounts of information. In addition, the cost of storing and maintaining this information is also reduced, requiring access to large data storage structures. A tremendous amount of data can be stored as a data warehouse, a database that typically represents an organization's business histories. Historical data is used in analyzes that support business decisions at many levels, from strategic planning to performance evaluation by individual organizational units. It can also include fetching data stored in a relational database and processing it to make it a more efficient tool for querying and analysis. In order to effectively manage data warehousing on a small scale, the concept of a data mart in which only a target subset of data is managed is employed.

OLAP (On-Line Analytical Processing) technology in databases and data warehouse systems, which are data analysis technologies that process complex queries at high speed using a multidimensional data structure called a data cube or a cube, have made many advances. OLAP has the characteristics of summarization, integration, observation, formal application, and synthesis of multidimensional data. In the multidimensional data model used in the OLAP system, the data cube is a 'dimension' And 'Measure', which is the target of analysis. OLAP technology is now a fundamental tool for data analysts and decision makers. Data cubes, OLAP's multidimensional data model, has been successfully applied to many multidimensional data analysis.

This OLAP is one of the decision support systems that can process complex queries at high speed and enable multi-dimensional analysis through an interactive approach. In other words, the multi-dimensional data model, which is the most important characteristic of OLAP, represents the characteristics of data using the concept of data cubes. The data queue voids gather to form a data cube lattice.

With the development of such technologies, various devices connected to the Internet have opened the Internet era, and data analysis and processing methods in a data stream environment that generates vast amounts of data in real time have become more important.

Examples of such techniques are described in documents 1 and 2 below.

For example, Patent Document 1 discloses a method for data mining on a multidimensional data cube in a computer system including a processor and a system memory, comprising: accessing multidimensional data; Accessing data mining extensions for performing data mining on data residing in the multidimensional data cube, accessing the multidimensional expressions that define how to view the multidimensional data cubes as a multidimensional data cube, Integrating the extensions into an input for generating a data mining model; generating a data mining model trained on the multidimensional data from the input by the processor; storing the data mining model; Data mining a multidimensional data cube to perform data predictions on the data contained within the multidimensional data cube, wherein the data mining includes performing data mining operations on portions of data contained within the multidimensional data cube in accordance with the multidimensional query element A computer-implemented method is disclosed.

In addition, Patent Document 2 discloses a method of displaying a rectangular parallelepiped model composed of a plurality of data cubes in a three-dimensional space of an OLAP system, selecting a first data cube among a plurality of data cubes constituting a first outer side of the rectangular- Selecting a second data cube from among a plurality of data cubes constituting a second outer side perpendicular to the first outer side, Displaying a second cube column including the selected second data cube in an active state, setting a data cube that is redundantly included in the first cube column and the second cube column as an analysis subject cube, outputting data corresponding to the analysis subject cube A method for selecting a data analysis object in an OLAP-based three-dimensional space.

Also, non-patent document 1 proposes a cuboid prefix tree based on a potential tree structure for processing an iceberg query in a data stream environment, and a cuboid potential tree is used for an iceberg query Since only a set of items consisting of group items is managed in the tree, the memory is used less than the dislocation tree, and unnecessary items are deleted from the transaction through 1-item management, thereby reducing unnecessary time required for updating. Discloses a method capable of efficiently processing multiple iceberg queries using a small memory by sharing nodes in accordance with group attributes commonly used in the system.

Korean Registered Patent No. 10-1034428 (registered on May 3, 2011) Korean Registered Patent No. 10-1117649 (registered on February 10, 2012)

 Journal of the Korean Institute of Information Scientists and Engineers ISSN: 1229-7739

However, existing OLAP processing methods in the real-time data stream environment have the following limitations.

First, since the disk-based data is accessed and the query is performed, there is a problem that the processing speed is delayed due to repeated disk scanning in a data stream environment where data is continuously generated.

Second, there is a problem that it is difficult to perform a query on a multi-dimensional hierarchical data simultaneously with a conventional OLAP query with a one-time query.

It is an object of the present invention to provide a multidimensional hierarchical data cube system and a processing method for processing multidimensional data on a continuous query basis in a real-time data stream environment.

It is another object of the present invention to provide a multidimensional hierarchical data cube system and a processing method for providing a variety of information to a user by constructing a multidimensional hierarchical data cube based on a continuous query and simultaneously performing aggregation.

It is yet another object of the present invention to provide a multidimensional hierarchical data cube system and method of processing a minimum cost cube tree by monitoring the cost between cuboids, thereby aggregating the results of the lowest cost cuboid among the previous aggregated queue voids .

According to an aspect of the present invention, there is provided a method of processing a multi-dimensional hierarchical data cube, the method comprising: (a) expressing multi-layer data in a SQL (Structured Query Language) (C) monitoring the number of generated tuples, which is the cost of each queue void, for the data cubes generated in step (b); (d) Counting a minimum cost parent queue void according to the monitoring in the child queue void and aggregating the minimum cost according to the result.

In the method of processing a multidimensional hierarchical data cube according to the present invention, a closure table data structure expressing all the paths of the tree is applied to the representation in the step (a).

In the multi-dimensional hierarchical data cube processing method according to the present invention, each of the hierarchical paths is defined as 'Path', the path ID is defined to identify the hierarchical path in the multi-hierarchical path, And a level difference (depth) and a hierarchical path ID (pathID) expressed in the closure table are applied to manage the multi-dimensional hierarchical data stream.

In the method of processing a multidimensional hierarchical data cube according to the present invention, the data cube in the step (b)

,

, …,

} … (1)Dimension D _i = {

,

, ... ,

} ... (One)

는 j 계층 레벨과 k 계층 경로 아이디의 속성값)에 의해 단일 조인으로 계층형 데이터 스트림 모델을 구축하는 것을 특징으로 한다.(remind

Is an attribute value of a j-th layer path ID and a k-th layer path ID), the hierarchical data stream model is constructed by a single join.

In the method of processing a multidimensional hierarchical data cube according to the present invention, in the step (d), when there is no hierarchical information, the parent cuboid having the lowest cost among the parent cuboids is selected considering only the connection relation between hypercuboids Forming a minimum cost cube tree, (d2) if there is attribute layer information of each dimension, forming a hierarchical queue void in a hypercube void and selecting a queue void in which the least cost occurs, (d3 ) Comprising the step of constructing a partial data cube tree that is repeatedly performed until a parent cuboid is found at a higher level, if there is no adjacent parent cuboid among the user interest queue voids, to connect with the ancestor cuboid .

In order to achieve the above object, a recording medium according to the present invention is a multidimensional hierarchical data cube processing system including a client terminal, a database, and a database management system (DBMS) And is a computer-readable recording medium on which a program for performing a cube processing method is recorded.

According to another aspect of the present invention, there is provided a multidimensional hierarchical data cube processing system including a client terminal, a database connected to the client terminal via a network and storing information requested by the client terminal, Wherein the database management system comprises a closure table data structure for applying a level difference (depth) and a hierarchical path ID (path ID) expressed in a closure table for managing a multi-dimensional hierarchical data stream, Forming a data cubes based on continuous queries for multi-layer data, selecting a least cost parent cuboid or ancestor queue void among the parent cuboid or ancestor queue voids, That includes a least-cost set of forming a re-characterized.

As described above, according to the multidimensional hierarchical data cube system and processing method of the present invention, hierarchical data is represented by existing SQL (Structured Query Language) statements, data cubes are generated based on continuous queries, By monitoring the number of cost-generating tuples, the performance of the OLAP (On-Line Analytical Processing) in the database and data warehouse system can be improved by aggregating the aggregate results of the least-cost parent cuboid with the child queue void Is obtained.

In addition, according to the multidimensional hierarchical data cube system and processing method of the present invention, multidimensional hierarchical data cubes are constructed on the basis of a continuous query, and aggregation is performed simultaneously, thereby providing various information to the user at high speed .

1 is a block diagram of a configuration of a multi-dimensional hierarchical data cube system according to the present invention;
FIG. 2 is a process diagram illustrating a process of a multidimensional hierarchical data cube according to the present invention;
3 is a diagram showing an example of data represented by a closure table,
4 is a conceptual diagram of a hyperdata cube,
5 is a diagram showing a connection state of a hierarchical queue void,
FIG. 6 is a partial data cube configuration example,
FIG. 7 is a graph showing a performance comparison by window size,
8 is a graph showing performance comparisons by jeep distribution,
9 is a graph illustrating performance comparisons according to datasets,
10 is a graph showing the maximum / minimum cost performance comparison (case 1, case 2)
11 is a graph showing the maximum / minimum cost performance comparison (case 3, case 4)
FIG. 12 is a graph showing a maximum / minimum cost performance comparison by hierarchical level,
13 is a graph showing a performance comparison according to the gap difference;

These and other objects and novel features of the present invention will become more apparent from the description of the present specification and the accompanying drawings.

First, the concept of the present invention will be described.

In the present invention, a closure table is used to implement an OLAP layering operator based on a continuous query. In addition, join operations were performed for each attribute, and aggregation operations were performed by layering a single tuple. In addition, the data cube tree is constructed based on the minimum cost by monitoring the cost per each cuboid for fast processing speed in a real time data stream environment. The experimental results show that the larger the cost difference between the queue voids, the more the performance difference between the minimum cost based cube tree and the maximum cost based cube tree. In addition, user interest queue voids are connected in an ancestor - child structure to partially improve performance.

That is, the present invention solves the problem of the related art that it is difficult to delay the processing speed in the existing OLAP processing method in the real-time data stream environment or to query the multi-dimensional hierarchical data simultaneously .

OLAP processing methods in existing data stream environments are tree based and continuous query based. The tree-based method manages the aggregate data in the form of a tree, which greatly increases the memory usage of the tree in a data stream environment in which various attribute data is constantly generated. On the other hand, the continuous query based method requires a relatively small amount of memory since the query is registered in advance and the query result is continuously obtained by querying the data in the window size.

The stream cube is a proposed structure for multi-dimensional data analysis in a data stream environment. The stream cube has the following characteristics.

First, the materialization cube is partially calculated using a tilted time frame. This structure stores the time in units of different levels. That is, the latest data is stored in a granular unit, and the historical data is summarized and stored. This is advantageous in terms of saving memory and disk costs, rather than processing all the data in the same unit, but it has a disadvantage in that it can not be inquired in detail about past data.

Second, it solves the cost problem of materializing all levels of the queue void by specifying only the interest queue voids using the concept of Critical Layers. In other words, the minimum attention layer (m-layer) and the observation layer (o-layer) are designated, and the data cubes are formed by only the queue voids located between the two layers. However, when a stream cube is constructed, it is impossible to observe the cuboid outside the layer, which can be a disadvantage in a real-time changing data stream environment.

Third, the stream cube stores the queue voids of popular paths frequently used by users among the queue voids from the lowest interest layer to the observation layer. In this case, the path is stored and managed in a data structure called H-tree, and the query time outside the stored path is calculated by moving to the closest queue void using the information stored in the H-tree. The popularity path can not be modified on the way to a fixed value determined by statistical analysis or experience, so that it can not actively respond to the data stream environment sensitive to rapid change.

To solve these drawbacks, a dynamic data cube is proposed. Dynamic data cubes save memory and processing time by dynamically grouping and managing attribute values by specifying user interest areas based on ratings of attribute values. The user's area of interest is the area with the minimum support rate S _min and the maximum support rate S _max or less.

The group having the support _rate less than the minimum support rate S _min is reduced and the support rate is increased to belong to the user interest area and the group having the support rate higher than the maximum support rate S _max is expanded to belong to the user interest area by lowering the support rate. This leads to a group of dimensional attribute values being in the user interest area as much as possible, not only efficiently managing the memory usage of the dynamic data cube, but also providing a detailed dimension property group range for the user interest area.

However, if a dynamic group tree is constructed for all the dimension attributes of the multidimensional data stream, all the attribute values are grouped, and a lot of memory is used and the processing time is long.

The continuous query applied to the present invention is an analytical technique for a data stream in a DSMS (Data Stream Management System), while a conventional query method is a one-time query to obtain results with repetitive scans for finite data, Since the data stream can not be stored in a limited memory space, the query is registered in advance and the data stream is queried successively to obtain the result.

The sliding window based aggregation performs the query by dividing the stream data by a predetermined number of times or a predetermined number of tuples, and moves the sliding window continuously. A time-based sliding window and a tuple-based sliding window that determine the window size based on time. Pane using linear data structure is a technique for sliding window aggregate query. Pane reduces the computational cost by partially aggregating the input stream to reduce the required window size and sharing partial aggregate values. Payne splits the stream in units of the greatest common divisor of the sliding window and the window size, and stores the aggregated value of the divided area instead of the tuple.

In recent data stream environment, continuous query based OLAP research performs cubic unit operator by mapping data into multi - dimensional structure in time - based window. The data stream is the same as the fact table and is converted into a multidimensional data stream. Multi-dimensional data stream types are S _- same as _{(D 1 .A 11, ...,} D n 1 .A n -1, Tt, m 1, ..., m m). D ₁ .A ₁₁ , ... , D _{n -1} .A _n-1 means the hierarchical level of the dimension and the attribute, Tt is the time (time stamp) at which the tuple occurred, m ₁ , ... , m _m means the measurement value. The source data stream is represented in the lowest layer format and can be moved (rolled up) to the upper layer using the mapping function g. Multi-Dimensional Stream Query Language, which is a study on multi-dimensional data stream query, has completed multi-dimensional data stream query in real-time data stream environment using PERIODIC and CONTINOUS mode. In addition, we implemented the hash table to support layer data.

In order to efficiently query a multidimensional data model, the conventional technique approaches a new operator in a data cube unit, but does not consider a processing technique between data cubes.

Hereinafter, a configuration of a multi-dimensional hierarchical data cube system according to the present invention will be described with reference to the drawings.

1 is a configuration block diagram of a multi-dimensional hierarchical data cube system according to the present invention.

1, a multidimensional hierarchical data cube system according to the present invention includes a client terminal 100, a client terminal 100, and a client terminal 100. The client terminal 100 is connected to the client terminal 100 via a network, And a database management system (DBMS) 300 for managing the database 200 through the network.

The client terminal 100 is a conventional computing terminal such as a PC, a notebook, a netbook, a PDA, a mobile phone, a smart phone, and the like. The user can access the database management system 300 using the client terminal 100 to retrieve necessary information. To this end, the client terminal 100 may be provided with a web browser to retrieve necessary information.

The database 200 is a conventional storage medium for storing necessary data in the client terminal 100 or the use database management system 300. The configuration is structured by the database construction theory in view of ease of access and search, .

It is sufficient that the network can interconnect the client terminal 100, the database 300 and the usage database management system 300 by wire or wireless, and is not limited to a specific communication means.

The database management system 300 may include one or more servers, and each server may include hardware and / or software (e.g., threads, processes, computing devices). For example, the database management system 300 may function as an algorithm for performing data warehousing and OLAP and as a server of a corresponding system. When the database management system 300 functions as an OLAP server, it includes various representations of warehouse data including software, metadata and a data relationship table designed to facilitate data representation .

1, a database management system 300 according to the present invention includes closure table data (data) for applying a level difference (depth) represented in a closure table and a hierarchical path ID (pathID) to manage a multi- A structure forming unit 310, a multi-dimensional hierarchical data cubing unit 320 for generating a data cube on the basis of a continuous query for multi-layer data, a lowest cost parent cuboid or ancestor queue void among the parent cuboid or ancestor queue void, And a minimum cost setting unit 330 for selecting a minimum cost cube tree.

The closet table data structure forming unit 310, the multidimensional hierarchical data cubing unit 320, and the minimum cost setting unit 330 may be configured to be executed by a program, or may be a computer-readable recording medium .

The functions of the closet table data structure forming unit 310, the multi-dimensional hierarchical data cubing unit 320, and the minimum cost setting unit 330 will be explained with reference to FIG.

2 is a flowchart illustrating a process of a multidimensional hierarchical data cube according to the present invention.

As shown in FIG. 2, in the method of processing a multidimensional hierarchical data cube according to the present invention, the closure table data structure forming unit 310 expresses multilevel data in a SQL (Structured Query Language) statement (S10) The multidimensional hierarchical data cubic structure unit 320 for the multi-layer data represented in step S10 creates a data cube on the basis of a continuous query (S20), and calculates the cost of each cuboid for the data cubes generated in step S20 The minimum cost setting unit 330 compares the minimum cost parent cuboid according to the monitoring in step S30 (S40), and according to the result, the child queue void counts the minimum cost S50).

That is, in the multidimensional hierarchical data cube processing method according to the present invention, hierarchical data is expressed by an existing SQL statement and a data cube is generated based on a continuous query. We will monitor the number of generated tuples, which are the cost of each queue void, and expect the performance improvement by aggregating the aggregate results of the least cost parent queue voids.

A representation of a multi-hierarchical structure of a multi-dimensional hierarchical data cube according to step S10 will be described.

For example, in a data warehouse system, a drill-down / roll-up function was set up as a decision-making tool by setting up a hierarchical structure. Drilldown is divided into two main meanings. "Region" attribute is used to move the hierarchy to perform aggregation or to add / remove dimensions. For example, you can drill down on the "Region" attribute to count sales for "Korea", "France", "UK" respectively. On the other hand, roll-ups raise the conceptual hierarchy level from "Korea" to "Asia" and move to "Europe" from "France" and "UK"

Hereinafter, a method for expressing hierarchical data in step S10 will be described in detail. In a typical SQL statement, various modeling techniques have been proposed for representing hierarchical data. Examples include Adjacency List, Path Enumeration, Nested Sets, and Closure Tables.

Existing models focused on representing a single hierarchical structure. However, in general, a dimension can have a multi-hierarchical structure, and can be classified as follows depending on how the hierarchical structure shares the levels.

FIG. 3 is a diagram showing an example of data represented by a closure table. FIG. 3 (a) is a diagram illustrating a data structure of a 'crossing multi-layer structure' sharing a high level, Hierarchical structure 'exists.

In order to reflect this point, in the present invention, the closure table data structure forming unit 310 uses a closure table data structure. A closure table represents all paths in the tree, as well as paths to parent-child relationships. The table stores all pairs of nodes with parent / child relationships in a single row. A closure table is the only model that allows a node to belong to multiple trees, and is the simplest way to store a hierarchy.

In FIG. 3, (a) is a cross-over hierarchical structure, and (b) is a relational table form. The lower level 'Seoul' has two hierarchical structures, and each layer path is defined as 'Path'. FIG. 3 (c) shows the depth difference of the hierarchy between parent-child relationships in a representation of a traditional closure table. Existing schemes can represent all hierarchical paths, but when a single data has a multi-hierarchical path, it is difficult to identify which hierarchical path is the data. Accordingly, in the present invention, an ID is assigned to identify the hierarchical path of each parent-child node relationship as shown in FIG. 3 (d), and it is defined as a path ID. The level difference (depth) and the hierarchical path ID (pathID) of the parent-child relationship expressed in the closure table are used to manage the multi-dimensional hierarchical data stream.

In step S20, the multi-dimensional hierarchical data cubic structure unit 320 constructs a hierarchical data stream model by a single join, taking advantage of the ability to represent data of all the hierarchical levels of the closure table structure. .

,

, …,

}Dimension D _i = {

,

, ... ,

}

: j 계층 레벨과 k 계층 경로 아이디의 속성값)(

: attribute value of j-layer level and k-layer path ID)

,

, …,

.

,

, …,

}S _stream = {timestamp,

,

, ... ,

.

,

, ... ,

}

,

.

, …,

.

,

, …,

}H _stream = {timestamp,

,

.

, ... ,

.

,

, ... ,

}

,,

.

, …,

.

,

, …,

}{timestamp,

,,

.

, ... ,

.

,

, ... ,

}

...

A single data stream is represented by an attribute value of the lowest layer, and the dimension D _i is composed of k layer paths and a maximum j level. | D _i | means all hierarchical levels of the attribute and the number of elements that the hierarchical path can represent. For example, if D ₁ = {day ⁰ , month ⁰ , year ⁰ , week ¹ , month ¹ , year ¹ }, then the hierarchy path is 2, the maximum hierarchy level is level 4, | D ₁ | . Thus, a single data stream can be stratified into a total of n ^k hierarchical data streams when all dimensions | D _i | = k of the n dimensional data streams.

Next, the minimum cost setting unit 330 sets a costume cost model.

A multidimensional data cube is formed of a grid structure of several cubic voids. Considering this connection relation, the calculation load of one cuboid can be reduced by calculating from the result of the other cuboid's aggregation operation. For example, (Company, Region, Color) and (Company, *, *) and Quboids can form a parent-child relationship with each other. The cuboid cost model is an indicator for making this connection. The cost of the cuboid is defined as follows.

Cost (C) = number of different groups within the schedule window size (W)

That is, it means the number of comparison operations when performing the aggregation operation based on the result from the parent queue void. For example, if the cost of (Company, Region, *) is 100 and the cost of (Company, *, Color) is 50, then the company (*, *) will process the result of the aggregation of the latter queue void It is a benefit to do.

Next, the data cube model that processes the hierarchical data and the cost-based data cube tree are described in detail.

First, we define a 'Multidimensional Data Cube' that applies a multi-dimensional data model of OLAP and a Hyper-Node concept to process hierarchical data.

A hyper node is represented by (G, N, E) and is a data structure including subnodes forming a relationship. G is the label of the hyper node, and N is the finite set of base nodes that the hyper node contains. E is the relation (edge) between nodes belonging to N.

The hypercube void can be expressed as (G, H, P), where G is the analytical dimension of the queue void, H is the hierarchical level of the dimension, and P is the hierarchical path ID of the dimension. That is, the hypercuboid contains a queue void that processes the hierarchical data.

4 is a conceptual block diagram of a hyperdata cube. For a three-dimensional fundamental hypercuboid, the first dimension Company has a single hierarchy and the second dimension Region has two hierarchical levels. The third dimension Color has single layer information. With the above conditions, the default hypercuboid will contain a data queue void that processes a total of two hierarchical data.

In order to identify the hierarchical queue void that processes such hierarchical data, a unique ID representing the hierarchical level and the hierarchical path is given as an integer, and this is called a 'control flag'. For example, h ₀₀₀ means that all dimensions are the lowest hierarchical level, that is, source data, and p ₀₀₀ means the hierarchical path ID of each dimension. (Company, City, Color) and (Company, Country, Color) possessed by the basic queue void shown in FIG. 2 can be represented by h ₀₀₀ p ₀₀₀ and h ₀₁₀ p ₀₀₀ , respectively. The control flags are used to connect the hyperdata queue voids of the parent-child relationship.

Next, the minimum cost data cube tree applied to the present invention at steps S40 and S50 will be described.

A hyperdata cube consists of 2 ⁿ hypercube voids. All hypercube voids have a parent-child relationship, except for the default hypercubic voids and vertex hypercubic voids. At this time, the child node may have one or more parent nodes. For example, HyperCuboid (A) has HyperCuboid (AB) and HyperCuboid (AC) as its parent. In order to efficiently process the data cube, we construct a 'Minimal Cost Data Cube tree' that connects all the nodes without forming a cycle. The cost is the number of tuples processed by a single node.

In the n-dimensional base-queue voids, the top-down method searches for adjacent subdimensional data cuboids and connects 1: 1 relationships between parent-child cube voids. The basic hyperdata cube will have multiple relationships in two cases, defined as follows.

Case 1. Non-tiered queue void connection

When there is no hierarchical information, only the connection relationship between hypercuboids is considered. In Figure 4, the Company, *, * has a parent-child relationship with the Company, Region, *, Company, *, Color. In a non-hierarchical data cube, when there are n or more parent cuboids, the least cost parent cuboid among the parent cuboids is selected to form the least cost cube tree.

Case 2. Hierarchical queue void connection

If there is attribute hierarchy information for each dimension, the hierarchy queue void is formed in the hypercube void.

5 is a diagram showing the connection state of the layer queue void. FIG. 5 (a) shows an example of hierarchical data cube structure when the maximum hierarchical level is 2 only in the Region dimension among the Company, Region, and Color in each dimension, and there are two hierarchical paths. The basic cuboid (Company, Region, Color) and the (Company, *, Color) cuboid have a 1: 1 relationship. However, the hierarchical queue voids included in the basic queue void have an n: 1 relationship, and the least cost hierarchy queue void is selected from the hierarchical queue voids. Figure 5 (b) is an example of the connection between layer queue voids when they have an n: 1 relationship between hypercuboids. (Company, *, *) The queue void has a redundant relationship with two parent queue voids, and the parent queue void has four hierarchical queue voids each. Of these, we choose the least costly queue void .

Next, a partial-data cube tree applied to the present invention will be described.

Partial Data on Cube Tree The user interest data cube tree construction methodology will be described. Analysis Data As the number of dimensions increases, the size of the entire data cube gradually increases, so you need to be able to configure the data cube tree in part. However, since the entire cube tree needs to be selected for all levels of the queue voids, even if only some of the cube trees are performed, the user must perform the process including the level of the queue voids that the user does not desire.

6 is a diagram illustrating an example of the configuration of a partial data cube. For example, when the user interest cupid is the default cube void (Company, Country, Color), (Company, City, Color) (*, City, *) to perform the aggregation operation of the queue void by executing the user's interest not the queue-void of interests (Company, City, *). In order to reduce the unnecessary computation load, It is called an ancestor-child connection. If there is no adjacent parent cuboid among the user interest queue voids, it is repeatedly performed until the parent cuboid is found at a higher level, so that a partial data cube tree connecting with the ancestor cuboid is constructed to improve performance do.

Next, the performance evaluation of the system according to the present invention will be described. First, the experimental environment according to the present invention will be described.

In the present invention, two data sets (jiff distribution data, actual purchase data) are used to verify the performance of the hierarchical multidimensional data model. The Zipf distribution data has a biased value as the Zipf value increases. The Actual Purchase Data Log is a purchase data from August 1, 2014 to August 27, 2014, consisting of 49,871 tuples, with four dimensions and two measurements. The present invention has been performed in an environment of Intel Core2Duo E6600 @ 2.4 GHz and memory 4G in CentOS 6.5 or higher.

Next, the entire data cube experiment according to the present invention was performed.

In other words, we analyzed the factors affecting the performance of the non - hierarchical data cube.

FIG. 7 is a graph showing performance comparison by window size. That is, in FIG. 7, the purchase data is composed of 4-dimensional data cubes and transmitted for 2 seconds to 10,000 tuples per second for 10 seconds, and the performance is compared for each window size of 10, 100, and 1,000. It showed the fastest processing time and small memory usage when the window size was 10, and it showed the slowest processing time and the large memory usage on Windows 1000. This is because the larger the window size, the larger the number of different groups in the window and the number of comparison operations increases.

FIG. 8 is a graph showing performance comparisons by jeep distribution. That is, FIG. 8 is a result of analyzing the distributional characteristics of data and the influence of dimension on the aggregate calculation load. In this experiment, we compared the performance by adjusting the jiff distribution when the fastest and smallest memory size was used and the window size was 10 and 10,000 tuples per second. The larger the jiff distribution value is, the more the distribution of data is shifted. As the number of different groups within the window size decreases, the memory usage decreases.

Next, the cost-based data cube tree was experimented.

First, a non-hierarchical data cube tree will be described.

In other words, we verified the performance of the data cube tree when it was constructed on a cost basis. In existing Jeep distribution data, the cardinality of all attributes is the same, so the cost difference between the same n-dimensional cube voids is insignificant. Therefore, in the present invention, actual online shopping mall purchase data is used. The cardinality of the data attributes is processed for the cost-based data cube tree experiment, and Table 1 shows the cost processing data.

9 is a graph illustrating performance comparisons according to datasets. That is, FIG. 9 shows that the case 4 data having the high degree of processing showed the fastest processing time and the small amount of memory usage due to the performance test of the processed data set. This is due to the fact that the number of different groups within a certain window size has been reduced, as confirmed in previous data distribution experiments.

FIG. 10 is a graph showing the maximum / minimum cost performance comparison (case 1, case 2). FIG. 10 is a graph comparing performance differences in the case of constructing the maximum / minimum cost cube tree of case 1 and case 2. In case 2, the number of groups of attributes was reduced, so the processing time was slightly faster than case 1 and less memory was used. When the cost - based cube tree was constructed, there was a slight difference in cost performance as the number of processing tuples increased.

FIG. 11 is a graph showing the maximum / minimum cost performance comparison (case 3, case 4). FIG. 11 is a graph showing performance differences when the data case 3 and case 4 are constructed based on cost. In the present invention, the number of groups of data is remarkably reduced, and the gaps can be confirmed as compared with the previous experiments. The gap between memory usage and processing time between minimum cost and maximum cost showed a slight increase as the transmission rate per second increased. Case 4 shows the largest cost difference when the initial transmission rate is small. This is because the number of groups is small, and when the number of tuples to be processed is small, the number of times the sub-queue void performs the aggregation operation is considerably reduced and seems to have influenced.

In the cost-based non-hierarchical data cube experiment, the difference between the maximum value and the minimum value in the same n-dimensional void voids was larger in the order of case 3, case 4, case 2 and case 1. That is, it can be seen that the performance of the algorithm according to the present invention is improved as the cost difference between the queue voids existing at the same level is larger.

Next, the hierarchical data cube tree was experimented.

In the present invention, the performance of the cost-based hyperdata cube tree is measured while increasing the hierarchical level of the dimension. We constructed a 4 - dimensional data cube and compared the performance differences by increasing the number (n) of hierarchical data cubes. That is, in the previous experiment, the processed data set case 2, in which the cost difference was small, and the case 3, in which the cost difference began to increase, were composed of the hyperdata cube tree.

FIG. 12 is a graph showing a maximum / minimum cost performance comparison according to a hierarchical level, and is a graph showing a performance difference when a cost-based tree is constructed. The larger the number of hierarchical data queue voids (n) the base queue void contains, the larger the cost difference between the same dimension queue voids. That is, when n = 4, it can be seen that the memory usage difference between the minimum cost and the maximum cost increases.

In the non-hierarchical data queue void, there is no maximum / minimum cost between 1: 1 relation hypercubes. However, since the hierarchical data queue void has an n: 1 relationship under the same conditions, the maximum / minimum cost difference occurs. Therefore, the memory usage gradually increased and the processing time remained almost constant.

We also experimented with a cost - based partial data cube tree.

In the present invention, the performance of the parent-child connection technique and the ancestor-child connection technique are tested when two cube voids, which are vertices of each tube tree, are selected from the entire cube tree structure. The user compared the parent - child connection method and the ancestor - child connection method on the assumption that the basic queue void and the one - dimensional queue void were set as the interest queue void.

FIG. 13 is a graph showing a performance comparison according to the gap difference. The performance is compared when performing a cuboid between a vertex and a vertex using 10-dimensional data having a jiff distribution of 1.8. Here, the interval is fixed by fixing the one-dimensional cuboid, and the number of dimensions of the basic cuboid is reduced by two. The larger the interval difference, the more unnecessary the queue void included in the path must be performed, so that the processing time and the memory usage gap can be confirmed.

As described above, the present invention provides a continuous query based hierarchical multidimensional data processing technique. That is, the hierarchical data is represented by the existing SQL (Structured Query Language) statement, the data cube is generated based on the continuous query, the aggregate result of the least cost parent cuboid is counted, By doing so, we could achieve a performance improvement for on-line analytical processing (OLAP) in databases and data warehouse systems.

Although the present invention has been described in detail with reference to the above embodiments, it is needless to say that the present invention is not limited to the above-described embodiments, and various modifications may be made without departing from the spirit of the present invention.

By using the multidimensional hierarchical data cube system and processing method according to the present invention, complex queries can be processed at high speed and performance can be improved.

100: Client terminal
200: Database
300: Database Management System

Claims (7)

A multi-dimensional hierarchical data cube processing system comprising a client terminal, a database connected to the client terminal through a network, the database storing information requested by the client terminal, and a database management system (DBMS) managing the database through the network In a data processing method,
The data processing method according to the database management system,
(a) expressing multi-layer data in a SQL (Structured Query Language) statement,
(b) generating a data cube based on the continuous query for the multi-layer data represented in the step (a)
(c) monitoring the number of generated tuples, which is the cost of each cuboid for the data cubes generated in step (b)
(d) aggregating the least cost parent cuboid according to the monitoring in step (c), and aggregating the minimum cost according to the result;
Dimensional data cube processing method. The method of claim 1,
Wherein the expression in step (a) applies a closure table data structure representing all paths of the tree. 3. The method of claim 2,
The closure table data structure defines each layer path as 'Path', defines a path ID as a path ID for identifying a layer path in a multi-layer path, and defines a path ID in the closure table to manage the multi- And a hierarchical path ID (pathID) is applied to the hierarchical data cube. The method of claim 1,
Wherein the data cube in step (b)
Dimension D _i = {

,

, ... ,

} ... (One)
(remind

Is the attribute value of the j-th layer level and the k-layer path ID)
And a hierarchical data stream model is constructed with a single join by a single join. The method of claim 1,
The step (d)
(d1) a step of forming a least cost cube tree by selecting the lowest cost parent void void among the parent queue voids considering only the connection relationship between hyper queue voids when there is no hierarchical information,
(d2) if there is attribute-layer information of each dimension, forming a hierarchical queue void in a hyper-cube void, selecting a queue void having the least cost,
(d3) If there is no adjacent parent cuboid among the user interest queue voids, the step of constructing the partial data cube tree connecting with the ancestor cuboid is repeatedly performed until the parent cuboid is found at the higher level Dimensional hierarchical data cube processing method. A computer program that, when executed in a multidimensional hierarchical data cube processing system including a client terminal, a database and a database management system (DBMS), executes a program for performing the multidimensional hierarchical data cube processing method of any one of claims 1 to 5 Readable recording medium. Client terminal,
A database which is connected to the client terminal through a network and stores information requested by the client terminal,
And a database management system (DBMS) for managing the database through the network,
The database management system
A closure table data structure forming unit for applying a level difference (depth) and a hierarchical path ID (pathID) expressed in a closure table to manage a multi-dimensional hierarchical data stream,
A multi-dimensional hierarchical data cube constructing unit for generating a data cube based on a continuous query for multi-layer data,
And a minimum cost setting unit for selecting the lowest cost parent cuboid or ancestor cuboid among the parent cuboid or the ancestor cuboid to form a least cost cube tree.

Images