KR20120118550A

KR20120118550A - An architecture of a high performance distributed main memory database management system for massive data

Info

Publication number: KR20120118550A
Application number: KR1020110035971A
Authority: KR
Inventors: 한상혁
Original assignee: 한상혁
Priority date: 2011-04-19
Filing date: 2011-04-19
Publication date: 2012-10-29

Abstract

In traditional applications requiring high performance transaction processing, such as mobile communications, finance, and defense, the application of the Main Memory Database Management System (MMDBMS) is increasing. In addition, applications such as ITS, LBS, Telematics, etc., that handle more than tens of thousands of high-performance transactions per second and manage large amounts of data over hundreds of gigabytes are expanding. In the case of building a large-capacity MMDB for high-performance transaction processing in these applications, there was a limit in applying the MMDBMS because of the burden of mounting the memory more than hundreds of GB rather than the technical problem. However, with the recent development of ultra-high speed networks such as Gigabit Ethernet, high-speed data transmission and reception between systems on the network has become possible, so that large-scale and high-performance transaction processing can be performed while lowering system construction cost by distributing the entire MMDB on the network. . In this paper, the present invention relates to the structure of Kairos-D, a Distributed Main Memory Database Management System (DMMDBMS), which provides a large capacity and high performance transaction processing using Kairos, an MMDBMS.

Description

An Architecture of A High Performance Distributed Main Memory Database Management System for Massive Data

The present invention is a configuration method of Kairos-D, a distributed database management system that provides high-speed transaction processing of large data based on Kairos, a commercial main memory database management system, to process large data at high speed. A distributed database management system comprising a distributed database server that performs processing, query analysis, distribution, and result processing, and one or more local database servers that perform decommissioned transactions and queries.

MySQL Cluster is a high-speed processing and cluster database developed for application requirements in the telecommunications field. 1 illustrates the structure of a MySQL Cluster, where a MySQL server accesses a clustered Data Node through an NDB API.

Oracle RAC is an Oracle clustering solution that can scale up to 100 instances accessing a database. 2 shows a typical clustering structure in which three instances use a RAC database. Between each instance, Heartbeat periodically checks if each instance is terminated.

Table 1 compares the structure of commercial solution and distributed database management system Kairos-D. Kairos-D's approach is similar in structure to MySQL Cluster rather than Oracle RAC. However, the existing proposed solutions do not provide detailed technical documentation, which makes it difficult to compare each element and makes sense of the scalability of the memory DB.

Main item

MySQL Cluster
Oracle RAC
Kairos-D
Approach
DISK DB Cluster MEMORY DB cluster
purpose
Space scalability Availability Space scalability
Data sharing
System-specific maintenance
Sharing between systems
System-specific maintenance

Storage engine type ISAM
INNODB
Memory
DISK
Memory
Access API
NDB API

-
CLI

Table 1 Comparison of Kairos-D with Commercial Solutions

The present invention has been proposed to solve the above problems and to improve the performance of the database system, and to provide a high-speed transaction processing of a large amount of data based on a commercial main memory database management system. To provide a component.

Distributed database management system for achieving the above object,

It is composed of a local database server including a main memory database management system capable of high-speed data processing, and a global database server managing the local database server.

Specifically, the global database server includes a distributed quality processor, a distributed server manager, a distributed directory manager, a distributed transaction manager, and a load balancing manager.

According to the distributed database management system of the present invention, a distributed database management system for managing the main memory database management system based on a number of main memory database management systems that have a high speed data processing function but has a limitation in processing a large capacity. By using, a large amount of data can be processed at high speed.

In addition, according to the distributed database management system, since it has a redundancy function for stability, an MMDBMS function for high-speed processing, and a distributed function for processing large amounts of information, high-speed transaction processing of large data such as LBS, ITS, USN, and horizontal partitioning of DB This is suitable for possible applications.

1 shows the structure of a MySQL Cluster.
2 shows the structure of an ORACLE RAC.
3 shows the Kairos-D structure.

Hereinafter, specific embodiments of components of a distributed database management system will be described with reference to the accompanying drawings.

In addition, the following components should be understood to include all radiuses, equivalents, or substitutes included in the spirit and scope of the present invention.

3 is a structural diagram of Kairos-D. Kairos-D consists of a distributed database server that performs distributed transaction processing, query analysis, distribution, and result processing, and one or more local database servers that are responsible for decomposing transactions and queries. Application programs are written using standard interfaces such as JDBC and ODBC. Similarly, global database servers and local database servers are written using standard interfaces such as JDBC, ODBC, and XA.

The start of a global transaction begins at the global database server. The global database server decomposes the unit operation, that is, the transmitted query, through the optimization algorithm and delivers it to the selected local database server. The local database server executes the passed query and passes the results to the global database server.

The global database server is composed of DCM, DQP, DTM, and DDM as shown in [Table 2].

Classification Component Explanation discharge
Database server DPM Data Partitioning Manager DQP Distributed Query Process DSM Distributed Server Manager DDM Distributed Directory Manager DTM Distributed Transaction Manager LBM Load Balancing Manager area
Database server TM Local Transaction Manager QM Local Query Manager RM Local Recovery Manager LM Local Lock Manager DM Local Duplication Manager

[Table 2] Major Components of Kairos-D

DPM has the ability to manage data partitions.

DSM has the ability to manage distributed local database servers.

DCM acts as a window for the client to access Kairos-D upon connection request. Upon initial connection, Kairos-D generates information for the connection between the client and server in DCM and loads it into the Distributed Connection Queue (DCQ). DCQ is used in DQP as a data structure that manages the connection information between Kairos-D and its clients. DQP is a process that processes a query from a real client and consists of one or more threads, each of which processes a query as one connection unit. At this time, the query is divided according to the characteristics of the query processed in the thread, and the selected local database server is requested to execute the query.

DTM registers and manages site information when executing each query, and when commit or rollback is required for distributed transaction, DTM also commits or rolls back local database servers related to the transaction.

DDM maintains and maintains a fragmentation list and a site list for accessing each site to find local database servers suitable for transaction execution.

DQP takes a socket from a Distributed Connection Dispatcher and receives a query from the Distributed Connection Dispatcher and sends it to the parser.

LBM load balancing in a distributed environment optimizes the performance of the system by balancing the availability of resources and load.

Claims

A distributed database management system comprising a local database server managing a local database and a global database server managing the local database server, the interface of which is characterized by its configuration.

In the global database server, the server comprises a distributed quality processor, distributed server manager, distributed directory manager, distributed transaction manager, load balancing manager, the configuration of the distributed database management system