KR101374533B1 - High performance replication system and backup system for mass storage data, method of the same - Google Patents

Abstract

An objective of the present invention is to copy data in real time through a multi-thread scheme in the process of copying data of a system to be copied. In order to accomplish the objective, the high performance copy system for mass storage data according to the present invention is characterized by including a copy-target disk in which data to be copied are stored, A copy-target server for dividing data blocks stored in the copy-target disk into a plurality of predetermined unit data, for computing a predetermined duration, for capturing the predetermined unit data for the predetermined duration by multiple threads, and for transmitting the predetermined unit data into a queue while computing another duration next to the predetermined duration to capture predetermined unit data during another duration by the thread which has finished the capturing work, a second agent for analyzing each unit data received from a first agent to create copy data by the multiple threads, and a copy server having a copy disk in which the copied data created by the second agent are stored. [Reference numerals] (100) Copy-target server; (110) First agent; (120) Copy-target disk; (200) Copy server; (210) Second agent; (220) Copy disk

Description

HIGH PERFORMANCE REPLICATION SYSTEM AND BACKUP SYSTEM FOR MASS STORAGE DATA, METHOD OF THE SAME

The present invention relates to a high performance replication and backup system for a large amount of data, and a high performance replication method. In particular, distributed parallel processing based on a primary key of a data table is required for high performance replication for a large DBMS (data). It is possible to automatically distribute based on these PKs and replicate them in parallel in each thread, but to distribute transactions with the same PK so that they can be processed in the same thread, ensuring data consistency even in parallel processing. The present invention relates to a high performance replication and backup system and a high performance replication method for a large amount of data that can be maximized through parallelism using distributed data, and in particular, a single table can be distributed and parallelized without violating consistency.

According to the U.S. Emergency Planning Institute, the average industry lost more than a million dollars due to data loss due to computer failure, and the backup of data is not only for companies but also for government offices that manage data resources in countries that have implemented e-government. It is emphasized that the importance of recovery and recovery is the most important issue directly related to national competitiveness and security, excluding economic losses.

And now, as the entire industry is shifting to the Internet environment, as data volume of individuals as well as corporate information data increases exponentially, data warehouse, enterprise resource planning, customer relationship management The establishment and expansion of advanced enterprise computing environments based on storage, such as customer relationship management and knowledge management, are increasing significantly.

Storage, which is being built in various industries as above, needs to expand storage space from hundreds of megabytes (MB) to several tens of gigabytes (GB) a day. Maintaining and protecting against disasters, obstacles and accidents such as earthquake or terrorism has become important to determine the company's existence.

 As a result of this change, many companies have developed a replication solution to provide software for copying data stored on a target disk, which is a main storage device connected to a main system, to a copy disk.

However, in the conventional replication solutions, as the number of files or the amount of data in the main storage device to be replicated increases, the speed of performing replication decreases. Therefore, it is important to minimize data replication time as much as possible. Within a limited space for storing data to be done, storing more data more efficiently is one of the important challenges.

The present invention has been made to solve the above problems, an object of the present invention is to enable data replication in real time using a multi-threaded method in the process of copying the data of the system to be replicated. .

Another object of the present invention is to enable data backup in real time using a multithreaded method.

In order to achieve the above object, a high-performance replication system for a large amount of data according to the present invention, after dividing the replication target disk and the block of data of the replication target data stored in a plurality of predetermined unit data, respectively By calculating an interval, the predetermined unit data of the predetermined interval is captured by a plurality of threads, and the captured thread transmits the predetermined unit data to a queue and calculates another interval following the predetermined interval. A replication target server having a first agent for capturing predetermined unit data of the another section; And a second agent for analyzing the predetermined unit data received from the first agent to generate duplicate data in a plurality of threads, and a duplicate disk storing the duplicate data generated from the second agent. It characterized by including.

In addition, the high-performance replication system for a large amount of data according to the present invention, the first agent is characterized in that for transmitting the predetermined unit data in a queue structure to the second agent.

In addition, the high performance replication system for a large amount of data according to the present invention, the second agent is characterized in that the predetermined unit data received from the queue in parallel to the processing based on the PK column or unique index column.

In addition, in the high performance replication system for a large amount of data according to the present invention, the second agent generates the data connection of the predetermined unit data of the queue structure received from the first agent by the number of the multi-threaded, the replication In disk, transactions having the same PK column or the same unique index column are replicated using only one of the connections.

In addition, the high performance replication system for a large amount of data according to the present invention, the second agent is a single thread when the value of the PK column or the unique index column is changed during the real-time data replication by the multi-threaded And the predetermined unit data is sequentially copied to the copy disk using only one data connection.

In addition, a high-performance backup system for large-capacity data according to the present invention includes an input / output unit for inputting a backup target disk in which backup target data is stored, a command including a backup operation command, and outputting a result for a predetermined command; A central control unit which processes a backup operation command applied to the input / output unit and controls the backup to be performed, and receives the backup operation command applied through the input / output unit and the central control unit to be transmitted to a backup manager module Receiving a backup master module for performing backup operation from the backup master module and managing the backup schedule information for each volume, collecting and managing backup status and backup history information for each volume, and a disk according to a backup schedule. Backups that transfer backup commands for volumes to the backup agent module. A backup target server having a manager module and a third agent for capturing predetermined unit data from the backup target disk; And a fourth agent for analyzing the predetermined unit data received from the third agent and generating backup data in a plurality of threads, and a backup disk storing the backup data generated from the fourth agent. N, including a server, wherein the third agent receives a backup command from the backup manager module, divides a volume of data of a backup target disk into unit data of a predetermined size, and performs several flows in one process. Generating four threads and sequentially compressing the divided unit data and transmitting the same to the fourth agent.

In addition, a high-performance backup system for a large amount of data according to the present invention, a backup master server including the backup master module, the backup manager module and the third agent, a plurality of backups provided with the backup target disk If the target server is configured separately, and receives a command including a backup operation command from the backup master server and transmits the command to the backup manager server side, the backup manager module manages backup schedule information for each volume, and backup status and backup history information for each volume. Collects and manages and transmits a backup command for a disk volume to a third agent module according to a backup schedule, and the third agent module selects a volume of data of the backup target disk according to a backup command applied from a backup manager module. Split into unit data of size, and To generate the N threads advancing multiple flows in the process is characterized in that by compressing the divided unit data is sequentially configured to transfer to the fourth agent.

In addition, a high performance backup system for a large amount of data according to the present invention, a backup master server including the backup master module; A plurality of backup manager servers including the backup manager module and having a backup target disk; And a third agent, and a backup dedicated server having a backup disk is separately configured, and when a command including a backup operation command is received from the backup master server and transmitted to a backup manager server, a backup manager module in the backup manager server Manages backup schedule information for each volume, divides the volume of data from the backup target disk into unit data of a certain size, reads it, and sends it to the backup server, and backup status and backup history information for each volume according to the backup progress on the backup server side. Collects and manages them, and transmits a backup command for a disk volume to a backup dedicated server according to a backup schedule, and the fourth agent module in the backup server creates N threads by a backup command authorized from a backup manager module. In turn, unit data of size To the son opening the generated N number of threads compressing the divided unit data sequentially it is characterized in that is configured to store to the backup disc.

In addition, the high-performance replication method for a large amount of data according to the present invention, a replication target server having a replication target disk storing the replication target data, and a first agent for capturing predetermined unit data from the replication target disk; And a second agent for analyzing the predetermined unit data received from the first agent to generate duplicate data in a plurality of threads, and a duplicate disk storing the duplicate data generated from the second agent. A high performance replication method for a large amount of data using a high performance replication system for a large amount of data comprising a step of initiating real-time data replication for the replication target disk of the replication target server (S100), and optionally by the second agent Generating a queue by the number of the set multi-thread (S200), storing the data received from the first agent in the generated queue (S300), and parsing the data stored in the queue (Sars) Extracting a column value to be used as a key (S400), and multithreading each of the column values. Branching each other, and storing the column value in each branched multi-threaded queue (S500), extracting the data from the queue in which the column value is stored, and applying the extracted data to the replication server. It characterized in that it comprises a step (S600).

In addition, the high-performance replication method for a large amount of data according to the present invention, in the step (S200), respectively generating a data connection in the multi-thread, and connecting the object of the queue to the generated data connection (S210) It characterized in that it further comprises.

In addition, the high-performance replication method for a large amount of data according to the present invention, a replication target server having a replication target disk storing the replication target data, and a first agent for capturing predetermined unit data from the replication target disk; And a second agent for analyzing the predetermined unit data received from the first agent to generate duplicate data in a plurality of threads, and a duplicate disk storing the duplicate data generated from the second agent. A high performance replication method for a large amount of data using a high performance replication system for a large amount of data, comprising: initiating real-time data capture of a replication target disk of a replication target server (S1000) and optionally set by the first agent. Generating a queue by the number of multithreads (S2000), determining whether the number of the multithreads in operation is smaller than a predetermined number of multithreads (S3000), and the number of the multithreads in operation. Is greater than or equal to the maximum number of multithreads, Wait until the data capture cycle of the thread is completed (S4000), and if the number of the multithreads in operation is less than the maximum number of multithreads, the data capture proceeds after driving the multithread to proceed with data capture (S5000). And determining whether there is more data to be captured on the copy target disk (S6000) when the data capturing process is completed, and when the data to be captured is further present (S3000). If there is no more data to be captured, the process proceeds to the waiting step (S4000).

According to the present invention, in the process of copying data of a replication target system, data replication can be performed in real time using a multithreaded method.

In addition, another object of the present invention has the effect of performing a data backup in real time using a multi-threaded method.

1 is a block diagram showing the configuration of a high performance replication system for large amounts of data according to an embodiment of the present invention.
2 is a diagram illustrating the replication of a large amount of data by multithreading in a high performance replication system for large amounts of data according to an embodiment of the present invention.
3 is a diagram illustrating the replication of a large amount of data by a single thread in a high performance replication system for a large amount of data according to an embodiment of the present invention.
4 is a floor chart showing the flow of a high performance replication method for large amounts of data according to an embodiment of the present invention.
5 is a floor chart illustrating a flow of a data capturing method of data to be replicated in a high performance replication method for a large amount of data according to an embodiment of the present invention.
6 is a block diagram showing the configuration of a high performance backup system for a large amount of data according to an embodiment of the present invention.
7 is a block diagram showing the configuration of a high performance backup system for a large amount of data according to another embodiment of the present invention.
8 is a block diagram showing the configuration of a high performance backup system for a large amount of data according to another embodiment of the present invention.

BRIEF DESCRIPTION OF THE DRAWINGS The present invention is capable of various modifications and various embodiments, and specific embodiments are illustrated in the drawings and described in detail in the detailed description. It is to be understood, however, that the invention is not to be limited to the specific embodiments, but includes all modifications, equivalents, and alternatives falling within the spirit and scope of the invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, the present invention will be described in detail with reference to the accompanying drawings.

The terms first, second, etc. may be used to describe various components, but the components should not be limited by the terms. The terms are used only for the purpose of distinguishing one component from another.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In this specification, the terms "comprises" or "having" and the like refer to the presence of stated features, integers, steps, operations, elements, components, or combinations thereof, But do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, or combinations thereof.

Also, the terms " part, "" module," and the like, which are described in the specification, refer to a unit for processing at least one function or operation, and may be implemented by hardware or software or a combination of hardware and software.

In the following description, well-known functions or constructions are not described in detail since they would obscure the invention in unnecessary detail.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram showing the configuration of a high performance replication system for a large amount of data according to an embodiment of the present invention.

Referring to FIG. 1, a high performance replication system for a large amount of data according to an exemplary embodiment of the present invention may include a replication target disk 120 in which replication target data is stored, and a block of data of the replication target disk 120 in a plurality of predetermined units. After splitting into data, each section is calculated to capture certain unit data of a certain section with a plurality of threads, and the captured thread transfers the predetermined unit data to the queue and other sections after the certain section. A replication target server 100 having a first agent 110 for estimating and capturing predetermined unit data of another section; And a second agent 210 which analyzes each predetermined unit data received from the first agent 110 to generate duplicate data in a plurality of threads, and stores the duplicate data generated from the second agent 210. A replication server 200 having a disk 220.

First, the first agent 110 divides each block of data of the target disk 120 into a plurality of predetermined unit data, calculates a predetermined section arbitrarily, and captures predetermined unit data of the predetermined section with a plurality of threads. do. In this way, the thread that has completed the capture transfers predetermined unit data to the queue. In addition, when the capturing of predetermined unit data for a predetermined section is completed, the next other section is calculated as a random and serves to capture the predetermined unit data of another section.

Next, the copy target disk 120 is the actual copy target data to be copied by the user, and is preferably file data of a general file system. However, the replication target data may be transaction information of the online redo log of the database. That is, when the replication target data of the replication target disk 120 is a database, the replication target server 100 includes an operation DB, which generates the information data of the online redo log as an archive log file. At this time, the first agent 110 captures transaction information of the online redo log and replicates the transaction information to a second agent, which will be described later, in real time as a transaction log file.

Next, the second agent 210 analyzes each piece of predetermined unit data received from the first agent 110 to generate duplicate data in a plurality of threads. This will be described later in more detail with reference to FIGS. 2 and 3.

Next, the clone disk 200 stores the duplicate data generated from the second agent 210. In addition, the first agent 110 preferably transmits predetermined unit data to the second agent 210 in a queue structure.

Next, FIG. 2 is a diagram illustrating replication of a large amount of data by multithreading in a high performance replication system for a large amount of data according to an embodiment of the present invention.

Referring to Fig. 2, a structure in which data is replicated by a multithreaded method will be described.

As described above, the predetermined unit data received from the first agent 110 to the second agent 210 is copied to the copy disk 220 by a multithreaded method.

As described above, the first agent 110 of the replication target server 100 captures the transaction information of the replication target disk 120 and transmits the captured transaction information to the second agent 210 to the transaction log file. Will be created.

The multi-threaded data replication method for such a transaction log file is as follows.

The transaction log file received from the first agent 110 to the second agent 210 has a structure of a queue 300. The queue 300 is distributed and processed in parallel with respect to the PK column 310 or the unique index column 320. For example, when the queue 300 configures the queue 300 based on the PK column 310, the queue 300 includes PK = 100, PK = 200, PK = 300, PK = 400, and the like. In addition, when the queue 300 configures the queue 300 based on the unique index column 320, (TX # 1), (TX # 2), (TX # 3), and (TX # 4). , (TX # 5), (TX # 6), (TX # 7), and the like. Of course, in the present invention, for convenience of description, seven transaction log files have been described as an example as shown in FIG. 2, but the present invention is not limited thereto.

Next, the second agent 210 receiving the queue 300 creates the data connection 330 as many as the number of multithreaded 340. Here, the queues 300 having the same multi-key (the same PK column 310 or the same unique index column 320) are sequentially replicated to the replication system 220 using one data connection 330. Here, the number of multithreaded 340 may be appropriately adjusted according to the generation amount of the transaction log file.

On the other hand, Figure 3 is a diagram showing the replication of a large amount of data by a single thread in a high-performance replication system for large amounts of data according to an embodiment of the present invention.

Referring to FIG. 3, when a corresponding multi-key column value is changed during data replication by a multithreaded method, by temporarily applying a single threaded method, the unit of a transaction log file must be reinforced to ensure data consistency.

For example, as shown in FIG. 3, when the queue 300, which is a collection structure of transaction log files, is analyzed and PK = 400 is changed to PK = 300, the multi-key column value is changed. Regardless, only one data connection 330 is used to replicate to the replication system 220 sequentially.

When the processing for this is completed, as described in FIG. 2, the transaction may be applied again using a multithreaded method.

Next, FIG. 4 is a floor chart showing the flow of a high performance replication method for a large amount of data according to an embodiment of the present invention.

Referring to FIG. 4, a high performance replication method for a large amount of data according to an embodiment of the present invention includes capturing predetermined unit data from a replication target disk 120 and a replication target disk 120 in which replication target data is stored. A replication target server 100 having a first agent 110; And a second agent 210 which analyzes each predetermined unit data received from the first agent 110 to generate duplicate data in a plurality of threads, and stores the duplicate data generated from the second agent 210. As a high performance replication method for a large amount of data using a high performance replication system for a large amount of data including a replication server 200 having a disk 220, real-time data replication to the replication target disk 120 of the replication target server 100 Starting step (S100), creating a queue (S200) by the number of multi-threads arbitrarily set by the second agent 210 (S200), and received from the first agent 110 to the generated queue Storing the data (S300), parsing the data stored in the queue (Parsing) and extracting a column value to be used as a key (S400), branching the column value for each multi-thread, Extracting data from the step (S500) to gidoen store the column values for each of the multiple threads per queue, the queue the column values are stored, and a step (S600) for applying the extracted data to the replication server 200.

Here, in the step (S200) of generating a queue by the number of multi-threads arbitrarily set by the second agent 210, each of the multi-threads 440 generates a data connection, respectively, to the generated data connection The method may further include a step S210 of connecting the objects of the queue.

Next, FIG. 5 is a flowchart illustrating a flow of a data capturing method of data to be replicated in a high performance copy method for a large amount of data according to an embodiment of the present invention.

Referring to FIG. 5, the flow of a data capturing method of data to be replicated in a high performance replication method for a large amount of data according to the present invention is as follows.

A replication target server (100) having a replication target disk (120) in which replication target data is stored, and a first agent (110) for capturing predetermined unit data from the replication target disk (120); And a second agent 210 which analyzes each predetermined unit data received from the first agent 110 to generate duplicate data in a plurality of threads, and stores the duplicate data generated from the second agent 210. A high performance replication method for a large amount of data using a high performance replication system for a large amount of data including a replication server 200 having a disk 220, the real-time data capture of the replication target disk 120 of the replication target server 100 Initiating a step (S1000), generating a queue by the number of multithreads arbitrarily set by the first agent 110 (S2000), and the number of operating multithreads is smaller than the maximum number of multithreads set arbitrarily Step (S3000) and, if the number of multi-threads in operation is greater than the maximum number of multi-threads, the data of the multi-threads in operation Waiting until the capture cycle is completed (S4000), and if the number of running multithreads is less than the maximum number of multithreads, driving the multithread to perform data capture and then performing data capture (S5000), and When the process of capturing is completed, it is determined whether there is more data to be captured on the copy target disk 120 (S6000), and when there is more data to be captured, the process proceeds to the above step (S3000). If there is no data to be performed, the process proceeds to the waiting step (S4000).

Next, Figure 6 is a block diagram showing the configuration of a high-performance backup system for a large amount of data according to an embodiment of the present invention.

Referring to FIG. 6, in the high performance backup system for a large amount of data according to an embodiment of the present invention, a backup target disk 440 in which backup target data is stored, and a command including a backup operation command are input, and An input / output unit 460 for outputting a result, a central control unit 450 for processing a backup operation command applied to the input / output unit 460 to perform backup, and an input / output unit 460 and a central control unit 450 The backup master command 410, which is authorized through the backup operation command and transmits the backup operation command to the backup manager module 420, and the backup operation command required for the backup operation from the backup master module 410. Management, collect and manage backup status and backup history information for each volume, and send backup commands for the disk volume to the third agent 430 according to the backup schedule. A backup target server 400 having a backup manager module 420 and a third agent 430 for capturing predetermined unit data from the backup target disk 440; And a fourth agent 510 which analyzes each predetermined unit data received from the third agent 430 and generates backup data in a plurality of threads, and a backup which stores backup data generated from the fourth agent 510. Backup server 500 with disk 520.

Referring to FIG. 6, the high performance backup system for a large amount of data according to the present invention may be configured by separately separating the backup target server 400 and the backup server 500, but may be integrated into one computer system. Portions of the system not directly related to the present invention will be omitted.

6 includes a backup master module 410, modules which are unit units for performing one or more unique functions, such as the backup manager module 420 and the third agent 430, and from outside. Modules 410 by an input / output unit 460 to which instructions including a backup operation command are input, a backup target disk 440 storing target data to be backed up, and a command applied through the input / output unit 450. It consists of a central control unit 450 for controlling the, 420, 430.

In detail, the backup master module 410 performs a function for managing and operating the backup target server 400. When the backup manager module 420 includes a plurality of backup manager modules 420, a plurality of backup manager modules 420 may be provided. It is configured to operate and manage the backup in groups, and manages backup schedule information for each volume, and provides a backup command to the backup manager module 420 according to the backup schedule.

In this case, the backup schedule information means that the backup operator sets data in a certain disk, which backup disk, at a certain time period, and every few days for automatic backup, and the backup master module 410 according to the scheduled backup schedule. This operation is automatically performed, and the backup manager module 420 and the third agent 430 are configured to proceed with the backup.

The backup manager module 420 is configured to receive a backup operation command required for backup operation from the backup master module 410 and transmit the backup operation command to the third agent 430, and all of the backup progresses performed by the third agent 430. The information is configured to collect information on the backup status and backup history for each volume and apply the information to the backup master module 410.

The third agent 430 receives the backup command from the backup manager module 420 and performs a backup according to the received command. When the third agent 430 receives a backup command for the backup target disk 440, the backup target disk is received. The capture of the predetermined unit data in 440 is performed.

The predetermined unit data received from the third agent 430 is analyzed to store the backup data generated in the backup data in a plurality of threads on the backup disk 520.

The third agent module 430 is configured to collect and manage backup information for each volume while the backup is in progress, and to notify the backup manager module 420 of the backup progress.

As described above, the high performance backup system for a large amount of data according to the present invention divides and reads the data in the backup target disk 440 to be backed up into unit data, and a plurality of threads simultaneously compress the read unit data. The time required for the backup is greatly reduced due to the feature of storing the backup disk 520 in advance, and the compression rate of the data is increased to store more data in the same backup disk 520 environment.

Next, FIG. 7 is a diagram illustrating a configuration of a high performance backup system for a large amount of data according to another embodiment of the present invention.

Referring to FIG. 7, a high performance backup system for a large amount of data according to the present embodiment may include a backup manager module 420, a third agent 430, and a backup target disk 440 as a separate backup manager server 700. And a backup master server 600 having a backup master module 410 that receives a backup command and applies the backup command to the backup manager server 700. Between the backup master server 600 and the backup manager server 700, It may have a configuration connected to the interface or connected by a network, it may be configured in a tree type to manage a plurality of backup manager server 700 having such a configuration in one backup master server 600.

As such, the configuration and operation shown in FIG. 7 is not significantly different from the configuration and operation shown in FIG. 6, and a plurality of backups having a concept of correspondence of the client to the backup master server 600 when connected by an open network such as the Internet. The manager server 700 is authorized to manage the backup operation command according to the backup information reserved by one backup master server 600, and the backup manager server 700 according to the backup command applied to the backup manager module 420. The backup command is transmitted to the third agent 430, and the third agent 430 divides and reads the volume of data of the backup target disk 440 into unit data having a predetermined size, and then generates N threads. The compressed unit data may be sequentially compressed to be stored in the backup disk 520.

As described above, the embodiment according to FIG. 7 also divides and reads data in the backup target disk 440 into unit data to be backed up, and a plurality of threads compress the read unit data at the same time. 520) significantly reduces the time required for backup and increases the compression rate of the data, making it possible to store more data in the same backup disk environment, as well as to clients connected through an open network such as the Internet. For example, temporary backup manager servers 700 may be grouped into groups to operate and manage backups.

Next, Figure 8 is a block diagram showing the configuration of a high-performance backup system for a large amount of data according to another embodiment of the present invention.

Referring to FIG. 8, the backup master server 600, the backup manager server 700, and the backup server 500 are configured as separate servers, and each server is connected to an interface or a network to perform backup. For example, a plurality of backup manager servers 700 are connected to one backup master server 600, and backup servers 500 are connected to each backup manager server 700, respectively.

In this case, the backup target disk 440 is configured in each backup manager server 700 to store data, and the backup disk 520 is configured in each backup server 500 to store data of the backup target disk 440. It is configured to compress and store.

In the configuration of FIG. 8, when the backup master server 600 receives a command including a backup operation command and transmits the command to the backup manager server 700, the backup manager module 420 in the backup manager server 700 controls volume by volume. The backup schedule information is managed, the volume of data is divided into unit data of a predetermined size in the backup target disk, and read and transmitted to the backup server 500.

On the backup server 500 side, N threads are generated by a backup command applied from the backup manager server 700 side, and the generated N data are sequentially received by the N threads generated by the backup manager server 700 side. Compress and store the data on the backup disk.

Although the embodiments of the present invention have been described above, various modifications may be made by those skilled in the art. Therefore, it should be understood that the present invention should not be construed as being limited to the above embodiments, but should be construed in accordance with the following claims.

100: replication target server
110: first agent
120: Clone Target Disk
200: replication server
210: second agent
220: clone disk
400: Backup destination server
410: backup master module
420: backup manager module
430: third agent
440: Backup destination disk
450: central control unit
460: input and output unit
500: backup server
510: fourth agent
520: backup disc
600: backup master server
700: backup manager server

Claims (11)

A replication target disk on which replication target data is stored;
After dividing the block of data of the disk to be copied into a plurality of predetermined unit data respectively, a predetermined section is calculated to capture the predetermined unit data of the predetermined section with a plurality of threads, and the finished thread is a queue. A replication target server having a first agent which transmits the predetermined unit data to the mobile station and calculates another section following the predetermined section to capture the predetermined unit data of the other section; And
A second agent which analyzes each of the predetermined unit data received from the first agent and generates duplicate data in a plurality of threads;
A replication server having a replication disk for storing the replication data generated from the second agent;
The first agent transmits the predetermined unit data to the second agent in a queue structure;
The second agent creates a data connection by the number of multithreads of the predetermined unit data of the queue structure received from the first agent,
In the replica disk, transactions having the same PK column or the same unique index column are replicated using only one of the connections,
When the value of the PK column or the unique index column is changed during the real-time data replication by the multithreaded, the second agent converts the multithreaded into a single thread and sequentially uses the single data connection for the duplicate disk. The high-performance replication system for a large amount of data, characterized in that for copying the predetermined unit data. delete The method of claim 1,
The second agent is a high-performance replication system for a large amount of data, characterized in that for processing in parallel to distribute the predetermined unit data received from the queue based on a PK column or a unique index column. delete delete A backup target disk on which the backup target data is stored;
An input / output unit for inputting a command including a backup operation command and outputting a result of the predetermined command;
A central control unit which controls a backup to be performed by processing a backup operation command applied to the input / output unit;
A backup master module which receives the backup operation command applied through the input / output unit and the central control unit and transmits the backup operation command to a backup manager module;
Receiving the backup operation command required for backup operation from the backup master module to manage the backup schedule information for each volume,
A backup manager module that collects and manages backup status and backup history information for each volume, and sends backup commands for disk volumes to a backup agent module according to a backup schedule;
A backup target server having a third agent for capturing predetermined unit data from the backup target disk; And
A backup server having a fourth agent for analyzing the predetermined unit data received from the third agent and generating backup data in a plurality of threads, and a backup disk storing the backup data generated from the fourth agent; Including,
The third agent receives a backup command from the backup manager module, divides the volume of data of the backup target disk into unit data having a predetermined size, and creates N threads for performing several flows in one process. High performance backup system for a large amount of data, characterized in that for sequentially compressing the divided unit data transmitted to the fourth agent. The method according to claim 6,
A backup master server including the backup master module;
A plurality of backup target server including the backup manager module and the third agent, the backup target disk is provided separately,
When a command including a backup operation command is received from the backup master server and transmitted to the backup manager server, the backup manager module manages backup schedule information for each volume, collects and manages backup status and backup history information for each volume, and backs up the backup manager server. The backup command for the disk volume is transmitted to the third agent module according to a schedule, and the third agent module divides the volume of the data of the backup target disk into unit data having a predetermined size according to the backup command applied from the backup manager module. And generating N threads that perform several flows in one process, and sequentially compressing the divided unit data and transmitting the divided unit data to the fourth agent. . The method according to claim 6,
A backup master server including the backup master module;
A plurality of backup manager servers including the backup manager module and having a backup target disk; And a third agent, and a backup-only server having a backup disk is configured separately.
When the backup master server receives a command including a backup operation command and transmits the command to the backup manager server, the backup manager module in the backup manager server manages backup schedule information for each volume, and manages the volume of data on the backup target disk. It divides the data into unit data and sends it to the backup server side. It collects and manages backup status and backup history information for each volume according to the backup progress on the backup server side. In the fourth agent module in the backup server, N threads are generated by a backup command applied from a backup manager module, and the N threads generated by sequentially receiving unit data of a predetermined size sequentially process the divided unit data. Compressed to above High-performance back-up system for the high-volume data, wherein configured to store up to the disk. A replication target server having a replication target disk storing replication target data and a first agent for capturing predetermined unit data from the replication target disk; And a second agent for analyzing the predetermined unit data received from the first agent to generate duplicate data in a plurality of threads, and a duplicate disk storing the duplicate data generated from the second agent. As a high performance replication method for a large amount of data using a high performance replication system for a large amount of data, including;
Initiating real-time data replication for the replication target disk of the replication target server (S100),
Generating a queue by the number of multithreads arbitrarily set by the second agent (S200);
Storing the data received from the first agent in the created queue (S300);
Parsing the data stored in the queue and extracting a column value to be used as a key (S400);
Branching the column value for each multithread, and storing the column value in each branched multithreaded queue (S500);
Extracting the data from the queue in which the column value is stored, and applying the extracted data to the replication server (S600). The method of claim 9,
In the step S200,
Generating a data connection in the multithread, and connecting the object of the queue to the generated data connection (S210). A replication target server having a replication target disk storing replication target data and a first agent for capturing predetermined unit data from the replication target disk; And a second agent for analyzing the predetermined unit data received from the first agent to generate duplicate data in a plurality of threads, and a duplicate disk storing the duplicate data generated from the second agent. As a high performance replication method for a large amount of data using a high performance replication system for a large amount of data, including;
Initiating a real-time data capture for the replication target disk of the replication target server (S1000),
Generating a queue by the number of multithreads arbitrarily set by the first agent (S2000);
Determining whether the number of multithreads in operation is smaller than a predetermined number of multithreads (S3000);
When the number of multithreads in operation is equal to or greater than the maximum number of multithreads, the processor waits until the data capture period of the multithreads in operation is completed (S4000), and the number of multithreads in operation is the maximum multithread. If the number is less than, driving the multi-thread to proceed with data capture and proceeding to capture the data (S5000),
When the data capture process is completed, determining whether there is more data to be captured on the copy target disk (S6000);
If there is more data to be captured, the process proceeds to the step S3000, and if there is no more data to capture, the process proceeds to the waiting step S4000.

Images