KR20150007435A - Method of duplicating database using pararell-processing and computer readable storage media storing the same) - Google Patents

Abstract

The present invention relates to a database duplication method using parallel processing and a computer-readable recording medium storing a program therefor, which may significantly reduce processing time by processing transaction in parallel in database duplication equipment. To achieve this, the database duplication method using parallel processing comprises: a first step of extracting a slot ID value and a transaction ID value from each query; a third step of determining whether the lock of a slot indicated by the extracted slot ID value is obtained; a fifth step of determining whether there is a pre-allocated applier in the slot; a seventh step of allocating an applier to the slot when an applier has not been allocated thereto; a ninth step of increasing a reference count value; and an eleventh step of cancelling the lock of the slot.

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a parallel processing method and a computer readable recording medium storing the program.

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a parallel processing type database duplication method and a computer readable recording medium on which the program is recorded. More particularly, the present invention relates to a transaction processing time in a database duplication processing system.

Recent advances in database-related technologies, such as large amounts of fixed or unstructured data sets beyond the ability to collect, store, manage, and analyze data with existing database management tools, and big data processing to extract values from these data and analyze the results Let this snow.

The development of data processing technology, which is characterized by the generation, collection, analysis, and expression of various kinds of large data, predicts the more diverse modern society more accurately, works efficiently, provides customized information for individualized modern society members, Management, and analysis, and realizes technologies that were not possible in the past.

In this way, data processing presents the possibility of providing valuable information to society and mankind in all areas such as politics, society, economy, culture, and science and technology.

Meanwhile, in order to increase the stability of the data processing, a plurality of physically separated databases are subjected to a replication process of replicating and managing the changed contents of the local database to the remote database.

Especially, considering the recent environment in which many industries such as banking, shopping, and advancement are performing services on the Internet, database backup and duplication processes are very important in order to prevent data loss or service interruption due to the nature of the Internet site. It is important. Backup is to prepare and save a copy of the data that was being worked on, so as to prepare for the loss or deletion of data due to user's mistake, computer error, virus infection, etc. However, And the same data is stored in several servers at the same time in order to increase the availability, so that even if the main server fails, another server is turned on and the service is immediately restored.

As a prior art relating to database duplication, Korean Patent Laid-Open Publication No. 10-2005-0064278 and others have been disclosed.

1A is a diagram illustrating a process of duplicating a conventional database.

In the process of duplicating a conventional database, data to be duplicated of the main apparatus 100 being serviced is sensed and the data to be duplicated is transmitted to the redundant apparatus 200 located at a remote place. After receiving the data, the redundancy device 200 reflects the data in its own database.

In the conventional database duplication process, the data of the redundant device 200 is sequentially processed to reflect the order applied to the original data of the main device 100. [ For example, assume that the following query is applied to the original database. In addition, if the following query is applied to a table T1 composed of columns Col1 (primary key), Col2, commit is performed after each query is executed.

First query: Insert Into T1 Values (1, 'ABC'); // Insert a new record into table T1

Second query: Update T1 Set Col2 = 'BCD' Where Col1 = 1; // Change the second column value 'ABC' in table T1 to 'BCD'

The third query: Update T1 Set Col2 = 'CDE' Where Col1 = 1; // Change the value of the second column 'BCD' in the table T1 changed to 'CDE'

The fourth query: Insert Into T1 Values (2, 'ABC'); // Insert a new record into table T1

In this case, after all the queries are applied to the original database, two records are generated. Finally, column 1 is' 1 ', column 2 is' CDE', column 1 is' 2 ', column 2 is' Two records having a value of ABC 'are generated. Also, in the redundancy process, when the above data is transmitted to the remote server and applied, the final value of the corresponding record must have the same value.

In this case, if the query is not applied to the remote server sequentially and processed in parallel, the query execution order can not be guaranteed. For example, if the query is processed in parallel and the third query is applied first and the second query is applied later, the final value of the corresponding record will have a different value from the original database because column 2 becomes 'BCD'. To avoid this problem, the order of the queries applied to the original data must be maintained and applied to the remote server.

However, since the primary query value is different in the case of the fourth query, it may be performed independently of the sequential processing of the first to third queries. However, in the conventional redundancy process, the query is executed sequentially for all the queries regardless of the nature of the result of the execution of the query, which causes the overall performance degradation.

It is an object of the present invention to provide a method for processing a query in parallel to a remote server.

According to an aspect of the present invention, there is provided a method for processing a query, the method comprising: a first step of extracting a SlotID value and a Transaction ID value from each Query; A third step of determining whether a slot lock indicated by the extracted slot ID value is acquired; A fifth step of determining whether there is a pre-allocated Applicator in the slot; A seventh step of allocating a reflector when the reflector is not allocated; A ninth step of increasing a reference count value; And canceling the lock of the slot; .

And a sixth step of determining whether the transaction ID value extracted in the query processed by the pre-allocated reflector is the same as the transaction ID value extracted in the first step .

In addition, if it is determined in the sixth step that it is the same, the ninth step may be performed without going through the seventh step.

Also, if it is determined in step 6 that it is not the same, it is possible to wait until there is no reflector in the slot.

Also, if it is determined that it is not acquired in the third step, it can wait until the lock of the slot is obtained.

On the other hand, a first step of completing a commit of a transaction; A third step of extracting an ID value of a slot used by the transaction; A fifth step of acquiring a lock of the slot;

A seventh step of subtracting the number of queries performed in the transaction from the reference count; A ninth step of checking whether there is a query to be committed by checking the value of the reference count; An eleventh step of deleting the reflector from the slot; A thirteenth step of releasing the lock of the slot; And determining whether all the slots indicated by the ID values of the slots extracted in the third step are released.

In addition, if it is determined that the slot lock is not obtained in the fifth step, it can wait until the lock of the slot is obtained.

Also, if it is determined that the ninth step does not exist, the thirteenth step may be performed without going through the eleventh step.

In addition, if it is determined in step 15 that all slots are not unlocked, it is possible to acquire the lock of the remaining unallocated slots in the fifth step.

On the other hand, the above methods can be stored in a computer-readable recording medium on which a program for carrying out the method is recorded.

It is possible to process the duplication process of the remote server in parallel, thereby solving the bottleneck that may occur in the sequential data processing of the remote server.

1A is a diagram illustrating a conventional database duplication process.
1B is a diagram illustrating a database duplication process according to the present invention.
2 is a flowchart illustrating an operation of a distributor according to the present invention.
3 is a flowchart illustrating a process of recovering a reflector according to the present invention.
FIG. 4 is a schematic flow chart showing points where the process of collecting the distributor and the reflector according to the present invention are combined.
5A and 5B are conceptual diagrams illustrating an operation in which a query is queued and performed in response to a change in reflector in the same slot.
6A and 6B are diagrams showing time consumed in performing a transaction.

Hereinafter, embodiments and examples of the present invention will be described in detail so that those skilled in the art can easily carry out the present invention.

It should be understood, however, that the present invention may be embodied in many different forms and is not limited to the embodiments and examples described herein.

Throughout this specification, when an element is referred to as " including " an element, it is understood that the element may include other elements as well, without departing from the other elements unless specifically stated otherwise.

Throughout this specification, when a step is positioned "before" or "after" another step, it includes not only when a step directly interacts with another step, but also when there is another step between the two steps .

        The terms " about ", " substantially ", etc. used to the extent that they are used throughout the specification are used in their numerical value or in close proximity to the numerical values when the manufacturing and material tolerances inherent in the meanings mentioned are presented, To prevent unauthorized exploitation by an unscrupulous infringer of precisely or absolutely stated disclosures.

The term " step " or " step of ~ " used throughout the specification does not mean " step for.

The term "electronic document " used in the present specification is a concept including an electronic certificate issued by an institution, which is information prepared in an electronic form, transmitted, received or stored by an information processing system.

BRIEF DESCRIPTION OF THE DRAWINGS The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this application, illustrate embodiments of the invention and, together with the description, serve to explain the principles of the invention.

1B is a diagram illustrating a database duplication process according to the present invention.

 Prior to describing the operation of the distributor, the concept of the transactions, distributor, slot, reflector, reference count and commit referred to herein will be described.

First, a transaction is a unit in which one or more queries are committed or rolled back. One transaction is reflected in the database of the redundant device by one reflector. If there are five reflectors, there are five transactions that can be performed simultaneously. As will be described in more detail below, as shown in the first embodiment, the first to third queries constitute the same transaction since the first to third queries are input and committed. Meanwhile, since the fifth and sixth queries are input and committed, the fifth and sixth queries constitute a transaction different from the first to third queries.

Next, the Distributor determines which records are reflected in the redundant equipment and distributes them to the reflectors. More specifically, when different transactions want to modify the same record, they are sequentially processed to prevent simultaneous processing on the same record.

Next, a slot (Slot) can be conceptualized as a processing place of queries to be processed in the same record. To determine if a query is processing the same record, you can analyze the query or analyze the record ID, which is the only value in the record. For example, you can determine if you are using the same primary key in a table, or a record ID that can identify the same record, to determine if you are processing the same record. It is needless to say that the method of processing the same record is not limited to that mentioned above. The primary key, on the other hand, uses 64 bits, increasing the overall computational complexity and increasing the loss. Of course, the primary key is not limited to the 64 bits mentioned, but may be a larger value or a smaller value. Therefore, in the present invention, the concept of a slot is used instead of using a primitive key or a record ID, thereby greatly reducing the amount of computation. As will be described in more detail below, as shown in the first embodiment, the first query through the third query and the sixth query are processed in the same record, so they have the same slot, and the fifth query is processed in the other record Have different slots. Meanwhile, the slot in the present invention can be recycled unlike the primary key. Assuming, for example, that there are 1,000 or more queries and the number of slots is limited to 100, it is assumed that three slots are used in the first embodiment, For the 233rd query processing a record, it can be assigned to any one of slots 1 to 100. In other words, a slot with a limited number of queries is allocated for a large number of queries. A module operation or the like may be used to allocate a limited number of slots to each query, but the present invention is not limited thereto.

The reflector is used to reflect the transaction to the actual redundant equipment. For example, if there are five reflectors, five trajectories can be performed at the same time.

The reference count refers to the number of records in use in the current slot. As will be described in more detail below, the first to third queries use the same slot, and thus have a reference count of 3, as shown in the first embodiment.

A commit is a generic term associated with a database that is used to permanently update the content of a database transaction. Normally, you use "commit" to commit the update at the end of the transaction.

FIG. 2 is a flow chart for a parallel processing of a query according to the present invention. The operation of a distributor for concurrently performing a transaction is described in detail.

Referring to FIG. 2, operations of the distributor include extracting an ID value of a slot and an ID value of a transaction (S210); Acquiring a lock of a slot relating to the extracted transaction (S220); A step S230 of confirming whether a reflector exists in the slot in which the lock is acquired; Allocating a new reflector if the reflector is not present in the acquired slot (S240); Incrementing the reference count value by 1 (S260); Unlocking the corresponding slot that has acquired the lock (S270); If it is determined that the slot lock can not be obtained (S220), a step S225 is awaited until the slot acquires the lock (S225); (S250) if it is determined that there is a reflector in the slot obtained by acquiring the lock, and whether the transaction ID value of the query whose reflector is set in the corresponding slot is the same as the transaction ID value extracted in step S210; (S255) if the reflector ID values are not identical.

Hereinafter, the present invention will be described in more detail with reference to the following first embodiment.

Query No. Processed content SQL The first query Insert a new Record. InsertIntoT1Values (1, 'ABC'); The second query Change the value of the second column 'ABC' to 'BCD'. UpdateT1SetCol2 = 'BCD'WhereCol1 = 1; The third query Change the above 'BCD' to 'CDE'. UpdateT1SetCol2 = 'CDE'WhereCol1 = 1; The fourth query Commit the transaction. commit; Fifth query Insert a new Record. InsertIntoT1Values (2, 'ABC'); The sixth query Change the value of the second column 'ABC' to 'XYZ'. UpdateT1SetCol2 = 'XYZ'WhereCol1 = 1; Seventh query Commit the transaction. commit; Eighth query Insert a new Record. InsertIntoT1Values (3, 'ABC'); · · · · · · · · ·

≪ First embodiment, queries received in the redundant equipment >

When eight queries are received in the duplication device as in the first embodiment, the distributor first extracts the slot ID value and the transaction ID value from the received query (S210). For the first query, the transaction ID value is 1 and the slot ID value is 1. The query may include specific information so that the distributor can extract the transaction ID value and the slot ID value from the received query.

Next, it is determined whether or not the slot having the ID value of 1 is locked. If the slot is not locked, the lock is performed (S220).

Then, it is determined whether there is a reflector in a slot having an ID value of 1 (S230).

If there is no reflector, a reflector is set in a slot having an ID value of 1 (S240). In the first embodiment, it is assumed that there are five reflectors. In this case, 1 is set as a reflector ID value in a slot having an ID value of 1. On the other hand, setting the reflector ID value may be performed randomly as setting the slot ID value.

Since the record using the slot having the ID value of 1 is the first record to be processed first by the first record, the reference count value is incremented by 1 (S260).

Thereafter, the lock of the slot having the ID value of 1 is canceled (S270).

This action is repeated when the distributor reads the second query and the third query, and increases the reference count because it processes the same first record. In the first embodiment, the reference count is increased from 0 to 3.

Next, a step in which the distributor reads the fifth query will be described. In this case, since the set of committed queries is changed, the transaction ID value is changed to 2, and the slot ID value is also changed to 2 (S210) because the transaction to be processed is also changed.

Next, it is determined whether or not the slot having the ID value of 2 is locked. If the slot is not locked, the lock is performed (S220).

Then, it is determined whether a reflector exists in a slot having an ID value of 2 (S230).

If there is no reflector, a reflector is assigned to a slot having an ID value of 2 (S240). In the first embodiment, the reflector ID value is set to 2 in the slot having the ID value of 2.

A record using a slot having an ID value of 2 is the second record on the table of the duplication equipment, and the fifth query processes another record unlike the first to third queries. Therefore, the reference count value of the fifth query using another slot is incremented by one. On the other hand, it is assumed that the reference count value is initialized to 0 in the first embodiment.

Thereafter, the lock of the slot having the ID value of 2 is canceled (S270).

Next, the distributor reads the sixth query. Since the sixth query is contained within the same set of queries that are committed as the fifth query, the transaction ID value is still extracted as two. Meanwhile, since the sixth query processes the same record as the first through third queries, the value of 1, which is the same value as the first through third queries, is used as the slot ID value (S210).

Next, it is determined whether or not the slot having the ID value 1 is locked. If the slot is not locked, the lock is performed (S220).

Then, it is determined whether there is a reflector in a slot having an ID value of 1 (S230). In the sixth query of the first embodiment, since the reflector has already been allocated to the slot having the ID value of 1, it can be determined that the reflector exists because the reflector is in operation. In step S250, it is determined whether the transaction ID value of the query being processed by the reflector assigned to the corresponding slot is the same as the transaction ID value extracted in step S210 (S250). For example, referring to the first embodiment, a reflector having an ID value of 1 is currently in operation in a slot having an ID value of 1, and a query in which the reflector is in operation has a transaction ID value of 1 to be. On the other hand, the ID value of the transaction of the query extracted in S210 is 2, and thus the values thereof are different from each other.

Therefore, the lock to the slot having the ID value of 1 is released and waits (S255, S225). This is because the reflector having 1 as the ID value in the slot having the ID value of 1 associated with the first to third queries must be waiting for the first to third queries to complete the task .

Thus, the distributor plays a role in regulating the reflector work order of the same slot. To this end, queries having the same transaction ID value have the same reflector ID value.

FIG. 3 is a flowchart mainly showing operations after a transaction is reflected in a database through a reflector.

After the transaction according to FIG. 3 is reflected in the database, the operation is performed in a transaction commit completion step S310; Acquiring slot information currently used by the transaction (S320); Acquiring a lock of the slot extracted in S320 (S330); Reducing the reference count (S340); Checking whether the reference count is 0 (S350); Deleting the reflector of the slot (S360); Releasing the lock of the slot (S370); And checking whether all slots have been processed (S380).

Hereinafter, the operation after the transaction is performed through the reflector will be described in detail based on the first embodiment.

First, an input transaction is performed (S310). Referring to the first embodiment, since the first to third queries constitute one transaction, the first to third queries are committed.

Then, information on a slot used by the transaction is obtained (S320). At this stage, information on a slot may be acquired by referring to a memory of a distributor, rather than extracting slot information by analyzing a query as in S210 of FIG.

A lock is acquired for the next slot (S330). It is determined whether or not the lock of the extracted slot can be obtained in the preceding step S320. Because it must not be affected by other trackings.

After acquiring the lock, the reference count is decreased by the number of queries executed in the transaction (S340). Since the transaction having the ID value of 1 in the first embodiment includes the first to third queries, the number of queries is reduced by 3 from the reference count value of three.

Thereafter, it is determined whether the reference count is 0 (S350). Thereafter, the reflector is deleted from the slot (S360), and then the corresponding slot is unlocked (S370). In the case of the sixth query of the first embodiment, since the reflector having 1 as the ID value is already allocated to the slot having the ID value of 1, the execution of the first to third queries is completed, We have already explained that we must wait until

FIG. 4 is a diagram showing the points where the processes associated with the respective drawings are combined in the simplification of FIGS. 2 and 3. FIG.

Referring to FIG. 4, if the reflector is deleted from the slot in S360, the reflector can be reassigned and the query can be continuously performed in conjunction with S230 of FIG.

Referring again to FIG. 3, it is checked whether the processing of the slot of the query included in the current transaction is completed (S380). For example, in the first embodiment, a transaction having an ID value of 2 uses a slot having an ID value of 1 and 2. Therefore, the above-described steps S330 to S370 should be processed for the query of all slots used.

On the other hand, if the lock of the slot is not obtained (S330), the process waits until the lock of the slot is obtained (S335).

On the other hand, if the reference count is not 0, the lock of the slot is immediately canceled without deleting the reflector of the slot (S370). If the reference count is not 0, it means that transactions and queries to be processed using the corresponding slot still exist, and it is not necessary to delete the reflector of the corresponding slot.

Meanwhile, it is assumed that the commit of the transaction having the ID value of 2 associated with the fifth query of the first embodiment is completed (S310).

Then, the ID value 2 is obtained from the slot information currently used by the transaction (S320). Thereafter, the lock of the slot having the ID value of 2 is obtained again (S330). In the first embodiment, since there is only one slot having an ID value of 2, it can be seen that the lock is not locked.

Since the fifth query has been executed since then, the reference count value is decremented by 1 from the reference count value of 1 (S340). As a result, the reference count value becomes 0, and it can be determined that the slot is not used any more at the present time. Thereafter, the reflector of the corresponding slot is deleted, and it is confirmed that the reflector having the ID value of 2 is no longer used (S360). Of course, it is not necessary to end the reference count value from '0' to '0', and it is enough to recognize the number of processing times of records processed in the current slot.

Thereafter, the lock of the slot having the ID value of 2 as the corresponding slot is canceled (S370), and it is confirmed whether there is any more query to be processed (S380).

By repeating this method, all the reflectors allocated in the slot can be retrieved and reallocated.

5A and 5B are conceptual diagrams illustrating an operation in which a query is queued and performed in response to a change in reflector in the same slot.

Referring to FIG. 5A, a reflector having 1 as an ID value and a reflector having 2 as an ID value are assigned to slots having IDs of 1 and slots having IDs of 2, respectively, and work is started. At this time, if a query having another transaction ID value in the same slot is input as in the sixth query, the sixth query is queued in the waiting queue. Of course, the slot is unlocked before queuing the queue.

Referring to FIG. 5B, when all the first to third queries are executed and the reflector is deleted in the corresponding slot, the wait state is canceled and the sixth query acquires a slot having ID as 1, As shown in FIG. As shown in FIG. 5B, the reflector moves to the slot having the ID value of 1 to perform the operation, and the seventh query, the eighth query, and the like can be performed consecutively.

Then, the lock of the slot whose ID value is 1 is canceled.

Finally, the above steps are repeated for the eighth query to set the transaction ID value to 3, the slot ID value to 3, the reflector ID value to 3, and the reference count value to 1.

The relationship between the transaction ID, the slot ID, the reflector ID, and the reference count value according to the steps from the first query to the eighth query is shown in the following table.

Query No. Transaction ID Slot ID Reflector ID Reference count The first query One One One One The second query One One One 2 The third query One One One 3 The fourth query Fifth query 2 2 2 One The sixth query 2 One 2 One Seventh query Eighth query 3 3 3 One

<ID value according to the first embodiment>

6A and 6B are diagrams showing time consumed in performing a transaction.

FIG. 6A is a diagram illustrating a time taken for performing a transaction according to a conventional method. As shown in FIG. 6A, although the record to be handled is different, a transaction is performed using a single reflector, and therefore, it is inevitable to sequentially perform the transaction. Therefore, the time consumed to complete an entire transaction is long.

FIG. 6B is a diagram illustrating a time for performing a transaction according to the present invention. Explaining with reference to the first embodiment, a transaction having an ID value of 1 and a transaction having an ID value of 2 will be sequentially executed by the distributor because they modify the same record. In the case of a transaction, it can be done in parallel by the distributor since it adds another record.

Thus, compared with the conventional method, the execution time by? Is reduced. If the number of transactions is large and the number of concurrent transactions is increased, the time required to duplicate the entire database can be drastically reduced.

Claims (10)

A first step of extracting a SlotID value and a Transaction ID value from each Query;
A third step of determining whether a slot lock indicated by the extracted slot ID value is acquired;
A fifth step of determining whether there is a pre-allocated Applicator in the slot;
A seventh step of allocating a reflector when the reflector is not allocated;
A ninth step of increasing a reference count value; And
An eleventh step of releasing the lock of the slot; And a database processing unit for executing the parallel processing. The method according to claim 1,
And a sixth step of determining whether the transaction ID value extracted in the query processed by the pre-allocated reflector is the same as the transaction ID value extracted in the first step, if it is determined to exist in the fifth step Wherein the parallel processing method is a database duplication method. 3. The method of claim 2,
And if it is determined to be the same in the sixth step, the ninth step is performed without going through the seventh step. 3. The method of claim 2,
And if it is determined in step 6 that it is not the same, waits until there is no reflector in the slot. 5. The method according to any one of claims 1 to 4,
And if it is determined in the third step that it is not acquired, waits until the lock of the slot is obtained. A first step of completing a commit of the transaction;
A third step of extracting an ID value of a slot used by the transaction;
A fifth step of acquiring a lock of the slot;
A seventh step of subtracting the number of queries performed in the transaction from the reference count;
A ninth step of checking whether there is a query to be committed by checking the value of the reference count;
An eleventh step of deleting the reflector from the slot;
A thirteenth step of releasing the lock of the slot; And
And determining whether all the slots indicated by the ID values of the slots extracted in the third step are unlocked. The method according to claim 6,
And if it is determined that the lock of the slot is not obtained in the fifth step, waiting until the lock of the slot is obtained. The method according to claim 6,
And if it is determined in step 9 that it does not exist, the step 13 is performed without going through step 11. The method according to claim 6,
And if it is determined in step 15 that all slots are not released, acquiring locks of remaining slots that have not been released in the fifth step. A computer-readable recording medium having recorded thereon a program for performing the method of any one of claims 1 to 9.

Images