JP2015156119A - Distributed database system, database device, and data synchronization method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To efficiently synchronize data.

SOLUTION: Each of database devices 10a-10d includes: a storage unit 111 for storing a plurality data tables, each having a record differing in size; order analysis means 101 for analyzing the order of sizes of records that each data table has; center of gravity calculation means 102 for obtaining the amount of change in size in ascending order of size for each record, and calculating the center of gravity value of the obtained amounts of change; ratio calculation means 103 for obtaining a first difference value that is a difference between the order of the minimum value and the center of gravity value of record sizes and a second difference value that is a difference between the order of the maximum value and the center of gravity value of record sizes, calculating a ratio of the first difference value to the second difference value, and making the calculated ratio to be a ratio of a first record that each transfer data includes to a second record larger in size than the first record; and transfer means 105 for transferring each data formed with the calculated ratio to another database.

COPYRIGHT: (C)2015,JPO&INPIT

Description

  The present invention relates to a distributed database system having a plurality of databases, a database apparatus, and a data synchronization method.

  There is a distributed database system that has a plurality of database devices (DB devices) and each DB device manages data in a multiplexed manner. In such a distributed database system, each device is set as an ACT system (active system) or an SBY system (standby system). Normally, access from an external device is connected to an ACT DB device, but the ACT system and the SBY system are switched when the ACT DB device fails. Thereafter, access from an external device is connected to a new ACT DB device.

  In the distributed data system, normal synchronization processing can be resumed by promptly switching between the ACT system and the SBY system. For example, Patent Document 1 describes a technique for reducing the load of data processing among a plurality of DB devices, completing switching quickly, and restarting normal processing in such a distributed database system.

  In the distributed database system, when the network load necessary for data synchronization between the DB devices increases, the network resources necessary for normal processing decrease. Therefore, it is preferable to reduce each DB apparatus and network load at the time of synchronizing data between each DB apparatus.

JP 2013-46346 A

  However, in Patent Document 1 described above, although the load at the time of switching is reduced, consideration is given to reducing the network load when synchronizing normal data and performing data synchronization efficiently. Absent.

  In view of the above problems, an object of the present invention is to efficiently synchronize data in a distributed database system.

  In order to achieve the above object, a distributed database system according to the present invention is connected to a plurality of database devices, transmits data stored in one of the database devices to another database device, and receives data from the plurality of database devices. Each database device stores a plurality of data tables each having a record having a different size, and a ranking for analyzing the order of the sizes of the records included in each data table stored in the storage unit. The analysis means, the centroid calculation means for calculating the size change amount in ascending order of the ranks analyzed by the rank analysis means, and calculating the centroid value of the calculated change amount, and the rank and centroid calculation of the minimum value of the record size The difference between the first difference value, which is the difference from the centroid value calculated by the means, and the rank of the maximum value of the record size and the centroid value A certain second difference value is obtained, a ratio between the first difference value and the second difference value is calculated, the first record including the obtained ratio in each transfer data, and the size is larger than that of the first record. A ratio calculating unit that sets the ratio of the second large record and a transfer unit that transfers each data formed at the ratio calculated by the ratio calculating unit to another database.

  According to the present invention, data can be efficiently synchronized in a distributed database system.

FIG. 1 is a schematic diagram illustrating a distributed database system according to an embodiment. FIG. 2 is a block diagram illustrating the configuration of the DB device according to the embodiment. FIG. 3 is a diagram illustrating an example of data transferred by the DB device. FIG. 4 is a diagram illustrating another example of data transferred by the DB device. FIG. 5 is a diagram for explaining the size of data transferred by the DB device. FIG. 6 is a diagram illustrating an example of data transferred by the DB device according to the embodiment. FIG. 7 is a diagram for explaining the relationship between the record sizes of data to be transferred by the DB device. FIG. 8 is a diagram for explaining a data synchronization method according to the embodiment. FIG. 9 is a diagram for explaining the data synchronization method according to the embodiment following FIG. FIG. 10 is a diagram for explaining the effect of the data synchronization method according to the embodiment. FIG. 11 shows the first simulation result. FIG. 12 shows the second simulation result.

  Hereinafter, a distributed database system, a database device, and a data synchronization method according to each embodiment of the present invention will be described with reference to the drawings. In the following description, the same components are denoted by the same reference numerals and description thereof is omitted.

  As shown in FIG. 1, the distributed database system 1 according to the embodiment includes a plurality of database devices (DB devices) 10a to 10d, and the DB devices 10a to 10d store data in synchronization. In the distributed database system 1 shown in FIG. 1, the first DB device 10a and the second DB device 10b are ACT systems (active system), and the third DB device 10c and the fourth DB device 10d are SBY systems (standby system). Further, the first DB device 10a and the third DB device 10c are set as masters, and the second DB device 10b and the fourth DB device 10d are set as slaves (Slave).

  Specifically, normally, when data update is requested from the user terminal 20, the data is updated by the master and the ACT first DB device 10a. Thereafter, the data is synchronized with the data in the first DB device 10a by the second ACT system DB device 10b as a slave and the SBY system third DB device 10c as a master device. After that, the slave is synchronized with the data of the second DB 10b by the SBY fourth DB device 10d.

  If the first DB device 10a is changed from the ACT system to the SBY system and the third DB device 10c is changed from the SBY system to the ACT system, the user terminal 20 is accessed not by the first DB device 10a but by the third DB device 10c, Data of the third DB device 10c is updated first. Thereafter, the data is synchronized with the other DB devices 10a, 10b, and 10d.

  As shown in FIG. 2, the first DB device 10a according to the embodiment includes a rank analysis unit 101 that analyzes records included in data to be transferred at the time of data transfer, and a size of a record to be transmitted from the analysis result of the records. Centroid calculation means 102 for calculating the centroid of the data, ratio calculation means 103 for calculating the ratio of two different sizes included in the transmission target, storage processing means 104 for storing the transfer data generated at the calculated ratio, and transfer And transfer means 105 for transmitting data to the transfer destination DB apparatuses 10b and 10c. Further, the first DB device 10a stores data input from the user terminal 20 in the data table storage unit 111, and stores transfer data generated at the time of data transfer in the transfer buffer 112.

  As shown in FIG. 1, the DB devices 10b and 10c that receive the transfer data transferred from the first DB device 10a set to Master in the ACT system store the received transfer data in the storage device 110 of the own device. The data is synchronized.

  In the distributed database system 1 shown in FIG. 1, the data input from the user terminal 20 is stored in the first DB device 10a set as the master in the ACT system, and the other DB devices 10b to 10d are the first DB device 10a. Synchronized with. However, if the settings of the DB devices 10b to 10d are changed, data input from the user terminal 20 may be stored in the DB devices 10b to 10d. Therefore, the other DB devices 10b to 10d have the same configuration as the first DB device 10a shown in FIG.

  As shown in FIG. 3A, the tables A to D, which are data stored in the distributed database system 1, are formed under conditions where the size and the number of records per record are different. For example, the table A is data having “8” records and the size of each record being “5”. Further, the table B is data having “20” records and the size of each record being “2”. Further, the table C is data having “9” records and the size of each record being “7”. The table D is data having “14” records and the size of each record being “1”.

  In the distributed database system 1, for example, when synchronizing the tables A to D formed as shown in FIG. 3A, it is efficient if the data of each table A to D is stored in the transfer buffer 112 and transferred sequentially. This is not an effective transmission. Specifically, as shown in FIG. 3B, data is transmitted at the timing when resource constraints such as the upper limit size of the transfer buffer 112 and the upper limit number of transfer records are reached, and the size of each transfer data Varies from transmission to transmission. In the case of FIG. 3B, the upper limit size of the transfer buffer 112 is “42” or more, but since the upper limit number of transfer records is “6”, transfer data with the record number “6” or more is transmitted. Can not do it.

  Therefore, in order to prevent the size of each transfer data from fluctuating for each transmission, for example, (1) a method for reducing the number of transfer records, (2) only the condition for the size of transfer data is set, and the number of transfer records is set. A method of making it variable is conceivable. For example, FIG. 4 illustrates an example in which transfer data is stored in the transfer buffer 112 when the methods (1) and (2) are used.

  FIG. 4A is the same as FIG. 3B, and shows an example in which transfer data is stored in the transfer buffer 112 with a predetermined resource constraint. FIG. 4B shows an example in which the transfer data is stored and transmitted as the data of each of the tables A to D by the “method of reducing the number of transfer records” of (1) above. Since the number of records stored in the transfer buffer 112 is limited to 3, the variation in the data size of the transfer data stored in the transfer buffer 112 each time is reduced as compared with the case of FIG. However, when the number of transfer records is reduced as shown in FIG. 4B, there is a problem that the number of data transfer increases and the transfer time becomes longer.

  In FIG. 4B, the transfer data is stored in the data of each table A to D according to the above-mentioned (2) “method of setting only the transfer data size condition and changing the number of transfer records”. It is an example to transfer. Thus, by not setting the condition for the number of transfer records, the number of records stored in the transfer buffer 112 can be increased and the number of transfers can be reduced. However, since the update of the target record is locked during the transfer, if the number of transfer records increases, processing restrictions using the target record occur, and the constraint rate increases.

  FIG. 5 shows an example of a simulation result when the upper limit size of the transfer buffer is “3000” and the upper limit number of transfer records is “30”. In FIG. 5A, the horizontal axis is the number assigned to the table to be transferred, and the vertical axis is the number of records in each table. In FIG. 5B, the horizontal axis is the number of the transfer buffer 112 that stores the transfer data, and the vertical axis is the size of the transfer data stored in each transfer buffer 112.

  This simulation is an example in the case of synchronizing data with the number of tables “100” as shown in FIG. Here, the number of records of each table is determined by a random number from 1 to 100, and the size of each record is determined by a random number from 1 to 100.

  When data is stored in the transfer buffer 112 sequentially under the conditions, specifically, in the order of the table numbers, and the records of the selected table are sequentially stored in the transfer buffer 112, as shown in FIG. The size of the transfer data stored in is random. For example, the transfer data stored in the transfer buffer 112 with the buffer numbers “4” and “5” has reached the upper limit value of the transfer record even though the upper limit size of the transfer buffer 112 has not been reached. The above data is not stored. In the simulation shown in FIG. 5B, the result that the size of the transfer data of each buffer number fluctuates with the standard deviation “683” was obtained.

  On the other hand, in the first DB device 10a according to the embodiment, as shown in FIG. 6, instead of storing data in the transfer buffer 112 in the order of table numbers, data including records having a plurality of different sizes is transferred as transfer data. Store in the buffer 112. Thereby, the variation in the data size of each transfer buffer 112 can be reduced.

  FIG. 6A shows tables A to D stored in the distributed database system 1. For example, the table A is data having “8” records and the size of each record being “5”. In FIG. 6A, in addition to the size of each record and the number of records, the size of a record in each table A to D is represented. According to FIG. 6A, the size of the record is larger in the order of the tables C, A, B, and D. From the tables A to D configured as described above, transfer data including a plurality of records of different sizes is stored in the transfer buffer 112 as shown in FIG.

  Here, when the data synchronization method is executed in the first DB device 10a according to the embodiment, in order to form the transfer data to be stored in the transfer buffer 112, the record of each table stored in the storage device 110 is analyzed, and which table Specify whether to combine each record. For example, it is assumed that the record size of each table is analyzed as shown in FIG. 7, and there are many tables with a small record size and few tables with a large record size. In this case, the first DB device 10a can reduce variation in the data size of each transfer data by generating transfer data with a small number of large records and a large number of small records.

  Specifically, when the rank analysis unit 101 synchronizes the table having the table numbers 1 to 100 as shown in FIG. 8A with the other DB devices 10a to 10d, as shown in FIG. 8B. The order of record size is set for the records included in each table. In FIG. 8B, the horizontal axis represents the size order of each record, and the vertical axis represents the record size. Here, the number of tables in FIG. 8A is 100, whereas the order of the record size in FIG. 8B is 100 or more, because each table includes a plurality of records. Here, each table is described as having a plurality of records of one type.

  Subsequently, the center-of-gravity calculation unit 102 uses the ranks of the record sizes analyzed by the rank analysis unit 101 to obtain the change amount when the size changes in the ascending order as shown in FIG. 9A. The center of gravity of the size change amount is calculated.

  In FIG. 9A, the horizontal axis represents each “record size rank”, and the vertical axis represents each “record size change amount” in ascending order. In FIG. 9A, the numbers attached to the horizontal axis are “record size ranks” that specify the ranks of records whose record sizes have changed. Therefore, for example, in the example shown in FIG. 9A, it can be seen that the size of the smallest record and the size of the second smallest record have changed to the rank “21.5”.

  The ratio calculation means 103 is a difference between the “first difference value” that is the difference between the rank of the minimum value of the record size and the centroid calculated by the centroid calculation means 102, and the difference between the rank of the maximum value of the record size and the centroid. A “second difference value” is obtained. Further, the ratio calculation unit 103 calculates the ratio between the first difference value and the second difference value, and sets the calculated ratio as the ratio between the first record and the second record included in the transfer data. Here, the first record and the second record are records included in different tables, respectively, and the size of the first record is smaller than the size of the second record.

  When the center of gravity is calculated as shown in FIG. 9A, the ratio calculation means 103 obtains the first difference value and the second difference value as shown in FIG. 9B. In the example shown in FIG. 9B, the ratio between the first record and the second record is “2423: 2568”. Here, the value calculated by the ratio calculation means 103 does not necessarily specify the ratio between the first record and the second record of each transfer data. For example, the ratio is “1: 1.0598...”, And data cannot be configured with such a ratio. Therefore, the ratio of the sum of the first records and the sum of the second records of all transmission data may be “2423: 2568”. Specifically, since the number of all records is “4991 (2423 + 2568)”, the total number of first records of all transmission data is “2423”, and the total number of second records is “2568”. If it is.

  The storage processing unit 104 reads the record data of each table from the data table storage unit 111 at the ratio obtained by the ratio calculation unit 103 and stores it in the transfer buffer 112 as transfer data.

  When the storage processing unit 104 stores the transfer data in the transfer buffer 112, the transfer unit 105 transmits the transfer data stored in the transfer buffer 112 to the target database devices 10b and 10c. Here, when the transfer data is stored in the transfer buffer 112, the transfer unit 105 does not need to wait until the next transfer data is stored, and may transmit immediately.

  Using FIG. 10, when transfer data is generated by extracting records sequentially (ie, in the order of the data table stored in the data table storage unit 111 (FIG. 10A)), as in the embodiment, The data size of each transfer data is compared with the case where the transfer data including the one record and the second record at the required ratio is generated (FIG. 10B). Here, both FIG. 10A and FIG. 10B are examples of cases where transfer data is generated from the records in the table of 100 shown in FIG. 8A.

  As shown in FIG. 10A, it can be seen that when transfer data is generated sequentially, the sizes of the transfer data vary. On the other hand, as shown in FIG. 10B, when generating transfer data in the method including the first record and the second record at the required ratio, that is, in the data synchronization method according to the embodiment, It can be seen that the variation in the transfer data is small. Actually, when the standard deviation of the transfer data size is obtained, it is “683” in the example shown in FIG. 10A, whereas it is “41” in the example shown in FIG. .

  A similar comparison was performed in another example. The first example is an example for a data table of record size and record size order as shown in FIG. When the transfer data is generated sequentially, as shown in FIG. 11B, the standard deviation of each transfer data is “286” and the data size varies. On the other hand, when the transfer data is generated by the data synchronization method according to the embodiment, the standard deviation of each transfer data is “45” as shown in FIG.

  The second example is an example for a data table of record size and record size order as shown in FIG. When the transfer data is generated sequentially, the standard deviation of each transfer data is “215” as shown in FIG. On the other hand, when the transfer data is generated by the data synchronization method according to the embodiment, as shown in FIG. 12C, the standard deviation of each transfer data is “25”, and the variation in data size becomes small.

  As described above, in the distributed database system 1 according to the embodiment, data can be efficiently synchronized by generating transfer data by combining records of different sizes.

  As mentioned above, although this invention was demonstrated in detail using embodiment, this invention is not limited to embodiment described in this specification. The scope of the present invention is determined by the description of the claims and the scope equivalent to the description of the claims.

1 Distributed Database System 10a to 10d Database device (DB device)
101 rank analysis means 102 centroid calculation means 103 ratio calculation means 104 storage processing means 105 transfer means 110 storage device 111 data table storage unit 112 transfer buffer 20 user terminal

Claims (3)

A distributed database system in which a plurality of database devices are connected, data stored in any one of the database devices is transmitted to another database device, and the transmitted data is stored synchronously in the plurality of database devices,
Each of the database devices is
A storage unit for storing a plurality of data tables each having records of different sizes;
Rank analysis means for analyzing the rank of the record size of each data table stored in the storage unit;
A center-of-gravity calculation unit that calculates the amount of change in size in ascending order of the ranks analyzed by the rank analysis unit, and calculates the value of the center of gravity of the calculated amount of change;
The first difference value, which is the difference between the rank of the record size minimum value and the centroid value calculated by the centroid calculation means, and the second difference between the rank of the record size maximum value and the centroid value. A difference value is obtained, a ratio between the first difference value and the second difference value is calculated, a first record that includes the obtained ratio in each transfer data, and a second that is larger in size than the first record. A ratio calculation means for the ratio of the records of
Transfer means for transferring each data formed at the ratio calculated by the ratio calculation means to another database;
A distributed database system comprising: The database apparatus in a distributed database system in which a plurality of database apparatuses are connected, data stored in one of the database apparatuses is transmitted to another database apparatus, and the transmitted data is stored in synchronization with the plurality of database apparatuses Because
A storage unit for storing a plurality of data tables having records of different sizes;
Rank analysis means for analyzing the rank of the record size of each data table stored in the storage unit;
A center of gravity calculating means for calculating the amount of change in size in ascending order of the ranking, and calculating the value of the center of gravity of the determined amount of change,
The first difference value, which is the difference between the rank of the record size minimum value and the centroid value calculated by the centroid calculation means, and the second difference between the rank of the record size maximum value and the centroid value. A difference value is obtained, a ratio between the first difference value and the second difference value is calculated, a first record that includes the obtained ratio in each transfer data, and a second that is larger in size than the first record. A ratio calculation means for the ratio of the records of
Transfer means for transferring each data formed at the ratio calculated by the ratio calculation means to another database;
A database apparatus comprising: In a distributed database system to which a plurality of database devices are connected, a data synchronization method for transmitting data stored in one of the database devices to another database device and synchronizing the transmitted data with the plurality of database devices. And
Analyzing the order of the size of the records included in the plurality of data tables stored in the table storage unit and having records of different sizes;
Obtaining each size change in ascending order;
Calculating a value of the center of gravity of the obtained change amount;
Obtaining a first difference value that is a difference between the rank of the minimum value of the record size and the value of the centroid, and a second difference value that is a difference between the rank of the maximum value of the record size and the value of the centroid When,
Calculating a ratio between the first difference value and the second difference value;
Identifying the determined ratio as a ratio of a first record included in each transfer data and a second record having a size larger than the first record;
Transferring each data formed at a specified ratio to another database;
A data synchronization method comprising:

Images