JP2016045610A - Database device and operation method thereof - Google Patents

Abstract

PROBLEM TO BE SOLVED: To reduce the consumption of a memory and to move a data group between database devices without requiring eliminating memory fragmentation at the time of scaling out in a distributed database system.SOLUTION: A group of data records managed as a group is represented as a cabinet. The cabinet represents the records showing a storage position of real data records connected in a list manner. When data is added to the data group, the list of the cabinet is extended. When the data is moved between the data base devices, one data group is collectively moved, and the cabinet is moved from a memory for cabinet of the database device of the moving destination using vacant records corresponding to the number of data records included in the moving cabinet. Even when the vacant memory of the memory for cabinet of the moving destination is discontinuous, it is used to be connected as a list.SELECTED DRAWING: Figure 6

Description

  The present invention relates to a database apparatus and an operation method thereof.

  With the development of IT in recent years, the amount of data handled by information systems has increased, and there are cases where the amount of data increases as performance and storage capacity become insufficient in one database device. In view of this, a distributed database has been researched and developed in which data is distributedly accommodated in a plurality of database devices so that the plurality of database devices can be regarded as one database. Distributing data to a plurality of database devices is called sharding.

  In sharding, there are various methods for associating data with a database device that accommodates the data. One of them is key range division that determines the accommodating device according to the range of the key value of the data. The key is an attribute that uniquely identifies data. For example, in a database relating to telephone numbers, the telephone number is used as a key. In many cases, the key is an attribute often used when a person expresses data. Therefore, it can be said that the key range division is a sharding method that makes it easy for a system administrator to understand the correspondence between data and a database device.

  In order to perform data operations on a distributed database divided into key ranges, it is necessary to manage the correspondence between the key range and the database device. In this paper, identifying a database device that contains a key from a key is called distributed solution. In order to operate data, it is necessary to know the correspondence between the key range and the database device. Data representing the correspondence between key ranges and database devices is called distributed solution data in this paper. In this paper, each database device of a distributed database system holds (shares) the distributed solution data of the data of the entire distributed database system as a table in the database, and any database device accepts data manipulation requests from the outside, and data A database device that accepts an operation request performs a distributed solution by indexing the distributed solution data table, identifies a database device that accommodates data to be manipulated, and a database device that accepts a data operation request moves to a database device that accommodates data Assume a distributed database system that requires actual data manipulation.

  Normally, a distributed database system has a mechanism that allows a database device to be added to raise the limit of the amount of data accommodated (called scale-out). When key range partitioning is used for sharding, the key range assignment is changed between the existing database device and the new database device, and the existing data is transferred to the key range assigned to the new database device. In some cases, it is necessary to move data from an existing database device to a new database device.

  The usage of the database varies depending on the application, but it is not only used to acquire one data record corresponding to the key from one key value, but also to manage a group of multiple related data records. Widely done. For example, in a telephone line management system, lines stored in one storage device are managed as a group in a database.

Shunsuke Tsurumi, Kenji Abe, Shigeki Tokunaga, Junko Sasaki "Study on Data Management Method in Large Scale Distributed Database System" IEICE Society Conference B-6-80, 2013.

  Non-Patent Document 1 shows a method for realizing a distributed database that uses key range partitioning for sharding and does not cause fragmentation of memory space when data is moved due to a key range allocation change between database devices. Yes. In this method, an actual data record table that is a memory area for storing data records, an index that indicates the position of the record in the actual data record table, and a group of indexes that indicate actual data records that belong to a certain key range are stored. A data structure called “INDEX table” is used. A plurality of “INDEX tables” correspond to one key range, and data movement between the database devices in accordance with the key range assignment change is performed in units of “INDEX tables”. By keeping the size of the “INDEX table” (the number of indexes that can be stored in the “INDEX table”) constant, it is possible to avoid fragmentation of the memory area of the “INDEX table” even if the “INDEX table” is moved. It is claimed that it can.

  If there are multiple tables with the same key range, the number of “INDEX tables” assigned to one key range may be the same in those tables, or a different number of “INDEX tables” may be assigned to each table. Could also be assigned.

  In the method of Non-Patent Document 1, since the “INDEX table” has the same size, if the number of indexes in the “INDEX table” is less than the size of the “INDEX table”, the memory area of the “INDEX table” is wasted. There is a problem. When using the method of assigning the same number of “INDEX tables” to multiple tables assigned to one key range, if the number of data records contained in the key range varies between tables, the memory area of the “INDEX table” Waste is even more pronounced.

  The present invention has been made in view of the above problems, and an object of the present invention is to provide a technique for managing a plurality of related data record groups while reducing memory consumption in a database.

  In order to solve the above-described problem, the first aspect of the present invention is a database device, and corresponds to one or more actual data tables in which records including actual data are stored in association with actual data indexes, and to actual data. Key range information indicating a range including a plurality of keys to be performed, a distributed solution data table storing a record including a cabinet ID for each actual data table, and a cabinet indicating a storage location of the cabinet ID and one record included in the cabinet A cabinet management table in which records including indexes are stored; a cabinet table in which records including actual data indexes and next indexes are stored; means for concatenating records in the cabinet tables to form a list; and Cabinet on record list Means for inserting a record in the table and extending the list; means for receiving a data operation signal including a key; and searching for a record including key range information indicating a range including the key from the distributed solution data table; Means for reading a cabinet ID of an actual data table to store actual data; means for reading a cabinet index corresponding to the cabinet ID from the cabinet table; and retrieving a record corresponding to the cabinet index from the cabinet table; Means for inserting a cabinet table record into a list including the inserted record, extending the list, and storing an actual data index in the inserted cabinet table record, and storing the received actual data. In the data table Kicking characterized in that it comprises a means for adding a record containing the actual data to which the received at positions corresponding to the actual data index.

  The second aspect of the present invention is an operation method of a database device, wherein the database device corresponds to one or more actual data tables in which records including actual data are stored in association with actual data indexes, and to actual data. Key range information indicating a range including a plurality of keys, a distributed solution data table storing a record including a cabinet ID for each actual data table, and a cabinet index indicating a storage location of the cabinet ID and one record included in the cabinet A cabinet management table in which records including a record are stored; a cabinet table in which records including an actual data index and a next index are stored; means for concatenating records in the cabinet table to form a list; and records in the cabinet table In the list of Means for inserting a record of the net table and extending the list, wherein the database device receives a data operation signal including a key, and the database device receives the key from the distributed solution data table. The record including the key range information indicating the range including the data is retrieved, the cabinet ID of the actual data table in which the actual data is to be stored is read, and the database device reads the cabinet index corresponding to the cabinet ID from the cabinet table. The record corresponding to the cabinet index is searched from the cabinet table, and the database device extends the list by inserting the record of the cabinet table into the list including the searched record, and the inserted cabinet table. The actual data index is stored in the record, and the database device adds a record including the received actual data at a position corresponding to the actual data index in the actual data table in which the received actual data is to be stored. It is characterized by

  According to the database device and the operation method thereof of the present invention, when changing the allocation of the key range between the database devices in the distributed database system using the key range division, it is not necessary to perform the defragmentation of the memory, and A database with low memory consumption can be realized.

It is a figure which shows the structure of the memory | storage part in the database apparatus which concerns on this Embodiment. It is a figure which shows the example of arrangement | positioning of the cabinet 6 in memory space (cabinet table 4). It is a figure which shows the structure of the memory | storage part in the database apparatus of a comparative example. It is a figure which shows the problem regarding the memory consumption of the database apparatus of a comparative example. It is a figure which shows operation | movement of the database apparatus concerning this Embodiment. It is a figure which shows typically the reduction effect of the memory consumption by the cabinet 6 of this Embodiment.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 is a diagram showing a configuration of a storage unit in the database apparatus according to the present embodiment.

  The figure shows one database device of a distributed database system including a plurality of database devices. Other database devices have the same configuration.

  The database device includes one or more actual data tables 1 in which records including actual data are stored in association with actual data indexes, key range information indicating a range including a plurality of keys corresponding to actual data, arrangement destination information, and actual data. The distributed solution data table 2 in which records including cabinet IDs for each data table are stored, the cabinet ID, the cabinet size corresponding to the cabinet ID, and the storage location of one record included in the cabinet A cabinet management table 3 in which records including cabinet indexes are stored, and a cabinet table 4 in which records including cabinet indexes, actual data indexes, and next indexes are stored.

  Records of the cabinet table 4 can form a list. To configure the list, a value indicating another record (L2) is stored in the next index of the record (L1) of the cabinet table 4, and another record (L3) is stored in the next index of the record (L2). By repeatedly storing the indicated value, it is possible to form a list in which records L1, L2, and L3 are linked or more records are linked. The end of the list can be expressed by storing NULL in the next index.

  A combination of the records of the cabinet table 4 as a list is called a cabinet 6. The cabinet 6 has a data structure corresponding to the “INDEX table” described in Non-Patent Document 1, and groups a plurality of data records. When data is sharded by key range division, the data arrangement destination is designated for each cabinet 6 by the distributed solution data table 2.

  The actual data table 1 stores different types of data from the data stored in other actual data tables 1. That is, the same actual data table 1 does not exist in the entire distributed database system or one database device.

  The distributed solution data table 2 is the same as the distributed solution data table 2 stored in another database device.

  The cabinet management table 3 and the cabinet table 4 are different from those stored in other database devices.

  Since the memory of the database device is finite, the size of the cabinet table 4 is fixed and the number of records “N” cannot be increased or decreased.

  In the present embodiment, the use of a telephone number as a distributed solution key will be described as an example. That is, the key range information of the distributed solution data table 2 being “050-1111” represents a telephone number from “050-1111-0000” to “050-1111-9999” as a key value.

  FIG. 2 is a diagram illustrating an arrangement example of the cabinets 6 in the memory space (cabinet table 4).

  The next index included in the record of the cabinet table 4 indicates the cabinet index in the record generated next to the record.

  The record generated first in the cabinet 6 with the cabinet ID “1” is stored in, for example, the first area of the memory space (cabinet table 4). The second record is stored in the third area, for example. The next index of the first record indicates the third area. The third record is stored in, for example, the sixth area. The next index of the second record indicates the sixth area.

  The record generated first in the cabinet 6 with the cabinet ID “2” is stored in, for example, the second area of the memory space. The second record is stored in the fourth area. The next index of the first record indicates the fourth area. The third record is stored in the fifth area. The next index of the second record indicates the fifth area. The fourth record is stored in the seventh area. The next index of the third record indicates the seventh area.

  As described above, in this embodiment, the memory space in the cabinet table 4 can be used without a gap.

  FIG. 3 is a diagram illustrating a configuration of a storage unit in the database device of the comparative example.

  One or more actual data tables 1, a distributed solution data table 2A in which records including addresses, key range information, placement destination information, next addresses, and cabinet IDs are stored, and records including cabinet IDs and empty statuses are stored. And a cabinet table 4A on which the cabinet 6A and the cabinet 6A are arranged for each cabinet ID.

  The actual data table 1 stores different types of data from the data stored in other actual data tables 1.

  The distributed solution data table 2A is the same as the distributed solution data table 2A stored in another database device. The distributed solution data table 2A is created for each type of the actual data table 1.

  The cabinet vacancy management table 3A and the cabinet table 4A are different from those stored in other database devices.

  The cabinet table 4A cannot increase or decrease the number of records “N”.

  The number of records in the cabinet 6A is fixed at a fixed length so that it cannot be increased or decreased.

  FIG. 4 is a diagram illustrating a problem related to memory consumption of the database device of the comparative example.

  For example, in a state where all records of the cabinet 6A (cabinet ID “1”) for use with respect to a certain real data table 1 (TB # 1 real data table 1) have a real data index (full state), a new real data table 1 is used. If it becomes necessary to store the data index, a cabinet 6A (cabinet ID “2”) for use with the same actual data table 1 is newly established, and a new actual data index is stored in the record.

  In this case, in the distributed solution data table 2 for the actual data table 1 of TB # 1, the NULL value included as the next address in the record including the cabinet ID “1” (referred to as an existing record) is, for example, the next address “0002”. The record including the same address as the next address “0002”, the key range information “050-1111” of the existing record, the placement destination information “DB # 1”, the NULL value as the next address, and the cabinet ID “2” It is added to the distributed solution data table 2A.

  In the cabinet 6A (cabinet ID “2”), the actual data index is not stored in most records at the beginning of the establishment, and thus the cabinet 6A at the beginning of the establishment consumes excessive memory.

  FIG. 5 is a diagram showing the operation of the database apparatus according to the present embodiment.

  Here, the operation when the database device stores the key “050-1111-dddd” in the data operation signal transmitted from the outside and the actual data “xxx” having the data type “n” in the storage unit. explain.

  First, the database device receives a data operation signal including a key, a type of data, and actual data (S1).

  Next, the database device searches the distributed solution data table 2 for a record including key range information indicating the range including the key (S2). If the record location information indicates another database device, the data is stored. While the actual data operation is requested to the database device to be accommodated and the process is terminated, if the placement destination information indicates itself (database device), the actual data table (in which the actual data in the received data operation signal is to be stored) A record including the cabinet ID is retrieved from the cabinet management table 3 for TB # 1) (S3), and the cabinet size is updated (S4). Here, the database apparatus increments (updates) the cabinet size from 3 to 4, for example (S4).

  Next, the database device reads the cabinet index (“1”) from the updated record, searches for the record corresponding to the index from the cabinet table 4 (S5), and is finally added in the cabinet 6 including the record. The NULL value included in the next record as the next index is changed to a next index that is not a NULL value (for example, “6”), a record including the same cabinet index as the next index is retrieved from the cabinet 6, and an actual data index (for example, "105") is stored (S6).

  Next, the database device adds a record including the received actual data “xxx” to the position corresponding to the actual data index (“105”) in the actual data table (TB # 1) (S7).

  FIG. 6 is a diagram schematically showing a memory consumption reduction effect by the cabinet 6 of the present embodiment.

  In the present embodiment, since the cabinet 6 is configured by connecting the records of the cabinet table 4, as long as there are empty records in the cabinet table 4, the cabinet 6 does not become full. The number of records included in 6 can be increased. When increasing the number of records included in the cabinet 6 when adding actual data to the database, the cabinet 6 consumes excessive memory because it consumes as many records in the cabinet table 4 as the number of actual records to be added. Can solve the problem.

  As described in [Background Art], when the cabinet 6 is transferred to another database device, the cabinet 6 needs to be transferred to the cabinet table 4 of the database device.

  In the case of the comparative example, even if the area of each record in the cabinet table 4 of the database apparatus to which the data is to be transferred is unused, the cabinet 6 cannot be moved if there is no continuous free area for the fixed size of the cabinet 6. However, in this embodiment, the records in the cabinet table 4 consume only the number of actual data records to be actually stored, and the empty records in the cabinet table 4 do not necessarily have to be continuous. The fact that the empty records do not necessarily have to be continuous means that it is not necessary to eliminate the fragmentation of the memory when the cabinet 6 is transferred. As described above, since the unused memory of the cabinet table 4 can be used even if it is discontinuous, it can be said that this embodiment has high memory reusability.

  A computer program for causing a computer to function as the database device of the present embodiment can be recorded on a computer-readable recording medium such as a semiconductor memory, a magnetic disk, an optical disk, a magneto-optical disk, a magnetic tape, and the Internet. It is possible to distribute widely through the communication network.

1 real data table 2, 2A distributed solution data table 3 cabinet management table 3A cabinet empty / busy management table 4, 4A cabinet table 6, 6A cabinet

Claims (2)

A database device,
One or more actual data tables in which records including actual data are stored in association with actual data indexes;
A distributed solution data table in which a key range information indicating a range including a plurality of keys corresponding to actual data and a record including a cabinet ID for each actual data table is stored;
A cabinet management table in which a record including a cabinet ID and a cabinet index indicating a storage position of one record included in the cabinet is stored;
A cabinet table in which records including the actual data index and the next index are stored;
Means for connecting the records of the cabinet table to form a list;
Means for inserting a cabinet table record into the cabinet table record list and extending the list;
Means for receiving a data manipulation signal including a key;
Means for retrieving a record including key range information indicating a range including the key from the distributed solution data table, and reading a cabinet ID of the actual data table in which the actual data is to be stored;
Means for reading a cabinet index corresponding to the cabinet ID from the cabinet table and searching for a record corresponding to the cabinet index from the cabinet table;
Means for inserting a cabinet table record into the list including the retrieved record to extend the list, and storing an actual data index in the inserted cabinet table record;
And a means for adding a record including the received actual data to a position corresponding to the actual data index in an actual data table in which the received actual data is to be stored. A method of operating a database device, comprising:
The database device includes:
One or more actual data tables in which records including actual data are stored in association with actual data indexes;
A distributed solution data table in which a key range information indicating a range including a plurality of keys corresponding to actual data and a record including a cabinet ID for each actual data table is stored;
A cabinet management table in which a record including a cabinet ID and a cabinet index indicating a storage position of one record included in the cabinet is stored;
A cabinet table in which records including the actual data index and the next index are stored;
Means for connecting the records of the cabinet table to form a list;
Means for inserting the cabinet table record into the cabinet table record list and extending the list;
The operation method is as follows:
The database device receives a data operation signal including a key;
The database device retrieves a record including key range information indicating a range including the key from the distributed solution data table, reads a cabinet ID of an actual data table in which the actual data is to be stored,
The database device reads a cabinet index corresponding to the cabinet ID from the cabinet table, searches the cabinet table for a record corresponding to the cabinet index,
The database device extends the list by inserting a cabinet table record into the list including the retrieved record, stores an actual data index in the inserted cabinet table record,
The database apparatus adds a record including the received actual data to a position corresponding to the actual data index in an actual data table in which the received actual data is to be stored. .

Images