JP5732082B2 - Data management apparatus and data management program - Google Patents

Description

  The present invention relates to a data management apparatus and a data management program for managing data by distributing it to a plurality of storage devices using KVS (Key-Value Store).

  In a system such as a large-scale communication service, a requirement for data operation turnaround time (TAT) is very strict, and real-time performance is required for data update.

  In order to realize high-speed processing of large-scale data, a database is constructed by a distributed KVS (Key-Value Store) system. The distributed KVS is a pair of key and value (value), and is distributed and stored in a plurality of storages. The distributed KVS is described in Non-Patent Document 1, for example.

"Realization of large-scale distributed key-value store with range search by Range-Key Skip Graph", IPSJ SIG Notes 2010-DPS-144 (9), 1-8, September 10, 2010

  When a database is constructed with the KVS system, data is basically accessed using one key for the purpose of increasing the data search speed. However, for example, in a large-scale communication service, one subscriber data (service order data) is accessed with a different key, such as accessing a telephone number in some cases and accessing with a SIP-URI in some cases. There is a case to do.

  When accessing a single piece of data with a plurality of different keys, it is necessary to hold the data redundantly for each key or to construct a separate index system. When data is held redundantly for each key, there is a problem that a large amount of storage capacity is required, consistency is ensured at the time of data update, and turnaround time is deteriorated. Further, when an index system is separately constructed, the scale of the index system itself becomes large, and there is a need for distributed accommodation, resulting in an increase in maintenance management operations.

  The present invention has been made in view of the above circumstances, and an object of the present invention is to hold data redundantly for each key and access data with a plurality of different keys without constructing an index system. It is an object of the present invention to provide a possible data management apparatus and data management program.

In order to achieve the above object, the present invention is a data management device for managing data distributed to a plurality of storage devices, and for each key type, a storage device for a key range and a storage destination of keys in the range To the storage means for storing the storage location resolution information associated with the storage location and the storage device determined with reference to the storage location resolution information for the secondary key, to access the secondary key and the data associated with the secondary key. And a registration means for storing a primary key and data associated with the primary key in a storage device determined by referring to storage location solution information for the primary key. The secondary key and the primary key are distributed and stored in a plurality of the storage devices .

The present invention is a data management program for managing data distributed to a plurality of storage devices, wherein the computer associates a key range with a storage device that stores keys in the range for each key type. a storage unit for storing the storage location resolution information, the computer, in a storage device as determined with reference to the storage location solution information for the sub key, sub key, access to data associated with the sub-key Storing a pointer to store the primary key and the data associated with the primary key in the storage device determined by referring to the storage solution for primary key storage, The secondary key and the primary key are distributed and stored in the plurality of storage devices.

  According to the present invention, it is possible to provide a data management apparatus and a data management program capable of accessing data with a plurality of different keys without duplicating and holding data for each key and without constructing an index system. .

It is a block diagram of the data management system which concerns on embodiment of this invention. It is a block diagram which shows the function of DB server (# 1). It is explanatory drawing for demonstrating a registration process. It is explanatory drawing for demonstrating a search process.

  Embodiments of the present invention will be described below with reference to the drawings.

  FIG. 1 is an overall configuration diagram showing a configuration of a data management system 1 in the present embodiment. The data management system 1 of the present embodiment is a distributed DB system that uses a distributed KVS to distribute (store) and manage a large amount of data in a plurality of DB servers (storage devices).

  The data management system 1 shown in FIG. 1 includes a plurality of DB servers 11 and 12, distributes data using keys, and distributes the data to each DB server 11 and 12 to store the data. From outside the data management system 1, it is not known in which DB server 11, 12 the data is stored.

  The client 2 accesses the data management system 1 via the network, searches and updates the data stored in any of the DB servers 11 and 12 of the data management system 1 by specifying a key, and registers / registers data. Perform data operations such as addition.

  In the present embodiment, the DB server (# 1) 11 has a function as a receiver (control device) that receives a request from the client 2 and also has a function as a storage (storage device) that stores data. To do.

  FIG. 2 is a block diagram showing the configuration of the DB server (# 1) 11 (data management apparatus). The illustrated DB server (# 1) 11 includes a registration unit 111, a search unit 112, a storage location solution information storage unit 113, and a storage unit 114.

  The storage location solution information storage unit 113 stores storage location solution information in which the DB servers 11 and 12 that store the keys are set for each key type. In the present embodiment, each data (actual data) is associated with a plurality of types of keys so that one data can be accessed with a plurality of different keys.

  In the storage location solution information storage unit 113 shown in FIG. 1, it is assumed that two types of storage location solution information, type 1 and type 2, are stored. In the illustrated example, a telephone number band is used for the type 1 key, and a contract ID band is used for the type 2 key. Further, in the present embodiment, a type 1 key will be described as a primary key and a type 2 key will be described as a sub key. Note that the number of subkeys is not limited to one and may be plural. When a plurality of types of subkeys are provided, storage destination solution information is stored in the storage destination solution information storage unit 113 for each type of subkey.

  Further, as shown in FIG. 2, the storage location solution information storage unit 113 may be included in the DB server (# 1) 11 or may be included in another device accessible by the DB server (# 1) 11. It is good as well.

  The registration unit 111 refers to the storage location solution information of the secondary key (key type 2) in the storage location solution information storage unit 113, determines a DB server that stores the secondary key, and stores the secondary key in the determined DB server. A pointer for accessing data associated with the subkey is stored. In addition, the registration unit 111 refers to the storage location solution information of the primary key (key type 1) in the storage location solution information storage unit 113 to determine the DB server that stores the primary key, and the determined DB server Stores a key and data associated with the primary key.

  The search unit 112 accepts a search request in which any type of subkey is specified, and refers to the storage destination solution information of the subkey to access the DB servers 11 and 12 in which the subkey is stored, and A pointer corresponding to the sub key is acquired, and the DB servers 11 and 12 storing data associated with the sub key are accessed using the acquired pointer, and the data is read.

  The storage unit 114 stores data and keys distributed according to the storage location solution information. In this embodiment, data is stored together with the primary key in the primary key storage location, and a pointer to the data is stored in the secondary key storage location along with the secondary key.

  The DB servers (# 2) to (# 4) 12 other than the DB server (# 1) 11 include at least a storage unit similar to the storage unit 114 of the DB server (# 1) 11.

  As the DB server (# 1) 11 described above, for example, a general-purpose computer system including a CPU, a memory, an external storage device such as an HDD, an input device, and an output device can be used. In this computer system, each function of the DB server (# 1) 11 is realized when the CPU executes a program (data management program) for the DB server (# 1) 11 loaded on the memory. The program for the DB server (# 1) 11 can be stored in a computer-readable recording medium such as a hard disk, a flexible disk, a CD-ROM, an MO, a DVD-ROM, or can be distributed via a network. it can.

  Next, the processing of this embodiment will be described.

  FIG. 3 is an explanatory diagram for explaining the registration / update processing to the DB server.

  The client 2 transmits a data registration request to the data management system 1 (S11). This registration request includes a primary key, a secondary key, and data (actual data) that is actually stored.

  Upon receiving the registration request, the registration unit 111 of the DB server (# 1) determines a DB server that stores the subkey specified in the registration request with reference to the storage location solution information storage unit 113 (S12). . Specifically, the storage destination DB server (in the example shown, DB server (# 2)) corresponding to the secondary key (in the example shown, A0001zzz) is determined from the storage destination solution information 113A for that secondary key. To do. Then, the registration unit 111 accesses the determined DB server and stores a record including the subkey, the key type, and the pointer (S12).

  The key type indicates the type of whether the target key is a primary key or a secondary key. In S12, the secondary key is set as the key type. The pointer is a pointer for accessing data. In the present embodiment, it is assumed that a primary key (0502222xxxx in the illustrated example) designated by the registration request is used as the pointer. The pointer may be anything as long as it can access data, and in addition to the primary key, an address of the DB server, an ID unique to the entire data management system 1, or the like may be used.

  Then, the registration unit 111 determines a DB server that stores the data specified in the registration request in S11 together with the primary key with reference to the storage location solution information storage unit 113 (S13). Specifically, the storage destination DB server (in the example shown, DB server (# 3)) corresponding to the primary key (in the example shown, 0502222xxxx) is determined from the storage destination solution information 113B for the primary key. To do. Then, the registration unit 111 accesses the determined DB server, and stores the main key, the key type, and data (yyyy in the illustrated example) (S13). In S13, the primary key is set as the key type.

  Through the above registration processing, the pointers associated with the subkeys can be distributed and stored in a plurality of DB servers, and the data associated with the primary key can be distributed and stored in a plurality of DB servers.

  If the request received in S11 is a primary key and secondary key binding request (a request for associating a secondary key with a primary key), and the primary key and data are already registered in the DB server. The registration unit 111 performs only the process of S12. Similar to the registration request, the bind request also includes a primary key, a secondary key, and data.

  FIG. 4 is an explanatory diagram for explaining a data search process.

  The client 2 transmits a data search request to the data management system 1 (S21). Here, it is assumed that the search request specifies a subkey. The determination of the key type of whether the key specified in the search request is a secondary key or a primary key is, for example, that the key type is specified in the search request transmitted from the client 2 or the search request is received. The search unit 112 performs the determination by some method, such as analyzing the key numbering system / configuration.

  When receiving the search request, the search unit 112 of the DB server (# 1) acquires the DB server storing the subkey specified in the search request with reference to the storage location solution information storage unit 113 (S22). . Specifically, the DB server that stores the sub key corresponding to the sub key (A0001zzz in the illustrated example) specified from the storage destination solution information 113A for the sub key (in the illustrated example, the DB server (# 2)) is acquired. Then, the search unit 112 accesses the acquired DB server and checks whether or not the key type corresponding to the subkey is a subkey. If the key type is the subkey, a pointer (0502222xxxx in the illustrated example) Is read (S22).

  Then, the search unit 112 specifies a DB server in which data is stored using the read pointer (S22). In the present embodiment, since the primary key is used for the pointer, the storage destination solution information 113B for the primary key in the storage location solution information storage unit 113 is referred to, and the acquired primary key (0502222xxxx in the illustrated example) is used. A DB server (DB server (# 3) in the illustrated example) that stores the corresponding data is acquired.

  Then, the search unit 112 accesses the acquired DB server and checks whether or not the key type corresponding to the primary key is a primary key. If the key type is the primary key, the search unit 112 stores the key type in association with the primary key. Data (in the example shown, yyyy) is read out and transmitted to the search requesting client 2 in S21 (S23).

  When the search request received by the search unit 112 in S21 is a search request specifying a primary key, the storage destination solution information 113B for the primary key in the storage location solution information storage unit 113 is referred to and the acquired primary key is used. Acquire the DB server that stores the corresponding data, and access the acquired DB server. Then, the registration unit 112 checks whether or not the key type corresponding to the primary key is a primary key. If the key type is the primary key, the registration unit 112 reads out data stored in association with the primary key, Sent to the original client 2

  As described above, in the present embodiment, the individual data is distributed and accommodated by the primary key, and the secondary key can be distributed and accommodated, the primary key is stored together with the data (actual data), and the secondary key is Stores data with a pointer indicating the storage location.

  Thereby, in this embodiment, it becomes possible to access one data with a plurality of keys. For example, in a system in which a large-scale communication service is provided, when subscriber data is held in the data management system of the present embodiment, subscriber data can be efficiently searched for other than the primary key.

  Further, since the data itself is stored in one place, it is not necessary to hold the data in duplicate. As a result, an increase in storage capacity due to data duplication can be avoided, consistency at the time of data update can be prevented from being lost, and deterioration in turnaround time can be prevented. Further, only the keys are distributed and accommodated, and it is not necessary to construct a separate index system, and there is no delay in turnaround time associated with index reconstruction when data is updated.

  In addition, this invention is not limited to the said embodiment, Many deformation | transformation are possible within the range of the summary.

1: Data management system 11: DB server (# 1)
111: Registration unit 112: Search unit 113: Storage location information storage unit 114: Storage unit 12: DB servers (# 2) to (# 4)
2: Client

Claims (4)

A data management device for managing data by distributing it to a plurality of storage devices,
Storage means for storing storage location solution information in which a key range and a storage device for storing a key in the range are associated with each key type;
The storage device determined by referring to the storage location resolution information for the secondary key stores the secondary key and a pointer for accessing data associated with the secondary key, and resolves the storage location for the primary key. The storage device determined with reference to the information includes a primary key and registration means for storing data associated with the primary key ,
The data management device, wherein the secondary key and the primary key are distributed and stored in the plurality of storage devices. The data management device according to claim 1,
Retrieving a search request in which the subkey is specified, referring to the storage resolution information for the subkey, accessing the storage device in which the subkey is stored, obtaining a pointer corresponding to the subkey, and obtaining the pointer A data management device, further comprising: search means for accessing the storage device storing the data associated with the subkey using and reading the data. A data management program for managing data distributed to a plurality of storage devices,
The computer includes, for each key type, a storage unit that stores storage location solution information that associates a key range with a storage device that stores a key in the range,
In the computer,
A storage device which is determined with reference to the storage location solution information for the sub key, and storing the sub-key, and a pointer for accessing data associated with the sub-key,
Storing the primary key and data associated with the primary key in the storage device determined by referring to the storage solution for primary key storage ,
The secondary key and the primary key are distributed and stored in a plurality of the storage devices.
A data management program characterized by A data management program according to claim 3, wherein
Receiving a search request in which a subkey is specified, referring to storage solution for subkey storage location, accessing a storage device storing the subkey, and obtaining a pointer corresponding to the subkey;
Using the obtained pointer to access the storage device storing the data associated with the subkey and reading the data;
Data management program for further execution.

Images