WO2012053219A1 - 分散型データベースシステムおよび分散型データ処理システム - Google Patents

分散型データベースシステムおよび分散型データ処理システム Download PDF

Info

Publication number
WO2012053219A1
WO2012053219A1 PCT/JP2011/005892 JP2011005892W WO2012053219A1 WO 2012053219 A1 WO2012053219 A1 WO 2012053219A1 JP 2011005892 W JP2011005892 W JP 2011005892W WO 2012053219 A1 WO2012053219 A1 WO 2012053219A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
node
database
information
key information
Prior art date
Application number
PCT/JP2011/005892
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
郁夫 中川
Original Assignee
株式会社インテック
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 株式会社インテック filed Critical 株式会社インテック
Publication of WO2012053219A1 publication Critical patent/WO2012053219A1/ja

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24—Querying
    • G06F16/245—Query processing
    • G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
    • G06F16/2471—Distributed queries

Definitions

  • the present invention relates to a technique for processing data stored in database nodes constituting a distributed database system.
  • a distributed database system includes a plurality of database servers, and a plurality of data is distributed and stored in a plurality of database servers.
  • KVS Key Value Store
  • KVS Key Value Store
  • examples of techniques using KVS include ROMA, kumofs, and Flare.
  • KVS a key / value pair is stored in each database server.
  • the value is data such as a character string or a byte string, and the key is information for specifying value (data).
  • KVS has high scalability. Therefore, when operating a system using KVS, the system scales by increasing the number of servers later as the system grows (the amount of data handled by the system increases) while operating at low cost with a small number of servers. Operation that is out is possible. Therefore, KVS is often operated so that a large amount of data can be appropriately distributed and stored in a large number of database servers (increasing distribution efficiency).
  • the client uses the key as a key to identify the database server in which the data (value) to be processed is stored, and the identified database
  • the client sends a key to the server to request data
  • data stored in correspondence with the key is sent from the database server to the client, and the client executes processing on the acquired data.
  • the database server is specified again based on the key, and the client sends the processed data together with the key to the specified database server for storage.
  • the data stored in the database server as a pair with the key is replaced with the processed data. In this way, processing of data distributed and stored in a plurality of database servers is performed.
  • the data to be processed is sent from the database server to the client for each processing, and when the processing is completed, the processed data is transferred from the client to the database server. Need to be sent to again.
  • the data acquired by the client from the database server is the entire data (value) stored corresponding to the key entered in the acquisition request (cannot be partially acquired).
  • the amount of data communication per process increases. Considering such a problem of the amount of data communication, in the conventional system, the amount of data that can be stored corresponding to each key is naturally limited.
  • the present invention has been made in view of the above-described problems.
  • the amount of data that can be stored in a database node is not limited as in the conventional system (KVS), and is large corresponding to one key. It is possible to store a large amount of data, increase the possibility that all the data necessary for one process exists in one database node, reduce the number of communications, and highly confidential data
  • An object is to provide a distributed database system that is suitable for handling a database.
  • a distributed database system includes a plurality of database nodes and one or more client nodes, and a plurality of data is distributed and stored in the plurality of database nodes.
  • the client node wherein the client node transmits a pair of key information and data to a database node identified based on the key information among the plurality of database nodes, and stores the data in the database node;
  • the pair of key information and program information is identified based on the key information among the plurality of database nodes.
  • Means for transmitting to the database node The database node executes the program according to the program information transmitted from the client node with respect to the data stored in pairs with the key information transmitted from the client node, thereby Means for performing processing, and means for transmitting information on the result of the processing to the client node.
  • the present invention for example, it is possible to store a large amount of data corresponding to one key, and increase the possibility that all data necessary for one process exists in one database node. The number of communications can be reduced. It is also adaptable to handle highly confidential data.
  • the figure for demonstrating an example of the distributed data processing system in embodiment of this invention 20 is a diagram for explaining an application example (communication log processing) of the system of FIG.
  • data paired with the key information is transmitted and stored in the database node specified by the key information.
  • a database node that stores certain data can be specified by key information paired with the data.
  • the data to be processed is not transmitted from the database node to the client node when the data processing is performed.
  • what is sent from the database node to the client node is the resulting information, which is much less data than the data to be processed.
  • the amount (data communication amount per process) needs only a small amount of data such as program information and information obtained as a result of processing. Therefore, the amount of data that can be stored in the database node is not limited as in the conventional system (KVS).
  • KVS conventional system
  • the number of times that the database node communicates with the client node can be reduced (for example, once).
  • the data itself is not transmitted from the database node to the client node (only the result information is transmitted), which is suitable for handling highly confidential data.
  • a program executed in the database node is an object in object-oriented programming
  • program information transmitted from the client node includes information indicating a class to which the object belongs, the object Information on the values of variables used in the above.
  • the program executed in the database node is an object in object-oriented programming
  • the program information transmitted from the client node includes information indicating the class to which the object belongs and information on the values of variables used in the object. Therefore, the program (object) can be executed at the database node. In this way, data processing (program execution) in the database node can be realized by using object-oriented programming.
  • the database node further includes means for storing information on a class library including the contents of methods executed in each class and definitions of variables to be used. It may be performed by executing the program using information on the class library.
  • the information about the class library stored in the database node includes, for example, the contents of the method executed in the class and the definition of the variable used for each class. (Including information on the value of the variable) is transmitted, it is possible to execute the program (object) at the database node using the information on the class library. In this way, data processing (program execution) in the database node can be realized by using object-oriented programming.
  • class information (original or a copy thereof) may be registered in each node that requires the class information.
  • a shared storage or a shared web server may be used.
  • a user of this system for example, a person who provides a service for processing data stored in a database node.
  • class information is registered in the shared file server (or shared web server), and for each database node, the storage location (or URL) is instructed.
  • the class information may be acquired from the shared file server (or shared web server) indicated by the stored location (or URL).
  • the data transmitted in pairs with the key information from the client node and stored in the database node is represented by an object in object-oriented programming and can have an arbitrary structure It may be.
  • the data stored in the database node is an object in object-oriented programming
  • data processing in the database node can be realized by using object-oriented programming.
  • the variable value information included in the program information transmitted from the client node is the variable has an arbitrary structure.
  • the variables used in the object executed according to the program information can be specified along the described structure. For example, if the data structure is a plurality of data object keys and values corresponding to each key, the program information transmitted from the client node may cause a key (in the data object) to exist in the data structure to be processed. It is possible to selectively perform processing on the value stored corresponding to the key.
  • the data structure here can be handled not only by the client node but also by the database node.
  • executing a program on the database node not only character strings and numerical values, but also a structure and size exceeding a certain level. It is possible to perform processing on data having
  • the program information transmitted from the client node includes an instruction for the process to be added and the new data.
  • the database node reads the data stored in pairs with the key information transmitted from the client node, and adds the new data transmitted from the client node to the data to obtain the data
  • a process of storing a pair with key information may be performed in the database node, and a notification indicating the success of the additional writing may be transmitted to the client node as information on the result of the process.
  • the program information including the instruction of the processing to be added and the data “xyz” from the client node Is transmitted together with the key information of the data “abc”.
  • the database node reads “abc” data based on the key information and adds “xyz” data thereto.
  • the new data “abcxyz” thus obtained is stored in the database node together with the key information.
  • a notification (a notification indicating the success of additional writing) is transmitted from the database node to the client node. In this way, the process of adding new data to the data stored in the database node can be realized in the database node.
  • the program information transmitted from the client node includes a rewrite process instruction and a rewrite process.
  • the database node reads the data stored in a pair with the key information transmitted from the client node, and includes an indication of data to be performed and new data, and is instructed by the client node in the data If there is data to be rewritten, the data obtained by replacing it with new data is stored in the database node as a pair, and the notification indicating the successful rewriting is performed in the database node.
  • the client It may be the one to be sent to the over de.
  • the program information transmitted from the client node includes an instruction for the process to be deleted and the data to be deleted.
  • the database node reads data stored in pairs with the key information transmitted from the client node, and if there is data to be deleted instructed from the client node in the data, Processing for storing the data obtained by cutting in pairs with the key information is performed in the database node, and a notification indicating the success of the deletion is transmitted to the client node as information on the result of the processing. It may be.
  • the process for the data is a process for comparing a data element included in the data with another data
  • the program information transmitted from the client node compares which data element.
  • the database node reads the data stored in pairs with the key information transmitted from the client node, and includes an instruction as to whether the processing is performed and the other data to be compared.
  • a process for comparing the data element instructed from the client node with the other data transmitted from the client node is performed in the database node, and the result obtained by the comparison is obtained as a result of the process.
  • Information may be transmitted to the client node.
  • “password” is stored as a data element included in data (data paired with key information), and compared with “password” entered by the user of the client node,
  • a program including an instruction from the client node that the process is to compare the data elements of “password” and a “password” to be compared (“password” entered by the user of the client node).
  • Information is transmitted along with key information.
  • the database node reads data based on the key information, and compares the “password” included in the data with the “password” to be compared.
  • the result (authentication OK or authentication NG) obtained by the comparison is transmitted from the database node to the client node. In this way, a process of comparing a data element included in data stored in a database node with another data can be realized in the database node.
  • the program information transmitted from the client node includes which data element
  • the database node reads the data stored in a pair with the key information transmitted from the client node, and indicates from the client node in the data
  • a process for performing the calculation instructed by the client node using the instructed data element is performed in the database node, and the result obtained by the calculation is used as information on the result of the process, and the client node It may be transmitted to.
  • some database nodes store several “scores of past tests” as one or more data elements included in the data (data paired with key information), and calculations using those data elements are performed. Assuming that “calculation of the average score of past tests” is performed. In this case, program information including an instruction to perform “calculation of the average score of past tests” from the client node using the data element “score of past tests” is transmitted together with the key information. Is done. The database node reads data based on the key information, and performs “calculation of past test average score” using “past test score” in the data. When the calculation process is completed, the result (calculated average score) obtained by the calculation is transmitted from the database node to the client node. In this manner, a process of performing calculation using one or more data elements included in data stored in the database node can be realized in the database.
  • the client node prepares a plurality of pairs of key information and program information in order to perform one large process for a plurality of data, and each program information includes the one large database.
  • each piece of program information including a processing instruction “calculation of the average number of user mails (specified by the key information)” from the client node as a pre-process of the large processing, together with each key information.
  • Each is sent to the database node.
  • each database node the calculation of the average number of emails is performed for each user stored in the database node, and the information (average of the number of emails of each user) resulting from the processing is the client node. Sent to. In the client node, the average of the total number of mails of all users can be calculated using the information received from each database node.
  • the processing for the data is processing in which the client node acquires a data element included in the data
  • the program information transmitted from the client node acquires which data element.
  • the database node reads the data stored in pairs with the key information transmitted from the client node, and extracts the data element indicated by the client node in the data
  • the processing may be performed in the database node, and the extracted data element may be transmitted to the client node as information on the result of the processing.
  • the test score for the past 10 years is stored as data (data paired with the key information), and the data elements included in the data are “tests for the last year”
  • program information including an instruction indicating that it is a process of acquiring a data element “test score for the last year” is transmitted from the client node together with the key information.
  • the data of “the test score for the past 10 years” is read, and the data element of “the test score for the most recent year” in the data is extracted. Then, the extracted data element (the test score for the last year) is transmitted to the client node. In this way, it is possible to realize a process in which a client node acquires a data element (a part of data) included in data stored in a database node.
  • the process on the data is a process in which the client node acquires the data
  • the program information transmitted from the client node includes an instruction for the process to acquire
  • the database The node performs a process of reading data stored in pairs with the key information transmitted from the client node, and transmits the read data to the client node as information on a result of the process. There may be.
  • program information including a processing instruction for acquiring data is transmitted from the client node together with key information.
  • the database node reads data based on the key information and transmits it to the client node. In this way, it is possible to realize a process in which the client node acquires data stored in the database node.
  • the program information transmitted from the client node includes an instruction to store and data to be stored
  • the database node is the client
  • a process for storing data included in the program information transmitted from the client node is performed, and notification indicating the success of the store is performed as information on the result of the process.
  • the client node It may be the one to trust.
  • a database node that stores a pair of key information and data transmitted from the client node has the key information and data stored in a database node that can be an alternative of the plurality of database nodes. There may be further provided means for storing a copy of the pair.
  • a copy of the key information / data pair is transferred to another database node (a database node that can be used as an alternative). Is also stored.
  • a failure occurs in a certain database node, the system can be easily restored by changing the responsible range of the database node to use a replica stored in another database node.
  • a key information and data pair is stored in a certain database node A, and a copy of the key information and data pair is stored in a database node B which can be an alternative.
  • the responsible range of the database node A is included in the responsible range of the database node B.
  • the database node B takes the place of the database node A. In this way, the system can be recovered by changing the scope of the database node.
  • the system can be easily recovered by replacing (adding / deleting) the database node using a replica stored in another database node.
  • a key information and data pair is stored in a certain database node A, and a copy of the key information and data pair is stored in a database node B which can be an alternative.
  • a new database node X is added. A copy of the key information / data pair is copied from the database node B and stored in the database node X.
  • the database node X is specified by the same key information as the database node A. In this way, the new database node X may be replaced with the old database node A to restore the system.
  • processing for the data involves updating the data stored in pairs with the key information or only reading the data, and only reading the data.
  • means for the client node to send the key information and program information pair to the database node specified based on the key information among the plurality of database nodes is indicated by the key information.
  • Processing that only reads data can be executed by a database node selected from the database node specified by the key information and a database that can be used as an alternative. That is, it is possible to distribute read-only processing.
  • a key information and data pair is stored in a certain database node A and a copy of the key information and data pair is stored in a database node B which can be used as an alternative, it is selected from the database nodes A and B.
  • Data reading processing can be executed from the database node (one or both of database nodes A and B). Therefore, data can be read from an appropriate database node according to the situation. For example, if the load on the database node A is high when attempting to execute a process for reading data, the data can be read from the database node B instead. As another example, when the load on the database nodes A and B is low, it is possible to speed up the process of reading data by sending a data read request to both and using the one with the faster response.
  • each of the plurality of database nodes has a range of values for which the database node is responsible, and a database node is identified based on the key information using a hash value calculated from the key information. Then, it may be performed by determining which database node is in the range of values for which it is responsible.
  • the identification of the database node based on the key information is performed using a hash value calculated from the key information.
  • the database node may be specified using a part of the hash value calculated from the key information (for example, the first two characters of the hash value). For example, assume that the hash value calculated from the key information is “39021650ae9e43ecb53f66bf865d9730135f43b1”. The first two characters of this hash value are “39”.
  • a range of data to be stored (range of charge) is set in each database node. For example, assume that the assigned range of node A is set to “08 to 34”, the assigned range of node B is set to “35 to 48”,. In such a case, the node B can be identified from a part “39” of the hash value of the key information.
  • the data is stored in the database node as an object with version information, and the processing for the data updates the data stored in pairs with the key information. If it is accompanied, the updated data may be stored as an object in which the version information is updated.
  • a key information / data pair is stored in two data database nodes (DB node A and DB node B), and one database node (DB node) is stored.
  • A) goes down temporarily due to a failure, and then the database node (DB node A) recovers and can provide services.
  • DB node A the database node
  • the two database nodes (DB node A and DB node B) The version number for the corresponding key information is compared between them, and the data can be transferred (copied) only when the update is necessary. In this way, it is possible to improve the efficiency of data exchange between database nodes.
  • the process performed according to the program information received from a certain client node is a process involving updating of the data
  • another process is performed while the process is being performed.
  • the program information for instructing the update of the data is received from the client node
  • the processing is notified to the other client node, or the update received from the other client node is instructed after completion of the processing.
  • a means for starting the processing of the program may be further provided.
  • a database node when a database node is executing data update in accordance with program information received from a client node A and receives program information instructing data update from another client node B, the The client node B is notified of the inability to process or waits for the end of the data update process from the client node A, and then starts the data update process from the client node B. In this way, it is possible to control data update conflicts by a plurality of client nodes.
  • the client node may further include means for creating the program information.
  • the client node user can create an original program by himself and process data stored in the database node using the program. Therefore, various data processing can be executed.
  • a program executed in the database node is an object in object-oriented programming, and the client node executes the program according to program information transmitted from the client node.
  • the information processing apparatus may further include means for creating information on a class used to do this.
  • the client node user can create an original class by himself and use the object defined in that class to Stored data can be processed. Therefore, various data processing can be executed.
  • the program information transmitted from the client node is information indicating a method to be executed by the object. And the information about the variables used in the object, and the client node creates the program information by referring to the information of the class that defines the called method in response to the method call. Also good.
  • the user of the client node can call the method as if the process is executed in the client node at hand, even though the process is executed in the database node existing on the other side of the network. It is possible to develop a program that performs
  • the client node includes key information in a proxy object (a pseudo object that can be used in the Proxy technology in the Java (registered trademark) language) executed when the method is called. By doing so, a pair of the key information and the program information may be created.
  • a proxy object a pseudo object that can be used in the Proxy technology in the Java (registered trademark) language
  • the proxy object can generate information indicating the method to be executed in the database node based on the method call in the client node, and to send this to the database node as a pair with the key information.
  • the client node user can develop a program without being aware that the destination of the pair of key information and program information is a database node existing on the other side of the network.
  • a program executed in the database node is an object in object-oriented programming, and the database node is configured to execute the program according to program information transmitted from the client node.
  • the information processing apparatus may further include a storage or a server that stores information on a class used for executing the information, and each of the plurality of database nodes may include a unit that acquires the information on the class from the storage or the server.
  • the class information is stored in the storage or server, and each database node receives the class information from the storage or server. It can be acquired and used. In this way, class information can be shared by a plurality of database nodes.
  • a pair of key information and program information transmitted from the client node is accompanied by process identification information that uniquely indicates a process to be performed in the database node.
  • the database node receives the previous request. It is possible to distinguish between whether the processing has not been performed and whether the communication from the database node to the client node after the processing has failed or not, and it is possible to handle a retry without performing duplicate processing.
  • a pair of key information and program information transmitted from the client node is accompanied by user information for specifying a user related to processing to be performed in the database node. You may make it provide the means to notify with respect to the process with respect to the identified user.
  • a database node or a server that manages a plurality of database nodes it is possible to proceed to the requested process after authenticating the user or to know which user has requested what process. Services such as billing, log management, and reporting can be provided.
  • the invention of the distributed database system described above may be an invention of a method of the entire system, an invention of a program (or a recording medium on which the program is recorded) for operating a general-purpose computer system as this system, or a client node device.
  • an invention of a database node device an invention of a program (or a recording medium on which the program is recorded) for operating a general-purpose computer as a client node or a database node of this system,
  • the invention of the method to be executed and the invention of the method to be executed in the database node are valid.
  • the principle of the present invention can be applied to a configuration in which data paired with key information that it is responsible for is distributed and stored in each of a plurality of database nodes, as in the above-described distributed database system.
  • the present invention can be applied to a configuration in which a plurality of data processing nodes having a shared storage in the back perform distributed writing and reading of data stored in the shared storage.
  • Such a distributed data processing system is distributed to a shared storage having a function of storing data corresponding to an index (information for uniquely specifying data).
  • a distributed data processing system that performs processing, the distributed data processing system comprising a plurality of data processing nodes and one or more client nodes, the client nodes based on an index for storing data Means for generating key information, and means for transmitting the key information and the data pair to a data processing node specified based on the key information among the plurality of data processing nodes,
  • the processing node requests the shared storage based on the key information transmitted from the client node. In association with identification information of the index which is provided with means to write the data sent from the client node.
  • the client node In the distributed data processing system, the client node generates a key information based on an index for searching for data stored in the shared storage, a pair of the key information and a search request, Means for transmitting to a data processing node identified based on the key information among a plurality of data processing nodes, wherein the data processing node is an index determined based on the key information transmitted from the client node Data stored corresponding to the identification information may be further read from the shared storage and transmitted to the client node.
  • a search request transmitted from the client node is set as program information including a search condition, and the data processing node searches for data to be read from the shared storage by executing a program according to the program information. You may make it perform the process to perform.
  • log reception and transmission (distribution) at each client node, index processing at each data processing node, and shared storage can be accessed through a pipeline, and by increasing the number of client nodes and data processing nodes, it is possible to realize scale-out that improves performance. Furthermore, it is possible to perform a search from such log records according to various search conditions.
  • the invention of the distributed data processing system described above may be an invention of a method of the entire system, an invention of a program for operating a general-purpose computer system as this system (or a recording medium on which the program is recorded), a client node
  • both the invention of the method executed in the node and the invention of the method executed in the data processing node are established.
  • each unit in each system described above can be configured as a unit.
  • FIG. 1 is an explanatory diagram showing the configuration of the present system.
  • the system 1 includes a plurality of database nodes 2 in which a plurality of data is distributed and stored, and a client node 3 used by a user of the system 1.
  • the service provided by the system 1 can be used from the user terminal 4 used by a plurality of end users.
  • the number of database nodes 2, client nodes 3, and user terminals 4 is illustrated as a small number for convenience of explanation, but the number of database nodes 2, client nodes 3, and user terminals 4 is limited to this. Of course not.
  • FIG. 2 is a block diagram showing the configuration of the client node 3 of this system.
  • the client node 3 includes a data transmission / reception unit 5, a data processing unit 6, and a program creation unit 7.
  • the data transmitter / receiver 5 has a function of performing data communication with the database node 2.
  • the data processing unit 6 has a function of processing data communicated with the database node 2.
  • the program creation unit 7 has a function of creating various programs used in the system 1.
  • FIG. 3 is a block diagram showing the configuration of the database node 2 of this system.
  • the database node includes a data transmission / reception unit 8, a data processing unit 9, a data storage unit 10, a class library storage unit 11, a conflict control unit 12, and a version management unit 13. .
  • the data transmission / reception unit 8 has a function of performing data communication with the client node 3.
  • the data processing unit 9 has a function of processing data communicated with the client node 3.
  • the data transmitter / receiver 8 also has a function of performing data communication with another database node 2, and the data processor 9 also has a function of processing data communicated with the other database node 2. I have.
  • the data received from the client node 3 is stored in the data storage unit 10.
  • the class library storage unit 11 stores information on a class of a program (a class to which an object in object-oriented programming belongs) used in the present system. For example, the contents of methods executed in each class, the definitions of variables used in each class, and the like are stored.
  • the conflict control unit 12 has a function of controlling a conflict when updating data stored in the data storage unit.
  • the version management unit 13 has a function of managing the version of data stored in the data storage unit.
  • Conflict control is executed when multiple client nodes request data update processing for the same key at the same time. For example, if a data update request is received from another client node B while a data update request is being processed from a client node A, the other client node B is incapable of processing. The data update process from the client node B is started after the notification or the data update process from the client node A is completed.
  • FIG. 4 is an explanatory diagram showing an overview of the present system.
  • DB nodes A to H database nodes
  • each database node is assigned an ID (for example, a hexadecimal number).
  • ID for example, a hexadecimal number
  • These database nodes are virtually arranged in a “ring shape” in the order of assigned ID numbers. That is, when the first database node (for example, DB node A) makes a round in order of ID numbers, the next database node (DB node H) is followed by the first database node (DB node A) again. Yes.
  • the “direction” is determined by the size of the assigned ID, and the database node (DB node B) having the next higher ID number than a certain database node (for example, DB node A) has its database node (DB) Called “successor” of node A). Further, the database node (DB node A) having the next lower ID number than a certain database node (for example, DB node B) is called a “predecessor” of the database node (DB node B).
  • the same technology as DHT (Distributed Hash Table) and Consistent Hash is used for the arrangement of database nodes, range allocation, backup, and other mechanisms.
  • the responsible range of a certain database node for example, DB node B
  • the ID of the database node ID of DB node B: 35
  • DB node C ID: 49 the ID of the next database node
  • the key-data pair sent from the client node is stored in the database node whose first two characters when the hash value of the key is calculated are included in the assigned range.
  • the hash value calculated from the key information is “3902650ae9e43ecb53f66bf865d9730135f43b1”
  • the first two characters of this hash value are “39” and are stored in the DB node B.
  • the number of data “replications” can be specified by setting the multiplicity of data, and it is possible to cope with the occurrence of failures in a plurality of database nodes at the same time. More specifically, in this system, in addition to the database node (DB node B) having information to be handled by a database node (for example, DB node B) having a certain ID, a copy (duplicate) of the information ) Is possessed by a database node (DB node A) which is a predecessor of the database node. Thus, it can be said that the multiplicity is set to “2” when the two database nodes have information. Multiplicity is common to all data.
  • the number of database nodes having copies should be increased in the direction of the predecessor.
  • data is copied between database nodes. That is, when data is stored, updated, or deleted in a certain database node (for example, DB node B), the information is also transmitted to the database node (DB node A) that is the predecessor of the database node. Similarly, data is stored, updated, and deleted at the database node (DB node A).
  • DB node A database node A
  • the “replication” possessed by the database node (DB node A) serving as a predecessor is data outside the scope of the database node (DB node A).
  • Program information (job) corresponding to the key reaches only the original database node (DB node B), and the update of the data of the database node (DB node A) is transmitted from the original database node (DB node B). This is done only with information that is available.
  • This “replication” can be used for parallel data reading. Therefore, in this system, the program information (job) is flagged so that it can be distinguished between “READ (read only)” and “EDIT (with update)”. As for the processing (read-only processing) with the flag “READ”, the processing can be distributed in the predecessor direction up to the number of multiplicity. Note that the writing process is performed only at the database node that is responsible for the original assigned range in order to maintain “consistency”.
  • DB node I a database node having a new ID
  • DB node H a database node having a new ID
  • DB node I a new database node
  • the database node to be deleted (for example, DB node H) is deleted. Thereafter, the program information (job) for the data in the range handled by the database node (DB node H) reaches the database node (DB node G) that is the predecessor of the database node. Since this database node (DB node G) has a copy of the data in the range handled by the database node to be deleted (DB node H), the provision of the service is continued. In this case, the downtime can be minimized by deleting the database node on the assumption that the database node serving as a predecessor has a copy of data.
  • this system has a scale and expandability equivalent to or better than KVS, and also supports a “distributed programming” interface.
  • this system not only data is sent to the database node and data is written, but also program information is sent to the database node and “processing” is performed on the data.
  • the data is stored in the database node as an “object”, and the object on the database node can be expressed as “Java (registered trademark) Object” (an arbitrary object that can be handled by Java (registered trademark)).
  • class information that is the definition of the program can be shared between the client node and the database node so that the program specified by the user of this system can be executed on the database node.
  • the first pattern is a pattern in which class information (original or a copy thereof) is registered in each node that requires the class information.
  • the second pattern uses a shared storage.
  • the third pattern uses a shared web server.
  • the sharing of class information will be further described.
  • the user registers class information in the shared file server, tells each database node the storage location, and starts the service. Then, each database node acquires class information from the shared file server.
  • the user registers class information in the shared web server, teaches the URL to each database node, and then starts the service. Then, each database node downloads class information from the URL (shared web server).
  • the processing content (job) is defined on the client node side
  • the program (job) is sent to the database node
  • the processing is distributed and executed on the database node side. Only get the result.
  • KVS conventional system
  • this system requires fewer communications with the database node than the conventional system (KVS), the amount of data sent from the database node is small, and the data does not leave the database node. It is suitable for handling highly confidential data.
  • FIG. 7 conceptually shows the difference between this system and the conventional system (KVS).
  • processing executed by the node or server is indicated by a “substantially square” graphic, and data to be processed is indicated by a “circular” graphic.
  • the processing itself is executed on the client side. Therefore, in the conventional system, the processing capability of the client is limited, and it is difficult to execute large-scale processing.
  • the processing itself can be distributed to database nodes (theoretically, it can be distributed by the number of database nodes). Therefore, in this system, large-scale processing can be executed without being limited by the processing capability of the client node (using the processing capability of a plurality of database nodes).
  • This system is superior to the conventional system (KVS) in that the processing can be easily implemented in the database node and the processing can be distributed. Therefore, this system is suitable for performing large-scale data processing with a service that handles a large amount of data (for example, tens of millions to tens of billions).
  • a service for example, tens of millions to tens of billions.
  • 100 million users' emails are 10,000 database servers. Are distributed and stored. In this case, data and its processing are moderately performed so that 10,000 users' mail (tens of thousands of mail per user) is stored and processed per database server. Will be distributed.
  • this system has high scalability (scalability), it is suitable for services with a high data increase rate.
  • FIG. 8 shows an application example to a system that handles personal information.
  • an identification number for example, 012345
  • key information key
  • various data password “hoge”, age “25”, name “Yamada” are paired with the key information.
  • this system can perform a “password check” process. In this case, when “user name and password” is input, “success / failure of authentication” is obtained as an output.
  • the job definition of this system will be described in detail by taking “Process 1” of “This system” in FIG. 8 as an example.
  • the first line shows the definition of a class for describing the contents of the program.
  • a class “Authenticate” is defined.
  • the second line shows the definition of a variable used in this program.
  • a String type variable “pass” is defined.
  • the third to fifth lines define a method (method) for generating a program as an object. In this example, a character string is used as an argument, and the internal variable “pass” is instructed to hold the argument. is doing. This method is called “constructor” in Java (registered trademark) programming.
  • the sixth to ninth lines define a method to be executed in the database node.
  • a method “Boolean applyTo” is defined.
  • Object o means that data (object) stored on the database node is received as an argument.
  • the seventh line describes the contents of the process to be performed on the object (on the database node) received as an argument.
  • the process of “compare the input password with the stored password” is performed. It is shown.
  • Map one of classes defined by Java (registered trademark) program
  • the data (object) held in the database node is handled as belonging to the class “Map”, and the password information of the user is set as a value corresponding to the key “password” in the data. Is stored.
  • the value corresponding to the key “password” in the object stored in the database node and the internal variable “ “pass” is compared to determine whether or not they match, and a process of returning a true / false value of “Boolean” (a value indicating whether or not they match) is performed.
  • the present system can perform the process of “Judgment as to whether it corresponds to a predetermined address and age”.
  • “identification number, address, and age range” is input, “whether or not is applicable” is obtained as an output.
  • a class “GetCampain” is used, and a String type method “applyTo” is defined. Then, the value corresponding to the key “address” of the object stored in the database node starts with “Tokyo”, and the value corresponding to the key “age (age)” is 20 If it is above, the process of returning the character string “let's go to the election” is performed.
  • FIG. 9 shows an application example to a system that handles purchase history of an online shop.
  • an identification number for example, 012345
  • key information key
  • various data password “hoge”, name “Taro Yamada”
  • this system can perform the process of “Judgment whether alcohol has been bought in the past”.
  • the “user ID” is input, and if there is a purchase history of liquor in the past, the information of “alcohol campaign” is returned as an output.
  • a class “GetCampain” is used, and a String type method “applyTo” is defined.
  • the information related to the purchase history is stored in an array stored in the key “history” of the object stored in the database node.
  • the past purchase history is searched from the array, and “beer” is stored in the purchase history.
  • the data stored in the database node can have data with a structure corresponding to the process as an object (in this example, not only character strings and numerical values but also arrays are stored in the map) ing).
  • FIG. 10 shows an application example to the commodity inventory management system.
  • a product ID for example, 012345
  • key information key
  • various data product name “chocolate”, price “150”, 2010 4) are paired with the key information.
  • the monthly sales number “1200”, the sales number “1500” in May 2010, etc.) are stored.
  • CellInfo a unique class called “SellInfo” is defined to represent the data to be stored.
  • the data stored in the database node can have an object of any unique class that can be defined by object-oriented programming.
  • the present system can perform a process for obtaining the “current stock quantity”.
  • “product ID” is input
  • “stock quantity” is obtained as an output.
  • a class “GetRemains” is used, and an Integer type method “applyTo” is defined.
  • the object stored in the database node is regarded as an object of the unique class “SellInfo”, and processing for returning the value stored in the variable “remains (stock quantity)” of the object is performed.
  • FIG. 11 shows an application example to a health management system.
  • a personal ID for example, 012345
  • key information key
  • various data as described data (acquired calories “1600” on September 1, 2010, Calories “1750” acquired on September 2, 2010, weight “65 kg”, etc.) are stored.
  • the present system can perform a process of “obtaining current weight”.
  • “current weight” is obtained as an output.
  • a class “GetWeight” is used, and a double type method “applyTo” is defined. Then, a process of returning a value corresponding to the key “weight” of the object stored in the database node is performed.
  • this system can perform the process of “Calculation of daily average calorie acquisition”.
  • “user ID” when “user ID” is input, “average value of acquired calories” is obtained as an output.
  • a class “GetAverage” is used, and a Double type method “applyTo” is defined.
  • the keys stored in the database node are “start (start date; in this example, September 1, 2010) calories acquired”, “2010 calories acquired on September 2, 2010”, and so on. The corresponding values are summed and the average value is returned.
  • FIG. 12 and FIG. 13 show an application example to the mail service of a communication carrier.
  • FIG. 12 shows an example in which processing can be performed efficiently in this system, whereas the data is too large in the conventional system (KVS).
  • KVS conventional system
  • a personal ID eg, 03a9e0
  • various data maximum number of stored mails “950”, first mail is paired with the key information. , The second mail text, etc.
  • this system can perform a process of “obtaining the latest mail”.
  • “the text of the latest mail” is obtained as an output.
  • a class called “GetNewMail” is used and a String type method called “applyTo” is defined. Then, a process of returning the mail text corresponding to the key “maxId (latest mail)” of the object stored in the database node is performed.
  • this system can perform a process of “determining the number of stored emails”.
  • “number of undeleted mails” is obtained as an output.
  • a class “GetCount” is used, and an Integer type method “applyTo” is defined.
  • an Integer type method “applyTo” is defined.
  • FIG. 13 is an example of processing that spans multiple keys.
  • this system uses a personal ID (for example, 03a9e0) as key information (key), and various data (maximum number of stored mails “950”, text of the first mail in pairs with the key information) , The text of the second mail, etc.) are stored.
  • a personal ID for example, 03a9e0
  • various data maximum number of stored mails “950”, text of the first mail in pairs with the key information
  • the present system performs a process of “calculating the average value of the number of retained mails” even when the system spans multiple “user IDs”. Can do.
  • a set of “user IDs” is input, “average value of the number of retained mails” is obtained as an output.
  • a class “GetCount” is used, and an Integer type method “applyTo” is defined.
  • this system including a framework for enabling an application provider who uses the distributed database system according to the present embodiment to perform program development and operation (operation management) more easily will be described. This will be described with reference to FIGS.
  • an Android terminal or a user terminal such as a computer on which Java (registered trademark) (including Applet, OSGI, and other arbitrary Java (registered trademark) virtual machines) is operated can be connected to the cloud (this system). It is possible to easily provide a service that makes it possible to easily access the above object and handle the data object on the cloud as if it were an object at hand of the user terminal.
  • Java registered trademark
  • this framework for example, applied to object-oriented programming such as Java (registered trademark)
  • communication between a server and a database, a place where data is stored, and a place where processing is performed It can be developed as if it were an application program that runs locally without being conscious of the above.
  • the proxy node for communicating with the object on the database node corresponding to the key is held in the client node as necessary.
  • the proxy object itself is known as Java (registered trademark) technology, but in this framework, the proxy object (client side object) holds key information in this system. And this proxy object is used at the time of the method call mentioned later.
  • the client node hooks a Java method call to the above proxy object and automatically converts it to a “remote procedure call”. Execute method call on the object.
  • the job automatically generated by this framework includes information on the method name and argument to be called, and is distributed to the database node specified by the key value and distributedly executed by this system.
  • FIG. 16 is a diagram for explaining the present system including the above-described framework (sometimes referred to as automatic job generation technology).
  • the client node holds a proxy object, and the proxy object includes a hook method that is executed when the method is called, and when the method is called, the original method name and arguments are related. Information (eg, argument class and value) is passed. As a result, the execution result of the hook method becomes the original method execution result.
  • Information eg, argument class and value
  • This key object holds key information that identifies the object on the database node.
  • a job is automatically generated and distributed to the database node specified based on the hash value of the key.
  • the key information and job are delivered to the database node.
  • the job has a method name, information about the argument (argument class and value), and the contents of processing to execute the above method and return the result. included.
  • the database node that receives this job loads the target object corresponding to the key from the database, executes the process specified by the job for the data in the target object, and returns the execution result to the client node. At this time, when the data is changed by the process, the database node stores the changed target object in the database.
  • the total amount is calculated from the price and number of mandarin oranges placed in a shopping cart in online shopping.
  • FIGS. 8 to 13 the description of the program for distributing the defined job to the database node specified by the key and executing the process is omitted. However, in FIG. Is described.
  • the program in “Application Example 1” first uses the method “jobcast.store” for registering the processing in this system, the key to be processed and the processing to be executed (here “orange”). And register. Then, in order to set the price and number of mandarin oranges, generate a corresponding job ("OrangeSet”), and use "jobcast.execute", a method for delivering jobs in this system. The job is distributed to the database node specified by the key, and the process is executed. Furthermore, in order to calculate the total amount of mandarin oranges, the corresponding job (“OrangeTotal”) is generated, and it is generated using the method for job distribution in this system called "jobcast.execute” Distribute the job to the database node specified by the key and execute the process.
  • the application program developer includes a program for defining a job for each process and a program for generating, distributing, and executing the job in the system. It will be described in a specific way.
  • FIG. 17 shows an example of Java (registered trademark) programming.
  • an interface for the process here, “Orange”
  • the processing content is defined as “OrangeImpl”.
  • “Application example 2” for executing the defined processing is described so as to generate an “Orange” object and call each method defined in “OrangeImpl”.
  • a virtual object (* 2) having a method (* 1) defined by a certain interface ("Orange" in this example) is generated, and the above
  • the method (* 1) is called, it is possible to define a virtual method that receives information on the method name and arguments (parameters when calling the method).
  • the method name and argument information at the time of the method call can be passed to the “specific method” (* 3) when the method is called. it can.
  • the key information is given to the object of (* 2) above, and the original method name (* 4) and method called in the “specific method” of (* 3) above.
  • a job (* 6) is generated from the argument information (* 5) when is called, the job is sent to the database node specified by the key, and the result (specified by the key in the destination database node) A result obtained by executing the job on the data object to be received. That is, the “specific method” in (* 3) above automatically performs job generation, job distribution, and result reception.
  • the job generated by the “specific method” in (* 3) above is an object for which the processing content to be executed in response to the method call is specified.
  • FIG. As described in “Definition Example”, a generalized job class (“classGenericJob”) is defined in advance.
  • the job of (* 6) generated individually is one object (instance) of the generalized job class, and the method name of (* 4) and the argument information of (* 5) above have.
  • the original method information defined in the target object stored on the database node is searched from the method name and argument information, and the method of the target object is called. This method will be executed on the database node.
  • the “specific method” selects a target database node based on the key information passed when the proxy object is generated, and sends the job instance to the database node.
  • the result of executing the job by the node is received via the network.
  • the received result is finally handled as the execution result of the above “obj.setOrange (10,100)”.
  • getTotal the “specific method” of the proxy object generated by “getProxy” in advance is called, and the method name that was originally called (“getTotal”) A job instance is created based on this. The contents of this job have information called getTotal, which is the method name.
  • the “specific method” selects a target database node based on the key information passed when the proxy object is generated, and sends the job instance to the database node.
  • the result of executing the job by the node is received via the network.
  • the received result is finally handled as the execution result of “obj. GetTotal ()”.
  • the implementation method of FIG. 15 can be applied to the application example of FIG. 14, but the implementation method using the automatic job generation technique is similarly applied to the application examples of FIGS.
  • automatic job generation technology it is possible to adopt If automatic job generation technology is used in this way, it will be possible to develop large-scale parallel processing programs that use this system, and by executing such developed programs, data processing and storage will be performed automatically in the cloud. It will be performed in a distributed manner.
  • the “with change” process is a process that needs to be saved again if the value is changed, and an exclusive lock is performed on the server side so that a plurality of processes cannot be executed simultaneously.
  • the “read” process is a process in which re-saving is skipped because it is not conscious of a change in value, and a plurality of processes can be executed at the same time.
  • “read” or “with change” is marked with a technique called “annotation” of Java (registered trademark). Marks can be defined for each method. Then, when automatically generating a job, a job that is actually generated is distinguished based on the presence or absence of a mark. For example, the default (when there is no mark) may be executed as a process involving a change.
  • a job to which an operation ID is assigned is not limited to a job that is automatically generated, and can be assigned to all jobs used in this system.
  • a unique operation ID (for example, using UUID) is assigned to each job, and then distributed to the database node.
  • the database node when the processing by a certain job is “executed”, the result is recorded in correspondence with the operation ID assigned to the job.
  • the database node that received the job receives the received operation ID and record.
  • the job has been previously received and has been "executed" (that is, it has failed after the job is executed on the server side and before the result arrives at the client) You can determine whether it is the first time.
  • the database node If it is determined that the database node is “executed”, the database node returns the result recorded in correspondence with the operation ID to the client node. If it is determined that it is not “executed”, the database node performs processing again. Execute and return the result to the client node.
  • FIGS. 18 and 19 summarize the advantages of this system.
  • this system has a small amount of data transfer at one time, and the number of communications is small even in a process that spans multiple elements. Moreover, since only the processing result is returned, it is suitable for handling confidential data.
  • FIG. 19 in this system, the amount of communication when accessing a single item is small, and the amount of communication when accessing a plurality of items is also small. And because of back-end processing (processing on the database node side), confidential data does not flow out to the network.
  • the “conventional” system in FIGS. 18 and 19 is a system in which data is divided and stored in detail with KVS, and the system of “reference example” is configured to store large data in KVS as a unit. System.
  • a distributed database system has been described in detail as an example of the system according to the embodiment of the present invention.
  • a distributed data processing system illustrated in FIG. It is also possible to configure this (also called “the present system”).
  • the distributed data processing system in FIG. 20 includes a client node that inputs log record information from a device and a plurality of data processing nodes that perform record storage / retrieval processing in a parallel and distributed manner.
  • the client node in FIG. 20 has a function corresponding to the client node of the distributed database system in addition to functioning as a reception unit. Specifically, the process is distributed by generating a key from the received record information and distributing the job to the data processing node specified by the key.
  • Each data processing node in FIG. 20 has a function corresponding to the database node of the distributed database system, except that the data storage destination is not the node but the shared storage provided in the back end.
  • the record is processed by executing the job sent from.
  • the above data processing node performs indexing (indexing) on the record information of the log and writes it to the storage according to the index.
  • indexing indexing
  • the log is accumulated by additionally writing the record received from the client node to the data area corresponding to the index each time. Therefore, log processing is possible continuously (in operation for 24 hours 365 days) in real time, not in batch processing.
  • This system is used for processing and storing communication logs of communication carriers, company network usage logs, machine sensor logs, etc., as well as sales / purchase records of shops, dealers, online shops, etc. Suitable for applications that record a large amount of data.
  • each device may transmit a record to any client node.
  • the data processing node distributes processing among a plurality of data processing nodes based on the key. Performance is improved).
  • the stages of log generation (device), log reception (client node), index processing (data processing node), and writing (storage) are connected in series, and one after another. Since the processing can be performed in a pipeline manner, and each stage is executed by parallel distributed processing of a large number of devices, high performance for processing a large amount of logs at high speed is possible.
  • the storage in FIG. 20 may be a shared storage, but the storage technology of Japanese Patent Application No. 2010-237828 (PCT / JP2011 / 005629; the contents of the application are incorporated herein by reference) proposed by the present inventor. If it is used, it is possible to realize scale-out and high performance in storage writing / reading. That is, if a data area for additional writing (accumulation) is provided on the storage of the prior application, the storage capacity can also be expanded in a scalable manner.
  • FIG. 21 shows communication log processing as an application example of the system of FIG.
  • a communication device as a device generates record information “from which address and which URL has been accessed” and transmits it to an arbitrarily selected client node.
  • the client node Based on the record information received from the device, the client node generates, for example, a job to be described later using “transmission source address” as a key, and transmits it to the data processing node specified according to the key.
  • Data processing node executes the job received from the client node. For example, a “source address” that is a key is treated as an index (which may correspond to a search key in the database), and record information is additionally written (accumulated) for each key.
  • the record (log contents) is “communication time (8 bytes); transmission source address (4 bytes); transmission destination address (4 bytes); transmission source port (2 bytes); transmission destination port (2 bytes) ); URL to access (character string) ”, and when the element to be indexed is the source address, the client node uses the 4 bytes from the 8th byte from the beginning of the record as a key. Then, the client node calculates a hash value of the key, and specifies a data processing node that is a job distribution destination according to the calculated value.
  • the data processing node that has received the above key and job additionally writes (accumulates) the record information included in the received job to the data area corresponding to the key that is the element to be used as the index (here, the source address).
  • the output destination file name is determined based on the received key, and the record information is output as a file stored in the shared storage.
  • the file name may be determined as “key-date-time.log” from the key and the received date and time.
  • an index here, a transmission source address
  • the data processing node returns all record information read from the data area indicated by the index in the result.
  • detailed search conditions for example, time, transmission destination address, port number, any combination thereof, etc.
  • jobs are distributed using the elements to be indexed as keys, so the data nodes to be processed are distributed according to the data area to be accessed, and the same data area from multiple data processing nodes Can reduce the possibility of contention being accessed.
  • the same data processing node that performs processing for storing (including writing to the storage) for the index also performs processing for searching (including reading from the storage).
  • a data processing node may be specified using a combination of a source address (element to be an index) and time information (other elements) as a key.
  • the data processing node does not use the entire key information as an index but extracts an index portion from the key information as an index.
  • the distributed database system or distributed data processing system according to the principle of the present invention is useful because it can provide various data storage services, for example.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computational Linguistics (AREA)
  • Probability & Statistics with Applications (AREA)
  • Software Systems (AREA)
  • Mathematical Physics (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Fuzzy Systems (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
PCT/JP2011/005892 2010-10-22 2011-10-20 分散型データベースシステムおよび分散型データ処理システム WO2012053219A1 (ja)

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
JP2010237102 2010-10-22
JP2010-237102 2010-10-22
JP2011229280A JP5984355B2 (ja) 2010-10-22 2011-10-18 分散型データベースシステムおよび分散型データ処理システム
JP2011-229280 2011-10-18

Publications (1)

Publication Number Publication Date
WO2012053219A1 true WO2012053219A1 (ja) 2012-04-26

Family

ID=45974948

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2011/005892 WO2012053219A1 (ja) 2010-10-22 2011-10-20 分散型データベースシステムおよび分散型データ処理システム

Country Status (2)

Country Link
JP (1) JP5984355B2 (enrdf_load_stackoverflow)
WO (1) WO2012053219A1 (enrdf_load_stackoverflow)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP5238915B1 (ja) * 2012-08-23 2013-07-17 ディジタル・ワークス株式会社 分散型データベースシステム

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR101480293B1 (ko) 2012-11-30 2015-01-12 주식회사 엘지씨엔에스 분산 파일시스템 처리 장치와 방법 및 이를 저장한 기록 매체
US9088573B2 (en) * 2013-03-22 2015-07-21 Dropbox, Inc. Local server for synced online content management system
WO2014199568A1 (ja) * 2013-06-12 2014-12-18 日本電気株式会社 永続記憶装置へのデータ書込制御方法
CN103886030B (zh) * 2014-03-05 2017-01-25 南京邮电大学 基于代价敏感决策树的信息物理融合系统数据分类方法

Non-Patent Citations (4)

* Cited by examiner, † Cited by third party
Title
MUGA MLHHIZJAWA: "ROMA to sono Plug-in KIKO", SOFTWAREDESIGN, 18 February 2010 (2010-02-18), pages 36 - 41 *
TAKAHIRO MAEBASHI: "Implementation of Distributed System for Large-Scale Data Processing and Its Application", THE TRANSACTIONS OF THE INSTITUTE OF ELECTRONICS, INFORMATION AND COMMUNICATION ENGINEERS, vol. J93-D, no. 7, 1 July 2010 (2010-07-01), pages 1072 - 1081 *
YOSHI TAKATSUKI: "Tokusen Free Soft ROMA Ruby de Ugoku Key Value-gata Data Store", NIKKEI LINUX, vol. 12, no. 5, 8 April 2010 (2010-04-08), pages 92 - 94 *
YUKIHIRO MATSUMOTO: "Yukihiro MATSUMOTO Gijutsu o Kiru", NIKKEI LINUX, vol. 11, no. LL, 8 October 2009 (2009-10-08), pages 150 - 156 *

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP5238915B1 (ja) * 2012-08-23 2013-07-17 ディジタル・ワークス株式会社 分散型データベースシステム
WO2014030235A1 (ja) * 2012-08-23 2014-02-27 ディジタル・ワークス株式会社 分散型データベースシステム
CN103765391A (zh) * 2012-08-23 2014-04-30 数创株式会社 分布式数据库系统

Also Published As

Publication number Publication date
JP2012108889A (ja) 2012-06-07
JP5984355B2 (ja) 2016-09-06

Similar Documents

Publication Publication Date Title
US8438238B2 (en) Master data access
US7200806B2 (en) System and method for generating pre-populated forms
JPWO2021002226A5 (enrdf_load_stackoverflow)
CN109964216A (zh) 识别未知数据对象
US20080306904A1 (en) System, method, and program product for integrating databases
WO2005055081A1 (en) Updating and maintaining data in a multi-system network using asynchronous message transfer
JP2001056810A (ja) データベースアクセスシステム
JPS62125445A (ja) デ−タ・オブジエクト・レプリカの管理方法
JP5984355B2 (ja) 分散型データベースシステムおよび分散型データ処理システム
WO1991004532A1 (fr) Systeme a centre temporaire dans un systeme a base de donnees decentralisee
CN106997557A (zh) 订单信息采集方法及装置
US10812322B2 (en) Systems and methods for real time streaming
JP4357827B2 (ja) ピアツーピア型文書共有ネットワークシステム
US20030163465A1 (en) Processing information about occurrences of multiple types of events in a consistent manner
US10657139B2 (en) Information processing apparatus and non-transitory computer readable medium for distributed resource management
WO2019030409A1 (en) SYSTEMS AND METHODS FOR ASSEMBLING DATA SETS
JP2006031608A (ja) 計算機、ストレージシステム、計算機が行うファイル管理方法、およびプログラム
JP3811615B2 (ja) 情報配信システム、装置及び方法並びに記録媒体
KR20210082481A (ko) 데이터베이스 관리 서비스 제공 시스템
JP2017010569A (ja) 分散型データベースシステムおよび分散型データ処理システム
JPH06290098A (ja) 分散データベース処理方法
JP3761911B2 (ja) ファイルサーバおよびファイル管理方法
JP3709905B2 (ja) ネットワークサーバ装置
JPH09114784A (ja) アクセス権評価装置
JP2003233517A (ja) 文書管理システム、文書管理サーバ、ディレクトリサーバ、クライアント端末及びプログラム

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 11834065

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 11834065

Country of ref document: EP

Kind code of ref document: A1