JP2022014633A - Distributed processing system, distributed processing device, database management device, and method - Google Patents

Abstract

To provide a distributed processing system, a distributed processing device, a database management device, and a method capable of accelerating processing on a query.SOLUTION: According to an embodiment, a distributed processing device calculates first processing time of a first database management device and second processing time of a second database management device for a first query for acquiring first and second records on the basis of management device information and creates second and third queries on the basis of the first and second processing times. First query execution means of the first database management device acquires a part of a plurality of second records included in a second table and executes processing on the second query when the first processing time is less than the second processing time. Second query execution means of the second database management device acquires a part of a plurality of first records included in a first table and executes processing on the third query when the second processing time is less than the first processing time.SELECTED DRAWING: Figure 16

Description

Embodiments of the present invention relate to distributed processing systems, distributed processing devices, database management devices and methods.

In recent years, a distributed processing system (database management system) capable of collectively acquiring data from a plurality of distributed databases has been known.

In such a distributed processing system, for example, when a query for acquiring a join operation result of a plurality of data (tables) stored in each database is received from a client terminal, the distributed processing system processes the query. It is possible to provide the result (combination operation result) of the above to the client terminal (user who uses).

By the way, the distributed processing system includes, for example, a distributed processing device and a plurality of database management devices (hereinafter referred to as DB management devices) that manage each of the plurality of databases, and the distributed processing device (parent node) is described above. The processing for the query can be pushed down (distributed) to each of the DB management devices (child nodes). According to such a distributed processing system, it is possible to reduce the load on the distributed processing apparatus, for example.

However, the processing amount of each DB management device in this case depends on the amount of data stored in the database managed by the DB management device. Therefore, for example, if there is a DB management device that manages a database having a larger amount of data than a database managed by another DB management device, it takes time to process the DB management device, so that distributed processing is performed. The operation of the entire system (that is, processing for queries) may be slow.

Japanese Patent No. 6253725

Therefore, an object to be solved by the present invention is to provide a distributed processing system, a distributed processing device, a database management device, and a method capable of speeding up processing for a query.

According to the embodiment, the distributed processing device, the first database management device that manages the first database communicably connected to the distributed processing device, and the second database that manages the second database different from the first database. A distributed processing system including a management device is provided. The first database stores a first table having a plurality of first records collected in the first database management device. The second database stores a second table having a plurality of second records collected in the second database management device. The first and second records are composed of the same field. The distributed processing apparatus includes a storage means, a receiving means, a calculation means, and a creating means. The storage means includes the arithmetic performance of each of the first and second database management devices, the number of first records of the first table and the number of second records of the second table, or the distributed processing device. Stores management device information including first communication performance between the first and second database management devices. The receiving means receives the first query for acquiring the first and second records of the first and second tables. The calculation means calculates the first processing time of the first database management device and the second processing time of the second database management device for the received first query based on the management device information. The creating means creates the second and third queries based on the calculated first and second processing times. The first database management device includes a first query execution means for executing a process for the created second query. The second database management device includes a second query execution means for executing a process for the created third query. When the calculated first processing time is less than the calculated second processing time, the first query executing means may obtain a part of a plurality of second records of the second table from the second database. Acquire and execute the process for the second query. When the calculated second processing time is less than the calculated first processing time, the second query executing means may use a part of the plurality of first records of the first table from the first database. Acquire and execute the process for the third query.

The figure which shows an example of the network configuration of the distributed processing system which concerns on embodiment. The figure which shows an example of the hardware composition of the distributed processing apparatus. The figure which shows an example of the table of the 1st DB management apparatus in the comparative example of this embodiment. The figure which shows an example of the table of the 2nd DB management apparatus in the comparative example of this embodiment. The figure which shows an example of the table of the 3rd DB management apparatus in the comparative example of this embodiment. The figure for demonstrating the outline of the processing executed in the distributed processing apparatus in the comparative example of this embodiment. The figure for demonstrating the outline of the process executed in the 1st DB management apparatus in the comparative example of this embodiment. The figure for demonstrating the outline of the process executed in the 2nd DB management apparatus in the comparative example of this embodiment. The figure for demonstrating the outline of the process executed in the 3rd DB management apparatus in the comparative example of this embodiment. The figure which shows the outline of the operation of the distributed processing system which concerns on this embodiment. The figure which shows an example of the functional configuration of a distributed processing apparatus. The figure which shows an example of the functional structure of the 1st DB management apparatus. The figure which shows an example of the data structure of the arithmetic performance information. The figure which shows an example of the data structure of the record number information. The figure which shows an example of the data structure of communication performance information. A sequence chart showing an example of the processing procedure of a distributed processing system. The figure for demonstrating concretely about the processing time of the 1st DB management apparatus. The figure for demonstrating concretely about the processing time of the 2nd DB management apparatus. The figure for demonstrating the difference between the distributed processing apparatus and a load balancer which concerns on this embodiment. The figure for demonstrating the route to acquire the record determined based on the communication performance information. The figure for demonstrating an example of the configuration which acquires a record using the connection information to the 1st DB management apparatus. The figure for demonstrating another example of the configuration which acquires a record using the connection information to the 1st DB management apparatus. The figure for demonstrating the structure which distributes the processing to the 4th DB management apparatus.

Hereinafter, embodiments will be described with reference to the drawings.
FIG. 1 shows an example of a network configuration of a distributed processing system according to the present embodiment. As shown in FIG. 1, the distributed processing system 1 includes a client terminal 10, a distributed processing device 20, and first to third database management devices (hereinafter referred to as DB management devices) 31 to 33.

In the present embodiment, the client terminal 10 is connected to the distributed processing device 20 so as to be able to communicate with each other. Further, the distributed processing device 20 is communicably connected to each of the first to third DB management devices 31 to 33.

The client terminal 10 is a terminal device used by the user. The client terminal 10 operates an application program or the like for the user to acquire and browse the data stored in the database described later.

The distributed processing device 20 is a server device (DBMS) that operates as a parent node for the first to third DB management devices 31 to 33 in the distributed processing system 1. The distributed processing device 20 has a function of distributing the processes to be executed in the distributed processing system 1 to the first to third DB management devices 31 to 33 and causing the first to third DB management devices 31 to 33 to execute the processes.

The first to third DB management devices 31 to 33 are server devices (DBMS) that operate as child nodes for the distributed processing device 20 in the distributed processing system 1 and manage a database for storing various data.

In this embodiment, the distributed processing device 20 and the first to third DB management devices 31 to 33 are assumed to be the same DBMS.

Further, in FIG. 1, the distributed processing system 1 has been described as including the client terminal 10, the distributed processing device 20, and the first to third DB management devices 31 to 33, but the distributed processing system 1 includes the distributed processing device 20 and the first. The configuration may include the first to third DB management devices 31 to 33 (that is, the client terminal 10 is arranged outside the distributed processing system 1).

Further, although the first to third DB management devices 31 to 33 are shown in FIG. 1, the distributed processing system 1 may be provided with a plurality of DB management devices.

FIG. 2 shows an example of the hardware configuration of the distributed processing apparatus 20 shown in FIG. As shown in FIG. 2, the distributed processing device 20 includes a CPU 21, a non-volatile memory 22, a main memory 23, a communication device 24, and the like.

The CPU 21 is a processor that controls the operation of each component in the distributed processing device 20. The CPU 21 executes various programs loaded from the non-volatile memory 22 which is a storage device into the main memory 23. This program includes a program for operating the operating system (OS) and the distributed processing apparatus 20 in the distributed processing system 1.

The communication device 24 is a device configured to execute wired or wireless communication with the client terminal 10, the first to third DB management devices 31 to 33, and the like.

Although only the CPU 21, the non-volatile memory 22, the main memory 23, and the communication device 24 are shown in FIG. 2, the distributed processing device 20 is, for example, an HDD (Hard Disk Drive) and an SSD (Solid State Drive). Other storage devices and the like may be further provided, and other devices may be further provided.

Although the hardware configuration of the distributed processing device 20 has been described here, it is assumed that the first to third DB management devices 31 to 33 also have the same configuration.

Here, the distributed processing system 1 according to the present embodiment is a database management system (distributed database) that collectively acquires data from the databases (distributed databases) managed by each of the first to third DB management devices 31 to 33. Operates as an integrated database).

Specifically, the user who uses the client terminal 10 can specify (input) a query for acquiring data from the database by operating the client terminal 10. The query specified by the user is, for example, an instruction statement (SQL statement) described in SQL (Structured Query Language).

When such a query is transmitted from the client terminal 10 to the distributed processing device 20, the distributed processing system 1 (distributed processing device 20 and the first to third DB management devices 31 to 33) executes processing for the query. , The result of the process (data acquired from the database) is returned to the client terminal 10.

For example, when two types of data are stored in the databases managed by the first to third DB management devices 31 to 33, for example, a join operation (a process of joining two types of data) is performed as a process for a query. ), And the result of the join operation can be returned to the client terminal 10.

For example, a database in which each of the first to third DB management devices 31 to 33 is provided in a different store (for example, a retail store or a mass retailer) and is managed by the first to third DB management devices 31 to 33. Is independently collected in the 1st to 3rd DB management devices 31 to 33 (stores in which the stores are provided) and is sold at the store and data related to the sales of the store (hereinafter referred to as sales data). It is assumed that the product data (hereinafter referred to as product data) is stored.

In such a case, when the query for acquiring the combined operation result of the sales data and the product data is specified by the user, it is stored in the database managed by the first to third DB management devices 31 to 33. The result of executing the join operation on the sales data and the product data is returned to the client terminal 10.

The sales data and product data described above are table format data and have a plurality of records. In this case, the distributed processing device 20 handles the sales data and the product data stored in the databases managed by the first to third DB management devices 31 to 33 as an external table, and handles the sales data and the product data, respectively. It is assumed that a virtual table is created and the processing for the query is executed. The external table is a function that behaves as if a table of a different data source is a table of its own database. In addition, virtual table creation realizes the acquisition of batch data from multiple data sources by virtualizing the tables of multiple data sources into one table using (the function of) an external table. It is a mechanism.

Here, before explaining the details of the distributed processing system 1 according to the present embodiment, the processing for the query executed in the distributed processing system according to the comparative example of the present embodiment will be specifically described.

Here, the distributed processing system according to the comparative example of the present embodiment includes the distributed processing devices 20'and the first to third DB management devices 31'to 33', and the first to third DB management devices 31'to 33' It is assumed that each of the managed databases stores a table t1 corresponding to the above-mentioned sales data and a table t2 corresponding to the product data.

In the present embodiment, it is assumed that the table t1 is the sales data and the table t2 is the product data, but the tables t1 and t2 may be other data. Specifically, the tables t1 and t2 may be, for example, power generation data at a power plant and operation logs of various devices installed at the power plant.

In the following description, the tables t1 and t2 stored in each of the databases managed by the first to third DB management devices 31'to 33' are, for convenience, the first to third DB management devices 31'to. It is referred to as tables t1 and t2 of 33'.

First, the tables t1 and t2 of the first to third DB management devices 31'to 33' will be described with reference to FIGS. 3 to 5. FIG. 3 shows an example of tables t1 and t2 (data structure) of the first DB management device 31'. FIG. 4 shows an example of the tables t1 and t2 (data structure) of the second DB management device 32'. FIG. 5 shows an example of tables t1 and t2 (data structure) of the third DB management device 33'.

As shown in FIGS. 3 to 5, the records of each of the tables t1 of the first to third DB management devices 31'to 33' are composed of the same fields (id, time and value). When the table t1 corresponds to the sales data as described above, the id corresponds to the identification information for identifying the product sold in the store. time corresponds to the time when the goods identified by id were sold. The value corresponds to additional information such as the amount of goods identified by id (that is, sales) and information about the purchaser who purchased the goods (eg, age, gender, etc.).

Similarly, the records of each of the tables t2 of the first to third DB management devices 31'to 33' are composed of the same fields (id and name). When the table t2 corresponds to the product data as described above, the id corresponds to the identification information for identifying the product sold in the store. name corresponds to the name of the product (that is, the product name) identified by id.

Here, it is assumed that the query for acquiring the join operation result of the tables t1 and t2 described above is specified by the user. The query in this case is, for example, "SELECT * FROM t1, t2 WHERE t1.id = t2.id". This query is meant to return the result of joining tables t1 and t2 based on the id (fields) that make up the records that the tables t1 and t2 have. In addition, "t1.id = t2.id" in the query "SELECT * FROM t1, t2 WHERE t1.id = t2.id" corresponds to the condition (expression) that the tables t1 and t2 are combined based on id. do. In the following description, this "t1.id = t2.id" is referred to as a conditional expression of the query.

In this case, as shown in FIG. 6, the distributed processing device 20'collects each of the tables t1 of the first to third DB management devices 31'to 33'from the first to third DB management devices 31'to 33'. Then, a table T1 is created by combining the collected tables t1.

Similarly, the distributed processing device 20'collects each of the tables t2 of the first to third DB management devices 31'to 33' from the first to third DB management devices 31'to 33', and the collected tables. Create a table T2 that combines t2.

Next, the distributed processing apparatus 20'executes a process (joining operation) of joining the tables T1 and T2 based on the conditional expression "t1.id = t2.id" of the above-mentioned query. As a result, the distributed processing apparatus 20 acquires the table T12 as the result of the join operation. This table T12 is returned from the distributed processing device 20'to the client terminal 10 as a response to the query.

In FIG. 6 described above, it is assumed that the join operation is executed by the distributed processing device 20'(parent node). In this case, the distributed processing device 20'is the first to third DB management devices 31'to 33'. Each of the plurality of records possessed by the table T1 in which the table t1 of ′ is combined and each of the plurality of records possessed by the table T2 in which the tables t2 of the first to third DB management devices 31 ′ to 33 ′ are combined are the above-mentioned queries. It is necessary to determine whether or not the records can be combined based on the conditional expression of, and to execute the process of combining the records when it is determined that the records can be combined. In this case, the calculation amount of the distribution processing device 20'is a direct product of N × M (N is the number of records in the table T1 and M is the number of records in the table T2), and the number of records in the table T1 and the number of records in the table T1. Depending on the number of records in the table T2, the load on the distributed processing device 20'will increase. Therefore, it may take a lot of time to process the query.

Therefore, in the distributed processing system according to the comparative example of the present embodiment, a part of the processing of the distributed processing device 20'(parent node) is pushed to the first to third DB management devices 31'to 33'(child node). It is down (distributed) to reduce the load on the distributed processing apparatus 20'.

Hereinafter, a case where the processing (joining operation) for the query executed in the comparative example of the present embodiment is distributed to the first to third DB management devices 31'to 33' will be described.

The tables t1 and t2 of the first to third DB management devices 31'to 33' are as described in FIGS. 3 to 5 described above, and the query "SELECT * FROM t1, t2 WHERE t1.id = t2" by the user. It is assumed that ".id" is specified.

In this case, the distributed processing device 20'uses one of the tables t1 and t2 of the first to third DB management devices 31'to 33' (here, the table t2) from the first to third DB management devices 31'to. Collect from 33'and create table T2. The tables T2 created in this way are distributed to the first to third DB management devices 31'to 33'. The process (join operation) for the query specified by the user is executed using the table T2 distributed from the distributed processing device 20'in this way. According to this, in each of the first to third DB management devices 31'to 33', the table t2 does not have a record composed of the same id as the id constituting the record possessed by the table t1. Also, if the table T2 has a record composed of the same id as the id constituting the record possessed by the table t1, the records can be appropriately combined.

Next, the processes executed by the first to third DB management devices 31'to 33' will be described.

First, FIG. 7 is a diagram for explaining an outline of processing executed in the first DB management device 31'. As shown in FIG. 7, the first DB management device 31'refers to the table t1 of the first DB management device 31'and the table T2 distributed from the distributed processing device 20', and the conditional expression "t1. The join operation based on "id = t2.id" is executed. According to this, the first DB management device 31'acquires the table t12 shown in FIG. 7 as the result of the join operation.

Further, FIG. 8 is a diagram for explaining an outline of the processing executed by the second DB management device 32'. As shown in FIG. 8, the second DB management device 32'refers to the table t1 of the second DB management device 32'and the table T2 distributed from the distributed processing device 20', and the conditional expression "t1. The join operation based on "id = t2.id" is executed. According to this, the second DB management device 32'acquires the table t12 shown in FIG. 8 as the result of the join operation.

Further, FIG. 9 is a diagram for explaining an outline of the processing executed by the third DB management device 33'. As shown in FIG. 9, the third DB management device 33'refers to the table t1 of the third DB management device 33'and the table T2 distributed from the distributed processing device 20', and the conditional expression "t1. The join operation based on "id = t2.id" is executed. According to this, the third DB management device 33'acquires the table t12 shown in FIG. 9 as the result of the join operation.

As described above, the table t12 acquired by the first to third DB management devices 31'to 33' is transmitted from the first to third DB management devices 31'to 33'to the distributed processing device 20'. The distributed processing device 20'acquires the same join operation result as the table T12 shown in FIG. 6 above by combining the tables t12 transmitted from each of the first to third DB management devices 31'to 33'. Can be done.

When the processing (join operation) for the query is distributed to each of the first to third DB management devices 31'to 33' in this way, the load on the distributed processing device 20'can be reduced.

However, the processing amount in each of the first to third DB management devices 31'to 33'is a direct product of the number of records in the table t1 x the number of records in the table T2, and the first to third DB management devices 31. It depends on the number of records in each table t1 of ′ to 33 ′. That is, if the number of records (that is, the data size) of the table t1 is biased between the first to third DB management devices 31'to 33', the processing in a specific DB management device takes an extremely long time. Things can happen. In this case, the distributed processing device 20'and other DB management devices must wait until the processing of the specific DB management device is completed, and the distributed processing system as a whole executes the processing for the query at high speed. Can't.

Therefore, in the present embodiment, when the processing for the query is distributed to the first to third DB management devices 31 to 33 as described above, a part of the processing of the DB management device having a large processing amount is partially processed by the DB having a small processing amount. By causing the management device to execute, the processing for the query is speeded up as the entire distributed processing system 1.

FIG. 10 shows an outline of the operation of the distributed processing system 1 according to the present embodiment. In FIG. 10, it is assumed that the number of records in the table t1 of the first DB management device 31 is larger than the number of records in the table t1 of the second DB management device 32, for example.

In this case, assuming that the first DB management device 31 has executed the join operation for the table t1 of the first DB management device 31 and the table T2 described above, the processing amount of the first DB management device 31 is the second DB management device 32. It will be more than the processing amount of.

Therefore, in the present embodiment, a part of the record held in the table t1 of the first DB management device 31 is transferred to the second DB management device 32, and a part of the record is processed on the second DB management device 32 side.

According to this, a part of the processing of the first DB management device 31 can be distributed to the second DB management device 32, and the processing amount of the first DB management device 31 can be reduced. Therefore, the processing of the distributed processing system 1 as a whole can be performed. It is possible to increase the speed.

In FIG. 10, the record held by the table t1 of the first DB management device 31 is transferred to the second DB management device 32, but the record is transferred to both the second and third DB management devices 32 and 33. Alternatively, it may be transferred only to the third DB management device 33.

Hereinafter, the configuration of the distributed processing system 1 according to the present embodiment will be described in detail. First, FIG. 11 shows an example of the functional configuration of the distributed processing apparatus 20 provided in the distributed processing system 1.

As shown in FIG. 11, the distributed processing device 20 includes a storage unit 201, an information management unit 202, a communication unit 203, a query analysis unit 204, a query creation unit 205, a query execution unit 206, and a result accumulation unit 207.

In the present embodiment, the storage unit 201 is realized by a non-volatile memory 22 or another storage device provided in the distributed processing device 20.

Further, in the present embodiment, a part or all of the information management unit 202, the communication unit 203, the query analysis unit 204, the query creation unit 205, the query execution unit 206, and the result accumulation unit 207 are the CPU 21 provided in the distributed processing device 20. It is assumed that (that is, the computer of the distributed processing apparatus 20) executes a predetermined program, that is, it is realized by software. This predetermined program is a program for the distributed processing device 20 to operate in the distributed processing system 1, and may be stored in advance in a computer-readable storage medium and distributed, or the distributed processing device 20 may be distributed via a network. May be downloaded to.

Here, a part or all of each part 202 to 207 described above has been described as being realized by software, but a part or all of each part 201 to 207 may be realized by hardware, for example, or software. And may be realized as a combination of hardware.

The storage unit 201 stores information (hereinafter, referred to as management device information) regarding each of the first to third DB management devices 31 to 33. The management device information includes the calculation performance of each of the first to third DB management devices 31 to 33, and the records of each table of the first to third DB management devices 31 to 33 (for example, tables t1 and t2). , Or the communication performance (information indicating) between the distributed processing device 20 and each of the first to third DB management devices 31 to 33.

In the following description, information indicating the calculation performance of each of the first to third DB management devices 31 to 33 is the calculation performance information, and each table of the first to third DB management devices 31 to 33 (for example, tables t1 and t2). ) Is referred to as record number information, and information indicating communication performance between the distributed processing device 20 and each of the first to third DB management devices 31 to 33 is referred to as communication performance information. The management device information may include at least one of calculation performance information, record number information, and communication performance information, but in the present embodiment, the management device information includes calculation performance information and record number information. And all of the communication performance information will be described.

The information management unit 202 is a functional unit for managing the management device information stored in the storage unit. Specifically, the information management unit 202 provides information (calculation performance information, record number information, and communication performance information) related to the first to third DB management devices 31 to 33 transmitted from the first to third DB management devices 31 to 33. ) Is collected, and the information is stored in the storage unit 201 as management device information. Further, the information management unit 202 periodically collects information on the first to third DB management devices 31 to 33 described above, and updates the management device information stored in the storage unit 201 based on the collected information. You may execute the processing to be performed.

The communication unit 203 is a functional unit for executing communication with the client terminal 10. The communication unit 203 receives, for example, a query specified by the user in the client terminal 10 from the client terminal 10.

The query analysis unit 204 analyzes the query received by the communication unit 203, and determines the processing for the query to be executed by the first to third DB management devices 31 to 33.

The query creation unit 205 has a processing time for processing a query to be executed in the first to third DB management devices 31 to 33 determined by the query analysis unit 204 (that is, the first to third DB management devices 31 to 33 for the query). Predicted value of each processing time of) is calculated. The processing time of each of the first to third DB management devices 31 to 33 is calculated based on the management device information stored in the storage unit 201.

The query creation unit 205 makes a query to the first to third DB management devices 31 to 33 (databases managed by each of them) based on the calculated processing time of each of the first to third DB management devices 31 to 33. create.

The query execution unit 206 is a functional unit for causing each of the first to third DB management devices 31 to 33 to execute a query for the first to third DB management devices 31 to 33 created by the query creation unit 205. In this case, the query execution unit 206 transmits a query to the first to third DB management devices 31 to 33 to each of the first to third DB management devices 31 to 33.

According to the query transmitted from the distributed processing device 20 (query creating unit 205) to the first to third DB management devices 31 to 33, it is possible to execute a process (for example, a join operation) for the query specified by the user. It is instructed to the first to third DB management devices 31 to 33. Of the first to third DB management devices 31 to 33, for example, for a DB management device having a short processing time, a part of a plurality of records held by a table of another DB management device having a long processing time may be used. It is instructed to acquire from the DB management device of the above and execute the join operation.

The result collection unit 207 collects the results of executing the processing for the queries transmitted from the distributed processing device 20 to the first to third DB management devices 31 to 33 from the first to third DB management devices 31 to 33. The result accumulated by the result accumulation unit 207 in this way is transmitted to the client terminal 10 via the communication unit 203 as a processing result for the query (query specified by the user) received by the communication unit 203.

Next, FIG. 12 shows an example of the functional configuration of the first DB management device 31 provided in the distributed processing system 1.

As shown in FIG. 12, the first DB management device 31 includes a database 311, a communication unit 312, a query execution unit 313, and an information management unit 314.

In the present embodiment, the database 311 is realized by a non-volatile memory or another storage device provided in the first DB management device 31.

Further, in the present embodiment, a part or all of the communication unit 312, the query execution unit 313, and the information management unit 314 are predetermined to the CPU provided in the first DB management device 31 (that is, the computer of the first DB management device 31). It is assumed that the program is executed, that is, it is realized by software. This predetermined program is a program for the first DB management device 31 to operate in the distributed processing system 1, and may be stored and distributed in advance in a computer-readable storage medium, or the first DB management device 31 may be managed via a network. It may be downloaded to the device 31.

Here, a part or all of each part 312 to 314 described above has been described as being realized by software, but a part or all of each part 312 to 314 may be realized by hardware, for example, or software. And may be realized as a combination of hardware.

The database 311 stores the table format data provided to the user by executing the process for the query specified by the user on the client terminal 10 as described above. Specifically, the database 311 stores tables t1 and t2 having a plurality of records as shown in FIG. 3, for example. Since the data structures of the tables t1 and t2 stored in the database are as described in FIG. 3 described above, detailed description thereof will be omitted here.

The communication unit 312 is a functional unit for executing communication with the distributed processing device 20. The communication unit 312 receives the query (query for the first DB management device 31) transmitted from the distributed processing device 20. The communication unit 312 may be configured to further execute communication with other DB management devices (second and third DB management devices 32 and 33).

The query execution unit 313 is a functional unit for executing processing for the query received by the communication unit 312.

When the processing time of the first DB management device 31 calculated by the distributed processing device 20 (query creation unit 205) is less than, for example, the processing time of the second DB management device 32 as described above, the first DB management device 31 is the second DB. The distributed processing device 20 is instructed to acquire a part of a plurality of records possessed by the table of the management device 32 (for example, the table t1). According to such an instruction, the query execution unit 313 acquires a part of a plurality of records included in the table of the second DB management device 32 from the second DB management device 32. In this case, the query execution unit 313 executes the processing (join operation) for the query by using the plurality of records in the table stored in the database 311 and the records acquired from the second DB management device 32.

On the other hand, when the processing time of the second DB management device 32 calculated by the distributed processing device 20 (query creation unit 205) is less than the processing time of the first DB management device 31 as described above, the table stored in the database 311. A part of the plurality of records possessed by the second DB management device 32 is transferred to the second DB management device 32 (that is, acquired by the second DB management device 32). In this case, the query execution unit 313 executes the processing (join operation) for the query by using the records that have not been transferred to the second DB management device 32 among the plurality of records in the table stored in the database 311. ..

The information management unit 314 is a functional unit that manages information (calculation performance information, record number information, and communication performance information) related to the first DB management device 31. It is assumed that the information regarding the first DB management device 31 is stored in the database 311. The information management unit 314 periodically transmits information about the first DB management device 31 to the distributed processing device 20 via the communication unit 312. Information about the first DB management device 31 may be transmitted to the distributed processing device 20 when the first DB management device 31 is connected to the distributed processing device 20, for example, and the calculation performance of the first DB management device 31. It may be transmitted to the distributed processing device 20 when the number of records in the table stored in the database 311 or the communication performance with the distributed processing device 20 changes.

Although the functional configuration of the first DB management device 31 has been described in FIG. 12, the functional configurations of the second and third DB management devices 32 and 33 are the same as those of the first DB management device 31. Hereinafter, not only the configuration of the first DB management device 31 but also the configurations of the second and third DB management devices 32 and 33 will be described with reference to FIG.

Here, an example of the data structure of the management device information stored in the storage unit 201 included in the distributed processing device 20 shown in FIG. 11 described above will be described with reference to FIGS. 13 to 15. The management device information is, for example, tabular information (data), and includes calculation performance information, record number information, and communication performance information as described above.

FIG. 13 shows an example of the data structure of the calculation performance information included in the management device information. As described above, the calculation performance information is information indicating the calculation performance of the first to third DB management devices 31 to 33, but in FIG. 13, the calculation performance of the first to third DB management devices 31 to 33 is used. In addition, a data structure of arithmetic performance information further indicating the arithmetic performance of the distributed processing apparatus 20 is shown.

As shown in FIG. 13, the calculation performance information includes the node ID and the calculation performance value in association with each other. The node ID is identification information for identifying each node (distributed processing device 20 and first to third DB management devices 31 to 33) constituting the distributed processing system 1. The calculation performance value is a value representing the calculation performance of the node identified by the node ID, and indicates, for example, the number of records that can be calculated (calculated) per second. In the present embodiment, "calculating a record" means, for example, when executing a join operation on two tables, one record possessed by one table and one record possessed by the other table are query conditional expressions. It is included to determine whether or not the records can be combined based on the above, and to combine the records when it is determined that the records can be combined. The calculation performance value may be another value as long as it is a value representing the calculation performance of each node.

In the example shown in FIG. 13, the calculation performance information includes the node “01” and the calculation performance value “100” in association with each other. The node "01" is, for example, identification information for identifying the distributed processing device 20. According to this calculation performance information, it is shown that the distributed processing apparatus 20 has a calculation performance capable of calculating 100 records per second.

Further, the calculation performance information includes the node "011" and the calculation performance value "10" in association with each other. The node "011" is, for example, identification information for identifying the first DB management device 31. According to this calculation performance information, it is shown that the first DB management device 31 has a calculation performance capable of calculating 10 records per second.

Further, the calculation performance information includes the node "012" and the calculation performance value "20" in association with each other. The node "012" is, for example, identification information for identifying the second DB management device 32. According to this calculation performance information, it is shown that the second DB management device 32 has a calculation performance capable of calculating 20 records per second.

Further, the calculation performance information includes the node "013" and the calculation performance value "30" in association with each other. The node "013" is, for example, identification information for identifying the third DB management device 33. According to this calculation performance information, it is shown that the third DB management device 33 has a calculation performance capable of calculating 30 records per second.

FIG. 14 shows an example of the data structure of the number of records information included in the management device information. Here, it is assumed that the tables t1 and t2 are stored in the databases 311 of the first to third DB management devices 31 to 33, respectively.

As shown in FIG. 14, the record number information includes the node ID, the table t1 and the table t2. Since the node ID is the same as the node ID included in the above-mentioned calculation performance information, detailed description thereof will be omitted here. Table t1 shows the number of records in table t1 stored in the database 311 of the node identified by the node ID. Table t2 shows the number of records held in table t2 stored in the database 311 of the node identified by the node ID.

In the example shown in FIG. 14, the record number information includes the node "011", the table t1 "100", and the table t2 "10" in association with each other. According to this record number information, it is shown that the table t1 of the first DB management device 31 identified by the node ID "011" has 100 records, and the table t2 has 10 records. ing.

Further, the record number information includes the node "012", the table t1 "10" and the table t2 "20" in association with each other. According to this record number information, it is shown that the table t1 of the second DB management device 32 identified by the node ID "011" has 10 records, and the table t2 has 20 records. ing.

Further, the record number information includes the node "013", the table t1 "25" and the table t2 "10" in association with each other. According to this record number information, it is shown that the table t1 of the third DB management device 33 identified by the node ID "013" has 25 records, and the table t2 has 10 records. ing.

FIG. 15 shows an example of the data structure of the communication performance information included in the management device information. Here, the communication performance information indicating the communication performance of the first DB management device 31 will be described.

As shown in FIG. 15, the communication performance information includes a node ID and a communication performance value in association with each other. Since the node ID is the same as the node ID included in the above-mentioned calculation performance information and record number information, detailed description thereof will be omitted here. The communication performance value is a value representing the communication performance between the first DB management device 31 and the node identified by the node ID, and indicates, for example, the number of records that can be communicated (transmitted) per second. The communication performance value may be another value as long as it is a value representing the communication performance of the first DB management device 31.

In the example shown in FIG. 15, the communication performance information includes the node "01" and the communication performance value "100" in association with each other. According to this communication performance information, it is shown that the first DB management device 31 can transmit 100 records per second to the distributed processing device 20 identified by the node "01".

Here, in the present embodiment, the first DB management device 31 has been described as being communicably connected to the distributed processing device 20, but the first DB management device 31 may be another, for example, via a mesh network or the like. It may be communicably connected to the DB management devices (second and third DB management devices 32 and 33). In this case, the communication performance information further indicates the communication performance between the first DB management device 31 and the second and third DB management devices 32 and 33.

Specifically, the communication performance information includes the node "012" and the communication performance value "20" in association with each other. According to this communication performance information, it is shown that the first DB management device 31 can transmit 20 records per second to the second DB management device 32 identified by the node "012".

Further, the communication performance information includes the node "013" and the communication performance value "25" in association with each other. According to this communication performance information, it is shown that the first DB management device 31 can transmit 25 records per second to the third DB management device 33 identified by the node "013".

In FIG. 15, the communication performance information indicating the communication performance of the first DB management device 31 has been described, but the management device information includes the communication performance indicating the communication performance of the distributed processing devices 20, the second and third DB management devices 32 and 33. Information is also included. Since the data structure of the communication performance information indicating the communication performance of the distributed processing devices 20, the second and third DB management devices 32 and 33 is the same as the communication performance information shown in FIG. 15, detailed description thereof will be omitted here. ..

In the example shown in FIG. 15, for example, the node "012" and the communication performance value "20" are associated with each other, but the first DB when the first DB management device 31 transmits a record to the second DB management device 32. The communication performance (value) of the management device 31 and the communication performance (value) of the second DB management device 32 when the second DB management device 32 transmits a record to the first DB management device 31 may be the same. However, it may differ depending on the communication environment and the like.

Although the data structures of the calculation performance information, the number of records information, and the communication performance information have been described in FIGS. 13 to 15 described above, the numerical values shown in FIGS. 13 to 15 are examples, and the numerical values are the distributed processing system 1. It may be different depending on the configuration and the like.

Next, an example of the processing procedure of the distributed processing system 1 according to the present embodiment will be described with reference to the sequence chart of FIG.

First, when browsing the table (data) stored in the databases 311 of the first to third DB management devices 31 to 33, the user acquires the table (the record possessed by) by operating the client terminal 10. Specify the query (SQL) for. In the client terminal 10 (client application), a query specified by the user (hereinafter referred to as a target query) is created (step S1).

Here, it is assumed that the above-mentioned tables t1 and t2 are stored in the databases 311 of the first to third DB management devices 31 to 33, respectively, and the target query acquires the result of the join operation between the table t1 and the table t2. ("SELECT * FROM t1, t2 WHERE t1.id = t2.id").

The target query created in step S1 is transmitted from the client terminal 10 to the distributed processing device 20 (step S2).

The target query transmitted from the client terminal 10 in step S2 is received by the communication unit 203 of the distributed processing device 20. The target query received by the communication unit 203 is analyzed by the query analysis unit 204. As a result, the query analysis unit 204 can determine the processing to be executed in the first to third DB management devices 31 to 33 based on the target query.

Here, in the present embodiment, the processing (that is, the join operation) for the target query is distributed to the first to third DB management devices 31 to 33 based on the analysis result (discrimination result) by the query analysis unit 204. In this case, the distributed processing device 20 collects one of the tables t1 and t2 from the first to third DB management devices 31 to 33 as described above, and the first table is a combination of the collected tables. It is necessary to distribute (transfer) to the third DB management devices 31 to 33.

Therefore, the query analysis unit 204 determines a table (hereinafter referred to as a collection table) to be collected from the first to third DB management devices 31 to 33 as described above (step S3).

Here, the time required for the process of collecting the tables (that is, the tables are transferred) from the first to third DB management devices 31 to 33 is reduced (that is, the operation of the distributed processing system 1 as a whole is performed at high speed). For that purpose, it is effective to collect tables having a small size (capacity) from the first to third DB management devices 31 to 33. Therefore, in step S3, among the tables t1 and t2 stored in the databases 311 of the first to third DB management devices 31 to 33, the table having a smaller size is determined as the collection table. The sizes of the tables t1 and t2 may be determined based on, for example, the management device information (information on the number of records included in the storage unit 201) stored in the storage unit 201, or are prepared separately from the management device information. It may be determined based on the information regarding the sizes of the tables t1 and t2.

Although omitted in FIG. 16, assuming that the table t2 is determined as the collection table in step S3, the distributed processing device 20 collects the table t2 from each of the first to third DB management devices 31 to 33. , Create a table T2 by combining the table t2. The table T2 created in this way is distributed to each of the first to third DB management devices 31 to 33.

Next, the query creation unit 205 calculates the processing time of the first to third DB management devices 31 to 33 based on the management device information stored in the storage unit 201 (step S4).

In this embodiment, the processing time of the first DB management device 31 means that the first DB management device 31 executes a join operation for the table t1 of the first DB management device 31 and the table T2 distributed from the distributed processing device 20. The time until the combined operation result is returned from the first DB management device 31 to the distributed processing device 20 (received by the distributed processing device 20). In other words, the processing time of the first DB management device 31 includes the time required to execute the join calculation (hereinafter referred to as the calculation time) and the result of the join calculation to be transmitted (transferred) to the distributed processing device 20. The time required (hereinafter referred to as transmission time) is included. The same applies to the processing times of the second and third DB management devices 32 and 33.

Hereinafter, the processing times of the first to third DB management devices 31 to 33 calculated in step S4 will be specifically described.

Here, the calculation (calculation) time per record is c, the transmission (transfer) time per record is t, the number of records possessed by the table t1 (the number of records possessed by the table t1) is r _t1 , and the table T2 has. Let _RT2 be the number of records (the number of records in table T2 created by collecting table t2 from all nodes).

Here, when the table t1 and the table T2 are joined (that is, the join operation for the tables t1 and T2 is executed), the combination of each of the records possessed by the table T1 and each of the records possessed by the table T2. It is necessary to perform an operation every time. Therefore, the calculation time of each of the first to third DB management devices 31 to 33 when executing the join calculation for the tables t1 and T2 is c * r _t1 * _RT2 .

On the other hand, for example, the time for transmitting the calculation results of all the records from the first to third DB management devices 31 to 33 to the distributed processing device 20 can be represented by _rt1 * _RT2 * t. However, the calculation results of all the records are not transmitted from the first to third DB management devices 31 to 33 to the distributed processing device 20, and only the combined records are transmitted as the join calculation result. Since the result of this join operation (the number of records generated as) changes depending on the conditional expression of the query, it is difficult to accurately predict the number of the records before executing the join operation. Therefore, in the present embodiment, the probability that the record matches the conditional expression of the query in the join operation for the tables t1 and T2 (that is, the record of the table t1 and the record of the table T2 are joined) is defined as p. do. In this case, the transmission time described above is r _t1 * _RT2 * p * t.

Thereby, each processing time (that is, calculation time + transmission time) of the first to third DB management devices 31 to 33 can be represented by c * r _t1 * _RT2 + r _t1 * _RT2 * p * t. ..

The above-mentioned calculation time c per record can be obtained based on the calculation performance information (calculation performance value) stored in the storage unit 201. Further, the transmission time t per record can be obtained based on the communication performance information (communication performance value) stored in the storage unit 201. Further, the number r _t1 of the records in the table t1 and the number _RT2 of the records in the table T2 can be obtained based on the record number information stored in the storage unit 201. Further, the probability p may be a value given by default, but may be, for example, a statistically calculated value or a value specified by the user.

Here, a specific example of the processing time of the first and second DB management devices 31 and 32 when executing the join operation for the tables t1 and T2 will be described with reference to FIGS. 17 and 18.

The calculation time per record, the number of records in the table t1, the number of records in the table t2, and the transmission time per record described here are exemplified for convenience of explanation. The values are different from the values based on the calculation performance information, the number of records information, and the communication performance information described in FIGS. 13 to 15 described above.

First, the processing time of the first DB management device 31 will be described with reference to FIG. In the example shown in FIG. 17, the number r _t1 of the records in the table t1 of the first DB management device 31 is 100,000, and the number of records in the table T2 distributed from the distributed processing device 20 to the first DB management device 31 _RT2 . Is assumed to be 100.

In this case, for example, the calculation time c per record of the first DB management device 31 is 0.01, the transmission time t per record is 100, and the records match the conditional expression of the query in the join operation for the tables t1 and T2. Assuming that the probability p is 0.001, the processing time of the first DB management device 31 is 0.01 (c) * 100000 ( _rt1 ) * 100 ( _RT2 ) + 100000 ( _rt1 ) * 100 ( _RT2 ). ) * 0.001 (p) * 100 (t) = 1100000.

Next, the processing time of the second DB management device 32 will be described with reference to FIG. In the example shown in FIG. 18, the number r _t1 of the records in the table t1 of the second DB management device 32 is 100, and the number of records in the table T2 distributed from the distributed processing device 20 to the second DB management device 32 _RT2 . Is assumed to be 100.

In this case, for example, the calculation time c per record of the second DB management device 32 is 0.01, the transmission time t per record is 100, and the records match the conditional expression of the query in the join operation for the tables t1 and T2. Assuming that the probability p is 0.001, the processing time of the second DB management device 32 is 0.01 (c) * 100 ( _rt1 ) * 100 ( _RT2 ) + 100 ( _rt1 ) * 100 ( _RT2 ). ) * 0.001 (p) * 100 (t) = 1100.

Although the processing time of the first and second DB management devices 31 and 32 has been described here, the processing time of the third DB management device 33 is also calculated in the same manner in step S4.

When the processing of step S4 is executed, the query creation unit 205 creates an execution plan of the target query based on the processing time of the first to third DB management devices 31 to 33 calculated in the step S4, and executes the execution. A query for the first to third DB management devices 31 to 33 according to the plan (a query to be executed by each of the first to third DB management devices 31 to 33) is created (step S5).

Here, it is assumed that the processing time of the first DB management device 31 calculated in step S4 is 1100000 and the processing time of the second DB management device 32 is 1100. In this case, when the processing for the target query is executed as in the comparative example of the present embodiment described above, even after the processing of the second DB management device 32 is completed, the target is until the processing of the first DB management device 31 is completed. The response to the query (that is, the result of the join operation) cannot be returned to the client terminal 10, and the operation of the distributed processing system 1 becomes slow.

Therefore, for example, the processing time of the first DB management device 31 is the maximum, and the processing time of the second DB management device 32 is the minimum (that is, the processing time of the second DB management device 32 is less than the processing time of the first DB management device 31). If there is), an execution plan is created in which records are transferred from the first DB management device 31 (hereinafter referred to as the maximum cost node) to the second DB management device 32 (that is, referred to as the minimum cost node).

In the present embodiment, based on the execution plan created in this way, the processing time is equalized among the nodes (here, the first and second DB management devices 31 and 32).

The number of records transferred from the maximum cost node to the minimum cost node can be, for example, the minimum x satisfying the following equation (1).
c * (r1 _t1 -x) * r1 _T2 + (r1 _t1 -x) * r1 _T2 * p * t + x * t
> C * (r2 _t1 + x) * r2 _T2 + (r2 _t1 + x) * r2 _T2 * p * t + x * t
Equation (1)

Here, in the above equation (1), r1 _t1 represents the number of records held in the table t1 of the maximum cost node, and r2 _t1 represents the number of records held in the table t1 of the minimum cost node. Further, r1 _T2 represents the number of records held by the table T2 distributed from the distributed processing device 20 to the maximum cost node, and r2 _T2 represents the number of records held by the table T2 distributed from the distributed processing device 20 to the minimum cost node. Represents a number. That is, r1 _T2 and r2 _T2 have the same value. Note that c and t on the left side are the calculation time and transmission time per record of the maximum cost node. C and t on the right side are the calculation time and the transmission time per record of the minimum cost node. p is the probability that the record matches the conditional expression of the query in the join operation for the tables t1 and T2.

According to the above equation (1), the left side represents the processing time of the maximum cost node when x records are transferred from the maximum cost node to the minimum cost node, and the right side represents the processing time of the maximum cost node from the maximum cost node to the minimum cost node. It represents the processing time of the minimum cost node when x records are transferred.

In this embodiment, the minimum x is calculated using such an equation (1). The x calculated in this way corresponds to the number of records that need to be transferred in order for the maximum cost node and the minimum cost node to have the same processing time.

For example, assuming that the maximum cost node is the first DB management device 31 and the minimum cost node is the second DB management device 32, the example described in FIGS. 17 and 18 described above is applied to the equation (1). , X = 39996 can be calculated. According to this, an execution plan of transferring 39996 records from the first DB management device 31 to the second DB management device 32 is created.

In this case, the query creation unit 205 creates, for example, "SELECT * FROM (SELECT * FROM LIMITED 60004), t2 WHERE t1.id = t2.id" as a query to the first DB management device 31. This query indicates that the 1st to 60004th records among the 100,000 records possessed by the table t1 of the first DB management device 31 are to be combined with the table T2 (records possessed by the table T2).

On the other hand, as a query to the second DB management device 32, the query creation unit 205 may use, for example, "GET 1st DB management device. T1's reserve (6005-10000)" and "SELECT * FROM t1, t2 WHERE t1.id = t2. Create "id". This query acquires the 60015th to 1000000th records (that is, 39996 records) from the 100,000 records held in the table t1 of the first DB management device 31 from the first DB management device 31, and manages the second DB. It is shown that the record possessed by the table t1 of the apparatus 32 and the record acquired from the first DB management apparatus 31 are combined with the table T2 (the record possessed by the table T2).

If the third DB management device 33 does not transfer the record to another DB management device or the record is not transferred from the other DB management device, the query for the third DB management device 33 is "SELECT * FROM t1, t2 WHERE". "t1.id = t2.id" is created.

Here, the number of records transferred from the first DB management device 31 (maximum cost node) to the second DB management device 32 (minimum cost node) has been described as being calculated using the above equation (1). The number of records to be transferred may be calculated by another method.

Further, in the present embodiment, the record is transferred from the maximum cost node to the minimum cost node, but the node that is the transfer source of the record and the node that is the transfer destination of the record may be determined by another method. good.

Further, for example, the record may be transferred between a plurality of nodes so that the processing times of all the nodes are about the same.

When the process of step S5 is executed, a query for each of the first to third DB management devices 31 to 33 created in step S5 is transmitted to each of the first DB management devices 31 to 33 (step S6). ).

Here, when the query transmitted from the distributed processing device 20 (query execution unit 206) is received by the first DB management device 31 (communication unit 312) in step S6, the query execution unit 313 of the first DB management device 31 , Executes the process for the query.

Here, assuming that the query to the first DB management device 31 is "SELECT * FROM (SELECT * FROM LIMIT 60004), t2 WHERE t1.id = t2.id" as described above, the query of the first DB management device 31 The execution unit 313 refers to the database 311 of the first DB management device 31, and executes a join operation for joining the table t1 having the first to 6004th records of the first DB management device 31 and the table T2 (step). S7).

The result of executing the process of step S7 (combination calculation result) is transmitted from the first DB management device 31 to the distributed processing device 20 via the communication unit 312 of the first DB management device 31 (step S8).

On the other hand, when the query transmitted from the distributed processing device 20 (query execution unit 206) is received by the second DB management device 32 (communication unit 312) in step S6, the query execution unit 313 of the second DB management device 32 receives the query. Execute the process for the query.

Here, as described above, the query to the second DB management device 32 is "GET 1st DB management device 31. t1's reserve (6005-10000)" and "SELECT * FROM t1, t2 WHERE t1.id = t2.id". Assuming that, the query execution unit 313 of the second DB management device 32 cooperates with the communication unit 312 and manages the first DB based on "GET 1st DB management device 31.t1's reserve (6005-10000)". Communication with the device 31 (communication unit 312) is executed (step S9). Communication between the first DB management device 31 and the second DB management device 32 is executed, for example, via a mesh network constructed by the first to third DB management devices 31 to 33.

When the process of step S9 is executed, the 6005-th to 100000th records (hereinafter referred to as target records) in the table t1 of the first DB management device 31 are transferred from the first DB management device 31 to the second DB management device 32. Will be done. As a result, the query execution unit 313 of the second DB management device 32 acquires the target record via the communication unit 312 (step S10).

Next, the query execution unit 313 of the second DB management device 32 refers to the database 311 of the second DB management device 32, and sets the target record acquired from the table t1 of the second DB management device 32 and the first DB management device 31. A join operation for joining the table in which the tables are combined and the table T2 is executed (step S11).

The result of executing the process of step S11 (combination calculation result) is transmitted from the second DB management device 32 to the distributed processing device 20 via the communication unit 312 of the second DB management device 32 (step S12).

Further, when the query transmitted from the distributed processing device 20 (query execution unit 206) is received by the third DB management device 33 (communication unit 312) in step S6, the query execution unit 313 of the third DB management device 33 receives the query. Execute the process for the query.

Here, assuming that the query to the third DB management device 33 is "SELECT * FROM t1, t2 WHERE t1.id = t2.id" as described above, the query execution unit 313 of the third DB management device 33 is concerned. With reference to the database 311 of the third DB management device 33, a join operation for joining the table t1 and the table T2 of the third DB management device 33 is executed (step S13).

The result of executing the process of step S13 (combination calculation result) is transmitted from the third DB management device 33 to the distributed processing device 20 via the communication unit 312 of the third DB management device 33 (step S14).

The join calculation result transmitted from the first DB management device 31 in step S8, the join calculation result transmitted from the second DB management device 32 in step S12, and the join calculation result transmitted from the third DB management device 33 in step S14 are , Are integrated by the distributed processing apparatus 20 (result accumulation unit 207) (step S15).

The join calculation result accumulated in step S15 is a response to the target query transmitted from the client terminal 10 to the distributed processing device 20 in step S2, from the distributed processing device 20 (communication unit 203) to the client terminal 10 (client application program). Is transmitted to (step S16).

The combination calculation result transmitted from the distributed processing apparatus 20 in step S16 is received by the client terminal 10 and displayed on the client terminal 10 (display or the like) (step S17). As a result, the user can view the result of the join operation.

In addition, in FIG. 16, a part (target record) of a plurality of records possessed by the table t1 of the first DB management device 31 is transferred to the second DB management device 32 (that is, the second DB management device 32 transfers the target record to the first DB. Although it has been described as (acquired from the management device 31), a part of the target record may be transferred to the second DB management device 32, and the remaining part of the target record may be transferred to the third DB management device 33. ..

Further, depending on the processing time calculated in step S4, for example, a part (target record) of a plurality of records included in the table t1 of the second DB management device 32 is transferred to the first DB management device 31 (or the third DB management device 33). It doesn't matter if it is done.

As described above, in the present embodiment, the distributed processing system 1 is described as including the first to third DB management devices 31 to 33, but the distributed processing system 1 according to the present embodiment includes at least two DB management devices. Any configuration may be provided. Further, the number of DB management devices provided in the distributed processing system 1 may be 4 or more.

Hereinafter, the operation and effect of the present embodiment will be described mainly with respect to the configuration in which the distributed processing system 1 includes two DB management devices (first and second DB management devices 31 and 32).

In the present embodiment, the database 311 of the first DB management device 31 stores the table t1 (first table) having a plurality of records (first records) collected in the first DB management device 31, and the second DB management device 32. Database 311 stores a table t1 (second table) having a plurality of records (second records) collected in the second DB management device 32. The record of the table t1 of the first DB management device 31 and the record of the table t1 of the second DB management device 32 are composed of the same fields.

Further, in the present embodiment, the distributed processing apparatus 20 has arithmetic performance information indicating the arithmetic performance of each of the first and second DB management devices 31 and 32, record number information indicating the number of records in the table t1, or distributed processing. A storage unit 201 for storing management device information including communication performance information indicating communication performance (first communication performance) between the device 20 and the first and second DB management devices 31 and 32 is included. Further, the distributed processing device 20 receives a query (first query) for acquiring the record of the table t1 from the client terminal 10, and the processing time (first processing time) of the first DB management device 31 for the query and the processing time (first processing time). The processing time (second processing time) of the second DB management device 32 is calculated, and the query to the first DB management device 31 (second query) and the query to the second DB management device 32 (third) based on the calculated processing time. Create a query).

Further, in the present embodiment, the first DB management device 31 executes the processing for the query created in the distributed processing device 20, and the second DB management device 32 executes the processing for the query created in the distributed processing device 20. When the processing time of the first DB management device 31 is less than the processing time of the second DB management device 32, the first DB management device 31 uses a part of the plurality of records in the table t1 of the second DB management device 32 as the first DB. It is acquired from the management device 31 (database 311) and the processing for the query is executed. On the other hand, when the processing time of the second DB management device 32 is less than the processing time of the first DB management device 31, the second DB management device 32 uses a part of the plurality of records in the table t1 of the first DB management device 31 as the first DB. It is acquired from the management device 31 (database 311) and the processing for the query is executed.

In the present embodiment, since it is possible to equalize the processing time between the first DB management devices 31 and 32 by such a configuration, it is possible to speed up the processing for the query in the distributed processing system 1 as a whole. It will be possible.

Here, for example, there is a system in which a load balancer is arranged to distribute (distribute) the load to a plurality of child nodes (child nodes A and B), and the load balancer is as shown on the left side of FIG. , It is assumed that the data is shared by the child nodes A and B. That is, the load balancer decentralizes the processing by changing the position where the data of each of the child nodes A and B is read, or by distributing the shared data to the child nodes A and B in advance. ..

On the other hand, in the distributed processing system 1 (integrated database) according to the present embodiment, as shown on the right side of FIG. 19, the child nodes A and B (for example, the first and second DB management devices 31 and 32) Since the processing for the query is executed for different data (tables) managed (possessed) by each, the data cannot be distributed in advance like the load balancer described above.

Therefore, in the present embodiment, by adopting a configuration that distributes and optimizes the processing of the node (for example, the first DB management device 31) whose processing amount is expected to be biased when executing the processing for the query. The operation of the entire distributed processing system 1 can be speeded up.

In the present embodiment, the management device information stored in the storage unit 201 has been described as including all of the calculation performance information, the number of records information, and the communication performance information, but the management device information is the first and second DBs. The information may be any information used for calculating the processing time of the management devices 31 and 32, and may include, for example, at least one of calculation performance information, record number information, and communication performance information. This management device information may be collected from, for example, the first and second DB management devices 31 and 32 and updated periodically.

Further, for example, the database 311 of the first DB management device 31 further stores the table t2 (third table), and the database 311 of the second DB management device 32 further stores the table t2 (fourth table). The record of the table t2 of 31 and the record of the table t2 of the second DB management device 32 are composed of the same fields. Further, in the query transmitted from the client terminal 10 to the distributed processing apparatus 20 in the present embodiment, the result of the join operation between the table t1 (first and second tables) and the table t2 (third and fourth tables) is acquired. Includes a query for.

In this case, the first and second DB management devices 31 and 32 execute the processing for the query by using the table T2 which is a combination of the tables t2 of the first and second DB management devices 31 and 32, but the first and second DB management devices 31 and 32 are managed. The capacity of the table t1 of the devices 31 and 32 is larger than the capacity of the table T2. According to this, the table t1 is collected from the first and second DB management devices 31 and 32, and the table in which the collected tables t1 are combined is distributed to the first and second DB management devices 31 and 32. As a result, the amount of communication between the distributed processing device 20 and the first and second DB management devices 31 and 32 can be reduced, so that high-speed processing can be realized.

Here, in the present embodiment, for example, when the processing time of the second DB management device 32 is less than the processing time of the first DB management device 31, the second DB management device 32 acquires the target record from the first DB management device 31. However, for example, when the difference between the processing time of the first DB management device 31 and the processing time of the second DB management device 32 is small, a small number of records are the target records from the first DB management device 31 to the second DB management device 32. Will be transferred to. In general, the transmission (transfer) process takes more time than the arithmetic process. Therefore, if only a small number of records are transferred from the first DB management device 31 to the second DB management device 32, the process of the first DB management device 31 is performed. The amount cannot be sufficiently reduced, and the time required for transferring the record may be a factor of slowing down the processing.

Therefore, in the present embodiment, when the difference between the processing time of the first DB management device 31 and the processing time of the second DB management device 32 is equal to or more than a predetermined value, the second DB management device 32 manages the first DB. The target record may be acquired from the device 31. In other words, if the difference between the processing time of the first DB management device 31 and the processing time of the second DB management device 32 is small, the target record may not be transferred. In this case, the join operation described with reference to FIGS. 7 and 8 may be executed in each of the first and second DB management devices 31 and 32.

Here, the case where the second DB management device 32 acquires the record from the first DB management device 31 has been described, but the same applies to the case where the first DB management device 31 acquires the record from the second DB management device 32.

Further, in the present embodiment, as described in FIG. 16, for example, the second DB management device 32 is mainly described as acquiring the target record directly from the first DB management device 31, but the target record is distributed processing. The configuration may be acquired via the device 20.

As described with reference to FIG. 15, the management device information (communication performance information) stored in the storage unit 201 of the distributed processing device 20 includes the distributed processing device 20 and the first and second DB management devices 31 and 32. When the communication performance between the first and second DB management devices 31 and 32 (second communication performance) is included in addition to the communication performance between the two, in the present embodiment, a record is acquired based on the communication performance information. It may be configured to determine the route to be performed.

Here, for example, assuming that the second DB management device 32 acquires a part of a plurality of records included in the table t1 of the first DB management device 31 as the target record, the first DB management device is used as a route for acquiring the target record. A first path directly from 31 and a second path acquired via the distributed processing apparatus 20 are included.

For example, as shown in FIG. 20, the communication performance (value) between the distributed processing device 20 and the first and second DB management devices 31 and 32 is 100, and the communication performance (value) between the first and second DB management devices 31 and 32 ( Assuming that the value) is 20, in the distributed processing device 20 (query creation unit 205), the first path acquired via the distributed processing device 20 is the second path directly acquired from the first DB management device 31. It can be determined that the target record can be acquired at a higher speed than that.

As described above, for example, when low-speed communication is executed between the first and second DB management devices 31 and 32, the query creation unit 205 determines the first route as the route for acquiring the record, and the first route is determined. It is possible to create an execution plan to acquire the target record in the route. According to this, when the second DB management device 32 executes the process (join operation) for the query created in the distributed processing device 20, the target record is set in the first path (that is, via the distributed processing device 20). ) Can be obtained.

On the other hand, for example, depending on the environment in which the distributed processing device 20 is arranged, the communication performance between the distributed processing device 20 and the first and second DB management devices 31 and 32 is more important than the first and second DB management devices 31 and The communication performance between 32 may be higher. In such a case, the query creation unit 205 may determine the second route as the route for acquiring the record, and create an execution plan for acquiring the record in the second route. According to this, when the second DB management device 32 executes the process (join operation) for the query created in the distributed processing device 20, the target record is set in the second path (that is, the first DB via the mesh network). It can be obtained (directly from the management device 31).

The first to third DB management devices 31 to 33 are connected to each other so as to be communicable with each other via, for example, a mesh network, but the first to third DB management devices 31 to 33 may not always be connected to each other. In such a case, for example, in order for the second DB management device 32 to directly acquire the target record from the first DB management device 31, the second DB management device 32 uses the connection information to the first DB management device 31. It is necessary to connect to the first DB management device 31. The connection information to the first DB management device 31 is information for communicating with the first DB management device 31 (connecting to the first DB management device 31), and includes, for example, an IP address and a password. For example, when the third DB management device 33 directly acquires the target record from the first DB management device 31, connection information to the first DB management device 31 is also required. The same applies to between other DB management devices.

Here, for example, when the distributed processing system 1 is configured to include a large number of DB management devices, it is difficult for one DB management device to hold connection information to all other DB management devices (all sibling nodes). Is. Therefore, for example, when the second DB management device 32 acquires the target record from the first DB management device 31, when the second route is determined as the route for acquiring the target record, the second route is determined as shown in FIG. 21. It is assumed that an execution plan including connection information to the first DB management device 31 is created, and the connection information is transmitted from the distributed processing device 20 to the second DB management device 32. According to this, even if the second DB management device 32 does not hold the connection information to the first DB management device 31 (including all the DB management devices) in advance, the connection information transmitted from the distributed processing device 20 is used. The target record can be directly acquired from the first DB management device 31. In this case, the connection information to the plurality of DB management devices (first to third DB management devices 31 to 33) provided in the distributed processing system 1 may be managed by, for example, the distributed processing device 20 (storage unit 201). ..

Here, it has been described that the connection information to the first DB management device 31 is transmitted from the distributed processing device 20 to the second DB management device 32, but the connection information is the second DB management device as shown in FIG. 22, for example. 32 may be acquired by accessing the distributed processing device 20.

Further, in the present embodiment, it has been described that the distributed processing system 1 includes the first to third DB management devices 31 to 33 that manage the databases storing the tables t1 and t2. For example, the third DB management device 33 is a table. It may be a DB management device that manages a database (third database) in which t1 and t2 are not stored. In the present embodiment, in the case of such a configuration, for example, the third DB management device 33 acquires the target record from the first DB management device 31 instead of the second DB management device 32, and the third DB management device 33 obtains the target record. The configuration may be such that the join operation (that is, the processing for the fourth query created in the distributed processing device 20) for the table t1 having the target record and the table T2 distributed from the distributed processing device 20 is executed. That is, in the present embodiment, the processing for the query may be distributed to the DB management devices (child nodes) that do not manage the data (tables t1 and t2) related to the query specified by the user.

Further, in the present embodiment, the processing (join operation) for the query is mainly described as being distributed to a plurality of DB management devices, but for example, the sizes of the tables t1 and t2 of each DB management device are small (the number of records is small). If the number is small) or if the calculation performance of the distributed processing device 20 is sufficiently higher than that of the plurality of DB management devices, the distributed processing device 20 may be configured to execute all the join operations. In this case, the processing as described with reference to FIG. 6 may be executed in the distributed processing apparatus 20.

Whether or not to execute all the join operations on the distributed processing apparatus 20 side is determined by calculating the processing time required when all the join operations are executed on the distributed processing apparatus 20 side, and the processing time and, for example, the figure. It may be determined by comparing with the processing time of each DB management device calculated in step S4 shown in 16.

Further, in the present embodiment, for example, based on the processing time of each DB management device, the processing for the query is distributed only to some DB management devices among the plurality of DB management devices, and the remaining processing is distributed processing device. It is also possible to configure it so that it is executed on the 20 side.

Although some embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other embodiments, and various omissions, replacements, and changes can be made without departing from the gist of the invention. These embodiments and variations thereof are included in the scope of the invention described in the claims and the equivalent scope thereof, as are included in the scope and gist of the invention.

10 ... client terminal, 20 ... distributed processing device, 31 ... first DB management device (first database management device), 32 ... second DB management device (second database management device), 33 ... third DB management device (third database management) Device), 21 ... CPU, 22 ... Non-volatile memory, 23 ... Main memory, 24 ... Communication device, 201 ... Storage unit, 202 ... Information management unit, 203 ... Communication unit, 204 ... Query analysis unit, 205 ... Query creation unit , 206 ... Query execution unit, 207 ... Result accumulation unit, 311 ... Database, 312 ... Communication unit, 313 ... Query execution unit, 314 ... Information management department.

Claims (14)

A distributed processing device including a first database management device that manages a first database communicably connected to the distributed processing device, and a second database management device that manages a second database different from the first database. In the processing system
The first database stores a first table having a plurality of first records collected in the first database management device.
The second database stores a second table having a plurality of second records collected in the second database management device.
The first and second records are composed of the same fields.
The distributed processing device is
The computing performance of each of the first and second database management devices, the number of first records in the first table and the number of second records in the second table, or the distributed processing device and the first and first. 2 A storage means for storing management device information including the first communication performance with the database management device, and
A receiving means for receiving the first query for acquiring the first and second records of the first and second tables, and
A calculation means for calculating the first processing time of the first database management device and the second processing time of the second database management device for the received first query based on the management device information.
Including a creation means for creating second and third queries based on the calculated first and second processing times.
The first database management device includes a first query execution means for executing a process for the created second query.
The second database management device includes a second query execution means for executing a process for the created third query.
When the calculated first processing time is less than the calculated second processing time, the first query executing means may obtain a part of a plurality of second records of the second table from the second database. Acquire and execute the process for the second query,
When the calculated second processing time is less than the calculated first processing time, the second query executing means may obtain a part of a plurality of first records of the first table from the first database. A distributed processing system that acquires and executes processing for the third query. The distributed processing system according to claim 1, wherein the storage means stores management device information collected from the first and second database management devices. The distributed processing system according to claim 2, wherein the management device information is periodically updated. The first database further stores a third table different from the first table.
The second database further stores a fourth table different from the second table.
The records of each of the third and fourth tables are composed of the same fields.
The distributed processing system according to any one of claims 1 to 3, wherein the first query includes a query for acquiring a join operation result between the first and second tables and the third and fourth tables. .. The distributed processing system according to claim 4, wherein the capacities of the first and second tables are larger than the capacities of the third and fourth tables. The first query execution means executes a process for the second query using a table in which the third table and the fourth table stored in the second database are combined.
The distributed process according to claim 4 or 5, wherein the second query execution means executes a process for the third query using a table in which the third table and the fourth table stored in the first database are combined. system. The first query execution means is when the first processing time is less than the second processing time and the difference between the first processing time and the second processing time is equal to or more than a predetermined value. , A part of the plurality of second records of the second table is acquired from the second database.
In the second query execution means, when the second processing time is less than the first processing time and the difference between the first processing time and the second processing time is equal to or more than the predetermined value. The distributed processing system according to any one of claims 1 to 6, wherein a part of a plurality of second records of the first table is acquired from the first database. The management device information stored in the storage means further includes a second communication performance between the first database management device and the second database management device.
The distributed processing device determines a route for acquiring a part of the plurality of first records or a part of the plurality of second records based on the first and second communication performance included in the management device information. The distributed processing system according to any one of claims 1 to 7, further comprising a determination means. The route is one of the first route for directly acquiring a part of the plurality of first records or a part of the plurality of second records from the first and second database management devices, and one of the plurality of first records. The distributed processing system according to claim 8, further comprising a second route for acquiring a unit or a part of the plurality of second records via the distributed processing apparatus. The distributed processing according to claim 9, wherein the distributed processing device further includes a transmission means for transmitting connection information for executing communication between the first and second database management devices to the first or second database management device. system. Further provided with a third database management device for managing a third database in which a table having a record composed of the same fields as the first and second records is not stored.
The third database management device is created in the distributed processing device by acquiring a part of the plurality of first records or a part of the plurality of second records in place of the first or second database management device. The distributed processing system according to claim 1, which includes a third query execution means for executing processing for the fourth query. A distributed processing apparatus used in the distributed processing system according to claim 1. A database management device used in the distributed processing system according to claim 1. A distributed processing device including a storage means for storing management device information, a first database management device for managing a first database for storing a first table having a plurality of first records, and the same field as the plurality of first records. It is a method executed by a distributed processing system including a second database management device that manages a second database that stores a second table having a plurality of second records composed of.
The step of receiving the first query for acquiring the first and second records of the first and second tables, and
A step of calculating the first processing time of the first database management device and the second processing time of the second database management device for the received first query based on the management device information.
The step of creating the second and third queries based on the calculated first and second processing times, and
The step to execute the process for the created second query and
It is provided with a step of executing a process for the created third query.
The management device information is the calculation performance of each of the first and second database management devices, the number of first records of the first table and the number of second records of the second table, or the distributed processing device. Includes first communication performance between the first and second database management devices.
When the calculated first processing time is less than the calculated second processing time, the step of executing the processing for the second query includes a part of the plurality of second records of the second table. Including the step of acquiring from the second database and executing the process for the second query.
In the step of executing the process for the third query, when the calculated second processing time is less than the calculated first processing time, a part of the plurality of first records of the first table is described. A method comprising a step of retrieving from a first database and executing processing for the third query.

Images