JPH10289187A - Parallel data transfer method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To accelerate data transfer between a client and a server when one or more servers hold a part or all of data required by the client and are connected by the communication paths respectively different in communication cost. SOLUTION: The client 1000 and the server 1002, the server 1004 and the server 1006 are respectively connected by communication channels and are capable of communication respectively by the communication path 1100, the communication path 1102 and the communication path 1104. Also, the respective servers hold a part or all of the same data respectively as the data 1202, the data 1204 and the data 1206. At the time, when the client 1000 issues a request for obtaining the data held by the servers as the data 1200, the communication costs of the communication paths between the respective servers and the client are taken into consideration, the data are divided and a part each of the data is respectively parallelly gathered from the respective servers, combined and obtained as the result of the request.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention generally relates to a computer system, and more particularly, to a client computer that generates a data acquisition request or a storage request on a computer network having uneven communication costs, and processes the request. The present invention relates to a method for transferring data between server computers.

[0002]

2. Description of the Related Art In a conventional client server system, one server holds master data for one data, and some other servers have a replica of the data as a replica or a cache. Then, when the client needs the data, it issues a data transfer request to the server having the master data or the server having the copy, and receives all the necessary data from the server using one communication path (Maekawa Mamoru, Mario Tokoro, Kentaro Shimizu: "Distributed Operating System-
What comes after UNIX-", Kyoritsu Publishing, 1.3 Why distributed processing?).

On the other hand, in a disk array or a striping file system, data is divided into a plurality of blocks.
When each block is stored on a separate disk and data is read, the speed of input / output is increased by reading the stored data in parallel (Mori Maekawa, Mario Tokoro, Kentaro Shimizu: "Distributed Operating System-UNIX"Next,-", Kyoritsu Shuppan, 6.3.1 File Server Implementation Technology).

[0004]

In the method of selecting one from a plurality of servers and acquiring all necessary data from the server, the method of selecting one from a plurality of servers, despite the fact that there are other servers having the necessary data, is also difficult. Since data is serially transferred from one server using one communication path, the time required for the transfer processing is limited by the amount of data and the speed of the communication path to be used, and further higher speed cannot be expected.

In the above-described disk array and striping file system, data transfer is speeded up by dividing data and performing input / output in parallel. However, in a network in which actual computers are connected, communication is not performed. Since the communication speed varies among the roads, even if the above technology is applied as it is, both the low-speed line and the high-speed line will transfer the same amount of data, so that efficient data transfer cannot always be performed. .

An object of the present invention is to realize data transfer with higher parallelism on a computer network where actual communication costs are not uniform, and to speed up data transfer between a client and a server. .

[0007]

According to the parallel data transfer method of the present invention, data is transferred in parallel by requesting a part of the data to two or more servers holding the data requested by the client. This realizes high-speed data transfer between the client and the server.

Further, by determining the range of data to be transferred to each server and the number of simultaneous requests to the server in consideration of the cost of the communication path between the server and the client, data transfer with higher parallelism can be performed. To reduce the data transfer processing time.

[0009]

FIG. 1 shows an example of a system configuration.

[0010] Client 1000 and server 1002,
The server 1004 and the server 1006 are connected by a communication line, respectively, and a communication path 1100 and a communication path 1102
And a communication path 1104. Further, the server holds some or all of the same data 1002, data 1004, and data 1006.

When the client 1000 obtains from the server group the same data 1200 as the data 1202, 1204, and 1206 managed by the server, the client 1000 uses the communication path 1100, the communication path 1102, and the communication path 1104. In consideration of the communication cost of the data transfer at the time of
, Data 1214 and data 1215 from the server 1004, and data 1216 from the server 1006, respectively. The data 1214 and data 1215 acquired from the server 1004 are
2 are further multiplexed and data is transferred in parallel.

FIG. 2 shows the configuration of the data acquisition request processing section in the client 1000 shown in FIG.

The request receiving unit 2000 receives the request for acquiring the data 1200 in FIG. 1 and passes it to the data maintenance information collecting unit 2006. The data maintenance information collection unit 2006
Communicates with the server 1002, the server 1004, and the server 1006 in FIG. 1 using the server communication unit 2010, and collects the holding information of the requested data held by each server, and communicates with the server. It is recorded as server data and data management information 2008 together with the communication cost required for communication. Further, the request destination server and the simultaneous request number determination unit 2004 request which part of the requested data among the servers to which server based on the server data and the data management information 2008, and how many requests are made at the same time. Then, it is determined whether to request the server. Then, the data collecting and combining unit 2002 transmits the server 10 through the server communication unit 2010 according to the determination.
02, while collecting and combining requested data while communicating with the server 1004 and the server 1006, the request receiving unit 20
Return to 00.

FIG. 3 shows an example of information held in the server data and the data management information 2008 in FIG.

The table in FIG. 3 exists one for each data.
One line is an entry. That is, row 3000,
Rows 3002, 3004, and 3006 are entries. Column 3100 is the server identifier of the entry, and column 3012 is information indicating which part of the data the server indicated by the identifier holds. A column 3104 is a value indicating a communication cost when the server and the client indicated by the server identifier communicate with each other. Here, the round trip time of the packet between the client and the server is used as the communication cost, but in addition, the line speed of the communication line, the cost of data communication, the average communication speed based on past communication records, and the administrator determine A fixed value can be used.

FIG. 4 shows a flow of processing of the data reception request in FIG.

Data 1200 of the client shown in FIG.
When a read request is generated (4002), the server holding the data and the range of data held by the server are collected by 2006 shown in FIG. 2 (4004), and based on the result, the server issuing the transfer request and the server The request destination server and the simultaneous request number determination unit 2 in FIG.
004 (4006), and based on the determination, FIG.
In the data collection and combination unit 2002, the data transfer request is transmitted to each server in parallel (4008), and the data is received (4010) until all the data are collected (4010).
12) Combining the collected results (4014),
002 is the result of the data read request generated. If the server holding the data and the range of the data held by the server are collected 4004, the process is not particularly performed if the server holding the data or the range of the data held is known in advance.

FIG. 5 shows the flow of the processing 4006 in FIG. 4 for determining the data transfer request and the number of simultaneous requests to the server.

First, the data to be collected is divided into one or more ranges (5002), and the acquisition destination servers of all ranges are determined (5004).
The server that has the lowest communication cost among the servers that manage the range is selected as the request destination server (5008). Once the acquisition destination servers of all ranges are determined, the next time the number of simultaneous requests of all ranges is determined (50
10) For the entire range (5012), the range has a data amount that is twice or more as large as the other range (5014), or the range is obtained as compared with the case of obtaining data in another range. More than twice the communication cost (501)
6) For the server, when issuing a transfer request in the range to the server, the number of requests is increased by dividing the request (5018).

FIG. 6 shows a processing flow of a method 4006 in FIG. 4 which is different from the method shown in FIG. 5 of the processing 4006 for determining the number of data transfer requests and the number of simultaneous requests to the server.

First, the range of the data for which the transfer request has been issued is divided into one or more ranges (6002), and the server holding the data is the most until the server of the request transfer destination in all ranges is determined (6004). A small range is selected to be a request destination server of the data in the range (6006), and a server which is least costly to transfer all the allocated ranges except the range is selected and a request destination of the range is selected. (6008).

FIG. 7 shows a process flow of a server 400 holding target data and a process 4004 for collecting a range of data held by the server in FIG.

First, a request for obtaining information of holding the target data is transmitted to a server that is thought to hold the data to be obtained (7002), and until all the replies of the request are returned (7004), each received reply is received. About (700
6), the time taken from when the request was sent to when the reply was returned (7008), and the holding information of the data in the reply is taken out (7010), and the server data and the data management information shown in FIG. The elapsed time is stored in the column 3104 of FIG. 3 as the communication cost, and the held information is stored in the column 3102 of FIG. 3 as the range of the server managed by each server (7012).

In FIG. 1, data 1200 stored in a client 1000 is stored in a server 1002 and a server 1004.
And data 1202 and data 120 of the server 1006
FIG. 8 shows the flow of the process when writing the data No. 4 and the data 1206.

When a request to save data 1200 is issued by the client 1000 in FIG. 1 (8000), the data range is divided in consideration of the communication cost required for communication with each server (8002), and the server for storing the data is divided. Are transferred in parallel to each other (8004), and the process waits until the transfer process is completed (8006). If an error has occurred during the transfer process (8008), the server other than the server in which the error occurred during the transfer is re-processed from the process 8002 for determining the storage destination only for the data for which the transfer failed.

FIG. 9 shows how data is transferred in parallel using two or more communication paths when a client and a server communicate.

In FIG. 9, a client 9100 is a server 9
The communication path 102 is connected to a communication path 9200, a communication path 9202, and a communication path 9204. At this time, the server 9102
When the data 9002 in the data path 900 is transferred to the client 9100 as data 9000, the communication cost of the communication path 9200, the communication path 9202, and the communication path 9204 is considered instead of transferring the entire data 9002 in one of the communication paths. Then, the data 9002 is divided into data 9004, data 9006, and data 9008,
002 is a communication path 9200, and data 9006 is a communication path 9
At 202, the data 9008 is transferred in parallel over the communication path 9204, and the client 9100 combines the transferred data to form data 9000.

[0028]

According to the present invention, data transfer between a client and a server can be efficiently parallelized and a high-speed transfer process can be performed in a computer network having an uneven communication speed.

[Brief description of the drawings]

FIG. 1 is a system configuration.

FIG. 2 shows a configuration of a client data acquisition request processing unit.

FIG. 3 shows server data and data management information.

FIG. 4 is a flowchart of a process performed by a data acquisition request processing unit.

FIG. 5 is a flowchart 1 of a data transfer request and a process of determining the number of simultaneous requests to the server.

FIG. 6 shows a flow 2 of a data transfer request and a process of determining the number of simultaneous requests to the server.

FIG. 7 shows a data management server and a retained data range information collection method.

FIG. 8 is a flow of a process at the time of data storage request processing.

FIG. 9 shows parallel data transfer using a multiplex communication path.

[Explanation of symbols]

1000: client, 1100: communication channel,
1200 data.

Claims (6)

[Claims] 1. A server which is a computer for storing and reading data in response to a request, and a client which is a computer for sending a request for storing and reading data to and from the server. Each client and one or more servers in the server group are connected by a communication path, respectively, and the client and the server are simultaneously connected on the communication path by one or more virtual communication paths multiplexed by software. In a client-server system capable of performing communication, when one or more servers in the server group respectively hold data requested by the client, the client issues a transfer request for a part or all of the data, In consideration of the cost of the communication path between each server and the client, Parallel data transfer method of determining a virtual channel number, and transfers the range of data that the determined respectively from the respective servers in parallel to use the range of data when communicating with each server simultaneously Request over server. 2. A server, which is a computer for saving and reading data in response to a request, and one or more clients, each being a computer for sending a request to save or read data to the server, Each client and one or more servers in the server group are connected by a communication path, respectively, and the client and the server are simultaneously connected on the communication path by one or more virtual communication paths multiplexed by software. In a client-server system capable of performing communication, when one or more servers in the server group respectively hold a part or all of data requested by the client, the client requests a transfer of part or all of the data. Issue, each server holds which part of the data Or information obtained from the server computer, among the servers holding a part of the requested data in consideration of the cost of the communication path between each server and the client, And a number of virtual communication paths to be used simultaneously at the time of communication for each server, and a parallel data transfer method for transferring the determined range of data from each server in parallel. 3. The parallel data transfer method according to claim 1, wherein the data requested by the client is divided into one or more ranges, and the communication cost is the lowest among a group of servers holding the range of the data. The process of selecting a server that does not cost is repeated for the entire range of the data, and among the servers, a virtual communication path that is used simultaneously for servers that have a wide range of data to be transferred or that are expected to take a long time for data transfer A parallel data transfer method for increasing the number of data to determine the range of the data requested from each server and the number of virtual communication paths to be used simultaneously during communication for each server. 4. The parallel data transfer method according to claim 1, wherein the data is divided into one or more ranges, and each of the ranges is stored in order from a server having a smaller number of servers. From among the server group, the server that transfers the data range is determined so that a server with a lower communication cost necessary to transfer the entire data range already allocated is allocated a wider range, When a server that processes a transfer request is determined for all of the ranges, a parallel data transfer method of transferring data in the determined range from each of the servers in parallel. 5. A server, which is a computer for storing and reading data in response to a request, and one or more clients, each being a computer for sending a data storage request or a read request to the server, Each client and one or more servers in the server group are connected by a communication path, respectively, and the client and the server are simultaneously connected on the communication path by one or more virtual communication paths multiplexed by software. In a client-server system capable of performing communication, when the client computer issues a data storage request to two or more of the servers, a server group that processes the data storage request among the server groups And considering the cost of the communication path between each server and the client, Parallel data transfer method to determine the range of data to be transferred to each server, and transfers the range of data that the determined to the server in parallel. 6. The parallel data transfer method according to claim 1, wherein when there are two or more communication paths between the server and the client, the data is transferred between the server and the client. The data to be transferred is determined in consideration of the communication cost of each of the communication paths, and a range of data to be transferred on each of the communication paths and the number of virtual communication paths to be used simultaneously during communication of each of the communication paths are determined. And transferring the data in parallel.