JP6924952B2 - Computer system and restore method - Google Patents

Description

The present invention generally relates to data restoration.

A wide area disaster such as an earthquake may damage the file server and cause the files stored in the file server to be lost. In order to anticipate such a wide area disaster and improve disaster resistance, in general, disaster recovery, that is, copying (backing up) files to a remote location through a network, the file server is damaged and files are lost due to the disaster. In that case, the backed up files are reversely duplicated (restored) to the reconstructed file server via the network. If the restoration can be performed at high speed, the waiting time until the start of using the file server after a disaster can be shortened, and the data of the file server can be used at an early stage.

As one method for performing restoration at high speed, the method disclosed in Patent Document 1 can be considered. The file transfer method disclosed in Patent Document 1 divides one file into fixed-size fragments before file transfer.

US6,085,251

Post-disaster networks are unstable and network delays (typically network delay times) can change (for example, in the event of a disaster, the network may become congested and the network delays may increase. ). This changes the maximum throughput per restore session.

In addition, delays vary in the time it takes to reach maximum throughput. In addition, the amount of data in the files transmitted during this period is different.

As described above, the characteristics of the network change as the delay of the network changes.

The file transfer method of Patent Document 1 divides a file into fixed-size fragments. In Patent Document 1, the change in delay is not taken into consideration, and the change in network characteristics due to the change in delay is not taken into consideration. Therefore, it may not be possible to achieve the optimum restore speed, such as dividing the file into a size that allows transmission to finish before reaching the maximum throughput. This is especially problematic for cases where files are restored through a post-disaster network, in other words a network that is considered more unstable.

Such challenges are not limited to disaster recovery, but may exist in other cases of restoring data such as files over the network.

The computer system measures the network delay between the computer system and one or more network storages. The computer system acquires chunks, which are at least a part of the data to be restored, from one or more network storages through the network according to at least one of the chunk size and the degree of parallelism according to the measured network delay. The computer system restores the data to be restored based on the acquired chunks.

According to the present invention, an optimum restore speed can be expected according to the network delay.

It is a block diagram which shows the configuration example of the backup restore system which concerns on Example 1. FIG. It is a block diagram which shows the internal structure of a file server. It is a block diagram which shows the internal structure of a cloud server. It is a figure which shows the configuration example of the cloud server management table. It is a figure which shows the configuration example of the data transmission / reception management table. It is a figure which shows the configuration example of the stored data management table. It is a schematic diagram which shows an example of the outline of parallel data restoration. It is a figure which shows the flow of a backup process. It is a figure which shows the flow of a restore process. It is a figure which shows the chunk acquired for each restore session in the restore process. It is a schematic diagram which shows an example of the restoration process which concerns on Example 2. FIG. It is a schematic diagram which shows an example of the backup process and the restore process which concerns on Example 3. FIG. It is a schematic diagram which shows an example of the flow control of TCP (Transmission Control Protocol). It is a schematic diagram which shows an example of the problem which concerns on one comparative example. It is a schematic diagram which shows an example of the outline of Example 1. FIG. It is a schematic diagram which shows an example of the effect of Example 1. FIG.

Hereinafter, some examples will be described. The examples described below are examples, and the present invention is not limited to these examples.

In the following description, the information may be described by the expression of "abc table", but the information may be expressed by a data structure other than the table. At least one of the "abc tables" can be referred to as "abc information" to show that it does not depend on the data structure. Further, in the following description, the configuration of each table is an example, and one table may be divided into two or more tables, or all or a part of the two or more tables may be one table. good.

Further, in the following description, the "interface unit" includes one or more interfaces. One or more interfaces may be one or more interfaces of the same type (eg, one or more NICs (Network Interface Cards)) or two or more heterogeneous interface devices (eg, NICs and HBAs (Host Bus Adapters)). There may be.

Further, in the following description, the "storage unit" includes one or more memories. At least one memory may be a volatile memory or a non-volatile memory. The storage unit is mainly used during processing by the processor unit.

Further, in the following description, the "processor unit" includes one or more processors. The at least one processor is typically a microprocessor such as a CPU (Central Processing Unit). Each of the one or more processors may be single-core or multi-core. The processor may include hardware circuits that perform some or all of the processing.

Further, in the following description, the processing unit (function) may be described by the expression of "kkk unit", but the processing unit may be realized by executing one or more computer programs by the processor unit. It may be realized by one or more hardware circuits (for example, FPGA (Field-Programmable Gate Array) or ASIC (Application Specific Integrated Circuit)). When the processing unit is realized by the processor unit in the program, the specified processing is appropriately performed using the storage unit and / or the interface unit (for example, the communication port), so that the processing unit is at least a part of the processor unit. May be. The processing described with the processing unit as the subject may be processing performed by the processor unit or a device having the processor unit. Further, the processor unit may include a hardware circuit that performs a part or all of the processing. The program may be installed on the processor from the program source. The program source may be, for example, a program distribution computer or a computer-readable recording medium (eg, a non-temporary recording medium). The description of each processing unit is an example, and a plurality of processing units may be combined into one processing unit, or one processing unit may be divided into a plurality of processing units.

Further, in the following description, the "computer system" includes one or more physical computers. At least one physical computer may execute a virtual computer (for example, VM (Virtual Machine)) or SDx (Software-Defined anything). As SDx, for example, SDS (Software Defined Storage) (an example of a virtual storage device) or SDDC (Software-defined Datacenter) can be adopted.

Further, in the following description, the common code among the reference codes will be used when the same type of elements are not distinguished, and the reference code will be used when the same type of elements are described separately.

Further, in the present specification, the network delay may be abbreviated as "delay" as appropriate. Delay typically means network delay time (in typically "milliseconds").

First, a problem relating to one comparative example and an outline of Example 1 for solving the problem will be described.

The issues related to one comparative example are as follows.

Restoring in parallel is faster than restoring files sequentially. Specifically, for example, it is assumed that the network bandwidth of one communication path is 1 MB / s and the size of the file to be restored is 4 MB. In this case, the communication in the sequential restoration takes 4 seconds (= 4MB / (1MB / s)). On the other hand, communication in parallel restore, in which a file is divided into 1 MB chunks and restored in 4 parallels, takes only 1 second. This is because four 1MB chunks are acquired in parallel through four communication paths, each of which has a network bandwidth of 1MB / s.

However, parallel restore, which divides the file into fixed-size chunks and acquires the chunks in parallel, is not always optimal. This is because network delays affect network characteristics, such as the maximum throughput per restore session and the time it takes to reach the maximum throughput.

In particular, this issue is a network in which communication is performed according to a protocol in which flow control is performed, including a slow start that gradually increases the data transmission / reception size per unit time, typically communication in accordance with TCP (Transmission Control Protocol). It is large in the case of getting chunks through the network where is done.

FIG. 13 is a schematic diagram showing an example of TCP flow control.

File transfer is performed according to, for example, HTTP (Hyper Text Transfer Protocol) or FTP (File Transfer Protocol), and both HTTP and FTP are protocols on TCP. In TCP, in order to avoid congestion, flow control including a slow start that gradually increases the data transmission / reception size per unit time is implemented.

However, the slow start speed (increase in data transmission / reception size) in flow control differs depending on the delay (see reference numeral 1301), as illustrated in FIG. The maximum bandwidth also depends on the delay (see reference numeral 1302).

FIG. 14 is a schematic diagram showing an example of a problem according to a comparative example.

If the file is divided into chunks of fixed chunk size that are not related to the delay and the chunks are acquired in parallel from the cloud server 1405, the optimum restore speed cannot always be realized. Specifically, for example, as illustrated in the left part of FIG. 14, if the chunk size of the file is too small for the delay, the chunk acquisition for each chunk ends during the slow start. (Chunk acquisition ends before the throughput reaches the maximum bandwidth) (see reference numeral 1401). In addition, if the chunk size is too small, the number of chunks will exceed the maximum number of restore sessions supported by the system. In this case, the total bandwidth (the product of the maximum bandwidth of one communication path and the degree of parallelism) does not reach the wire speed limit of 1350 because all chunks cannot be restored in parallel (see reference numeral 1402). Sometimes. On the other hand, for example, as illustrated in the right part of FIG. 14, even if the chunk size is too large for the delay (extremely, for example, if the file is not divided), the total bandwidth is similarly the wire speed limit 1350. May not be reached.

The above is the problem related to one comparative example. The outline of Example 1 for solving the problem is as follows.

FIG. 15 is a schematic view showing an example of an outline of the first embodiment.

In this embodiment, the file server (restore) 103B (an example of the second computer system) holds the data transmission / reception management table 213 and the file system volume 203. The file server (restore) 103B executes a delay measurement program (an example of a delay measurement means) 212, a data transmission / reception program (an example of an acquisition means) 210, and a restore program (an example of a restore means) 209.

The data transmission / reception management table 213 holds the chunk size and the degree of parallelism for each of the plurality of delays. The delay measurement program 212 measures the network delay between the file server (restore) 103B and the cloud server (an example of network storage) 105. The data transmission / reception program 210 specifies the chunk size and the degree of parallelism according to the measured network delay from the data transmission / reception management table 213, and acquires chunks that are at least a part of the files to be restored from the cloud server 105 through the network. .. The restore program 209 restores the restore target file to the file system volume 203 based on the chunk acquired for the restore target file.

FIG. 16 is a schematic view showing an example of the effect of Example 1.

The chunk size of the file is an appropriate size for the delay. Therefore, for each chunk (each of chunks 1 to 3), the chunk acquisition does not end during the slow start (chunk acquisition ends before the throughput reaches the maximum bandwidth) ( See reference numeral 1601). Also, the total bandwidth reaches the wire speed limit of 1650, resulting in shorter restore times (see reference numeral 1402).

The above is the outline of the first embodiment. Hereinafter, the first embodiment will be described in detail. The graphs illustrated in FIGS. 13, 14 and 16 are simplified experimental graphs (for example, graphs that do not consider throughput changes according to congestion control). Further, in this embodiment, the "restore speed" is the speed of one restore (restoration of one or more files targeted for restoration). The "restore time" is the time required for one restore.

FIG. 1 is a block diagram showing a configuration example of the backup / restore system according to the first embodiment.

The backup restore system 100 includes one or more file servers 103. Each of the one or more file servers 103 is communicably connected to the management server 104 and the one or more cloud servers 105 through the network 102. The network 102 is a network that typically performs communication according to TCP, and is, for example, a LAN (Local Area Network) or the Internet. In the present invention, the configuration of the network 102 is not limited.

One or more clients 101 are communicably connected to the network 102. Take one client 101 as an example. The client 101 is a computer that uses the file server (operation) 103A or the file server (restore) 103B. The end user who uses the client 101 connects to the file servers 103A and 103B by using the file access program 110 of the client 101, and reads and writes the file stored in the file server 103A or 103B.

The file server (operation) 103A is an example of the first computer system. The file server (operation) 103A is a server device that provides a file access service to the client 101 before a disaster occurs. On the other hand, the file server (restore) 103B is an example of the second computer system. The file server (restore) 103B is a server device that provides a file access service to the client 101 after a disaster occurs. The file access service refers to a network service that enables reading and writing of files using protocols such as NFS (Network File System) and CIFS (Common Internet File System). The present invention does not limit the form of each file access service of the file server (operation) 103A and the file server (restore) 103B. Further, the file server (restore) 103B may be a file server after the replacement of the file server (operation) 103A. The structure and processing operation of the file server 103 will be described in detail later.

The management server 104 is a computer that sets the backup / restore system 100 and the like. The administrator of the backup / restore system 100 sets up the backup / restore system using the management server 104. For example, the administrator sets the IP address of the file server 103 and the like. The function of the management server 104 may be introduced into the client 101 or the file server 103. In the present invention, the place where the function of the management server 104 is introduced and the place where the management server 104 is installed are not limited.

Cloud servers 105A and 105B are exemplified as one or more cloud servers 105. The cloud servers 105A and 105B are computers that hold backups of files stored in the file server (operation) 103A. The file server (operation) 103A periodically transmits files to the cloud servers 105A and 105B, and the cloud servers 105A and 105B store the files on the internal disk. The structure and processing operation of the cloud server 105 will be described in detail later.

When the file server (operation) 103A is physically damaged due to a disaster and the stored file is lost, the file server (restore) 103B provides a file access service. Therefore, the file server (restore) 103B acquires the files saved in the cloud servers 105A and 105B through the network 102 and stores (restores) them in the internal disk. If the necessary files can be obtained from the cloud server 105, the file server (restore) 103B restarts the file access service in the same manner as the file server (operation) 103A. In this way, the backup / restore system allows the file access service to continue even after a disaster.

FIG. 2 is a block diagram showing an internal configuration of the file server 103.

The file server 103 includes a network I / F 201, a CPU 202, a file system volume 203, and a memory 205, and they are connected by an internal communication path 204.

The network I / F201 is an example of an interface unit. The network I / F 201 is a device that is interconnected with the network 102 and is used when receiving a file access request from the client 101.

The CPU 202 is an example of a processor unit. The CPU 202 is a device that executes a program stored in the memory 205.

The file system volume 203 is a device for storing information such as a program file and a data file created by an end user. An external storage device (not shown) may be adopted instead of the file system volume 203. For example, the restore destination by the file server (restore) 103B may be an external storage device connected to the file server (restore) 103B in place of or in addition to the file system volume 203 in the file server (restore) 103B.

The memory 205 is an example of a storage unit. The memory 205 is a memory (for example, a volatile memory) that temporarily holds a file stored in the file system volume 203 when the CPU 202 processes the file. When the CPU 202 executes a program, the CPU 202 reads a program file or a data file from the file system volume 203 to the memory 205. Hereinafter, unless otherwise specified, the program shall be executed by the CPU 202. Further, it is assumed that the program is read from the file system volume 203 into the memory 205 and executed.

The memory 205 stores programs such as a file sharing server program 206, a file system program 207, a creation program 208, a restore program 209, a transmission / reception program 210, a management program 211, and a delay measurement program 212. Further, the memory 205 stores information such as a data transmission / reception management table 213, a cloud server management table 214, and a storage data management table 215.

The file sharing server program 206 is a program that processes a file access request from the file access program 110 of the client 101. There are a file write request and a file read request as file access requests. The file data received in association with the file write request is stored in the file system volume 203 as a data file (file). Of the files stored in the file system volume 203, the files that have not yet been backed up are backed up to the cloud server 105.

The file system program 207 is a program that manages the data of the file stored in the file system volume 203. When the file sharing server program 206 receives the file access request from the file access program 110, the file sharing server program 206 passes the request to the file system program 207. Then, the file system program 207 accesses the file system volume 203. For example, the file access program 110 transmits a file write request and file data to the file sharing server program 206. The file sharing server program 206 receives the file write request and receives the file data. The file sharing server program 206 passes a file write request and file data to the file system program 207. The file system program 207 writes the file data to the file system volume 203.

The creation program 208 is a program that changes the file stored in the file system volume 203 into a format stored in the cloud server 105. For example, when performing simple duplication of a file, the creation program 208 does not change the format of the file. On the other hand, when the file is divided and duplicated, the creation program 208 divides the file into fragments of a certain size. The data created by the creation program 208 is transmitted (backed up) to the cloud server 105 by the data transmission / reception program 210 described later.

The restore program 209 is a program that restores data (chunks) acquired from the cloud server 105 using the data transmission / reception program 210 described later to the original file (for example, integrates a plurality of chunks to create the original file). Is.

The data transmission / reception program 210 is a program for transmitting / receiving data through the cloud server 105 and the network 102. The data transmission / reception program 210 stores the file and its fragments in the cloud server 105. Further, the data transmission / reception program 210 acquires a chunk of files from the cloud server 105. In this embodiment, HTTP (Hypertext Transfer Protocol) is used for communication between the data transmission / reception program 210 and the cloud server 105. In the present invention, the communication protocol between the data transmission / reception program 210 and the cloud server 105 is not limited.

The management program 211 is a program for setting an address or the like for accessing the cloud server 105 for transmitting and receiving files. The management program 211 is started by the administrator. The administrator connects the management server 104 to the file server 103 using SSH (Secure Shell) protocol or the like. The connection protocol between the management server 104 and the file server 103 does not have to be the SSH protocol. In the present invention, the connection protocol is not limited. Further, when a console device such as a keyboard or a display is connected to the file server 103, the administrator may directly log in to the file server 103 for setting.

The data transmission / reception management table 213 is a table set by the management program 211 that defines a data transmission / reception method due to network delay. The data transmission / reception management table 213 will be described in detail later.

The cloud server management table 214 is a table that holds the access address of the cloud server 105, which is set by the management program 211. The cloud server management table 214 will be described in detail later.

The storage data management table 215 is a table that manages the storage destination cloud server 105 that stores the backup file for each file. The stored data management table 215 will be described in detail later.

FIG. 3 is a block diagram showing an internal configuration of the cloud server 105.

The cloud server 105 is composed of a network I / F 301, a CPU 302, a memory 303, and a file system volume 310, and they are connected to each other by an internal communication path 307.

The network I / F 301 is a device connected to the network 102. The CPU 302 is a device that executes a program stored in the memory 303. The file system volume 310 is a device for storing program files and backup data 309.

The file system program 304 is a program that manages the backup data 309 stored in the file system volume 310 as a file.

The data transmission / reception program 305 is a program that receives file data sent from the file server 103. Further, the data transmission / reception program 305 receives the acquisition request of the backup data 309 from the file server 103, and transmits the backup data 309. The backup data 309 is an example of data to be restored, and is typically a file.

When the data transmission / reception program 305 reads the backup data 309 of the file system volume 310, it transmits a file read request to the file system program 304. The file system program 304 reads the backup data 309 from the file system volume 310 and returns it to the data transmission / reception program. On the other hand, when the data transmission / reception program 305 receives the file data from the file server 103, the data transmission / reception program 305 transmits a file write request and the file data to the file system program 304. The file system program 304 writes a file to the file system volume 310 as backup data 309.

FIG. 4 is a diagram showing a configuration example of the cloud server management table 214.

The cloud server management table 214 is composed of a site ID 401, a site name 402, and a URL 403. The site ID 401, the site name 402, and the URL 403 are set as one record 404 as a set. The record 404 exists for each cloud server 105.

The site ID 401 is an ID (for example, a serial number) of the cloud server 105. The site name 402 is the name of the cloud server 105. The URL 403 is a root path (URL (Uniform Resource Locator)) used by the file server 103 to access the cloud server 105. The file server 105 generates an access path based on the URL 403 when backing up a file or accessing the backed up file. The usage of the cloud server management table 214 will be described in detail later.

FIG. 5 is a diagram showing a configuration example of the data transmission / reception management table 213.

The data transmission / reception management table 213 is composed of a delay 501, a chunk size 502, and a degree of parallelism 503. The delay 501, chunk size 502, and parallelism 503 are set as a set as a record 504. Record 504 exists for each delay.

The delay 501 is a value representing the delay of the network 102. The delay 501 is a representative value. For example, if the delay 501 is set to 10 milliseconds (msec), it is interpreted as 10 milliseconds or less. If the delay at a given time is 5 ms, then record 504A with a delay of 10 ms is selected. In the present invention, the interpretation of the delay 501 is not limited.

The chunk size 502 is the size when the file server 103B acquires the file stored in the cloud server 105. For example, when the file server (restore) 103B acquires a 1 MB file from the cloud server 105, it acquires it from the cloud server 105 as a block of four data of 0.25 MB shown in the chunk size 502. The degree of parallelism 503 indicates the number of parallel acquisitions when the file server (restore) 103B acquires the files stored in the cloud server 105. For example, at a certain time, when the file server (restore) 103B acquires a 1 MB file, if the delay 501 is 20 milliseconds, the file server (restore) 103B is regarded as a block of two data of 0.5 MB. Data is acquired in two parallels using two restore sessions. The method of using the data transmission / reception management table 213 will be described in detail later.

According to the example of FIG. 5, if the delay is relatively small, the chunk size is small and the number of parallels is large, at least one of them. For example, the value of the chunk size 502 corresponding to the delay 501 "10 msec" is smaller than the value of the chunk size 502 corresponding to the delay 501 of a value greater than "10 msec". Further, for example, the value of the degree of parallelism 503 corresponding to the delay 501 "10 msec" is larger than the value of the degree of parallelism 503 corresponding to the delay 501 of a value larger than "10 msec".

Further, according to the example of FIG. 5, if the delay is relatively large, the chunk size is large and the number of parallels is small, at least one of them. For example, the value of the chunk size 502 corresponding to the delay 501 "30 msec" is larger than the value of the chunk size 502 corresponding to the delay 501 of a value smaller than "30 msec". Further, for example, the value of the degree of parallelism 503 corresponding to the delay 501 "30 msec" is smaller than the value of the degree of parallelism 503 corresponding to the delay 501 of a value larger than "30 msec".

Further, in this embodiment, the data transmission / reception management table 213 prepared in advance is used, but instead of or in addition to the use of the table 213, at least one of the chunk size and the degree of parallelism according to the delay is set. It may be determined by calculation. If the maximum number of restore sessions and the maximum number of backup sessions supported by the system are known in advance, the chunk size 502 and the degree of parallelism 503 may be determined with reference to the values. For example, the degree of parallelism 503 is a value that does not exceed the maximum number of restore sessions, and the chunk size 502 is a value obtained by dividing the file size by the degree of parallelism 503.

FIG. 6 is a diagram showing a configuration example of the stored data management table 215.

The storage data management table 215 is composed of a file path 601, a size 602, and a storage destination cloud server ID 603. The file path 601 and the size 602 and the storage destination cloud server ID 603 are managed as a set as a record 604. Record 604 exists for each backed up file.

The file path 601 indicates the path of the file stored in the file system volume 203 of the file server 103. For example, if a file is stored in the file system volume 203 with the path /aaa/bbb.db, it is described as such in the file path 601. The method of using the stored data management table 215 will be described in detail later.

FIG. 7 is a diagram showing an outline of data restoration when the file server (restore) 103B restores a certain file from the cloud servers 105A and 105B in parallel.

In the cloud servers 105A and 105B, files of 1 MB from the file server (operation) 103A are stored as backup data 306 by simple duplication. That is, the same backup data 306 is stored in the cloud servers 105A and 105B. The file server (restore) 103B acquires backup data 306 from the cloud servers 105A and 105B through the network 102 with a delay of 20 milliseconds. At this time, based on the chunk size 502 of the record 504B (delay 501 “20 msec”) of the data transmission / reception management table 213, the file server (restore) 103B makes a chunk of 0.5 MB and acquires the file (backup data 306). .. Further, the file server (restore) 103B acquires data (chunks) in two parallels based on the degree of parallelism 503 of the record 504B of the data transmission / reception management table 213. In this example, the file server (restore) 103B acquires data (chunks) in parallel from the cloud servers 105A and 105B. If the product of the chunk size and the degree of parallelism is less than or equal to the amount of data that the cloud server 105A can transmit per unit time, the file server (restore) 103B transfers the chunk of the chunk size from the cloud server 105A to the degree of parallelism. You may get it with. In the present invention, the method of selecting the acquisition destination cloud server 105 is not limited. In this way, the file server (restore) 103B determines at least one of the chunk size and the degree of parallelism when acquiring the backup data 306 based on the delay of the network 102, and determines the chunk size and the degree of parallelism. Acquire chunks according to at least one of them.

FIG. 8 shows a flow of file backup processing executed by the file server (operation) 103A.

The backup process is executed by the data transmission / reception program 210 and the creation program 208 in cooperation with each other. The administrator of the file server (operation) 103A instructs the management program 211 to back up the files stored in the file system volume 203. The instruction is sent from the management program 211 to the data transmission / reception program 210.

In response to the instruction, the data transmission / reception program 210 selects one of the files stored in the file system volume 203 that has not been backed up (step 801).

Next, the data transmission / reception program 210 selects the cloud server 105 as the transmission destination (backup destination) of the file selected in step 801 (“selected file” in the description of FIG. 8) from the cloud server management table 214 (step). 802). At this time, the data transmission / reception program 210 selects two or more cloud servers 105 when transmitting to a plurality of cloud servers 105.

Next, the data transmission / reception program 210 calls the creation program 208 and requests the creation of duplicate data. The creation program 208 does not make any changes to the selected file in the case of simple duplication. On the other hand, in the case of split duplication, the creation program 108 divides the selected file (file data) into a size such as 1 MB. Then, the data transmission / reception program 210 transmits the backup data created by the creation program 208 to the cloud server 105 selected in step 802 (step 803).

Next, the data transmission / reception program 210 adds information about the selected file that has been transmitted to the cloud server 105 and has been backed up to the stored data management table 215 as a new record 404 (step 804).

Then, the data transmission / reception program 210 determines whether or not there is a file that has not been backed up (step 805). If the determination result of step 805 is true (step 805: Yes), the data transmission / reception program 210 continues the process from step 801. On the other hand, if the determination result in step 805 is false (step 805: No), the backup processing flow ends.

As a result, the files stored in the file server (operation) 103A are backed up to the cloud server 105. This backup process is executed periodically or irregularly (for example, each time an unbacked up file is stored in the file server (operation) 103A) before a disaster occurs. That is, after the disaster occurs, the file system volume 203 of the file server (operation) 103A at the time when the backup process is executed can be restored.

FIG. 9 shows a flow of file restoration processing executed by the file server (restore) 103B.

The restore process is executed by the data transmission / reception program 210 and the restore program 209 in cooperation with each other.

The administrator of the file server (restore) 103B instructs the management program 211 to restore the backup data 306 stored in the cloud server 105. The instruction is sent from the management program 211 to the data transmission / reception program 210.

In response to the instruction, the data transmission / reception program 210 searches the stored data management table 215 and selects one file that has not been restored to the file system volume 203 (step 901).

Next, the data transmission / reception program 210 determines the unrestored data of the [remaining amount] (delay time of the network 102) and the [remaining amount] (file selected in step 901 (“selected file” in FIG. 9)). Amount) and is set (step 902). Since the delay is unknown at the start of restoration of the selected file, the data transmission / reception program 210 sets an initial value (for example, 30 milliseconds) in [Delay]. Instead, the data transmission / reception program 210 may communicate data such as dummy data through the network 102, and set the delay time measured by the delay measurement program for the communication to [delay]. Further, the data transmission / reception program 210 sets a value of size 602 corresponding to the selected file in [Remaining amount].

Next, the data transmission / reception program 210 searches the data transmission / reception management table 213 based on the delay (value set in [delay]) (step 903). For example, when the [delay] is 30 milliseconds, the data transmission / reception program 210 selects the record 504C including the delay 501 “30 msec”. When the number of chunks of the size indicated by the chunk size 502 corresponding to the delay is less than the parallel degree (value) indicated by the parallel degree 503 corresponding to the delay, the data transmission / reception program 210 selects the file recently selected in step 901. Alternatively or additionally, the total file size is one or more such that the number of chunks of the size indicated by the chunk size 502 corresponding to the delay is equal to or greater than the parallelism (value) indicated by the parallelism degree 503 corresponding to the delay. You may select a file.

Then, the data transmission / reception program 210 transmits a chunk acquisition request of the size indicated by the chunk size 502 in the record 504C in parallel to one or more cloud servers 105 by the degree of parallelism indicated by the parallelism degree 503 in the record 504C, and the parallelism thereof. Chunks are acquired (received) from one or more cloud servers 105 in response to the acquisition requests (step 904). In the iterative process from step 902 to step 906, if the delay does not change from the time of the previous chunk acquisition, the operation of acquiring the subsequent chunks is performed in step 904. Details of the repetitive operation will be described later with reference to FIG.

When acquiring chunks from the cloud server 105 in step 904, the delay measurement program 212 measures the delay in communication between the file server (restore) 103B and the cloud server 105 (step 905).

Subsequently, the data transmission / reception program 210 determines whether or not the [remaining amount] is greater than 0 (step 906). For example, when restoring an 8MB file, if the data transmission / reception program 210 acquires 1MB chunks in 4 parallels according to the record 504C of the data transmission / reception management table 213, a total of 4MB of data has been acquired, so 4MB is still available. There is a remaining amount of. That is, [remaining amount] = 4 MB. Therefore, in this case, the determination result of step 906 is true (step 906: Yes), and the data transmission / reception program 210 continues the process from step 902. On the other hand, when "remaining amount" = 0 MB, the determination result in step 906 is false (step 906: No), and the process proceeds to step 907.

Then, the data transmission / reception program 210 searches the stored data management table 215, and if there is a file to be restored (step 907: Yes), the process continues from step 901. On the other hand, if there is no file to be restored (step 907: No), the restore process is terminated.

FIG. 10 is a diagram showing repeated chunk acquisition of one file (repetition of steps 902 to 906 in FIG. 9). Assuming a case of restoring an 8 MB file, the flow of chunk acquisition will be described with reference to FIG. In the description of FIG. 10, the “loop” refers to the process from step 902 to step 906. The "loop" is an example of the number of restores. Further, in FIG. 10, the white-painted chunk is a chunk as at least a part of the unprocessed data, and the gray chunk is a processed (acquired) chunk.

First, since the delay (delay time) of the first loop is unknown in the data transmission / reception program 210, "30 milliseconds" is set as the initial value of [delay]. Further, the data transmission / reception program 210 sets "8MB" in the [remaining amount]. Then, the record 504C having a delay 501 of the data transmission / reception management table 213 of 30 milliseconds is selected. The chunk size 502 of record 504C is "1MB" and the degree of parallelism 503 is "4". Therefore, the data transmission / reception program 210 sets the file to be restored as a chunk of 1 MB, and acquires the first four chunks in parallel. That is, the first chunk acquisition in the loop is to acquire chunks 1 to 4 in parallel. As a result, the acquisition of 4MB of file data is completed. Therefore, the remaining amount is 4MB (= 8MB-4MB), and therefore, the data transmission / reception program 210 updates the [remaining amount] to "4MB".

In the second loop of the next loop, it is assumed that the delay (measured delay time) when acquiring chunks 1 to 4 in the first loop is 20 milliseconds. Therefore, by the second loop, the data transmission / reception program 210 updates the [delay] to "20 milliseconds". In the second loop, the data transmission / reception program 210 selects the record 504B in which the delay 501 of the data transmission / reception management table 213 is "20 milliseconds". The chunk size 502 of record 504B is "0.5MB" and the degree of parallelism 503 is "2". Therefore, it is assumed that the data transmission / reception program 210 has 8 chunks of 0.5 MB of the remaining data of 4 MB. Since the degree of parallelism is "2", the data transmission / reception program 210 acquires two chunks A and B in parallel. At this point, 4 MB for the first loop and 1 MB for the second loop are acquired, and the remaining amount is 3 MB. Therefore, the data transmission / reception program 210 updates the [remaining amount] to "3MB".

In the third loop, it is assumed that the delay (measured delay time) when the chunks A and B are acquired in the second loop is 20 milliseconds. Therefore, the [delay] remains "20 milliseconds". That is, the data transmission / reception program 210 determines that the delay time has not changed. Therefore, in the third loop, the data transmission / reception program 210 maintains the chunk size "0.5 MB" and the degree of parallelism "2" corresponding to the delay "20 milliseconds", and acquires the subsequent chunks C and D. At this point, another 1 MB is acquired, and the remaining amount is 2 MB. Therefore, the data transmission / reception program 210 updates the [remaining amount] to "2MB".

Since the delay does not change at "20 ms" in the 4th loop and the 5th loop, the data transmission / reception program 210 has a chunk size of "0.5 MB" and a degree of parallelism of "2" corresponding to the delay of "20 ms". Continue the restore with ". If chunk G and chunk H can be acquired in the fifth loop, the remaining amount becomes 0 MB, and the restoration of this file is completed.

In this way, the data transmission / reception program 210 changes the chunk size and the number of parallels according to the recent delay for each loop so that the optimum restore speed corresponding to the delay can be achieved. Therefore, the optimum restore speed can be expected to be maintained.

According to the above embodiment, the chunk size (file acquisition size) and the degree of parallelism of acquisition are dynamically changed based on the delay of the network 102. As a result, it can be expected that chunks will be acquired using the maximum bandwidth of the network 102. As a result, it can be expected that the restore speed will be improved, in other words, the restore time will be shortened. By shortening the restore time, the file sharing service after a disaster can be restarted at an early stage.

In this embodiment, the data transmission / reception program 210 in the file server (operation) 103A may back up each file in the file server (operation) 103 to each of the cloud servers 105 for the maximum degree of parallelism. The data transmission / reception program 210 in the file server (restore) 103B may acquire chunks for the parallel degree from the cloud storage 105 for the parallel degree in parallel among the cloud servers 105 for the maximum parallel degree. Since the restore source is distributed to a plurality of cloud servers 105, it can be expected that high-speed restore can be realized more reliably.

The second embodiment will be described with reference to FIG. Since this embodiment corresponds to a modified example of the first embodiment, the differences from the first embodiment will be mainly described.

FIG. 11 is a schematic diagram showing an example of the restore process according to the second embodiment.

The restore method according to this embodiment is a restore method for restoring a small file. When restoring a small file, if the files are divided by the chunk size 502 of the data transmission / reception management table 213, the degree of parallelism may be smaller than 503. On the other hand, if the files are divided by the degree of parallelism 503, the size may be smaller than the chunk size 502. In this case, the appropriate chunk size and parallel number cannot be satisfied. Therefore, in this embodiment, the chunk size 502 is protected, and the degree of parallelism is satisfied by having a plurality of files to be restored in one restore process (main process).

In FIG. 11, 2 MB of File 1 and File 2 are backed up to the cloud servers 105A and 105B, respectively. And the delay of the network 102 is 30 milliseconds. Since the delay is 30 milliseconds, according to the data transmission / reception management table 213 (FIG. 5), the chunk size 502 adopted is "1.0 MB", and the degree of parallelism 503 adopted is "4". Further, it is assumed that the file selected as the restoration target is File1.

In this case, even if File 1 is divided into chunks of 1 MB, restoration in two parallels is the limit, and the degree of parallelism "4" is not satisfied. On the other hand, when File 1 is divided into four in parallel (when divided into four), the size of each chunk becomes 0.5 MB, and the chunk size "1.0 MB" is not satisfied.

Therefore, the data transmission / reception program 210 in the file server (restore) 103B restores File 2 in chunks of 1 MB in addition to File 1. This makes it possible to restore a total of 4 in parallel. That is, in order to satisfy the chunk size "1.0 MB" and the degree of parallelism "4", the size to be restored needs to be 4.0 MB or more (1.0 x 4 = 4.0 MB). Therefore, the data transmission / reception program 210 satisfies that the total file size is 4.0 MB or more by targeting the 2 MB File 2 as the restore target in addition to the 2 MB File 1.

In order to achieve this operation, the data transmission / reception program 210 determines the chunk size and the degree of parallelism in step 903 of FIG. 9, and then determines whether or not the degree of parallelism is insufficient. If this determination result is true, the data transmission / reception program 210 starts the restore process as another thread or process (starts the sub restore process), and operates steps 901 to 906. That is, a new file to be restored is selected, and the process proceeds. The restore process is started in the same manner until the degree of parallelism corresponding to the delay is satisfied (until the total degree of parallelism achieved by the restore process becomes equal to or higher than the degree of parallelism corresponding to the delay). However, the restore process does not operate individually, but waits for the process in step 904. That is, in step 904, a plurality of restore processes are executed at the same time. This achieves the degree of parallelism of the restore. This individual restore process may be an example of a restore session. At this time, in order to adjust the degree of parallelism, the data transmission / reception program 210 may change the chunk size and the degree of parallelism for each restore process (restore session). For example, the data transmission / reception program 210 reduces the chunk size and the degree of parallelism of any of the restore processes when the total degree of parallelism of all restore processes does not match the target degree of parallelism (the degree of parallelism corresponding to the delay). You may make the adjustment.

As described above, according to the present embodiment, when the file is small and the degree of parallelism cannot be achieved by acquiring the data of one file, the degree of parallelism is achieved by collectively targeting a plurality of files as one restore target. be able to. As a result, it can be expected that the restore speed can be increased without depending on the file size of the file to be restored.

In the above description, the chunk size priority restore process in which the chunk size is prioritized is adopted, but the parallel degree priority restore process in which the parallel degree is prioritized may be adopted. For example, the following processing may be performed.

The data transmission / reception program 210 performs the following processing,
(P) Assuming that the file to be restored is divided into chunks for the degree of parallelism according to the measured delay, it is determined whether or not the chunk size is equal to or larger than the chunk size according to the measured delay. matter,
(Q) If the judgment result of (p) is true, the restore target file is divided into chunks for the degree of parallelism according to the measured delay, and the chunks obtained by the division are divided according to the measured delay. Obtaining with the degree of parallelism,
May be executed. As a result, the degree of parallelism according to the delay is maintained.

In addition, the data transmission / reception program 210 performs the following processing.
(R) When the judgment result of (q) is false, the restore target file and one or more other restore target files are divided into chunks for the degree of parallelism according to the measured delay, and the result is obtained by the division. Obtaining the chunks obtained with the degree of parallelism according to the measured delay,
May be executed. The total size of the restore target file and one or more other restore target files is greater than or equal to the product of the chunk size according to the measured delay and the degree of parallelism according to the measured delay. As a result, the chunk size according to the delay is maintained.

Even if the judgment result of (q) is false, the data transmission / reception program 210 performs the following processing instead of (r) if there is no other restore target file for which the total size equal to or larger than the above product can be obtained. ,
(S1) The file to be restored is divided into chunks of chunk size according to the measured delay, and the chunks obtained by the division are acquired at the maximum degree of parallelism less than the degree of parallelism according to the measured delay. To do,
May be executed. As a result, the chunk size according to the delay is protected as the minimum chunk size, and chunks can be acquired with a degree of parallelism that is less than the degree of parallelism according to the delay but as high as possible.

The data transmission / reception program 210 performs the following processing,
(S2) When the judgment result of (q) is false, the restore target file is divided into chunks of chunk size according to the measured delay, and the chunks obtained by the division are divided according to the measured delay. Performing to get at the maximum degree of parallelism, less than the degree of parallelism,
May be executed. As a result, the chunk size according to the delay is kept as the minimum chunk size regardless of the presence or absence of other restore target files that can obtain the total size equal to or larger than the above product, and the parallelism is less than the parallelism according to the delay but as high as possible. You can get chunks by degree.

The above is an example of parallelism priority restore processing. The data transmission / reception program 210 selects whether to adopt the chunk size priority restore process or the parallel degree priority restore process (for example, selects based on the file to be restored or the measured delay), and the selected process. May be executed. It may be predetermined whether to adopt the chunk size priority restore process or the parallel degree priority restore process for each file or for each delay (for example, a representative value).

The third embodiment will be described with reference to FIG. Since this embodiment corresponds to a modified example of the first embodiment, the differences from the first embodiment will be mainly described.

FIG. 12 is a schematic diagram showing an example of the backup process and the restore process according to the third embodiment.

The restore method according to this embodiment is a split restore method in which a file is divided into a plurality of fragments. In this example, each of the plurality of fragments overlaps at least one of the posterior portion of the immediately preceding fragment and the anterior portion of the immediately preceding fragment. Therefore, the chunk size and the degree of parallelism when restoring the file data can be adjusted.

According to FIG. 12, an example of the outline of the third embodiment is as follows. That is, the file of the file server (operation) 103A4MB is divided into a plurality of fragments, and the plurality of fragments are backed up to the cloud servers 105A and 105B. The file server (restore) 103B acquires chunks of a plurality of fragments in parallel from the cloud servers 105A and 105B, and restores a 4 MB file.

The details will be described below.

The data in the file is 4MB. The 4MB file data is divided into three fragments 1 to 3, and the three fragments 1 to 3 are backed up to the cloud servers 105A and 105B. Fragment 1 is the 0th to 2nd MB portion of the file data, fragment 2 is the 1st to 3rd MB portion of the file data, and fragment 3 is the 2nd to 4th MB portion of the file data. The division into the fragments 1 to 3 is carried out by the creation program 208. The size of each fragment is the same. The fragment size is preferably a common multiple of a plurality of chunk sizes (eg, 0.25 MB, 0.5 MB, 1.0 MB according to the example of FIG. 5). This is because one or more chunks are obtained from one fragment. Further, the fragment size is preferably M times the least common multiple of a plurality of types of chunk sizes (M is an integer of 2 or more). Because to get more than one chunk from one fragment).

Fragments 1 and 2 are stored in the cloud server 105A. Fragment 3 is stored in the cloud server 105B.

When the file server (restore) 103B restores the file data, if the communication delay of the network 102 is 20 milliseconds, the data transmission / reception program 210 of the file server (restore) 103B uses the record 504B of the data transmission / reception management table 213. Select. At this time, the chunk size 502 is "0.5 MB" and the degree of parallelism 503 is "2". Therefore, the data transmission / reception program 210 acquires the fragment 1 and the fragment 3 in parallel in 0.5 MB units from the beginning. As a result, restoration can be performed in two parallels, and the chunk size and the degree of parallelism set in the data transmission / reception management table 213 can be achieved.

On the other hand, when the communication delay of the network 102 is 30 milliseconds, the data transmission / reception program 210 selects the record 504C of the data transmission / reception management table 213. At this time, the chunk size 502 is "1 MB" and the degree of parallelism 503 is "4". Therefore, the data transmission / reception program 210 includes a chunk 1-1 of the first half 1 MB of the fragment 1, a chunk 2-1 of the first half 1 MB of the fragment 2, a chunk 3-1 of the first half 1 MB of the fragment 3, and a chunk 3-2 of the second half 1 MB. Is acquired in 4 parallels.

As described above, according to the present embodiment, the divided and duplicated file can be restored by dividing the file into a plurality of fragments and duplicating the file. Further, by partially superimposing the fragment ranges, the communication delay of the network 102 increases, and even when a large chunk and a large degree of parallelism are restored, the chunk size and the degree of parallelism can be achieved and restored.

Although some examples have been described above, these are examples for explaining the present invention, and the scope of the present invention is not limited to these examples. The present invention can also be implemented in various other forms.

103: File server

Claims (14)

A computer system that acquires restore target data from one or more network storages through a network.
A delay measuring means for measuring network delay between one or more network storages, and
An acquisition means for acquiring chunks, which are at least a part of the data to be restored, from the one or more network storages through the network according to at least one of the chunk size and the degree of parallelism according to the measured network delay. ,
A computer system including a restore means for restoring the data to be restored based on the acquired chunks. The acquisition means is the following processing,
(A) Dividing the unprocessed data of the data to be restored into chunks of chunk size according to the recently measured network delay.
(B) Of the chunks obtained by the division of (a), the chunks corresponding to the degree of parallelism according to the recently measured network delay are acquired.
(C) Judging whether or not there is a difference between the chunk obtained by the division of (a) and the chunk corresponding to the degree of parallelism.
(D) If the judgment result of (c) is true, execute (a) with the difference as unprocessed data.
To execute,
The computer system according to claim 1. The acquisition means, the following processing,
The (h) raw data of the restoration target data, assuming that the dividing into chunks chunk size corresponding to the measured network delay, the number of chunks, according to the network delay that is recently measured Judging whether or not the degree of parallelism is higher,
(I) When the determination result of (h) is true, the unprocessed data in the restore target data is divided into chunks of chunk size according to the measured network delay.
(J) When the judgment result of (h) is false, the total data size is greater than the product of the chunk size according to the recently measured network delay and the degree of parallelism according to the recently measured network delay. In addition to the restore target data, one or more other restore target data are selected, and the unprocessed data of the restore target data and the selected one or more other restore target data is measured recently. Dividing into chunks of chunk size according to the network delay
(K) Of the chunks obtained by the division of (i) or (j), the chunks corresponding to the degree of parallelism according to the measured network delay are acquired.
To execute,
The computer system according to claim 1. The data to be restored is stored in each of the network storages for a predetermined maximum degree of parallelism.
The one or more network storages are network storages corresponding to the maximum degree of parallelism.
The acquisition means acquires chunks corresponding to the degree of parallelism in parallel from the network storage corresponding to the degree of parallelism among the network storages corresponding to the degree of parallelism.
The computer system according to any one of claims 1 to 3. The acquisition means is the following processing,
(P) If it is assumed that the data to be restored is divided into chunks for the degree of parallelism according to the measured network delay, does the chunk size become larger than or equal to the chunk size according to the measured network delay? To judge whether or not
(Q) When the determination result of (p) is true, the restore target data is divided into chunks for the degree of parallelism according to the measured network delay, and the chunks obtained by the division are measured. Obtaining with the degree of parallelism according to the network delay,
To execute,
The computer system according to claim 1. The acquisition means is the following processing,
(R) When the judgment result of (p ) is false, the restore target data and one or more other restore target data are divided into chunks for the degree of parallelism according to the measured network delay, and the chunks are divided. Acquiring the chunk obtained by the division with the degree of parallelism according to the measured network delay,
And
The total data size of the restore target data and the other restore target data of 1 or more is equal to or more than the product of the chunk size according to the measured network delay and the degree of parallelism according to the measured network delay. ,
The computer system according to claim 5. Even if the judgment result of (p ) is false, the acquisition means performs the following processing instead of (r) if there is no other restore target data for which the total size equal to or larger than the product can be obtained.
(S) The restore target data is divided into chunks of chunk size according to the measured network delay, and the chunks obtained by the division are divided into chunks having a degree of parallelism less than the measured network delay. obtaining the maximum degree of parallelism,
To execute,
The computer system according to claim 6. The acquisition means is the following processing,
(S) When the determination result of (p ) is false, the restore target data is divided into chunks of chunk size according to the measured network delay, and the chunks obtained by the division are divided into the measured chunks. executing the obtaining the maximum degree of parallelism of less than parallelism in accordance with the network delay,
To execute,
The computer system according to claim 5. One or more restore target data including the restore target data is stored in a plurality of network storages.
The acquisition means acquires chunks of one or more restore target data from two or more network storages among the plurality of network storages.
The computer system according to claim 1. A plurality of fragments of the data to be restored are stored in the plurality of network storages.
Each of the plurality of fragments overlaps with at least one of the posterior portion of the immediately preceding fragment and the anterior portion of the immediately preceding fragment.
The acquisition means acquires chunks of the plurality of fragments from the plurality of network storages.
The computer system according to claim 9. The acquisition means acquires chunks according to at least one of the recently measured chunk size and degree of parallelism according to the recently measured network delay for each restore session.
The computer system according to any one of claims 1 to 10. The data to be restored is a file.
The network is TCP (Transmission Control Protocol) in which flow control including a slow start that gradually increases the data transmission / reception size per unit time is performed.
The computer system according to any one of claims 1 to 11. A first computer system that backs up data to one or more network storages,
A second computer system for restoring the data to be restored among the data stored in the one or more network storages is provided.
The second computer system is
The network delay between the one or more network storages and the second computer system is measured and
According to at least one of the chunk size and the degree of parallelism according to the measured network delay, a chunk that is at least a part of the data to be restored is acquired.
Restore the restore target data based on the acquired chunks,
Backup restore system. It is a restoration method by a computer system that acquires data to be restored from one or more network storages through a network.
The computer system
Measure the network delay between the one or more network storages and
Chunks, which are at least a part of the data to be restored, are acquired from the one or more network storages through the network according to at least one of the chunk size and the degree of parallelism according to the measured network delay.
Restore the restore target data based on the acquired chunks,
Restore method.

Images