JP2022082006A - Storage device - Google Patents

Abstract

To solve the problem that it is difficult to refer to the most recent snap shot promptly in a replica site side.SOLUTION: A storage device for performing replication with a master site has a replication part for receiving attribute information of a snap shot created in the master site and a hash list of data as a replication object from the master site, and updating index information by registering data existing in an own device on the basis of the hash list, and the replication part can refer to a snap shot without waiting for completion of data transfer from the master site after updating the index information.SELECTED DRAWING: Figure 8

Description

The present invention relates to a storage device, a replication method, and a program.

Replication may occur in deduplication storage.

For example, Patent Document 1 describes a replication system including a master-side storage system including a master-side storage device and a replica-side storage system including a replica-side storage device. The master-side storage system includes a plurality of data transfer means for transferring data stored in the master-side storage device to the replica-side storage system in parallel. Further, the replica-side storage system includes a replica means for storing the data transmitted from the master-side storage system in the replica-side storage device and generating a replica of the data stored in the master-side storage device. Further, according to Patent Document 1, each data transfer means of the master side storage system generates data feature amount information representing the feature amount of the data based on the content of the data which is a transfer candidate for the replica side storage system. It is checked whether or not the data feature amount information of the data that is a transfer candidate exists in the transfer data list in which the data feature amount information of the data being transferred or transferred to the replica side storage system is stored. The data of the transfer candidate whose data feature amount information exists in the transfer data list is not transferred to the replica side storage system, and the data of the transfer candidate whose data feature amount information does not exist in the transfer data list is sent to the replica side storage system. Forward.

Japanese Unexamined Patent Publication No. 2014-178827

When replication is performed in the deduplication storage as described in Patent Document 1, the latest snapshot can be referred to after the replication is completed. In other words, until the replication was completed, the replica site could see the previous snapshot and could not refer to the latest snapshot.

In this way, there has been a problem that it is difficult for the replica site to refer to the latest snapshot at an early stage. Therefore, an object of the present disclosure is to provide a storage device, a replication method, and a program that solve the problem that it is difficult for the replica site side to refer to the latest snapshot at an early stage.

In order to achieve such an object, the storage device which is one form of the present disclosure is
A storage device that replicates with the master site.
The attribute information of the snapshot created at the master site and the hash list of the data to be replicated are received from the master site, and the data existing in the own device is registered and indexed based on the hash list. It has a replication unit that updates information.
After updating the index information, the replication unit has a configuration in which a snapshot can be referred to without waiting for the completion of data transfer from the master site.

In addition, the replication method, which is another form of the present disclosure,
The storage device that replicates with the master site
The attribute information of the snapshot created at the master site and the hash list of the data to be replicated are received from the master site, and the data existing in the own device is registered and indexed based on the hash list. Update the information,
After updating the index information, the snapshot can be referred to without waiting for the completion of data transfer from the master site.

In addition, the program, which is another form of the present disclosure,
For storage devices that replicate to and from the master site
The attribute information of the snapshot created at the master site and the hash list of the data to be replicated are received from the master site, and the data existing in the own device is registered and indexed based on the hash list. Update the information,
This is a program for realizing a process that enables a snapshot to be referenced without waiting for the completion of data transfer from the master site after updating the index information.

According to each configuration as described above, it is possible to refer to the latest snapshot at an early stage on the replica site side.

It is a figure which shows the configuration example of the replication system in 1st Embodiment of this disclosure. It is a figure which shows the image of the file system structure in the deduplication storage. It is a block diagram which shows the configuration example of the deduplication storage. It is a figure for demonstrating an example of replication. It is a figure which shows the operation example at the time of performing replication. It is a flowchart which shows the operation example when the data which does not exist in a replica site is accessed. It is a figure which shows the hardware configuration example of the storage apparatus in the 2nd Embodiment of this disclosure. It is a block diagram which shows the configuration example of a storage device.

[First Embodiment]
The first embodiment of the present disclosure will be described with reference to FIGS. 1 to 6. FIG. 1 is a diagram showing a configuration example of the replication system 300. FIG. 2 is a diagram showing an image of the file system structure in the deduplication storages 110 and 120. FIG. 3 is a block diagram showing a configuration example of the deduplication storages 110 and 120. FIG. 4 is a diagram for explaining an example of replication. 5 and 6 are diagrams showing an operation example when performing replication.

In the first embodiment of the present disclosure, the replication system 300 having the deduplication storage 110 and the deduplication storage 120 will be described. As will be described later, in the case of the replication system 300 described in the present embodiment, the deduplication storage 110 on the master site side has the inode (attribute information) of the snapshot created by the start of replication and the hash list of the data. To the deduplication storage 120 on the replica site side. Further, the deduplication storage 120 registers the data existing in the deduplication storage 120 based on the hash list. Then, the deduplication storage 120 makes it possible to refer to the latest snapshot after updating the inode index B + Tree which is the index information. In this way, the deduplication storage 120 makes it possible to refer to the latest snapshot without waiting for the completion of transfer of data that the deduplication storage 120 does not have.

Further, the deduplication storage 120 may receive a reference request such as a read request for data that does not exist in the deduplication storage 120 after the latest snapshot can be referred to. In this way, when a reference request is made for data that does not exist in the deduplication storage 120, the deduplication storage 120 requests the deduplication storage 110 on the master side to preferentially transmit the data. do. As a result, necessary data is preferentially transmitted from the deduplication storage 110, and even if there is a reference to data that does not exist in the deduplication storage 120, the latest snapshot can be referred to without any problem.

FIG. 1 shows a configuration example of the replication system 300. Referring to FIG. 1, the replication system 300 has a deduplication storage 110, a deduplication storage 120, a business server 210, and a business server 220.

As shown in FIG. 1, the business server 210 and the deduplication storage 110 are connected so as to be able to communicate with each other. For example, the business server 210 can read / write to the file system of the deduplication storage 110. Further, the business server 220 and the deduplication storage 120 are connected so as to be able to communicate with each other. For example, the business server 220 can read / write to the file system of the deduplication storage 120. Further, the deduplication storage 110 and the deduplication storage 120 are connected so as to be able to communicate with each other. For example, in the replication in the present embodiment, the snapshot of the file system created in the deduplication storage 110 is moved to the deduplication storage 120.

The file system in the deduplication storage 110 and the deduplication storage 120 has, for example, a structure as shown in FIG. FIG. 2 shows a file system structure image in deduplication storage. In the deduplication storage 110 and the deduplication storage 120, the unit surrounded by the fine dotted line is treated as a block. As shown in FIG. 2, in the deduplication storage 110 and the deduplication storage 120, it is possible to access the super block which is the highest block of the inode index B + Tree based on the file system + version. In addition, it is possible to trace from the super block to each block based on the CA (content address).

Further, as shown in FIG. 2, by specifying and tracing the CA corresponding to fid (= inode), the data of the directory or file can be referred to. If the contents of multiple data are the same, the hash values will also be the same, and each CA will point to the same data. As a result, the area can be reduced.

The CA is composed of information such as a hash value of data (block). Therefore, when the data or block in the tree is changed, the CA to that data or block changes. Also, when a block containing the changed CA is updated, the CA to that block changes. By repeating the above changes, the CA of the super block will eventually be updated.

The deduplication storage 110 is a storage device that is on the master site side in replication. The deduplication storage 110 may be composed of one storage device, or may be composed of a plurality of server computers functioning as an accelerator node or a storage node as shown in Patent Document 1.

FIG. 3 shows a configuration example used when performing replication. Referring to FIG. 3, the deduplication storage 110 includes a replication unit 111 and a storage device 112. For example, the deduplication storage 110 has an arithmetic unit such as a CPU and a storage device such as a storage device 112. For example, the deduplication storage 110 can realize the above-mentioned processing unit by the arithmetic unit executing a program stored in the storage device such as the storage device 112.

The replication unit 111 performs replication by creating a snapshot of the file system and moving it to the deduplication storage 120.

For example, the replication unit 111 creates a snapshot of the file system stored in the storage device 112 of the deduplication storage 110 in response to the start of replication execution. Then, the replication unit 111 transmits the created snapshot inode to the deduplication storage 120. In other words, the replication unit 111 creates a snapshot and sends an inode, which is attribute information for managing files and directories, to the deduplication storage 120 in response to the start of replication execution.

Further, the replication unit 111 is stored in the storage device 112 and generates a hash list of each data to be replicated. Then, the replication unit 111 transmits the generated hash list to the deduplication storage 120. The hash list may be created in advance, for example.

Further, the replication unit 111 is stored in the storage device 112 and transmits each data to be replicated to the deduplication storage 120. The replication unit 111 may be configured to transmit only the data that the deduplication storage 120 does not have among the above data to the deduplication storage 120. Whether or not the deduplication storage 120 has data can be confirmed by referring to the hash value of the transferred data and the like. As will be described later, the deduplication storage 110 may be requested to preferentially transmit data from the deduplication storage 120. Upon receiving such a request, the replication unit 111 preferentially transmits the requested data to the deduplication storage 120.

In the present embodiment, the timing at which the replication unit 111 starts replication is not particularly limited. The replication unit 111 may be configured to start replication at any timing.

The deduplication storage 120 is a storage device that is on the replica site side in replication. Like the deduplication storage 110, the deduplication storage 120 may be composed of one storage device, or may be composed of a plurality of server computers that function as accelerator nodes and storage nodes.

Referring to FIG. 3, the deduplication storage 120 has a replication unit 121, a reference reception unit 122, a confirmation unit 123, a priority transmission request unit 124, and a storage device 125. For example, the deduplication storage 120 has an arithmetic unit such as a CPU and a storage device such as a storage device 125. For example, in the deduplication storage 120, the above-mentioned processing unit can be realized by the arithmetic unit executing a program stored in the storage device such as the storage device 125.

The replication unit 121 cooperates with the replication unit 111 of the deduplication storage 110 to perform replication.

For example, the replication unit 121 receives the inode, which is the attribute information transmitted by the replication unit 111, and the hash list. Then, the replication unit 121 registers the data already stored in the storage device 125 of the deduplication storage 120 based on the received hash list. For example, the replication unit 121 compares the received hash list with the own device hash list generated based on the data stored in the storage device 125, thereby displaying the data stored in the storage device 125. Identify. Then, the replication unit 121 registers the specified data.

In addition, the replication unit 121 updates the inode index B + Tree, which is index information. Then, the replication unit 121 makes it possible to refer to the latest snapshot. That is, the replication unit 121 registers the data stored in the storage device 125, but makes it possible to refer to the latest snapshot at the timing before the transfer of the data not stored in the storage device 125 is completed.

As described above, the replication unit 111 transmits each data to be replicated. When the replication unit 121 receives data from the replication unit 111, the replication unit 121 can appropriately register the received data.

FIG. 4 is a diagram showing an example of replication by the replication unit 111 and the replication unit 121. As shown in FIG. 4, at the timing when the latest snapshot can be referred to, undetermined blocks remain because there is untransferred data under the tree. The undetermined block is appropriately determined by registering the untransferred data under the tree.

The replication unit 121 can leave the old superblock when updating the superblock which is the highest block of the inode index B + Tree. You can keep multiple snapshots by leaving the old superblock when updating the superblock.

The reference receiving unit 122 receives a reference request such as a read request from the business server 220 or the like. In other words, the reference receiving unit 122 receives a reference request requesting a reference to the snapshot that can be referred to.

The confirmation unit 123 confirms whether or not the data requesting the reference is stored in the storage device 125 in response to the reference request received by the reference reception unit 122. For example, the confirmation unit 123 confirms whether or not the data for which reference is requested is stored in the storage device 125 by checking the hash list of the data stored in the storage device 125 or the like. When the data for which reference is requested is stored in the storage device 125, the confirmation unit 123 acquires the data from the storage device 125 and returns the acquired data to the business server 220. On the other hand, when the data for which the reference is requested is not stored in the storage device 125, the priority transmission request unit 124 is notified that the data is not stored.

The priority transmission request unit 124 requests the deduplication storage 110 to preferentially transmit the necessary data in response to the notification from the confirmation unit 123. For example, the priority transmission request unit 124 transmits the hash value of the data stored in the storage device 125 to the deduplication storage 110, so that the data having the hash value is preferentially transmitted. Request for exclusion storage 110.

The above is a configuration example of the replication system 300. Subsequently, an operation example when the replication system 300 performs replication will be described with reference to FIG. The operation when performing replication is not limited to the case illustrated in FIG.

FIG. 5 is a diagram showing an operation example when performing replication. Referring to FIG. 5, the replication unit 111 included in the deduplication storage 110 creates a snapshot of the file system stored in the storage device 112 included in the deduplication storage 110 in response to the start of replication execution. Then, the replication unit 111 transmits the created snapshot inode to the deduplication storage 120 (step S101).

Further, the replication unit 111 is stored in the storage device 112 and generates a hash list of each data to be replicated. Then, the replication unit 111 transmits the generated hash list to the deduplication storage 120 (step S102).

For example, the replication unit 121 of the deduplication storage 120 receives the inode, which is the attribute information transmitted by the replication unit 111, and the hash list.

The replication unit 121 registers the data already stored in the storage device 125 of the deduplication storage 120 based on the received hash list (step S201). For example, the replication unit 121 compares the received hash list with the own device hash list generated based on the data stored in the storage device 125, thereby displaying the data stored in the storage device 125. Identify. Then, the replication unit 121 registers the specified data.

Further, the replication unit 121 updates the inode index B + Tree (step S202). Then, the replication unit 121 makes it possible to refer to the latest snapshot (step S203).

The above is an operation example when the replication system 300 performs replication. For example, by the above operation, the deduplication storage 120 makes it possible to refer to the latest snapshot. Subsequently, with reference to FIG. 6, an operation example in the case of receiving a reference request requesting a reference to data that does not exist in the deduplication storage 120 will be described.

Referring to FIG. 6, the reference receiving unit 122 receives a reference request such as a read request from the business server 220 or the like (step S301).

The confirmation unit 123 confirms whether or not the data requesting the reference is stored in the storage device 125 in response to the reference request received by the reference reception unit 122 (step S302). For example, the confirmation unit 123 confirms whether or not the data for which reference is requested is stored in the storage device 125 by checking the hash list of the data stored in the storage device 125 or the like.

When the data for which reference is requested is stored in the storage device 125 (step S302, Yes), the confirmation unit 123 acquires the data from the storage device 125 and returns the acquired data to the business server 220. On the other hand, when the data for which reference is requested is not stored in the storage device 125 (step S302, No), the priority transmission request unit 124 is notified that the data is not stored.

The priority transmission request unit 124 requests the deduplication storage 110 to preferentially transmit the necessary data in response to the notification from the confirmation unit 123 (step S303). For example, the priority transmission request unit 124 transmits the hash value of the data stored in the storage device 125 to the deduplication storage 110, so that the data having the hash value is preferentially transmitted. Request for exclusion storage 110. In response to this request, the deduplication storage 110 will preferentially transmit the required data for which the reference is requested to the deduplication storage 120.

As described above, the deduplication storage 120 has a replication unit 121. With such a configuration, the replication unit 121 can refer to the latest snapshot without waiting for the completion of transfer of data that the deduplication storage 120 does not have. This makes it possible to refer to the latest snapshot at an early stage on the replica site side.

Further, the deduplication storage 120 has a confirmation unit 123 and a priority transmission request unit 124. With such a configuration, the priority transmission requesting unit 124 can request the deduplication storage 110 to preferentially transmit the necessary data for which the reference is requested. With such a configuration, the replica site can refer to the latest snapshot earlier without any problem. Further, according to the above configuration, since the latest snapshot can be referred to at an early stage and the referenced data is preferentially transmitted, it is less likely to be affected even if the replication bandwidth is limited. As a result, for example, it becomes possible to effectively utilize resources with functions other than replication.

Also, the deduplication storage 120 can be configured to retain the old superblock when updating the superblock. This makes it possible to leave multiple snapshots.

[Second Embodiment]
Next, a second embodiment of the present disclosure will be described with reference to FIGS. 7 and 8. In the second embodiment, the storage device 400 that performs replication with the master site will be described.

FIG. 7 shows a hardware configuration example of the storage device 400. Referring to FIG. 7, the storage device 400 has the following hardware configuration as an example.
-CPU (Central Processing Unit) 401 (arithmetic unit)
-ROM (Read Only Memory) 402 (storage device)
-RAM (Random Access Memory) 403 (storage device)
-Program group 404 loaded in RAM 403
A storage device 405 for storing the program group 404.
-Drive device 406 that reads and writes the recording medium 410 external to the storage device.
-Communication interface 407 that connects to the communication network 411 outside the storage device
-I / O interface 408 for inputting / outputting data
-Bus 409 connecting each component

Further, the storage device 400 can realize the function as the replication unit 421 shown in FIG. 8 by the CPU 401 acquiring the program group 404 and executing the program group 404. The program group 404 is stored in, for example, a storage device 405 or a ROM 402 in advance, and the CPU 401 loads the program group 404 into a RAM 403 or the like and executes the program group 404 as needed. Further, the program group 404 may be supplied to the CPU 401 via the communication network 411, or may be stored in the recording medium 410 in advance, and the drive device 406 may read the program and supply the program to the CPU 401.

Note that FIG. 7 shows an example of the hardware configuration of the storage device 400. The hardware configuration of the storage device 400 is not limited to the above case. For example, the storage device 400 may be configured from a part of the above-mentioned configuration, such as not having the drive device 406.

The replication unit 421 receives the attribute information of the snapshot created at the master site and the hash list of the data to be replicated from the master site, and registers the data existing in the own device based on the hash list. Update the index information. Further, the replication unit 421 makes it possible to refer to the snapshot without waiting for the completion of data transfer from the master site after updating the index information.

As described above, the storage device 400 has the replication unit 421. According to such a configuration, the replication unit 421 can refer to the snapshot after updating the index information without waiting for the completion of data transfer from the master site. This makes it possible to refer to the latest snapshot at an early stage on the replica site side.

The storage device 400 described above can be realized by incorporating a predetermined program into the storage device 400. Specifically, the program according to another embodiment of the present invention receives the attribute information of the snapshot created at the master site and the hash list of the data to be replicated from the master site in the storage device 400. , Registers the data existing in the own device based on the hash list and updates the index information, and after updating the index information, it is possible to refer to the snapshot without waiting for the completion of data transfer from the master site. It is a program to realize.

Further, in the replication method realized by the above-mentioned storage device 400, the storage device 400 receives the attribute information of the snapshot created at the master site and the hash list of the data to be replicated from the master site. , Registers the data existing in the own device based on the hash list and updates the index information, and after updating the index information, it is possible to refer to the snapshot without waiting for the data transfer from the master site to be completed. The method.

Even the invention of the program (or recording medium) or the replication method having the above-mentioned configuration achieves the above-mentioned object of the present invention in order to have the same operation and effect as the above-mentioned storage device 400. Can be done.

<Additional Notes>
Part or all of the above embodiments may also be described as in the appendix below. Hereinafter, the outline of the storage device and the like in the present invention will be described. However, the present invention is not limited to the following configuration.

(Appendix 1)
A storage device that replicates with the master site.
The attribute information of the snapshot created at the master site and the hash list of the data to be replicated are received from the master site, and the data existing in the own device is registered and indexed based on the hash list. It has a replication unit that updates information.
The replication unit is a storage device that enables a snapshot to be referred to after updating the index information without waiting for the completion of data transfer from the master site.
(Appendix 2)
The storage device according to Appendix 1, which has a confirmation unit for confirming whether or not the own device has the data when a data reference request is received from the external device.
(Appendix 3)
The storage device according to Appendix 2, which has a request unit that requests the master site to preferentially transmit necessary data according to the result of confirmation by the confirmation unit.
(Appendix 4)
The storage according to Appendix 3 in which the requesting unit requests the master site to preferentially transmit data that does not exist in the own device when the data to be referred to does not exist in the own device as a result of the confirmation by the confirmation unit. Device.
(Appendix 5)
The storage device according to any one of Supplementary note 1 to Supplementary note 4, wherein the replication unit holds a superblock to be updated when the superblock which is the uppermost block is updated.
(Appendix 6)
The storage device according to any one of Supplementary note 1 to Supplementary note 5, wherein the replication unit receives an inode as the attribute information.
(Appendix 7)
The storage device that replicates with the master site
The attribute information of the snapshot created at the master site and the hash list of the data to be replicated are received from the master site, and the data existing in the own device is registered and indexed based on the hash list. Update the information,
A replication method that enables a snapshot to be referenced without waiting for the completion of data transfer from the master site after updating the index information.
(Appendix 8)
When receiving a data reference request from an external device, check whether the own device has the data, and request the master site to preferentially send the necessary data according to the confirmation result. 7. The replication method according to 7.
(Appendix 9)
For storage devices that replicate to and from the master site
The attribute information of the snapshot created at the master site and the hash list of the data to be replicated are received from the master site, and the data existing in the own device is registered and indexed based on the hash list. Update the information,
A program for realizing a process that enables a snapshot to be referenced without waiting for the completion of data transfer from the master site after updating the index information.
(Appendix 10)
When a data reference request is received from an external device, it is confirmed whether or not the own device has the data, and the master site is requested to preferentially transmit the necessary data according to the confirmation result. The program described in Appendix 9 for realizing the above.

In addition, the program described in each of the above-described embodiments and appendices may be stored in a storage device or recorded in a recording medium readable by a computer. For example, the recording medium is a portable medium such as a flexible disk, an optical disk, a magneto-optical disk, and a semiconductor memory.

Although the invention of the present application has been described above with reference to each of the above embodiments, the invention of the present application is not limited to the above-described embodiment. Various changes that can be understood by those skilled in the art can be made to the structure and details of the present invention within the scope of the present invention.

110 Deduplication storage 111 Replication unit 112 Storage device 120 Deduplication storage 121 Replication unit 122 Reference reception unit 123 Confirmation unit 124 Priority transmission request unit 125 Storage device 210 Business server 220 Business server 300 Replication system 400 Storage device 401 CPU
402 ROM
403 RAM
404 Program group 405 Storage device 406 Drive device 407 Communication interface 408 Input / output interface 409 Bus 410 Recording medium 411 Communication network 421 Replication unit

Claims (10)

A storage device that replicates with the master site.
The attribute information of the snapshot created at the master site and the hash list of the data to be replicated are received from the master site, and the data existing in the own device is registered and indexed based on the hash list. It has a replication unit that updates information.
The replication unit is a storage device that enables a snapshot to be referred to after updating the index information without waiting for the completion of data transfer from the master site. The storage device according to claim 1, further comprising a confirmation unit for confirming whether or not the own device has the data when a data reference request is received from the external device. The storage device according to claim 2, further comprising a requesting unit that requests the master site to preferentially transmit necessary data according to the result of confirmation by the confirmation unit. The third aspect of claim 3, wherein the requesting unit requests the master site to preferentially transmit data that does not exist in the own device when the data to be referred to does not exist in the own device as a result of the confirmation by the confirmation unit. Storage device. The storage device according to any one of claims 1 to 4, wherein the replication unit holds a super block to be updated when the super block, which is the highest block, is updated. The storage device according to any one of claims 1 to 5, wherein the replication unit receives an inode as the attribute information. The storage device that replicates with the master site
The attribute information of the snapshot created at the master site and the hash list of the data to be replicated are received from the master site, and the data existing in the own device is registered and indexed based on the hash list. Update the information,
A replication method that enables a snapshot to be referenced without waiting for the completion of data transfer from the master site after updating the index information. When a data reference request is received from an external device, it is confirmed whether or not the own device has the data, and a request is made to the master site to preferentially transmit the necessary data according to the confirmation result. Item 7. The replication method according to Item 7. For storage devices that replicate to and from the master site
The attribute information of the snapshot created at the master site and the hash list of the data to be replicated are received from the master site, and the data existing in the own device is registered and indexed based on the hash list. Update the information,
A program for realizing a process that enables a snapshot to be referenced without waiting for the completion of data transfer from the master site after updating the index information. When a data reference request is received from an external device, it is confirmed whether or not the own device has the data, and the master site is requested to preferentially transmit the necessary data according to the confirmation result. The program according to claim 9.

Images