JP5263237B2 - Storage system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a storage system for, even when a volume of a buffer installed in a copy destination device is smaller than that of a buffer installed in a copy source device as regards a buffer to be used for a remote copy whose sequentiality is secured, validly using the buffer installed in the copy source device. <P>SOLUTION: A copy source device 101 includes: an identification information acquisition part for acquiring buffer identification information for reception to be used for identifying a buffer for reception installed in a copy destination device 111; a buffer for transmission for storing data to be transferred to the buffer for reception; a data transfer control part for associating the buffer for transmission with the buffer for reception on the basis of buffer identification information for transmission installed in the buffer for transmission and buffer identification information for reception in performing transfer processing; and a transfer processing part for transferring the data of the buffer for transmission to the buffer for reception on the basis of the association. <P>COPYRIGHT: (C)2011,JPO&amp;INPIT

Description

  The present invention relates to a storage system having one or more storage devices and having an asynchronous copy function that guarantees the order.

  Conventionally, a RAID (Redundant Array of Inexpensive Disks) apparatus that employs a distributed cache memory type storage system has a redundant configuration including a plurality of control modules that control input / output of data to / from storage in order to improve performance and reliability. It has been adopted. Data read / write processing for the logical volume is executed for each control module.

In addition, such a RAID device is provided with a remote copy function that guarantees ordering, called advanced copy, in order to improve reliability and the like.
FIG. 33 is a diagram showing a conventional example of a RAID apparatus having a remote copy function that guarantees order. FIG. 33 shows a storage system including a RAID device 3301 including control modules 00 and 01 and a RAID device 3302 including control modules 10 and 11.

  Each RAID device stores a buffer that temporarily stores data stored in a storage medium or stores data into a plurality of divided areas, and a BIT (Buffer Index) that stores an index assigned to each area of the data stored in the buffer. 1 or 2 provided in a copy destination device that stores data sent from the copy source device and one or more buffers provided in the copy source device in which the data to be copied is stored. A buffer set storage unit that stores a buffer set composed of a combination of the above buffers and a storage medium that stores data are provided.

  In remote copy that guarantees order, the copy source device reads data to be copied from a storage medium and sequentially stores it in a recording buffer. When all the areas in the recording buffer are used up or a certain time has elapsed, the copy source device collects the data in the recording buffer to the copy destination device 3302 connected via the network according to the buffer set. Transmit (this transmission method is hereinafter referred to as “collective transmission method”). At this time, the buffer set is also transmitted simultaneously.

  On the other hand, when receiving data from the copy source device 3301, the copy destination device 3302 temporarily stores the data in the recording buffer. Further, the copy destination apparatus 3302 also stores the received buffer set in its own buffer set storage unit. When all the data is received, the copy destination apparatus 3302 expands the data in the recording buffer in a predetermined recording medium.

  As described above, data transmission is performed in a collective manner using a recording buffer, the buffer set is collectively controlled, and data is expanded on a storage medium in the copy destination device 3302, thereby guaranteeing order. doing.

  FIG. 33 shows an example where the number of control modules in the copy source apparatus 3301 and the number of recording buffers included in the control module are the same as the number of recording buffers included in the control module and control module in the copy destination apparatus 3302.

  Similarly, when the number of control buffers in the copy source apparatus 3301 and the number of recording buffers in the control module are larger than the number of recording buffers in the copy destination apparatus 3302, the copy source apparatus 3301 All buffer sets can be used effectively.

  However, if the number of control buffers in the copy source apparatus 3301 and the number of recording buffers in the control module are larger than the number of control buffers in the copy destination apparatus 3302 and the number of recording buffers in the control module, all the buffers in the copy source apparatus 3301 There was a problem that the set could not be used effectively.

  FIG. 33 is a diagram illustrating an example in which the number of control modules in the copy source apparatus is greater than the number of control modules in the copy destination apparatus. FIG. 33 shows a storage system including a RAID device 3301 including control modules 00, 01, 02, and 01 and a RAID device 3302 including control modules 10 and 11.

  Similarly to FIG. 33, each RAID device temporarily stores a data stored in a storage medium or data to be stored in a plurality of divided areas, and an index assigned to each area for the data stored in the buffer. A recording buffer having a BIT to be stored; a buffer set storage unit that stores a buffer set including an ID of a copy source buffer and an ID of a copy destination buffer; and a storage medium that stores data.

  For example, when there are eight buffer areas in the control module, there are only two control modules in the copy destination apparatus 3302 compared to four control modules in the copy source apparatus 3301, so data by the above-described batch sending method is used. When transmission is performed, the number of buffer areas that can be effectively used by the control module of the copy source apparatus 3301 is half, which is four, and the remaining buffers cannot be used.

  In relation to the above technology, the usage status of the write buffer that temporarily stores the write data is monitored, and when the free space of the buffer decreases, the information in the buffer is written to a separately prepared high-speed disk system, and the usage status 2. Description of the Related Art A backup device for a disk array system is known in which the saved data is written back to the buffer when the situation improves.

  Also, a virtual ordered writing method that provides consistency (restorability) of the secondary device at the time of failure by achieving a low latency for each host write regardless of the distance between the primary device and the secondary device It has been known.

  Further, in a system including a management computer, a host computer, a first storage device, and a second storage device, when maintaining duplicate data of the first storage device in the second storage device by remote data copy, from the host computer A data copy method switching method is known in which a change in the access amount to the first storage device is monitored, and the management computer instructs the storage device to switch the remote data copy method in accordance with the change.

  A storage system having a remote copy function that guarantees the order guarantee while maintaining the superiority of scalability of a distributed cache memory type storage system is also known.

  There is also known a storage apparatus that prevents interruption of application processing by securing a reservation buffer for remote copy in advance and avoiding a buffer shortage of communication performed using a buffer reservation target.

There is also known a remote data copy method between disk array devices that reduces the overhead in the update data transmission protocol and realizes a data update order guarantee function and a copy stop function.
JP 2006-268420 A JP 2006-523890 A JP 2006-236019 A JP 2006-260292 A JP 2005-122235 A JP 2003-167684 A

  The present invention has been made in view of the above-described problems, and a problem to be solved by the present invention relates to a buffer used for remote copy that guarantees the order, and the buffer provided in the copy destination apparatus is included in the copy source destination apparatus. To provide a storage system that can effectively use a buffer of a copy source device even when the number of buffers is smaller than that provided.

  In order to solve the above-described problem, the storage system includes a first storage volume that stores data transmitted from a host device and a first control unit that controls storage in the first storage volume. A first storage device, a second storage volume to which data stored in the first storage device is copied via a network, and a second control unit that controls storage in the second storage volume; And a second storage device, wherein the first control unit receives reception buffer identification information used for identification of a reception buffer provided in the second storage device. An identification information acquisition unit to be acquired, a transmission buffer for storing data to be transferred to the reception buffer of the second storage device, and a transmission provided in the transmission buffer when performing transfer processing A data transfer control unit for associating the transmission buffer with the reception buffer based on the buffer identification information for reception and the reception buffer identification information, and for receiving the data in the transmission buffer based on the association A transfer processing unit for transferring data to the buffer.

  According to this storage system, when the first control unit performs the transfer process, the transmission buffer and the reception buffer based on the transmission buffer identification information and the reception buffer identification information provided in the transmission buffer Therefore, even if there is no space in the reception buffer, reception processing can be performed if there is space in the transmission buffer, so that the copy processing can be continued. Further, if there is a space in the transmission buffer while there is a space in the reception buffer, a copy process from the first storage device to the second storage device can be performed.

  As described above, according to the disclosed storage system, a storage system that can effectively use the buffer of the copy source device even when the copy destination device has fewer buffers than the copy source device. It becomes possible to provide.

It is a figure which shows the outline | summary of a structure of the storage system concerning a present Example. It is a figure which shows the specific structural example of the memory with which the control module which concerns on a present Example is provided. It is a figure explaining the outline | summary of the remote copy process which concerns on a present Example. It is a figure explaining the outline | summary of the remote copy process which concerns on a present Example. It is a figure explaining the outline | summary of the remote copy process which concerns on a present Example. It is a figure explaining the outline | summary of the remote copy process which concerns on a present Example. It is a figure explaining the outline | summary of the remote copy process which concerns on a present Example. It is a figure explaining the outline | summary of the remote copy process which concerns on a present Example. 7 is a flowchart illustrating an overview of remote copy processing according to the embodiment. 7 is a flowchart illustrating an overview of remote copy processing according to the embodiment. It is a flowchart which shows the specific process of the copy destination buffer allocation process based on a present Example. It is a flowchart which shows the process which determines the buffer set transfer process in the copy source apparatus after the path | route block | blocking state cancellation based on a present Example. It is a flowchart which shows the state change process of the buffer in a copy source apparatus when the path | route obstruction | occlusion state concerning a present Example is detected. 10 is a flowchart showing I / O processing in the copy source apparatus in a path blocked state according to the embodiment. It is a flowchart (the 1) which shows the unused buffer ID acquisition process and buffer set transfer resumption process after the path | route blockage state cancellation which concerns on a present Example. It is a flowchart (the 2) which shows the unused buffer ID acquisition process and buffer set transfer resumption process after the path | route blockage state cancellation which concerns on a present Example. It is a flowchart which shows the unused buffer ID acquisition process and buffer set transfer resumption process after the path | route blockage state cancellation which concerns on a present Example. 7 is a flowchart illustrating processing in a copy destination apparatus when a path blockage state is detected according to the embodiment. It is a figure which shows the prior art example of the remote copy process which guaranteed the order. It is a figure which shows the prior art example of the remote copy process which guaranteed the order. It is a figure explaining the outline | summary of the buffer evacuation process based on a present Example. It is a figure explaining the outline | summary of the buffer evacuation process based on a present Example. It is a flowchart which shows the process which produces | generates the area | region of the save buffer which concerns on a present Example. It is a flowchart which shows the specific process of allocation to the control module of the save buffer which concerns on a present Example. It is a figure which shows the structural example in the case of comprising the save buffer which concerns on a present Example by one RAID group. It is a figure which shows the structural example in the case of comprising the save buffer which concerns on a present Example by two RAID groups. It is a figure explaining the save buffer which concerns on a present Example. It is a figure explaining the write-back pointer information and stage pointer information in the buffer evacuation processing according to the present embodiment. It is a flowchart which shows the update process of the write-back pointer information which concerns on a present Example. It is a flowchart which shows the update process of the stage pointer information which concerns on a present Example. It is a flowchart which shows the write-back process which concerns on a present Example. It is a figure which shows the specific example of the write-back process which concerns on a present Example. It is a figure which shows the specific example of the write-back process which concerns on a present Example. It is a flowchart which shows the staging process which concerns on a present Example. It is a flowchart which shows the staging process which concerns on a present Example. It is a figure which shows the prior art example of a RAID apparatus provided with the remote copy function which guaranteed the order. It is a figure which shows the prior art example of a RAID apparatus provided with the remote copy function which guaranteed the order.

Hereinafter, an example of the embodiment will be described with reference to FIGS.
FIG. 1 is a diagram illustrating an outline of a configuration of a storage system 100 according to the present embodiment.
A storage system 100 shown in FIG. 1 includes a RAID device 101 having control modules 102a, 102b, 102c and 102d, and a RAID device 111 having control modules 112a, 112b, 112c and 112d, and a distributed cache memory type storage system. It is.

  The control module 102a includes a CPU (Central Processing Unit) 103a that executes a predetermined program instruction to operate the control module 102a, a memory 104a used for a BIT, a recording buffer, a buffer set storage unit, and the like described later, and a host computer ( Hereinafter, a CA (Channel Adapter) 105a, which is an interface control unit with a "host" 109, and an RA (Remote Adapter), which is an interface control unit, between another RAID device or the like connected via a network or a dedicated line 120. ) 106a and a disk device 1 including an FC (Fibre Channel) 107a that is an interface control unit between the disk device 108a and a storage device such as a magnetic disk device. 08a.

  The control modules 102b, 102c and 102d included in the RAID device 101 and the control modules 112a, 112b, 112c and 112d included in the RAID device 111 have the same configuration.

  In the above configuration, for example, the “identification information acquisition unit”, “data transfer control unit”, “transfer processing unit”, “evacuation control unit”, “restoration control unit”, and the like are stored in a predetermined area on the memory 104a by the CPU 103a. It can be realized by executing the program instruction expanded in (1).

  The “transmission buffer”, “first pointer”, “second pointer”, and the like can be realized by using a partial area of the memory 104a, and the “evacuation volume” is 1 or 2 This can be realized by using a part of the disk device 108a constituted by the magnetic disk device described above.

  In the following embodiment, a case where remote copy is performed from the RAID apparatus 101 to the RAID apparatus 111 will be described. In this case, the RAID device 101 is referred to as “copy source device 101”, and the RAID device 111 is referred to as “copy destination device 111”.

  Although FIG. 1 shows the case where the RAID devices 101 and 111 are configured by four control modules, the present invention is not limited to this. Similarly, the numbers of CA, RA, and FC are not limited to the numbers shown in FIG.

FIG. 2 is a diagram illustrating a specific configuration example of a memory included in the control module according to the present embodiment, for example, the memories 104a and 114a illustrated in FIG.
As shown in FIG. 2, the memory 200 provided in the control module of the RAID device 101 or 111 according to the present embodiment includes a recording buffer 201 that stores in the disk device or temporarily stores and manages the stored data, and a buffer set Are stored in the buffer set storage unit 202, and an unused buffer ID storage unit 203 that stores buffer IDs of unused buffers in the copy destination apparatus 111.

  The buffer 201a is a storage unit that manages data for each of a plurality of areas (hereinafter referred to as “generations”) divided into a certain size, and each generation stores data in a disk device such as the disk device 108a. Data to be stored (for example, write data described later) is temporarily stored.

  Regarding (0000), (0001), (0002),... Described in the buffer 201a, the numbers 0000, 0001, 0002,... In parentheses indicate the buffer IDs assigned to the respective generations and are enclosed in parentheses. The numbers (0000), (0001), (0002),... Indicate that arbitrary data is stored in the buffer of the numbered buffer ID in parentheses.

  The BIT 201b is a table that stores, for each generation of the buffer 201a, an LU (Logical Unit) or LBA (Logical Block Address) in which data stored in the generation is expanded, a data size, a copy session number, and the like. is there.

  0000, 0001, 0002,... Described in the BIT 201b indicate buffer IDs assigned for each generation of the buffer 201a. For example, 0000 in the BIT 201b indicates that the LU or LBA in which the data stored in the buffer with the buffer ID 0000 is expanded, the data size, the copy session number, and the like are stored. When performing remote copy between RAID devices, the buffer set storage unit 202 includes a buffer ID (hereinafter referred to as “copy source ID”) of a buffer provided in the copy source device 101 storing the copy target data, and a copy destination. A combination (hereinafter referred to as “buffer set”) of a buffer ID (hereinafter referred to as “copy destination ID”) of the buffer provided in the apparatus 111 is stored.

  When the copy source apparatus 101 and the copy destination apparatus 111 are provided with a plurality of control modules as in the present embodiment, the buffer set is a buffer in which data to be copied is stored and a plurality of controls provided in the copy source apparatus 101. This is a combination of a buffer ID of the module buffer and buffer IDs of buffers of a plurality of control modules provided in the copy destination apparatus 111 that store data transmitted from the copy source apparatus 101.

  For example, in the buffer set storage unit 202 illustrated in FIG. 2, the control modules 00, 01, 02, and 03 included in the copy source apparatus 101 have buffers IDs of 0000, 0100, 0200, and 0300, respectively, as illustrated in FIG. FIG. 6 shows a buffer set when the control modules 00, 01, 02 and 03 provided in the copy destination apparatus 111 have buffers with buffer IDs 1000, 1100, 1001 and 1101, respectively.

  That is, the buffer ID “0000” of the buffer of the control module 00 provided in the copy source apparatus 101, the buffer ID “1000” of the buffer of the control module 00 provided in the copy destination apparatus 111, and the buffer ID of the control module 01 provided in the copy source apparatus 101. The buffer ID “0100”, the buffer ID “1100” of the buffer of the control module 01 provided in the copy destination apparatus 111, the buffer ID “0200” of the buffer of the control module 02 provided in the copy source apparatus 101, and the control module provided in the copy destination apparatus 111 Buffer ID “1001” of the buffer 02, buffer ID “0300” of the buffer of the control module 03 provided in the copy source apparatus 101, and buffer ID “1101” of the buffer of the control module 03 provided in the copy destination apparatus 111, The combination has been defined.

  In the following description, the generation of a buffer newly stored in the recording buffer 201 or the buffer set for the buffer is “new generation”, and the buffer set already stored in the recording buffer 201 or the buffer set for the buffer is The generation is called the “old generation”.

  The unused buffer ID storage unit 203 stores a buffer ID of an unused buffer provided in the copy destination apparatus 111. For example, upon receiving a notification of an unused buffer ID from the copy destination device 111 in response to a request for notification of an unused buffer ID of the copy source device 101, the copy source device 101 sets the notified buffer ID to an unused buffer ID. This is stored in the used buffer ID storage unit 203.

  In the above configuration, in the remote copy processing according to the present embodiment, for example, the write data in the copy source apparatus 101 is stored in the recording buffer 201, transferred to the copy destination apparatus 111, and the transfer data disk apparatus in the copy destination apparatus 111. The order of the entire storage system is guaranteed by performing processing such as expansion to 118a in a batch in units of buffer sets.

  The copy source apparatus 101 according to this embodiment monitors the network status between the copy source apparatus 101 and the copy destination apparatus 111, and performs remote copy processing between the copy source apparatus 101 and the copy destination apparatus 111 according to the network status. Control. The remote copy process is shown in FIGS. 11, 12, 14A, 14B, 15 and the like, for example.

  Further, the copy source apparatus 101 according to the present embodiment determines a buffer set immediately before data transfer from the copy source apparatus 101 to the copy destination apparatus 111. That is, immediately before data transfer, the copy source apparatus 101 associates the buffer of the copy source apparatus 101 storing the transfer target data with the buffer of the transfer destination copy destination apparatus 111.

  As a result, the copy source apparatus 101 does not need to secure a buffer for the copy destination apparatus 111 that is a transfer destination when storing write data or the like in the buffer. Even if there is no empty buffer, it is possible to store write data or the like in the buffer of the copy source apparatus 101.

  When an empty buffer is generated in the copy destination apparatus 111, the copy source apparatus 101 determines a buffer set at this stage and transfers data to the copy destination apparatus 111. Therefore, even if the buffer provided in the copy destination apparatus 111 is smaller than the buffer provided in the copy source apparatus 101, it is possible to perform remote copy processing by effectively using the buffer of the copy source apparatus 101.

That is, even when the size of the buffer used for remote copy provided in the copy source apparatus 101 and the size of the buffer used for remote copy provided in the copy destination apparatus 111 do not match, the buffer is used effectively. Efficient remote copy processing can be performed.
(Overview of remote copy processing according to this embodiment)
The outline of the remote copy processing according to the present embodiment will be described with reference to FIGS. 3 to 6 and FIG. 8 are diagrams illustrating states of the recording buffer 201, the buffer set storage unit 202, and the unused buffer ID storage unit 203 in the copy source apparatus 101 or the copy destination apparatus 111 at the time of remote copy processing. is there.

  When the copy source apparatus 101 receives, for example, a write I / O command from the host 109, the control module 00, 01, 02, or 03 that has received the write I / O command controls the control according to the write I / O command. Write data is stored in the disk devices 108a, 108b, 108c or 108d provided in the module.

  At the same time, the control module 00, 01, 02 or 03 that has received the write I / O instruction stores the write data received together with the instruction in a predetermined generation of the buffer 201a, 301a, 401a or 501a as shown in FIG. In addition, the index assigned to the generation storing the write data is stored in the BIT 201b.

Further, the copy source apparatus 101 sets only the copy source ID in the buffer set storage unit 202 and creates a buffer set in which the copy destination ID is indeterminate.
On the other hand, when the copy source apparatus 101 requests the copy destination apparatus 111 to notify the unused buffer and receives a notification from the copy destination apparatus 111 regarding the unused buffer, The unused buffer ID corresponding to the used buffer is stored in the unused buffer ID storage unit 203.

  As described above, in the remote copy processing according to the present embodiment, when creating a buffer set, only the copy source ID is set in the buffer set storage unit, and no copy destination ID is set. In other words, the matching process between the copy source buffer and the copy destination buffer is not performed when the buffer set is created, but is performed immediately before the buffer data is transferred to the transfer destination, as will be described later.

  As a result, the buffer provided in the copy source apparatus 101 can be used effectively. As a result, even if the buffer provided in the copy destination device is smaller than the buffer provided in the copy source device, the buffer of the copy source device can be used effectively.

  As shown in FIG. 4, (a) the copy source apparatus 101 switches the next buffer set to the storage target when a fixed time has passed or no data storage area exists in the buffer set to be stored. Switch ”. When buffer set switching is performed, the old buffer set becomes a transfer target.

  Further, (b) the copy source apparatus 101 assigns an unused copy destination ID acquired from the unused buffer ID storage unit 203 to the buffer set to be transferred. Then, the assigned buffer ID is deleted from the unused buffer ID storage unit 203.

  Specifically, the copy source apparatus 101 refers to the unused buffer ID storage unit 203 when switching buffer sets, and acquires as many unused buffer IDs as necessary. Then, the copy source apparatus 101 sets the acquired unused buffer ID as the copy destination ID of the buffer set storage unit 202 in the buffer set to be transferred by the buffer set switching.

When (c) buffer set setting (the above-described processes (a) and (b)) is completed, the copy source apparatus 101 transfers information stored in the buffer in units of buffers.
The copy source apparatus 101 deletes the unused buffer ID acquired from the unused buffer ID storage unit 203 from the unused buffer ID storage unit 203. If a necessary number of unused buffer IDs cannot be acquired from the unused buffer ID storage unit 203, the buffer transfer process is suspended until a required number of unused buffer IDs can be acquired.

In the remote copy processing according to the present embodiment, the following processing is performed in addition to the processing described above.
The copy source apparatus 101 according to this embodiment monitors the state of the network 501 between the copy source apparatus 101 and the copy destination apparatus 111. Then, as shown in FIG. 5, when it is detected that the data transfer path between the apparatuses is blocked (hereinafter referred to as “path blocked state”), the copy source apparatus 101 (a) Holds the combination of the buffer set. That is, the state of the recording buffer 201 and the buffer set storage unit 202 is held. Also, the copy source apparatus 101 clears all unused buffer IDs stored in (b) the unused buffer ID storage unit 203.

  Further, when the copy source apparatus 101 receives a write I / O command from the host 109 while the path is blocked, (c) accepts the write I / O command as much as possible. The copy source apparatus 101 stores the write data in the recording buffer 201, 301, 401, or 501.

  For example, when a write I / O command from the host 109 is received while the path is blocked, the copy source apparatus 101 stores data in the buffer of each control module. For example, in the control module a shown in FIG. 5, data is sequentially stored in the buffers 0002, 0003,. If the data is completely stored in the buffer of the recording buffer of any control module while the path is closed, the copy source apparatus 101 writes the data stored in the buffer back to the bitmap of each session according to the BIT information. Buffer halt processing is performed.

Therefore, if the path blockage state is resolved before the recording buffer 201 is depleted, the influence on the host 109 can be suppressed.
As a result, even if a temporary path blockage occurs, as much write I / O as possible can be stored in the copy source apparatus 101. If the buffer is not used up until the path is restarted, Since it is possible to resume the transmission of data stored in the buffer while the path is closed when the transfer is resumed, it is possible to continue the remote copy in which the order is guaranteed.

  In addition, the copy source apparatus 101 sets (d) a buffer set that has not been released as a buffer transfer is complete or is being transferred when a path blockage state is detected, and retransfers the buffer set after the path blockage state is resolved. Do.

  Similar to the copy source apparatus 101, the copy destination apparatus 111 monitors the state of the network 601 between the copy source apparatus 101 and the copy destination apparatus 111. As illustrated in FIG. 6, when the path blockage state is detected, the copy destination apparatus 111 does not perform (a) the expansion process of the generation waiting for expansion. For example, for the buffer set “in transfer” or “transferred and waiting for release”, the data stored in the buffer in the recording buffer 601, 701, 801 or 901 is not expanded in the disk device, and the buffer set storage unit The contents of 602, 702, 802 or 902 are held, and the copy source apparatus 101 waits for transfer again while maintaining the matching with the copy source ID. Then, the process waits until an “unused buffer notification request command” for requesting notification of an unused buffer ID is issued from the copy source apparatus 101.

  In addition, the copy destination apparatus 111 performs (b) the generation that was performing the expansion process when the path blockage was detected until the expansion process is completed. In other words, when the path blockage state is detected, the process of expanding the data in the recording buffers 601, 701, 801 and 901 to the disk device is continued until completion.

  The copy destination apparatus 111 shown in FIG. 6 may also include the unused buffer ID storage unit 203, but is not shown for the sake of simplicity. However, the omission of illustration is not intended to limit the present invention to the configuration of FIG.

FIG. 7 is a diagram showing a remote copy process between the copy source apparatus 101 and the copy destination apparatus 111 when the path blocking state is resolved.
(A) When the path blocking state is resolved, the copy source apparatus 101 issues a buffer notification request command to the copy destination apparatus 111. At this time, the copy source apparatus 101 adds information on the presence / absence of execution of the buffer halt processing in the path blocked state to the buffer notification request command.

  (B) Upon reception of the buffer notification request command, the copy destination apparatus 111 refers to the presence / absence of execution of the buffer halt processing notified together with the command. When the buffer halt process is performed in the copy source apparatus 101, the copy destination apparatus 111 also performs the buffer halt process and returns an error response to the copy source apparatus 101. When the buffer halt process is completed, the copy destination apparatus 111 again waits to receive a buffer notification request command from the copy source apparatus 101.

  On the other hand, when the response from the (b ′) copy destination apparatus 111 is an error response, the copy source apparatus 101 performs a retry process. That is, the buffer notification request command is issued again to the copy destination apparatus 111.

  (C) When the buffer halt process is not performed in the copy source apparatus 101, the copy destination apparatus 111 issues a buffer information notification command to the copy source apparatus 101, and the buffer information such as an unused buffer ID is copied to the copy source apparatus 101. The device 101 is notified and the remote copy process is resumed.

  (D) The copy source apparatus 101 stores the notified buffer information in the unused buffer ID storage unit and restarts the consistency operation. That is, the copy source apparatus 101 stores the unused buffer ID notified from the copy destination apparatus 111 in the unused buffer ID storage unit 203. When the copy source apparatus 101 confirms that there is no contradiction between the notified unused buffer ID and the copy destination ID in the buffer set stored in the buffer set storage unit 202 as a retransfer target, the remote copy The process is resumed and a normal response is returned to the copy destination apparatus 111.

FIG. 8 shows the states of the state recording buffer 201, buffer set storage unit 202, and unused buffer ID storage unit 203 of the copy source apparatus 101 at this time.
The copy source apparatus 101 stores the unused buffer ID in the unused buffer ID storage unit 203 when the path blockage state is resolved and the unused buffer ID is notified from the copy destination apparatus 111.

  Here, (a) a data transfer is completed when a path blockage is detected, but an unreleased buffer, or an unreleased buffer during transfer or the like is set as a target of the retransfer process. Since it becomes the object of the retransfer process, the area that is the object of the retransfer process is not released and the contents of the recording buffer 201 are retained.

  In FIG. 8, for example, buffer sets stored in the buffer set storage units 202, 302, 402, and 502, respectively, where (copy source ID−copy destination ID) is (0000-1000), (0100-1100), A combination buffer set consisting of (0200-1001) and (0300-1101), and a combination buffer set consisting of (0001-1002), (0101-1102), (0201-1003) and (0301-1103), Indicates a case where is subject to retransmission.

  Here, 0000 and 0001, 0100 and 0101, 0200 and 0201, 0300 and 0301 are buffer IDs of the buffers 201a, 301a, 401a and 501a included in the control modules 00, 01, 02 and 03 of the copy source apparatus 101, respectively. is there.

  Further, 1000 and 1002, 1100 and 1102, 1001 and 1003, 1101 and 1103 are buffers 601a, 701a, 801a and 901a respectively included in the control modules 00, 01, 02 and 03 provided in the copy destination apparatus 111 shown in FIG. Buffer ID.

A part of the buffer ID is not shown in the recording buffers 201, 301, 401, and 501 shown in FIG. The buffer set storage units 202, 302, 402, and 502 store a copy source ID (eg, 0002, 0102, 0202, 0302) as a part of the buffer set.

  (B) When the path resumption is detected, the copy source apparatus 101 refers to the unused buffer ID storage unit 203 and stores the unused buffer ID included in the unused buffer ID storage unit 203 and the buffer set storage unit 202. It is confirmed that the copy destination ID in the retransfer target buffer set is not duplicated.

  If it is determined that a contradiction has occurred due to duplicate unused buffer IDs, the copy source apparatus 101 acquires a new unused buffer ID from the copy destination apparatus 111 so that no contradiction occurs from the unused buffer ID storage unit 203. Set to the copy destination ID.

  (C) When the unused buffer ID notified from the copy destination apparatus 111 is the same as that before the path is blocked, that is, the unused buffer ID included in the unused buffer ID storage unit 203 in the buffer set to be retransferred. If there is no contradiction such as an overlap between the copy destination ID in the buffer set to be retransferred stored in the buffer set storage unit 202, the copy source apparatus 101 performs transfer processing (remote copy processing) as it is. ) To resume. Even when the contradiction is resolved, the copy source apparatus 101 resumes the transfer process.

  9A and 9B are flowcharts showing an overview of the remote copy processing according to the present embodiment. The outline of the remote copy processing according to this embodiment will be described below with reference to FIGS. 9A and 9B.

In step S900a, when the remote copy process is started, the copy source apparatus 101 moves the process to step S901a.
In step S901a, the copy source apparatus 101 performs buffer initial configuration processing. The copy source apparatus 101, for example, secures areas having the configuration shown in FIG. 2 in the memory 200 provided in each control module according to configuration information set in advance, and initializes each area.

In step S902a, the copy source apparatus 101 issues an unused buffer notification request command to the copy destination apparatus 111 in order to request notification of an unused buffer.
On the other hand, in step S901b, the copy destination apparatus 111 performs buffer initial configuration processing in the same manner as in step S901a. Then, the copy destination apparatus 111 shifts the processing to step S902b and monitors the command until an unused buffer notification request command is received from the copy source apparatus 101 (S902b NO).

  In step S902b, when an unused buffer notification request command from the copy source apparatus 101 is detected (S902b YES), the copy destination apparatus 111 moves the process to step S903b. Then, the copy destination apparatus 111 searches for an empty buffer whose area has been released in the recording buffer 201 of the control module included in the copy destination apparatus 111. When the copy destination apparatus 111 detects an empty buffer, the copy source apparatus uses the buffer ID of the buffer as an unused buffer ID. 101 is notified.

  In step S903a, the copy source apparatus 101 performs buffer set initial creation processing. For example, when the write I / O command from the host 109 has already been received and the write data received together with the write I / O command is recorded in the buffer 201a of the recording buffer 201, the buffer data recorded in the buffer 201a is stored. A buffer set whose copy destination ID is undetermined with the buffer ID as the copy source ID is generated, and the generated buffer set is stored in the buffer set storage unit 202. In this step, as described with reference to FIG. 3, a buffer set including only the buffer ID of the copy source apparatus 101 is generated and stored in the buffer set storage unit 202.

  In step S903a, upon receiving notification of an unused buffer ID from the copy destination apparatus 111, the copy source apparatus 101 stores the unused buffer ID in the unused buffer ID storage unit 203. Then, the process proceeds to step S904a.

  In step S904a, for example, when the copy source apparatus 101 receives a write I / O command from the host 109, the copy source apparatus 101 stores the write data received together with the command in the buffer 201a of the recording buffer 201. Similarly to step S903a, a buffer set whose copy destination ID is undetermined with the buffer ID of the stored buffer as a copy source ID is generated, and the generated buffer set is stored in the buffer set storage unit 202.

  In step S905a, the copy source apparatus 101 determines whether or not the buffer set storage in the buffer set storage unit 202 has been completed for all the received Write I / O instructions. When determining that the storage of the buffer set in the buffer set storage unit 202 has not been completed, the copy source apparatus 101 repeats the process of step S905a (NO in step S905a).

  If it is determined in step S905a that all buffer sets have been stored in the buffer set storage unit 202 (YES in step S905a), the copy source apparatus 101 moves the process to step S906a.

  In step S906a, the copy source apparatus 101 refers to the unused buffer ID storage unit 203. Then, the copy source apparatus 101 determines whether or not there are a necessary number of unused buffer IDs. For example, it is determined whether or not there are unused buffer IDs that can set copy destination IDs in all buffer sets stored in the buffer set storage unit 202.

  If the necessary number of unused buffer IDs does not exist (NO in S906a), the copy source apparatus 101 repeats the process in step S906a until the necessary number of unused buffer IDs can be confirmed. If it is determined that there are a required number of unused buffer IDs, the copy source apparatus 101 moves the process to step S907a (YES in S906a).

  In step S907a, the copy source apparatus 101 acquires a necessary number of unused buffer IDs from the unused buffer ID storage unit 203. Then, the copy source apparatus 101 sets the acquired unused buffer ID as the copy destination ID of the buffer set set in the buffer set storage unit 202 (see, for example, FIG. 4). This process is called “matching process”.

  In step S908a, the copy destination apparatus 111 sets the copy source ID and the copy destination ID according to the buffer set stored in the buffer set storage unit 202, and the copy source ID set in the buffer set. The data stored in the buffer and its BIT are transmitted to the copy destination apparatus 111.

On the other hand, when the copy destination apparatus 111 receives a buffer set or the like from the copy source apparatus 101, the process proceeds to step S904b.
In step S904b, the copy destination apparatus 111 performs buffer set reception processing. For example, the copy destination apparatus 111 stores the received buffer set in the buffer set storage unit 202 and stores the data received together with the buffer set in the recording buffer 201.

  In step S905b, the copy destination apparatus 111 determines whether all the buffer sets have been received. If it is determined that all the buffer sets have not been received yet (NO in S905b), the copy destination apparatus 111 moves the process to step S904b and repeats the processes in steps S904b to S905b.

If it is determined in step S905b that all the buffer sets have been received (YES in S905b), the copy destination apparatus 111 moves the process to step S906b. In step S906b, the copy destination apparatus 111 stores the data in the buffer 201a of the recording buffer 201. It is determined whether or not the buffer set acquired in step S904b including the processed data can be expanded on the disk device of the storage device included in the buffer set. If it is determined that expansion is possible (S906b YES), the copy destination apparatus 111 moves the process to step S907b.

  In step S907b, the copy destination apparatus 111 expands the data acquired in step S904b on the disk apparatus provided in itself. When the expansion ends, the copy destination apparatus 111 moves the process to step S908b and notifies the copy source apparatus 101 that the expansion of the buffer set has been completed. Hereinafter, this notification is referred to as “buffer set expansion completion notification”.

  When the expansion of the buffer set to the disk device is completed, in step S909b, the copy destination device 111 performs a buffer set release process. For example, the areas of the buffer set storage unit 202 for storing the buffer set and the recording buffer 201 for storing the buffer ID of the buffer are released.

  In step S910b, similarly to step S901b, the copy destination apparatus 111 newly secures the areas of the buffer set storage unit 202 and the recording buffer 201 in the released area, and constructs the configuration shown in FIG. 2, for example.

  When the above processing ends, the copy destination apparatus 111 shifts the process to step S911b, and notifies the copy source apparatus 101 of the unused buffer ID of the buffer that has become newly usable by the processes of steps S909b to S910b.

  On the other hand, in step S909a, the copy source apparatus 101 determines whether or not a buffer set expansion completion notification from the copy destination apparatus 111 has been received. If the buffer set expansion completion notification is not received (NO in S909a), the process in step S909a is repeated. When the buffer set completion notification is received from the copy destination apparatus 111 (YES in S909a), the copy source apparatus 101 moves the process to step S910a.

  In step S910a, the copy source apparatus 101 performs buffer set release processing on the buffer set for which data transfer has been completed in step S908a, as in step S909b. In step S911a, the copy source apparatus 101 performs a buffer reconstruction process in the same manner as in step S901a.

When the above processing is completed, the copy source apparatus 101 ends the remote copy processing, shifts the processing to step S902a to continue the remote copy processing, or receives a new Write I / O command from the host 109. Then, the process proceeds to S904a and the like, and the remote copy process is continued. Further, when the above-described processing ends, the copy destination apparatus 111 ends the remote copy processing, or shifts the processing to step S902b or 904b and continues the remote copy processing.
(Specific processing of the remote copy function according to this embodiment)
In the remote copy processing according to the present embodiment shown in FIGS. 9A and 9B, the following processing will be described more specifically.

(1) Matching processing between the copy source buffer and the copy destination buffer at the time of buffer set transfer (step S907a).
(2) A process for determining a transfer process to be started when the route blockage state is resolved.

(3) I / O processing in a route blockage state.
(4) An unused buffer ID acquisition process and a buffer set transfer restart process after the path blockage state is resolved.

(5) Processing at the time of detecting a path blocking state in the copy destination apparatus.
(1) Matching process between copy source buffer and copy destination buffer at the time of buffer set transfer (step S907a)
FIG. 10 is a flowchart showing a specific process of the copy destination buffer allocation process (step S907a) shown in FIG. 9B. FIG. 10 shows specific processing for a control module (hereinafter referred to as “master control module”) that controls the entire control module provided in the copy source apparatus 101 and other control modules.

  In step S1001a, the master control module refers to the unused buffer ID storage unit 203. Then, it is confirmed whether or not the number of unused buffer IDs recorded in the unused buffer ID storage unit 203 is a necessary number as copy destination IDs in the buffer set of the buffer set storage unit 202.

  If the necessary number of unused buffer IDs cannot be confirmed in step S1001a (NO in S1001a), the master control module moves the process to step S1002a. In step S1002a, the master control module waits for an unused buffer notification from the copy destination apparatus 111. When the master control module receives the unused buffer notification, the process proceeds to step S1001a.

  If the necessary number of unused buffer IDs is confirmed in step S1001a (YES in S1001a), the master control module shifts the process to step S1003a.

  In step S1003a, the master control module obtains the necessary number of unused buffer IDs from the unused buffer ID storage unit 203. Then, the acquired unused buffer ID is set as the copy destination ID in the buffer set already set in the buffer set storage unit 202 in step S904a. Thus, matching between the copy source buffer and the copy destination buffer is performed.

When this matching process is completed, the master control module notifies the other control modules of a buffer set information multiplexing request.
On the other hand, in step S1001b, upon receiving a buffer set information multiplexing request notification from the master control module, each control module performs buffer set information multiplexing processing. Here, the multiplexing process of buffer set information is a process of holding the same buffer set information in all the control modules provided in the copy source apparatus 101 or the copy destination apparatus 111.

  For example, the buffer set that has undergone the matching process in step S1003a is acquired from the master control module. Then, the acquired buffer set is reflected in the buffer set storage unit 202 provided in itself.

  When the buffer set information multiplexing process in S1001b is completed, each control module notifies the master control module of the completion of the buffer set information multiplexing process.

  In step S1005a, the master control module confirms whether or not responses have been received from all of the control modules that have requested the buffer set information multiplexing process. If it is determined that the response has been received from all control modules (YES in S1005a), the copy destination buffer allocation process is terminated (step S1005a), and the process proceeds to the process in step S908a shown in FIG. 9B.

  As described above, in the conventional remote copy process, the matching process between the copy source ID and the copy destination ID is performed when the buffer set is created. However, in the remote copy process according to the present embodiment, the process of step S904a is performed. It is not performed at the timing, but is performed immediately before the transfer process is performed after the buffer storage process is completed (step S907a).

  For this reason, since data storage processing can be performed in parallel with the transfer processing for buffers other than those related to the transfer processing, all the buffers including the recording buffer 201 and the buffer set storage unit 202 provided in the copy source apparatus 101 are used. It becomes possible. For example, data related to Write I / O from the host 109 can be stored until the buffer capacity of the copy source apparatus 101 runs out.

  As a result, regarding the buffers used for remote copy that guarantees the order, even if the copy destination apparatus 111 has fewer buffers than the copy source apparatus 101, the buffers of the copy source apparatus are effectively used. It becomes possible to do.

(2) Processing for determining transfer processing to be started when route blockage state is resolved In the remote copy processing according to the present embodiment, when the path blockage state is detected in copy source apparatus 101, if buffer halt processing has been executed. The buffer halt process is stopped, and the retransfer target when the path blockage state is resolved is determined according to the transfer status of each buffer set. In this embodiment, a buffer set that satisfies the following conditions is a retransfer target.

1) Previous generation buffer set that has been transferred 2) Old generation buffer set that is being transferred 3) Old generation buffer set that is waiting for transfer processing start In the remote copy processing according to this embodiment, the path blockage state is resolved In the retransfer of the buffer set at the time, the buffer set is transferred from the old generation.

FIG. 11 is a flowchart showing processing for determining the buffer set transfer processing after the path blockage state is resolved in the copy source apparatus 101.
In step S1100a, when a path blockage state is detected, the master control module in the copy source apparatus 101 moves the process to step S1101a.

  In step S1101a, the master control module refers to the recording buffer 201 or the buffer set storage unit 202. Then, the master control module checks whether there is an old generation buffer set that is 1) already transferred, 2) being transferred, or 3) waiting to start transfer processing.

  In step S1101a, when a buffer set corresponding to the condition 1), 2) or 3) is not detected (NO in S1001a), the master control module shifts the process to step S1104a and ends the process of FIG. If a corresponding buffer set is detected (S1101a YES), the master control module moves the process to step S1102a.

  In step S1102a, the master control module adds the corresponding buffer set detected in step S1101a to a queue used for buffer transfer order control (hereinafter referred to as “transfer queue”). At this time, queuing is performed so as to increase the priority of the transfer queue related to the old generation buffer set.

When the queuing process is completed, the master control module notifies other control modules to request multiplexing of the transfer queue information described above.
On the other hand, when receiving a request for multiplexing transfer queue information from the master control module, each control module acquires the transfer queue information from the master control module and reflects it in the transfer queue information provided in itself (S1101b).

When the transfer queue information is completely reflected by the processing of S1101b, each control module notifies the master control module that the transfer queue information has been multiplexed.
In step S1103a, the master control module confirms the completion notification of transfer queue information multiplexing from another control module. When the completion notification is received from all the control modules (YES in S1103a), the master control module shifts the process to step S1104a and ends the process of determining the buffer set transfer process after the path blockage state is resolved.

  When detecting that the route blockage state has been resolved, the master control module performs the buffer set retransfer process according to the queue information after performing the process described in FIG. 8, for example.

  The above-mentioned transferred buffer set indicates a buffer set that has been transferred to the copy destination apparatus 111 but has not received a buffer set expansion completion notification from the copy destination apparatus 111. In this case, it is necessary to wait for a buffer set expansion completion notification from the copy destination apparatus 111 without releasing the buffer.

(3) I / O processing in path blocked state In the conventional remote copy processing, when a path blocked state is detected in the copy source device, the data stored in the buffer by the buffer halt processing is changed according to the BIT information. I was writing back to the session bitmap. Then, when the path blockage state after the buffer halt processing is resolved, remote copy processing that does not guarantee the order according to the written back bitmap is performed. As for the write I / O processing from the host while the remote copy that does not guarantee the order is operating, the remote copy that does not guarantee the order is performed.

  In the remote copy processing according to the present embodiment, when the path blockage state is detected, the recording buffer 201 and the buffer set storage unit 202 (hereinafter simply referred to as “buffer”) can be stored without performing the buffer halt processing. Then, write I / O from the host is stored in the buffer as much as possible.

  However, when a new I / O or the like is received from the host in a state where there is no new data storage area in the buffer, a buffer switching request is made. If the buffer cannot be switched, a buffer halt process is performed.

FIG. 12 is a flowchart showing buffer state change processing when a path blockage state is detected in the copy source apparatus 101.
In step S1200a, when the path blockage state is detected, the master control module shifts the process to step S1201a.

  Here, the buffer according to the present embodiment transits at least three states (Halt, Active, and Buffering). For example, when a path blockage state is detected, the buffer enters a buffering state in conjunction with it. In the buffering state, data can be stored in the buffer, but data stored in the buffer cannot be transmitted. In the case of Halt, data cannot be stored in the buffer even if a write I / O command is received from the host 109. In the Active state, data can be stored in the buffer, and data stored in the buffer can also be transmitted.

  In step S1201a, the master control module changes the buffer state from the active state to the buffering state. Then, the master control module notifies the other control modules of the buffer status multiplexing request.

  On the other hand, when receiving a buffer status multiplexing request notification from the master control module, each control module proceeds to step S1201b. Then, its own buffer state is changed to the Buffering state.

  The control module that has finished changing the buffer state to the buffering state in S1201b notifies the master control module that the multiplexing of the buffer state has been completed.

  In step S1202a, the master control module confirms the completion notification of the buffer status multiplexing from the control module. When the completion notification is received from all the control modules (S1202a YES), the master control module shifts the process to step S1203a and ends the buffer state change process.

FIG. 13 is a flowchart showing I / O processing in the copy source apparatus 101 in the path blocked state.
When the write I / O command is received from the host 109 in S1300, the control module in the copy source apparatus 101 that has received the write I / O command shifts the processing to step S1301.

  In step S1301, the control module that has received the Write I / O command refers to the states of the recording buffer 201 and the buffer set storage unit 202 (hereinafter, both are simply referred to as “buffer” if necessary). Then, it is confirmed whether or not the buffer is in a state of storing data. If the buffer is in a state other than the active state or the buffering state (NO in S1301), the control module determines that the data cannot be stored in the buffer, and the process proceeds to step S1302. In step S1302, the control module records Write I / O in a bitmap prepared in advance in a memory provided in the control module.

  On the other hand, in step S1301, when the buffer state is the active state or the buffering state (YES in S1301), the control module determines that data can be stored in the buffer, and the process proceeds to step S1303.

  In step S1303, the control module checks whether there is an area where data can be stored in the buffer to be stored. If it is determined that there is no area where data can be stored (NO in S1303), the control module moves the process to step S1304. In step S1304, the control module requests the master control module to switch the buffer set.

  Upon accepting the buffer switching request in S1304, the master control module switches its own buffer set, and also notifies other control modules of buffer switching requests. The control module stores Write I / O in the switched buffer. If the buffer cannot be switched, buffer halt processing is performed.

  If it is determined in step S1303 that there is an area for storing data (YES in S1303), the control module moves the process to step S1305 to acquire an area for storing data. Then, the process proceeds to step S1306.

  In step S1306, a write I / O process by the host 109 is performed. At this time, the control module writes data to the disk device provided in the control module in accordance with the Write I / O command from the host 109. Then, the control module moves the process to step S1307 or S1308.

  For example, if there is no write data to be stored in the buffer, the process proceeds to step S1307. If there is write data to be stored in the buffer, the process proceeds to step S1308 to copy the write data to the buffer.

(4) Unused buffer ID acquisition processing and buffer set transfer resumption processing after route blockage state cancellation In the copy source apparatus 101, the route block state is detected, and the process (2) described in FIG. 12 and described in FIG. When the process of (3) is performed and the path blocking state is resolved, the following two patterns are transferred from the copy source apparatus 101 to the copy destination apparatus 111 in accordance with the state of the copy source apparatus 101 in the path blocking state. Perform buffer notification request processing.

1) When the buffer halt processing is not performed in the route blockage state.
2) When the buffer halt process is performed in the path blocked state.
In the case of the above pattern 2), since the buffer halt processing has already been performed in the copy source apparatus 101, the buffer is cleared. Therefore, the copy destination apparatus 111 also needs to clear the buffer by performing the buffer halt processing. As a trigger for performing the buffer halt process in the copy destination apparatus, a buffer notification request indicating that the buffer halt process has been performed in the copy source apparatus 101 is issued to the copy destination apparatus. Upon receiving the buffer notification request, the copy source apparatus 101 performs buffer halt processing of the copy destination apparatus while responding to the copy source apparatus with an error intended to be retried.

  The copy source apparatus 101 that has received an error from the copy destination apparatus issues a buffer notification request to the copy destination apparatus after a certain period of time has elapsed since the error reception. A buffer notification request is issued with a notification that the buffer halt processing has not been performed.

1) When Buffer Halt Processing is not Performed in Route Blocked State FIGS. 14A and 14B are flowcharts showing unused buffer ID acquisition processing and buffer set transfer restart processing after the route blocked state is resolved. Hereinafter, as necessary, the master control module of the copy source apparatus 101 is referred to as a “copy source master control module”, and the control module of the copy source apparatus 101 is referred to as a “copy source control module”. The master control module of the copy destination apparatus 111 is referred to as a “copy destination control module”.

In step S1400a, when the copy source master control module detects resumption of the path, the process proceeds to step S1401a.
In step S1401a, the copy source master control module issues a buffer notification request command to the copy destination apparatus 111. At this time, information indicating that the buffer halt processing is not performed is attached to the command.

  In step S1402a, the copy source master control module changes its buffer state from the Buffering state to the Active state. Further, it requests another copy source control module provided in the copy source apparatus 101 to change the buffer state from the Buffering state to the Active state.

  On the other hand, in step S1401b, upon receiving a buffer status change request from the copy source master control module, each copy source control module changes its buffer status from the Buffering status to the Active status, and indicates that the change has been completed. Notify the master control module.

  In step S1403a, the copy source master control module checks whether or not a notification of completion of buffer state change has been received from all requested control modules. When the notification is received from all requested control modules (YES in S1403a), the copy source master control module moves the process to step S1404a.

On the other hand, when the copy destination master control module receives a buffer notification request from the copy source apparatus 101 in step S1401c, the process proceeds to step S1402c.
The copy destination master control module confirms the buffer usage status and issues a buffer notification command in step S1402c to notify the copy source master control module of an unused buffer, that is, a released buffer.

In step S1404a, when receiving a buffer notification command from the copy destination apparatus 111, the copy source master control module shifts the processing to step S1405a.
In step S1405a, the copy source master control module refers to its own buffer and confirms whether there is a generation to be retransferred. If it is determined that there is no generation to be retransmitted (NO in S1405a), the copy source master control module moves the process to step S1406a and resumes normal operation. If it is determined that there is a generation to be transferred (S1405a YES), the copy source master control module shifts the processing to step S1407a.

  In step S1407a, the copy source master control module checks whether there is a contradiction between the buffer ID set in the buffer set of the generation to be retransferred and the buffer ID received in step S1404a. To do.

  For example, the copy source master control module checks whether or not the buffer ID set in the copy destination ID of the buffer set of the generation to be retransferred overlaps with the buffer ID received in step S1404a. If they overlap, it is determined that there is a contradiction (NO in S1407), the process proceeds to step S1408a, and the buffer halt process is executed.

  In step S1407a, when the buffer ID set in the buffer set of the generation to be retransferred and the buffer ID received in step S1404a do not overlap (YES in S1407a), the copy source master control module The processing moves to step S1409a and starts transfer processing.

2) When Buffer Halt Processing is Performed in Route Blocked State FIG. 15 is a flowchart showing unused buffer ID acquisition processing and buffer set transfer restart processing after the route blocked state is resolved.

In step S1500a, when the copy source master control module detects resumption of the path, the process proceeds to step S1501a.
In step S1501a, the copy source master control module issues a buffer notification request command to the copy destination apparatus 111. At this time, information indicating that the buffer halt processing has been performed is attached to the command.

  On the other hand, when the copy destination master control module receives the buffer notification request command sent from the copy source apparatus 101 in step S1501b, the process proceeds to step S1502b.

  In step S1502b, when the copy destination master control module detects from the information attached to the buffer notification request command that the buffer halt process has been performed in the copy source apparatus 101, it returns an error response to the copy source apparatus 101. . Then, the process proceeds to step S1503b, and the process ends.

  In step S1502a, when the copy destination master control module receives an error response from the copy destination device 111, the copy destination master control module waits for a predetermined time from the reception of the error response and then executes the process of step S1501a only once. Then, the process proceeds to step S1503.

(5) Processing at the time of detecting the path blocking state in the copy destination apparatus FIG. 16 is a flowchart showing the processing at the copy destination apparatus 111 when the path blocking state is detected.

  In step S1600, when the master control module of the copy destination apparatus 111 completes the process of expanding the data of the generation buffer in which the data already transferred from the copy source apparatus 101 is stored in the disk apparatus, the process proceeds to step S1601. Transition. Then, it is checked whether a path blockage has occurred.

  If a path blockage is detected in step S1601 (YES in S1601), the master control module shifts the process to step S1602, and waits for execution of the process shown in FIG. 14A and FIG. 14B or FIG.

If no path blockage is detected in step S1601 (NO in step S1601), the master control module shifts the process to step S1603 and stores it in the buffer of the next generation of the buffer that has been expanded on the disk. The process of expanding data to the disk device is started.
(Modified example of remote copy processing according to this embodiment)
The copy source device 3301 and the copy destination device 3302 shown in FIGS. 17 and 18 have the same configuration as the copy source device 3301 and the copy destination device 3302 shown in FIG. 33, but in order to facilitate the description of this modification example. The buffer set storage unit is not shown.

  In the remote copy function that guarantees the order, as shown in FIG. 17, (a) when a write I / O command is received from the host 109, each control module stores the write data in the disk devices 108a and 108b according to the command. To do. At the same time, (b) each control module transfers Write data and stores it in the buffer. At this time, the write data is managed in units of buffer sets.

  Then, (c) when the writing of the write data to the buffer is completed, the control module executes data transfer for each write data in units of buffer sets, and starts remote copy processing.

  (D) Write data sent from the copy source apparatus 101 by remote copy is stored in a buffer provided in each control module of the copy destination apparatus 111. Then, the control module reflects the write data stored in the buffer on the disk devices 1701a and 1701b.

When the above processing is completed, (e) the control modules of the copy source device 101 and the copy destination device 111 release the buffer and BIT.
However, as shown in FIG. 18, the buffer and BIT memory areas transferred by the copy source apparatus 101 are held without being released until the data is expanded on the disk apparatus by the copy destination apparatus 111. For this reason, (f) if a delay occurs in the transfer process due to a delay in the reflection process in the copy destination apparatus 111 or a line state between the casings, it is not possible to release the buffer or the memory storing the BIT in the copy destination apparatus 111.

  For example, if a delay occurs in the transfer process due to low bandwidth or instability between cabinets, the buffer area of the copy destination device 111 and the memory area of the BIT become longer in use, and the memory is released during that time. become unable.

  (G) When a write I / O command is received from the host to the copy source apparatus 101 in such a state, each time the command is received, the copy source apparatus 101 stores buffers one after another in order to store write data in the buffer. And consume.

  (H) It is conceivable that the memory area of the buffer capable of storing data cannot be secured, and the buffer of the copy source apparatus is depleted. In addition, even when the write data is larger than the set buffer size, it is possible that the buffer depletion similarly occurs.

  In the buffer depleted state, the write I / O from the host is stopped without being processed inside the copy source apparatus 101, and this state cannot be continued for a long time. For this reason, (i) I / O processing is temporarily stopped, and buffer halt processing is performed when the buffer exhaustion state is not resolved even after a predetermined time has elapsed. The buffer state is cleared by the buffer halt process and the I / O process is restarted.

  At this time, information such as Write data in the buffer is written back to a dedicated bitmap, and remote copy transfer that does not guarantee the order based on the bitmap is performed. Therefore, the remote copy that ensures the order is interrupted.

  Therefore, in the modification according to the present embodiment, an area for temporarily saving the buffer set on the memory is secured in the disk device. Hereinafter, the area secured in the disk device is referred to as “evacuation buffer”. Then, the buffer set is saved in the save buffer 190 according to the usage status of the buffer on the memory (hereinafter referred to as “cache buffer” as necessary). This series of processing is referred to as “buffer saving processing”.

  For example, in the copy source apparatus 101, when the free space of the buffer is reduced in the memory, the buffer set to be transferred is immediately transferred to the save buffer 190 without being transferred to the copy destination apparatus 111 and is stored in the memory. By ensuring a certain amount of free space in the buffer, it is possible to prevent the buffer from being exhausted even if a new Write I / O instruction is received.

  When the buffer set area is released by being transferred / reflected to the copy destination apparatus 111, the buffer set saved in the save buffer 190 is stored in the released buffer area, and the transfer process is continued. 19 to 21 show an outline of the buffer saving process.

19 and 20 are diagrams for explaining buffer save processing according to the present embodiment.
FIG. 19 is a diagram for explaining buffer save processing when a write I / O instruction is received in a state where the buffer set 1-4 is being transferred or is being used for waiting for expansion processing in the copy destination apparatus 111. It is.

  (A) Upon receiving a write I / O command, the control module stores write data in its own disk device, and (b) stores copy data of the write data in the buffer set 5. Hereinafter, the copy data of the write data in this case is also referred to as “write data”.

  Here, the control module has, for example, a buffer set usage rate (for example, “the number of generations of buffers used in the buffer set” / “the number of generations of all buffers”) of a certain value or more (in this embodiment, 50% or more). ), The control module saves, for example, data in the buffer set 5 scheduled to be transferred to the copy destination apparatus 111 to the save buffer 190 and stores a new write data storage destination from the buffer set 5. Switch to buffer set 6.

  Then, (d) when a new Write I / O command is received, the control module stores the Write data in the disk device and stores the Write data in the switched buffer set 6.

  However, since the buffer set older than the buffer set 6, in FIG. 19, the buffer set 5 is saved in the save buffer 190, (e) the control module saves the data in the buffer set 6 in the save buffer 190.

  Then, when the buffer set 1 in use is released, (f) the control module reads the data of the buffer set 5 stored in the save buffer 190 and stores it in the buffer set 5.

In the modification of the remote copy process according to the present embodiment, the following process will be described more specifically.
(1) Processing for creating a save buffer area.

(2) Allocation process for saving buffer to control module.
(3) Write back pointer information / stage pointer information update processing.
(4) Write back processing from the cache buffer to the save buffer.

(5) Staging processing from the save buffer to the cache buffer.
In this embodiment, storing the data stored in the cache buffer in the save buffer is called “write-back processing”, and storing the data stored in the save buffer in the cache buffer is called “staging process”. "

(1) Process for Creating Save Buffer Area FIG. 21 is a flowchart showing a process for creating a save buffer area according to this embodiment.

  In step S2101, the copy source apparatus 101 selects a buffer in which a save buffer is to be set according to an input from the user. Then, the copy source apparatus 101 shifts the processing to step S2102, and selects a disk apparatus for configuring a RAID according to an input from the user.

  In step S2103, the copy source apparatus 101 checks whether the disk apparatus selected in step S2102 satisfies the conditions for configuring the RAID. If the condition is not satisfied (NO in S2103), for example, the copy source apparatus 101 displays on the display device or the like that another disk apparatus should be designated, and the process proceeds to step S2102.

If the condition is satisfied in step S2103 (YES in step S2103), the copy source apparatus 101 moves the process to step S2104.
In step S2104, the copy source apparatus 101 generates a RAID group composed of the disk apparatuses selected in step S2102, and reflects the information in the configuration information that holds the apparatus configuration of the copy source apparatus 101.

  In step S2105, the copy source apparatus 101 creates a plurality of volumes in the RAID group created in step S2104. The copy source apparatus 101 moves the process to step S2106, and reflects the configuration of the logical unit created in step S2105 in the configuration information.

  In step S2107, the copy source apparatus 101 determines a volume to be used in each control module (hereinafter referred to as “in-charge volume” as necessary) by the processing shown in FIG. Each control module uses the determined volume as a save buffer area.

(2) Saving Buffer Allocation Processing to Control Module FIG. 22 is a flowchart showing specific processing of saving buffer allocation to the control module (step S2107) according to this embodiment.

In step S2201, the copy source apparatus 101 refers to the configuration information and acquires the number (A) of RAID groups to be allocated to the buffer.
Further, the copy source apparatus 101 shifts the process to step S2202, and acquires the number (B) of control modules installed in the copy source apparatus from the configuration information.

  In step S2203, the copy source apparatus 101 calculates the number of control modules per RAID group (C) = (B) / (A). Then, the process proceeds to step S2204, and (C) control modules are assigned to each RAID group.

  In step S2205, the copy source apparatus 101 divides the number of volumes in each RAID group by (C) to calculate the number of volumes in charge (D). Then, the copy source apparatus 101 shifts the processing to step S2206, and assigns (D) assigned volumes to the control modules assigned for each RAID group from the calculation result of step S2205.

  In step S2207, the copy source apparatus 101 reflects the configuration determined by the above processing in the configuration information and also in the setting information included in each control module, and ends the processing (step S2208).

FIG. 23 is a diagram illustrating a configuration example when the save buffer according to the present embodiment is configured by one RAID group.
A RAID group 2301 used as a save buffer shown in FIG. 23 includes logical units LU # 0 to LU # F. In addition, LU # 0-LU # 3, LU # 4-LU # 7, LU # 8-LU # B, and LU # C-LU # F surrounded by dotted lines in the horizontally long form volume groups, respectively. Configure. The RAID group 2301 is used in units of volume groups.

  A cache buffer and a volume group are assigned to each control module. For example, logical units LU # 0, # 4, # 8 and #C are allocated to the cache buffer of the control module # 00. Similarly, logical units LU # 1, # 5, # 9 and #D are allocated to the cache buffer of the control module # 01, and logical units LU # 2, # 6, and #D are allocated to the cache buffer of the control module # 02. #A and #E are assigned, and logical units LU # 3, # 7, #B and #F are assigned to the cache buffer of the control module # 03.

When the number of logical units constituting the volume group exceeds the number of mounted control modules, a plurality of logical units may be assigned to one control module.
Similarly to the cache buffer, the logical units constituting each volume group have the same size as one generation of the cache buffer and the area is divided for each generation. The data of the buffer set of each control module is, for example, As indicated by an arrow 23, the information is stored in an area of an arbitrary generation of the volume group assigned to each control module. FIG. 24 is a diagram illustrating a configuration example when the save buffer according to the present embodiment is configured by two RAID groups.

  The save buffer illustrated in FIG. 24 includes a RAID group 2401 including logical units LU # 0 to LU # F and a RAID group 2402 including logical units LU # 10 to LU # 1F.

  In the RAID group 2401, LU # 0, LU # 1, LU # 4, LU # 5, LU # 8, LU # 9, LU # C and LU # D, LU # 2, LU # 3, LU #. 6, LU # 7, LU # A, LU # B, LU # E and LU # F constitute a volume group.

  Similarly, in the RAID group 2402, LU # 10, LU # 11, LU # 14, LU # 15, LU # 18, LU # 19, LU # 1C and LU # 1D, LU # 12, LU # 13, LU # 16, LU # 17, LU # 1A, LU # 1B, LU # 1E, and LU # 1F constitute a volume group.

  A cache buffer and a volume group are assigned to each control module. For example, the cache buffer of the control module # 00 has a volume composed of logical units LU # 0, LU # 1, LU # 4, LU # 5, LU # 8, LU # 9, LU # C, and LU # D. A group is assigned. Similarly, the cache buffer of the control module # 02 includes logical units LU # 10, LU # 11, LU # 14, LU # 15, LU # 18, LU # 19, LU # 1C, and LU # 1D. A volume group is assigned.

  In addition, the logical units constituting each volume group are divided into areas for each generation, like the cache buffer, and the buffer set data of each control module is, for example, as shown by the arrows shown in FIG. It is stored in an area of an arbitrary generation of a volume group assigned to each control module.

  As shown in FIG. 24, when two or more RAID groups are allocated to the cache buffer, an effect of distributing the load of each RAID group can be obtained. FIG. 24 shows the case where two RAID groups are used, but it is natural that the same effect can be obtained even when two or more RAID groups are used.

  As described above, the save buffer according to this embodiment is used by dividing an area for each generation with the same size as the cache buffer. The outline is shown in FIG. The buffer set, BIT, and buffer stored in the cache buffer are stored in each generation of the save buffer.

(3) Write Back Pointer Information / Stage Pointer Information Update Processing In the buffer save processing according to this embodiment, a write back pointer that stores a position (generation) at which a write-back target generation buffer set is written back to the save buffer. The information (first pointer) and the stage pointer information (second pointer) for storing the generation from which data has been read from the save buffer and the staging process are stored are stored in the save buffer. The write position (generation) of data to be read and the read position (generation) of data read from the save buffer are managed.

FIG. 26 is a diagram for explaining write-back pointer information / stage pointer information in the buffer saving process according to the present embodiment.
For example, when the x-th buffer set switching is performed in the control module, the stage pointer information holds the last processed generation as information. The write-back pointer information holds the volume group generation x as information. Since whether or not to perform the buffer saving process varies depending on the usage state of the cache buffer, in this embodiment, a write area to the cache buffer is determined when the buffer set x is created.

  In addition, when the generation x buffer set is immediately transferred to the copy destination apparatus 111 at the time of the (x + 1) th buffer set switching in the control module, the data stored in the generation x is staged and the transfer process is completed. The pointer information is updated from generation x to generation x + 1. The write-back pointer information holds the volume group generation x + 1 as information. However, the write area to the cache buffer is determined when the buffer set x + 1 is created.

  Further, when generation x + 1 is written back to the save buffer at the time of x + 2 buffer set switching in the control module, if the generation of the volume group held in the stage pointer information is generation x, the next staging target is generation x + 1. Become. The write-back pointer information holds the volume group generation x + 2. However, the write area to the cache buffer is determined when the buffer set x + 2 is created.

  As shown in FIG. 25, in the write-back pointer information, the next generation of the created buffer set is held as information every time the buffer set is created. The stage pointer information is updated every time processing of the switched old generation buffer set is completed, for example, every time data transfer to the copy destination is completed and data expansion is completed at the copy destination. Is done. When there are multiple volume groups assigned as evacuation buffers, the volume groups are used in ascending order of the numbers assigned to the volume group. When the volume group is used up to the end, the volume group is returned to the first and cyclically. use.

Hereinafter, specific update processing of the write-back pointer information and the stage pointer information will be described with reference to FIGS.
FIG. 27 is a flowchart illustrating the write back pointer information update process according to the present embodiment.

In step S2700a, when the master control module of the copy source apparatus 101 starts buffer set switching processing, the processing proceeds to step S2701a.
In step S2701a, the master control module advances the write-back pointer information currently held by one. For example, the generation currently held is incremented by 1, and the next generation is held as information.

  In step S2702a, the master control module creates a new buffer set in the save buffer and sets the generation of the buffer set in the write-back pointer information. For example, the generation of step S2701a is set as the generation of the buffer set.

Further, the master control module requests other control modules included in the copy source apparatus 101 to perform the same process as in step S2702a.
On the other hand, in step S2701b, each control module creates a new buffer set in the save buffer, and sets the generation of the buffer set in the write-back pointer information. Then, when the setting of the write back pointer information is completed, the master control module is notified accordingly.

  In step S2703a, the master control module checks whether or not responses have been received from all requested control modules. If there is a control module that has not received a response (NO in step S2703a), the process of step S2703a is performed. repeat.

  If it is determined in step S2703a that responses have been received from all requested control modules (YES in S2703a), the master control module moves the process to step S2704a and ends the update process of the write-back pointer information.

FIG. 28 is a flowchart illustrating the stage pointer information update process according to the present embodiment.
In step S2800a, when the master control module of the copy source apparatus 101 starts the buffer set release process, the process proceeds to step S2801a.

  In step S2801a, the master control module determines whether there is a buffer set to be released. If there is no buffer set to be released (S2801a NO), the master control module moves to step S2804a and ends the process.

  In this embodiment, the buffer set that has been notified that the transfer to the copy destination device 111 has been completed and the transfer of the data transferred from the copy destination device 111 to the disk device has been completed is subject to area release. Judge.

If there is a buffer set to be released in step S2801a (YES in S2801a), the master control module moves the process to step S2802a.
In step S2802a, the master control module releases the buffer set (including BIT) to be released and its buffer area, and updates the stage pointer information. Then, the master control module requests other control modules included in the copy source apparatus 101 to perform the same process as in step S2802a.

  In step S2801, when receiving a request from the master control module, each control module releases the buffer set (including BIT) to be released and its buffer area, and updates the stage pointer information. When the processing is completed, each control module notifies the master control module of completion.

  In step S2803a, the master control module confirms responses from the requested control module, and determines whether or not responses have been received from all the control modules. If there is a control module that has not received a response (NO in S2803a), the process of step S2803 is repeated.

  If it is determined in step S2803a that responses have been received from all control modules (YES in S2803a), the master control module proceeds to step S2804a and ends the buffer set release processing.

(4) Writeback Processing from Cache Buffer to Saving Buffer FIG. 29 is a flowchart showing writeback processing according to this embodiment.
In step S2900a, for example, when the process of storing the write data in the buffer set is completed, the master control module in the copy source apparatus 101 moves the process to step S2901a.

  In step S2901a, the master control module acquires the number of generations of the buffer set being transferred to the cache buffer or waiting to be released (hereinafter referred to as “number of generations in use”). Then, the number of generations in use is compared with a predetermined threshold, for example, a value of 50% of the total number of generations. If the number of generations in use is less than the threshold (NO in S2901a), the process proceeds to step S2906a. If the number of generations in use is equal to or greater than the threshold (YES in S2901a), the process proceeds to step S2902a.

  In step S2902a, the master control module performs a write-back process for saving the buffer set, BIT, and buffer data of generations other than the generation in use to the save buffer assigned to the buffer set.

When the write-back process is completed, the master control module requests the same control as that in step S2902a to other control modules provided in the copy source apparatus 101.
In step S2901b, each control module saves the buffer set and BIT requested from the master control module in the save buffer assigned to the buffer set.

  In step S2902b, each control module checks whether or not the write data is stored in the buffer corresponding to the buffer set to be written back in S2901b, and if the write data is in the buffer (YES in step S2902b). Goes to step S2903b.

  In step S2903b, each control module saves the write data stored in the buffer to the save buffer assigned to the corresponding buffer set. Then, the master control module is notified of the completion of the requested process.

  On the other hand, in step S2903a, the master control module checks whether or not responses have been received from all requested control modules. If there is a control module that has not received a response (NO in step S2903a), the master control module performs step S2903a. Repeat the process.

  When responses are received from all the control modules (YES in S2903a), in step S2904a, the master control module releases the buffer set, BIT, and buffer areas. When the release of the buffer set or the like is completed, the master control module requests the same processing as step S2904a to the other control module provided in the copy source apparatus 101.

  On the other hand, in step S2904b, each control module releases the corresponding buffer set or the like in response to a request from the master control module. When the release of the buffer set or the like is completed, each control module notifies the master control module of completion.

  In step S2905a, the master control module checks whether or not responses have been received from all requested control modules. If there is a control module that has not received a response (NO in step S2906a), the process of step S2905a is performed. repeat.

  If it is determined in step S2905a that responses have been received from all requested control modules, the master control module moves the process to step S2906a and ends the buffer saving process.

30 and 31 are diagrams illustrating a specific example of the write-back process according to the present embodiment.
30 and 31 illustrate a case where the copy source apparatus 101 includes control modules # 0 to # 3 and each control module includes an 8th generation buffer, and the threshold described in FIG. 29 is 4th generation. It is a figure explaining a write-back process.

  Consider the case where (a) generations 1 to 4 of the buffer set are in transfer, waiting for transfer completion, or waiting for release, and (b) generation 5 is in write-back processing. In this case, the generation of the buffer set for storing (c) Write data is generation 6.

  As shown in FIG. 31, (d) when the area used by generation 1 is released, the master control module updates the stage pointer information and sets the data storage destination in the next staging process to generation 1 To do.

  Since generation 5 is now a write-back processing target, the staging target is also generation 5. When generation 5 write-back processing is in progress, (e) because there is data in the cache buffer, generation 5 write-back processing is interrupted and transfer processing of the generation 5 buffer set to the copy destination device 111 is started. To do.

  When generation 5 write-back processing is complete, staging processing is performed in which data of the generation 5 buffer set is read from the save buffer and stored in the generation 1 area.

(5) Staging Processing from Saving Buffer to Cache Buffer FIGS. 32A and 32B are flowcharts showing staging processing according to the present embodiment.

In step S3200a, when the old generation release process is completed, the master control module in the copy source apparatus 101 shifts the process to step S3201a.
In step S3201a, the master control module determines a generation to be staged. For example, the master control module refers to the stage pointer information, increments the generation stored in the stage pointer information by 1, and determines the incremented generation as a new generation to be staged.

  In step S3202a, the master control module checks whether the staging target area to be staged is waiting for release processing. If the staging destination area is awaiting release (S3202a YES), the master control module shifts the process to step S3203a.

  In step S3203a, the master control module sets a generation obtained by adding 1 to the staging target generation as a new staging target generation. Then, the master control module shifts the process to step S3202a.

  On the other hand, if the staging destination area is not waiting for release in step S3202a (NO in S3202a), the master control module shifts the process to step S3204a.

  In step S3204a, the master control module checks whether or not the staging destination buffer set is being transferred. When the buffer set of the staging destination is being transferred (S3204a YES), the master control module shifts the process to step S3203a.

  On the other hand, if the staging destination buffer set is not being transferred in step S3204a (NO in S3204a), the master control module shifts the processing to step S3205a.

  In step S3205a, the master control module checks whether or not the staging target is being staged. When the staging target is in the staging process (S3205a NO), the master control module shifts the process to step S3203a.

  On the other hand, in step S3205a, when the staging target is not in the staging process (S3205a YES), the master control module shifts the process to step S3206a.

  In step S3206a, the master control module checks whether the staging target is being written back. If the staging target is in the write-back process (S3206a YES), the master control module shifts the process to step S3207a.

  In step S3207a, the master control module interrupts its own write-back process and requests other control modules provided in the copy source apparatus 101 to interrupt the write-back process.

  In step S3201b, when receiving a request from the master control module, each control module interrupts the write-back process being processed. Then, the master control module is notified that the write-back process has been interrupted.

  In step S3208a, the master control module checks whether or not responses have been received from all requested control modules. If there is a control module that has not received a response (NO in step S3208a), the process of step S3208a is performed. repeat.

  If it is determined in step S3208a that responses have been received from all requested control modules (YES in S3208a), the master control module moves the process to step S3209a and ends the staging process.

  On the other hand, if it is determined in step S3206a that the staging target is not in the write-back process (NO in S3206a), the master control module moves the process to step S3210a.

  In step S3210a, the master control module checks whether or not the staging destination buffer set is in the process of storing write data or the like. If it is determined that the staging destination buffer set is being stored (YES in S3210a), the master control module shifts the process to step S3211a and ends the staging process.

  If it is determined in step S3210a that the staging destination buffer set is not being stored (YES in S3210a), the master control module shifts the process to step S3212a.

  In step S3212a, the master control module acquires a staging destination buffer set. Then, the master control module shifts the processing to step S3213a and checks whether or not staging target data is already stored in the stage processing buffer set.

  If there is data to be staged (S3213a YES), the master control module proceeds to step S3217a and ends the staging process. If there is no data to be staged (S3213a NO), the master control module moves the process to step S3214a.

  In step S3214a, the master control module acquires the generation data to be staged from the save buffer and stores it in the buffer set acquired in step S3212a. Then, the process proceeds to step S3215a.

  In step S3215a, the master control module requests each responsible control module to acquire the data of the generation to be staged from the save buffer and perform the staging process, as in step S3214a.

  In step S3202b, the control module that has received the request performs staging processing on the buffer set requested by the master control module. Then, the process proceeds to step S3203b.

  In step S3203b, each control module refers to the save buffer and determines whether there is write data to be staged in the buffer of the generation to be staged. If there is write data (YES in S3203b), each control module performs a staging process on the write data (S3204b). When the staging process is completed, each control module notifies the master control module of completion.

  In step S3203b, if there is no write data to be staged in the buffer of the generation to be staged (NO in S3203b), each control module notifies the master control module of completion.

  In step S3216a, the master control module monitors completion notification until responses are received from all requested control modules. And if a response is received from all the requested control modules (S3216a YES), the master control module moves the process to step S3217a and ends the staging process.

  As described above, in the remote copy processing according to the present embodiment, as described in step S907a, for example, immediately after the storage processing in the buffer is completed and the transfer processing is performed, the copy source ID of the buffer set is set. Since the matching process with the copy destination ID is performed and the buffer set is established, the write data related to the write I / O from the host 109 is stored in the buffer such as the recording buffer 201 and the buffer set storage unit 202 provided in the copy source apparatus 101. Can be stored until the capacity of the buffer runs out.

  As a result, regarding the buffers used for remote copy that guarantees the order, even if the copy destination apparatus 111 has fewer buffers than the copy source apparatus 101, the buffers of the copy source apparatus are effectively used. It becomes possible to do.

  Further, even when a problem occurs in the network between the copy source apparatus 101 and the copy destination apparatus 111 and the path is blocked, the write data can be stored in the buffer provided in the copy source apparatus 101 as long as the capacity allows. Therefore, if the path blockage state is eliminated before the buffer is depleted, there is an effect that it is not necessary to interrupt the remote copy processing that guarantees the order. As a result, there is an effect that the reliability and stability of the remote copy process is improved.

  Further, by performing the buffer saving process using the saving buffer 190 shown in the modification of the remote copy process in this embodiment, the buffer provided in the copy source apparatus 101 is not exhausted. -Even if a route is blocked due to some trouble in the network between the copy destination devices 111, it is possible to continue remote copy processing with guaranteed order as long as conditions such as restrictions on the host side allow. It has the effect of becoming.

Regarding the embodiment including the above Examples 1 to n, the following supplementary notes are further disclosed.
(Appendix 1)
A first storage device comprising a first storage volume for storing data transmitted from a host device and a first controller for controlling storage in the first storage volume; and the first storage device A storage device comprising: a second storage device that includes a second storage volume to which data stored in the storage device is copied via a network; and a second control unit that controls storage in the second storage volume. A system,
The first controller is
An identification information acquisition unit for acquiring reception buffer identification information used for identification of a reception buffer provided in the second storage device;
A transmission buffer for storing data to be transferred to the reception buffer of the second storage device;
A data transfer control unit that associates the transmission buffer and the reception buffer based on the transmission buffer identification information and the reception buffer identification information provided in the transmission buffer when performing the transfer process;
A storage system comprising: a transfer processing unit that transfers data in the transmission buffer to the reception buffer based on the association.
(Appendix 2)
The first storage device further comprises a save volume for temporarily saving data stored in the transmission buffer, and a save control unit for controlling data save to the save volume,
The appending control unit according to appendix 1, wherein when the use state of the transmission buffer reaches a predetermined state, the save control unit saves the data stored in the transmission buffer to the save buffer in units of generations. Storage system.
(Appendix 3)
The supplementary control unit according to appendix 2, wherein the save control unit saves data of a generation whose transfer preparation is completed and whose transfer order is low among the data stored in the transmission buffer to the save volume. Storage system.
(Appendix 4)
The control unit includes a first pointer indicating a write state of the transmission buffer,
The storage system according to appendix 3, wherein the save control unit saves data of a generation having a lower transfer order to the save volume based on the first pointer.
(Appendix 5)
A second pointer indicating a read state of the transmission buffer;
The storage system according to any one of appendices 1 to 4, further comprising a restoration control unit that restores the generation of data having a lower transfer order from the save volume based on the second pointer.
(Appendix 6)
When the generation data having a low transfer order stored in the transmission buffer is usable, the restoration control unit only transfers control data associated with the generation having a low transfer order from the save volume. The storage system according to any one of appendices 1 to 4, wherein the storage system is restored to the transmission buffer.
(Appendix 7)
A first storage volume for storing data transmitted from the host device;
A storage device comprising: a control unit that performs storage control to the first storage volume and copy control to a second storage volume connected via a network;
The controller is
An identification information acquisition unit for acquiring reception buffer identification information used for identification of a reception buffer provided in a reception-side storage device including the second storage volume;
A transmission buffer for storing data to be transferred to the reception buffer of the second storage device;
A data transfer control unit that associates the transmission buffer and the reception buffer based on the transmission buffer identification information and the reception buffer identification information provided in the transmission buffer when performing the transfer process;
A storage device comprising: a transfer processing unit that transfers data in the transmission buffer to the reception buffer based on the association.
(Appendix 8)
The transmission device further comprises a save volume for temporarily saving data stored in the transmission buffer, and a save control unit for controlling data save to the save volume,
The appending control unit according to claim 7, wherein the save control unit saves the data stored in the transmission buffer to the save buffer in units of generations when the use state of the transmission buffer reaches a predetermined state. Storage device.
(Appendix 9)
The supplementary control 8 according to appendix 8, wherein the save control unit saves, to the save volume, data of a generation whose transfer preparation is completed and whose transfer order is low among the data stored in the transmission buffer Storage device.
(Appendix 10)
The control unit includes a first pointer indicating a write state of the transmission buffer,
The storage device according to appendix 9, wherein the save control unit saves the generation data having a low transfer order to the save volume based on the first pointer.
(Appendix 11)
A second pointer indicating a read state of the transmission buffer;
11. The storage device according to any one of appendices 8 to 10, further comprising a restoration control unit that restores the generation data having a lower transfer order from the save volume based on the second pointer.
(Appendix 12)
When the generation data having a low transfer order stored in the transmission buffer is usable, the restoration control unit only transfers control data associated with the generation having a low transfer order from the save volume. The storage device according to any one of appendices 8 to 10, wherein the storage device is restored to the transmission buffer.
(Appendix 13)
A storage device control method comprising: storage control to a first storage volume that stores data transmitted from a host device; and a control unit that performs copy control to a second storage volume connected via a network. There,
An identification information acquisition step of acquiring reception buffer identification information used for identification of a reception buffer provided in a reception-side storage device comprising the second storage volume;
A data storage step of storing in the transmission buffer data to be transferred to the reception buffer of the reception-side storage device;
A data transfer control step for associating the transmission buffer with the reception buffer based on the transmission buffer identification information and the reception buffer identification information provided in the transmission buffer when performing the transfer process;
A storage device control method comprising: a transfer processing step of transferring data in the transmission buffer to the reception buffer based on the association.
(Appendix 14)
A saving control step of temporarily saving data stored in the transmission buffer to a saving volume provided in the storage device;
14. The saving control step according to claim 13, wherein when the use state of the transmission buffer reaches a predetermined state, the saving control step saves the data stored in the transmission buffer in the saving buffer in units of generations. Storage device control method.
(Appendix 15)
Item 15. The supplementary note 15, wherein the save control step saves data of a generation whose transfer preparation is completed and whose transfer order is low among the data stored in the transmission buffer to the save volume. Storage device control method.
(Appendix 16)
16. The storage device according to appendix 15, wherein the saving control step saves the generation data having a low transfer order to the saving volume based on a first pointer indicating a writing state of the transmission buffer. Control method.
(Appendix 17)
Supplementary notes 14 to 16, further comprising: a restoration control unit that restores the generation data having a lower transfer order from the save volume based on a second pointer indicating a read state of the transmission buffer. Storage device control method.
(Appendix 18)
In the restoration control step, when generation data with a lower transfer order stored in the transmission buffer is usable, only the control data accompanying the generation with lower transfer order is transferred from the save volume. 17. The storage device control method according to any one of appendices 14 to 16, wherein the data is restored to the transmission buffer.

101 Copy source device 104a Memory 108a Disk device 109 Host 111 Copy destination device 190 Evacuation buffer 201 Recording buffer 201a Buffer 201b BIT
202 Buffer set storage unit 203 Unused buffer ID storage unit

Claims (6)

A first storage device including a first storage unit that stores data transmitted from a host device and a first control unit that controls storage in the first storage unit; and the first storage device. stored data is a storage system comprising a second storage device having a second storage unit are copied over the network to,
The first controller is
A transmission buffer having a plurality of areas for storing data to be transferred to the second storage device;
The data stored in a certain area of the transmission buffer is stored in the second storage device based on information that associates the area with any one of the reception buffers included in the second storage device . a transfer unit which performs transmission copy rolling with guaranteed ordering for,
When it is detected that the usage rate of the transmission buffer is equal to or greater than a predetermined value, the transmission buffer has a transmission buffer that temporarily saves data stored in the transmission buffer according to the save order information. a saving process of saving the data in any region of the one or more regions transfer waiting data is stored you, storage for storing data of a new transfer waiting in a region saved the data A processing unit that performs processing and a re-storing process that re-stores any of the saved data in the transmission buffer according to re-storing order information when the area in use of the transmission buffer is released When,
A storage system characterized by comprising: The save order information indicates the position of data to be stored in the save buffer;
Wherein the processing unit, the storage system according to claim 1, characterized in that to save the data in the order rather based on the save order information. Re-storing order information is re-storing order information indicating the position of data to be re-stored in the transmission buffer from the save buffer,
Wherein the processing unit, the storage system according to claim 1 or 2, characterized in that re-stored in the transmission buffer in the order rather based on the re-storing order information. When the data stored in the area included in the transmission buffer is usable, the processing unit transfers the data stored in the area included in the transmission buffer to the second storage device . The storage system according to claim 3, wherein the storage system is transferred. A first storage unit that stores data transmitted from the host device, a control unit that performs storage control to the first storage unit and copy control to other storage devices connected via a network;
A storage device comprising:
The controller is
A transmission buffer having a plurality of areas for storing data to be transferred to the other storage device;
The data stored in a certain area of the transmission buffer is stored in the other storage device based on information that associates the area with any one of the reception buffers included in the other storage device . a transfer unit which performs transmission copy rolling with guaranteed ordering,
When it is detected that the usage rate of the transmission buffer is equal to or greater than a predetermined value, the transmission buffer has a transmission buffer that temporarily saves data stored in the transmission buffer according to the save order information. a saving process of saving the data in any region of the one or more regions transfer waiting data is stored you, storage for storing data of a new transfer waiting in a region saved the data A processing unit that performs processing and a re-storing process that re-stores any of the saved data in the transmission buffer according to re-storing order information when the area in use of the transmission buffer is released When,
A storage device comprising: A control method for a storage device including a control unit that performs storage control to a first storage unit that stores data transmitted from a host device and copy control to another storage device connected via a network. And
The data stored in the region of the transmission buffer having a plurality of areas for storing data to be transferred to the other storage device, one of the receiving buffer to the a region other storage device has based on the information that associates region, performs transmission copy rolling was guarantee the order with respect to the other storage devices,
When it is detected that the usage rate of the transmission buffer is equal to or greater than a predetermined value, the transmission buffer has a transmission buffer that temporarily saves data stored in the transmission buffer according to the save order information. a saving process of saving the data in any region of the one or more regions transfer waiting data is stored you, storage for storing data of a new transfer waiting in a region saved the data performing a processing, when the area in use of the transmission buffer is released, and re-storing process of restoring any data that the save accordance restoring order information in the transmission buffer, and this And a storage device control method.