JP2019191801A - Storage system and control device - Google Patents

Abstract

To make data of a copy source device and a copy destination device equivalent in a short time during failover.SOLUTION: A control unit 23 stores first data transmitted from a control unit 13 by synchronous copying in a recording medium 21, and stores control information 24a indicating the first data in a divided area 22a. When notification of a buffer change is made, the control unit stores second data transmitted from the control unit 13 in the recording medium 21 after the notification, The control unit stores a control information 24b indicating the second data in a divided area 22b, and erases the control information 24a from divided area 22b upon receiving a completion notification. When the control device 10 stops after the buffer switching notification, and if the control information 24a remains in the divided area 22a, the control unit 23 reads the first data from the recording medium 21 based on the control information 24a and transmits the first data to the control device 30.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a storage system and a control device.

  Synchronous copying and asynchronous copying are known as remote copy methods for copying data recorded in a storage device to another storage device for backup. In synchronous copy, the copy source device stores data requested to be written by the host device, copies the data to the copy destination device, and notifies the host device of the completion of writing after the copy is completed. On the other hand, in the asynchronous copy, the copy source device notifies the host device of the completion of writing at the stage where the data requested to be written is stored, and copies the data to the copy destination device at a timing asynchronous with this.

  Synchronous copying has the property that data can always be duplicated, but the longer the distance between the devices, the longer the RTT (Round-Trip Time) and the worse the write response performance. For this reason, asynchronous copying is suitable for applications in which a copy destination apparatus is arranged at a long distance assuming a large-scale disaster over a wide area.

  In this connection, the first device that receives a data write request from the host device and performs the write control process performs a synchronous copy to the second device and an asynchronous copy from the first device to the third device. There has been proposed a storage system that performs the above. In this storage system, when the operation of the first device stops, the second device takes over the write control process according to the request from the host device, and asynchronous copy is performed from the second device to the third device.

  Also, in this storage system, asynchronous copying is performed in which the order of writing is guaranteed. The copy source apparatus for asynchronous copying stores the data of the first recording medium in the first recording-dedicated buffer, and when this buffer becomes full, the data in the buffer are collectively transmitted to the copy destination apparatus. The copy destination apparatus stores the received data in the second recording-dedicated buffer, and when the reception of the data is completed, the data in this buffer is expanded on the second recording medium.

  As another technique related to the above, when an instruction for forming an asynchronous copy pair using the first volume and the third volume is received, a second volume in which a high availability pair is formed with the first volume, There has been proposed a storage system that rejects a formation command when a standby pair with three volumes is not formed.

JP 2017-167602 A International Publication No. 2016/084231

  In the above storage system in which the first device performs the synchronous copy from the second device and the first device performs the asynchronous copy from the third device, the first device to the third device by asynchronous copy. During the data transfer, the operation of the first device may stop. In this case, the data held by the second device does not match the data held by the third device. Therefore, when the second device takes over the write control process, there is a problem that it is desired to make the data equivalent between the second device and the third device in the shortest possible time using the data held by the second device.

  In one aspect, an object of the present invention is to provide a storage system and a control device that can equalize each data of a copy source device and a copy destination device in a short time at the time of failover.

  In one scheme, a storage system having a first control device, a second control device, and a third control device is provided. In this storage system, the first control device writes a buffer having a first area and a second area, and data requested to be written by the host device to the first recording medium and performs a second control. Synchronous copy processing for transmitting to the device and notifying the host device of the completion of writing, and storing the data written to the first recording medium in the first area, and switching the buffer when the predetermined condition is satisfied. And a storage process for switching the storage destination in the buffer of data written to the first recording medium after the notification to the second area, and the first data stored in the first area. And asynchronous copy processing for transmitting the second data stored in the second area together to the third control device, to the third control device. That has the transmission of the first data is completed, the first control unit for transmitting a completion notice to the second control device.

  The second control device includes a control information storage unit having a third region associated with the first region and a fourth region associated with the second region, and a first control unit. The transmitted first data is stored in the second recording medium, the first control information indicating the first data is stored in the third area, and when the buffer switching is notified, the first data is notified after the notification. When the second data transmitted from the first control unit is stored in the second recording medium, the second control information indicating the second data is stored in the fourth area, and the completion notification is received, When the first control information is erased from the area 3 and the first control device stops after notification of buffer switching, the write request reception processing from the host device is taken over from the first control unit, and the write is requested. The third data is stored in the second recording medium, and then Having a second control unit for transmitting a third data at a predetermined timing to the third control device.

  The third control device stores the first data in the third recording medium when receiving the first data from the first control device, and receives the second data from the first control device when receiving the second data. Data is stored in the third recording medium, and when the third data is received from the second control device, the third control unit stores the third data in the third recording medium.

  The second control unit further determines the first control information based on the first control information when the first control device is stopped after the buffer switching notification and the first control information remains in the third area. The first data is read from the second recording medium, the read first data is transmitted to the third control device, and the third control unit further receives the first data from the second control device. 1 data is stored in the third recording medium.

  In one proposal, a control device having a buffer and a control unit is provided. In this control device, the buffer has a first area and a second area. The control unit writes the data requested to be written from the host device to the recording medium and transmits it to the first other control device, and notifies the host device of the completion of writing, and the data is written to the recording medium. When the data is stored in the first area and the predetermined condition is satisfied, the buffer switching is notified to the first other control device, and the storage destination in the buffer of the data to be written in the recording medium after the notification is set to the second area. Storage processing for switching to the first area and the first data stored in the first area are collectively transmitted to the second other control device, and the second data stored in the second area are collectively Asynchronous copy processing to be transmitted to two other control devices is executed, and when the transmission of the first data to the second other control device is completed, a completion notification is transmitted to the first other control device.

  When at least a part of the first data writing destination area overlaps with the second data writing destination area, the control unit sets the first data when storing the second data in the recording medium. It is determined whether the process of storing data from the recording medium into the first area is completed. If the process is not completed, the second data is stored in the recording medium after the process is completed.

  In one aspect, the data of the copy source device and the copy destination device can be equivalent in a short time during failover.

1 is a diagram illustrating a configuration example and a processing example of a storage system according to a first embodiment. It is a figure which shows the structural example of the storage system which concerns on 2nd Embodiment. It is a figure which shows the hardware structural example of CM. It is a block diagram which shows the structural example of the processing function with which each CM is provided. It is a figure for demonstrating the basic process sequence of an asynchronous copy. It is a figure which shows the information memorize | stored in the management information storage part of each CM. It is a figure for demonstrating a buffer set management table. It is a figure which shows the structural example of a buffer management table. It is a figure which shows the example of a write I / O process according to the write request from a host apparatus. It is a figure which shows the example of a switching process of a buffer set. It is a figure which shows the example of a transfer process of a write data transfer, and a buffer set. It is a figure which shows the release process example of a buffer set. It is a figure which shows the example of a process when a failover occurs. It is an example of a sequence diagram showing a procedure of write I / O processing. It is an example of the sequence diagram which shows the procedure of the switching process of a buffer set. It is an example of the flowchart which shows the procedure of a buffer storage process. It is an example of a sequence diagram showing a procedure of transfer / development / release processing. It is an example of the sequence diagram which shows the procedure of the resynchronization process after failover. It is an example of the flowchart which shows the procedure of the write I / O process in a copying state. It is an example of the flowchart which shows the procedure of the write I / O process in an active state.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
[First Embodiment]
FIG. 1 is a diagram illustrating a configuration example and a processing example of the storage system according to the first embodiment. The storage system shown in FIG. 1 includes control devices 10, 20, and 30. In addition, the control devices 10 and 20 can communicate with the host device 1.

  The control devices 10, 20, and 30 have recording media 11, 21, and 31, respectively. In FIG. 1, as an example, the recording media 11, 21, and 31 are mounted inside the control devices 10, 20, and 30, respectively, but may be connected to the outside of the control devices 10, 20, and 30, respectively. .

  The control device 10 controls access to the recording medium 11 in response to a request from the host device 1. Further, the control device 10 has a function of copying data written in the recording medium 11 to the recording media 21 and 31 of the control devices 20 and 30 for backup. Synchronous copy is executed from the control device 10 to the control device 20, and asynchronous copy is executed from the control device 10 to the control device 30.

  The control device 20 receives data requested to be written to the recording medium 11 from the control device 10 and writes the data to the recording medium 21. In addition, when the operation of the control device 10 stops due to the occurrence of a failure or the like, the control device 20 takes over the write control processing corresponding to the write request from the higher-level device 1 from the control device 10. That is, in this state, the control device 20 controls access to the recording medium 21 in response to a request from the higher-level device 1. At the same time, the control device 20 backs up the data written in the recording medium 21 to the recording medium 31 of the control device 30 by asynchronous copy.

The control device 30 writes the data transmitted from the control device 10 or the control device 20 in the recording medium 31.
The control device 10 further includes a buffer 12 and a control unit 13. The buffer 12 is secured in, for example, a RAM (Random Access Memory) included in the control device 10. The control unit 13 is realized as, for example, a processor included in the control device 10. The control device 20 further includes a control information storage unit 22 and a control unit 23. The storage area of the control information storage unit 22 is secured in, for example, a RAM included in the control device 20. The control unit 23 is realized as a processor included in the control device 20, for example. The control device 30 further includes a control unit 33. The control unit 33 is realized as a processor included in the control device 30, for example.

  In the buffer 12, data transmitted from the recording medium 11 to the control device 30 is temporarily stored. The buffer 12 has a plurality of divided buffer areas. In the example of FIG. 1, the buffer 12 has divided buffer areas 12a and 12b.

  The control information storage unit 22 has divided areas respectively associated with the divided buffer areas of the buffer 12. In the example of FIG. 1, the control information storage unit 22 has divided areas 22a and 22b associated with the divided buffer areas 12a and 12b, respectively. Control information is stored in the divided areas of the control information storage unit 22. The control information is information necessary for storing the data stored in the corresponding divided buffer areas in the recording media 11, 21, 31. For example, the control information indicates a write destination area for each piece of data stored in the corresponding divided buffer area.

The control unit 13 executes the following “synchronous copy processing”, “storage processing”, and “asynchronous copy processing”.
In the synchronous copy process, the control unit 13 writes the data requested to be written from the higher-level device 1 to the recording medium 11 and transmits the data to the control device 20 to notify the higher-level device 1 of the completion of writing. The control unit 23 stores the data transmitted from the control unit 13 in the recording medium 21 and stores control information indicating the data in the control information storage unit 22. Note that the completion of writing to the higher-level device 1 by the control unit 13 is performed after storage of data in the recording medium 21 and storage of control information in the control information storage unit 22 are completed. By the above synchronous copy process, at least the data stored in the recording medium 11 and the data stored in the recording medium 21 are always equivalent.

  In the storage process, the control unit 13 stores the data written in the recording medium 11 in the divided buffer area 12a. Then, when a predetermined condition is satisfied, the control unit 13 notifies the control device 20 of buffer switching, and also stores the storage destination of data written from the higher level device 1 to the recording medium 11 from the divided buffer region 12a to the divided buffer region 12b. Switch to. Therefore, in the storing process after the buffer switching notification, the data written in the recording medium 11 is stored in the divided buffer area 12b.

  Here, when the control unit 23 receives the data from the control unit 13 and stores it in the recording medium 21 before the buffer switch is notified, the control information 24 a corresponding to the data is stored in the control information storage unit 22. Store in the divided area 22a. On the other hand, when the control unit 23 receives data from the control unit 13 and stores the data in the recording medium 21 after the buffer switching is notified, the control information 24b corresponding to the data is divided into the divided areas of the control information storage unit 22. 22b. In this way, the control device 20 can distinguish and hold the control information corresponding to the data stored in the recording medium 21 before and after receiving the buffer switching notification.

  In the asynchronous copy process, the control unit 13 collectively transmits the data stored in the divided buffer area 12a to the control device 30. That is, after the storage of data in the divided buffer area 12a is completed, the control unit 13 transmits the data in the divided buffer area 12a to the control device 30. Further, the control unit 13 collectively transmits the data stored in the divided buffer area 12b to the control device 30.

  The control unit 33 receives the data in the divided buffer area 12 a transmitted from the control unit 13 and stores the received data in the recording medium 31. Further, the control unit 33 receives the data of the divided buffer area 12 b transmitted from the control unit 13 and stores it in the recording medium 31.

  According to such asynchronous copy, data is copied in units of divided buffer areas. Therefore, asynchronous copy in which the order of writing from the host device 1 is guaranteed is executed between the control device 10 and the control device 30.

  When the transmission of data to the control device 30 in the asynchronous copy process is completed, the control unit 13 transmits a completion notification to the control device 20. Upon receiving the completion notification, the control unit 23 deletes the control information 24a from the divided area 22a. As described above, the control unit 23 can recognize that the asynchronous copy process for the data corresponding to the control information 24a, that is, the data stored in the divided buffer area 12a, is completed by receiving the completion notification.

Next, the case where the control apparatus 10 stops will be described.
For example, when the control device 10 is stopped after notification of the buffer switching described above, the control unit 23 takes over the write request reception processing from the higher-level device 1 from the control unit 13 and records the data requested to be written. Store in the medium 21. Then, the control unit 23 transmits the data stored in the recording medium 21 to the control device 30 at a predetermined timing thereafter.

  At the same time, when the control device 10 is stopped after the buffer switching is notified, the control unit 23 transmits data from the recording medium 21 based on the control information 24a if the control information 24a remains in the divided area 22a. And the read data is transmitted to the control device 30. The control unit 33 receives this data from the control unit 23 and stores it in the recording medium 31.

  Here, if the control device 10 stops while the data in the divided buffer area 12a is being transmitted from the control device 10 to the control device 30, the data in the recording medium 11 and the data in the recording medium 31 may not match. There is. On the other hand, the control unit 23 controls the data in the divided buffer area 12a corresponding to the divided area 22a because the control information 24a remains in the divided area 22a when the control device 10 is stopped after the buffer switching is notified. It can be determined that there is a possibility of not being transmitted to the device 30. When the determination is made as described above, the control unit 23 reads the same data as the data in the divided buffer area 12 a from the recording medium 21 based on the control information 24 a and transmits the data to the control device 30. When the control unit 33 stores the transmitted data in the recording medium 31, the data on the recording medium 31 becomes equivalent to the data on the recording medium 21.

  By such processing, the control unit 23 copies only data that may not be stored in the recording medium 31 from the recording medium 21 to the recording medium 31 among the data stored in the recording medium 21. Can do. As a result, at the time of failover in which the control device 20 takes over the write control processing, the data of the control device 20 that is the copy source device of asynchronous copy and the control device 30 that is the copy destination device can be made equivalent in a short time. it can.

  The process of storing data in the recording medium 21 in response to a write request from the host apparatus 1 is executed after the process of transmitting data based on the control information 24a from the recording medium 21 to the control apparatus 30 is completed, for example. Also good. Alternatively, these processes may be executed in parallel. In the latter case, it is possible to shorten the time from when the control device 10 is stopped until the write control process according to the request from the host device 1 is resumed.

[Second Embodiment]
FIG. 2 is a diagram illustrating a configuration example of a storage system according to the second embodiment. The storage system shown in FIG. 2 includes storage apparatuses 100, 200, 300 and a host apparatus 400. The host device 400 and the storage devices 100 and 200 are connected via, for example, a SAN (Storage Area Network) 51. The storage apparatus 100 and the storage apparatus 200 are connected through, for example, a dedicated communication line, and the storage apparatuses 100 and 200 and the storage apparatus 300 are connected through, for example, a WAN (Wide Area Network) 52. . Compared to the distance between the storage apparatus 100 and the storage apparatus 200, the storage apparatus 300 is arranged at a long distance.

  The storage apparatus 100 includes CM (Controller Module) 110 and 110a and drive units 120 and 120a. A plurality of storage devices are mounted on each of the drive units 120 and 120a. These storage devices are, for example, HDDs (Hard Disk Drives), and in this case, the drive units 120 and 120a are disk array devices. The CM 110 is a control device that controls access to the storage device in the drive unit 120, and the CM 110a is a control device that controls access to the storage device in the drive unit 120a.

  Similarly, the storage apparatus 200 includes CMs 210 and 210a and drive units 220 and 220a, and the storage apparatus 300 includes CMs 310 and 310a and drive units 320 and 320a. In the storage system of FIG. 2, CMs 110, 110a, 210, 210a, 310, 310a are connected to each other.

  The CM 110 can control access to a logical volume realized by a storage device in the drive unit 120 in response to a request from the host device 400. Similarly, the CM 110a can control access to a logical volume realized by a storage device in the drive unit 120a in response to a request from the host device 400. The CMs 110 and 110a perform access control using a distributed cache method in which caches used for accessing corresponding logical volumes are held respectively.

  The storage apparatus 200 is in a standby state (standby state) when the storage apparatus 100 is in an active state (operation state). In this state, the CMs 210 and 210a receive and back up the cache data in the CMs 110 and 110a, respectively. The cache data in the storage apparatus 100 is backed up to the storage apparatus 200 by synchronous copy. When a failure occurs in the storage apparatus 100, the storage apparatus 200 transitions from the standby state to the active state (that is, fails over), and takes over the access control to the logical volume according to the request from the host apparatus 400 in a short time. .

  The storage apparatus 300 also receives and backs up cache data in the storage apparatus 100 when the storage apparatus 100 is in an active state. Specifically, the CMs 310 and 310a receive and back up the cache data in the CMs 110 and 110a, respectively. However, the cache data in the storage apparatus 100 is backed up to the storage apparatus 300 by asynchronous copy. When a failure occurs in the storage apparatus 100, the storage apparatus 300 backs up the cache data in the storage apparatus 200 that has transitioned to the active state. Also at this time, the cache data in the storage apparatus 200 is backed up to the storage apparatus 300 by asynchronous copy.

  As described above, in the storage system shown in FIG. 2, when the storage apparatuses 100, 200, and 300 are operating normally, the backup data corresponding to the cache data is stored twice, thereby causing the cache data to be lost. Risk is reduced. In addition, when data is backed up by synchronous copying to a storage apparatus 300 located at a remote location, the RTT increases and the response speed to the host apparatus 400 is greatly reduced. However, according to the storage system of FIG. 2, since data is backed up to the storage apparatus 300 by asynchronous copy, such a decrease in response speed can be prevented. In other words, since the storage apparatus 300 can be remotely located without degrading the response performance with respect to the host apparatus 400, for example, the risk of data loss in the event of a large disaster that affects a wide range can be reduced. .

FIG. 3 is a diagram illustrating a hardware configuration example of the CM. In FIG. 3, the CM 110 is illustrated.
The CM 110 includes a processor 111, a RAM 112, an SSD (Solid State Drive) 113, a host interface 114, a drive interface 115, and remote adapters 116 and 117. These components are connected via a bus 118.

  The processor 111 controls the CM 110 as a whole. The processor 111 may be a multiprocessor. The processor 111 is, for example, a central processing unit (CPU), a micro processing unit (MPU), a digital signal processor (DSP), an application specific integrated circuit (ASIC), or a programmable logic device (PLD). Further, the processor 111 may be a combination of two or more elements among CPU, MPU, DSP, ASIC, and PLD.

  The RAM 112 is used as a main storage device of the CM 110. The RAM 112 temporarily stores at least part of an OS (Operating System) program and application programs to be executed by the processor 111. The RAM 112 stores various data necessary for processing by the processor 111.

  The SSD 113 is used as an auxiliary storage device for the CM 110. The SSD 113 stores an OS program, application programs, and various data. As the auxiliary storage device, another type of storage device such as an HDD may be used.

  The host interface 114 is an interface for communicating with the host device 400. The drive interface 115 is an interface for communicating with a storage device in the drive unit 120. The drive interface 115 is, for example, a SAS (Serial Attached SCSI, SCSI: Small Computer System Interface) interface. The remote adapter 116 is an interface for communicating with the storage apparatus 200, and the remote adapter 117 is an interface for communicating with the storage apparatus 300.

  The processing functions of the CM 110 are realized by the above hardware configuration. Note that the CMs 110a, 210, 210a, 310, and 310a are also realized by the same hardware configuration as in FIG.

FIG. 4 is a block diagram illustrating a configuration example of processing functions included in each CM.
First, the CM 110 includes a control unit 130, a cache 140, a buffer 150, and a management information storage unit 160. The processing of the control unit 130 is realized by the processor 111 executing a predetermined program. The cache 140 and the buffer 150 are realized by a part of the storage area of the RAM 112. The management information storage unit 160 is realized by a part of the storage area of the RAM 112 or the SSD 113.

  The control unit 130 uses the cache 140 to control access to the logical volume in response to a request from the host device 400. For example, the control unit 130 controls access to the logical volume using a write back method. In addition, the control unit 130 executes a copy process for the data 210 and 310 written in the cache 140. The control unit 130 performs synchronous copy for the CM 210 and performs asynchronous copy for the CM 310.

  For example, when the write back method is used, when the control unit 130 receives a write request for a logical volume, the control unit 130 stores data (write data) requested to be written in the cache 140. At the same time, the control unit 130 transmits the write data to the CM 210 and stores it in the cache of the CM 210 (a cache 240 described later). When the above processing is completed, the control unit 130 transmits a write completion notification to the host device 400. The control unit 130 writes the write data stored in the cache 140 to the storage device in the drive unit 120 at the subsequent asynchronous timing. Further, the control unit 130 transmits the write data stored in the cache 140 to the CM 310 at an asynchronous timing after the transmission of the write completion notification, and writes it in a cache of the CM 310 (a cache 340 described later).

  The cache 140 temporarily stores at least a part of data included in the logical volume to be accessed from the host device 400. The buffer 150 temporarily stores write data transferred from the cache 140 to the CMs 210 and 310. The buffer 150 is divided into a plurality of divided areas, and is used to execute asynchronous copy while maintaining the order of writing.

  The management information storage unit 160 stores various types of information referred to in the copy process. For example, a control information area corresponding to each divided area of the buffer 150 is set in the management information storage unit 160. Write control information corresponding to each write data stored in the cache 240 is registered in the control information area. The write control information is information indicating the write destination of the write data. The LUN (Logical Unit Number) for identifying the logical volume, the LBA (Logical Block Address) indicating the start address of the write destination in the logical volume, and the data length in block units. The block count indicated by is included. Such write control information is used to store write data in the buffer 150 at an asynchronous timing after storage in the cache 140.

  Although not shown, the CM 110a also has the same processing function as the CM 110. However, regarding the control of the entire remote copy processing such as buffer generation management, which will be described later, the CM 110 operates as a master and the CM 110a operates as a slave.

  The CM 210 includes a control unit 230, a cache 240, a buffer 250, and a management information storage unit 260. The processing of the control unit 230 is realized by a processor included in the CM 210 executing a predetermined program. The cache 240 and the buffer 250 are realized by a part of the storage area of the RAM included in the CM 210. The management information storage unit 260 is realized by a part of a RAM or SSD storage area provided in the CM.

  The control unit 230 executes a process of storing the write data transferred from the CM 110 in the cache 240 when the storage apparatus 100 is in an operating state. At this time, the control unit 230 registers the write control information for the write data stored in the cache 240 in the management information storage unit 260. The registered write control information is used to transmit to the CM 310 write data that is not stored in the cache of the CM 310 among the write data stored in the cache 240 after failover.

  Further, when the storage apparatus 100 stops due to the occurrence of a failure and the storage apparatus 200 transitions to an active state (failover), the control unit 230 takes over access control for the logical volume from the control unit 130 of the CM 110. At this time, the control unit 230 controls access to the logical volume according to the request from the host device 400 using the cache 240. At the same time, the control unit 230 executes an asynchronous copy process for asynchronously copying the write data of the cache 240 to the cache of the CM 310.

  The buffer 250 temporarily stores write data transferred from the cache 240 to the CM 310 when the storage apparatus 200 is in an active state. The management information storage unit 260 stores various types of information referred to in the copy process.

Although not shown, the CM 210a also has the same processing function as the CM 210.
The CM 310 includes a control unit 330, a cache 340, a buffer 350, and a management information storage unit 360. The processing of the control unit 330 is realized by a processor included in the CM 310 executing a predetermined program. The cache 340 and the buffer 350 are realized by a part of the storage area of the RAM provided in the CM 310. The management information storage unit 360 is realized by a part of a RAM or SSD storage area provided in the CM.

  The control unit 330 stores the write data transferred from the active CM among the CMs 110 and 210 in the cache 340. The cache 340 stores copy data corresponding to the write data stored in the cache 140 of the CM 110 or the cache 240 of the CM 210. The buffer 350 temporarily stores write data transferred from the CM 110 or the CM 210. The management information storage unit 360 stores various types of information referred to in the copy process, that is, the process of storing the received write data in the cache 340.

Although not shown, the CM 310a has the same processing function as the CM 310.
Next, asynchronous copy processing executed in the above storage system will be described. In the present embodiment, when the storage apparatus 100 is in an active state, asynchronous copy with guaranteed ordering is executed between the storage apparatus 100 and the storage apparatus 300.

  FIG. 5 is a diagram for explaining a basic processing procedure of asynchronous copying. In FIG. 5, processing related to synchronous copying between the storage apparatus 100 and the storage apparatus 200 is omitted.

  When the storage apparatus 100 is in the active state, the storage apparatus 100 becomes a copy source apparatus in asynchronous copy, and the storage apparatus 300 becomes a copy destination apparatus in asynchronous copy. As described above, the CM 110 of the storage apparatus 100 includes the cache 140 serving as a copy source and the buffer 150 used during asynchronous copy. Similarly, the CM 110a of the storage apparatus 100 includes a cache 140a serving as a copy source and a buffer 150a used for asynchronous copy.

  Further, as described above, the CM 310 of the storage apparatus 300 includes the cache 340 serving as a copy destination and the buffer 350 used during asynchronous copy. Similarly, the CM 310a of the storage apparatus 300 includes a cache 340a serving as a copy destination and a buffer 350a used for asynchronous copying.

  The buffers 150, 150a, 350, and 350a are each divided into the same number of divided regions. In FIG. 5, a number written in each divided area indicates an ID (Identification) of each divided area. In the following description, a divided area whose ID is “xxxx” is referred to as “divided area (xxxx)”.

  In the asynchronous copy process, one divided area is selected from each of the buffers 150, 150a, 350, and 350a, and the selected divided area is used to store write data written from the host device 400 in the same period. The Therefore, in the following description, each divided region may be identified by the concept of “generation”. In the example of FIG. 5, the divided areas (0000), (0100), (2000), and (2100) belong to the same generation. Further, the divided areas (0001), (0101), (2001), and (2101) belong to the same generation. The divided areas (0002), (0102), (2002), and (2102) belong to the same generation. The divided areas (0003), (0103), (2003), and (2103) belong to the same generation. A combination of divided areas belonging to the same generation is called a “buffer set”.

  When the CM 110 receives a write request from the host device 400, the CM 110 writes the write data transmitted from the host device 400 into the cache 140 (step S11a), and transmits a write completion notification to the host device. Similarly, the CM 110a writes the write data transmitted from the host device 400 to the cache 140a (step S11b), and transmits a write completion notification to the host device 400.

  The CM 110 stores (copies) the write data written in the cache 140 in the divided area (0000) at a timing asynchronous with the writing to the cache 140 (step S12a). Similarly, the CM 110a stores (copies) the write data written to the cache 140a in the divided area (0100) at a timing asynchronous with the writing to the cache 140a (step S12b).

  Thereafter, in the copy source storage apparatus 100, the buffer set is switched when the remaining capacity of at least one of the divided areas (0000) and (0100) runs out, or when a certain time elapses. The switching of the buffer set refers to switching of a divided area as a write data storage destination. When the buffer set is switched, the storage destination of the write data written in the caches 140 and 140a is switched to the divided areas (0001) and (0101) belonging to the next-generation buffer set, respectively. At the same time, the divided areas (0000) and (0100) belonging to the previous generation buffer set are to be transferred to the copy destination apparatus.

  When the remaining capacity of the divided area (0000) runs out or a certain time elapses, the CM 110 collectively transfers the write data stored in the divided area (0000) to the CM 310 (step S13a). Similarly, the CM 110a also collectively transfers the write data stored in the divided area (0100) to the CM 310a (step S13b).

  The CM 310 temporarily stores the write data received from the CM 110 in the divided area (2000), and when all the write data is stored, the write data stored in the divided area (2000) is written in the cache 340 (step S14a). Similarly, the CM 310a temporarily stores the write data received from the CM 110a in the divided area (2100). When all the write data is stored, the write data stored in the divided area (2100) is written in the cache 340a. (Step S14b). Hereinafter, writing from the buffers 350 and 350a to the caches 340 and 340a is referred to as “development”.

  When the expansion of the data stored in the divided areas (2000) and (2100) is completed, the buffer set to which the divided areas (0000), (0100), (2000), and (2100) belong is returned to the unused area. This return of the divided area to the unused area is called “release”.

  As described above, by using a buffer for copying and transferring write data between devices in units of buffer sets and developing write data from buffers to cache, the order of writing in units of buffer sets is guaranteed. The

  Incidentally, as described above, when a failure occurs in the storage apparatus 100, the storage apparatus 200 takes over access control in response to a request from the host apparatus 400 (failover). Here, between the storage apparatus 100 and the storage apparatus 200, the same data is always held in the cache by synchronous copying, and this point does not change even when a failure occurs in the storage apparatus 100. On the other hand, if a failure occurs in the storage apparatus 100 during data transfer by asynchronous copy between the storage apparatus 100 and the storage apparatus 300, the cache data is not equivalent between the storage apparatus 200 and the storage apparatus 300. Therefore, at the time of failover, it is possible to restart the access control according to the request from the host apparatus 400 in a short time while maintaining the cache equivalence between the storage apparatus 200 and the storage apparatus 300. It becomes one problem in.

  In the present embodiment, in the synchronous copy destination storage apparatus 200, write data from the copy source is written to the cache 240, and write control information indicating the write data is stored in the management information storage unit. In addition, buffer set switching and release processing is synchronized in the storage apparatuses 100, 200, and 300 so that the storage apparatus 200 can identify write control information corresponding to each buffer set before switching, after switching, and unreleased. To do.

  When a failure occurs in the storage apparatus 100, the storage apparatus 200 reads the write data corresponding to the write control information from the cache based on the write control information corresponding to the unreleased buffer set, and stores it in the storage apparatus 300. Forward. The storage apparatus 300 writes the transferred write data to the cache. Thereby, the cache data is made equivalent between the storage apparatus 200 and the storage apparatus 300. At this time, only write data that may not have been transferred from the storage apparatus 100 to the storage apparatus 300 is transferred from the storage apparatus 200 to the storage apparatus 300. Shortened.

  Further, the storage apparatus 200 takes over the access control to the logical volume according to the request from the host apparatus 400 from the storage apparatus 100 while executing the data transfer from the cache to the storage apparatus 300 as described above. Thereby, the access control according to the request from the host device 400 is resumed in a short time.

FIG. 6 is a diagram illustrating information stored in the management information storage unit of each CM.
The management information storage unit 160 of the CM 110 includes a session management table 161, a buffer set management table 162, a buffer management table 163, switching state information 164, storage buffer set information 165, a switching waiting queue 166, a storage completion waiting queue 167, and a copy bit. A map 168 is stored. Although not shown, similar information is also stored in the management information storage unit of the CM 110a.

  The management information storage unit 260 of the CM 210 includes a session management table 261, a buffer set management table 262, a buffer management table 263, switching state information 264, storage buffer set information 265, a switching waiting queue 266, a storage completion waiting queue 267, and A copy bitmap 268 is stored. Although not shown, similar information is also stored in the management information storage unit of the CM 210a.

  Further, the management information storage unit 360 of the CM 310 stores a session management table 361, a buffer set management table 362, and a buffer management table 363. Although not shown, similar information is also stored in the management information storage unit of the CM 310a.

  In the session management tables 161, 261, and 361, setting information related to a copy session set between storage apparatuses is registered. In the present embodiment, a synchronous copy session is set between the storage apparatus 100 and the storage apparatus 200. In addition, an asynchronous copy session is set between the storage apparatus 100 and the storage apparatus 300. Further, an asynchronous copy session is set between the storage device 200 and the storage device 300.

  In the session management tables 161, 261, and 361, for example, synchronous copy and asynchronous copy types in each copy session, information indicating a copy source device and a copy destination device, and a session status are registered. In the session status, information indicating the status of the copy session is registered.

  In a state where the storage apparatus 100 is normally operated, the session statuses of the synchronous copy session between the CM 110 and the CM 210 and the asynchronous copy session between the CM 110 and the CM 310 are “active”. In this state, the synchronous copy and the asynchronous copy in which the order of writing is guaranteed are normally executed. In addition, the session status of the asynchronous copy session between the CM 210 and the CM 310 becomes “standby” indicating that the CM 210 and the CM 310 are not activated.

  When a failover occurs, the session status of the asynchronous copy session between the CM 210 and the CM 310 is first changed to “copying”. In this state, the CM 210 records the copy location corresponding to the write control information in the copy bitmap 268 based on the write control information corresponding to the unreleased buffer set. Then, the CM 210 reads out data corresponding to the copy location recorded in the copy bitmap 268 from the cache 240 and transmits it to the CM 310. As a result, of the volume data, difference data that has not been transferred from the CM 110 to the CM 310 before the failover is transferred to the CM 310. When the data transfer for all the copy locations recorded in the copy bitmap 268 is completed, the session status is changed to “active”.

  When a write request is received from the host device 400 with the session status being “copying”, the following processing is performed. The CM 210 writes the write data to the cache 240 and records the copy location corresponding to the write request in the copy bitmap 268. This copy location is handled as the position of the data that has not been transferred to the CM 310, and the data corresponding to this copy location is transmitted to the CM 310 by the difference data transfer processing based on the copy bitmap 268 as described above. By using the copy bitmap 268, the CM 210 can execute a process of receiving the write request and writing the write data to the cache 240 and a process of transferring the difference data to the CM 310 in parallel. As a result, it is possible to shorten the time from when the storage apparatus 100 is stopped until the write control process according to the request from the host apparatus 400 is resumed.

  Then, when the transfer of the difference data is completed and the session status becomes “active”, the asynchronous copy in which the order of writing is guaranteed between the CM 210 and the CM 310 is started. That is, the CM 210 resumes the asynchronous copy equivalent to the CM 110.

  Information indicating the buffer set used in the asynchronous copy session is registered in the buffer set management tables 162, 262, and 362. In the buffer management tables 163 and 263, write control information indicating write data to be stored in each divided area in the buffer set is registered. In the buffer management table 363, write control information indicating the write data stored in each divided area in the buffer set is registered.

Here, FIG. 7 is a diagram for explaining the buffer set management table.
The buffer 150 of the CM 110 temporarily stores write data transferred from the CM 110 to the CM 310 when the CM 110 is in an active state. The buffer 250 of the CM 210 temporarily stores write data transferred from the CM 210 to the CM 310 when the CM 210 is in an active state. The write data transferred from the CM 110 or the CM 210 is temporarily stored in the buffer 350 of the CM 310. The buffers 150, 250, and 350 are each divided into the same number of divided areas. In addition, the number described in each division area in FIG. 7 shows ID of a division area.

  In the buffer set management table 162 of the CM 110, information indicating a buffer set used in asynchronous copy session processing is registered. The information indicating the buffer set includes the ID of the copy source divided area and the ID of the copy destination divided area. For example, in FIG. 7, the combination of the divided area (0000) as the copy source and the divided area (2000) as the copy destination and the combination of the divided area (0100) as the copy source and the divided area (2100) as the copy destination are the same generation. It is registered as a buffer set.

  In the buffer set management table 262 of the CM 210, information indicating a buffer set used in the asynchronous copy session processing is registered. For example, in FIG. 7, the combination of the divided area (1000) as the copy source and the divided area (2000) as the copy destination and the combination of the divided area (1100) as the copy source and the divided area (2100) as the copy destination are the same generation. It is registered as a buffer set.

  Between the buffer set management table 162 and the buffer set management table 262, a buffer set in which a divided area having the same ID is registered as a copy destination is registered, and such a buffer set is used as a buffer set of the same generation. The According to such an information registration method, the generation can be notified by transmitting the ID of the copy-destination divided area from the CM 110 to the CM 210. As a result, generation (buffer set) switching and release processing can be synchronized between the CM 110 and the CM 210.

  Although not shown, the CM 110a management information storage unit also holds a buffer set management table in which the same information as the buffer set management table 162 is registered. The management information storage unit of the CM 210a also holds a buffer set management table in which the same information as the buffer set management table 262 is registered.

FIG. 8 is a diagram illustrating a configuration example of the buffer management table. FIG. 8 illustrates the buffer management table 163 of the CM 110.
The buffer management table 163 is divided into control information areas 163a to 163d. The control information areas 163a, 163b, 163c, and 163d are associated with the divided areas (0000), (0001), (0002), and (0003) of the buffer 150, respectively. In each of the control information areas 163a to 163d, the write control information 163-1 indicating the write data stored in the corresponding divided area is registered with the storage state information 163-2 added.

  The write control information 163-1 is referred to store the corresponding write data from the cache 140 in the corresponding divided area in the buffer 150. The storage state information 163-2 is flag information indicating whether or not the write data corresponding to the write control information 163-1 has been stored in the buffer 150 from the cache 140. When the write control information 163-1 is registered in the control information area, the storage state information 163-2 indicating “not stored” is added to the write control information 163-1.

  Similarly to the buffer management table 163 of the CM 110, the buffer management table 263 of the CM 210 is also divided into control information areas corresponding to the divided areas of the buffer 250. In the control information area of the buffer management table 263, the same write control information as the control information area of the same generation in the buffer management table 163 is registered with storage state information added. When the CM 110 is in the active state and the CM 210 is in the standby state, the write control information registered in the control information area is referred to in order to transfer the corresponding write data from the cache 240 to the CM 310 when a failover occurs. On the other hand, the write control information registered in the control information area after the CM 210 changes to the active state is referred to in order to store the corresponding write data from the cache 240 in the corresponding divided area in the buffer 150.

  Similarly to the buffer management table 163 of the CM 110, the buffer management table 363 of the CM 310 is also divided into control information areas corresponding to the divided areas of the buffer 350. In the control information area of the buffer management table 363, write control information indicating the write data stored in the corresponding divided area is registered. The write control information is referred to in order to store (decompress) the corresponding write data in the cache 340 from the corresponding divided area in the buffer 350.

Hereinafter, returning to FIG.
The switching status information 164 and 264 is information indicating whether or not the buffer set switching processing is being executed. The switching state information 164 is used when the CM 110 is in an active state, and the switching state information 264 is used when the CM 210 is in an active state.

  The storage buffer set information 165 and 265 is information indicating a buffer set currently targeted for storage reservation. “Storage reservation” refers to registering write control information indicating write data to be stored in a divided area of the buffer in the control information area of the buffer management table corresponding to the divided area. The buffer sets indicated by the storage buffer set information 165 and 265 are associated with the divided areas included in the buffers 150 and 250, respectively. For example, in this embodiment, the buffer set to be reserved for storage is divided into areas (0000), (0001), (0002), (0003), (0000),.・ It can be switched in the order.

  The switching wait queues 166 and 266 temporarily hold write control information indicating the write request until the switching process is completed when a write request is received from the host device 400 during the switching process. This is a FIFO (First In First Out) type queue. The switching waiting queue 166 is used when the CM 110 is in an active state, and the switching waiting queue 266 is used when the CM 210 is in an active state.

  The storage completion waiting queues 167 and 267 are queues that temporarily hold the write control information. In the storage completion waiting queues 167 and 267, when a write request is received from the host device 400, write data is requested for an area overlapping the write destination area in the previous generation and has not yet been stored in the buffer. If there is, the write control information indicating the received write request is temporarily held until the storage is completed. The storage completion waiting queue 167 is used when the CM 110 is in an active state, and the storage completion waiting queue 267 is used when the CM 210 is in an active state.

  The copy bitmap 268 is used to transfer the write data of the unreleased buffer set from the cache 240 to the CM 310 when the CM 210 transitions to the active state due to failover. The copy bitmap 268 has a bit for each block of the copy processing unit in the logical volume. If there is write data to be transferred for a block corresponding to a bit, “1” is set to that bit. By using this copy bitmap 268, only the difference data can be transferred from the cache 240 to the CM 310 when the data in the caches 240 and 340 are made equivalent at the time of failover.

The copy bitmap 168 is used for the same purpose as the copy bitmap 268 when resuming the operation of the CM 110 after failback.
Next, an overview of copy processing in a state where the storage apparatus 100 is operating normally (active state) will be described with reference to FIGS.

First, FIG. 9 is a diagram illustrating an example of a write I / O (Input / Output) process in response to a write request from the host device.
As shown in FIG. 9, in the initial state, the host device 400 transmits a write request to the CM 110. Further, it is assumed that the divided area (0000) of the buffer 150 and the divided area (1000) of the buffer 250 are set as storage targets. That is, the divided areas (0000) and (1000) belong to the current generation buffer set.

  When the CM 110 receives a write request and write data from the host device 400 (step S11a), the CM 110 stores the write data in the cache 140 and transmits the write data to the CM 210 (step S11b). The transmitted write data is stored in the cache 240 of the CM 210.

  Further, the CM 110 registers the write control information indicating the write data write destination in the control information area corresponding to the divided area (0000) in the control information area of the buffer management table 163 (step S12a). At this time, storage state information indicating “not stored” is added to the write control information. The CM 220 registers the write control information indicating the write data write destination in the control information area corresponding to the divided area (1000) in the control information area of the buffer management table 263 (step S12b).

  When the above processing is completed, the CM 110 transmits a write completion notification to the host device 400. In this way, synchronous copying between the CM 110 and the CM 210 is realized, and the data written in the caches 140 and 240 are always equivalent. At the same time, the CMs 110 and 210 register write control information in the buffer management table, that is, reserve storage for buffer storage.

  In the CM 110, the following processing is executed at a timing asynchronous with the writing of the write data to the cache 140. As processing of the asynchronous copy session, the CM 110 stores (copies) the write data written in the cache 140 in the divided area (0000) of the buffer 150 based on the write control information registered in Step S12a (Step S13). When the storage is completed, the CM 110 updates the storage state information added to the write control information to “stored”. As described above, the storage process in the buffer 150 is executed asynchronously with the write process in the cache 140, so that the response time to the write request from the host apparatus 400 can be shortened.

  On the other hand, the CM 210 does not store write data in the buffer 250. That is, when the CM 210 is in a standby state, the buffer 250 is not used, and it is not necessary to reserve the buffer 250 area in the RAM of the CM 210.

  Although not shown, the host device 400 can also send a write request to the CM 110a. In response to this write request, the CMs 110a and 210a execute the same processing as the CMs 110 and 210, respectively.

FIG. 10 is a diagram illustrating an example of buffer set switching processing.
When the process of FIG. 9 is continued, the write data to be stored in the divided area (0000) of the buffer 150 increases. When the CM 110 receives a data write request from the host device 400 (step S21), the CM 110 determines that there is no free area in the divided area (0000) and the data cannot be stored. This determination is made based on the write control information registered in the control information area corresponding to the divided area (0000) of the buffer management table 163.

  At this time, the CM 110 notifies the CM 210 of buffer set switching (step S22). As a result, the divided area that becomes the storage destination of the write data received from the host apparatus 400 thereafter is switched between the CM 110 and the CM 210 in synchronization. In other words, the buffer set generation is switched between the CM 110 and the CM 210 in synchronization.

  After the switching notification, the write data received in step S21 is stored in the cache 140, transmitted to the CM 210, and stored in the cache 240 (step S23). Unlike before switching, the CM 110 registers the write control information indicating the write data write destination in the control information area corresponding to the divided area (0001) in the control information area of the buffer management table 163 (step S110). S24a). At this time, storage state information indicating “not stored” is added to the write control information. Further, the CM 220 registers the write control information indicating the write data write destination in the control information area corresponding to the divided area (1001) in the control information area of the buffer management table 263 (step S24b). When the above processing is completed, the CM 110 transmits a write completion notification to the host device 400.

  Further, the CM 110 stores (copies) the write data written in the cache 140 in the divided area (0001) of the buffer 150 based on the write control information registered in step S24a at a timing asynchronous with the above (step). S25). When the storage is completed, the CM 110 updates the storage state information added to the write control information to “stored”.

  In this way, the divided area that is the storage reservation destination of the write data is switched. Further, the notification of switching of the buffer set is also notified to the CMs 110a and 210a, and the divided areas as the data storage reservation destinations are also switched in the CMs 110a and 210a. Thereby, the generation of the buffer set of the data storage reservation destination is switched between the storage apparatuses 100 and 200 in synchronization.

  In the processing of FIG. 10, when the CM 110 stores the write data from the host device 400 in the cache 140, the old write data in the previous generation whose write data and write destination overlap are stored in the buffer 150. May not be completed. In this case, the CM 110 suspends storage of new write data in the cache 140 and registers the write control information indicating the write data in the storage completion waiting queue 167. As a result, the write I / O processing with the new write data enters a waiting state. Thereafter, when the storage of the old write data in the buffer 150 is completed, the CM 110 reads the write control information from the storage completion waiting queue 167, stores the write data in the cache 140 based on the write control information, and writes the write control information. Are registered in the control information area of the buffer management table 163. By such processing, the order of writing of write data can be accurately maintained between generations.

On the other hand, for the previous generation buffer set, the following processing shown in FIG. 11 is executed.
FIG. 11 is a diagram illustrating an example of a write data transfer and buffer set expansion process.
When all the data written in the cache 140 before the occurrence of switching is stored in the divided area (0000) of the buffer 150, the CM 110 transfers the data in the divided area (0000) to the CM 310 (step S31). . That is, the CM 110 collectively transfers the data in units of divided areas to the CM 310. At this time, the CM 110 transfers the ID “2000” of the storage area in the buffer 350 together with the write data. The CM 310 temporarily stores the received write data in the divided area (2000) of the buffer 350, and stores the write control information indicating the write data in the control information area corresponding to the divided area (2000) in the buffer management table 363. Register with. When these processes are completed, the CM 310 writes the write data stored in the divided area (2000) into the cache 340 (step S32). That is, the write data in the buffer 350 is expanded in the cache 340.

  Actually, the above processing is executed not only between the CM 110 and the CM 310 but also between the CM 110a and the CM 310a. That is, the write data is transferred from the buffer of the CM 110a, temporarily stored in the buffer of the CM 310a, and then developed in the cache of the CM 310a.

  Further, the write I / O processing by the CM 110 is continued in parallel with the above transfer and expansion processing. That is, when the CM 110 receives a write request and write data from the host device 400 (step S33a), the CM 110 stores the write data in the cache 140 and transmits the write data to the CM 210 (step S33b). The transmitted write data is stored in the cache 240 of the CM 210.

  Further, the CM 110 registers write control information indicating the write data write destination in the control information area corresponding to the divided area (0001) in the control information area of the buffer management table 163 (step S34a). On the other hand, the CM 210 registers the write control information indicating the write data write destination in the control information area corresponding to the divided area (1001) in the control information area of the buffer management table 263 (step S34b).

  When the above processing is completed, the CM 110 transmits a write completion notification to the host device 400. Further, the CM 110 stores the corresponding write data in the divided area (0001) of the buffer 150 based on the write control information registered in the control information area at an asynchronous timing thereafter (step S35).

  FIG. 12 is a diagram illustrating an example of buffer set release processing. When the development process shown in step S32 of FIG. 11 is completed for all the write data stored in the divided area (2000), the process shown in FIG. 12 is executed.

  The CM 310 notifies the CM 110 of the release of the buffer set (Step S41a). The CM 310 that has received the notification notifies the CM 210 of the release of the buffer set (step S41b). The CM 310 releases the divided area (2000) (step S42a), and the CMs 110 and 210 that have received the release notification also release the divided areas (0000) and (1000), respectively (steps S42b and 42c). The release of the divided area is to return the divided area to an unused empty area. Through the above processing, the divided areas (0000), (1000), and (2000) belonging to the same generation buffer set are released in synchronization.

  In actual processing, not only the CM 310 but also the CM 310a notifies the release of the buffer set when the expansion into the cache is completed. The release notification is transmitted from the CM 310 to the CM 310a, the CM 110 to the CM 110a, and the CM 210 to the CM 210a, respectively, thereby releasing the divided areas belonging to the same generation buffer set in the CMs 310a, 110a, and 210a.

  9 to 12, the synchronous copy is executed between the CM 110 and the CM 210, and the asynchronous copy is executed between the CM 110 and the CM 310. Further, the release of the buffer set is performed synchronously between the CMs 110, 210, and 310. In the CM 210, until the buffer set is released, write control information indicating the write data written in the cache 240 in the generation of the buffer set is held in the buffer management table 263. The write control information held in the buffer management table 263 in this way is used for matching cache data between the CM 210 and the CM 310 when a failover occurs during data transfer from the CM 110 to the CM 310. Is done.

  FIG. 13 is a diagram illustrating a processing example when a failover occurs. FIG. 13 shows a processing example when the operation of the storage apparatus 100 stops while data is being transferred from the divided area (0000) of the buffer 150 to the divided area (2000) of the buffer 350 as shown in FIG. .

  When the operation of the storage apparatus 100 stops, the write request destination from the host apparatus 400 is automatically changed from the storage apparatus 100 to the storage apparatus 200 by the access destination control function provided in the host apparatus 400. At this time, the physical write destination is changed without changing the logical write destination volume recognized by the host apparatus 400.

  In addition, the storage apparatus 200 changes the session status of the asynchronous copy session set with the storage apparatus 300 to “copying”, and starts receiving a write request from the host apparatus 400. At this time, in parallel with the processing according to the write request, the following processing relating to the previous generation buffer set, that is, the divided areas (1000) and (2000) is executed in the background.

  When the operation of the storage apparatus 100 is stopped, the write control information is registered in the control information area corresponding to the divided area (1000) in the buffer management table 263. The write data corresponding to the write control information is data that may not be transferred from the CM 110 to the CM 310. Therefore, the CM 210 reads the write data corresponding to the write control information from the cache 240 and transfers it to the CM 310 (step S51).

  The CM 310 stores the transferred write data in the cache 340. When the storage in the cache 340 is completed, the release of the buffer set is notified, and the divided areas (1000) and (2000) are returned to the unused free areas.

  By such processing, even if data transfer from the divided area (0000) to the divided area (2000) is not completed before failover, the write control information indicating the write data that may not be transferred is obtained. It is held in the buffer management table 263 of the CM 210. Therefore, the CM 210 can read write data that may not be transferred from the cache 240 and transfer it to the CM 310 based on the held write control information. Thereby, the data in the caches 240 and 340 can be made equivalent.

  Also, the CM 210 extracts only difference data that may not be transferred to the CM 310 from the write data stored in the cache 240 based on the held write control information, and transfers it to the CM 310. Can do. Thus, the amount of data transferred from the CM 210 to the CM 310 to equalize the data in the caches 240 and 340 can be suppressed, and the time until the data in the caches 240 and 340 become equivalent can be shortened.

  On the other hand, in parallel with such processing, the CM 210 executes processing in response to a write request from the host device 400. When the CM 210 receives a write request from the host device 400, the CM 210 writes the write data received from the host device 400 into the cache 240 and transmits a write completion notification to the host device 400 (step S52). The CM 210 then transfers the write data written in the cache 240 to the CM 310 (step S53). The CM 310 stores the transferred write data in the cache 340.

  As described above, the transfer processing of the write data that may not be completely transferred from the CM 110 to the CM 310 at the time of the failover and the processing corresponding to the write request from the host device 400 are executed in parallel. . As a result, when the operation of the storage apparatus 100 stops, the CM 210 can take over the access control according to the write request from the host apparatus 400 in a short time, and in the background, the remaining of the buffer set of the previous generation Can be executed.

  In the actual process, the above process is executed using the copy bitmap 268. When a failover occurs, the CM 210 sets the bit corresponding to the write control information registered in the control information area of the buffer management table 263 among the bits of the copy bitmap 268 to “1”. As a result, the position of the write data to be copied from the cache 240 to the CM 310 is recorded in the copy bitmap 268. The CM 210 reads the write data indicated by the bit whose bit value is “1” in the copy bitmap 268 from the write data in the cache 240 from the cache 240 and transfers it to the CM 310 (corresponding to the processing in step S51). The CM 210 executes data transfer based on the copy bitmap 268 until all the bits of the copy bitmap 268 become “0”.

  On the other hand, when the CM 210 receives a write request from the host device 400 and writes the write data to the cache 240, the bit corresponding to the write data write destination among the bits of the copy bitmap 268 is also set to “1”. As described above, the CM 210 executes data transfer based on the copy bitmap 268 until all the bits of the copy bitmap 268 become “0”. Therefore, the write data written to the cache 240 in response to the write request is also transferred from the cache 240 to the CM 310 based on the copy bitmap 268.

  As described above, actually, the data transfer process to the CM 310 based on the copy bitmap 268 and the write data write process and the update process of the copy bitmap 268 in response to the write request from the host apparatus 400 are performed in parallel. Executed. As a result, when the operation of the storage apparatus 100 stops, the CM 210 can take over the access control according to the write request from the host apparatus 400 without leaving time.

  When all the bits of the copy bitmap 268 become “0”, the CM 210 sets the session status of the asynchronous copy session with the CM 310 to “active”. As a result, the asynchronous copy equivalent to the CM 110 is resumed in the CM 210. That is, the CM 210 executes an asynchronous copy in which the order of writing is guaranteed while receiving a write request from the host device 400.

  In the example of FIG. 13, when the CM 210 receives a write request after the session status becomes “active”, the CM 210 writes the write data to the cache 240. At the same time, the CM 210 registers write control information indicating a write data write destination in a control information area corresponding to the divided area (1001) in the control information area of the buffer management table 263. When the above processing is completed, the CM 210 transmits a write completion notification to the host device 400.

  Further, at the subsequent asynchronous timing, the CM 210 stores (copies) the write data written in the cache 240 in the divided area (1001) of the buffer 250 based on the write control information registered in the control information area. After the buffer set is switched, the CM 210 transfers the write data stored in the divided area (1001) to the CM 310. The transferred write data is stored in the cache 340 via the divided area (2001) of the buffer 350.

By such processing, when the session status is “active”, the CM 210 can execute an asynchronous copy equivalent to the CM 110 before failover.
Note that the same processing as described above is executed between the CM 210a and the CM 310a.

  By the way, according to the process described in Patent Document 1, communication between the CM 210 and the CM 310 occurs while the CM 110 is in the active state. For example, in Patent Document 1, when write data is stored in the buffer 150 in the CM 110 (for example, corresponding to step S13 in FIG. 9), the write data is also stored in the buffer 250 in the CM 210. When the write data transferred from the CM 110 is stored in the buffer 350 of the CM 310, the write control information is transmitted from the CM 210 to the CM 310. In the CM 310, the received write control information is used to collate the write data stored in the buffer 350. When data matching of the same generation buffer set of the CMs 110, 210, and 310 is confirmed by this collation, the write data is expanded in the cache 340 of the CM 310, and the release of the buffer set is notified. This release notification is transmitted directly from CM 310 to CM 210 as well as from CM 310 to CM 110.

  On the other hand, as shown in FIGS. 9 to 12, in the present embodiment, no communication is performed between the CM 210 and the CM 310 while the CM 110 is in the active state. For this reason, it is possible to reduce the amount of communication between CMs when the CM 110 is active. In particular, the CM 310 is often set at a remote location with respect to the CMs 110 and 210, and the communication charge on the communication path between the CMs 110 and 210 and the CM 310 is often high. In this embodiment, communication between the CM 210 and the CM 310 is not performed at all, so that a necessary communication fee can be reduced.

  Further, in the present embodiment, when the write data stored in the buffer 350 in the CM 310 is expanded in the cache 340, there is no need to wait for transmission information from the CM 210, and collation processing based on the transmission information is not performed. Therefore, it is possible to shorten the time from the start of data transfer from the buffer 150 to the buffer 350 until the data is expanded in the cache 340 and the buffer set is released. By shortening the transfer / development / release processing time, the following effects can be obtained.

  When the buffer set is switched as shown in FIG. 10, if there is no empty divided area in the buffer 150, the process waits until the divided area (0001) is released, and the switching process is temporarily stopped. As described above, the time until the divided area (0001) is released is shortened, so that the time until the switching process is resumed is also shortened. As a result, the overall processing time required for switching can be shortened.

  Also, as shown in the case of FIG. 10, when a write request is received from the host apparatus 400, if there is no space in the storage reservation target divided area (0000), a write request is required. Write control information corresponding to is registered in the switching wait queue 166. Then, the write I / O process according to the write request is suspended until the switching is completed. In this case, as described above, the switching processing time is shortened, so that the time until the write I / O processing is restarted is also shortened. As a result, the response time for the write request from the host device 400 can be shortened.

Next, a processing procedure in the storage system will be described with reference to a sequence diagram and a flowchart.
First, processing when the storage apparatuses 100, 200, and 300 are operating normally, the storage apparatus 100 is in the active state, and the storage apparatus 200 is in the standby state will be described with reference to FIGS.

FIG. 14 is an example of a sequence diagram showing the procedure of the write I / O process.
The control unit 130 of the CM 110 receives a write request and write data from the host device 400 (step S101). The control unit 130 refers to the switching state information 164 and determines whether the buffer set switching process is currently being executed (step S102). If it is determined in step S102 that the process is being executed, the control unit 130 registers the write control information indicating the write request in the switching wait queue 166 (step S103), and ends the write I / O process. At this time, the write data is temporarily held in the RAM 112. As will be described later, when the switching process is completed, the write control information is extracted from the switching waiting queue 166. Then, the processing after step S102 is executed for this write control information and the corresponding write data (corresponding to step S130 in FIG. 15).

  On the other hand, if it is determined in step S102 that the switching process is not being executed, the control unit 130 receives the received write data in the storage area in the buffer 150 (in this example, the division area (0001)). It is determined whether there is enough free space to store (step S104). In this determination, the control unit 130 calculates the total size of the corresponding write data based on the write control information registered in the control information area corresponding to the divided area (0001) in the buffer management table 163. When the value obtained by adding the size of the received write data to the calculated total size is larger than the size of the divided area, the control unit 130 determines that there is no free area.

  When it is determined in step S104 that there is no free space, the control unit 130 registers write control information indicating a write request in the switching wait queue 166, and starts buffer set switching processing (step S105). Thereby, the switching process shown in FIG. 15 is started. The write data is temporarily stored in the RAM 112. Similarly to the above, when the switching process is completed, the write control information is taken out from the switching waiting queue 166, and the processes after step S102 are executed for this write control information and the corresponding write data (step of FIG. 15). Corresponding to S130).

  On the other hand, if it is determined in step S104 that there is a free area, the control unit 130 requests the previous generation to write to an area that overlaps the write destination area of the write data and does not store in the buffer 150. It is determined whether there is any write data (step S106). In this determination, the control unit 130 refers to a control information area corresponding to a buffer set that is a generation before the present and is not released, among the control information areas of the buffer management table 163. The control unit 130 includes write control information in which the write destination area overlaps with the write destination of the new write data received in step S101 and the storage status information indicates “unstored” in the referenced control information area. In this case, it is determined that the condition of step S106 is satisfied.

  When this condition is satisfied, if new write data is stored in the cache 140, at least part of the old write data that satisfies the condition of step S106 is overwritten. For this reason, old write data cannot be correctly left in the divided areas belonging to the previous generation, and the order of writing cannot be maintained. Therefore, when the condition of step S106 is satisfied, the control unit 130 registers the write control information indicating the write request in the storage completion waiting queue 167 (step S107), and ends the write I / O processing. At this time, new write data is temporarily held in the RAM 112. As will be described later, when the old write data is stored in the divided area of the buffer 150, the write control information is extracted from the storage completion waiting queue 167. Then, the processing after step S102 is executed for this write control information and corresponding write data (corresponding to step S137 in FIG. 16).

  On the other hand, when the condition of step S106 is not satisfied, the control unit 130 executes a synchronous copy process and a storage reservation process. That is, the control unit 130 registers the write control information indicating the received write data in the control information area corresponding to the divided area (0001) in the buffer management table 163 (step S108). Further, the control unit 130 transmits the write data together with the write control information to the CM 210 (step S109). At this time, for example, identification information indicating the storage area division area (1001) in the buffer 250 is added to the write data and transmitted. Alternatively, a generation number or a buffer set identification number may be added instead of the divided region identification information.

  The control unit 230 of the CM 210 registers the received write control information in the control information area corresponding to the divided area (1001) in the buffer management table 263, and writes the received write data in the cache 240 (step S110). When these processes are completed, the control unit 230 transmits a completion notification to the CM 110 (step S111).

  The control unit 130 of the CM 110 writes the write data to the cache 140 (Step S112). Then, the control unit 130 transmits a write completion notification to the host device 400 and makes a response to the write request (step S113).

  FIG. 15 is an example of a sequence diagram illustrating a procedure of buffer set switching processing. When the buffer set switching condition is satisfied, the processing of FIG. 15 is started. The case where the switching condition is satisfied is a case where it is determined in step S104 in FIG. 14 that there is no free space, or a case where a certain time has elapsed since the most recent switching process was executed. The process of FIG. 15 is executed in parallel with the process of FIG.

  The control unit 130 of the CM 110 updates the switching state information 164 to indicate that switching processing is being performed (step S121). And the control part 130 confirms the empty state of the division area in the buffer 150, and performs the following determinations (step S122). If there is no empty divided area (released divided area), the control unit 130 enters a waiting state for releasing the divided area (step S123). In this case, the control unit 130 executes the process of step S121 again after a predetermined time.

  In step S121, if there is an empty divided area, specifically, if a divided area corresponding to the next generation is released, the control unit 130 switches the buffer set (step S124). Specifically, the control unit 130 updates the storage buffer set information 165 to indicate the next generation buffer set. Further, the control unit 130 notifies the CM 210 of buffer set switching (step S125).

  The control unit 230 of the CM 210 switches the buffer set (step S126). Specifically, the control unit 230 updates the storage buffer set information 265 to indicate the next generation buffer set. Then, the control unit 230 transmits a completion notification to the CM 110 (step S127).

  The control unit 130 of the CM 110 updates the switching state information 164 to indicate that the switching process is not being performed (step S128). Then, the control unit 130 determines whether the write control information is registered in the switching wait queue 166 (step S129). When the write control information is registered, the control unit 130 retrieves the write control information from the switching wait queue 166, and starts the write I / O processing from step S102 of FIG. 14 for the write control information (step S102). S130). Thereby, the write I / O processing for the extracted write control information is resumed. When a plurality of write control information is registered in the switching wait queue 166, the write I / O processing is activated by selecting them sequentially. On the other hand, if it is determined in step S129 that the light control information is not registered, the switching process is terminated.

  FIG. 16 is an example of a flowchart illustrating the procedure of buffer storage processing. When the switching process shown in FIG. 15 is completed, the buffer storage process shown in FIG. 16 is started for the buffer set after the switching. The process of FIG. 16 is executed in parallel with the process of FIG. 14 and the process of FIG. 15 for other buffer sets.

  The control unit 130 of the CM 110 determines whether the write control information is registered in the control information area corresponding to the buffer set to be processed in the buffer management table 163 (step S131). When the write control information is not registered, the control unit 130 determines whether the buffer set switching process has been executed (step S132). The control unit 130 determines that the switching process has been executed when the generation of the buffer set indicated by the storage buffer set information 165 does not match the buffer set to be processed. In this case, since all the write control information registered in the control information area corresponding to the switched buffer set has been processed, the buffer storage process is completed. On the other hand, when the switching process is not executed, the control unit 130 executes the process of step S131 after a predetermined time.

  When the light control information is registered in step S131, the control unit 130 selects the earliest registered light control information among the light control information registered in the control information area (step S133). Next, the control unit 130 reads the write data indicated by the selected write control information from the cache 140 and stores (copies) the write data in the divided area corresponding to the buffer set to be processed in the buffer 150 (step S134). Then, the control unit 130 updates the storage state information added to the selected write control information so as to indicate that it has been stored (step S135).

  Next, the control unit 130 refers to the storage completion waiting queue 167 and determines whether or not write control information indicating new write data with overlapping write destination areas is registered (step S136). The new write data is data in which at least a part of the write destination area overlaps with the write data write destination area selected in step S133.

  If the corresponding write control information is registered in step S136, the control unit 130 extracts the write control information from the storage completion waiting queue 167. And the control part 130 starts the write I / O process from step S102 of FIG. 14 about the taken-out write control information (step S137). Thereby, the write I / O processing for the extracted write control information is resumed. Then, the control part 130 performs the process of step S131.

On the other hand, when the corresponding write control information is not registered in step S136, the control unit 130 executes the process of step S131.
FIG. 17 is an example of a sequence diagram illustrating a procedure of transfer / development / release processing. When it is determined “Yes” in step S132 of FIG. 16 and the buffer storage process is completed, the process shown in FIG. 17 is started for the buffer set for which the process has been completed (that is, the buffer set that has been switched and the buffer has been stored). . The process of FIG. 17 is executed in parallel with the process of FIG. 14 and the processes of FIGS. 15 and 16 for other buffer sets.

  The control unit 130 of the CM 110 transfers all the write data stored in the divided area (in this example, the divided area (0001)) of the buffer 150 to the CM 310 together with the corresponding write control information (Step S141). . At this time, for example, identification information indicating the storage area division region (2001) in the buffer 350 is added to the write data and transmitted. Alternatively, a generation number or a buffer set identification number may be added instead of the divided region identification information.

  The control unit 330 of the CM 310 stores the received write data in the divided area (2001) of the buffer 350 and registers the received write control information in the control information area corresponding to the divided area (2001) in the buffer management table 363. (Step S142). When these processes are completed, the control unit 330 writes the write data stored in the divided area (2001) into the cache 340 (step S143). As a result, the write data is expanded in the cache 340.

  The control unit 330 notifies the CM 110 of the release of the buffer set (step S144). Upon receiving the release notification, the control unit 130 of the CM 110 notifies the CM 210 of the release of the buffer set (Step S145). At the same time, the control unit 130 performs processing for releasing the processing target buffer set (step S146). Specifically, the control unit 130 returns the divided area (0001) to an empty area, and erases the write control information registered in the control information area corresponding to the divided area (0001) in the buffer management table 163.

  Upon receiving the release notification, the control unit 230 of the CM 210 performs processing for releasing the processing target buffer set (step S147). Specifically, the control unit 230 returns the divided area (1001) to an empty area, and erases the write control information registered in the control information area corresponding to the divided area (1001) in the buffer management table 263.

  The control unit 230 notifies the CM 110 that the buffer set has been released (step S148). Upon receiving the release completion notification, the control unit 130 of the CM 110 notifies the CM 310 of the completion of buffer set release (step S149). The control unit 330 of the CM 310 performs processing for releasing the processing target buffer set (step S150). Specifically, the control unit 330 returns the divided area (2001) to an empty area, and erases the write control information registered in the control information area corresponding to the divided area (2001) in the buffer management table 363.

By such processing, the buffer sets of the same generation in the CMs 110, 210, and 310 are released synchronously.
In the process of FIG. 17 described above, when compared with Patent Document 1, it is not necessary to wait for transmission information from the CM 210 at the time of development in step S143, and a matching process based on the transmission information is not performed. For this reason, it is possible to shorten the entire processing time of transfer / development / release in FIG.

  Further, by shortening the transfer / development / release processing time, for example, the release completion waiting time in step S123 of FIG. 15 can be shortened, thereby reducing the buffer set switching processing time. As a result, for example, the write I / O processing for the write control information registered in the switching wait queue 166 in steps S103 and S105 in FIG. 14 can be resumed in a short time, thereby responding to the write request from the host device 400. Response time can be shortened.

  Next, processing when the operation of the storage apparatus 100 is stopped due to a failure or the like will be described with reference to FIGS. In the description of FIGS. 18 and 19, as an example, it is assumed that the divided areas (1000) and (1001) in the buffer 250 are not released at the time of occurrence of failover.

  FIG. 18 is an example of a sequence diagram illustrating a procedure of resynchronization processing after failover. Resynchronization is a process of making the cache 240 of the CM 210 and the cache 340 of the CM 310 equivalent.

  When the control unit 230 of the CM 210 detects that the storage apparatus 100 has stopped (step S151), it notifies the CM 310 of the occurrence of failover (step S152). At the same time, the control unit 230 changes the session status of the asynchronous copy session with the storage apparatus 300 to “copying” (step S153). On the other hand, when receiving the notification of the occurrence of failover, the control unit 330 of the CM 310 changes the session status of the asynchronous copy with the storage apparatus 200 to “copying” (step S154).

  The control unit 230 of the CM 210 refers to the control information area of the buffer management table 263 and confirms whether the write control information is registered. When the write control information is registered, the following processing is executed. Here, it is assumed that the write control information is registered in the control information area corresponding to (1000) and (1001). The control unit 230 creates a copy bitmap 268 based on the write control information registered in these control information areas (step S155). The copy bitmap 268 has a bit for each block of the copy processing unit in the logical volume to be written. The initial value of each bit is “0”.

  The control unit 230 acquires the write control information one by one from the control information area, and among the bits of the copy bitmap 268, sets the bit corresponding to the block included in the write destination area specified by the acquired write control information. Update to “1”. The write control information registered in the control information area indicates write data that may not be transferred from the buffer 150 of the CM 110 to the CM 310. Through the above processing, the position on the logical volume where such write data exists is registered in the copy bitmap 268.

  Note that when the copy bitmap 268 is created in this manner, acceptance of an access request from the host device 400 is started. For example, when a write request is received from the host apparatus 400, the later write I / O processing shown in FIG. 19 is executed in parallel with the following processing in FIG.

  The control unit 230 selects one bit whose bit value is “1” from the created copy bitmap 268. The control unit 230 reads the write data corresponding to the selected bit from the cache 240 and transfers it to the CM 310 together with the corresponding write control information (step 156).

  The control unit 330 of the CM 310 stores the received write data in the cache 340 (step S157). When receiving the write data storage completion notification from the CM 310, the control unit 230 of the CM 210 updates the bit corresponding to the write data in the copy bitmap 268 to “0” (step S158). Then, the control unit 230 determines whether all the bits of the copy bitmap 268 are “0” (step S159).

  When the bit having the bit value “1” remains, the control unit 330 newly selects the bit having the bit value “1” from the copy bitmap 268 and executes the process of step S156. In this way, the processes of steps S156 to S158 are repeated until all bits of the copy bitmap 268 become “0”.

  When all the bits in the copy bitmap 268 become “0” in step S159, the cache 240 of the CM 210 and the cache 340 of the CM 310 are synchronized (that is, the data is equivalent to each other). At this time, the control unit 230 of the CM 210 notifies the CM 310 of the release of the buffer set (Step S160). At the same time, the control unit 230 performs a process of releasing the unreleased buffer set (step S161). Specifically, the control unit 230 returns the divided areas (1000) and (1001) to the free area, and writes the write information registered in the control information area corresponding to the divided areas (1000) and (1001) in the buffer management table 263. Erase control information.

  When receiving the release notification, the control unit 330 of the CM 310 performs a process of releasing the unreleased buffer set (step S162). Specifically, the control unit 230 executes a process of returning the divided areas (2000) and (2001) to a free area. When the release process is completed, the control unit 330 notifies the CM 210 of the completion of the release of the buffer set (step S163).

  Through the above processing, the divided areas (1000), (1001), (2000), and (2001) are released synchronously. At this time, all buffer sets are released.

  The control unit 230 of the CM 210 changes the session status of the asynchronous copy session with the storage apparatus 300 to “active” (step S164). Further, the control unit 230 instructs the CM 310 to change the status, and the control unit 330 of the CM 310 changes the session status of the asynchronous copy session with the storage apparatus 200 to “active” according to the instruction ( Step S165). As the session status is changed to “active”, an asynchronous copy in which the order of writing is ensured is started between the CM 210 and the CM 310.

  Note that when the session status is changed to “active”, all the buffer sets are released. For this reason, when a write I / O process corresponding to a write request from the host device 400 is started, any buffer set may be selected as the current generation buffer set (buffer set to be stored). For example, when changing the session status to “active”, the CMs 210 and 310 select a predetermined buffer set as the current generation buffer set.

  Here, in FIG. 18 described above, the process of performing data transfer using the copy bitmap 268 has been described. However, as another method, a method may be used in which the write control information is acquired one by one from the control information area referenced in step S155, and the write data indicated by the acquired write control information is transferred from the cache 240 to the CM 310. . However, by using the copy bitmap 268, there is a possibility that the data transfer amount from the CM 210 to the CM 310 can be reduced.

  For example, even when there are a plurality of unreleased buffer sets and there is write control information indicating the write data with the same write destination area between the buffer sets, the cache 240 of the CM 210 includes the write data. Only the latest write data is stored. For this reason, even if there is write control information indicating such write data, the corresponding write data need only be transferred to the CM 310 once.

  By using the copy bitmap 268, write data having the same write destination area can be transferred from the CM 210 to the CM 310 by a single transfer process. As a result, the amount of data transferred from the CM 210 to the CM 310 can be reduced, and the time until the data in the cache 240 and the data in the cache 340 become equivalent can be shortened.

  In the copy bitmap 268, in addition to the position of write data that may not be transferred as described above, the position at which write data is newly written in the next write I / O processing shown in FIG. To be recorded. For this reason, in the processing of steps S156 to S158, the write data newly written in the write I / O processing is also transferred to the CM 310.

  Thus, in parallel with the cache resynchronization, the write I / O processing according to the write request from the host device 400 can be executed. Therefore, the CM 210 can start accepting a new write request from the host apparatus 400 in a short time after the occurrence of failover. Therefore, the CM 210 can take over the write I / O processing corresponding to the write request from the host apparatus 400 from the CM 110 in a short time.

  Further, by using the copy bitmap 268 in the write I / O processing, the data newly written by the write request can be transferred to the CM 310 asynchronously with the writing to the cache 240 (that is, by the write back method). . For this reason, the response performance with respect to the write request can be improved.

  FIG. 19 is an example of a flowchart showing the procedure of write I / O processing in the copying state. As described above, when the session status becomes “copying” in steps S153 and S154 in FIG. 18 and the copy bitmap 268 is created in step S155, acceptance of an access request from the host device 400 is started. When the write request is received from the host apparatus 400 in the state where the session status is “copying”, the processing of FIG. 19 is executed.

  The control unit 230 of the CM 210 receives a write request and write data from the host device 400 (step S171). Then, the control unit 230 writes the write data to the cache 240 (Step S172). Also, the control unit 230 updates the bit corresponding to the block included in the write data write destination area among the bits of the copy bitmap 268 to “1” (step S173). Then, the control unit 230 transmits a write completion notification to the host device 400 and makes a response to the write request (step S174).

  In FIG. 19 described above, the position where the new write data is written is recorded in the copy bitmap 268 by the process of step S173. As a result, the processing of steps S156 to S158 in FIG. 18 is executed, so that the write data newly written in the cache 240 is transferred to the CM 310 asynchronously with the response to the write request.

  Next, processing of the CM 210 when a write request is received from the host apparatus 400 after the session status of the asynchronous copy session becomes “active” in steps S164 and S165 in FIG. 18 will be described with reference to FIG.

  FIG. 20 is an example of a flowchart showing the procedure of the write I / O process in the active state. As shown below, the write I / O process executed in the CM 210 basically includes processes related to communication with the CM 210 (steps S109 to S111) from the write I / O process in the CM 110 shown in FIG. This is the same as the processing that is excluded.

  That is, the control unit 230 of the CM 210 receives a write request and write data from the host device 400 (step S181). The control unit 230 refers to the switching state information 264 to determine whether the buffer set switching process is currently being executed (step S182). If it is determined in step S182 that the process is being executed, the control unit 230 registers the write control information indicating the write request in the switching queue 266 (step S183), and ends the write I / O process. At this time, the write data is temporarily held in the CM 210 RAM. When the switching process is completed, the write control information is extracted from the switching wait queue 266. Then, the processing after step S172 is executed for this write control information and corresponding write data (corresponding to step S130 in FIG. 15).

  On the other hand, when it is determined in step S182 that the switching process is not being executed, the control unit 230 receives the received write data in the storage area of the buffer 250 (in this example, the divided area (1001)). It is determined whether there is enough free space to store (step S184). When it is determined that there is no free space, the control unit 230 registers the write control information indicating the write request in the switching wait queue 266 and starts the buffer set switching process (step S185). Thereby, the switching process is started in the same procedure as in FIG. The write data is temporarily held in the CM 210 RAM. Similarly to the above, when the switching process is completed, the write control information is extracted from the switching waiting queue 266, and the processes after step S182 are executed for this write control information and the corresponding write data (step of FIG. 15). Corresponding to S130).

  On the other hand, if it is determined in step S184 that there is an empty area, the control unit 230 requests a previous generation for an area overlapping the write data write destination area, and the buffer 250 does not store the write data. It is determined whether there is any write data (step S186). When this condition is satisfied, the control unit 230 registers the write control information indicating the write request in the storage completion waiting queue 267 (step S187), and ends the write I / O processing. At this time, new write data is temporarily held in the RAM of the CM 210. When the old write data is stored in the divided area of the buffer 150, the write control information is extracted from the storage completion queue 267. Then, the processing after step S182 is executed for this write control information and the corresponding write data (corresponding to step S137 in FIG. 16).

  On the other hand, when the condition of step S186 is not satisfied, the control unit 230 registers the write control information indicating the received write data in the control information area corresponding to the divided area (1001) in the buffer management table 263 (step S188). . Then, the control unit 230 writes the write data to the cache 240 (Step S189). The control unit 230 transmits a write completion notification to the host device 400 and makes a response to the write request (step S190).

  Although illustration is omitted, when the control unit 230 of the CM 210 executes the process of FIG. 20 even once, the process of the CM 110 in FIG. 15 (except for the notification of switching to the CM 210), the process shown in FIG. The same process as the CM 110 process (excluding notification of buffer set release to the CM 210) is executed. For example, write data is copied from the cache 240 to the divided area (1001) illustrated in FIG. Further, when a condition such as no space in the divided area (1001) is satisfied, the buffer set is switched, and then the write data stored in the divided area (1001) is transmitted to the CM 310. As a result of such processing, as in the case of the CM 110 before failover, asynchronous copying is performed in which the order of writing is guaranteed.

  Note that the processing functions of the devices (for example, the control devices 10, 20, and 30, the CMs 110, 110a, 210, 210a, 310, and 310a) described in each of the above embodiments can be realized by a computer. In that case, a program describing the processing contents of the functions that each device should have is provided, and the processing functions are realized on the computer by executing the program on the computer. The program describing the processing contents can be recorded on a computer-readable recording medium. Examples of the computer-readable recording medium include a magnetic storage device, an optical disk, a magneto-optical recording medium, and a semiconductor memory. Magnetic storage devices include hard disk devices (HDD) and magnetic tapes. Optical discs include CD (Compact Disc), DVD (Digital Versatile Disc), Blu-ray Disc (BD, registered trademark) and the like. Magneto-optical recording media include MO (Magneto-Optical disk).

  When distributing the program, for example, a portable recording medium such as a DVD or a CD on which the program is recorded is sold. It is also possible to store the program in a storage device of a server computer and transfer the program from the server computer to another computer via a network.

  The computer that executes the program stores, for example, the program recorded on the portable recording medium or the program transferred from the server computer in its own storage device. Then, the computer reads the program from its own storage device and executes processing according to the program. The computer can also read the program directly from the portable recording medium and execute processing according to the program. In addition, each time a program is transferred from a server computer connected via a network, the computer can sequentially execute processing according to the received program.

DESCRIPTION OF SYMBOLS 1 High-order apparatus 10, 20, 30 Control apparatus 11, 21, 31 Recording medium 12 Buffer 12a, 12b Divided buffer area 13, 23, 33 Control part 22 Control information storage part 22a, 22b Divided area 24a, 24b Control information

Claims (7)

A storage system having a first control device, a second control device, and a third control device,
The first control device includes:
A buffer having a first region and a second region;
The data requested to be written by the host device is written to the first recording medium and transmitted to the second control device, and the host device is notified of the completion of writing, and the first recording medium The written data is stored in the first area, and when a predetermined condition is satisfied, the buffer switching is notified to the second control device, and the data written to the first recording medium after the notification is notified. The storage process for switching the storage destination in the buffer to the second area and the first data stored in the first area are collectively transmitted to the third control device and stored in the second area. Asynchronous copy processing for collectively transmitting the second data thus transmitted to the third control device, and when the transmission of the first data to the third control device is completed, A first control unit that transmits a notification to the second controller,
Have
The second control device includes:
A control information storage unit having a third region associated with the first region and a fourth region associated with the second region;
The first data transmitted from the first control unit is stored in a second recording medium, and first control information indicating the first data is stored in the third area, and the buffer When the switching is notified, the second data transmitted from the first control unit after the notification is stored in the second recording medium, and second control information indicating the second data is stored. When storing in the fourth area and receiving the completion notification, the first control information is erased from the third area, and when the first control device stops after notification of the buffer switching, Taking over the reception process of the write request from the apparatus from the first control unit, the third data requested to be written is stored in the second recording medium, and the third data is transmitted at a predetermined timing thereafter. The third system A second control unit for transmitting device,
Have
The third control device includes:
When the first data is received from the first control device, the first data is stored in a third recording medium, and when the second data is received from the first control device, the second data is stored. Is stored in the third recording medium, and upon receiving the third data from the second control device, a third control unit stores the third data in the third recording medium,
Have
The second control unit may further include the first control information if the first control information remains in the third area when the first control device is stopped after the buffer switching notification. Reading the first data from the second recording medium based on the control information, transmitting the read first data to the third control device;
The third control unit further stores the first data in the third recording medium when receiving the first data from the second control device.
Storage system. The second control unit stores the third data in the second recording medium when the first control device stops after notification of the buffer switching, and then stores the third data in the second recording medium. Based on the first control information remaining in the third area, the first data from the second recording medium is read out, and processing to be transmitted to the third control device is performed. Executing in parallel with the process of transmitting data to the third control unit;
The storage system according to claim 1. The second control device stores a bitmap having a bit for each storage area of the second recording medium,
The second controller is
When the first control device stops after the notification of the buffer switching, the second data storage area of the second recording medium among the bits of the bitmap is based on the second control information. Set the corresponding bit to the first value indicating the presence of data,
When the third data requested to be written is stored in the second recording medium, the bit corresponding to the storage area of the third data in the second recording medium among the bits of the bitmap , Set to the first value,
The bit map is referenced, data is read from the storage area of the second recording medium corresponding to the bit for which the predetermined value is set, is transmitted to the third control device, and the bit is set to the second value. To update all the bits of the bitmap until the second value,
The storage system according to claim 1. When at least a part of the write destination area for the first data overlaps with the write destination area for the second data, the first control unit transfers the second data to the first recording medium. When storing the first data in the first area from the first recording medium, the process is completed if the process is not completed. And then storing the second data in the first recording medium,
The storage system according to any one of claims 1 to 3. When the predetermined condition is satisfied, the first control unit is requested to write from the host device before that and is stored in the second area, and is transferred to the third control device. If there is third data that has not been completed, and if there is the third data, the buffer switching is performed after the transfer of the third data to the third control device is completed. Notify the second controller;
The storage system according to any one of claims 1 to 4. The first control unit is requested to write the second data from the host device as the predetermined condition, and there is no free space in the first area to store the second data. When the above condition is satisfied, the third data that has been requested to be written by the host device and stored in the second area before the transfer to the third control device is completed. If there is the third data, after the transfer of the third data to the third control device is completed, the buffer switching is notified to the second control device, and After the second data is written to the first recording medium and transmitted to the second control device, the second device is notified of the completion of the writing of the second data, and then the second data is sent to the second recording device. Store in the area of
The storage system according to any one of claims 1 to 4. A buffer having a first region and a second region;
The data requested to be written by the host device is written to the recording medium and transmitted to the first other control device, and the host device is notified of the completion of writing, and the data written to the recording medium is transmitted. When it is stored in the first area and a predetermined condition is satisfied, the buffer switching is notified to the first other control device, and the storage destination in the buffer of the data written to the recording medium after the notification is notified. The storage process for switching to the second area and the first data stored in the first area are collectively transmitted to the second other control device, and the second data stored in the second area is stored. Asynchronous copy processing for collectively transmitting data to the second other control device, and when the transmission of the first data to the second other control device is completed, a completion notification And a control unit which transmits to said first other control device,
Have
When at least a part of the write destination area of the first data overlaps with the write destination area of the second data, the control unit, when storing the second data in the recording medium, It is determined whether the process of storing the first data from the recording medium into the first area is completed. If the process is not completed, the second data is stored after the process is completed. Store in a recording medium,
Control device.

Images