JP4808747B2 - Storage subsystem - Google Patents

Description

  The present invention relates to a storage subsystem and a storage subsystem control method.

  The storage subsystem can provide a large capacity and high performance storage service to the host computer. In the storage subsystem, for example, a large number of disk drives are arranged in an array, and a storage area based on RAID (Redundant Array of Independent Inexpensive Disks) is constructed. A logical volume, which is a logical storage area, is formed on the physical storage area of each disk drive. A LUN (Logical Unit Number) is associated with a logical volume in advance. The host computer issues a predetermined format write command or read command to the storage subsystem by specifying the LUN, block address, and the like. As a result, the host computer can read / write desired data from / to the storage subsystem.

  A plurality of host computers can be connected to the storage subsystem. When a group of data managed by a certain host computer or application program can be read and written from another host computer or application program, inconvenience may occur. Therefore, for example, an access control technique such as zoning or LUN masking is used. Zoning is a technology in which one or a plurality of zones are set in the storage subsystem, and data transfer is permitted only to specific communication ports and WWNs (World Wide Names) belonging to the zones. LUN masking is a technology that allows a specific host computer to access a specific LUN.

Incidentally, data exchange between the host computer and the disk drive is performed via a cache memory. When a write command is output from the host computer, for example, data is once stored in a cache memory and then written to a disk drive. When a read command is output from the host computer, for example, data read from the disk drive is provided to the host computer via the cache memory. Therefore, a technique for appropriately allocating the cache memory allocation amount to the disk drive is also known (Patent Document 1).
JP-A-4-264940

  Access restrictions from the host computer to the storage subsystem can be set by conventional zoning or LUN masking. However, the conventional technology only allows simple access restriction to the logical volume, and does not realize the division management of the resources of the storage subsystem. When the storage subsystem is used jointly by multiple users, for example, if a user issues a large number of I / O requests (IO requests), many areas of cache memory are used to process this request. The Therefore, it becomes impossible to use a cache memory sufficient for input / output processing from other users, and the service performance of the entire subsystem is lowered. In other words, the service to one user affects the service to another user.

  In the above patent document, a cache memory is allocated to each storage device, but this is for efficiently writing data stored in the cache memory to the storage device, and for providing it to a plurality of users. It does not perform resource (cache memory) division management.

  Accordingly, one object of the present invention is to allocate a cache area to each of a plurality of users so that they can be used together, and to reduce the influence on other users due to the service provided to a certain user. It is an object of the present invention to provide a storage subsystem and a storage subsystem control method that can be used. One object of the present invention is to provide a storage subsystem and a storage subsystem that can flexibly allocate resources to a plurality of users and that can be managed so that the management range of each user does not affect each other with a smaller amount of information. It is to provide a control method. One object of the present invention is to provide a storage subsystem that can logically divide and use cache resources by configuring management information according to the management unit of the cache memory, and a storage subsystem control method. It is in. Other objects of the present invention will become clear from the description of the embodiments described later.

  In order to solve the above problems, a storage subsystem according to the present invention includes a plurality of channel adapters that respectively control data exchange with a host device, a plurality of storage device groups that respectively provide logical storage areas, and each storage device A plurality of disk adapters that respectively control data exchange with the group, a cache memory that is used by each channel adapter and each disk adapter, a plurality of cache division regions that are configured by logically dividing the cache memory, And a control memory for storing management information for managing each cache division area. The management information includes partition management information provided for each cache partition area and common management information applied to the entire cache partition area.

  The cache memory and the control memory can be mounted as separate memory substrates, respectively, or a memory substrate in which both are mounted may be used. Further, an area with a memory may be used as a cache memory, and another area may be used as a control memory. The cache memory is used for temporarily (or over a long period of time) storing data exchanged between the host device and the storage device group. The cache memory is logically divided into a plurality of cache divided areas. Each cache division area is managed independently by management information. Each cache division area can be provided (assigned) for each channel adapter, for example. Each cache division area can be used by different users, for example. The management information is composed of divided management information and common management information. The partition management information is information provided for each cache partition area. The common management information is information applied to all cache division areas. By configuring the management information from the divided management information and the common management information, it is possible to use the plurality of cache divided areas individually by efficiently using the storage resources of the control memory.

  Each division management information and common management information can be set based on an attribute of a cache management unit. Examples of the cache management unit include a slot and a segment. One slot can be composed of at least one segment, and one segment has a data size of about 16 KB, for example. Examples of the attribute of the cache management unit include a free state, a dirty state, and a clean state. The free state indicates a state where the cache management unit is unused. The dirty state indicates a state in which data stored in the cache management unit is not written to the storage device. The clean state indicates a state in which data stored in the cache management unit has been written to the storage device. Depending on the attribute of the cache management unit, it may be preferable to provide individual management information for each cache division area. On the contrary, depending on the attribute of the cache management unit, there is also management information applicable to all cache division areas.

  For example, if the management information is composed of multiple types of sub-management information, a part of the sub-management information is divided into each cache partition area, and the remaining sub-management information is applied to all cache partition areas. Can be made. In this case, which of the sub management information is used as the divided management information and which is used as the common management information can be determined based on the attribute of the cache management unit.

  For example, the management information can be configured to include a plurality of types of queues and counters associated with the respective queues. In this case, each division management information can be configured by providing a part of each queue and each counter for each cache division area based on the attribute of the cache management unit. The remainder of each queue and the remainder of each counter can be used as common management information. “Based on the attribute of the cache management unit” can be rephrased as “based on the attribute of the queue (and counter)”.

  Here, one of the queues and the counters associated with each other can constitute division management information, and the other can be used as common management information. That is, for example, among the queues and counters associated with each other, only the counter can be divided and assigned to each cache division area.

  Furthermore, a queue management table can be associated with each queue. A queue management table associated with the queues constituting the partition management information can be provided for each cache partition area. That is, the queue and the queue management table are always used integrally, and when a queue is provided for each cache partition area, the queue management table is also provided for each cache partition area.

  Management information is connected to a free queue to which an unused cache management unit is connected, a free queue counter associated with the free queue, and a cache management unit for storing dirty state data before reflection in the storage device group. Use of a dirty queue and a dirty queue counter associated with the dirty queue, a clean queue to which a cache management unit for storing clean data reflected in the storage device group is connected, and a clean queue counter associated with the clean queue And a busy counter that counts the total amount of the cache management unit. A free queue counter, a clean queue, a clean queue counter, and a busy counter can be provided for each cache partition area as partition management information. Further, the free queue, the dirty queue, and the dirty queue counter can be used as common management information.

  It is also possible to set a new cache partition area by setting one cache partition area among the cache partition areas as a shared area and allocating resources belonging to this shared area. For example, in the initial state, all cache areas are used as shared areas. When a new cache partition area is generated, a part of the cache area is cut out from the shared area and assigned to the new cache partition area. When deleting a cache partition area, the cache area allocated to the cache partition area may be returned to the shared area.

  Hereinafter, embodiments of the present invention will be described with reference to FIGS. In the present embodiment, a plurality of host interface control units that respectively control data exchange with a host device, a plurality of storage device groups that respectively provide logical storage areas, and a data transfer between each storage device group are controlled. A storage subsystem is disclosed that includes a plurality of lower-level interface controllers, and memory units used by the upper-level interface controllers and the lower-level interface controllers. In this embodiment, the cache area provided by the memory unit is divided into a plurality of cache division areas. Furthermore, in this embodiment, the management information is divided for each cache division area according to the attribute of the cache management unit for managing data in the memory unit, and each cache division area is divided based on each management information. Let each manage the data.

  FIG. 1 is a schematic perspective view showing an external configuration of the storage subsystem 10. The storage subsystem 10 can be composed of a basic enclosure 11 and a plurality of additional enclosures 12, for example. The basic chassis 11 is a minimum configuration unit of the storage subsystem 10 and has both a storage function and a control function. The additional chassis 12 is an option of the storage subsystem 10 and is controlled by the control function of the basic chassis 11. For example, a maximum of four additional enclosures 12 can be connected to the basic enclosure 11.

  A plurality of control packages 13, a plurality of power supply units 14, a plurality of battery units 15, and a plurality of disk drives 80 are detachably provided on the basic casing 11. A plurality of disk drives 80, a plurality of power supply units 14, and a plurality of battery units 15 are detachably provided in the additional enclosure 12. The basic chassis 11 and each additional chassis 12 are provided with a plurality of cooling fans 16, respectively.

  The control package 13 is a module for realizing a channel adapter 20, a disk adapter 30, a cache memory 40, and the like, which will be described later. That is, a plurality of channel adapter packages, a plurality of disk adapter packages, and one or more memory packages are detachably provided in the basic casing 11 and can be replaced in units of packages.

  FIG. 2 is a block diagram showing an overall outline of the storage system including the storage subsystem 10. The storage subsystem 10 is connected to a plurality of host computers 1A to 1C (hereinafter referred to as “host computer 1” unless otherwise specified) via a communication network CN1 so as to be capable of bidirectional communication. Here, the communication network CN1 is, for example, a LAN (Local Area Network), a SAN (Storage Area Network), the Internet, a dedicated line, or the like. When the LAN is used, data transfer between the host computer 1 and the storage subsystem 10 is performed according to a TCP / IP (Transmission Control Protocol / Internet Protocol) protocol. When using a SAN, the host computer 1 and the storage subsystem 10 perform data transfer according to the fiber channel protocol. When the host computer 1 is a mainframe, for example, FICON (Fibre Connection: registered trademark), ESCON (Enterprise System Connection: registered trademark), ACONARC (Advanced Connection Architecture: registered trademark), and FIBARC (Fibre Connection Architecture: registered trademark). Data transfer is performed according to a communication protocol such as a trademark.

  Each host computer 1 is realized as, for example, a server, a personal computer, a workstation, a main frame, or the like. For example, each host computer 1 is connected to a plurality of client terminals located outside the figure via another communication network. Each host computer 1 provides a service to each client terminal by, for example, reading / writing data from / to the storage subsystem 10 in response to a request from each client terminal. Although only one is shown in the figure, a plurality of virtual enclosures (SLPR: Storage Logical Partition) can be set in the storage subsystem 10.

  SLPR is an area configured by dividing and allocating various physical resources and logical resources in the storage subsystem 10 for each user (or for each application program). That is, for example, the SLPR includes a dedicated channel adapter 20, a dedicated cache area 40, a dedicated virtual logical device (VDEV) 70, and the like. That is, SLPR behaves as if it is a virtual small storage subsystem.

  In SLPR, at least one cache partition area (CLPR) can be provided. Although only one SLPR is shown on the left side in FIG. 2, a plurality of SLPRs can be provided. One or a plurality of CLPRs can be set in one SLPR.

  CLPR is a logical division of the cache memory 40 into a plurality of areas. CLPR can be set for each channel adapter 20. For example, when n channel adapters 20 are mounted, n CLPRs can be provided. For example, although n = 32 can be set, the present invention is not limited to this. Each of the CLPR0 to CLPRn is used independently of each other, and each host computer 1 can exclusively use only the CLPR that can be used. An operation from a host computer 1 with respect to a certain CLPR is configured so as not to affect other CLPR as much as possible. That is, even when accesses from a specific host computer 1 are concentrated, a necessary cache capacity is secured only by CLPR that can be used by the host computer 1, and other CLPR resources (cache areas) are not taken away. It is configured.

  Note that CLPR0 shown on the right side of the figure is a shared area where SLPR is not defined, and various information used in the entire storage subsystem 10 is stored in this shared area CLPR0. As will be described later, in the initial state, the entire area of the cache memory 40 is assigned to the shared area CLPR0. A new CLPR is set by cutting out a predetermined amount of the cache area from the shared area CLPR0.

  In the example shown in FIG. 2, the host computer 1A can access only the shared area CLPR0 and input / output data. The host computer 1B can access only CLPR1, the host computer 1C can access only CLPR2, and the host computer 1N can access only CLPRn, and cannot use or reference other CLPRs.

  The SVP (Service Processor) 90 is a computer device for managing and monitoring the storage subsystem 10 and provides a management server function. For example, the SVP 90 collects various environmental information and performance information from each channel adapter 20 and each disk adapter 30 via the internal network CN3 such as a LAN provided in the storage subsystem 10 (see FIG. 3). To do. Examples of information collected by the SVP 90 include device configuration, power supply alarm, temperature alarm, input / output speed (IOPS), and the like. The SVP 90 and each of the management terminals 2A to 2N, 3 are connected via a communication network CN2 such as a LAN, for example. By logging in to the SVP 90 via the management terminal, the administrator can set, for example, a RAID configuration and block various packages (channel adapter package, disk adapter package, memory package, disk drive, etc.) within an authorized range. Processing and various setting changes can be performed.

  A plurality of management terminals 2A to 2N, 3 can be connected to the SVP 90. Here, the management terminals 2 </ b> A to 2 </ b> N are terminals provided for each SLPR, and the management terminal 3 is a terminal provided for managing the entire storage subsystem 10. In the following description, a case where one CLPR is provided for one SLPR will be described as an example. Accordingly, the management terminals 2A to 2N are sub-terminals operated respectively by managers (hereinafter referred to as division managers) that manage each CLPR. The management terminal 3 is an overall terminal operated by a system administrator (or may be referred to as an overall administrator) that manages the entire storage subsystem 10.

  The division manager of each of the sub-terminals 2A to 2N can change various settings only for the CLPR for which he / she has management authority, and is not permitted to refer to or change other CLPR configurations or the like. On the other hand, the system administrator can change various settings of the entire storage subsystem 10 including each CLPR.

  The system administrator can set SLPR (CLPR) for each user by logging into the SVP 90 via the overall terminal 3 and appropriately dividing the physical resources and logical resources of the storage subsystem 10. . The system administrator can also issue a user ID and the like to each division manager. The division administrator can log in to the SVP 90 using a dedicated user ID issued by the system administrator. The division manager can change the setting in the CLPR under his / her management by operating the sub-terminal 2.

  FIG. 3 is a block diagram focusing on the logical configuration of the storage subsystem 10. The storage subsystem 10 includes a plurality of channel adapters (hereinafter referred to as CHAs) 20, a plurality of disk adapters (hereinafter referred to as DKAs) 30, at least one cache memory 40 and shared memory 50, a switch unit 60, and a plurality of switches. Virtual logical device (VDEV) 70, a plurality of disk drives 80, and an SVP 90.

  Each CHA 20 controls data transfer with each host computer 1 and includes a communication port 21. For example, 32 CHAs 20 can be provided in the storage subsystem 10. The CHA 20 is prepared according to the type of the host computer 1 such as an open system CHA, a mainframe system CHA, and the like.

  Each CHA 20 receives a command and data for requesting reading / writing of data from the host computer 1 connected thereto, and operates according to the command received from the host computer 1. For example, when the CHA 20 receives a data read request from the host computer 1, the read command is stored in the shared memory 50. The DKA 30 refers to the shared memory 50 at any time. When an unprocessed read command is found, the DKA 30 reads data from the disk drive 80 and stores it in the cache memory 40. The CHA 20 reads the data transferred to the cache memory 40 and transmits it to the host computer 1 that issued the command.

  When the CHA 20 receives a data write request from the host computer 1, the CHA 20 stores the write command in the shared memory 50. Further, the CHA 20 stores the received data (user data) in the cache memory 40. Here, since the data requested to be written by the host computer 1 is “dirty data” that has not been written to the disk drive 80, it is stored and multiplexed, for example, at a plurality of locations.

  The CHA 20 stores the data in the cache memory 40 and then reports the completion of writing to the host computer 1. Then, the DKA 30 reads the data stored in the cache memory 40 according to the write command stored in the shared memory 50 and stores it in a predetermined disk drive 80. The attribute of the data written in the disk drive 80 changes from “dirty data” to “green data” and is released from multiple management by the cache memory 40. In this specification, “dirty data” means data that is not written in the disk drive 80. “Clean data” means data already written in the disk drive 80.

  A plurality of DKAs 30 can be provided in the storage subsystem 10 such as four or eight. Each DKA 30 controls data communication with each disk drive 80. Each DKA 30 and each disk drive 80 are connected via a communication network CN4 such as a SAN, for example, and perform block-unit data transfer according to a fiber channel protocol. Each DKA 30 monitors the state of the disk drive 80 as needed, and the monitoring result is transmitted to the SVP 90 via the internal network CN3. Each CHA 20 and each DKA 30 include, for example, a printed circuit board on which a processor, a memory, and the like are mounted, and a control program (not shown) stored in the memory, and cooperation between these hardware and software. A predetermined function is realized by each work.

  The cache memory 40 stores, for example, user data. The cache memory 40 is composed of, for example, a nonvolatile memory. The cache memory 40 can be composed of a plurality of memories, and can manage multiple dirty data. In this embodiment, each of the CLPR0 to CLPRn is set by dividing the entire cache area of the cache memory 40 into a plurality of parts.

  The shared memory (or control memory) 50 is composed of, for example, a nonvolatile memory. For example, control information and management information are stored in the shared memory 50. Information such as control information can be multiplexed and managed by a plurality of shared memories 50. A plurality of shared memories 50 and cache memories 40 can be provided. Further, the cache memory 40 and the shared memory 50 can be mixed and mounted on the same memory board. Alternatively, a part of the memory can be used as a cache area and the other part can be used as a control area.

  The switch unit 60 connects each CHA 20, each DKA 30, the cache memory 40, and the shared memory 50. Thereby, all the CHAs 20 and DKAs 30 can access the cache memory 40 and the shared memory 50, respectively. The switch unit 60 can be configured as, for example, an ultrahigh-speed crossbar switch or the like.

  The storage subsystem 10 can mount a large number of disk drives 80. Each disk drive 80 can be realized as, for example, a hard disk drive (HDD) or a semiconductor memory device. The disk drive 80 is a physical storage device (PDEV). For example, a virtual logical device (VDEV) 70 is constructed on a physical storage area provided by a set of four disk drives 80, although it differs depending on the RAID configuration or the like. Further, a virtual logical device can be set on the VDEV 70. Further, it is not necessary to provide all storage resources used by the storage subsystem 10 in the storage subsystem 10, and storage resources existing outside the storage subsystem 10 can be used. For example, by assigning a storage device of an external third-party storage subsystem to a specific VDEV 70 and managing an access path to the third-party storage device, the third-party storage device is as if it were its own storage device. Can be used.

  FIG. 4 is an explanatory diagram schematically showing how each CLPR divides and uses resources. In the figure, a case where the cache memory 40 is divided and used by the shared area CLPR0 and the two dedicated CLPR1 and 2 is shown. Each CLPR is assigned a cache area. The maximum value (maximum allocation amount) of the cache area that can be used by each CLPR is set by the system administrator. Data (write data, read data) used by the host computer 1 using each CLPR is stored in the cache area allocated to each CLPR. In FIG. 4, the area in which the data is stored is shaded and indicated as “in use”. Each CLPR can use a cache area up to the maximum allocation amount that can be used by each CLPR, and cannot use a cache area of another CLPR.

  Further, each CLPR manages other cache related data in addition to the respective cache areas. Examples of the cache-related data include DCR (F1), side file F2, and PCR (F3).

  The DCR (Dynamic Cache Residency) 100 is a cache resident function, and makes data existing in a specific area of the VDEV 70 resident in the cache area. Thereby, the access performance with respect to an important data group can be improved.

  The side file F2 is a file that stores data for transferring and copying data to a remote site (not shown). For example, when a predetermined amount or more of remote copy target data is accumulated in the side file F2, the data is transferred and held via a communication network to another storage subsystem located at a distance. In the initial copy of remote copy, the entire designated data group is copied to the remote site, and the data updated thereafter is remotely copied as a differential file.

  The PCR (Partical Cache Residence) 102 is used when, for example, a NAS (Network Attached Storage) function is provided in the storage subsystem 10, and is a function that makes the data resident in the cache area for each type of data. DCR (F1) makes data existing in a specific storage space resident, while PCR (F3) makes data of a specific type resident. The cache-related data DCR (F1), the side file F2, and the PCR (F3) are provided only in the CLPR that requires them. That is, the data regarding the cache residency is not stored in the CLPR in which the residency function is not set. Similarly, the side file F2 is not stored in the CLPR for which remote copy is not set.

  As shown in FIG. 4, at least one VDEV 70 is assigned to each CLPR. Data written to the cache area of each CLPR is written to a predetermined area of the VDEV 70 assigned to each CLPR. The data read from the VDEV 70 is held in the corresponding CLPR cache area.

  FIG. 5 is an explanatory diagram showing a method for managing data stored in the cache area. The storage subsystem 10 manages the data stored in the cache memory 40 in the following hierarchical structure in order to efficiently search the data.

  First, when there is a data input / output request from the host computer 1, a VDEVSLOT number (VDEVSLOT #) is obtained based on an LBA (Logical Block Address) included in the input / output request. Then, based on the VDEVSLOT number, a pointer to the next hierarchy is obtained by referring to the VDSLOT-PAGE table T1. The VDSLOT-PAGE table T1 includes a pointer to the PAGE-DIR table T2. The PAGE-DIR table T2 includes a pointer to the PAGE-GRPP table T3. Further, the PAGE-GRPP table T3 includes a pointer to the GRPT1 table T4. The GRPT1 table T4 includes a pointer to the GRPT2 table T5. The GRPT2 table T5 includes a pointer to an SLCB (slot control table) T6.

  In this manner, the SLCB table T6 is reached by sequentially referring to the tables T1 to T5 based on the LBA. At least one SGCB (segment control block) table T7 is associated with the SLCB table T6. The SGCB table T7 stores control information related to a segment that is a minimum unit of cache management. One to four segments can be associated with one slot. One segment can store, for example, 48 KB or 64 KB of data.

  The smallest unit of cache management is a segment, but the transition between each state of dirty data (the state before writing to the physical disk) and clean data (the state after writing to the physical disk) is performed in units of slots. . Further, securing (reserving) and releasing (releasing) the cache area is also performed in slot units or segment units.

  As shown in FIG. 8, the SLCB table T6 includes a backward pointer, a forward pointer, a queue status, VDSLOT #, slot status, CLPR information, and at least one SGCB pointer. be able to. The queue status includes a queue type and a queue number associated with the SLCB table T6. The slot status includes the status of the slot corresponding to the SLCB table T6. The CLPR information includes information (CLPR number and the like) related to the CLPR to which the SLCB table T6 belongs. The SGCB pointer includes a pointer for specifying the SGCB table T7 associated with the SLCB table T6.

  The SLCB table T6 and the SGCB table T7 are exclusively controlled in units of VDEV 70 so that, for example, a conflict such as a pointer change occurs and the consistency is not lost. That is, the SLCB table T6 or the SGCB table T7 can be operated only when the lock is acquired for the VDEV 70 to which the SLCB table T6 or the SGCB table T7 belongs. In this specification, this VDEV exclusive control is referred to as “VDEV lock”. In other words, VDEV lock is exclusive control in units of logical volumes.

  FIG. 6 is an explanatory diagram showing state transition of the SLCB table T6. The SLCB table T6 can take, for example, five statuses: an unused state ST1, a device reflected state ST2, no parity & pre-device reflection state ST3, a parity generating state ST4, and a parity present & drive pre-reflection state ST5. .

  The unused state ST1 is a state indicating that the slot is unused. The unused slot (SLCB) is managed by a free SGCB queue (hereinafter abbreviated as free queue). The device reflected state ST2 is a state indicating that the data stored in the slot has been written to the physical device (disk drive 80). The slots in the device reflected state ST2 are managed by a clean queue.

  No parity & pre-device reflection state ST3 is a state indicating that parity data has not been generated for data stored in the slot when parity data is required, such as RAID5. . The slots in the state ST3 with no parity and before device reflection are managed by the intermediate dirty queue. The parity generation state ST4 is a state indicating that parity data is being generated for the data stored in the slot. Slots in the parity generation state ST4 are managed by the host dirty queue.

  The parity state & drive pre-reflection state state ST5 is a state indicating that parity data has been generated for data stored in the slot but has not yet been written to the physical device. A slot in the state ST5 with parity & before drive reflection is managed by a physical dirty queue (hereinafter abbreviated as a dirty queue).

  When there is a write access from the host computer 1, the state transitions from ST1 to ST3 to ST3, and generation of parity data is started. When parity data is generated for the data requested to be written, the state changes from ST3 to ST4 to ST5. When data is written from the host computer 1 to one or more slots in the CLPR cache area, a write completion report is transmitted to the host computer 1. At this time, since the data requested to be written exists only in the cache area, it is in a dirty state (ST5).

  When the data of the slot in the dirty state is written to the predetermined physical device (destage), the slot changes from the dirty state to the clean state (ST5 → ST2). Since the data in the clean state is written in the physical device, it may be erased from the cache area if necessary. When the data stored in the slot in the clean state is erased from the cache, the slot returns from the clean state to the unused state (ST2 → ST1).

  When there is a read access from the host computer 1, the requested data is specified by sequentially tracing the hierarchical tables T1 to T6 described with reference to FIG. Then, the data requested by the host computer 1 is read from the physical device and stored in one or more slots of the cache area. A slot for storing data read from the physical device has a clean state.

  FIG. 7 is an explanatory diagram showing a queue structure. The queue 100 is configured as a queue that connects the SLCB table T6 in the same state in one direction or in both directions. When the state of the SLCB table T6 is changed, for example, based on a preset algorithm such as FIFO (First-In First-Out), LRU (Least Recently Used), etc. Configuration changes.

  Exclusive control is performed for each queue 100 in order to prevent a loss of consistency due to conflicting operations on the SLCB table T6. That is, an operation on the queue can be performed only when the lock of the queue to be operated is acquired. In this specification, this exclusive control for each queue is referred to as “queue lock”. Accordingly, the SLCB table T6 can be operated under the above-described two-stage exclusive control of the VDEV lock and the queue lock. That is, the contents of the SLCB table T6 can be changed only when two locks are acquired: the lock of the VDEV 70 to which the SLCB table T6 to be operated belongs and the lock of the queue to which the SLCB table T6 is connected. it can.

  As described above, since operations on the queue 100 are exclusively controlled, a plurality of queues of the same type are provided in consideration of overhead at the time of acquiring a lock. Each of the clean queue, the dirty queue, the free queue, and the like includes a plurality of queues.

  A counter 200 is associated with each queue constituting the queue set. The counter 200 manages the number of SLCB tables T6 and SGCB tables T7 belonging to the queue. Based on the number of SLCBs managed by the counter 200, overload control described later can be performed.

  In addition, a queue management table 300 is also associated with each queue 100 constituting the queue set. The queue management table 300 manages the structure of the queue 100 and the like. The queue 100, the counter 200, and the queue management table 300 are stored in the shared memory 50, respectively.

  FIG. 9 is an explanatory diagram schematically showing the relationship between the usage state of the cache area of each CLPR and each queue. The cache area of each CLPR can be divided into “in-use area” and “unused area” according to the use state. The in-use area means a slot group (segment group) storing data. An unused area means an empty slot group (segment group).

  The product of the total number of segments used and the data size per segment is the size of the area in use in the CLPR. The segments that are used include segments that are in a clean state and segments that are in a dirty state. Therefore, the size of the currently used area among the cache areas allocated to the CLPR can be expressed as (number of clean segments + number of dirty segments). The size of the unused area can be represented by the number of segments in the free state.

  The free queue 101 manages the SLCB table T6 in a free state (unused state). The number of SLCB tables T6 (and SGCB table T7) connected to the free queue 101 is grasped by the free counter 201 associated with the free queue 101. The control unit of the free queue 101 is a VDEV, and the queue lock unit is also a VDEV unit. The free queue 101 is used according to, for example, a FIFO. The free queue 101 is provided twice as many as the number of mounted VDEVs 70, for example. Accordingly, if the maximum number of mountable VDEVs 70 is 512, the total number of free queues 101 is 1024.

  The clean queue 102 manages the SLCB management table T6 in the clean state. The number of SLCB tables T6 (and SGCB table T7) connected to the clean queue 102 is grasped by the clean counter 202 associated with the clean queue 102. The control unit of the clean queue 102 is a VDEV, and the queue lock unit is also a VDEV unit. The clean queue 102 is used according to, for example, LRU. The clean queue 102 is provided, for example, four times as many as the number of mounted VDEVs (512 × 4 = 2048).

  The dirty queue 103 manages the SLCB management table T6 in the dirty state. The number of SLCB tables T6 (and SGCB table T7) connected to the dirty queue 103 is known by the dirty counter 203 associated with the dirty queue 103. The control unit of the dirty queue 103 is a physical drive. The dirty queue 103 is provided, for example, 32 times the number of disk drives 80 mounted. When 2048 disk drives 80 are mounted, 65536 dirty queues 103 can be provided. The dirty queue 103 is for managing data (segments) before being written to the physical device. Therefore, the physical device (disk drive 80) is more relevant than the logical device (VDEV 70). Therefore, the dirty queue 103 is set depending on the number of physical devices.

  The in-use counter 206 counts the number of segments in use in each CLPR (the number of dirty segments + the number of clean segments).

  FIG. 10 is an explanatory diagram showing the relationship between each CLPR and each queue and counter. Each queue, counter, and management table described above constitute management information for managing a cache management unit (segment or slot). In this embodiment, the management information (divided management information) provided for each CLPR and the common management information applied to all CLPRs are distinguished according to the nature (status) of the segment or slot.

  As shown in FIG. 10, each CLPR includes a clean queue 102, a clean counter (more precisely, a clean queue counter) 202, a free queue counter 201, a color-coded queue counter 204, a busy counter 206, and a BIND queue. A counter 205 is provided. Although not shown, a clean queue management table for managing the configuration of the clean queue 102 is also provided for each CLPR. As described above, the management information provided for each CLPR constitutes the divided management information.

  On the other hand, the free queue 101, the dirty queue 103, the dirty counter (exactly the dirty queue counter) 203, the color coding queue 104, the BIND queue 105, and the queues 101 and 103 to 105 are managed. The queue management table 301 constitutes common management information that is commonly applied to all CLPRs.

  The color classification queue 104 is a queue for managing data that is made resident in the cache area by the PCR function. The color coding queue counter 204 counts the number of segments that are made resident by PCR.

  The BIND queue 105 is a queue for managing data that is resident in the cache area by the DCR function. The BIND queue counter 205 counts the number of segments (or the number of slots) resident by the DCR.

  By providing the color coding queue 104 and the BIND queue 105 for managing the segments for storing the data to be resident, the segments for storing the data to be resident in each CLPR can be prevented from being changed to the free state. can do.

  Next, the reason why each sub management information (queue, counter, queue management table) constituting the management information is provided for each CLPR and which is applied to all CLPR will be described.

  The clean queue 102, the clean counter 202, and the clean queue management table are provided for each CLPR. When free segments are depleted, clean slots are freed to become free slots. Therefore, the clean queue 102 is the most frequently used queue. If a set of clean queues is used for all CLPR, if a free segment is depleted in one CLPR, a clean slot for storing data used by other CLPR may be released. There is sex. That is, the cache usage state in one CLPR affects the cache usage state in the other CLPR. Therefore, in this embodiment, a clean queue 102 and a clean counter 202 can be provided for each CLPR. The queue management table is handled integrally with the queue. Therefore, when the clean queue 102 is provided for each CLPR, a clean queue management table is also provided for each CLPR.

  A busy counter 206 is provided for each CLPR. The busy counter 206 counts a value obtained by adding the number of clean segments and the number of dirty segments. Therefore, the value obtained by subtracting the count value of the clean counter 202 from the count value of the in-use counter 206 is nothing but the number of dirty segments in the CLPR. Therefore, if the clean counter 202 and the in-use counter 206 are provided for each CLPR, it is not necessary to provide the dirty counter 203 for each CLPR.

  The dirty queue 103 is a queue related to a physical device, and the control unit is also a physical device unit. Therefore, if the dirty queue 103 is provided for each CLPR, the storage capacity for storing the dirty queue management table increases and the storage area of the shared memory 50 is compressed. For example, the amount of information required per queue is 16 bytes for the queue management table and 4 bytes for the counter. Since the division of the queue means the division of the queue management table, many storage areas of the shared memory 50 are consumed for storing the division management information. Therefore, in this embodiment, in order to efficiently use the storage resources of the shared memory 50, the dirty queue 103 and the dirty counter 203 are not provided for each CLPR, but are applied to all the CLPRs.

  Further, from the viewpoint of effectively using the storage resources of the shared memory 50, the free queue 101 is not provided for each CLPR, and only the free counter 201 having a small number of bytes is provided for each CLPR.

  For the same reason as the free queue 101, the color classification queue 104 and the BIND queue 105 are also provided with only the respective counters 204 and 205 for each CLPR, and the queue itself is applied to all the CLPR.

  Thus, in this embodiment, the sub management information is distributed according to the attribute of the sub management information, that is, according to the attribute of the cache management unit managed by the sub management information. Therefore, it is possible to efficiently use the storage resources of the shared memory 50 and ensure independence between the CLPRs.

  FIG. 11 is an explanatory diagram showing the relationship between the cache management structure and exclusive control. In FIG. 11, only a part of the management information described with FIG. 10 is shown. “VDEV lock Gr” in FIG. 11 indicates a group of VDEV locks. In order to operate the directory, it is necessary to acquire a VDEV lock, and in order to operate the queue, it is necessary to operate the queue lock. In other words, in order to move the data managed by the CLPR, both the operation authority to the queue that manages the segment storing the data and the operation authority to the directory to which the segment belongs are required.

  As shown on the right side of FIG. 11, one or more VDEVs 70 can be associated with each CLPR. A VDEV 70 associated with one CLPR can also be associated with another CLPR. FIG. 12 is an explanatory diagram showing how VDEV # 1 associated with CLPR0 is replaced with CLPR1.

  When changing the affiliation CLPR to which the VDEV 70 belongs, it is necessary to change the association between the VDEV 70 and the CLPR, and further associate the data group belonging to the migration target VDEV 70 with the new CLPR. That is, the segment (or slot) to be moved is removed from the queue associated with the source CLPR and connected to the queue associated with the destination CLPR. For this operation, both VDEV lock and queue lock are required.

  FIG. 13 shows an example of processing when the CLPR configuration is changed. FIG. 13 shows an outline in the case where the VDEV 70 associated with the source CLPR is moved to the destination CLPR.

  First, when the system administrator requests the migration of the VDEV 70 via the management terminal 3 (S1: YES), the representative processor of the CHA 20 or DKA 30 changes the unique information of the VDEV 70 designated as the migration target (S2). That is, the information regarding the CLPR to which the VDEV 70 belongs (affiliation destination CLPR information) is rewritten with the information regarding the CLPR of the movement destination.

  Next, the representative processor searches and extracts the SLCB table T6 corresponding to the migration target VDEV 70, and changes the CLPR information for the migration target SLCB table T6 (S3). That is, as described above with reference to FIG. 8, for each SLCB table T6 that is a movement target, the information regarding the affiliation destination CLPR is changed to the information regarding the movement destination CLPR.

  Then, the representative processor checks whether or not the slot status of the SLCB table T6 to be moved is a clean state (S4). If the slot is in the clean state (S4: YES), that is, if the SLCB table T6 to be moved is connected to the clean queue 102, the representative processor matches the queue status of the SLCB table T6 with the destination CLPR. (S5). That is, information such as the number of clean queues 102 provided in the movement destination CLPR is rewritten.

  After changing the queue status, the representative processor removes the SLCB table T6 to be moved from the current clean queue 102 (S6). Then, the representative processor decrements the count values of the in-use counter 206 and the clean counter 202 provided in the movement source CLPR according to the number of clean slots to be moved (S7). Next, the representative processor connects the SLCB table T6 to be moved to the clean queue 102 of the movement destination CLPR (S8). Then, the representative processor increases the count values of the in-use counter 206 and the clean counter 202 in the destination CLPR (S9), and ends this processing.

  On the other hand, when the SLCB table T6 corresponding to the movement target VDEV 70 is not in the clean state (S4: NO), the representative processor checks whether or not the movement target SLCB table T6 is in the free state (S10). When the SLCB table T6 to be moved is in a free state (S10: YES), the representative processor changes the free counter 201 of the movement source and the movement destination, respectively (S11). That is, in accordance with the number of free slots to be moved, the movement source free counter 201 is subtracted, the movement destination free counter 201 is added, and the process is terminated. If the SLCB table T6 to be moved is in a dirty state (S10: NO), the process is terminated without doing anything. Since the dirty queue 103 and the dirty counter 203 are commonly applied to all CLPRs, there is no need to perform queue movement or increase / decrease of the counter.

  Although omitted in FIG. 13, the SLCB table T6 connected to the color classification queue 104 and the BIND queue 105 is processed in the same manner as the free queue 101. The color separation queue 104 and the BIND queue 105 are not provided separately for each CLPR, but only the counters 204 and 205 are provided for each CLPR. Therefore, there is no movement between the queues, and only a change of the counter value at the movement source and the movement destination occurs.

  FIG. 14 is a flowchart showing a schematic process for changing the CLPR cache allocation amount. When a change operation of the cache allocation process is performed by the system administrator (S31: YES), the representative processor detects the SLCB table T6 related to the change of the cache allocation (S32). Then, the representative processor changes the CLPR information of the SLCB table T6 related to the change according to the change of the cache allocation (S33).

  For example, when the cache allocation amount of CLPR1 is decreased and the cache allocation amount of CLPR2 is increased by that amount, the attribution destination CLPR of the SLCB table T6 located in this variable region is changed from CLPR1 to CLPR2. Then, according to the cache area allocation change, the value of the clean counter 202 or the like is increased or decreased, and the process is terminated (S34).

  FIG. 15 is a flowchart showing an outline of a free segment collection process for securing a free segment. Here, the free segment means a segment in a free state. This processing can be performed periodically or when the free segment is insufficient (overload).

  First, the least frequently used SLCB table T6 in the clean state is detected (S41), and the status of the detected SLCB table T6 is changed from the clean state to the free state (S42). Along with this status change, the SLCB table T6 is changed from the clean queue 102 to the free queue 101. Further, the count values of the clean counter 202 and the free counter 201 are also changed. The processes of S41 and S42 are repeated until a predetermined amount of free segments are secured (S43).

  FIG. 16 is a schematic diagram showing how CLPR is defined. In this embodiment, CLPR0 is used as a shared pool area. In CLPR0, various types of information commonly used in the storage subsystem 10 are stored. As described above, various types of management information are stored in the shared memory 50.

  As shown in FIG. 16A, in the initial state where no user-specific CLPR is set, all the cache areas of the cache memory 40 belong to CLPR0. As shown in FIG. 16B, the cache area allocated to CLPR0 can be distinguished into “in-use area” and “unused area” that store common information and the like. When a user-specific CLPR1 is newly set, as shown in FIG. 16C, as many cache areas as necessary are cut from the pool area CLPR0 and allocated to CLPR1. Further, when a new CLPR2 is set for another user, as shown in FIG. 16D, the cache area is cut out from the CLPR0 used as a pool area by a necessary amount and assigned to the newly generated CLPR2. . When deleting the user-specific CLPR1 or CLPR2, the cache area assigned to the CLPR to be deleted is returned to CLPR0.

  FIG. 17 is a flowchart showing the above-described processing. In the initial state, all cache areas of the cache memory 40 are allocated to the shared area CLPR0 used as a pool area (S51). Various information used in the entire storage subsystem 10 is stored in the CLPR0 and the operation is started (S52).

  When a new CLPR is added (S53: YES), the CLPR configuration is changed (S54). In this CLPR configuration change processing, processing such as cutting a cache area from CLPR0 and assigning it to a new CLPR is performed. On the other hand, when deleting the already generated CLPR (S53: NO, S55: YES), processing such as returning the cache area assigned to the CLPR to be deleted to CLPR0 is performed (S56). In principle, CLPR0 cannot be deleted.

  FIG. 18 is an explanatory diagram showing a state in which, for each sub-terminal 2, information related to CLPR managed by each sub-terminal 2 is displayed.

  The SVP 90 includes an authentication unit 91 and a configuration management unit 92. Each CLPR administrator accesses the SVP 90 via the sub-terminal 2 and inputs a user ID and the like. As a result of user authentication by the authentication unit 91, if it is recognized that the user is a valid user, the CLPR administrator can change the configuration of the CLPR under his / her management. This configuration change operation is reflected on each CLPR by the configuration management unit 92. Further, the configuration information regarding the CLPR acquired from the configuration management unit 92 can be displayed on the sub-terminal 2.

  Information that can be displayed on the sub-terminal 2 includes, for example, the maximum cache amount allocated to the CLPR, information on the VDEV allocated to the CLPR (VDEV number, volume size, etc.), the cache size in use, and unused You can list cache size, IO frequency to CLPR, etc. Note that a system administrator who manages the entire storage subsystem 10 can obtain information on all the CLPRs via the entire terminal 3, and can change the configuration of each CLPR.

  As described above, according to this embodiment, the management information is divided for each CLPR set in the storage subsystem 10. Therefore, interference between CLPRs can be suppressed, and usability can be improved. That is, even when access to one CLPR increases, it is possible to prevent the other CLPR from being affected as much as possible.

  In this embodiment, the management information is distributed based on the segment or slot attributes. Therefore, the data size of the entire management information can be reduced and the shared memory 50 can be used efficiently compared to a configuration in which all management information is simply provided in each CLPR.

  The present invention is not limited to the above-described embodiment. A person skilled in the art can make various additions and changes within the scope of the present invention.

1 is an external view of a storage subsystem according to an embodiment of the present invention. It is a block diagram which shows the logical schematic structure of a storage subsystem. FIG. 3 is a more detailed block diagram of the storage subsystem. It is explanatory drawing which shows the relationship between the cache area allocated for every CLPR, and VDEV. It is explanatory drawing which shows the cache management method which has a hierarchical structure. It is a state transition diagram of a SLCB table. It is explanatory drawing which shows the relationship between a queue, a counter, and a queue management table. It is explanatory drawing which shows schematic structure of a SLCB table. It is explanatory drawing which shows the relationship between a cache area | region, each queue, and a counter. It is a schematic diagram which shows a mode that the management information of CLPR is distributed. It is explanatory drawing which shows a mode that data movement is attained by VDEV lock and queue lock. It is explanatory drawing which shows a mode that a VDEV is moved from CLPR0 to CLPR1. It is a flowchart which shows the process in the case of moving VDEV. It is a flowchart in the case of changing the cache allocation to CLPR. It is a flowchart which shows the collection process of a free segment. It is explanatory drawing which shows a mode that a cache area | region is cut out from CLPR0 used as a pool area | region, and a new CLPR is set. It is a flowchart which shows the process in the case of defining CLPR. It is explanatory drawing which shows a mode that the information regarding each CLPR is displayed on a sub management terminal.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Host computer, 2 ... Sub-terminal, 3 ... Whole terminal, 10 ... Storage subsystem, 11 ... Basic enclosure, 12 ... Expansion enclosure, 13 ... Control package, 14 ... Power supply unit, 15 ... Battery unit, 16 ... Cooling fan, 20 ... channel adapter, 21 ... communication port, 30 ... disk adapter, 40 ... cache memory, 50 ... shared memory, 60 ... switch unit, 70 ... logical device, 80 ... disk drive, 90 ... service processor, 91 ... Authentication unit, 92 ... Configuration management unit, 100 ... Queue, 101 ... Free queue, 102 ... Clean queue, 103 ... Dirty queue, 104 ... Color-coded queue, 105 ... BIND queue, 200 ... Counter, 201 ... Free queue counter, 202 ... Clean queue counter, 203 ... Dirty queue counter, 2 04 ... Color-coded queue counter, 205 ... BIND queue counter, 206 ... In-use counter, 300 ... Queue management table, CLPR ... Logical cache partition area, SLPR ... Virtual enclosure, SLCB ... Slot control table, SGCB ... Segment control block, F1 ... DCR, F2 ... side file, F3 ... PCR, CN ... communication network, ST ... status, T1-T7 ... table

Claims (2)

A plurality of channel adapters each controlling data exchange with the host device;
A plurality of storage devices each providing a logical storage area;
A plurality of disk adapters for controlling data exchange with each storage device group;
Cache memory respectively used by each channel adapter and each disk adapter;
A plurality of cache division regions configured by logically dividing the cache memory;
A control memory for storing management information for managing the data stored in each cache partition area for each cache partition area ;
A logical device provided on a physical storage area of each storage device group and assigned to each cache partition area;
With
The management information is composed of partition management information provided for each cache partition area, and common management information applied to the entire cache partition area , and
The management information is
A free queue to which an unused cache management unit is connected;
A free queue counter that is associated with the free queue and counts the number of unused cache management units connected to the free queue;
A dirty queue to which a cache management unit for storing dirty state data before reflection in the storage device group is connected;
A dirty queue counter that is associated with the dirty queue and counts the number of cache management units that store the dirty data connected to the dirty queue;
A clean queue to which a cache management unit for storing clean data reflected in the storage device group is connected;
A clean queue counter that counts the number of the cache management units that are associated with the clean queue and store the clean data connected to the clean queue;
And a busy counter that counts the total amount of cache management units in use,
The free queue counter, the clean queue, the clean queue counter, and the in-use counter are provided for each cache partition area, respectively, and configure the partition management information, respectively.
The free queue, the dirty queue and the dirty queue counter are used as the common management information,
In the case of changing the allocation destination of the migration target logical device assigned to the migration source cache division area among the cache division areas to the migration destination cache division area of the cache division areas,
When the migration target cache management unit corresponding to the migration target logical device stores the clean state data,
Removing the migration target cache management unit from the clean queue corresponding to the migration target cache management unit provided in the migration source cache partition area;
The clean queue counter associated with the clean queue provided in the migration source cache partition area and the in-use counter provided in the migration source cache partition area are transferred to the migration target cache. Subtract by the number of administrative units,
The cache management unit to be moved that stores the data in the clean state is connected to the clean queue corresponding to the cache management unit to be moved, provided in the cache partition area of the move destination,
The clean queue counter associated with the clean queue provided in the migration-destination cache partition area and the in-use counter provided in the migration-destination cache partition area are converted into the cache to be moved. Add the number of management units,
When the cache management unit to be moved is in the unused state,
Subtract the free queue counter provided in the migration source cache partition area by the number of cache management units to be migrated,
The free queue counter provided in the cache partition area of the transfer destination is added by the number of cache management units to be moved,
Storage subsystem. The storage subsystem according to claim 1, wherein each cache partition area can be set for each channel adapter.

Images