JP4028833B2 - Apparatus, method and computer program for controlling I / O operations of storage area in network of storage area controller node - Google Patents

Description

The present invention relates to the field of computer storage controllers, and more particularly to high performance storage controllers connected in an n-way storage configuration with shared storage.

In the field of advanced computer storage, there is a growing demand for what is called "high functionality". These functions go beyond the simple I / O functions of conventional storage systems. Advanced functionality is well known in the art and relies on control of metadata used to maintain state information about real or “user” data stored in the system. Operations that can be performed using advanced features allow various actions to be immediately applied to the virtual image of the data while the user application remains available to the “real” data. An example of such a high function is “flash copy” and the present invention is not limited to this function, but the description of one embodiment of the present invention using flash copy as an example only is Suitable for explaining the background.

Flash copy at the highest level is a feature that makes a second image of “some data” available. This function is also known as point-in-time copy, or T0 copy. The content of the second image is initially identical to that of the first. The second image is available “instantly”. This actually means that the second image will be available in much less time than is required to create another real physical copy, making the operation of the application in use unacceptable. It means that it can be established without interruption.

Once established, the second copy can be used for several purposes, including backups, system trials, and performing data mining. The first copy continues to be used for its original purpose by the original busy application. On the other hand, in the case of backup without flash copy, the application must be shut down and the application must be restarted after taking the backup. Finding time to block a window when the application is sufficiently idle becomes increasingly difficult. The cost of taking backups is also increasing. The ability of FlashCopy to take backups without disrupting business has an increasingly important business value.

By performing a flash copy, the read I / O addressed to the second image (hereinafter referred to as “target”) is transferred to the original image (hereinafter referred to as “source”) Unless the region is the target of writing, the presence of the second image is illusioned. If the region is subject to writing (either to the source or target), to maintain the illusion that both the source and target own their own copy of the data, Suspend, issue a read of the affected area from the source without enabling it, write the read data to the target by writing, and then write (only if all steps were performed successfully) The process of canceling the interruption is called. Since the target will already have its own copy of the data, subsequent writes to the same area need not be interrupted. This copy-on-write technique is well known and is used in many environments.

There are various variations in the method of performing flash copy. These variations are shown through various features of implementation. For example, certain implementations can read and write to the target, while other implementations can only read. Some implementations allow only limited updates to the target. Some targets must be the same size as the source, others may be smaller.

However, all implementations must suspend the above decision, i.e., whether a read received at the target is issued to the source or target, as well as writing to be able to perform copy-on-write Rely on some data structure that governs the decision on whether or not. The data structure essentially tracks the area copied from the source to the target as different from the non-copied area.

The maintenance of this data structure (hereinafter referred to as metadata) is the key to executing the algorithm behind the flash copy. Other “high functionality” such as remote copy (also known as PPRC or continuous copy, or remote mirroring) or virtualization relies on similar data structures and is therefore within the scope of the present invention. Although the metadata for each of these functions is different, in all cases it is used to maintain state information such as the location of the data and the mapping of the virtualized file to the actual storage object. Metadata is kept in persistent storage.

Functions such as flash copy are relatively easy to implement in a single (or with SMP processor) CPU complex, as can be seen from the fact that they are often used in modern storage controllers. is there. With very little effort, a fault tolerant flash copy can be implemented that allows (at least) two CPU complexes to access a copy of the metadata. If the first CPU complex fails, the second can be used to continue operation without losing access to the target image.

However, the I / O function of a single CPU complex is limited. There is a limit to improving the performance of a single CPU complex measured in terms of I / O per second or bandwidth (MB / s), which ultimately limits the performance of the application in use. . This limitation occurs in many flash copy implementations, a good example of which occurs in the storage controller. A typical storage controller has a single (or redundant pair) CPU complex that determines limits on the performance of the controller.

Many more storage controllers can be installed. However, because separate storage controllers do not share access to metadata, they do not cooperate in managing flash copy images. The storage space will be fragmented by functions such as flash copy that are limited to the scope of a single controller system. Both source and target disks must be managed within the same storage controller. Even if a single storage controller disk space fills up and some extra space remains on the other side, separate the source and target disks and place the target disk under the control of the new controller. A disk cannot be placed. (This is particularly disadvantageous for new flash copies where target movement is an inexpensive operation because there is no physical data associated with the target.)

The overall performance constraints possible for source / target pairs, as well as the single controller storage capability constraints, further complicate the management of the storage environment. Management costs are often quoted as the largest costs in the total cost of storage ownership. It is very advantageous to reduce the cost and complexity of the system by removing all possible constraints.

Typically, today's storage management systems do not attempt to solve this problem. In order to implement high functionality limited to a single controller, it is constrained by the capabilities of that controller.

A simple way to allow multiple controllers to participate in a shared flash copy relationship is to assign one controller as the owner of the metadata and all the read and write requests to that one controller. It is to make it forward to. The owner controller uses the algorithm described above to process the request as if it had come directly from its own connected host server and return each request back to the originating controller for completion.

The main drawbacks of these systems and the reasons they are not widely used are that the burden of transferring each I / O request is too heavy, doubling the total cost of the entire system, resulting in nearly half the system performance. It is to end.

In the field of distributed parallel database processing, assemble resources into "lock clubs", assign owners to lock clubs, and then control all locks in the region to which they are assigned, and send lock control messages to I / O It is known to have a distributed lock management mechanism that allows it to be issued to O requesting clients. These systems are implemented at the logical resource level and do not allow control of locks between storage controller systems, but also provide any form of lock management at the metadata level. do not do. Considerable overhead is also introduced during storage virtualization if the real data segments can be held in widely distributed physical media.

Academic literature, such as “Scalable Concurrency Control and Recovery for Shared Storage Arrays” by Amiri et al. Suggests that distributed lock management at the device level in storage controller networks is possible. However, such lock management techniques must perform messaging between controllers, which increases the latency period and increases the likelihood of deadlocks and repeated retries. It is known to be deterred by this burden.
K. Amiri, two others, "Scalable Concurrency Control and Recovery for Shared Storage Arrays", [online], February 1999, Technical Report ( Technical Report), CMUS 99.111 (CMU-CS-99-111), [October 10, 2003 search], Internet <URL: http: //reports-archive.adm.cs.cmu. edu / cs1999.html>

Thus, low cost, high performance, allowing N storage controllers (“n-way systems”) to participate in extended storage capabilities that can distribute ownership and locks at the metadata level among devices, And scalable schemes have significant advantages.

The present invention provides, in a first aspect, an apparatus for controlling storage I / O operations within a network of storage controller nodes. This device
(A) means for designating one of the storage controller nodes as an owner storage controller node and designating the rest of the storage controller nodes as client storage controller nodes; ,
The owner storage controller node owns metadata to control all I / O operations associated with one region of storage ;
(B) In response to the client storage controller node receiving an I / O request for the area, interrupt the processing of the I / O request and the data in the area has already been copied Means for sending a request message to the owner storage controller node to ask whether or not
(C) Whether the data in the region has already been copied by examining the metadata associated with the region in response to the owner storage controller node receiving the request message A means of determining
(D1) means for transmitting to the client storage controller node a response message indicating that the data in the area has already been copied in response to the positive determination result of the means for determining (c); ,
(D2) means for resuming processing of the I / O request in response to receipt of the response message by the client storage controller node;
(E1) placing a lock record for the metadata associated with the area under the control of the owner storage controller node in response to a negative decision result of the means for determining (c) And means for sending an authorization message to the client storage controller node indicating that the lock has been granted;
(E2) means for copying data in the region in response to the client storage controller node receiving the authorization message;
(E3) means for transmitting, upon completion of the copy operation by the means for copying (e2), a lock release message requesting release of the lock to the owner storage controller node;
(E4) further comprising means for removing the lock record from the metadata associated with the region in response to the owner storage controller node receiving the unlock message.

Preferably, the means for copying (e2) includes executing a copy-on-write operation .

Preferably, said means for copying (e2) performs a copy operation by using a flash copy algorithm .

In a second aspect, the present invention provides a method for controlling storage I / O operations within a network of storage controller nodes. This method
(A) designating one of the storage controller nodes as an owner storage controller node and designating the rest of the storage controller nodes as client storage controller nodes;
The owner storage controller node owns metadata to control all I / O operations associated with one region of storage;
(B) In response to receiving an I / O request for the region, the client storage controller node interrupts processing of the I / O request and the data in the region has already been copied Sending a request message to the owner storage controller node to ask whether or not
(C) In response to receiving the request message, whether the data in the region has already been copied by the owner storage controller node examining the metadata associated with the region Determining whether or not,
(D1) In response to the positive determination result of the determining step (c), the owner storage controller node sends a response message indicating that the data in the area has already been copied to the client. Sending to the storage controller node;
(D2) in response to receiving the response message, the client storage controller node resumes processing the I / O request;
(E1) In response to the negative determination result of the determining step (c), the owner storage controller node places and locks a lock record for the metadata associated with the region. Sending an authorization message to the client storage controller node indicating that
(E2) in response to receiving the authorization message, the client storage controller node copying the data in the region;
(E3) upon completion of the copying step (e2), the client storage controller node sends an unlock message requesting unlocking to the owner storage controller node;
(E4) in response to receiving the unlock message, the owner storage controller node further removes the lock record from the metadata associated with the region .

Preferably, the copying step (e2) includes performing a copy-on-write operation .

Preferably, the copying step (e2) is performed by using a flash copy algorithm .

The present invention, in the third aspect, when on a computer Ru is executed, provides a computer-program for executing the method according to the second aspect.

Thus, the presently preferred embodiment of the present invention uses message communication to coordinate metadata level locking activity among multiple storage controllers (or nodes) in an n-way system. Although message communication coordinates activity between nodes in the system, each node is still responsible for performing its own I / O.

The preferred embodiment of the present invention uses message communication to coordinate activity among multiple storage controllers in an n-way storage system. Although message communication coordinates activity between nodes in the system, each node is still responsible for performing its own I / O. Such message communication reduces the processing burden on the nodes in the system compared to the above-described prior art I / O transfer scheme for known storage areas. Protocol messages are much less burdensome because they are less (total bytes transferred) than the I / O they are involved in, but because messages are suitable for coalescing or batching, System overhead is further reduced.

As an example, consider an n-way system that performs flash copy. Assume that every node has access to storage managed by a cooperating set of n nodes. One of the nodes is designated as the owner of the metadata that controls all I / O relationships of one area of storage (102). Other nodes are designated as clients. In the current most preferred embodiment, one of the client nodes is further designated as the backup owner and a copy of the metadata to provide continued availability if the owner node fails To maintain.

Consider that a host I / O request reaches a particular client node ("C") (104). Assume that the host I / O request is either a read or write to the target disk, or perhaps a write to the source disk. Client C initiates processing by interrupting I / O (106). C then sends a message REQ to the owner node O (108) asking if the grain has been copied.

Upon receipt of the message REQ, O checks its own metadata structure. If O finds that the region has already been copied, it responds with a NACK message (110). If O finds that the region has not yet been copied, it will place a lock record for the appropriate metadata for the region in its own metadata structure and reply with a GNT message indicating that the lock has been granted. (112). Lock records guarantee compatibility between the request just received and accepted and other requests that may affect the same metadata while processing at C continues It is required to be. The details about how lock records are maintained and the compatibility constraints are the same as if the I / O was received locally by the O and are well known to those skilled in the art.

Upon receipt of the NACK message, C resumes (unpends) the original I / O request (114).

Upon receipt of the GNT message, C continues by performing the data transfer requested by the flash copy algorithm (116). For target reads, this means performing a read to the source disk. After a while, C processes the completion of the read request (118) and issues an UNL message to O to request to release the lock (120), at the same time, the host system that issued it Complete the original I / O request to

Upon receipt of the UNL message, O removes the lock record from its metadata table (122), possibly releasing other I / O requests that were suspended due to the lock. In the current most preferred embodiment, O then delivers an UNLD message to C indicating that the lock has been released (124), allowing C to reuse the resources associated with the original request. However, this is not required by the flash copy algorithm itself.

If it is a write (to either target or source), C must perform a copy-on-write (127). If all copy-on-write steps are complete and the original write I / O request is still suspended, C releases the lock and updates the metadata to record the copy-on-write An UNLC request for requesting to be issued is issued to O (126).

Upon receipt of the UNLC message, O marks the affected area as copied in the metadata (128), removes the lock record (130), and confirms that the part has been copied. An arbitrary waiting request is notified (132), and then an UNLD message indicating that the lock has been released is issued to C (134).

Upon receipt of the UNLD message, C releases the suspended write operation (136), which will be completed after some time, after which C completes the write operation to the host (138).

A recovery path is required to respond to disk I / O failures, message communication system failures, or node failures, but these requirements and implementations are well known to those skilled in the art. There is no need to explain here.

The above description was for a single I / O and a single client C. However, if you are one of ordinary engineers in the field, if there are multiple I / Os from multiple client nodes, O will continue to process all requests using the same algorithm. However, it will be clear how this scheme continues to work.

Turning now to FIG. 2, there is shown an apparatus according to the presently preferred embodiment of the present invention, which includes an owner (202), a client (204) I / O execution component, a storage area network. Embodied in a storage controller network comprising a portion of metadata (206) regarding data (208) held under the control of and a copy (209) of data (208). The device may be a ownership assignment component (210) for assigning ownership of metadata to the owner (202) and a flash copy image, a remote copy image, or a storage virtualization image. Including a lock management component (212) operable to control locking at the metadata (206) level during I / O activity to ensure consistency with any copy (209) . It also includes a messaging component (214) connected to the owner (202) and the client (204), which requests a response regarding the metadata status, grants the lock, locks Is operable to pass one or more messages between the client (204) and the owner (202) to request release and to signal that the lock has been released. The client (204) can receive the metadata by the owner (202), provided that the client (204) complies with the locking protocol at the metadata level controlled by the owner (202). It is operable to perform I / O on owned data.

The current most preferred embodiment of the present invention assigns different nodes the role of becoming the metadata owner of different regions of storage. This ensures that the single owner node itself is not a bottleneck in the system. An efficient scheme for balancing domain ownership (albeit at the database table level) can be seen from known prior art distributed parallel database processing schemes.

The foregoing method will typically be implemented in software running on one or more processors (not shown), which may be any suitable data such as magnetic or optical computer disks. It will be appreciated that it can be provided as a computer program element that is carried on a carrier (also not shown). Similarly, data transmission channels can include all described storage media as well as signal carrier media such as wired or wireless signal media.

The present invention can be suitably embodied as a computer program product for use in a computer system. Such implementations include a series of computer readable instructions fixed on a tangible medium such as a disk readable medium such as a diskette, CD-ROM, ROM, or hard disk, or an optical or analog communication line. Through a tangible medium that is not limited to these, or intangibly using a radio technique, including but not limited to microwave, infrared, or other transmission techniques. A series of computer readable instructions that can be transmitted to the computer system via any of the interface devices. The series of computer readable instructions embodies all or part of the functionality previously described herein.

Those skilled in the art will appreciate that such computer readable instructions can be written in a number of programming languages for use in many computer architectures or operating systems. Further, such instructions may be stored using any current or future memory technology including, but not limited to, semiconductor, magnetic or optical, or may include optical, infrared, or microwave. It can be transmitted using any current or future communication technique, including but not limited to these. Such a computer program product can be distributed as a removable medium with attached printed or electronic documents such as, for example, shrink-wrapped software, can be preloaded into a computer system, for example on a system ROM or fixed disk, or It is contemplated that it can be distributed from a server or electronic bulletin board via a network such as, for example, the Internet or World Wide Web.

One of ordinary skill in the art will appreciate that various modifications to the above-described embodiments will be apparent.

4 is a flow diagram illustrating the steps of a method according to a preferred embodiment of the present invention. 1 is a block diagram illustrating an apparatus according to a preferred embodiment of the present invention.

Explanation of symbols

202 Owner 204 Client 206 Metadata 208 Data 209 Data Image 210 Ownership Control 212 Lock Management Component 214 Message Communication Component

Claims (7)

An apparatus for controlling an I / O operation of the serial憶域in the network storage controller node,
(A) means for designating one of the storage controller nodes as an owner storage controller node and designating the rest of the storage controller nodes as client storage controller nodes; ,
The owner storage controller node owns metadata to control all I / O operations associated with one region of storage ;
(B) In response to the client storage controller node receiving an I / O request for the area, interrupt the processing of the I / O request and the data in the area has already been copied Means for sending a request message to the owner storage controller node to ask whether or not
(C) Whether the data in the region has already been copied by examining the metadata associated with the region in response to the owner storage controller node receiving the request message A means of determining
(D1) means for transmitting to the client storage controller node a response message indicating that the data in the area has already been copied in response to the positive determination result of the means for determining (c); ,
(D2) means for resuming processing of the I / O request in response to receipt of the response message by the client storage controller node;
(E1) placing a lock record for the metadata associated with the area under the control of the owner storage controller node in response to a negative decision result of the means for determining (c) And means for sending an authorization message to the client storage controller node indicating that the lock has been granted;
(E2) means for copying data in the region in response to the client storage controller node receiving the authorization message;
(E3) means for transmitting, upon completion of the copy operation by the means for copying (e2), a lock release message requesting release of the lock to the owner storage controller node;
(E4) further comprising means for removing the lock record from the metadata associated with the region in response to the owner storage controller node receiving the unlock message . The apparatus of claim 1, wherein said means for copying (e2) comprises performing a copy-on-write operation . The apparatus according to claim 1, wherein said means for copying (e2) performs a copy operation by using a flash copy algorithm . A method of controlling storage I / O operations within a network of storage controller nodes, comprising:
(A) designating one of the storage controller nodes as an owner storage controller node and designating the rest of the storage controller nodes as client storage controller nodes;
The owner storage controller node owns metadata to control all I / O operations associated with one region of storage;
(B) In response to receiving an I / O request for the region, the client storage controller node interrupts processing of the I / O request and the data in the region has already been copied Sending a request message to the owner storage controller node to ask whether or not
(C) In response to receiving the request message, whether the data in the region has already been copied by the owner storage controller node examining the metadata associated with the region Determining whether or not,
(D1) In response to the positive determination result of the determining step (c), the owner storage controller node sends a response message indicating that the data in the area has already been copied to the client. Sending to the storage controller node;
(D2) in response to receiving the response message, the client storage controller node resumes processing the I / O request;
(E1) In response to the negative determination result of the determining step (c), the owner storage controller node places and locks a lock record for the metadata associated with the region. Sending an authorization message to the client storage controller node indicating that
(E2) in response to receiving the authorization message, the client storage controller node copying the data in the region;
(E3) upon completion of the copying step (e2), the client storage controller node sends an unlock message requesting unlocking to the owner storage controller node;
(E4) The method further comprising: in response to receiving the unlock message, the owner storage controller node removes the lock record from the metadata associated with the region . The method of claim 4 , wherein the copying (e2) comprises performing a copy-on-write operation . The method of claim 4 , wherein the copying (e2) is performed by using a flash copy algorithm . Computer-program for executing the method according to any one of claims 4 to 6 when on the computer Ru is executed.

Images