KR20130139883A - Offload reads and writes - Google Patents

Abstract

Aspects of the subject matter described herein relate to offload reading and writing. In an aspect, a requestor wishing to send data sends a request for representation of the data. In response, the requester receives one or more tokens representing the data. The requestor may then provide one or more of these tokens to the component having a request to record the data represented by one or more tokens. In some example applications, such components may use one or more tokens to identify the data, reading the data or logically writing the data without further interaction with the requestor. Tokens may be invalidated by request or based on other factors.

Description

OFFLOAD READS AND WRITES}

One mechanism for transferring data is to read data from a file at a source location into main memory and write data from this main memory to a destination location. In some circumstances, this may work satisfactorily for relatively little data, but as the data increases, the time taken to read the data and transfer the data to another location increases. In addition, when data is accessed over the network, the network may impose additional delays when transferring data from the source location to the destination location. Moreover, security issues combined with the complexity of storage devices can complicate data transmission.

The subject matter claimed in this specification is not limited to embodiments that solve any shortcomings or operate only in environments as described above. Rather, these backgrounds are merely provided to illustrate exemplary technical areas in which some embodiments described herein may be practiced.

In short, aspects of the subject matter described herein relate to offload reading and writing. In an aspect, a requestor wishing to send data sends a request for a representation of the data. In response, the requester receives one or more tokens representing the data. The requestor can then provide one or more of these tokens to the component requesting to record the data represented by the one or more tokens. In some example applications, these components may use one or more tokens to identify the data and then read or logically write the data without further interaction with the requestor. Tokens can be invalidated by request or based on other factors.

This summary is provided to simply identify some aspects of the subject matter described further in the detailed description below. This Summary is not intended to identify key or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.

The phrase “subject matter described herein” refers to the subject matter described in the detailed description unless the context clearly indicates otherwise. The term "aspects" should be read as "at least one aspect". The identification of aspects of the subject matter described in the detailed description is not intended to identify key or essential features of the claimed subject matter.

The foregoing and other aspects of the subject matter described in this specification are illustrated by way of example and not by way of limitation in the figures of the accompanying drawings in which like references indicate similar elements.

1 is a block diagram illustrating an exemplary general-purpose computing environment in which aspects of the subject matter described herein may be incorporated.
2-5 are block diagrams illustrating an exemplary arrangement of components of a system in which aspects of the subject matter described herein may operate.
6-8 are flow diagrams illustrating exemplary operations that may generally occur in accordance with aspects of the subject matter described herein.

Justice

As used herein, the term "includes" and variations thereof is an open-ended term meaning "includes, but is not limited to." Should be read as The term "or" is to be understood as "and / or" unless the context clearly dictates otherwise. The term "based on" should be understood as "based at least in part on." The terms "one embodiment" and "an embodiment" are to be understood as "at least one embodiment". The term "another embodiment" should be understood as "at least one other embodiment". Other explicit and implicit definitions may be included below.

The terms "first", "second", "third" and the like are sometimes used here. In particular, the use of these terms in the claims is not intended to be in the order and in fact are used for identification purposes. For example, the phrases "first data" and "second data" do not necessarily mean that the first data is physically or logically located before the second data or that the first data is requested or operated before the second data. Rather, such phrases are used to identify sets of data that may or may not be distinct. That is, the first data and the second data may represent different data, the same data, the same data, a part of some different data, and the like. The first data may be a subset of the second data, potentially an appropriate subset, or vice versa.

The phrases "data of the store" and "data in the store" are sometimes used herein, but limit the above data to data physically stored on the storage device. There is no intention to use it. Rather, the phrase is to limit the data to data logically on the storage device even if the data is not physically on the storage device. For example, storage abstraction (described below) does not mean that a significant number of zeros (or other data values) are actually stored on the underlying storage medium, but rather shortened data (e.g., values and Length) can be performed. Another example is provided below.

Exemplary operating environment

1 illustrates an example of a suitable computing system environment 100 in which aspects of the subject matter described herein may be implemented. The computing system environment 100 is only one example of a suitable computing environment and is not intended to suggest any limitation as to the scope of use or functionality of aspects of the subject matter described herein. Neither side should be interpreted that the computing environment 100 has dependencies or requirements on any one or combination of components illustrated in the exemplary operating environment 100.

Aspects of the subject matter described herein operate in numerous other general purpose or special purpose computing system environments or configurations. Examples of well-known computing systems, environments or configurations that may be suitable for use with aspects of the subject matter described herein include personal computers, server computers, handheld or laptop devices, multiprocessor systems, microcontroller based systems, set top boxes Programmable appliances, programmable consumer electronics, networked PCs, minicomputers, mainframe computers, personal digital assistants (PDAs), gaming devices, printers, set-tops, media centers or other appliances, in-vehicle or attached computing devices , Other mobile devices, distributed computing environments including any of the above systems or devices, and the like.

Aspects of the subject matter described in this specification may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., which perform particular tasks or implement particular abstract data types. Aspects of the subject matter described herein may also be practiced in distributed computing where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media including memory storage devices.

Referring to FIG. 1, an exemplary system for implementing aspects of the subject matter described herein includes a general purpose computing device in the form of a computer 110. The computer may include any electronic device capable of executing instructions. Components of the computer 110 may include a processing unit 120, a system memory 130, and a system bus 121, and the system bus 121 may process various system components, including system memory. ) The system bus 121 may be any one of various types of bus structures including a memory bus or a memory controller, a peripheral bus, and a local bus using any of a variety of bus architectures. As a non-limiting example, this architecture is a Peripheral PCI, also known as an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MCA) bus, an Enhanced ISA (EISA) bus, a Video Electronics Standards Association (VESA) local bus, and also a Mezzanine bus. Component Interconnect (PCI) bus, Peripheral Component Interconnect Extended (PCI-X) bus, Advanced Graphics Port (AGP) and PCI Express (PCIe).

Computer 110 typically includes a variety of computer readable media. Computer readable media can be any available media that can be accessed by computer 110 and includes both volatile and nonvolatile media, and removable and non-removable media. By way of example, and not limitation, computer readable media may comprise computer storage media and communication media.

Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Computer storage media may include RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or And any other medium that can be used to store desired information and can be accessed by computer 110.

Communication media typically embody computer readable instructions, data structures, program modules, or other data in a modulated data signal, such as carrier waves or other transmission mechanisms, and include any information delivery media. The term "modulated data signal" means a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. By way of example, and not limitation, communication media includes wired media such as a wired network or direct wired connection, and wireless media such as acoustic, RF, infrared, and other wireless media. Combinations of any of the above should also be included within the scope of computer readable media.

System memory 130 includes computer storage media in the form of volatile and / or nonvolatile memory, such as read only memory (ROM) 131 and random access memory (RAM) 132. Basic input / output system 133 (BIOS), which includes basic routines that help transfer information between elements within computer 110, such as during startup, is typically stored in ROM 131. The RAM 132 typically has direct access to the processing unit 120, or contains data and / or program modules currently operated by the processing unit 120. As a non-limiting example, FIG. 1 illustrates an operating system 134, an application program 135, another program module 136, and program data 137.

Computer 110 may also include other removable / non-removable, volatile / nonvolatile computer storage media. By way of example only, FIG. 1 illustrates a hard disk drive 141 that reads from or writes to non-removable, nonvolatile magnetic media, and a magnetic disk drive that reads from or writes to a removable, nonvolatile magnetic disk 152 (see FIG. 151 and an optical disk drive 155 that reads from or writes to a removable, nonvolatile optical disk 156, such as a CD ROM or other optical medium. Other removable / non-removable, volatile / nonvolatile computer storage media that can be used in the exemplary operating environment include magnetic tape cassettes, flash memory cards, digital general purpose disks, other optical disks, digital video tapes, solid state RAM. , Solid state ROM, and the like. The hard disk drive 141 may be connected to the system bus 121 through the interface 140, and the magnetic disk drive 151 and the optical disk drive 155 may interface to a removable nonvolatile memory such as the interface 150. It may be connected to the system bus 121 by.

The drive and associated computer storage media described above and illustrated in FIG. 1 store computer readable instructions, data structures, program modules, and other data for the computer 110. In FIG. 1, for example, hard disk drive 141 is illustrated as storing operating system 144, application program 145, other program module 146, and program data 147. Note that these components may be the same as or different from operating system 134, application program 135, other program modules 136, and program data 137. Operating system 144, application program 145, other program module 146, and program data 147 have at least a different counterpart and corresponding number in RAM 132 to illustrate that they are different copies. Is given.

A user may enter commands and information into the computer 110 through input devices such as a keyboard 162 and pointing device 161, commonly referred to as a mouse, trackball or touch pad. Other input devices (not shown) may include a microphone, joystick, game pad, satellite dish, scanner, touch-sensitive screen, recording tablet, and the like. These and other input devices are often connected to the processing unit 120 via a user input interface 160 coupled to the system bus, but by other interfaces and bus structures such as parallel ports, game ports or universal serial buses (USB). Can be connected.

The monitor 191 or other type of display device is also connected to the system bus 121 via an interface such as a video interface 190. In addition to the monitor, the computer may also include other peripheral output devices, such as speakers 197 and printer 196, which may be connected via an output peripheral interface 195.

Computer 110 may operate in a network environment using logical connections to one or more remote computers, such as remote computer 180. Remote computer 180 may be a personal computer, server, router, network PC, peer device, or other common network node, and only memory storage 181 is shown in FIG. 1, but is typically described above with respect to computer 110. Include many or all elements. The logical connection shown in FIG. 1 includes a local area network (LAN) 171 and a wide area network (WAN) 173, but may also include other networks. Such network environments are commonplace in offices, enterprise-wide computer networks, intranets and the Internet.

When used in a LAN network environment, computer 110 is connected to LAN 171 via a network interface or adapter 170. When used in a WAN network environment, computer 110 may include a modem 172 or other means for establishing communications over WAN 173, such as the Internet. Modem 172, which may be internal or external, may be coupled to system bus 121 via user input interface 160 or other suitable mechanism. In a networked environment, program modules depicted for the computer 110 or portions thereof may be stored in the remote memory storage device. As a non-limiting example, FIG. 1 illustrates a remote application program 185 residing in the memory device 181. It will be appreciated that the network connections shown are exemplary and other means of establishing a communications link between the computers can be used.

Off road  Read and write

As mentioned above, some traditional data transfer operations may not be efficient or may not even work in today's storage environment.

2-5 are block diagrams illustrating an exemplary arrangement of components of a system in which aspects of the subject matter described herein may operate. The components illustrated in FIGS. 2-5 are illustrative and are not meant to include all of the components that may be necessary or included. In other embodiments, components and / or functions described in connection with FIGS. 2-5 may be included in other components (not shown or not shown) without departing from the spirit or scope of aspects of the subject matter described herein. Or may be placed in subcomponents. In some embodiments, the components and / or functions described in connection with FIGS. 2-5 may be distributed across multiple devices.

Referring to FIG. 2, system 205 may include requestor 210, data access component 215, token manager 225, storage 220, and other components (not shown). System 205 may be implemented via one or more computing devices. Such devices may include, for example, personal computers, server computers, handheld or laptop devices, multiprocessor systems, microcontroller based systems, set top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, cell phones, Devices including personal digital assistants (PDAs), gaming devices, printers, set tops, media centers or other devices, in-vehicle or attached computing devices, other mobile devices, distributed computing environments including any of the above systems or devices, and the like. It includes.

If system 205 includes a single device, an example device that may be configured to act as system 205 includes computer 110 of FIG. 1. If system 205 includes multiple devices, one or more of the multiple devices may include similar or differently configured computer 110 of FIG. 1.

The data access component 215 transfers data to the storage device 220 and can be used to transfer data from the storage device 220. The data access component 215 is, for example, one of an I / O manager, a filter, a driver, a file server component, a component on a storage area network (SAN) or other storage device, and other components (not shown). It may contain the above. A SAN can be implemented with, for example, devices exposing logical storage targets, communication networks including such devices, and the like.

In one embodiment, the data access component is provided with an opportunity to examine the I / O between the requestor 210 and the storage device 220 and may change, complete or fail the I / O or perform something else based thereon. It may contain any component that may or may not work at all. For example, if system 205 resides on a single device, data access component 215 may include any object in the I / O stack between requestor 210 and storage 220. When system 205 is implemented with multiple devices, data access component 215 may be a component on a device hosting requestor 210, a component on a device providing access to storage device 220, and / or Or components on other devices. In other embodiments, data access component 215 may be used by components that I / O passes through (eg, services, databases, etc.) even when data does not travel through the components used. It may include components.

As used herein, the term component is intended to include all or part of a device, a collection of one or more software modules or portions thereof, one or more software modules or portions thereof and some combination of one or more devices or portions thereof, and the like. Should be read.

In one embodiment, storage device 220 is any storage medium capable of storing data. Such storage device 220 may include volatile memory (eg, cache) and nonvolatile memory (eg, persistent storage). The term data should be read broadly to include anything that may be represented by one or more computer storage elements. Logically, data can be represented by a series of ones and zeros of volatile or nonvolatile memory. In computers with non-binary storage media, data may be represented according to the capabilities of the storage media. The data may be composed of several types, such as simple data types such as numbers, characters, etc., data structures including hierarchies, links, or other related data types, data structures including many other data structures, or simple data types. Some examples of data include information, program code, program status, program data, instructions, other data, and the like.

Storage device 220 may include hard disk storage, solid state storage, or other non-volatile storage, volatile memory such as RAM, other storage, some combination thereof, and the like, and may include multiple devices (eg, , Multiple SANs, multiple file servers, combinations of heterogeneous devices, and the like. The devices used to implement the storage device 220 may be physically located together or geographically distributed (eg, in a single device, data center, etc.). The storage device 220 can be located in a hierarchical storage or a non-hierarchical storage device. Storage device 220 may be an external device, an internal device, or may include components both internal and external to one or more devices implementing system 205. Storage device 220 may be formatted (eg, into a file system) or unformatted (eg, native).

In another embodiment, storage device 220 may be implemented with a storage abstraction rather than direct physical storage. Storage abstractions may include, for example, files, volumes, disks, virtual disks, logical units, data streams, alternative data streams, metadata streams, and the like. For example, storage device 220 may be implemented by a server having multiple physical storage devices. In such an example, the server may provide an interface that allows data access components to access data on storage devices that are implemented using one or more of the server's physical storage devices or portions thereof.

This level of abstraction can be repeated at any arbitrary depth. For example, a server that provides a storage abstraction to the data access component 215 may also rely on the storage abstraction to access and store the data.

In another embodiment, storage device 220 may include a component that provides a view with data that may be persistent or nonpersistent to nonvolatile storage.

One or more of the data access components 215 may reside on the device hosting the requestor 210, while the other one or more of the data access components 215 may host the storage device 220 or the storage device 220. Can reside in a device that provides access to the network. For example, if the requestor 210 is an application running on a personal computer, one or more of the data access components 215 may reside in an operating system hosted on the personal computer. As another example, if the storage device 220 is implemented by a storage area network (SAN), one or more of the data access components 215 manages or provides access to the storage device 220. A storage operating system can be implemented. When requestor 210 and storage device 220 are hosted on a single device, all or many data access components 215 may also reside on the device.

In order to begin an offload reading (described below) of data in storage 220, requestor 210 may acquire a token representing the data using a predefined command (eg, via an API). Request can be sent. In response, one or more of the data access components 215 may respond to the requestor 210 by providing one or more tokens representing the data or a subset thereof.

For example, for various reasons, it may be desirable to return a token that represents less data than the original requested data. When a token is returned, it can be returned with the length of data the token represents or even multiple data ranges. The length may be less than the length of the originally requested data.

One or more of the data access components 215 may use less than the requested length associated with the token in either the offload read or the offload write. The length of data actually used is often referred to herein as the "effective length." Using a length less than the requested length may be desirable for various reasons. The effective length may be returned to the requestor or other data access component to recognize how many bytes were actually used by the command.

The data access component 215 can act in a variety of ways in response to offload reads or writes, including for example.

1. Partitioning The data access component can adjust the offset of such a request before sending an offload read or write request to the next lower data access component.

2. The RAID data access component can split offload read or write requests and send the pieces to the same or different data access components. In the case of RAID-0, the received request can split along a stripe boundary (which produces a short effective length), whereas in the case of RAID-1, the entire request is directed to more than one data access component. May be sent (as a result of which multiple tokens are generated for the same data).

3. The caching data access component may write the portion of the cache that contains the data that is to be obtained by the offload read request.

4. The caching data access component may invalidate the portion of the cache that contains the data that is about to be overwritten by the offload write request.

5. Data Verification The data access component may invalidate any cached checksums of data that are about to be overwritten by an offload write request.

6. The encrypted data access component may fail the offload read or write request.

7. Snapshot data access components can copy data where it is about to be overwritten by an offload write request. This can be done in part so that the user can 'go back' to the 'old version' of the file later if necessary. The snapshot data access component can use offload read and write commands to copy the data of the location it is trying to overwrite to the backup location. In this example, the snapshot data access component may be considered a "downstream requester" (described below).

The above examples are not intended to be all or all inclusive. Based on the subject matter herein, one skilled in the art can recognize other scenarios in which the subject matter can be applied without departing from the spirit or scope of the aspects of the subject matter described herein.

If the data access component 215 fails to offload read or write, an error code may be returned that causes another data access component or requester to try another mechanism to read or write data. Capability discovery may be performed, for example, during initialization. When a storage device or even lower layer data access component does not support a particular operation, other operations may be performed by the higher data access component or requestor to achieve the same result. For example, if the storage system (described below) does not support offload reading and writing, the data access component can manage tokens and maintain a view of the data, and thus the parent data access component. Does not recognize that the storage device or sub-data access component does not provide this capability.

The requestor may include an originating requestor or a downstream requestor. For example, the requestor may include an application that requests a token so that the application can perform offload writing. This type of requestor may be referred to as an originating requester. As another example, the requestor may include a server application (eg, a server message block (SMB) server) that receives a copy command from a client. The client may request to copy the data from the source storage device to the destination storage device through a copy command. The SMB server can use this offload read and write to receive this request and consequently perform the copy. In this case, the requestor may be referred to as a downstream requestor.

As used herein, unless otherwise specified or otherwise specified in the context, the term requestor should be understood to include both originating requesters and downstream requestors. The originating requester is the first requestor sending a request for offload reading or writing. In other words, the term requestor is intended to cover not only when the requester voluntarily sends offload reads or writes, but also when the requester has additional components other than the requestor responding to the start of the offload read.

For example, the originating requester may be an application that wants to transfer data from a source to a destination. This type of originating requester may send one or more offload read and write requests to data access component 215 to send data.

A downstream requestor is a requestor who makes one or more offload reads or writes to satisfy a request from another requestor. For example, one or more data access components 215 may serve as a downstream requestor and may initiate one or more offload reads or writes to satisfy a request made from another requestor. Some examples of downstream requestors have been described above with respect to RAID-0, partitioning, and snapshot data access components although these examples are not intended to be all or all inclusive.

In one embodiment, the token contains random or pseudorandom numbers that are difficult to guess. The difficulty of guessing numbers can be chosen by the size of the numbers as well as the mechanism used to generate them. The numbers represent data on storage device 220 but may be much smaller than the data. For example, the requestor can request a token of a 100 gigabyte file. In the response, the requestor may receive a token of 512 bytes or other size, for example.

As long as the token is valid, the token represents data. In some implementations, the token can represent data that was logically present when the token was bound to the data. The term logically is used when data is not all on the storage device or even cannot be persisted. For example, some of the data may be in a cache that needs to be flushed before the token can be provided. As another example, some of the data may be derived from other data. As another example, data from completely different sources may need to be manipulated to produce data that is combined or represented in tokens. The binding may occur after a request for the token is received and before or at the time the token is returned.

In other implementations, the data represented by the token can be changed. The operation of whether data can be changed during the validity period of the token can be negotiated with the requestor or can be negotiated between the components. This is explained in more detail below.

Tokens can be invalidated by expiration or invalidated before they expire explicitly. For example, if the file represented by the token is closed, the computer hosting the requestor 210 is turned off, the volume with the data represented by the token is detached, or the intended use of the token is completed, A message may be sent to explicitly invalidate the token.

In some implementations, messages that invalidate tokens can be treated as mandatory and follow. In other implementations, a message invalidating a token can be treated as a hint as to which to follow or not to follow. After the token is invalidated, it can no longer be used to access data.

Tokens may be protected by the same security mechanism that protects the data represented by the token. For example, if a user has permission to open and read a file, this may allow the user to obtain a token that allows the user to copy the file elsewhere. Once the channel for reading the file is reserved, the token can be passed over the reserved channel. If data can be provided to another entity, the token can be passed to another entity as if the data can be delivered to another entity. The receiving entity can use the token to obtain the data as if the receiving entity used that data itself is the data itself sent to the receiving entity.

Tokens may not be changed. That is, if a token is changed in any way, it can no longer be used to access the data representing the token.

In one embodiment, only one token representing data is provided. However, in other embodiments, multiple tokens may be provided, each representing a portion of data. In another embodiment, some or all of the data may be represented by multiple tokens. Such tokens may be encapsulated in other data structures or provided separately.

If encapsulated, a non-advanced requester may simply pass the data structure to the data access component when the requestor wishes to perform an action on the data (eg, offload write, token invalidation). The more advanced requestor 210 may relocate the token to an encapsulated data structure and may use separate tokens separately from other tokens to perform data operations, or take other actions when multiple tokens are passed back.

After receiving the token, requestor 210 may request that all or part of the data represented by the token be logically recorded. Sometimes this operation is referred to as offload writing here. Requestor 210 may perform this operation by sending a token to data access component 215 along with one or more offsets and lengths.

In the case of offload recording, for each relevant token, a token-related offset as well as a destination-relative offset may be indicated. Either or both offsets may be implicit or explicit. The token related offset may indicate the number of bytes (or other units), for example, from the beginning of the data represented by the token. The destination-related offset may indicate the number of bytes (or other units) from the start of data for the destination. The length can indicate the number of bytes (or other units) to copy the start at offset.

One or more of the data access components 215 may receive a token and verify that the token represents data on the storage device, and if so, the component has the ability of the storage system to host the underlying storage device 220. As a result, the portion of data represented by the token can be logically recorded. The storage system hosting basic storage device 220 includes one or more SANs, dedicated file servers, general servers or other computers, network appliances, any other device suitable for implementing the computer 110 of FIG. can do.

For example, if the storage device 220 is hosted through a storage system such as a SAN, and the requestor 210 requests an offload write to the SAN using a token representing the data present on the SAN, the SAN can You can take advantage of SAN's proprietary mechanism to log data logically without making other physical copies. For example, reference counting or other mechanism can be used to indicate the number of logical copies of the data. For example, the reference count may be used at the block level where the block may be logically duplicated in the SAN by incrementing the reference count of the block.

As another example, storage device 220 may be hosted through a storage system, such as a file server, which may have other mechanisms useful for performing offload writing such that offload writing does not involve physically copying data.

As another example, storage device 220 may be hosted through a “dumb” storage system that physically copies data from one location to another in the storage system in response to offload writing.

The above examples are not intended to be all or all inclusive. In fact, from the requester's point of view, it may be irrelevant to how the storage system implements data transfer corresponding to offload writing.

As described above, the data transfer operation of the storage system may be delayed. In some scenarios, no data transfer operation can occur at all. For example, the storage system may quickly respond that the offload write is complete, but may receive a command to trim the basic store before the storage system actually starts transmitting data. In this case, the data transfer operation in the storage system can be canceled.

The requestor 210 can share the token with one or more other entities. For example, the requestor may send the token to an application hosted on a device outside the device on which the requestor 210 is hosted. This application can then use the token to record data in the same way that requestor 210 may have. This scenario is illustrated in FIG. 5.

Referring to FIG. 5, using the data access component 215, the requestor 210 requests and obtains a token representing data from the storage device 220. The requestor 210 then passes this token to the requestor 510. The requestor 510 may then send the token through the data access component 515 to record the data.

One or more data access components 215 and 515 may be identical. For example, if requestors 210 and 510 are hosted on the same device, all data access components 215 and 515 may be the same for both requestors. If the requestor 210 and 510 are hosted on different devices, some components (e.g., components that implement the device hosting or providing access to storage 220) are the same. However, other components (eg components of different devices) may be different.

Returning to FIG. 2, in one embodiment, one or more of the data access components 215 may include or consult with a token manager (eg, such as token manager 225). Token managers can generate or obtain tokens representing data in storage 220, provide these tokens to authorized requesters, respond to requests to record data using tokens, and determine when to invalidate tokens. It can include one or more components. As described in more detail below, token managers may be distributed across multiple devices such that token managers that are logically identical acquire both tokens in offload reads and both are used to use tokens in offload writes. In this case, the distributed components of the token managers can communicate with each other to obtain information about the tokens as needed. In one embodiment, the token manager may generate a token, store the token in a token storage device that associates the token with data in storage device 220, and verify that the token received from the requester is found in the token storage device. have.

Token manager 225 may associate the token with data that identifies where the data may be found. Such data may also be distributed among multiple devices to obtain token information (what data the token represents, other data if the token has expired, other data, etc.) from the distributed components of the token manager 225. Can be used in the case. Token manager 225 may also partially associate the token with the length of the data to ensure that the requestor cannot obtain data past the end of the data associated with the token.

When the data of the storage device 220 is changed or deleted, the token manager 225 may take various actions depending on how the token manager 225 is configured. For example, when configured to preserve the data represented by the token, token manager 225 may ensure that a copy of the data that exists at the time the token was created is maintained. Some storage systems may have sophisticated mechanisms to maintain these copies even when data changes. In this case, token manager 225 may instruct the storage system (where storage device 220 may be part of) to maintain a copy of the original data for a while or until otherwise indicated.

In other cases, the storage system may not implement a mechanism for maintaining a copy of the original data. In this case, token manager 225 or other data access component 215 may keep a copy of the original data for a while or until otherwise indicated.

Note that maintaining a copy of the original data may include maintaining a logical copy rather than a duplicate copy of the original data. The logical copy contains data that can be used to create an exact copy. For example, the logical copy may include a changelog with the current state of the data. Conversely, by applying the change log to the current state, an original copy can be obtained. As another example, copy-on-write techniques may be used to maintain a logical copy that may be used to reconstruct original data. The above examples are not intended to limit the skilled person's understanding that there are many ways in which logical copies can be implemented without departing from the spirit or scope of the aspects of the subject matter described herein.

Token manager 225 may be configured to invalidate tokens when data changes. In this case, with respect to causing the data associated with the token to change, token manager 225 may indicate that the token is no longer valid. This can be done, for example, by deleting the token or marking the token as invalid in the token storage. When token manager 225 is implemented by a component of the storage system, one or more failure codes may be passed to one or more other data access components and passed to requestor 210.

Token manager 225 may manage the expiration of tokens. For example, a token can have a time to live (TTL). After the TTL expires, the token can be invalidated. In other embodiments, the token may remain valid according to various factors, including the following.

1. Storage constraints. Maintaining the original copy of the data can consume space above the threshold. In that regard, one or more tokens may be invalidated to free memory space.

2. Memory constraints. Memory consumed by maintaining multiple tokens may exceed a threshold. In that regard, one or more tokens may be invalidated to free memory space.

3. Number of tokens. The system may allow a set number of activated tokens. After the maximum number of tokens has been reached, the token manager can invalidate existing tokens before providing another token.

4. Input / output (IO) overhead. IO overhead with too many tokens may allow token managers to invalidate one or more tokens to reduce IO overhead.

5. IO cost / latency. Tokens may be invalidated based on the cost and / or latency of data transfer from source to destination. For example, if the cost exceeds a threshold, the token can be invalidated. Similarly, if the wait time exceeds the threshold, the token can be invalidated.

6. Priority. Some tokens may have priority over other tokens. If the token is invalidated, the low priority token may be invalidated. The priority of a token may be adjusted based on various policies (eg, usage, explicit or implicit knowledge of the token, request by the requestor, other policies, etc.).

7. Storage Provider Request. The storage provider (eg, SAN) may request to reduce the number of activated tokens. In response, the token manager may invalidate one or more tokens as appropriate.

Based on the token, the token may be invalidated at any time, even before or after one or more offload records are successful.

In one embodiment, the token only contains a value representing the data. In other embodiments, the token may also include or be associated with other data. Such other data may include, for example, data that may be used to determine storage devices, storage systems, or other entities from which data may be obtained, identification information of network storage systems, routing data and hints, information about access control mechanisms, Checksums about the data represented by the token, type of data (e.g., system, metadata, database, virtual hard drive, etc.), data access pattern (e.g., sequential, arbitrary), usage pattern (e.g., Often, sometimes, such as rare access, etc.), desired alignment of data, data to optimize the placement of data during offload writing (eg, in a hybrid environment with different types of storage), and so forth.

The above examples are not intended to include all or encompass all of the other data that may be included in or associated with a token. Indeed, based on the subject matter herein, those skilled in the art can recognize other data that can be conveyed in a token without departing from the spirit or scope of the aspects of the subject matter described herein.

The read / write request to the storage device may split a read request to a lower layer of the storage stack when it encounters a file fragment boundary, a RAID stripe boundary, a volume spanning boundary, and the like internally. Such splitting can occur because the source / destination differs over the split. Such splitting can be hidden by the splitter by not completing requests that need to be split until all generated split IO is complete.

This hiding of splitting within the splitting layer of the storage stack is convenient in that the layer of the storage stack does not need to know about splitting. In one embodiment, with the token-based approach described herein, splitting can be visual. In particular, if splitting occurs due to a different source / destination over the splitted portion, the offload provider (described below) may differ across the splitted portion. For example, if data is duplicated (or not duplicated), there may be multiple offload providers that provide access to the data. As another example, there may be multiple file servers destined for the SAN. In addition to a SAN, one or more servers or other data access components may be considered offload providers.

An offload provider is a logical entity (perhaps including a number of components distributed across multiple devices) that provides access to data associated with a storage source or destination. As used herein, access can include reading data, writing data, deleting data, updating data, a combination including two or more of these, and the like. Logically, the offload provider can perform an offload read or write. Physically, the offload provider may include one or more data access components 215 and may also include a token manager 225.

The offload provider may send data from the source storage device, write the data to the destination storage device, and maintain the data to be provided upon receipt of a token associated with the data. In some implementations, the offload provider can indicate that the offload write command is complete after the data has been logically written to the destination storage device. In addition, the offload provider may indicate that the offload write command is complete, but may postpone physically recording data associated with the offload write until convenient.

When data is split, the offload provider may provide access to a portion of the requested data, but may not provide access to other portions of the requested data. In this case, the separation token may be provided in the part before the split point and in the part after the split point. Other implementation dependent constraints at the storage stack's layer or offload provider may prevent the token from spanning the split range for other reasons. In such an embodiment, splitting may be visible to the requestor because the requestor may see the token returned in the read.

The following are two example approaches to dealing with splitting.

1. The read request may return more than one token, with each token associated with a different range of requested data. As noted above, these multiple tokens may be returned in a single data structure. When a requestor wants to record data, it can pass the entire data structure or, if done in an improved manner, pass one or more tokens of the data structure.

2. If a single token is returned, the token may indicate a short range of the original requested data. The requestor can then use the token to perform one or more offload writes within the length limit of the shortened range. If offload writing is requested, the length of the requested recording can also be truncated. For both read and write, the requestor can make a request for another range starting at an offset that was not processed by the previous request. In this way, the requestor can work through the entire required scope of the requestor.

The approach is illustrative only. Based on the subject matter herein, those skilled in the art can recognize other approaches for dealing with splitting that can be utilized without departing from the spirit or scope of the aspects of the subject matter described herein.

There can be multiple offload providers in the same stack. For the given range returned in the offload read request (the only range possible in case of range truncation), there may be multiple offload providers who are willing to provide tokens. In one embodiment, multiple such tokens may be returned in the request for the same data and used by the requestor in the offload record.

For example, the requestor may select one of the tokens for use in offload recording. By passing only one token to the offload provider, the requestor can determine the source offload provider used to obtain the data in this manner. In another example, the requestor may pass more than one token to the destination offload provider. The destination offload provider can then select one or more source offload providers associated with the token to obtain the data represented by the token.

In another example, multiple tokens may be returned to enable offloaded copies of bulk data, and offloaded copies of auxiliary data other than the bulk data. One example of auxiliary data is metadata about the data. For example, to ensure that the full offload copy is successful, the file system offload provider might include two tokens (eg, base data token and metadata token) to ensure that the offload write request is successfully used on the destination stack. It can be specified.

In contrast, multiple tokens used to support multiple large data offload providers in the stack may require that only one token be used on the destination stack in order for offload writes to succeed.

When multiple offload providers can transfer data from source to destination, the requestor can select one or more specific offload providers available. In one embodiment, this may include using a skip N instruction indicating that "skip N" skips the first N offload provider. In other embodiments, there may be other mechanisms used to identify a particular offload provider (eg, the offload provider's ID). In another embodiment, selecting one of the many tokens does not allow some offload providers to copy the data represented by the token, but selects an offload provider to copy the data when another offload provider can do so. Can be used.

In some embodiments, if more than one offload provider can copy the data represented by the token, the first, last, any, at least loaded, most efficient, low latency, or otherwise determined offload provider is automatically selected. Can be.

Tokens can represent data starting from any sector of a hard disk or other storage medium. The data represented by the token may be exactly a number of sectors, but in most cases it may not. If the token is used in a file operation on data that is beyond the end of length, the data returned may be null, zero, or some other indication of no data. Thus, if a requestor attempts to copy past the end of the data represented by the token, the requestor cannot physically obtain data past the end of the data through this mechanism.

Tokens can be used to offload large files. For example, the token can represent a null, zero, or other "no data" file. By using this token in offload writing, the token can be used to initialize a file or other data.

3 is a block diagram generally illustrating an exemplary arrangement of components of a system hosted by a device where a token manager hosts the storage device. As illustrated, the system 305 includes the requestor 210 and storage 220 of FIG. 2. The data access component 215 of FIG. 3 is the data access component 310 in the device 330 hosting the requestor 210 and the data access component in the device 335 hosting the storage device 220. Divided between 315. In other embodiments, there may be additional data access components that provide access to storage device 220 when storage device 220 is external to device 335.

Device 335 may be considered to be an offload provider when such device provides a token and the token contains the necessary components for recording the given data.

The token manager 320 may generate and verify a token as described above. For example, when the requestor 210 requests a token for data from the storage device 220, the token manager 320 may generate a token representing the data. This token can then be sent back to the requestor 210 via the data access components 310 and 315.

In connection with generating a token, token manager 320 may generate an entry in token storage 325. Such an entry may associate the token with data indicating where in the storage device 220 the data represented by the token may be found. The entry may also include other data used in managing the token, such as when invalidating a token, TTL for the token, other data, and the like.

When the requestor 210 or some other entity provides the token to the token manager 320, the token manager may perform a lookup on the token storage 325 to determine if the token exists. If the token exists and is valid, token manager 320 may provide location information to such component so that data access component 315 may logically record the data as requested.

If multiple physical devices provide access to storage device 220, token manager 320 and / or token storage device 325 may have components hosted by one or more physical devices. For example, token manager 320 can replicate token status across devices, have a centralized token component referenced by other token components, and provide token status from peer token managers as needed. Distributed systems, and the like.

Logically, token manager 320 manages tokens. Physically, token manager 320 may be hosted by a single device or may have components distributed across more than one device. Token manager 320 may be hosted on a device that is separate from any device that hosts storage device 220. For example, token manager 320 may exist as a service that data access component 315 may call to generate and verify a token and provide location information associated therewith.

In one embodiment, token storage device 325 may be stored in storage device 220. In other embodiments, token storage 325 may be separate from storage 220.

4 is a block diagram illustrating another exemplary arrangement of components of a system operating generally in accordance with aspects of the subject matter described herein. As illustrated, the device 405 hosts the data access component 310 and the virtualization layer 430 as well as the requestor 210. Data access component 310 is arranged in a stacked manner and includes N components including components 415, 420, 425 and other components (not shown). The number N is variable and can vary between devices.

The requestor 210 accesses one or more data access components 310 through an application programming interface (API) 410. The virtualization layer 430 indicates that the requestor or either data access component can be in a virtual environment.

A virtual environment is an environment that is simulated or emulated by a computer. The virtual environment can simulate or emulate a physical machine, an operating system, a set of one or more interfaces, portions thereof, combinations thereof, and the like. When a machine is simulated or emulated, it is sometimes called a virtual machine. A virtual machine is a machine that appears to be a physical machine for the software running on the virtual machine. The software can store files in virtual storage devices such as virtual hard drives, virtual floppy disks, and the like, can read files from virtual CDs, and communicate via virtual network adapters and the like.

Files on a virtual hard drive, floppy, CD or other virtual storage device may be supported by physical media, which may be local or remote to device 405. The virtualization layer 430 places the data on a physical medium and can provide the data to the virtual environment in a manner that does not recognize that one or more components that access the data access the data in the virtual environment.

More than one virtual environment can be hosted on a single computer. That is, two or more virtual environments can run on a single physical computer. For software running in each virtual environment, the virtual environment is a resource of its own (for example, hardware), even though a virtual environment hosted on a single computer can physically share one or more physical devices with each other and with the hosting operating system. Seems to have

Source storage 435 represents a storage device from which requestor 210 requests a token. Destination storage 440 represents a storage device that requests the requester to record data using a token. In an implementation, source storage 435 and destination storage 440 may be implemented as a single storage device (eg, a SAN with multiple volumes) or two or more storage devices. If source storage 435 does not support maintaining a copy of the original data, one or more components 415-425 may operate to maintain a copy of the original data during the lifetime of the token.

If the source storage device 435 and the destination storage device 440 are implemented as two separate storage devices, additional components (eg, storage server or other component) do not include the device 405 and store data. The source storage device 435 may transfer the data from the source storage device 435 to the destination storage device 440. However, in one embodiment, if the source storage device 435 and the destination storage device 440 are implemented as two separate storage devices, the one or more data access components 310 store data in the source storage device 435. It may serve to copy to the destination storage device 440. The requestor 210 may or may not recognize, or may not know, how to perform the basic copy.

There may be multiple paths between requestor 210 and source storage 435 and / or destination storage 440. In one embodiment, the token method described herein is independent of the path provided where information indicative of data (eg, available through the token manager) is available. In other words, if the requestor 210 has a path through the virtualization layer 430, a network path that does not pass through the virtualization layer 430, an SMB path, or some other path to the source or destination storage device, the requestor ( 210 may use one or more of these paths to perform offload writing to destination storage 440. In other words, the path to the source storage device and the path to the destination storage device may be the same or different.

In offloaded recording, the token is passed to the destination storage device 440 along with one or more offsets and lengths of the data to write. The data access component (not necessarily one of the data access component 310) receives the token, uses the token to obtain location information from the token manager, and stores the data in the source storage device 435. Logically recording to device 440 may begin.

One or more components 415-425 or other components (not shown) may implement a token manager.

The following are some example definitions of some data structures that may be used with aspects of the subject matter described herein.

#define FSCTL _{_} OFFLOAD _{_} READ

CTL_CODE (FILE_DEVICE_FILE_SYSTEM, 153, METHOD _{_} BUFFERED,

FILE _{_} READ _{_} ACCESS) // 153 is used to indicate an offload read.

typedef struct _{_} FSCTL _{_} OFFLOAD _{_} READ _{_} INPUT {

   ULONG Size;

   ULONG Flags;

   ULONG TokenTimeToLive; // (for example, milliseconds)

   ULONG Reserved;

   ULONGLONG FileOffset;

   ULONGLONG CopyLength;

} FSCTL _{_} OFFLOAD _{_} READ _{_} INPUT, * PFSCTL_OFFLOAD_READ _{_} INPUT;

typedef struct _{_} FSCTL _{_} OFFLOAD _{_} READ _{_} OUTPUT {

   ULONG Size;

   ULONG Flags;

   ULONGLONG TransferLength;

   UCHAR Token [512]; // can be greater than or less than 512

} FSCTL _{_} OFFLOAD _{_} READ _{_} OUTPUT,

* PFSCTL_OFFLOAD_READ_OUTPUT;

#define FSCTL _{_} OFFLOAD _{_} WRITE

CTL_CODE (FILE_DEVICE_FILE_SYSTEM, 154, METHOD _{_} BUFFERED,

FILE _{_} WRITE _{_} ACCESS) // 154 is used to indicate offload records

typedef struct _{_} FSCTL _{_} OFFLOAD _{_} WRITE _{_} INPUT {

   ULONG Size;

   ULONG Flags;

   ULONGLONG FileOffset;

   ULONGLONG CopyLength;

   ULONGLONG TransferOffset;

   UCHAR Token [512];

} FSCTL _{_} OFFLOAD _{_} WRITE _{_} INPUT, * PFSCTL_OFFLOAD_WPJTE _{_} INPUT;

typedef struct _{_} FSCTL _{_} OFFLOAD _{_} WRITE _{_} OUTPUT {

   ULONG Size;

   ULONG Flags;

   ULONGLONG Length Written;

} FSCTL _{_} OFFLOAD _{_} WRITE _{_} OUTPUT,

* PFSCTL_OFFLOAD_WRITE_OUTPUT;

//

// this flag, when OR'd into an action indicates that the given action is non-destructive

// If this flag is set then storage stack components which do not understand the action should forward the given request

//

#define DeviceDsmActionFlag _{_} NonDestructive 0x80000000

#define IsDsmActionNonDestructive ( _{_} Action) ((BOOLEAN) ((_ Action &

DeviceDsmActionFlag _{_} NonDestructive)! = 0))

typedef ULONG DEVICE _{_} DATA _{_} MANAGEMENT _{_} SET _{_} ACTION;

   #define DeviceDsmAction_OffloadRead (3 |

DeviceDsmActionFlag _{_} NonDestructive)

#define DeviceDsmAction _{_} OffloadWrite 4

//

// Flags that are global across all actions

//

typedef struct _{_} DEVICE DATA SET RANGE {

   LONGLONG Starting Offset; // for example, bytes

   ULONGLONG LengthlnBytes; // for example, multiple sector sizes

} DEVICE _{_} DATA _{_} SET _{_} RANGE, * PDEVICE_DATA_SET_RANGE;

Exemplary IOCTL data structures for implementing aspects of the subject matter described herein may be defined as follows.

//

// input structure for IOCTL _{_} STORAGE _{_} MANAGE _{_} DATA _{_} SET _{_} ATTRIBUTES

//One. ParameterBlockOffset or ParameterBlockLength has a value of 0, which indicates that a parameter block does not exist.

//

//2. The value of DataSetRangesOffset or DataSetRangesLength is 0, indicating that the DataSetRanges Block does not exist.

// If DataSetRanges Block exists, it contains

// Continuous DEVICE _{_} DATA _{_} SET _{_} RANGE structure.

// 3. The total size of the buffer is at least

// sizeof

(DEVICE_MANAGE_DATA_SET_ATTRIBUTES) + ParameterBlockLength +

     DataSetRangesLength

typedef struct _{_} DEVICE _{_} MANAGE _{_} DATA _{_} SET _{_} ATTRIBUTES {

   ULONG Size; // Size of structure

                       //

DEVICE _{_} MANAGE _{_} DATA _{_} SET _{_} ATTRIBUTES

DEVICE _{_} DATA _{_} MANAGEMENT _{_} SET _{_} ACTION Action;

   ULONG Flags; // global flags across all actions

   ULONG ParameterBlockOffset; // sorted by corresponding structure order

   ULONG ParameterBlockLength; // 0 means no parameter block exists.

ULONG DataSetRangesOffset; // Sort by DEVICE _{_} DATA _{_} SET _{_} RANGE structure alignment.

   ULONG DataSetRangesLength; // 0 means that the DataSetRanges Block does not exist.

} DEVICE _{_} MANAGE _{_} DATA _{_} SET _{_} ATTRIBUTES,

* PDEVICE_MANAGE_DATA_SET_ATTRIBUTES;

//

// Define Parameter Structure For Copy Offload Behavior

//

// The offload copy interface operates with phase 2: offload read and offload write.

//

// Input to OffloadRead operation is a set of extents in DSM structure

// The output parameter of OffloadRead is a token returned by the target that identifies a "point in time" snapshot of the extent taken by the target.

//

// The format of the token is unclear to the requestor and is clear to the target.

//

// Note: The token length of 512 is exemplary. The SCSI interface to OffloadCopy can be negotiable in size.

// New behavior can be created for tokens of variable size.

#define DSM _{_} OFFLOAD _{_} MAX _{_} TOKEN _{_} LENGTH 512

         // Keep as ULONG multiple

typedef struct DEVICE DSM OFFLOAD READ PARAMETERS {

   ULONG Flags;

   ULONG TimeToLive; // token TTL (eg, milliseconds) may be requested by the requestor.

} DEVICE _{_} DSM _{_} OFFLOAD _{_} READ _{_} PARAMETERS,

* PDEVICE_DSM_OFFLOAD_READ_PARAMETERS;

typedef struct _{_} DEVICE _{_} DSM _{_} OFFLOAD _{_} WRITE _{_} PARAMETERS {

   ULONG Flags;

   ULONG Reserved; // reserved for future use

  ULONGLONG TokenOffset; // start offset from the data represented by the token

   UCHAR Token [DSM_OFFLOAD_MAX_TOKEN_LENGTH]; // token

} DEVICE _{_} DSM _{_} OFFLOAD _{_} WRITE _{_} PARAMETERS,

* PDEVICE_DSM_OFFLOAD_WRITE_PARAMETERS;

typedef struct _{_} STORAGE _{_} OFFLOAD _{_} READ _{_} OUTPUT {

   ULONG OffloadReadFlags; // Outbound flags

   ULONG Reserved;

  ULONGLONG LengthProtected; // Length of data represented by token from lowest StartingOffset

   ULONG TokenLength; // Length of token in bytes.

   UCHAR Token [DSM_OFFLOAD_MAX_TOKEN_LENGTH];

         // consecutively generated tokens.

} STORAGE _{_} OFFLOAD _{_} READ _{_} OUTPUT,

* PSTORAGE_OFFLOAD_READ_OUTPUT;

//

// Define STORAGE OFFLOAD READ OUTPUT Flag

//

#define STORAGE _{_} OFFLOAD _{_} READ _{_} RANGE _{_} TRUNCATED (0x0001)

typedef struct _{_} STORAGE _{_} OFFLOAD _{_} WRITE _{_} OUTPUT {

   ULONG OffloadWriteFlags; // output flag

   ULONG Reserved; // reserved for future use

   ULONGLONG LengthCopied; // output parameter: the length of the content copied from the beginning of the data represented by the token

} STORAGE _{_} OFFLOAD _{_} WRITE _{_} OUTPUT,

* PSTORAGE_OFFLOAD_WRITE_OUTPUT;

//

// STORAGE OFFLOAD WRITE OUTPUT flag definitions-used in

OffloadWriteFlags mask

//

// recording is performed, but in truncated range

#define STORAGE_OFFLOAD _{_} WRITE _{_} RANGE _{_} TRUNCATED (0x0001)

//

// DSM output structure for bidirectional operation.

//

// The output parameter block is placed in the buffer created at the offset contained in the OutputBlockOffset field.

// offset is calculated from the beginning of the buffer,

// The callee sorts it according to the requirements of the action-specific structure template.

// example: In order for the OffloadRead operation to obtain a pointer to the output structure, the caller does the following:

//

// PSTORAGE _{_} OFFLOAD _{_} READ _{_} OUTPUT pReadOut =

// (PSTORAGE _{_} OFFLOAD _{_} READ _{_} OUTPUT) ((UCHAR *) pOutputBuffer +

//

((PDEVICE_MANAGE_DATA_SET_ATTRIBUTES_OUTPUT) pOutputBuffer)

//-> OutputBlockOffset)

//

typedef struct _{_} DEVICE _{_} MANAGE _{_} DATA _{_} SET _{_} ATTRIBUTES _{_} OUTPUT {

   ULONG Size; // size of the structure

DEVICE _{_} DATA _{_} MANAGEMENT _{_} SET _{_} ACTION Action;

          // action is requested and performed

   ULONG Flags; // common output flags for DSM behavior

  ULONG OperationStatus; // operation state used for offload operation

                              // (placeholder in various meanings like PENDING)

   ULONG ExtendedError; // extended error information

   ULONG TargetDetailedError; // Target specific error can be used for offload operation.

                            // (SCSI sense code)

   ULONG ReservedStatus; // reserved field

   ULONG OutputBlockOffset; // certain actions are sorted by corresponding structure alignment

   ULONG OutputBlockLength; // 0 means that there is no output parameter block.

} DEVICE _{_} MANAGE _{_} DATA _{_} SET _{_} ATTRIBUTES _{_} OUTPUT,

* PDEVICE _{_} MANAGE _{_} DATA _{_} SET _{_} ATTRIBUTES _{_} OUTPUT;

6-8 are flow diagrams illustrating exemplary operations that may generally occur in accordance with aspects of the subject matter described herein. For simplicity of explanation, the method described in connection with FIGS. 6-8 is depicted and described as a series of acts. It is to be understood that aspects of the subject matter described herein are not limited by the illustrated acts and / or order of acts. In one embodiment, the actions occur in the order described below. However, in other embodiments, the actions may occur in parallel in different orders and may occur with other actions not presented and described herein. Moreover, not all illustrated acts may be required to implement a methodology in accordance with aspects of the subject matter described herein. In addition, those skilled in the art will understand and appreciate that such methods may alternatively be represented by a series of correlation states or events via state diagrams.

Referring to FIG. 6, at block 605, operation begins. At block 610, a request for representation of data on a storage device is received. The request is conveyed with a description (eg, location and length) that identifies a portion of the storage device. Here, the word "portion" may be a storage device of all storage devices or less. For example, referring to FIG. 2, the requestor 210 may request a token for data from the storage device 220. Upon request, requestor 210 may send the location of the data (eg, file name, handle to open file, physical offset to file, volume or raw disk, etc.) along with the length.

At block 615, in response to the request, a token is received that logically represents data stored in a portion of the storage device when the token is bound to data. As mentioned above, the token may represent less data than requested. For example, referring to FIG. 2, one or more of the data access components 215 may return a token to the requestor 210 representing the requested data or a subset thereof. The token may be a size (eg, a predetermined number of bits or bytes) independent of the size of the data represented by the token. Such tokens may be received with other tokens in the data structure where each token of the data structure is associated with a different part of the data or more than one token is associated with the same part of the data.

Receiving a token may be accompanied by an indication that the data represents data in which the token is a subset of the requested data. Such an indication may take the form of the length of the data, for example represented by a token.

At block 620, a token is provided to perform offload writing. The token may be provided with information indicating whether to logically record all or part of the data via the offload provider. Such information may include, for example, a destination related offset, a token related offset, and a length. A token-related offset of zero, and a token-related offset of length equal to the full length of the data represented by the token, may indicate that all data is to be copied, but any offset with a length less than the full length of the data copies less than the full data. Can be shown.

For example, referring to FIG. 2, the requestor may pass the token to data access component 215, which may pass the token to token manager 225 to obtain the location of the indicated data. If token manager 225 is part of a storage system that provides access to storage device 220 (eg, in a SAN), the token identifies the data and tokens to logically record the data indicated by the request. May be provided to a data access component of a SAN that may utilize.

As mentioned above, the offload provider may be external to the device sending the request. In addition, when the offload provider receives the request, the offload provider can logically record the data regardless of any further interaction with any component of the device sending the request. For example, referring to FIG. 3, when a token to be written and a request arrive at the data access component 315, the component of the device 335 logically logs the data as requested without any further assistance from the device 330. Can record

At block 625, other operations may be performed, if any. At block 630, at any time after the token is generated, the requestor (or other data access component) may explicitly request to invalidate the token. In one embodiment, if such a request is sent in the middle of a copy operation, the copy may be allowed to complete. In other implementations, copying may be aborted, errors may occur, and other operations may occur.

Referring to FIG. 7, at block 705, operation begins. At block 710, a request for representation of data in storage is received. The request is conveyed with a description identifying the portion of the storage device where the data is located. The request may be received at a component of the storage area network or at another data access component. For example, referring to FIG. 3, one or more data access components 315 may receive a request for a token along with an offset, length, logical unit number, file handle, etc. that identifies data in storage 220. have.

At block 715 a token is generated. The generated token may represent data that is logically stored (eg, in storage device 220 of FIG. 3). As mentioned above, such data may or may not change during the validity period of the token, depending on the implementation. The token may represent a subset of the requested data as indicated previously. For example, referring to FIG. 3, the token manager 320 may generate a token to represent data requested by the requestor 210 in the storage device 220.

At block 720, the token is associated with the data represented through the data structure. For example, referring to FIG. 3, the token manager 320 may store an association in the token storage device 325 that associates the indicated data with the generated token.

In block 725 a token is provided to the requestor. For example, referring to FIG. 3, token manager or one data access component 315 may provide a token to data access component 310 for providing to requestor 210. The token may be returned in length indicating the size of the data represented by the token.

At block 730, other operations may be performed, if any. At block 735, note that at any time after the token has been generated, the token manager may invalidate the token according to various factors as described above. In one implementation, if the token is invalidated during a write operation that affects the data, the write may be allowed to complete. In other implementations, recording may be interrupted, errors may occur, or other operations may occur.

8 is a block diagram generally illustrating exemplary operations that may occur when offload recording is received at an offload provider in accordance with various aspects of the subject matter described herein. At block 805, operation begins.

At block 810, a token is received. The token may be received with data indicating whether to logically record all or some of the data represented by the token. For example, referring to FIG. 3, one data access component 315 may receive a token at one data access component 310 of FIG. 3.

At block 815, a determination is made as to whether the token is valid. For example, referring to FIG. 3, the token manager 320 may query the token storage device 325 to determine whether the received token is valid. If the token is valid, operation continues at block 820; otherwise, the request may fail, and operation continues at block 817.

In block 817 the request fails. For example, referring to FIG. 3, data access component 315 may indicate that the copy has failed.

At block 820, requested data of the offload copy is identified. For example, referring to FIG. 3, token manager 320 may query token storage 325 to obtain a location or other identifier of data associated with the token. The token may include or be associated with data representing a device hosting the data represented by the token.

In block 825, a logical recording of the data represented by the token is performed. For example, referring to FIG. 3, the device 335 may logically record the data represented by the token.

At block 830, other operations may be performed, if any.

As can be seen from the above detailed description, aspects relating to offload reading and writing have been described. While aspects of the subject matter described herein allow for various modifications and alternative configurations, certain illustrative embodiments thereof are shown in the drawings and described in detail above. However, there is no intention to limit aspects of the claimed subject matter to the specific forms disclosed, while the intent is to limit all modifications, alternative configurations, and equivalents within the spirit and scope of the various aspects of the subject matter described herein. It should be understood that it is encompassing.

Claims (15)

CLAIMS What is claimed is: 1. A method implemented at least in part by a computer,
Sending a request for a representation of the first data of the storage device, the request carried with a description identifying a portion of the storage device,
In response to said request, receiving a token representing second data logically stored in said portion of said storage device, said second data being a subset of said first data, ie a potentially appropriate subset, and
Providing the token with information instructing to logically record the third data via an offload provider operable to use the token to retrieve at least third data, the third data being a portion of the second data. Subset, that is, a potentially appropriate subset
Way.
The method of claim 1,
Sending a request comprising a description of the portion of the storage device transmitting an offset and a length, wherein the offset indicates the position of the first data in the storage device and the length indicates the size of the first data. Containing
Way.
The method of claim 1,
Receiving the token comprises receiving a number available to obtain the second data with the second data present when the token is bound to the second data, the number identifying the second data. Available by the offload provider and wherein the number is generated by a random number or pseudo random mechanism
Way.
The method of claim 1,
Receiving the token comprises receiving the token along with another token in a data structure, wherein each token in the data structure has a different portion of the second data present when the token is bound to the other portion. Available to obtain as part
Way.
The method of claim 1,
Receiving one or more other tokens, each also representing the second data, and providing one or more of the other tokens in conjunction with providing the token;
Way.
A computer storage medium with computer executable instructions,
The computer executable instructions, when executed
Receiving a request from a requestor for a representation of first data logically stored on a storage device, the request being conveyed with a description identifying the portion of the storage device where the first data is located;
Generating a token representing second data logically stored in the portion of the storage device, the second data being a subset of the first data, ie a potentially appropriate subset;
Associating the token with the second data via a data structure, the token being available to obtain the second data as the second data present when the token is bound to the second data, and
Performing an operation including providing the token to the requestor
Computer storage media.
The method according to claim 6,
The computer executable instructions are
Receiving the token with third data indicating whether to record all or part of the second data,
Determining whether the token is valid,
If the token is invalid, further performing the operation of failing the request.
Computer storage media.
The method according to claim 6,
Receiving a request for a representation of first data logically stored on a storage device comprises receiving the request at a data access component of a storage area network device,
Generating a token representing the second data comprises generating a value by a data access component of the storage area network device,
Associating the token with the second data via a data structure includes placing an entry in a table, wherein the entry is before or after the token and the request are received at the data access component, and the Including an identifier of the second data as the second data present before or when a token is returned to the requestor
Computer storage media.
The method according to claim 6,
The computer executable instructions are
Receive a request to change the first data, and further perform an operation to invalidate the token in response.
Computer storage media.
The method according to claim 6,
The computer executable instructions are
Further invalidating the token based on one or more of memory constraints, write activity, disk constraints, network bandwidth constraints, latency constraints, and time to live (TTL)
Computer storage media.
The method according to claim 6,
The computer executable instructions are
Receiving a request to change the first data, making the change in response, and further performing maintaining a logical copy of the second data present when the token is bound to the second data;
Computer storage media.
In a computing environment,
Operable to send a request for a representation of the first data of the storage device, and further operable to receive a token representing the subset of the first data, that is, the second data, which is a potentially appropriate subset; A requestor further operable to provide the token with third data instructing to logically record all or part of the data,
A token manager operable to generate the token and associate the token with the second data token via a data structure, and
Operable to receive the token with the third data, further operable to contact the token manager to determine if the token is valid, the second data being logically unaltered while at least the token is valid Including offload providers maintained
system.
13. The method of claim 12,
The offload provider is further operable to logically record all or part of the second data as indicated by the third data if the token is valid, the third data also being a destination for entering the recorded data. Containing
system.
13. The method of claim 12,
The requestor includes components of a device external to the device hosting the offload provider.
system.
13. The method of claim 12,
The token manager and the offload provider are both hosted on a device in a storage area network.
system.

Images