EP3111623A1

EP3111623A1 - System, method, and computer program product for providing a unified namespace

Info

Publication number: EP3111623A1
Application number: EP15751600.6A
Authority: EP
Inventors: Ajay Bakre; Dhruva KRISHNAMURTHY; Kartheek Muthyala; Chhavi Sharma; Rukma Talwadker
Original assignee: NetApp Inc
Current assignee: NetApp Inc
Priority date: 2014-02-24
Filing date: 2015-02-23
Publication date: 2017-01-04
Also published as: EP3111623A4

Abstract

A system and computer-based method for performing a data transaction in a network storage system is provided. In an embodiment, the method includes receiving, from a client, a first data transaction request including a first data identifier associated with a first namespace protocol; determining a storage system associated with the first data identifier and associated with a second namespace protocol; determining a second data identifier associated with the first data identifier, wherein the second data identifier conforms to the second namespace protocol; and forwarding a second data transaction request including the second data identifier to the storage system based on determining an association between the second data identifier and the first data identifier, wherein the first namespace protocol and the second namespace protocol are different.

Description

SYSTEM, METHOD, AND COMPUTER PROGRAM PRODUCT

FOR PROVIDING A UNIFIED NAMESPACE

CROSS-REFERENCE TO RELATED APPLICATK)NS(S)

[Θ001] This application claims priority to Indian Patent Application No, 1463/CHE/20I filed February 24, 2014, which is hereby incorporated by reference in its entirety.

[Θ002] This application claims priority to U.S. Patent Application No. 14/318,460 filed June 27, 2014, which is hereby incorporated by reference in its entirety. TECHNICAL FIELD

[0003J The present description relates generally to data storage and, more specifically, to providing a single unified namespace across multiple storage systems, each having its own independent namespace. BACKGROUND

[Θ004] Network-based storage and cloud storage has become a common way for entities to save their data. However, as such storage options have become more common, they have also become more complex. In a simplified example, a single client accesses data from a single data store of a networked storage system by sending data transactions (e.g., read and/or write requests) to the storage system, which executes the requested transactions on a connected data store. The client refers to the desired data using a data identifier, which in various examples includes a file name, a path name, and/or a virtual memory address. The storage system, and typically a storage controller thereof, translates the data identifier into a unique location for the data within the data store. In this way, the storage system virtualizes the organization of data on the data store and hides the complexity of the underlying data storage techniques from the client. The identifier used by the client in the data transaction does not necessarily contain the location or address within the data store; rather, the storage system translates the identifier into a corresponding address. The storage controller then performs the requested operation on the data at the address in the data store and sends a response to the client.

[Θ005] The data identifier used by the client conforms to a first namespace, which in this example, includes a protocol for identifying data objects. Different storage systems may use different (heterogeneous) namespaces, and the addition of further storage systems to the example above introduces complexity to the system due to the additional namespace to data location transformations. In that regard, in some conventional systems, a client in

communication with two or more storage systems may have to be cognizant of the heterogeneous namespaces and may have to conform its data transactions to the namespace associated with whichever storage system the transaction is sent to.

[0006] Furthermore, conventional namespace techniques encounter problems of scale when the number of storage systems and namespaces grows. For example, conventional file-based storage systems do not scale smoothly because each file path must resolve to a single storage system. In other words, each directory of files must reside on one and only one system. [0007] In order to relieve the burden of managing multiple namespaces from the clients, it would be desirable to have a storage system in which a client can access multiple storage systems without being exposed to the complexity of multiple disparate namespaces. That is to say, creating a federation of heterogeneous namespaces would allow a client to use a single identifier format to specify data regardless of the actual namespace in which the data resides. It would be additionally beneficial if the techniques for providing translation of the identifiers scaled seamlessly as data set size grows.

BRIEF DESCRIPTION OF THE DRAWINGS

[0008] The present disclosure is best understood from the following detailed description when read with the accompanying figures.

[0009] Fig. I is an illustration of an example storage architecture, adapted according to one embodiment.

[0010] Figs. 2A and 2B are illustrations of exemplary Key- Value ( V) databases according to various embodiments. [0011] Fig. 3 is an illustration of a further exemplary KV database according to one embodiment.

[0012] Fig. 4 is an illustration of an example storage architecture, adapted according to one embodiment, and using a distributed database for KV pairs.

[0013] Fig. 5 is an illustration of an example storage architecture according to one embodiment. [0014] Fig. 6 is an illustration of an example storage architecture integrating a unified namespace gateway module into a storage system according to one embodiment.

[Θ015] Fig. 7 is an illustration of an example process adapted according to one

embodiment. [0016J Fig. 8 is an illustration of an example process adapted according to one

embodiment.

DETAILED DESCRIPTION

[Θ017] All examples and illustrative references are non- limiting and should not be used to limit the claims to specific implementations and embodiments described herein and their equivalents. For simplicity, reference numbers may be repeated between various examples. This repetition is for clarity only and does not dictate a relationship between the respective embodiments. Finally, in view of this disclosure, particular features described in relation to one aspect or embodiment may be applied to other disclosed aspects or embodiments of the disclosure, even though not specifically shown in the drawings or described in the text.

[ΘΘ18] Various embodiments include systems, methods, and computer program products that join any number of storage systems including storage systems that have incompatible namespaces for specifying data. The embodiments provide a single unified namespace by which clients can refer to data located on different storage systems. Many embodiments include a gateway system that translates data transactions directed to the unified namespace into a protocol and namespace associated with a particular storage system and that forwards the translated transaction on to the respective system, in this way, the gateway system simplifies the task of specifying data located on a group of heterogeneous storage systems.

[0019] In an example, one or more clients are in communication with multiple, different storage systems, w here some of the storage systems use different namespaces. Storage system refers generally to any suitable data storage system. In one example, a storage system includes a network storage system with multiple storage drives, one or more storage controllers, and a storage server that communicates with the storage controllers. Thus, the storage system can be a network storage system that, by itself, could provide storage to multiple clients. The clients communicate with the storage system by sending and receiving data transactions to the storage system and in particular, to the storage controller of the storage system. In order to reference a particular data object, a transaction include a data identifier that uniquely identities a data object and that conforms to a namespace of the storage system.

[ΘΘ20] However, namespaces may differ between storage systems as noted above, and even where namespaces are the same, underlying storage systems may be incompatible due to complications such as incompatible software configurations or software release versions. Accordingly, in an example, a data storage environment including a number of storage systems further includes a gateway that is disposed communicatively between the client and the storage systems. In some embodiments, a gateway is a separate computing system distinct from the client and the storage systems. In some embodiments, a gateway is a module of a storage system. In yet further embodiments, the storage environment includes some storage systems that include gateway modules and some storage systems that interface with discrete gateway systems.

[0021] The gateway receives transactions that refer to data using a first namespace and translates the transaction to refer to data using a second namespace particular to the storage system. In this way, the gateway hides the underlying storage architecture from the client. Thus, in these examples, the client is unaware of the multitude of different storage systems. When the client sends a read or write request, the gateway translates the data identifier in the request to a namespace appropriate for whichever storage system will handle the request. For example, one storage system may use an NFS protocol, while a second uses an CIFS/SMB protocol and a third uses HTTP.

[0022] A particular technique for facilitating the namespace translation at the gateway includes the use of a database of Key-V alue (KV) pairs. At its most basic, querying a database using a key returns a corresponding value. In an example of a database used for namespace translation (i.e., namespace resolution), the keys includes the data identifier used by the client in its request. The values associated with the keys each include an identification of the appropriate storage system (e.g., in the case of a read request, the storage system that stores the data), an identifier of the namespace of the storage system, and another data identifier that is understood by the storage system to refer to the data identified by the key. This latter data identifier conforms to the namespace of the storage system. Thus, upon receipt of a data transaction from a client, the gateway uses the data identifier in the transaction to traverse the KV pairs, finding the corresponding key. The gateway accesses the data in the value to determine which storage system owns the data and what data identifier to use. The gateway then sends a modified transaction to the identified storage system using the translated data identifier from the value.

[ΘΘ23] To provide multiple points of access, gateways may be clustered and a database of namespace translations may be distributed over the gateways with each gateway storing at least a portion of the database. The gateways may act as peers so that when any gateway receives a transaction, it is operable to query its peer nodes to determine an appropriate translation if the relevant portion of the database is not stored locally. In this way, the systems and method s of these embodiments can join heterogeneous pools of storage across networks and sites in a manner that is transparent to the client systems. Many embodiments also provide transparent redundancy. For example, transactions can be handled by other gateways in the cluster should a particular gateway fail, and redundant storage systems can be brought on line to replace a failing device without interruption.

[0024] Fig. 1 is an illustration of exemplary storage architecture 100, adapted according to one embodiment. The storage architecture 100 includes clients 1 10-112, gateways 120 and 121 , and storage systems 130-133, Clients 1 10-1 12 in this example include clients that use the storage systems 130- 133 for storage. For instance, each of clients 1 10- 1 12 may run one or more applications (e.g., word processing or database programs) that utilize the storage architecture. Each client 1 10-1 12 includes a computer system that interacts with gateways 120 and 121 for submitting read/write access requests and for receiving or transmitting data from storage systems 130-133 via gateways 120 and 121.

[0025] Gateways 120 and 12.1 each include a computer system that executes applications and interacts with the clients 110-1 12 for receiving read/write access requests and receiving or transmitting data from or to the clients 1 10-1 12. Gateways 120 and 121 in this example may be connected to the clients 1 10-1 12 over a network such as a local area network (LAN), an Ethernet subnet, a PCI or PCIe subnet, a switched PCIe subnet, a wide area network (WAN ), a metropolitan area network (MA N), the Internet, or the like. The gateway functionality described herein may be included in applications that are run on the computer systems that constitute gateways 120 and 12.1 . Each gateway 120 and 12.1 may include any appropriate computer hardware and software, such as, e.g., a general-purpose computer configured to execute any of a vari ety of operating systems, including the Unix™, Linux™, and Microsoft Windows™ operating systems. [0026] The storage systems 130-133 in this example each include a storage controller (e.g. , storage controllers 140-142) that exercises low-level control over a set of networked storage devices referred to col lectively as a data store (e.g. , data stores 150-153) in order to read and write data to the storage devices of the data store. Examples of storage hardware that can be used in each data store 150-153 include hard disk drives (HDDs), solid state drives (SSDs), optical drives, and/or any other suitable volatile or non-volatile data storage medium, though the scope of embodiments is not limited to any particular storage hardware. The storage controllers 140-142 each have an associated namespace and process transactions identifying data in a respective namespace protocol. To do so, the storage controllers 140-143 may translate addresses within the associated namespace to a data location on a storage device of a data store.

[Θ027] As shown in Fig. 1 , each of the clients 1 10-1 12 is in communication with at least one of the gateways 120 and 121 . For instance, client 1 10 is in communication with gateway 120; client 1 1 1 is in communication with gateways 120 and 121 ; and client 1 12 is in communication with gateway 121. In turn, each of the gateways 120-121 is in

communication with each of the storage systems 130-133. Thus, gateway 120 is in communication with all of storage systems 130-133, as is gateway 121 , and each of the clients 1 10-1 12 can reach any of the storage systems 1 30-133 via one of the gateways 120 or 121. Gateways 120 and 121 may also be in communication with each other, e.g., over a LAN or other type of network.

[0028] Each of the clients 1 10-1 12 can send a request to create a file, read a fi le, write a file, change a directory, look up a file, or any other suitable data transaction to a gateway with which it is in communication. The gateways 120 and 121 translates an identifier of the file or directory into a namespace of an underlying storage device and thereby provides a unified namespace technique so that complexity of th e storage architecture is hidden from clients 110-1 12, as described in detail below. The following example use case illustrates a read request from client 1 1 1 to gateway 120, and it is understood that similar action may be performed by any of the cl ients 1 10- 1 12 with any of the gateways 120 and 121 , assuming a network connection therebetween. [0029] Continuing with the example, client 1 1 1 issues a read request to gateway 120.

Client 1 1 1 is in communication with both of gateways 120 and 121 and may issue the read request to gateway 121 rather than to gateway 120 for any appropriate reason, e.g., load balancing. [0030] The read request from client 111 includes a first data identifier in the unified namespace. The first data identifier in this example uniquely identifies the object (e.g., an object, file, or block), although it does not comply with a namespace used by any of the storage systems 130-133 and would not properly identify data if sent directly to any of the storage systems 130-133.

[0031 ] Gateway 120 receives the read request and determines a particular storage system 130- 133 to which to send the read request. Gateway 120 also determines a second data identifier to use when sending the read request to the particular storage system. Various embodiments may use any appropriate technique to resolve a particular storage system or data identifier upon receipt of a request from a client. In the present example, gateways 120 and 121 use a database of V pairs to resolve storage systems and data identifiers. The gateway 120 creates a second read request that references data using the second data identifier and forwards the second request to the identified storage system of systems 130- 133. [Θ032] Moving to Fig. 2A, an illustration of an exemplary KV database 200 according to one embodiment is provided. On the left side, the keys include the unified namespace identifiers that are provided from and to clients 1 10-112. In various embodiments, the unified namespace identifiers include block-level identifiers, file-level identifiers, and/or object identifiers. Block-level data identifiers refer to data using an address that corresponds to a sector of a storage device and may include a physical address (i.e., an address that directly map to a storage device) and/or a logical address (i.e., an address that is translated into a physical address of a storage device). Exemplary block-level protocols include iSCSI, Fibre Channel, and Fibre Channel over Ethernet (FCoE). In contrast, file-level protocols specify data locations by a file name. A file name is an identifier within a file system that can be used to uniquely identify corresponding memory addresses. File-level protocols rely on a system such as one of the unified namespace gateways 120 and 121 and/or a storage system to translate the file name into respective storage device addresses. Exemplar}' file-level protocols include CIFS/SMB, SAMBA, and NFS. Object-level protocols are similar to file- level protocols in that data is specified via an object identifier that is eventually translated by another system into a storage device address. However, objects are more flexible groupings of data and may specify a cluster of data within a file or spread across multiple files. Object- level protocols include CDML HTTP, and S3.

/ [0033] In the embodiments of Fig. 2A, the unified namespace identifiers are file-level identifiers. Commonly, a file-level namespace identifier includes a filename suffix and a path prefix that indicates a directory structure in which the data corresponding to the filename is said to reside. In a conventional system, the path determines the storage system on which the data resides, with each directory being assigned to a particular system. This does not allow data residing within a given directory to be split across more than one storage system. However, by using the V database 200 discussed below, many embodiments of the present disclosure allow each individual file to reside on any storage system (e.g., systems 130-133) regardless of the path prefix. [0034] Upon receiving a unified namespace identifier, a computing system such as one of gateways 120 and 121 can query the KV database 200 using the identifier. On the right side of the illustrated KV database 200, the associated values include an identifier of a storage system, an identifier of a namespace, and an identifier corresponding to the object in the namespace of the particular storage system. In this way, a query using the unified namespace identifier will return a value that specifies which storage system the data resides on, a namespace used by the storage system, and/or an identifier specifying the data at the storage system. As can be seen, the database 200 may be divided into database portions 202, which in some embodiments are distributed among multiple computing systems such as gateways 120 and 121 , As discussed in further detail below, a distributed database 200 has many advantages such as redundancy and load balancing that become particularly important as database size increases.

[Θ035] Continuing with the example in which the client 1 1 1 sends a read request to gateway 120, the exemplary read request issued from client 1 1 1 includes the data identifier "/project/alpha/design ___spec.txt" which is a file-level unified namespace data identifier. The gateway 120 traverses the KV database to find a corresponding identifier particular to a storage system and within the namespace of the storage system. In this example, the gateway 120 traverses the KV database using a key that includes the data identifier

('Vproject/alpha/design jspec.txt"), although in other examples, the data value may be tokenized such that the key corresponds to the data identifier without necessarily including the data identifier. Returning to this example, the gateway 120 reads the value corresponding to the key, which in this example includes "Storage System C, Namespace B, #09548a22". Therefore, gateway 120 has information indicating that the read request should be sent to Storage System C using Namespace B and data should be identified to Storage System C using the block-level identifier #09548a22. It is noted that the identifier used by the storage system need not be the same or even the same type (e.g., file-level/block-level) as the unified namespace data identifier.

[0036] Assuming in this example that "Storage System C" corresponds to storage system 132 of Fig. 1, gateway 120 then forwards the read request to storage system 132 indicating that the read request is for object #09548a22. Storage controller 142 of storage system 132 then maps the data identifier #09548a22 to one or more storage locations within its physical stores drives (using its own internal mapping tables), reads the corresponding data, and returns the results to gateway 120. Gateway 120 forwards the results on to requesting client 111 using the unified namespace protocol.

[0037] Referring now to Fig. 2B, an illustration of a further exemplar}' KV database 250 according to one embodiment is provided. On the left side, the keys include the unified namespace identifiers that are provided from and to clients 110-112. In contrast to the embodiments of Fig. 2A, the unified namespace identifiers include block-type data identifiers that correspond to addresses where the data is stored. On the right side, the associated values include an identifier of a storage system, an identifier of a namespace, and an identifier corresponding to the object in the namespace of the particular storage system. In this way, a query using the block-level unified namespace identifier will return a value that specifies which storage system the data resides on, what namespace is used by the storage system, and what identifier specifies the data at the storage system . Similar to the examples of Fig. 2A, the database 250 may be divided into database portions 202, which in some embodiments are distributed among multiple computing systems such as gateways 120 and 121.

[0038] While the exemplary databases of Figs. 2A and 2B contain keys of a single identifier type (file-level and block-level, respectively), further exemplary databases include keys of any number of identifier types. Thus, the unified namespace may support block-level identifiers, file-level identifiers, and/or object identifiers, concurrently. This provides "file object duality", whereby different clients can access the same element of data using different protocols.

[0039] Fig. 3 is an illustration of one such database 300 according to various embodiments. The database 300 includes records comprising various fields. The illustrated examples include a IJUID 302 (a randomly generated unique value assigned to the record), a name 304 particular to the object or file, a prefix 306 corresponding to a path of a file or a prefix of an object, a file handle 308 uniquely identifying the file for NFS compatibility, a file handle 310 uniquely identifying a parent of the file for NFS compatibility, a creation time 312, an access time 314, a size 316, a type 318 such as file or directory, and a location 320. Similar to the values of databases 200 and 250, the location 320 may specify a storage system the data resides on, a namespace used by the storage system, and/or an identifier specifying the data at the storage system.

[0040] Expanding on the KV concept, multiple fields of the database 300 may correspond to searchable keys. For example, to facilitate searches, the database 300 may be queried using the name field 304. To facilitate director listings, the database 300 may also be queried using the prefix field 306. To facilitate NFS file handling, the database 300 may also be queried using the file handle 308 and/or parent file handle 310 fields. The available fields and the fields indexed for querying have an impact on the database performance, and, in many embodiments, the database 300 is configured to optimize those transactions expected to be most common. For example, in the illustrated database 300, moving and renaming a file is performed by updating the file name 304 field of a single record. However, moving or renaming a directory is performed by updating at least the prefix 306 field of all contained files and subdirectories. If this proves unduly burdensome, the prefix 306 field may be omitted in favor of the parent file handle 310, This may optimize director}' moves and renaming, but possibly at the expense of other transactions such as a directory listing transaction. According!)', it can be seen that the database 300 provides sufficient flexibility to optimize gateway performance for nearly any given environment.

[0041] The examples above refer to a read request, although the scope of embodiments includes other data transactions, such as write requests, as well. In one example, the gateway 120, upon receiving a write request creates a new KV pair in its database to indicate the unified namespace identifier, storage system identifier, namespace identifier, and data identifier for use with the storage system. The gateways 120 and 121 may keep their KV tables up to date by modifying them in response to newly-written data and modified data, thereby ensuring that subsequent data transactions access the correct data.

[ΘΘ42] Furthermore, the examples above illustrate a case in which gateway 120 includes a comprehensive database to map the KV pairs. Other embodiments use a distributed database to store the KV pairs, and in such embodiments, a given gateway may not have direct, local access to a requested data identifier. Fig. 4 is an illustration of a storage architecture 400, adapted according to one embodiment and using a distributed database for KV pairs. [0043] In Fig. 4, gateway 120 includes namespace database portion 402, and gateway 121 includes namespace database portion 404. Therefore, in a scenario in which a client sends a read request to gateway 120 for a particular object, gateway 120 may or may not have local access to the part of the namespace database including that object's identifiers. The same is true for gateway 121. The storage architecture 400 may provide one or more techniques to provide comprehensive access to the clients 110-1 12 so that each client can access any storage system from any gateway, despite individual gateway 120 and 121 having less than a full database stored locally.

[0044J Gateways 120 and 121 may use any appropriate di stributed database technology in the example of Fig. 4. One example database technology includes the open source NoSQL distributed database under the name CASSANDRA, available from Apache Software Foundation, Another example database technology includes the open source NoSQL distributed database under the name R.1A , available from Basho Technologies. Of course, the scope of embodiments may include any appropriate distributed database technology, whether open source or otherwise.

[0045] Gateways 120 and 121 are in communication with each other via a network, such as a LAN or other suitable connection. In fact, gateways 120 and 121 may be in the same facility or geographic area or may be in different facilities or different geographic areas. For geographically-distributed systems, gateways 120 and 121 may communicate with each other via the Internet. As discussed further below, gateways 120 and 121 communicate with each other to provide access from any client 110-1 12 to any storage system 130-133.

[0046] In one example, client 1 1 1 sends a read request to gateway 120 using the unified namespace data identifier 'Yproject aipha/design spec.txt" from the previous example.

Gateway 120 queries its database portion 402 using the key corresponding to the identifier. In a scenario in which the gateway 120 finds the data identifier

"/project/alpha/desi gn_spec.txt" within database portion 402, gateway 120 forwards the read request to Storage System C (storage system 132 in the previous example) using data identifier #09548a22. This is consistent with example database 200 of Fig, 2.

[0047J In a scenario in which gateway 120 does not find data identifier

'VpiOject/alpha/design spec.txt", it sends a message to other gateways to inquire whether those other gateways can resolve the unified namespace data identifier. In the present example, the other gateway is gateway 121, and gateway 120 sends a message to gateway 121 requesting that the unified namespace identifier "/project/alpha/design, spec.txt" be resolved. In some examples, the key or a hash thereof uniquely identifies those gateways storing the database portion containing the key. In these examples, knowing that gateway 121 can resolve the identifier, gateway 120 only inquires of gateway 121. [Θ048] Storage architecture 400 may resolve the data identifier and handle the read request in any appropriate manner. According to one example, gateway 121 takes over responsibility for the read request if it is determined that namespace database portion stored on gateway 121 (i.e., database portion 404) includes an entry for identifier "/project/alpha/design spec.txt". Taking over a data transaction may include one or more of: performing the translation of the data transaction from the unified namespace to a namespace associated with a storage system, providing the translated data transaction to the storage system, forwarding a response from the storage system to the requesting client, and/or translating the response from the namespace associated with the storage system to the unified namespace prior to forwarding it.

[0049J in such a scenario, gateway 121 reads the value associated with the key and determines that the corresponding value includes "Storage System C, Namespace B,

#09548a22". Assuming that Storage System C corresponds to storage system 132, gateway

121 forwards the read request to storage system 132 with the data identifier #09548a22.

Storage system 132 receives the read request, resolves data identifier #09548a22 using its own internal mapping to find one or more locations within its physical storage drives, reads the data from the location(s), and returns the data to gateway 121 . Gateway 121 then either returns the requested data directly to requesting client 1 11 or sends the requested data to gateway 120 to forward to client 111.

[0050] In another example, gateway 120 searches its namespace database portion 302 using data identifier "/projeet/alpha/design_spec.txt" and, not finding a match, sends a message to gateway 121 with the data identifier. However, in this example, gateway 121 does not take over responsibility for the read request, instead, gateway 121 sends the value of the V pair "Storage System C, Namespace B, #09548a22" to gateway 120. Gateway 120 then forwards the read request to storage system 132 with the data identifier #09548a22. Storage system 132 receives the read request, resolves data identifier #09548a22 using its own internal mapping to find one or more locations within its physical storage drives, reads the data from the location(s), and returns the data to gateway 120. Gateway 120 then returns the requested data to requesting client 111 using the unified namespace protocol. [0051 ] Such a scenario is shown in Fig. 5, which is an illustration of storage architecture 400 according to one embodiment. The arrow between client 1 1 1 and gateway 120 indicates the client 11 1 issuing the request to gateway 120 and the gateway 120 returning the requested data to client 1 1 1. The arrow⁷ between gateway 120 and storage system 132 indicates sending the request from gateway 120 to storage system 132 with the resolved identifier and receiving the requested data from storage system 132. Arrows between gateways 120 and 121 indicate that gateways 120 and 121 communicate with each other to resolve a unified namespace data identifier wherever the V pair may be stored,

[0052J The example in which the originating gateway follows through in completing the data transaction, despite not having resolving information stored locally for the data transaction, may be especial!)' applicable to architectures in which some clients lack communication with some gateways. In those scenarios, the requested data is returned by the same gateway that received the request, thereby ensuring that a communication connection exists between the client and the gateway. Other examples in which another gateway takes over from the first gateway when the first gateway does not resolve the data identifier may be especially applicable to architectures in which clients are connected to multiple gateways. In many architectures, clients will be in communication with multiple gateways to provide fault tolerance.

[0053] In some embodiments, a unified namespace gateway is integrated into a storage system. Fig. 6 illustrates a storage architecture 600 according to one embodiment. In many respects, the unified gateway modules 612 and 613 operate substantially simi lar to unified namespace gateways such as gateways 120 and 121, and the storage systems 602 and 603 operate substantially similar to storage systems such as storage systems 130-133. In such embodiments, the storage systems 602 and 603, and in particular the gateway modules 612 and 613 of the storage systems, perform translation of a unified namespace identifier to an identifier within the namespace of a storage system. In this way, the gateway modules 612 and 613 provide frontends that allow respective storage systems 602 and 603 to service data access requests from a client even if the data does not reside on the storage system.

[0054] To perform the translation, the gateway module 602 may contain some or all of a namespace database. In some embodiments, if a unified namespace identifier is not contained within the portion of the namespace database stored by the gateway module 602, the gateway module 602 provides the unified namespace identifier to another gateway module or to a unified namespace gateway to translate. Once the unified namespace identifier is translated, the gateway module 602 may provide the request with the translated identifier to the corresponding storage system. Additionally, or in the alternative, the gateway module 602 may respond to the client, thereby causing the client to redirect the request to the corresponding storage system. The response from the gateway module to the client may include the translated identifier, an identifier of the corresponding storage system, an identifier of the associated namespace, and/or any other suitable information.

[0055] Fig. 7 is an illustration of example process 700 adapted according to one embodiment. Process 700 may be performed by one or more computer processors executing code to provide the functionality described above with respect to gateways 120 and 121 and gateway module 602,

[0056] At action 710, the gateway receives from a client a first data transaction request including a first data identifier. The data transaction request may include, for example, a read request or a write request. Furthermore, in this example, the first data identifier conforms to a first namespace protocol (e.g., a universal namespace protocol) that is not directly actionable by various underlying storage systems. This is due in part to the storage systems using one or more namespace protocols that are different from the first namespace protocol .

[0057] At action 720, the gateway determines a storage system associated with the first identifier. In one example, the gateway traverses a database of KV pairs to find an entry that corresponds to the first identifier. The value associated with the first identifier provides an indication of the storage system that should receive the data transaction request.

[Θ058] At action 730, the gateway determines a second data identifier associated with the first data identifier, wherein the second data identifier conforms to a namespace protocol associated with the storage system. In one example, the gateway determines the second data identifier by examining the same KV database entry that indicates the storage system in action 720.

[0059] Whereas the first data identifier conforms to the first namespace protocol, the second data identifier conforms to a namespace protocol of the storage system. For instance, the storage system uses a different namespace protocol than does the client, and the storage system's namespace protocol may also be different from namespaces used by other storage systems in the architecture. It is not necessary that any of the namespace protocols be compatible, in other words, the first data identifier conforms to a protocol that is not compatible with that of the storage system; similarly, the second data identifier conforms to a protocol that is not compatible with that of the client. Instead , the gateway translates between the namespace protocols to allow the client to perform data transactions on any storage system in the architecture while at the same time hiding the underlying storage architecture complexity from the clients. Thus, in some embodiments, the clients may only know and use the unified namespace protocol (of the first data identifier) and be unaware of the namespace protocols used by the storage systems.

[ΘΘ60] At action 740, the gateway forwards a second data transaction request referencing the second data identifier to the storage system. A storage controller of the storage system maps the second data identifier to one or more physical storage addresses in its own data store and returns results to the gateway.

[0061 ] The scope of embodiments is not limited to the actions shown in Fig. 7, as other embodiments may add, omit, rearrange, or modify one or more actions. For instance, some processes may include repeating actions 710-740 for each transaction request received.

Moreover, in examples in which the requested data transaction includes a write request, process 700 may also include updating a database of data identifiers to create or modify a record consistent with the written data, if the database is distributed among a multitude of gateways, updating the database may include identifying at least one gateway device to store the entry of the distributed database by, e.g., considering load balancing among the gateways. In a scenario in which the requested data transaction is a read request, process 700 may also include forwarding the read results to the client from the gateway.

[0062] Fig. 8 is an il lustration of example process 800 adapted according to one

embodiment. Process 800 may be performed by one or more computer processors executing code to provide the functionality described above with respect to gateways 120 and 121 and gateway module 602, In the process 800, a first gateway, having received a first data transaction request, finds that it does not have local access to a distributed database entry corresponding to requested data. The actions of Fig. 8 are discussed above with respect to Figs. 4 and 5 as well,

[0063J At action 81 0, the gateway receives from a client a first data transaction request including a first data identifier. Action 810 is substantially similar to action 710 (Fig. 7), discussed above.

[0064] At action 820, the gateway searches a portion of a distributed database for an entr to resolve the first data identifier into a second data identifier and a corresponding storage system. An example of action 820 is given above, wherein the gateway traverses a database of KV pairs, where the keys in the KV pairs correspond to first data identifiers. Further, in action 820, the gateway has local access to a portion of a distribu ted database, where the portion is less than the full database and does not, by itself, provide comprehensive resolution of first data identifiers because some first data identifiers may be missing from the portion.

[0065] At action 830, upon determining that no database entry exists locally for resolving the first data identifier, the gateway communicates with another gateway to complete the first data transaction. For example, the gateway may forward the first data transaction request to the other gateway or may merely pass an inquiry to the other gateway, where the inquiry references the first data identifier. Any appropriate communication between gateways can be used by various embodiments to facilitate completion of the first data transaction.

[0066] At action 840, the first data transaction is completed, as facilitated by action 630. In some embodiments, the first gateway passes the first data transaction request to the second gateway, and the second gateway takes over responsibility for the first data transaction. The second gateway resolves the first data identifier into a corresponding storage system and second data identifier and requests the transaction from the storage system using the second data identifier (using, e.g. , actions 720-740 of Fig. 7). The second gateway can communicate results of th e data transaction to the requesting client itself or can forward the results to the first gateway, thereby allowing the first gateway to pass the results to the requesting client. [0067] in another embodiment, the first gateway requests that the second gateway resolve the first data identifier and return the second data identifier and storage system identifier to the first gateway. Upon receipt of the second data identifier and storage system identifier, the first gateway then completes the transaction according to action 740 of Fig. 7.

[Θ068] Various embodiments may include one or more advantages over conventional systems. For instance, various embodiments provide a unified namespace that hides the complexity of the underlying data storage architecture from the clients. Such action allows the cl ients to u se a single namespace and pass the burden of keeping track of namespaces to the gateways.

[ΘΘ69] Furthermore, various embodiments use a distributed database of KV pairs to facilitate resolving data identifiers of the unified namespace. The distributed database may provide scalability because it can be used for a large volume of entries and can be split among a multitude of gateways. In other words, storage systems and gateways can be added to the storage architecture by updating the database and providing at least a portion of the database to added gateways.

[ΘΘ70] When implemented via computer-executable instructions, various elements of embodiments of the present di sclosure are in essence the software code defining the operations of such various elements. The executable instructions or software code may be obtained from a non-transitory, tangible readable medium (e.g., a hard drive media, optica] media, RAM, EPROM, EEPROM, tape media, cartridge media, flash memory, ROM, memory stick, network storage device, and/or the like). In fact, readable media can include any medium that can store information. [ΘΘ71] in the embodiments described above, example clients, gateways, and storage systems include processor-based devices and may include general-purpose processors or specially-adapted processors (e.g., an Application Specific Integrated Circuit). Such processor-based devices may include or otherwise access the non-transitory, tangible, machine readable media to read and execute the code. By executing the code, the one or more processors perform the actions of processes 700 and/or 800 as described above.

[0072] Thus, the systems and methods of the present disclosure provide a unified or federated namespace capabl e of identifying data across heterogeneous storage systems using a single universal identifier. In many embodiments a database used in translating data identifiers to and from the unified namespace is distributed across multiple gateway systems. This distributes the translatioiial burden across the gateways and reduces system bottlenecks particularly as the size of the unified namespace grows.

[0073] The foregoing outlines features of several embodiments so that those skilled in the art may better understand the aspects of the present disclosure. Those skilled in the art should appreciate that they may readily use the present disclosure as a basis for designing or modifying other processes and structures for carrying out the same purposes and/or achieving the same advantages of the embodiments introduced herein. Those skilled in the art. should also realize that such equivalent constructions do not depart from the spirit and scope of the present disclosure, and that they may make various changes, substitutions, and alterations herein without departing from the spirit and scope of the present disclosure.

Claims

WHAT IS CLAIMED IS:

1. A computer-based method for performing a data transaction in a network storage system, the method comprising:

receiving, from a client, a first data transaction request including a first data identifier associated with a first namespace protocol;

determining a storage system associated with the first data identifier and associated with a second namespace protocol;

determining a second data identifier associated with the first data identifier, wherem the second data identifier conforms to the second namespace protocol; and

forwarding a second data transaction request including the second data identifier to the storage system based on determining an association between the second data identifier and the first data identifier,

wherein the first namespace protocol and the second namespace protocol are different.

2. The method of claim 1, further comprising determining the namespace protocol associated with the storage system from a plurality of namespace protocols associated with a plurality of storage systems.

3. The method of claim 1, wherein the determining of the second data identifier includes querying a distributed database based on the first data identifier.

4. The method of claim 3, wherein the querying of the distributed database includes:

sending a communication from a first gateway device to a second gateway device storing a portion of the distributed database, wherein the portion includes the second data identifier.

5. The method of claim 3, wherein the distributed database stores data identifiers as key-value pairs distributed over a plurality of gateway devices.

6. The method of claim 1 further comprising adding an entry to a distributed database, wherein the entry expresses a relationship between the first data identifier and the second data identifier.

7. The method of claim 6, wherein the adding of the entry to the distributed database comprises:

identifying at least one gateway device to store the entry of the distributed database from a plurality of gateway devices associated with the distributed database; and

providing the entry to the at least one gateway device.

8. The method of claim 1 further comprising:

receiving a response to the second data transaction from the storage system; and providing the response to the client.

9. The method of claim 8, wherein the received response conforms to the namespace protocol associated with the storage system, and wherein the providing of the response to the client includes modifying the response to conform to a namespace protocol associated with the data transaction.

10. A computer program product having a non-transitory computer readable medium tangibly recording computer program logic for performing a data transaction, the computer program product comprising:

code to receive, from a client, a first data identifier associated with a first namespace; code to deterniine a second data identifier corresponding to the first data identifier, wherein the second data identifier is associated with a second namespace from a plurality of namespaces each associated with at least one storage system; and

code to provide the second data identifier for use in a data transaction directed to a storage system associated with the second namespace.

11. The computer program product of claim 10,

wherein a relationship between th e first namespace and the plurality of namespaces is expressed in a database,

wherein the database is distributed over a plurality of node systems, and

wherein the code to determine the second data identifier includes code to quer a node system of the plurality of node systems based on the node system storing a portion of the database associated with the first data identifier.

12, The computer program product of claim 1 1 further comprising code to add a key- value pair to the database by providing the first data identifier to the node system, wherein the key-value pair expresses a relationship between the first data identifier and the second data identifier.

13, The computer program product of claim 10 further comprising:

code to receive a response to the data transaction from the storage system, vv erein the response conforms to the second namespace;

code to modify the response to conform to the first namespace; and

code to provide the modified response to the client.

14. A namespace gateway computing system, the system comprising:

a first gateway of a plurality of gateways in a network storage system, wherein the first gateway stores a first portion of a namespace database, and other gateways of the pi uraiity store other respective portions of the namespace database, and wherein the first gateway is operable to:

receive a first data identifier in accordance with a first namespace, the first data identifier being received from one of the other gateways of the plurality, the first namespace being incompatible with storage systems of the network storage system; traverse the first portion of the namespace database using the first data identifier;

determine, using th e fi rst portion of the namespace database, a second data identifier associated with the first data identifier, wherein the second data identifier is in accordance with a second namespace compatible with at least one storage system of the network storage system.

15. The computing system of claim 14, wherein the namespace database includes at least one data identifier in accordance with each of a plurality of namespaces.

16. The computing system of claim 14, wherein the namespace database includes a plurality of key-value pairs associating data identifiers in accordance with the first namespace and data identifiers in accordance with a plurality of namespaces that are each different from the first namespace.

17, The computing system of claim 14, wherein the first gateway is further operable to perform a data transaction associated with the first data identifier on behalf of the one of the other gateways from which the first data identifier was received.

18, The computing system of claim 14, wherein the first gateway is further operable to add an entry to the namespace database.

19. The computing system of claim 14, where the first gateway is further operable to provide a data transaction including the second data identifier to a storage system associated with the second data identifier.

20. The computing system of claim 19, wherein the first gateway is further operable to receive a response to the provided data transaction from the storage system and to provide a corresponding response to a client.