KR20190056295A - On-demand storage provisioning using distributed and virtual namespace management - Google Patents

Abstract

The present invention relates to an on-demand storage provisioning using dispersed virtual name space management. According to an embodiment of the present invention, a storage node comprises: one or more local storage apparatuses; and a storage node controller. The storage node controller includes: a host interface formed to be connected to an application driving on a host computer; a storage manager formed to manage one or more virtual name spaces; and a storage apparatus controller formed to manage each of the name spaces relevant to one or more storage apparatuses. The storage manager is additionally formed to, when the storage node includes a sufficient storage space, extend the storage space relevant to the virtual name space in accordance with a demand on one or more local storage apparatuses of the storage node in a request of the application driving on the host computer. When the storage node includes an insufficient storage space, the storage manager is formed to communicate with a peer storage manager of a second storage node on the network and to extend a storage space relevant to the virtual name space in accordance with a demand on the local storage apparatus of the second storage node from a request of the application driving on the host computer.

Description

[0001] ON-DEMAND STORAGE PROVISIONING USING DISTRIBUTED AND VIRTUAL NAMESPACE MANAGEMENT USING DISTRIBUTED VIRTUAL NOMENACE SPACE MANAGEMENT [0002]

The present invention relates to data storage systems, and more particularly to data storage devices, particularly systems that provide on-demand storage provisioning using distributed virtual namespace management for NVMe-SSDs and their variants. And methods.

Applications running on computer systems can access data stored on non-volatile data storage devices such as solid-state drives (SSDs). In particular, a nonvolatile memory express (NVMe) -based SSD is connected to a host computer system via a Peripheral Component Interconnect express (PCIe) bus. In general, applications can not increase on-demand storage capacity from NVMe SSDs because the storage capacity of NVMe SSDs is limited to the available physical flash capacity of NVMe SSDs.

The limitation on the number of physical connections to physical NVMe-SSD devices limits device connection management and prevents the use of connection multiplexing and management of multiple flash devices in the cluster. Multiplexing of multiple NVMe connections into a group of devices (each physical device can manage more than one connection) can help improve scaling and performance through the tiers of devices at the backend. Additionally, multiplexing can help address issues of failover, failback, better namespace, or capacity provisioning during run-time in response to unpredictable application run-time requests. The allocation of storage capacity in an NVMe SSD is very static, and its physical storage space can not be increased dynamically on demand during run-time. The non-scalable storage capacity of the NVMe SSD prevents applications running on one node of the network from taking advantage of unused storage space on other peer NVMe SSDs in the same network. This results in over-provisioning of data on the local node or on the physical host, over-provisioning of data storage capacity over the data storage layer level in the data center over time, and the total cost of operating the data center The total cost of ownership (TCO) increases.

According to one embodiment of the invention, a storage node at a storage node comprises: one or more local storage devices; And a storage node controller. The storage node controller comprising: a host interface configured to connect with an application running on the host computer; A storage manager configured to manage one or more virtual namespaces; And a storage device controller configured to manage respective namespaces associated with the one or more storage devices. Wherein the storage manager is further adapted to, in response to a request from the application running on the host computer if the storage node includes sufficient storage space, to provide storage associated with the virtual namespace in response to a request on the one or more local storage devices of the storage node Is further configured to expand the space. Wherein the storage manager is configured to communicate with a peer storage manager of a second storage node on a network if the storage node includes insufficient storage space, wherein in the request of the application running on the host computer, Is further configured to expand the storage space associated with the virtual namespace according to a request on the local storage device.

According to another embodiment of the present invention, a storage system comprises: a plurality of host computers; And a plurality of storage nodes coupled to the plurality of host computers via a network. The plurality of storage nodes include a storage node including a storage node controller. The storage node controller comprising: a host interface configured to connect with an application running on the host computer; A storage manager configured to manage one or more virtual namespaces; And a storage device controller configured to manage respective namespaces associated with the one or more storage devices. Wherein the storage manager is further adapted to, in response to a request from the application running on the host computer if the storage node includes sufficient storage space, to provide storage associated with the virtual namespace in response to a request on the one or more local storage devices of the storage node Is further configured to expand the space. Wherein the storage manager is configured to communicate with a peer storage manager of a second storage node on a network if the storage node includes insufficient storage space, wherein in the request of the application running on the host computer, Is further configured to expand the storage space associated with the virtual namespace according to a request on the local storage device.

The communication to the peer storage manager may be shared (cluttered or distributed) information indicating the status of the NVMe namespaces, available storage capacity, used space, etc. (e.g., Quot;). ≪ / RTI > The peer communication between the storage manager processes manages keeping the metadata information up-to-date synchronously.

According to another embodiment of the present invention, a method includes: allocating storage space to a first storage device of a first storage node using a virtual namespace; Receiving a write request for storing data in the storage space from an application running on a host computer; Determining if the allocated storage space includes insufficient storage space to store the data associated with the write request; Expanding the storage space by including additional storage space in the first storage node if the first storage node includes sufficient storage space to store the data; Identifying a second storage node that includes additional storage space if the first storage node includes insufficient storage space to store the data; Negotiating with the second storage node to allocate the additional storage space; Expanding the storage space by including the additional allocated storage space in the second storage node; Writing the data to the additional storage space at the second storage node by peer-to-peer communication between the first storage node and the second storage node over the network; And updating the global mapping table to update the mapping information of the virtual namespace.

BRIEF DESCRIPTION OF THE DRAWINGS The above and other appropriate features, which include combinations of various novel descriptions and events of the implementation, will be more fully described with reference to the accompanying drawings and will be referred to in the claims. It is to be understood that the specific systems and methods described herein are shown by way of example only, and not limitation. As will be appreciated by those skilled in the art, the theories and features described in the text may be used in numerous and numerous embodiments without departing from the spirit of the invention.

BRIEF DESCRIPTION OF THE DRAWINGS The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate preferred embodiments of the invention and, together with the general description given above, the detailed description of the preferred embodiments given below serves to teach and explain the theory described in the text.
1 is a block diagram of an exemplary framework capable of providing on-demand storage provisioning, in accordance with one embodiment.
2 is a block diagram of an exemplary framework, in accordance with one embodiment.
3 is a block diagram of an exemplary storage system, in accordance with one embodiment.
4 is a flowchart of dynamically allocating storage space in a cluster of storage nodes, according to one embodiment.
The drawings are not necessarily drawn to scale, and similar elements or elements of the functions are generally designated with like reference numerals throughout the drawings for purposes of illustration. The drawings are intended to facilitate describing the various embodiments described herein. The drawings are not intended to illustrate all of the teachings of the teachings herein and are not intended to limit the scope of the claims.

Each of the features and teachings described herein may provide a framework capable of on-demand storage provisioning and virtual namespace management in connection with or related to other features and teachings. Representative embodiments utilizing various additional features and teachings, either individually or in combination, are described in further detail with reference to the accompanying drawings. This description is merely intended to provide those of ordinary skill in the art with a more detailed description of how to implement the aspects of the invention, and is not intended to limit the scope of the claims. Therefore, combinations of the features described above in the detailed description may not be necessary to practice the widest conceptual teachings, and are intended only to describe exemplary embodiments of the teachings of the present invention.

In the following description, for purposes of explanation only, specific nomenclatures provide a thorough understanding of the detailed description. However, it will be understood by those skilled in the art that the detailed description is not required to practice the teachings of the present invention.

Some portions of the detailed description in the text are expressed in terms of algorithms and symbolic representations of operations on data bits in computer memory. These algorithmic explanations and symbols are used by those skilled in the art in data processing applications to effectively convey the content of their work to other persons skilled in the art. The algorithm is understood to be a self-sustaining sequence of steps leading to the intended result. Steps are physical manipulation of physical quantities . Generally, though not necessarily, these quantities can take the form of electrical or magnetic signals that can be stored, transmitted, combined, compared, and otherwise manipulated. It has proven convenient to refer to these signals as bits, values, elements, symbols, letters, terms, numbers, etc., primarily for common usage reasons.

It should be borne in mind, however, that all of these and similar terms should be associated with appropriate physical quantities, and are merely labels applied to these quantities. Quot; computing ", " calculating ", " determing ", " displaying ", and the like, unless the context clearly dictates otherwise. The descriptions utilizing such terms generally refer to data expressed in physical (electronic) quantities in registers and memories of a computer system as data in computer system memories, registers, or other information storage, transmission, or display devices, Refers to the operations and processes of a computer system or similar electronic computing device that converts and modulates other data represented as < RTI ID = 0.0 > < / RTI >

Moreover, the various figures and the appended claims of the exemplary embodiments may be combined in a manner not specifically mentioned or expressly set forth in order to provide additional useful embodiments of the invention. It also specifies that all value ranges or indications of an enterprise group provide all possible value-added or intermediate entities for the purpose of limiting the claims as well as the purpose of initial disclosure. Also, the shapes and dimensions of the structures shown in the figures are designed to facilitate understanding of how the present invention is implemented, and do not limit the shapes and dimensions shown in the embodiments.

An application running on the host computer may require device management and data storage provisioning at run time for NVMe-based SSDs. Limitations on on-demand device management and provisioning imposed by requests for physical connections between the host computer and NVMe-based SSDs can be solved by virtualizing the management of NVMe-SSD devices / namespaces. The present invention provides a framework for extracting physical data storage devices (e.g., NVMe-SSD devices) from virtual disk entities (called namespaces) consumed by an application . In particular, device management and data provisioning can be used to create objects or key-values (KVs) that use virtual disks and virtual namespaces in a distributed storage cluster forming the foundations of a SaaS (as-a- -value) repositories.

The framework of the present invention can create and extend storage NVMe namespaces used by application on-demand and can provide dynamic and continuous provisioning of storage space. The framework of the present invention can optimize disk usage by managing the storage of objects / KV in a distributed storage cluster in a resilient, on-demand provisioned manner and by managing virtual disks and virtual namespaces. The framework of the present invention can be exported as a SaaS (SSD-as-a-Service) model in a cloud environment or data center.

The framework of the present invention synthesizes NVMe SSDs over clusters of storage nodes on a network (e.g., a fabric network such as Ethernet, Fiber Channel, and Infiniband) The application provides virtual disks as on-demand. These virtual disks can be mapped to physical disks through virtual namespace mapping. Thus, a virtual disk can be mapped to a physical disk or a virtual namespace that can be mapped into the namespace itself. The mapping may be 1: 1: 1 or 1: M: N. The framework of the present invention intelligently manages and virtualizes distributed disks and their namespaces across a plurality of storage nodes to dynamically (or elastically) and automatically expand the virtual disk capacity do. In one embodiment, the framework of the present invention manages the namespaces of distributed NVMe-SSD devices and distributed NVMe-SSD devices in an application in an application-unaware way, storing and retrieving object / KV data . Because the framework of the present invention can manage both data and management planes, it can be extended to manage or include other services, such as HA (High Availability), replication, and other storage services. Accordingly, the framework of the present invention can be used to export SSD-as-a-Service (SaaS) in the cloud and data center.

Currently available NVMe disks allow only one NVMe-oF connection to each namespace. The framework of the present invention removes these limitations by virtualizing or extracting physical NVMe disks and namespaces. Multiple connections through multiple NVMe-oF devices may be useful for data center applications to provide services such as High Availability (HA), Failover, and Disaster Recovery, or distributed storage features .

According to one embodiment, the framework of the present invention provides storage node controllers on each storage node or node designated to accept or intercept connections to the backend NVMe SSD. A storage node controller of a designated node or storage node for multiple back-end NVMe SSDs can communicate with other peer storage node controllers on other nodes in the same cluster or storage pool of storage nodes. The storage node controller can be implemented as a device / node-level, lightweight, thin software stack (or module) that can enable connection of requesting applications and other storage node controllers in the same cluster as the underlying NVMe SSDs. Peer-to-peer communication between storage node controllers may be performed over a fabric network such as Ethernet, Fiber Channel, and Infiniband.

According to one embodiment, the framework of the present invention is to uncept the NVMe / NVMe-oF request issued by one or more target NVMe SSDs or namespaces in the backend by an application running on the host computer, It responds to the application as a virtual namespace encapsulation. Here, an NVMe-oF request is an example of a data path request issued by an application to target NVMe SSDs or namespaces, and may include NVMe commands over a physical connection (i.e., a PCIe connection). The data path request may include both I / O commands and admin commands. Because the framework of the present invention eliminates the overhead of applications and establishes and manages connections to NVMe SSDs, the framework of the present invention can support logical connections as well as physical connections, via virtual namespace encapsulation. Moreover, the framework of the present invention can support any number of connections (virtual connections) to backend NVMe SSDs without being limited by the available number of physical connections. In the case of NVMe requests, only one physical connection to each NVMe SSD is possible via the PCIe bus.

The framework of the present invention can efficiently establish and manage connections with one or more backend NVMe SSDs on demand. The framework of the present invention can provide communication protocols between peer storage node controllers through clusters of storage nodes and establish connections to backend NVMe SSDs that can be distributed through clusters. In addition, the framework of the present invention can be implemented as separate, lightweight, fast, concurrent threads that can be scheduled in the user's space of the application, or run in separate I / O queues for input and output separately for each backend NVMe SSD You can manage input / output (I / O) paths asynchronously through operations. The actual I / O data path may include other optimizations depending on the actual implementation and operating environment. For example, the I / O data path of the framework of the present invention may be optimized for direct device access from user space without user-mode disk I / O scheduling and kernel or kernel mode drivers intervention.

1 is a block diagram of an exemplary framework capable of providing on-demand storage provisioning, in accordance with one embodiment. The plurality of client applications 110a-110d run on one or more host computers (not shown). In one embodiment, each client application 110 is connected to a node 130 based on its physical connection. For example, client 110a is connected to node 130a, and clients 110b and 110d are connected to node 130b. In another embodiment, the connection between the client 110 and the node 130 may be on the fabric network 150, such as Ethernet, Fiber Channel, and InfiniBand. Each node 130 includes a storage controller 120 that is capable of establishing and managing connections to the local storage 160. The local storage device 160 may be connected to the node 130 via the PCIe bus. In addition, the storage controller 120 can further establish and manage connections to one or more backend storage devices 140 via a network connection. The connection between the node 130 and the backend storage devices 140 may be through the same fabric network as the fabric network 150 connecting the clients 110 to the nodes 130. The storage controllers 120 of the nodes 130 may communicate with one another via an internal-node communication path that may be established on the fabric network 150 or may be an independent back-channel that guarantees minimum network latencies .

Figure 2 shows a block diagram of an exemplary framework in accordance with one embodiment. The framework 200 includes a plurality of hosts 210 including a host 210a and a host 210b and a plurality of storage nodes 230 including a node 230a and a node 230b . Each of the hosts may be connected to the at least one storage node 230 either locally via the PCIe bus or remotely via the network 250. When the host 210 and the storage node 230 are connected via the PCIe bus, the host 210 and the connected storage node 230 may be located in the same rack of the data center.

The storage node 230 includes a storage node controller 220 and one or more local storage devices 260. Of the plurality of nodes 230 in a cluster of storage nodes, each storage node 230 may include a storage node controller 220. In some embodiments, the storage node 230 may be designated to provide services to manage the associated physical disks 260 (e.g., backend NVMe SSDs) and their associated namespaces.

Each storage node controller 220 includes a host connection manager 221, a namespace / disk manager 222, and an NVMe controller 223. Host connection manager 221 manages connections either locally or on network 250 between applications running on host 210 and node 230.

The host connection manager 221 processes the request received from the client application and provides it to the NS / disk manager 222. Here, NS / disk manager 222 may be referred to as a storage device manager or a virtual namespace manager, as it manages the connection from a remote node to a network-connected storage device using a virtual namespace. For example, the host connection manager 221 intercepts NVMe-oF requests from the client application and provides NVMe-oF requests to the NS / disk manager 222 for further processing. The NS / disk manager 222 is coupled to one or more back-end NVMe SSDs 260, which may connect to one or more back-end NVMe SSDs 260 and locally or remotely through the fabric network in the same cluster Manages mapping to namespaces. The mapping of the remote backend SSD 260 to the virtual namespaces may be performed through a peer-to-peer connection between the two NS / disk managers 222.

The NVMe controller 223 can establish, control, manage, and assign a namespace identifier (nsid) to the I / O data paths and connections to the attached backend NVMe SSDs 260. After the connection is established, the NVMe controller 223 can route the I / O data to the namespace and from the namespace using the namespace identifier.

Here, the namespace NS refers to a list of logical block addresses (LBAs) addressable to each NVMe SSD 260. In an embodiment of the invention, node 230a includes one or more NVMe SSDs 260a, 260b. One or more NVMe SSDs 260a and 260b may be designated by their namespace (NS A, NS B) using their unique identifiers (nsid1, nsid2). Similarly, node 230b may include one or more NVMe SSDs 260c and 260d. One or more NVMe SSDs 260c and 260d are addressable by their namespace (ns c, ns d) using their identifiers (nsid3, nsid4).

According to one embodiment, the NS / disk manager 222 uses one or more storage devices 260 and their namespace, which may be associated with them, using the framework 200 of the present invention to another node You can map it virtually. The virtually mapped namespaces are named in the text as virtual namespaces (VNS). In order to support associations between namespaces NS of NVMe SSDs and virtual namespaces NVS, the NS / disk manager 222 may include a particular node 220, which may be local or remote to the node 220, Control, and manage virtual namespaces that can be associated with the NVMe SSDs 260 of the device. In comparison, the NVMe controller 223 can establish, control, and manage namespaces (NS) associated with physically connected local NVMe SSDs 260. The virtual namespace may be mapped to one or more namespaces that may be distributed through the cluster, but the NS / disk manager 222 may be operatively coupled to the namespace NS managed by the NVMe controller 223, Manages virtual namespaces.

The NS / disk manager 222 may associate the namespace NS of the NVMe SSD 260 with one or more virtual namespaces, according to the configuration and virtualization scheme of the operating system of the host 210. Using the virtual namespace mapping capability of the NS / disk manager 222, the framework 200 of the present invention can flexibly expand or shrink the storage space as required by requests from applications.

In an embodiment of the present invention, the NS / DX manager 222a maps the virtual namespace vnsid200 to two distributed namespaces NSB and NSC. The namespace NS B is physically attached to the NVMe controller 223a of the node 220a and the namespace NS C is physically attached to the NVMe controller 223b of the node 220b. The storage node controller 220a can internally manage mappings (including virtual mappings) by using distributed metadata on the fabric network 250. [ Mapping to the virtual namespaces by the framework 200 of the present invention is performed by the host 210 (or host 210), since the virtual mapping and distributed storage settings and connections are performed through the peer NS / disk managers 222. [ (Such as a client application running on the backend NVMe SSDs 210) or backend NVMe SSDs 260.

The distributed metadata may be stored in a metadata table that can be accessed by the NS / disk manager 222. The global metadata table stores information on use / free / available space at the unit-host level for all disks / namespaces in the host. The metadata table may be maintained as global or shared information at the cluster level. Since space requests are not frequent, updates to the metadata table may not happen often. In an embodiment of the present invention, the NS / disk manager 222a searches the metadata table to determine the amount of disk space required in the cluster and sends a request to the NS / disk manager 222b to create a namespace NS C ).

The framework 200 of the present invention integrates the NVMe SSDs 260 of the distributed network 250 as a single pool of storage, resiliently expanding or reducing the virtual namespace according to the requested storage capacity, Disk or virtual namespace to the client application. The framework 200 of the present invention can establish multiple connections to remote storage devices and their virtual namespaces on demand and use the network of NS / disk managers 222 to connect connections internally . Each NS / disk manager 222 may be a device / node-level, lightweight, thin software stack (or module) that is deployed in nodes that manage the physical connection to the attached NVMe SSDs 260.

Although an embodiment of the present invention shows two hosts 210 and two nodes 230, it will be appreciated that any number of hosts and nodes may reside in a cluster of storage nodes. Additionally, embodiments of the present invention show that each node 230 includes NVMe SSDs 260, and each NVMe SSD 260 can be managed by its unique name space identifier. It will be appreciated that a number of other and any number and any type of storage devices may be present at each node 230. For example, the storage device 260 may be an NVMe SSD or an NVMe-oF SSD.

3 shows a block diagram of an exemplary storage system, in accordance with one embodiment. The storage system 300 can use the framework of the present invention to provision storage space between a plurality of storage nodes 330 or storage devices 370 and a client application 301 running on a host computer . The storage node 330 includes a storage node controller (e.g., storage node controllers 220a, 220b of FIG. 2) capable of controlling one or more storage devices 370 of the storage node 330 do. Each storage node controller may be connected to other storage node controllers via the fabric network 350. Fabric network 350 may be referred to as a back-channel or internal communication network or management channel. Each storage node controller may export mount points for client applications 301 to mount and initiate regular block read / write and object / KV stores on distributed storage devices 370. [ The exported mount point may be created by mapping virtual namespaces from the backend directly attached to the storage node controller itself or from the other peer storage node 330 through the fabric network 350 to the actual physical namespace attached to the physical storage device . Each virtual namespace contains a unique virtual namespace identifier (vnsid). Assignment of the virtual namespace identifiers may be performed by consensus with the peer storage controllers of the storage system 300. The assignment and distribution of the virtual namespace identifiers can be dynamically managed or computed a priori when the connection request arrives. The consensus algorithm may be a generic distributed consensus algorithm that is well known in the art or may be located in a virtual namespace table and accessed by all storage controllers via shared storage Or may simply be a simple identifier that is simply distributed and copied to all storage nodes when atomically updated by a single storage node controller that is part of the storage system 300. [

For example, the mount point / mnt / NVMe0n1 contains a virtual namespace table that contains a set of vnsids.

{ vnsid1 = (IP addr1, nsid-xyz),

vnsid2 = (IP addr2, nsid-xyz),

vnsid2 = (IP addr3, nsid-xyz)

}.

The vnids can be created utilizing the controller ID-NSID, and can be associated with an IP address to aid in controller migration. In another embodiment, vnsid may be created utilizing the subsystem ID that includes NVMe controllers and their namespaces. vnsids can be generated by inference.

Since the new NVMe-oF SSDs can be added or removed as mount points without affecting the client application, it is possible to easily create more storage space, the mapping table can be extended on demand. The client application can request an initial capacity name space and can increase or decrease the initial storage capacity as required because the client application can consume the initially allocated capacity. The interface / application programming interface (API) option may be passed to the framework at connection setup time and / or as part of the extended NVMe command set. The framework of the present invention may fail in managing a write request by extending the namespace during operation, or in a client application attempting to write over the initially negotiated capacity (failing to detect the requested application).

According to one embodiment, the framework of the present invention can organize communications between peer storage controllers to ensure that the assigned virtual namespace identifiers are unique. Additionally, the framework can manage the complexities that may arise when the data storage space (or object or key-value) is assigned or written to a particular namespace, The storage space may be extended to other physical storage nodes by the virtual addition of one or more physical namespaces.

According to one embodiment, the framework of the present invention can manage object requests (read / write) through virtual namespaces. For example, the application is connected to the first storage node SN1, and the storage node controller of the storage node SN1 allocates the virtual namespace indicated by vnsid100 to the application. While the virtual namespace vnsid 100 is being allocated, the storage node controller can create a large object (VLOB_ 100) that can span the virtual namespace (vnsid 100) filling the initially allocated storage space. Assignment of the virtual namespace can be performed by the application by calling an API command, for example " PUT key-vlob100 ". The object VLOB_100 may be located entirely at the storage node SN1 of either a local attached storage (DAS) or at least one of the other physical storage devices of the remote storage node. The storage node controller manages the allocation and allocation of the object (VLOB_100) and the management and creation of the virtual namespace according to a request to one or more configuration drive namespaces. After allocating the object (VLOB_100), the storage node controller communicates with the application to notify that the object (VLOB_100) has been successfully allocated. The application updates the object (VLOB_100) on the application side and uses the allocated storage space Start.

While initially using the allocated space, the application may consume the initially allocated storage space as a whole. In this case, the object put command by the application converting from the primary storage device to a COW (copy-to-write) command may return an error (e.g., ENOSPACE). The storage node controller can intercept and manage errors by allocating additional storage space.

The storage node controller can extend the namespace by allocating another namespace and appending (writing) it to the current namespace. For example, by allocating additional storage space to the storage node SN2 together with the IP address IPaddr2, the former vnsid = (IPaddr1, nsid-nvme0) becomes vnsid100 = {(IPaddr1, nsid-nvme0), (IPaddr2, nsid -nvme9)}. After negotiation with the storage node SN2, the object VLOB_100 may be extended through the physical storage device belonging to the storage node SN2 and the excess content of the VLOB_100 may be transferred to the physical storage device available from the storage node SN2 Can be written. The physical namespace (nsid-nvme9) is mapped to the physical storage device labeled nvme9 of the local storage node SN2.

Other commands, such as READs or GETs of objects or KV data, may proceed in a substantially similar manner. The global mapping table stores the updated virtual namespace identifiers. The vnsid index in the global mapping table indicates the mapping information for a particular object. Parallel reads can be performed to read all the chunks of an object through the storage spaces or namespaces of the storage nodes that form the virtual namespace. Global metadata may be maintained in a distributed manner by the framework of the present invention.

When the same application or other applications request data associated with an object (VLOB_100), the framework of the present invention includes the ability to return the latest copy of VLOB_100 from the cluster to the requesting application. The framework can perform what is described above, by versioning each object using a version-stamp with a timestamp. For example, a first copy (currently outdated ) of VLOB_100 on ( IPaddr1, nsid-NVMe0 ) is marked for deletion and storage node sn1 is marked for garbage collection (an application sends an explicit DELETE object request to TRIM 0.0 > VLOB-100 < / RTI > When the storage node sn1 completes the garbage collection (GC), the storage node sn1 completely erases the first copy of the VLOB_100, and the framework sets the cluster internal location of the VLOB_100 as ( IPaddr2, nsid-nvme9 ).

4 is a flowchart of dynamically allocating storage space in a cluster of storage nodes, according to one embodiment. The client application requests the namespace of the initial storage capacity to the first storage node. In response, the storage manager of the first storage node allocates storage space to the first storage device of the first storage node, and assigns a virtual namespace to the allocated storage space (401). The storage manager receives a write request to store data in the storage space from the client application (402). The storage space initially allocated to the first storage device may be insufficient to store the data. In this case, the storage manager checks whether the currently allocated storage space includes sufficient storage space to store data associated with the write request (403). If the allocated storage space is sufficient, the storage manager writes the data to the allocated storage space (413) and updates the global mapping table accordingly (409). Otherwise, the storage manager checks whether the associated storage node (the first storage node) includes local disk space for writing data (404). If additional local disk space is available, the storage manager scales 414 the storage space in the local disk space, writes 408 data to the additional allocated storage space in the local disk space, and updates the global mapping table accordingly (409). If local disk space is not available, the storage manager looks up the global mapping table to identify a second storage node containing additional storage space (405). Because the storage administrators of the storage system can communicate with each other through the established network, the storage administrator can include information about the availability of additional storage space at other storage nodes in the same cluster. The storage manager of the first storage node negotiates 406 with the storage manager of the second storage node to allocate additional storage space. If the negotiation is successful (407), the storage manager of the first storage node expands the initially allocated storage space to include additional storage space allocated to the second storage device of the second node (417). If the negotiation fails, the storage manager of the first storage node communicates with the other storage nodes to expand the storage space. The storage manager of the first storage node writes 408 data to the additional storage space allocated at the second storage node by peer-to-peer communication between the first storage node and the second storage node through the network, The mapping table is updated to update the mapping information of the virtual namespace (409).

It is assumed that the framework of the present invention hashes to a range ensuring that each object is managed only by one storage node (SN) of the cluster. Since this consistent hash algorithm is known to all the storage nodes of the cluster, the request for VLOB_100 is sent to SN1 in the above example, thereby ensuring strict consistency of reads to the object data. The framework of the present invention can expand and replicate objects through multiple storage nodes of a cluster on a fabric network, further cache objects on replicas, and improve performance and capabilities by invalidating objects on designated replicas .

According to one embodiment, a storage node of a cluster of storage nodes comprises: one or more local storage devices; And a storage node controller. The storage node controller comprises: a host interface configured to connect with an application running on the host computer; A storage manager configured to manage one or more virtual namespaces; And a storage device controller configured to manage respective namespaces associated with the one or more storage devices. If the storage node includes sufficient storage space, the storage manager may further configure to extend the storage space associated with the virtual namespace on demand at one or more local storage devices of the storage node at the request of an application running on the host computer do. If the storage node contains insufficient space, the storage manager is configured to communicate with the peer storage manager of the second storage node on the network, and in response to a request from an application running on the host computer, Thereby extending the storage space associated with the virtual namespace.

One or more storage devices may be coupled to the storage node controller via the PCIe bus.

One or more local storage devices may be NVMe (nonvolatile memory express) devices.

The network may be a fabric network.

The fabric network may be one of Ethernet, Fiber Channel, and Infiniband.

After the storage space is extended to the second storage node, the storage manager can update the global mapping table to update the virtual namespace.

The virtual namespace may be mapped to the at least one storage device of the second storage node in whole or in part and the request to access the data stored in at least one storage device of the second storage node may be a storage manager of the storage node and a second May be handled by peer-to-peer communication between the node's peer storage space managers.

The data stored in the storage space may be an object or key-value (KV) data.

According to another embodiment, a storage system comprises: a plurality of host computers; And a plurality of storage nodes coupled to the plurality of host computers via the network. The plurality of storage nodes includes a storage node including a storage node controller. The storage node controller comprises: a host interface configured to connect with an application running on the host computer; A storage manager configured to manage one or more virtual namespaces; And a storage device controller configured to manage respective namespaces associated with the one or more storage devices. If the storage node includes sufficient storage space, the storage manager may further configure to extend the storage space associated with the virtual namespace on demand on one or more local storage devices of the storage node at the request of an application running on the host computer do. If the storage node contains insufficient space, the storage manager is configured to communicate with the peer storage manager of the second storage node on the network, and upon request of the application running on the host computer, to request on the local storage device of the second storage node Thereby extending the storage space associated with the virtual namespace.

The storage system may further include a global mapping table that updates the virtual namespace after the storage space has been extended to the second storage node.

The local storage device of the second storage node may be coupled to the second storage node controller via the PCIe bus.

The local storage device of the second storage node may be a nonvolatile memory express (NVMe) device.

The network may be a fabric network.

The fabric network may be one of Ethernet, Fiber Channel, and Infiniband.

The data stored in the storage space may be an object or key-value (KV) data.

According to yet another embodiment, a method includes: allocating storage space to a first storage device of a first storage node using a virtual namespace; Receiving a write request for storing data in a storage space from an application running on a host computer; Determining that the allocated storage space includes insufficient storage space to store data associated with the write request; Expanding the storage space by including additional storage space in the first storage node when the first storage node includes sufficient storage space to store data; Writing data to an additional storage space of the first storage node; Identifying a second storage node that includes additional storage space if the first storage node includes insufficient storage space to store data; Negotiating with the second storage node to allocate additional storage space; Expanding the storage space by including storage space additionally allocated to the second storage node; Writing data to a storage space additionally allocated to a second storage node by peer-to-peer communication between the first storage node and the second storage node via the network; And updating the global mapping table to update the mapping information of the virtual namespace.

The method includes: intercepting an error from a storage device controller of the first storage node; And identifying that the second storage node includes additional storage space.

The method may further comprise: accessing the first storage device of the first storage node and the second storage device of the second storage node in parallel.

The method includes: invalidating data stored in the first storage device; Deleting a local copy of the data stored in the first storage device; And updating the virtual namespace to map to a second storage device of the second storage node.

The data contained in the storage space may be an object or key-value (KV) data.

The above-described exemplary embodiments provide an approach for managing object read and write operations by the framework, along with extensions over physical storage nodes and virtual namespace management in a distributed manner, and on-demand storage provisioning using virtual namespace management. ≪ RTI ID = 0.0 > and / or < / RTI > Various modifications from the described exemplary embodiments can be made by those skilled in the art. Configurations within the scope of the invention are set forth in the following claims.

Claims (10)

In a storage node,
One or more local storage devices; And
A storage node controller,
The storage node controller comprising:
A host interface configured to connect with an application running on the host computer;
A storage manager configured to manage one or more virtual namespaces; And
And a storage device controller configured to manage respective namespaces associated with the one or more storage devices,
Wherein the storage manager is further adapted to, in response to a request from the application running on the host computer when the storage node includes sufficient storage space, to store, in response to a request on the one or more local storage devices of the storage node, Further configured to expand the space,
Wherein the storage manager is configured to communicate with a peer storage manager of a second storage node on a network if the storage node includes insufficient storage space, wherein in the request of the application running on the host computer, The storage node being configured to expand the storage space associated with the virtual namespace according to a request on a local storage device of the storage node. The method according to claim 1,
Wherein the one or more storage devices are connected to the storage node controller via a PCIe bus. 3. The method of claim 2,
Wherein the one or more storage devices are nonvolatile memory express (NVMe) devices. The method of claim 3,
Wherein the network is a fabric network. 5. The method of claim 4,
Wherein the fabric network is one of Ethernet, Fiber Channel, and InfiniBand. The method according to claim 1,
After the storage space is extended to the second storage node, the storage manager updates the global mapping table to update the virtual namespace. The method according to claim 1,
Wherein the virtual namespace is mapped in whole or in part to at least one storage device of the second storage node and a request to access data stored in the at least one storage device of the second storage node is stored in the storage node A storage node being processed by peer-to-peer communication between the storage manager and the peer storage manager of the second storage node. The method according to claim 1,
Wherein the data stored in the storage space is an object or key-value (KV) data. In a storage system,
A plurality of host computers; And
A plurality of storage nodes coupled to the plurality of host computers via a network,
The plurality of storage nodes including a storage node including a storage node controller,
The storage node controller comprising:
A host interface configured to connect with an application running on the host computer;
A storage manager configured to manage one or more virtual namespaces; And
And a storage device controller configured to manage respective namespaces associated with the one or more storage devices,
Wherein the storage manager is further adapted to, in response to a request from the application running on the host computer when the storage node includes sufficient storage space, to store, in response to a request on the one or more local storage devices of the storage node, Configured to expand space,
Wherein the storage manager is configured to communicate with a peer storage manager of a second storage node on a network if the storage node includes insufficient storage space, wherein in the request of the application running on the host computer, And to extend the storage space associated with the virtual namespace upon request on a local storage device of the storage system. In the method,
Allocating storage space to a first storage device of a first storage node using a virtual namespace;
Receiving a write request for storing data in the storage space from an application running on a host computer;
Determining that the allocated storage space includes insufficient storage space for storing the data associated with the write request;
Expanding the storage space by including additional storage space in the first storage node if the first storage node includes sufficient storage space to store the data;
Writing data to an additional storage space of the first storage node;
Identifying a second storage node that includes additional storage space if the first storage node includes insufficient storage space to store the data;
Negotiating with the second storage node to allocate the additional storage space;
Expanding the storage space by including the additional allocated storage space in the second storage node;
Writing the data to the additional storage space at the second storage node by peer-to-peer communication between the first storage node and the second storage node over the network; And
Updating the global mapping table to update the mapping information of the virtual namespace.

Images