US11579781B2 - Pooling distributed storage nodes that have specialized hardware - Google Patents

Pooling distributed storage nodes that have specialized hardware Download PDF

Info

Publication number
US11579781B2
US11579781B2 US17/078,504 US202017078504A US11579781B2 US 11579781 B2 US11579781 B2 US 11579781B2 US 202017078504 A US202017078504 A US 202017078504A US 11579781 B2 US11579781 B2 US 11579781B2
Authority
US
United States
Prior art keywords
storage
node
storage nodes
specialized hardware
specialized
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active, expires
Application number
US17/078,504
Other versions
US20220129170A1 (en
Inventor
Gabriel Zvi BenHanokh
Brett Niver
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Red Hat Inc
Original Assignee
Red Hat Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Red Hat Inc filed Critical Red Hat Inc
Priority to US17/078,504 priority Critical patent/US11579781B2/en
Assigned to RED HAT, INC. reassignment RED HAT, INC. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: BENHANOKH, GABRIEL ZVI, NIVER, BRETT
Publication of US20220129170A1 publication Critical patent/US20220129170A1/en
Application granted granted Critical
Publication of US11579781B2 publication Critical patent/US11579781B2/en
Active legal-status Critical Current
Adjusted expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/062—Securing storage systems
    • G06F3/0623—Securing storage systems in relation to content
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/0604—Improving or facilitating administration, e.g. storage management
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/061—Improving I/O performance
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0638—Organizing or formatting or addressing of data
    • G06F3/0644—Management of space entities, e.g. partitions, extents, pools
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0655—Vertical data movement, i.e. input-output transfer; data movement between one or more hosts and one or more storage devices
    • G06F3/0658—Controller construction arrangements
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0655—Vertical data movement, i.e. input-output transfer; data movement between one or more hosts and one or more storage devices
    • G06F3/0659—Command handling arrangements, e.g. command buffers, queues, command scheduling
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0662—Virtualisation aspects
    • G06F3/0664—Virtualisation aspects at device level, e.g. emulation of a storage device or system
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/0671—In-line storage system
    • G06F3/0683—Plurality of storage devices

Definitions

  • the present disclosure relates generally to distributed storage systems. More specifically, but not by way of limitation, this disclosure relates to pooling storage nodes of a distributed storage system that have specialized hardware, such as hardware with built-in compression or encryption capabilities.
  • Distributed storage systems can include storage nodes in communication with each other over a network for synchronizing, coordinating, and storing data.
  • the storage nodes can work together so that the distributed storage system behaves as one storage system.
  • Distributed storage systems can implement block storage, file storage, or object storage techniques.
  • distributed storage systems can be easily scaled horizontally, in the sense that they can combine many storage nodes into a single, shared storage system.
  • Distributed storage systems can also store many copies of the same data for high availability, backup, and disaster recovery purposes.
  • some distributed storage systems can execute compute workloads on the same storage nodes that are also used to store data, thereby yielding a hyper-converged infrastructure that is highly efficient.
  • FIG. 1 is a block diagram of an example of a distributed storage system according to some aspects of the present disclosure.
  • FIG. 2 is a sequence diagram of an example of a process for enabling specialized functionality on a storage node according to some aspects of the present disclosure.
  • FIG. 3 is a block diagram of an example of data migration in a distributed storage system according to some aspects of the present disclosure.
  • FIG. 4 is a block diagram of another example of a distributed storage system according to some aspects of the present disclosure.
  • FIG. 5 a flow chart of an example of a process for pooling distributed storage nodes according to some aspects of the present disclosure.
  • a distributed storage system can include storage nodes formed from relatively heterogeneous servers in communication with each other over a network, such as a local area network.
  • Some of the storage nodes may have specialized hardware.
  • Specialized hardware is a piece of hardware that is preconfigured by a manufacturer to implement specialized functionality, prior to receipt of the specialized hardware by an end user.
  • the specialized hardware can include specialized circuitry configured to implement the specialized functionality.
  • the specialized circuitry may include a custom or semi-custom integrated circuit with a special hardware design or firmware specifically tailored for performing the specialized functionality, where the specialized circuitry is built into the specialized hardware by a manufacturer during a manufacturing process.
  • specialized hardware is different from generic hardware on which aftermarket software is installed by the end user to implement the specialized functionality.
  • the specialized functionality can be functionality that is supplemental to the main purpose of the piece of hardware.
  • the specialized hardware can include a storage adapter, a storage device, an input/output (I/O) interface, or a processing device, where the specialized hardware has appropriate circuitry built-in to implement specialized functionality such as data encryption, data compression, and checksum calculation capabilities.
  • the specialized hardware can be a self-encrypting hard drive with inline data-encryption capabilities. Using specialized hardware to perform the specialized functionality can be faster and more robust than executing aftermarket software on generic hardware.
  • the distributed storage system may operate as if all of the storage nodes lack the specialized hardware, since distributed storage systems generally operate on the basis of the lowest common denominator. As a result, the distributed storage system may not implement the specialized functionality of the specialized hardware, even though the specialized functionality could improve the distributed storage system.
  • Some examples of the present disclosure can overcome one or more of the abovementioned problems by identifying storage nodes in a distributed storage system that have specialized hardware with specialized functionality, enabling the specialized functionality on the storage nodes, grouping the storage nodes together into a pool of storage nodes (“node pool”) that have the specialized functionality enabled, and using the node pool to utilize the specialized functionality in relation to data requests.
  • the data requests can include read requests for reading data, write requests for writing data, or both.
  • Grouping the storage nodes in this way can prevent the data requests from being divided up among some storage nodes that have the specialized hardware and other storage nodes that lack the specialized hardware, so that the data requests arrive at a node pool in which all of the storage node members have the specialized hardware or none of the storage node members have the specialized hardware.
  • the distributed storage system can leverage the specialized functionality to achieve various improvements, such as improvements to data security and throughput.
  • the distributed storage system can include hundreds or thousands of storage nodes. Each storage node can determine if it has specialized hardware capable of performing specialized functionality, such as data encryption, data compression, or both. Each storage node may determine if it has the specialized hardware by scanning its hardware. For example, a storage node can analyze its hardware upon booting up to determine if the specialized hardware is connected. As another example, a storage node may periodically analyze its hardware at predefined intervals to determine if the specialized hardware is connected. As yet another example, a storage node may analyze its hardware in response to an event determine if the specialized hardware is connected. After scanning their hardware, the storage nodes can then transmit status communications indicating whether or not they have the specialized hardware capable of performing the specialized functionality.
  • specialized hardware capable of performing specialized functionality
  • a centralized management node of the distributed storage system can receive the status communications from the storage nodes. Based on the status communications, the management node can determine a subset of the storage nodes that have the specialized hardware. The management node may then transmit communications to the subset of storage nodes for causing the storage nodes to enable the specialized functionality. Alternatively, the storage nodes may automatically enable the specialized functionality upon discovering that they are connected to the specialized hardware. Either way, the specialized functionality can be enabled on the storage nodes.
  • the management node can assign the storage nodes in the subset to the same node pool.
  • the node pool may only contain storage nodes with specialized hardware capable of performing the specialized functionality.
  • the node pool can then be used to service data requests, so that the specialized functionality can be performed in relation to the data requests.
  • Such node pools may be considered higher-tiered pools with better performance or data protection, given their specialized functionality.
  • a service provider that is selling access to the distributed storage system may charge higher fees for using the node pool than for other node pools, such as node pools that lack specialized hardware with the specialized functionality.
  • FIG. 1 is a block diagram of an example of a distributed storage system 100 according to some aspects of the present disclosure.
  • the distributed storage system 100 includes storage nodes 102 a - e .
  • the storage nodes 102 a - e may be physical servers for storing data.
  • Some storage nodes 102 a - c can have specialized hardware 108 a - c , while other storage nodes 102 d - e may lack the specialized hardware.
  • Examples of the specialized hardware 108 a - c can include a storage adapter for interacting with storage devices, a storage device such as a hard disk or solid-state drive, a processing device such as a central processing unit (CPU) or graphics processing unit (GPU), or an I/O interface such as a peripheral component interface (PCI) card.
  • the specialized hardware 108 a - c may be internal to the storage nodes 102 a - c or external and coupled to the storage nodes 102 a - c .
  • the specialized hardware 108 a - c is configured to implement specialized functionality.
  • Examples of the specialized functionality can include data compression/decompression capabilities, data encryption/decryption capabilities, checksum calculation capabilities, or any combination of these.
  • the specialized functionality may be selectively enabled and disabled.
  • the storage nodes 102 a - e can each analyze their hardware to determine if they have a corresponding piece of specialized hardware. For example, storage nodes 102 a - c may analyze their hardware and determine that the specialized hardware 108 a - c is attached. And storage nodes 102 d - e may analyze their hardware and determine that they lack the specialized hardware. The storage nodes 102 a - e may analyze their hardware to determine if they have the specialized hardware in response to any suitable event. For example, the storage nodes 102 a - e may each analyze their hardware to detect the presence of a piece of specialized hardware in response to a boot up event.
  • the storage nodes 102 a - e may each analyze their hardware to detect the presence of the specialized hardware in response to the passage of a predefined amount of time, such as one hour.
  • the storage nodes 102 a - e may each analyze their hardware to detect the presence of the specialized hardware in response to a request 110 from a management node 112 of the distributed storage system 100 .
  • a request 110 is shown in FIG. 1 , but the management node 112 can transmit similar types of requests to many or all of the storage nodes 102 a - e.
  • the storage nodes 102 a - e can generate respective status information indicating whether or not they have the specialized hardware. Each of the storage nodes 102 a - e can then transmit a status communication that includes the respective status information over a network to the management node 112 .
  • the network may be a local area network or the Internet.
  • One example of such a status communication 114 with status information 116 is shown in FIG. 1 , but many or all of the storage nodes 102 a - e may transmit similar types of status communications to the management node 112 .
  • the management node 112 is configured to manage one or more aspects of the distributed storage system 100 .
  • the management node 112 can generate node pools and manage which virtual storage units are mapped to which nodes.
  • the management node 112 may also manage which storage nodes 102 a - e have write-back caching enabled or disabled, as described below.
  • the management node 112 can receive the status communications from the storage nodes 102 a - e and determine a subset of storage nodes 102 a - c that have the specialized hardware 108 a - c based on the status information in the status communications. The specialized functionality of the specialized hardware 108 a - c can then be enabled on the subset of storage nodes 102 a - c . For example, the management node 112 can transmit signals, such as signal 118 , to the subset of storage nodes 102 a - c for causing the storage nodes 102 a - c to enable the specialized functionality.
  • Each signal can include a command or other information configured to cause a corresponding storage node to enable the specialized functionality.
  • the storage nodes 102 a - c may be in a first state in which the specialized functionality is disabled.
  • the management node 112 can transmit signals with activation commands to the storage nodes 102 a - c .
  • the storage nodes 102 a - c can receive the signals, detect the activation commands in the signals, and responsively switch from the first state to a second state in which the specialized functionality is enabled.
  • At least some of the storage nodes 102 a - c may automatically enable the specialized functionality in response to determining that they have back the specialized hardware 108 a - c .
  • a system administrator may manually enable the specialized functionality on at least some of the storage nodes 102 a - c based on determining that the storage nodes 102 a - c have specialized hardware 108 a - c .
  • the management node 112 can assign the subset of storage nodes 102 a - c to a node pool 122 .
  • a node pool can be a defined group of storage nodes configured to implement storage functionality to service one or more read/write requests.
  • the node pool 122 only includes the storage nodes 102 a - c that have the specialized hardware 108 a - c .
  • the node pool 122 may be designated as a higher-tiered pool, since it may have better performance characteristics than another node pool (e.g., formed from storage nodes 102 d - e ) that lacks the specialized hardware or specialized functionality.
  • a user can obtain access to the node pool 122 for storing data. For example, a user may purchase a subscription to the node pool 122 , allowing the user to store and retrieve data therefrom by submitting data requests. Upon the user being granted such access, the distributed storage system 100 can cause the node pool 122 to perform the specialized functionality in relation to the data requests submitted by the user.
  • the specialized functionality may yield better performance or security than is otherwise possible absent the specialized functionality.
  • storage nodes may have more than one type of specialized hardware or may have specialize hardware capable of implementing more than one type of specialized functionality.
  • the management node 112 can group together storage nodes having the same amounts and/or types of specialized functionality into the same node pool.
  • the distributed storage system 100 can include a first set of storage nodes with specialized hardware for implementing both data encryption and data compression.
  • the distributed storage system 100 can also include a second set of storage nodes with specialized hardware for implementing data encryption but not data compression.
  • the distributed storage system 100 include a third set of storage nodes with specialized hardware for implementing data compression but not data encryption.
  • the first set of storage nodes may be grouped into a first node pool capable of implementing both data encryption and data compression.
  • the second set of storage nodes may be grouped into a second node pool capable of implementing data encryption but not data compression.
  • the third set of storage nodes may be grouped into a third node pool capable of implementing data compression but not data encryption.
  • a user may purchase a subscription to whichever of the node pools meets the user's needs, where the first node pool may be more expensive than the second node pool or the third node pool.
  • FIG. 1 is intended to be illustrative and non-limiting. Other examples may include more components, fewer components, different components, or a different arrangement of the components shown in FIG. 1 .
  • the distributed storage system 100 includes five storage nodes in the example of FIG. 1 , the distributed storage system 100 may have hundreds or thousands of storage nodes in other examples.
  • FIG. 2 is a sequence diagram of an example of a process for enabling specialized functionality on distributed storage nodes according to some aspects of the present disclosure. Although the example shown in FIG. 2 includes a certain sequence of steps, other examples may involve more steps, fewer steps, different steps, or a different order of the steps shown in FIG. 2 .
  • the process begins with a management node 112 of a distributed storage system transmitting a request for status information to a storage node 102 .
  • the storage node 102 can receive the request and responsively determine if specialized hardware is coupled to the storage node. In this example, the storage node 102 has determined that the specialized hardware is coupled to the storage node 102 .
  • the storage node 102 can transmit a response to the request, where the response is in the form of a status communication with status information indicating that the storage node 102 is coupled to the specialized hardware.
  • the management node 112 can receive the status communication and determine that the storage node 102 is coupled to the specialized hardware based on the status information.
  • the management node 112 may then transmit a signal to the storage node 102 for causing the storage node 102 to enable the specialized functionality of the specialized hardware.
  • the storage node 102 can receive the signal and responsively enable the specialized functionality.
  • the distributed storage system can use the storage node 102 to service a data request.
  • the data request may be a higher-priority data request for which the specialized functionality may be desirable.
  • a data request may be higher-priority if it is more critical, demanding, or higher cost than other data requests. For example, a user may pay a premium for the user's data requests to be deemed higher priority.
  • FIG. 3 is a block diagram of an example of a migration process in a distributed storage system 100 according to some aspects of the present disclosure.
  • the distributed storage system 100 includes a first node pool 302 for servicing data requests 308 from a client device, such as a laptop computer, desktop computer, server, or mobile device.
  • the client device may be external to the distributed storage system 100 .
  • a storage node 102 d that lacks specialized hardware.
  • the storage node 102 d may be incapable of implementing the specialized functionality corresponding to the specialized hardware.
  • the storage node 102 d may include any number of virtual storage units (VSU) 306 d .
  • Virtual storage units can be logical devices that are mapped to physical storage devices for reading and writing data associated with data requests.
  • the node locations and physical storage-device mappings of the VSUs in the distributed storage system 100 may be adjustable by the management node 112 .
  • the management node 112 can generate a second node pool 304 using the techniques described above, where the second node pool 304 includes storage nodes 102 a - b on which the specialized functionality is enabled.
  • the specialized functionality may be enabled on the storage nodes 102 a - b based on the storage nodes 102 a - b having corresponding specialized hardware 108 a - b .
  • the management node 112 can determine that VSU 306 d _ 1 corresponds to (e.g., is a destination for) the data requests 308 , and then migrate a VSU 306 d _ 1 from storage node 102 d to storage node 102 a . This migration is represented in FIG. 1 by a dashed arrow.
  • the management node 112 can also transmit a communication to the client device from which the data requests 308 originated, to notify the client device of the change in location of the VSU 306 d _ 1 .
  • the client device can direct future data requests 308 corresponding to VSU 306 d _ 1 to storage node 102 a , so that the data requests 308 can be serviced by the second node pool 304 using the specialized functionality.
  • the management node 112 can determine that the data requests 308 have a particular priority level, such as a high priority level associated with sensitive data. Different data requests 308 may have different priority levels assigned by a user or the system.
  • the management node 112 can also determine a VSU 306 d _ 1 associated with the data requests 308 is located on a particular storage node 102 d of the distributed storage system 100 .
  • the management node 112 can communicate with the particular storage node 102 d to determine that the particular storage node 102 d lacks a specialized storage adapter with inline data-encryption capabilities.
  • the management node 112 can receive status information from the particular storage node 102 d indicating that the particular storage node 102 d lacks the specialized storage adapter. Since the particular storage node 102 d lacks the specialized storage adapter, the storage node 102 d is likely incapable of implementing the inline data-encryption.
  • the management node 112 can migrate the VSU 306 d _ 1 from the particular storage node 102 d to another storage node 102 a that has the specialized hardware 108 a .
  • the data requests 308 can be serviced by the other storage node 102 a using the specialized functionality moving forward, given the presence of the specialized hardware 108 a.
  • FIG. 3 is intended to be illustrative and non-limiting. Other examples may include more components, fewer components, different components, or a different arrangement of the components shown in FIG. 3 .
  • the distributed storage system 100 may have any number of storage nodes spread across any number of node pools.
  • FIG. 4 is a block diagram of another example of a distributed storage system 400 according to some aspects of the present disclosure.
  • the distributed storage system 400 includes a management node 112 and storage nodes 102 a - e with specialized hardware 108 a - c.
  • the management node 112 includes a processor 402 communicatively coupled with a memory 404 .
  • the processor 402 can include one processor or multiple processors. Non-limiting examples of the processor 402 include a Field-Programmable Gate Array (FPGA), an application-specific integrated circuit (ASIC), a microprocessor, etc.
  • the processor 402 can execute instructions 406 stored in the memory 404 to perform operations.
  • the instructions 406 can include processor-specific instructions generated by a compiler or an interpreter from code written in any suitable computer-programming language, such as C, C++, C#, etc.
  • the memory 404 can include one memory or multiple memories.
  • Non-limiting examples of the memory 404 can include electrically erasable and programmable read-only memory (EEPROM), flash memory, or any other type of non-volatile memory.
  • EEPROM electrically erasable and programmable read-only memory
  • flash memory or any other type of non-volatile memory.
  • At least some of the memory 404 includes a non-transitory computer-readable medium from which the processor 402 can read the instructions 406 .
  • the non-transitory computer-readable medium can include electronic, optical, magnetic, or other storage devices capable of providing the processor 402 with computer-readable instructions or other program code. Examples of the non-transitory computer-readable medium can include magnetic disks, memory chips, ROM, random-access memory (RAM), an ASIC, optical storage, or any other medium from which a computer processor can read the instructions 406 .
  • the processor 402 of the management node 112 can execute the instructions 406 to perform operations.
  • the processor 402 can determine a subset of storage nodes 102 a - c that include specialized hardware 108 a - c based on status information 410 received from a plurality of storage nodes 102 a - e of the distributed storage system 400 .
  • the processor 402 can receive the status information 410 from the plurality of storage nodes 102 a - e .
  • the status information 410 can indicate whether each storage node in the plurality of storage nodes 102 a - e has (e.g., is coupled to) a corresponding piece of specialized hardware.
  • the status information 410 can indicate that storage nodes 102 a - c have specialized hardware 108 a - c , and that storage nodes 102 d - e does not have the specialized hardware.
  • the processor 402 can determine a subset of storage nodes 102 a - c , from among the plurality of storage nodes 102 a - e , having the specialized hardware 108 a - c .
  • the processor 402 can then generate a node pool 122 that includes the subset of storage nodes 102 a - c .
  • the specialized functionality 408 a - c can be enabled on the subset of storage nodes 102 a - c .
  • the node pool 122 can be configured to perform the specialized functionality 408 a - c in relation to a data request 412 .
  • the processor 402 can implement some or all of the steps shown in FIG. 5 .
  • Other examples can include more steps, fewer steps, different steps, or a different order of the steps than is shown in FIG. 5 .
  • the steps of FIG. 5 are discussed below with reference to the components discussed above in relation to FIG. 4 .
  • a processor 402 determines a subset of storage nodes 102 a - c that include specialized hardware 108 a - c .
  • the specialized hardware 108 a - c can be preconfigured with specialized functionality 408 a - c for compressing data or encrypting data, in some examples.
  • the processor 402 can determine the subset of storage nodes 102 a - c based on status information 410 received from the plurality of storage nodes 102 a - e of a distributed storage system 400 .
  • the status information 410 can indicate whether each storage node in the plurality of storage nodes 102 a - e has a corresponding piece of specialized hardware.
  • the processor 402 generates a node pool 122 that includes the subset of storage nodes 102 a - c with the specialized hardware 108 a - c .
  • generating the node pool 122 may involve transmitting one or more commands to an application programming interface (API) of the distributed storage system 100 for causing the distributed storage system 100 to assign the storage nodes into the node pool 122 .
  • API application programming interface
  • the node pool 122 can perform the specialized functionality in relation to a data request 412 .
  • data request 412 can correspond to a particular VSU that is part of the node pool 122 .
  • the data request 412 can be serviced by the node pool 122 using the specialized functionality.

Abstract

Distributed storage nodes having specialized hardware can be pooled for servicing data requests. For example, a distributed storage system can include a group of storage nodes. The distributed storage system can determine a subset of storage nodes that include the specialized hardware based on status information received from the group of storage nodes. The specialized hardware can be preconfigured with specialized functionality. The distributed storage system can then generate a node pool that includes the subset of storage nodes with the specialized hardware. The node pool can be configured to perform the specialized functionality in relation to a data request.

Description

TECHNICAL FIELD
The present disclosure relates generally to distributed storage systems. More specifically, but not by way of limitation, this disclosure relates to pooling storage nodes of a distributed storage system that have specialized hardware, such as hardware with built-in compression or encryption capabilities.
BACKGROUND
Distributed storage systems can include storage nodes in communication with each other over a network for synchronizing, coordinating, and storing data. The storage nodes can work together so that the distributed storage system behaves as one storage system. Distributed storage systems can implement block storage, file storage, or object storage techniques.
There are numerous advantages to using distributed storage systems, such as improved scalability, redundancy, and performance. In particular, distributed storage systems can be easily scaled horizontally, in the sense that they can combine many storage nodes into a single, shared storage system. Distributed storage systems can also store many copies of the same data for high availability, backup, and disaster recovery purposes. Additionally, some distributed storage systems can execute compute workloads on the same storage nodes that are also used to store data, thereby yielding a hyper-converged infrastructure that is highly efficient.
BRIEF DESCRIPTION OF THE DRAWINGS
FIG. 1 is a block diagram of an example of a distributed storage system according to some aspects of the present disclosure.
FIG. 2 is a sequence diagram of an example of a process for enabling specialized functionality on a storage node according to some aspects of the present disclosure.
FIG. 3 is a block diagram of an example of data migration in a distributed storage system according to some aspects of the present disclosure.
FIG. 4 is a block diagram of another example of a distributed storage system according to some aspects of the present disclosure.
FIG. 5 a flow chart of an example of a process for pooling distributed storage nodes according to some aspects of the present disclosure.
 DETAILED DESCRIPTION
A distributed storage system can include storage nodes formed from relatively heterogeneous servers in communication with each other over a network, such as a local area network. Some of the storage nodes may have specialized hardware. Specialized hardware is a piece of hardware that is preconfigured by a manufacturer to implement specialized functionality, prior to receipt of the specialized hardware by an end user. For example, the specialized hardware can include specialized circuitry configured to implement the specialized functionality. The specialized circuitry may include a custom or semi-custom integrated circuit with a special hardware design or firmware specifically tailored for performing the specialized functionality, where the specialized circuitry is built into the specialized hardware by a manufacturer during a manufacturing process. Thus, specialized hardware is different from generic hardware on which aftermarket software is installed by the end user to implement the specialized functionality. The specialized functionality can be functionality that is supplemental to the main purpose of the piece of hardware. By way of example, the specialized hardware can include a storage adapter, a storage device, an input/output (I/O) interface, or a processing device, where the specialized hardware has appropriate circuitry built-in to implement specialized functionality such as data encryption, data compression, and checksum calculation capabilities. As one particular example, the specialized hardware can be a self-encrypting hard drive with inline data-encryption capabilities. Using specialized hardware to perform the specialized functionality can be faster and more robust than executing aftermarket software on generic hardware.
While some storage nodes may have the specialized hardware, other storage nodes may not. In such situations, the distributed storage system may operate as if all of the storage nodes lack the specialized hardware, since distributed storage systems generally operate on the basis of the lowest common denominator. As a result, the distributed storage system may not implement the specialized functionality of the specialized hardware, even though the specialized functionality could improve the distributed storage system.
Some examples of the present disclosure can overcome one or more of the abovementioned problems by identifying storage nodes in a distributed storage system that have specialized hardware with specialized functionality, enabling the specialized functionality on the storage nodes, grouping the storage nodes together into a pool of storage nodes (“node pool”) that have the specialized functionality enabled, and using the node pool to utilize the specialized functionality in relation to data requests. The data requests can include read requests for reading data, write requests for writing data, or both. Grouping the storage nodes in this way can prevent the data requests from being divided up among some storage nodes that have the specialized hardware and other storage nodes that lack the specialized hardware, so that the data requests arrive at a node pool in which all of the storage node members have the specialized hardware or none of the storage node members have the specialized hardware. With the data requests arriving at a node pool in which all of the storage nodes have the specialized hardware, the distributed storage system can leverage the specialized functionality to achieve various improvements, such as improvements to data security and throughput.
One particular example can involve a distributed storage system, such as Ceph Storage by Red Hat®. The distributed storage system can include hundreds or thousands of storage nodes. Each storage node can determine if it has specialized hardware capable of performing specialized functionality, such as data encryption, data compression, or both. Each storage node may determine if it has the specialized hardware by scanning its hardware. For example, a storage node can analyze its hardware upon booting up to determine if the specialized hardware is connected. As another example, a storage node may periodically analyze its hardware at predefined intervals to determine if the specialized hardware is connected. As yet another example, a storage node may analyze its hardware in response to an event determine if the specialized hardware is connected. After scanning their hardware, the storage nodes can then transmit status communications indicating whether or not they have the specialized hardware capable of performing the specialized functionality.
A centralized management node of the distributed storage system can receive the status communications from the storage nodes. Based on the status communications, the management node can determine a subset of the storage nodes that have the specialized hardware. The management node may then transmit communications to the subset of storage nodes for causing the storage nodes to enable the specialized functionality. Alternatively, the storage nodes may automatically enable the specialized functionality upon discovering that they are connected to the specialized hardware. Either way, the specialized functionality can be enabled on the storage nodes.
Next, the management node can assign the storage nodes in the subset to the same node pool. As a result, the node pool may only contain storage nodes with specialized hardware capable of performing the specialized functionality. The node pool can then be used to service data requests, so that the specialized functionality can be performed in relation to the data requests. Such node pools may be considered higher-tiered pools with better performance or data protection, given their specialized functionality. Thus, a service provider that is selling access to the distributed storage system may charge higher fees for using the node pool than for other node pools, such as node pools that lack specialized hardware with the specialized functionality.
These illustrative examples are given to introduce the reader to the general subject matter discussed here and are not intended to limit the scope of the disclosed concepts. The following sections describe various additional features and examples with reference to the drawings in which like numerals indicate like elements but, like the illustrative examples, should not be used to limit the present disclosure.
FIG. 1 is a block diagram of an example of a distributed storage system 100 according to some aspects of the present disclosure. The distributed storage system 100 includes storage nodes 102 a-e. The storage nodes 102 a-e may be physical servers for storing data.
Some storage nodes 102 a-c can have specialized hardware 108 a-c, while other storage nodes 102 d-e may lack the specialized hardware. Examples of the specialized hardware 108 a-c can include a storage adapter for interacting with storage devices, a storage device such as a hard disk or solid-state drive, a processing device such as a central processing unit (CPU) or graphics processing unit (GPU), or an I/O interface such as a peripheral component interface (PCI) card. The specialized hardware 108 a-c may be internal to the storage nodes 102 a-c or external and coupled to the storage nodes 102 a-c. The specialized hardware 108 a-c is configured to implement specialized functionality. Examples of the specialized functionality can include data compression/decompression capabilities, data encryption/decryption capabilities, checksum calculation capabilities, or any combination of these. The specialized functionality may be selectively enabled and disabled.
In some examples, the storage nodes 102 a-e can each analyze their hardware to determine if they have a corresponding piece of specialized hardware. For example, storage nodes 102 a-c may analyze their hardware and determine that the specialized hardware 108 a-c is attached. And storage nodes 102 d-e may analyze their hardware and determine that they lack the specialized hardware. The storage nodes 102 a-e may analyze their hardware to determine if they have the specialized hardware in response to any suitable event. For example, the storage nodes 102 a-e may each analyze their hardware to detect the presence of a piece of specialized hardware in response to a boot up event. As another example, the storage nodes 102 a-e may each analyze their hardware to detect the presence of the specialized hardware in response to the passage of a predefined amount of time, such as one hour. As yet another example, the storage nodes 102 a-e may each analyze their hardware to detect the presence of the specialized hardware in response to a request 110 from a management node 112 of the distributed storage system 100. One example of such a request 110 is shown in FIG. 1 , but the management node 112 can transmit similar types of requests to many or all of the storage nodes 102 a-e.
After analyzing their hardware, the storage nodes 102 a-e can generate respective status information indicating whether or not they have the specialized hardware. Each of the storage nodes 102 a-e can then transmit a status communication that includes the respective status information over a network to the management node 112. The network may be a local area network or the Internet. One example of such a status communication 114 with status information 116 is shown in FIG. 1 , but many or all of the storage nodes 102 a-e may transmit similar types of status communications to the management node 112.
The management node 112 is configured to manage one or more aspects of the distributed storage system 100. For example, the management node 112 can generate node pools and manage which virtual storage units are mapped to which nodes. The management node 112 may also manage which storage nodes 102 a-e have write-back caching enabled or disabled, as described below.
In some examples, the management node 112 can receive the status communications from the storage nodes 102 a-e and determine a subset of storage nodes 102 a-c that have the specialized hardware 108 a-c based on the status information in the status communications. The specialized functionality of the specialized hardware 108 a-c can then be enabled on the subset of storage nodes 102 a-c. For example, the management node 112 can transmit signals, such as signal 118, to the subset of storage nodes 102 a-c for causing the storage nodes 102 a-c to enable the specialized functionality. Each signal can include a command or other information configured to cause a corresponding storage node to enable the specialized functionality. For instance, the storage nodes 102 a-c may be in a first state in which the specialized functionality is disabled. To enable the specialized functionality on the storage nodes 102 a-c, the management node 112 can transmit signals with activation commands to the storage nodes 102 a-c. The storage nodes 102 a-c can receive the signals, detect the activation commands in the signals, and responsively switch from the first state to a second state in which the specialized functionality is enabled. In other examples, at least some of the storage nodes 102 a-c may automatically enable the specialized functionality in response to determining that they have back the specialized hardware 108 a-c. In still other examples, a system administrator may manually enable the specialized functionality on at least some of the storage nodes 102 a-c based on determining that the storage nodes 102 a-c have specialized hardware 108 a-c. By using one or more of the above techniques, the specialized functionality can be enabled on the storage nodes 102 a-c.
With the specialized functionality engaged on the subset of storage nodes 102 a-c, the management node 112 can assign the subset of storage nodes 102 a-c to a node pool 122. A node pool can be a defined group of storage nodes configured to implement storage functionality to service one or more read/write requests. In this example, the node pool 122 only includes the storage nodes 102 a-c that have the specialized hardware 108 a-c. The node pool 122 may be designated as a higher-tiered pool, since it may have better performance characteristics than another node pool (e.g., formed from storage nodes 102 d-e) that lacks the specialized hardware or specialized functionality.
A user can obtain access to the node pool 122 for storing data. For example, a user may purchase a subscription to the node pool 122, allowing the user to store and retrieve data therefrom by submitting data requests. Upon the user being granted such access, the distributed storage system 100 can cause the node pool 122 to perform the specialized functionality in relation to the data requests submitted by the user. The specialized functionality may yield better performance or security than is otherwise possible absent the specialized functionality.
In some examples, storage nodes may have more than one type of specialized hardware or may have specialize hardware capable of implementing more than one type of specialized functionality. In some such examples, the management node 112 can group together storage nodes having the same amounts and/or types of specialized functionality into the same node pool. For example, the distributed storage system 100 can include a first set of storage nodes with specialized hardware for implementing both data encryption and data compression. The distributed storage system 100 can also include a second set of storage nodes with specialized hardware for implementing data encryption but not data compression. And the distributed storage system 100 include a third set of storage nodes with specialized hardware for implementing data compression but not data encryption. As a result, the first set of storage nodes may be grouped into a first node pool capable of implementing both data encryption and data compression. The second set of storage nodes may be grouped into a second node pool capable of implementing data encryption but not data compression. And the third set of storage nodes may be grouped into a third node pool capable of implementing data compression but not data encryption. A user may purchase a subscription to whichever of the node pools meets the user's needs, where the first node pool may be more expensive than the second node pool or the third node pool.
It will be appreciated that FIG. 1 is intended to be illustrative and non-limiting. Other examples may include more components, fewer components, different components, or a different arrangement of the components shown in FIG. 1 . For instance, although the distributed storage system 100 includes five storage nodes in the example of FIG. 1 , the distributed storage system 100 may have hundreds or thousands of storage nodes in other examples.
FIG. 2 is a sequence diagram of an example of a process for enabling specialized functionality on distributed storage nodes according to some aspects of the present disclosure. Although the example shown in FIG. 2 includes a certain sequence of steps, other examples may involve more steps, fewer steps, different steps, or a different order of the steps shown in FIG. 2 .
The process begins with a management node 112 of a distributed storage system transmitting a request for status information to a storage node 102. The storage node 102 can receive the request and responsively determine if specialized hardware is coupled to the storage node. In this example, the storage node 102 has determined that the specialized hardware is coupled to the storage node 102. Next, the storage node 102 can transmit a response to the request, where the response is in the form of a status communication with status information indicating that the storage node 102 is coupled to the specialized hardware. The management node 112 can receive the status communication and determine that the storage node 102 is coupled to the specialized hardware based on the status information. The management node 112 may then transmit a signal to the storage node 102 for causing the storage node 102 to enable the specialized functionality of the specialized hardware. The storage node 102 can receive the signal and responsively enable the specialized functionality.
In some examples, the distributed storage system can use the storage node 102 to service a data request. The data request may be a higher-priority data request for which the specialized functionality may be desirable. A data request may be higher-priority if it is more critical, demanding, or higher cost than other data requests. For example, a user may pay a premium for the user's data requests to be deemed higher priority.
FIG. 3 is a block diagram of an example of a migration process in a distributed storage system 100 according to some aspects of the present disclosure. In this example, the distributed storage system 100 includes a first node pool 302 for servicing data requests 308 from a client device, such as a laptop computer, desktop computer, server, or mobile device. The client device may be external to the distributed storage system 100. Within the first node pool 302 is a storage node 102 d that lacks specialized hardware. As a result, the storage node 102 d may be incapable of implementing the specialized functionality corresponding to the specialized hardware. The storage node 102 d may include any number of virtual storage units (VSU) 306 d. Virtual storage units can be logical devices that are mapped to physical storage devices for reading and writing data associated with data requests. The node locations and physical storage-device mappings of the VSUs in the distributed storage system 100 may be adjustable by the management node 112.
It may be desirable for the data requests 308 to be serviced by storage nodes that have the specialized functionality enabled, for example to obtain the performance or security improvements associated with the specialized functionality. For example, if the data requests 308 are higher priority, it may be desirable to service the data requests 308 using higher-performance storage nodes that have the specialized functionality enabled. To that end, the management node 112 can generate a second node pool 304 using the techniques described above, where the second node pool 304 includes storage nodes 102 a-b on which the specialized functionality is enabled. The specialized functionality may be enabled on the storage nodes 102 a-b based on the storage nodes 102 a-b having corresponding specialized hardware 108 a-b. The management node 112 can determine that VSU 306 d_1 corresponds to (e.g., is a destination for) the data requests 308, and then migrate a VSU 306 d_1 from storage node 102 d to storage node 102 a. This migration is represented in FIG. 1 by a dashed arrow. The management node 112 can also transmit a communication to the client device from which the data requests 308 originated, to notify the client device of the change in location of the VSU 306 d_1. As a result, the client device can direct future data requests 308 corresponding to VSU 306 d_1 to storage node 102 a, so that the data requests 308 can be serviced by the second node pool 304 using the specialized functionality.
As one particular example, the management node 112 can determine that the data requests 308 have a particular priority level, such as a high priority level associated with sensitive data. Different data requests 308 may have different priority levels assigned by a user or the system. The management node 112 can also determine a VSU 306 d_1 associated with the data requests 308 is located on a particular storage node 102 d of the distributed storage system 100. The management node 112 can communicate with the particular storage node 102 d to determine that the particular storage node 102 d lacks a specialized storage adapter with inline data-encryption capabilities. For example, the management node 112 can receive status information from the particular storage node 102 d indicating that the particular storage node 102 d lacks the specialized storage adapter. Since the particular storage node 102 d lacks the specialized storage adapter, the storage node 102 d is likely incapable of implementing the inline data-encryption. Based on determining that (i) the data requests 308 have the particular priority level and (ii) the particular storage node 102 d having the VSU 306 d_1 associated with the data requests 308 does not have the specialized hardware, the management node 112 can migrate the VSU 306 d_1 from the particular storage node 102 d to another storage node 102 a that has the specialized hardware 108 a. As a result, the data requests 308 can be serviced by the other storage node 102 a using the specialized functionality moving forward, given the presence of the specialized hardware 108 a.
It will be appreciated that FIG. 3 is intended to be illustrative and non-limiting. Other examples may include more components, fewer components, different components, or a different arrangement of the components shown in FIG. 3 . For instance, although the distributed storage system 100 includes three storage nodes in two node pools in the example of FIG. 3 , the distributed storage system 100 may have any number of storage nodes spread across any number of node pools.
FIG. 4 is a block diagram of another example of a distributed storage system 400 according to some aspects of the present disclosure. The distributed storage system 400 includes a management node 112 and storage nodes 102 a-e with specialized hardware 108 a-c.
In this example, the management node 112 includes a processor 402 communicatively coupled with a memory 404. The processor 402 can include one processor or multiple processors. Non-limiting examples of the processor 402 include a Field-Programmable Gate Array (FPGA), an application-specific integrated circuit (ASIC), a microprocessor, etc. The processor 402 can execute instructions 406 stored in the memory 404 to perform operations. The instructions 406 can include processor-specific instructions generated by a compiler or an interpreter from code written in any suitable computer-programming language, such as C, C++, C#, etc.
The memory 404 can include one memory or multiple memories. Non-limiting examples of the memory 404 can include electrically erasable and programmable read-only memory (EEPROM), flash memory, or any other type of non-volatile memory. At least some of the memory 404 includes a non-transitory computer-readable medium from which the processor 402 can read the instructions 406. The non-transitory computer-readable medium can include electronic, optical, magnetic, or other storage devices capable of providing the processor 402 with computer-readable instructions or other program code. Examples of the non-transitory computer-readable medium can include magnetic disks, memory chips, ROM, random-access memory (RAM), an ASIC, optical storage, or any other medium from which a computer processor can read the instructions 406.
The processor 402 of the management node 112 can execute the instructions 406 to perform operations. In particular, the processor 402 can determine a subset of storage nodes 102 a-c that include specialized hardware 108 a-c based on status information 410 received from a plurality of storage nodes 102 a-e of the distributed storage system 400. For example, the processor 402 can receive the status information 410 from the plurality of storage nodes 102 a-e. The status information 410 can indicate whether each storage node in the plurality of storage nodes 102 a-e has (e.g., is coupled to) a corresponding piece of specialized hardware. For example, the status information 410 can indicate that storage nodes 102 a-c have specialized hardware 108 a-c, and that storage nodes 102 d-e does not have the specialized hardware. Based on the status information 410, the processor 402 can determine a subset of storage nodes 102 a-c, from among the plurality of storage nodes 102 a-e, having the specialized hardware 108 a-c. The processor 402 can then generate a node pool 122 that includes the subset of storage nodes 102 a-c. The specialized functionality 408 a-c can be enabled on the subset of storage nodes 102 a-c. The node pool 122 can be configured to perform the specialized functionality 408 a-c in relation to a data request 412.
In some examples, the processor 402 can implement some or all of the steps shown in FIG. 5 . Other examples can include more steps, fewer steps, different steps, or a different order of the steps than is shown in FIG. 5 . The steps of FIG. 5 are discussed below with reference to the components discussed above in relation to FIG. 4 .
In block 502, a processor 402 determines a subset of storage nodes 102 a-c that include specialized hardware 108 a-c. The specialized hardware 108 a-c can be preconfigured with specialized functionality 408 a-c for compressing data or encrypting data, in some examples. The processor 402 can determine the subset of storage nodes 102 a-c based on status information 410 received from the plurality of storage nodes 102 a-e of a distributed storage system 400. The status information 410 can indicate whether each storage node in the plurality of storage nodes 102 a-e has a corresponding piece of specialized hardware.
In block 504, the processor 402 generates a node pool 122 that includes the subset of storage nodes 102 a-c with the specialized hardware 108 a-c. In some examples, generating the node pool 122 may involve transmitting one or more commands to an application programming interface (API) of the distributed storage system 100 for causing the distributed storage system 100 to assign the storage nodes into the node pool 122.
After the node pool 122 is generated, the node pool 122 can perform the specialized functionality in relation to a data request 412. For example, data request 412 can correspond to a particular VSU that is part of the node pool 122. As a result, the data request 412 can be serviced by the node pool 122 using the specialized functionality.
The foregoing description of certain examples, including illustrated examples, has been presented only for the purpose of illustration and description and is not intended to be exhaustive or to limit the disclosure to the precise forms disclosed. Numerous modifications, adaptations, and uses thereof will be apparent to those skilled in the art without departing from the scope of the disclosure. For instance, any examples described herein can be combined with any other examples to yield further examples.

Claims (20)

The invention claimed is:
1. A management node of a distributed storage system, the management node comprising:
a processor; and
a memory including instructions that are executable by the processor for causing the processor to:
receive status information from a plurality of storage nodes of the distributed storage system, the plurality of storage nodes being separate from the management node, and the status information indicating which storage nodes of the plurality of storage nodes include specialized hardware that is preconfigured with specialized functionality for compressing data or encrypting data;
identify, based on the status information, a subset of storage nodes that include the specialized hardware from among the plurality of storage nodes;
in response to identifying the subset of storage nodes that include the specialized hardware, group the subset of storage nodes together into a node pool that includes the subset of storage nodes with the specialized hardware, the node pool being configured to perform the specialized functionality in relation to a data request;
determine that a particular data request has a particular priority level;
determine that a particular virtual storage unit (VSU) associated with the particular data request is located on a particular storage node of the distributed storage system;
determine, based on the status information, that the particular storage node does not have the specialized hardware; and
based on determining that the particular data request has the particular priority level and that the particular storage node does not have the specialized hardware, migrate the particular VSU from the particular storage node to another storage node of the distributed storage system that has the specialized hardware.
2. The management node of claim 1, wherein each storage node in the plurality of storage nodes is configured to determine whether the storage node includes a respective piece of specialized hardware and responsively transmit a status communication to the management node, the status communication including respective status information indicating whether the storage node includes the respective piece of specialized hardware.
3. The management node of claim 1, wherein the node pool consists only of the storage nodes in the subset.
4. The management node of claim 1, wherein the memory further includes instructions that are executable by the processor for causing the processor to transmit signals to the subset of storage nodes for causing each storage node in the subset of storage nodes to enable the specialized functionality.
5. The management node of claim 4, wherein the signals are configured to cause each storage node in the subset of storage nodes to switch from (i) a first state in which the specialized functionality is disabled to (ii) a second state in which the specialized functionality is enabled.
6. The management node of claim 1, wherein the specialized hardware is manufactured with specialized circuitry for implementing the specialized functionality, prior to the specialized hardware being incorporated into the distributed storage system.
7. A method comprising:
receiving, by a processing device of a management node of a distributed storage system, status information from a plurality of storage nodes of the distributed storage system, the plurality of storage nodes being separate from the management node, and the status information indicating which storage nodes of the plurality of storage nodes include specialized hardware that is preconfigured with specialized functionality for compressing data or encrypting data;
identifying, by the processing device and based on the status information, a subset of storage nodes that include the specialized hardware from among the plurality of storage nodes;
in response to identifying the subset of storage nodes that include the specialized hardware, grouping, by the processing device, the subset of storage nodes together to form a node pool that includes the subset of storage nodes with the specialized hardware, the node pool being configured to perform the specialized functionality in relation to a data request;
determining, by the processing device, that a particular data request has a particular priority level;
determining, by the processing device, that a particular virtual storage unit (VSU) associated with the particular data request is located on a particular storage node of the distributed storage system;
determining, by the processing device and based on the status information, that the particular storage node does not have the specialized hardware; and
based on determining that the particular data request has the particular priority level and that the particular storage node does not have the specialized hardware, migrating, by the processing device, the particular VSU from the particular storage node to another storage node of the distributed storage system that has the specialized hardware.
8. The method of claim 7, wherein each storage node in the plurality of storage nodes is configured to determine whether the storage node includes a respective piece of specialized hardware and responsively transmit a status communication, the status communication including respective status information indicating whether the storage node includes the respective piece of specialized hardware.
9. The method of claim 7, wherein the node pool consists only of the storage nodes in the subset.
10. The method of claim 7, further comprising transmitting signals to the subset of storage nodes for causing each storage node in the subset of storage nodes to enable the specialized functionality.
11. The method of claim 7, wherein the specialized hardware includes a storage adapter, a storage device, an input/output (I/O) interface, or a processing device, and wherein the specialized hardware has built-in circuitry configured for implementing data encryption or data compression.
12. The method of claim 7, further comprising:
determining, based on the status information, that a storage node of the plurality of storage nodes does not have the specialized hardware; and
excluding the storage node from the node pool based on determining that the storage node does not have the specialized hardware.
13. A non-transitory computer-readable medium comprising program code that is executable by a processor for causing the processor to:
receive status information from a plurality of storage nodes of a distributed storage system, the status information indicating which storage nodes of the plurality of storage nodes include specialized hardware that is preconfigured with specialized functionality for compressing data or encrypting data;
identify, based on the status information, a subset of storage nodes that include the specialized hardware from among the plurality of storage nodes;
in response to identifying the subset of storage nodes that include the specialized hardware, group the subset of storage nodes together to form a node pool that includes the subset of storage nodes with the specialized hardware, the node pool being configured to perform the specialized functionality in relation to a data request;
determine that a particular data request has a particular priority level;
determine that a particular virtual storage unit (VSU) associated with the particular data request is located on a particular storage node of the distributed storage system;
determine, based on the status information, that the particular storage node does not have the specialized hardware; and
based on determining that (i) the particular data request has the particular priority level and (ii) the particular storage node does not have the specialized hardware, migrate the particular VSU from the particular storage node to another storage node of the distributed storage system that has the specialized hardware.
14. The non-transitory computer-readable medium of claim 13, wherein each storage node in the plurality of storage nodes is configured to determine whether the storage node includes a respective piece of specialized hardware and responsively transmit a status communication, the status communication including respective status information indicating whether the storage node includes the respective piece of specialized hardware.
15. The non-transitory computer-readable medium of claim 13, further comprising program code that is executable by the processor for causing the processor to transmit signals to the subset of storage nodes for causing each of storage node in the subset of storage nodes to enable the specialized functionality.
16. The non-transitory computer-readable medium of claim 13, further comprising program code that is executable by the processor for causing the processor to:
determine, based on the status information, that a storage node of the plurality of storage nodes does not have the specialized hardware; and
exclude the storage node from the node pool based on determining that the storage node does not have the specialized hardware, wherein the node pool consists only of storage nodes having the specialized functionality enabled.
17. The management node of claim 1, wherein each storage node in the plurality of storage nodes is a server.
18. The management node of claim 1, wherein the memory further includes instructions that are executable by the processor for causing the processor to group the subset of storage nodes together into the node pool by transmitting one or more commands to an application programming interface associated with the distributed storage system.
19. The method of claim 7, wherein grouping the subset of storage nodes together into the node pool comprises transmitting one or more commands to an application programming interface associated with the distributed storage system.
20. The non-transitory computer-readable medium of claim 13, further comprising program code that is executable by the processor for causing the processor to:
group the subset of storage nodes together into the node pool by transmitting one or more commands to an application programming interface associated with the distributed storage system.
US17/078,504 2020-10-23 2020-10-23 Pooling distributed storage nodes that have specialized hardware Active 2041-01-21 US11579781B2 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US17/078,504 US11579781B2 (en) 2020-10-23 2020-10-23 Pooling distributed storage nodes that have specialized hardware

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
US17/078,504 US11579781B2 (en) 2020-10-23 2020-10-23 Pooling distributed storage nodes that have specialized hardware

Publications (2)

Publication Number Publication Date
US20220129170A1 US20220129170A1 (en) 2022-04-28
US11579781B2 true US11579781B2 (en) 2023-02-14

Family

ID=81258319

Family Applications (1)

Application Number Title Priority Date Filing Date
US17/078,504 Active 2041-01-21 US11579781B2 (en) 2020-10-23 2020-10-23 Pooling distributed storage nodes that have specialized hardware

Country Status (1)

Country Link
US (1) US11579781B2 (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114077578A (en) * 2020-08-12 2022-02-22 富泰华工业（深圳）有限公司 Distributed storage method, server and storage medium

Citations (26)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20040117621A1 (en) 2002-12-12 2004-06-17 Knight Erik A. System and method for managing resource sharing between computer nodes of a network
US20050220305A1 (en) * 2004-04-06 2005-10-06 Kazuhisa Fujimoto Storage system executing encryption and decryption processing
US7277431B2 (en) 2002-10-31 2007-10-02 Brocade Communications Systems, Inc. Method and apparatus for encryption or compression devices inside a storage area network fabric
US20080126813A1 (en) * 2006-09-21 2008-05-29 Hitachi, Ltd. Storage control device and method of controlling encryption function of storage control device
US20100031062A1 (en) * 2008-07-29 2010-02-04 Hitachi, Ltd. Storage Device and Data Processing Method of Storage Device
US7860255B2 (en) * 2003-12-17 2010-12-28 Panasonic Corporation Content distribution server, key assignment method, content output apparatus, and key issuing center
US20100329256A1 (en) * 2009-06-26 2010-12-30 Srinivasa Aditya Akella Architecture and system for coordinated network-wide redundancy elimination
US7877601B2 (en) * 2004-11-23 2011-01-25 Cisco Technology, Inc. Method and system for including security information with a packet
US8041941B2 (en) 2002-10-31 2011-10-18 Brocade Communications Systems, Inc. Method and apparatus for compression of data on storage units using devices inside a storage area network fabric
US8166314B1 (en) * 2008-12-30 2012-04-24 Emc Corporation Selective I/O to logical unit when encrypted, but key is not available or when encryption status is unknown
US8434087B2 (en) 2008-08-29 2013-04-30 International Business Machines Corporation Distributed acceleration devices management for streams processing
US20130151770A1 (en) * 2011-12-08 2013-06-13 Hitachi, Ltd. Remote copy system and remote copy control method
US9448846B2 (en) 2011-12-13 2016-09-20 International Business Machines Corporation Dynamically configurable hardware queues for dispatching jobs to a plurality of hardware acceleration engines
US20160344711A1 (en) * 2015-05-20 2016-11-24 Alcatel-Lucent Canada, Inc. Policy based cryptographic key distribution for network group encryption
CN106341417A (en) 2016-09-30 2017-01-18 贵州白山云科技有限公司 Content delivery network-based HTTPS acceleration method and system
US20170017550A1 (en) * 2014-07-31 2017-01-19 Hitachi, Ltd. Storage system
US20170180247A1 (en) * 2014-09-05 2017-06-22 Huawei Technologies Co., Ltd. Method and apparatus for compressing label stack, and method and apparatus for decompressing label stack
US9787522B1 (en) * 2011-06-29 2017-10-10 EMC IP Holding Company LLC Data processing system having failover between hardware and software encryption of storage data
US10133505B1 (en) * 2016-09-29 2018-11-20 EMC IP Holding Company LLC Cooperative host and data storage system services for compression and encryption
US10454498B1 (en) * 2018-10-18 2019-10-22 Pure Storage, Inc. Fully pipelined hardware engine design for fast and efficient inline lossless data compression
US10489215B1 (en) * 2016-11-02 2019-11-26 Nutanix, Inc. Long-range distributed resource planning using workload modeling in hyperconverged computing clusters
US20210036714A1 (en) * 2019-08-01 2021-02-04 EMC IP Holding Company LLC Techniques for determining compression tiers and using collected compression hints
US11100267B1 (en) * 2020-05-05 2021-08-24 Xilinx, Inc. Multi dimensional memory compression using bytewide write enable
US20210263759A1 (en) * 2020-02-26 2021-08-26 Red Hat, Inc. Encryption and remote attestation of containers
US20210281643A1 (en) * 2020-03-09 2021-09-09 International Business Machines Corporation Method for a network of storage devices
US20210374568A1 (en) * 2020-05-29 2021-12-02 EMC IP Holding Company LLC System and method for prioritizing and preventing backup failures

Patent Citations (26)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7277431B2 (en) 2002-10-31 2007-10-02 Brocade Communications Systems, Inc. Method and apparatus for encryption or compression devices inside a storage area network fabric
US8041941B2 (en) 2002-10-31 2011-10-18 Brocade Communications Systems, Inc. Method and apparatus for compression of data on storage units using devices inside a storage area network fabric
US20040117621A1 (en) 2002-12-12 2004-06-17 Knight Erik A. System and method for managing resource sharing between computer nodes of a network
US7860255B2 (en) * 2003-12-17 2010-12-28 Panasonic Corporation Content distribution server, key assignment method, content output apparatus, and key issuing center
US20050220305A1 (en) * 2004-04-06 2005-10-06 Kazuhisa Fujimoto Storage system executing encryption and decryption processing
US7877601B2 (en) * 2004-11-23 2011-01-25 Cisco Technology, Inc. Method and system for including security information with a packet
US20080126813A1 (en) * 2006-09-21 2008-05-29 Hitachi, Ltd. Storage control device and method of controlling encryption function of storage control device
US20100031062A1 (en) * 2008-07-29 2010-02-04 Hitachi, Ltd. Storage Device and Data Processing Method of Storage Device
US8434087B2 (en) 2008-08-29 2013-04-30 International Business Machines Corporation Distributed acceleration devices management for streams processing
US8166314B1 (en) * 2008-12-30 2012-04-24 Emc Corporation Selective I/O to logical unit when encrypted, but key is not available or when encryption status is unknown
US20100329256A1 (en) * 2009-06-26 2010-12-30 Srinivasa Aditya Akella Architecture and system for coordinated network-wide redundancy elimination
US9787522B1 (en) * 2011-06-29 2017-10-10 EMC IP Holding Company LLC Data processing system having failover between hardware and software encryption of storage data
US20130151770A1 (en) * 2011-12-08 2013-06-13 Hitachi, Ltd. Remote copy system and remote copy control method
US9448846B2 (en) 2011-12-13 2016-09-20 International Business Machines Corporation Dynamically configurable hardware queues for dispatching jobs to a plurality of hardware acceleration engines
US20170017550A1 (en) * 2014-07-31 2017-01-19 Hitachi, Ltd. Storage system
US20170180247A1 (en) * 2014-09-05 2017-06-22 Huawei Technologies Co., Ltd. Method and apparatus for compressing label stack, and method and apparatus for decompressing label stack
US20160344711A1 (en) * 2015-05-20 2016-11-24 Alcatel-Lucent Canada, Inc. Policy based cryptographic key distribution for network group encryption
US10133505B1 (en) * 2016-09-29 2018-11-20 EMC IP Holding Company LLC Cooperative host and data storage system services for compression and encryption
CN106341417A (en) 2016-09-30 2017-01-18 贵州白山云科技有限公司 Content delivery network-based HTTPS acceleration method and system
US10489215B1 (en) * 2016-11-02 2019-11-26 Nutanix, Inc. Long-range distributed resource planning using workload modeling in hyperconverged computing clusters
US10454498B1 (en) * 2018-10-18 2019-10-22 Pure Storage, Inc. Fully pipelined hardware engine design for fast and efficient inline lossless data compression
US20210036714A1 (en) * 2019-08-01 2021-02-04 EMC IP Holding Company LLC Techniques for determining compression tiers and using collected compression hints
US20210263759A1 (en) * 2020-02-26 2021-08-26 Red Hat, Inc. Encryption and remote attestation of containers
US20210281643A1 (en) * 2020-03-09 2021-09-09 International Business Machines Corporation Method for a network of storage devices
US11100267B1 (en) * 2020-05-05 2021-08-24 Xilinx, Inc. Multi dimensional memory compression using bytewide write enable
US20210374568A1 (en) * 2020-05-29 2021-12-02 EMC IP Holding Company LLC System and method for prioritizing and preventing backup failures

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
Andrei, P.S., et al., "Hardware Acceleration in CEPH Distributed File System," IEEE 12th International Symposium on Parallel and Distributed Computing, 2013, https://ieeexplore.ieee.org/abstract/document/6663583.

Also Published As

Publication number Publication date
US20220129170A1 (en) 2022-04-28

Similar Documents

Publication Publication Date Title
US10102153B2 (en) System and method for intercept of UEFI block I/O protocol services for BIOS based hard drive encryption support
CN107949842B (en) Virtual file system supporting multi-tier storage
US8959307B1 (en) Reduced latency memory read transactions in storage devices
US8131872B2 (en) Affinity-based transaction processing
US10146718B2 (en) Mechanism to boot multiple hosts from a shared PCIe device
US11416166B2 (en) Distributed function processing with estimate-based scheduler
CN108228084B (en) Method and apparatus for managing storage system
CN116204456A (en) Data access method and computing device
CN108475201B (en) Data acquisition method in virtual machine starting process and cloud computing system
US11579781B2 (en) Pooling distributed storage nodes that have specialized hardware
US20110072433A1 (en) Method to Automatically ReDirect SRB Routines to a ZIIP Eligible Enclave
US10956173B2 (en) Capturing pre-fetch blocks of an operating system to improve boot performance in a cloud environment
US9477518B1 (en) Method to automatically redirect SRB routines to a zIIP eligible enclave
US11347512B1 (en) Substitution through protocol to protocol translation
US10552067B2 (en) Method and system for delivering message in storage system
US11347520B2 (en) Unavailable memory device initialization system
US11397678B2 (en) Pooling distributed storage nodes that have backup power supplies and write-back caching capabilities
US10261722B2 (en) Performing caching utilizing dispersed system buffers
US10795824B2 (en) Speculative data return concurrent to an exclusive invalidate request
US20190339903A1 (en) Drive utilization in multi-tiered systems with read-intensive flash
US8688643B1 (en) Systems and methods for adaptively preferring mirrors for read operations
US11334261B2 (en) Scalable raid storage controller device system
US11500683B2 (en) Workload compliance governor system
US20240028477A1 (en) Systems and methods for backing up clustered and non-clustered data
CN114860350B (en) Data processing method and device based on cloud diskless tree-like mirror image

Legal Events

Date Code Title Description
AS Assignment

Owner name: RED HAT, INC., NORTH CAROLINA

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:BENHANOKH, GABRIEL ZVI;NIVER, BRETT;REEL/FRAME:054149/0852

Effective date: 20201022

FEPP Fee payment procedure

Free format text: ENTITY STATUS SET TO UNDISCOUNTED (ORIGINAL EVENT CODE: BIG.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

STPP Information on status: patent application and granting procedure in general

Free format text: NON FINAL ACTION MAILED

STPP Information on status: patent application and granting procedure in general

Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER

STPP Information on status: patent application and granting procedure in general

Free format text: FINAL REJECTION MAILED

STPP Information on status: patent application and granting procedure in general

Free format text: ADVISORY ACTION MAILED

STPP Information on status: patent application and granting procedure in general

Free format text: RESPONSE AFTER FINAL ACTION FORWARDED TO EXAMINER

STPP Information on status: patent application and granting procedure in general

Free format text: NOTICE OF ALLOWANCE MAILED -- APPLICATION RECEIVED IN OFFICE OF PUBLICATIONS

STCF Information on status: patent grant

Free format text: PATENTED CASE