JP7370801B2 - System that supports erasure code data protection function with embedded PCIe switch inside FPGA + SSD - Google Patents

Description

TECHNICAL FIELD The present invention relates to computer systems, and more particularly, to erasure coding within PCIe switches.

Currently, most NVMe (Non-Volatile Memory Express)-based SSDs (Solid State Drives) with RAID (Redundant Array of Independent Disks) protection function are heral Component Interconnect Express) AIC (Add-In-Card ). To optimize the bus bandwidth between the host CPU and the AIC RAID controller, the bus typically supports X16 PCIe lanes. However, due to the physical limitations of the standard form factor of PCIe cards, few U.S. Only 2 connectors are supported by each AIC RAID controller. That is, usually two or four U.S. 2 connectors.

To support up to 24 NVMe SSDs in a 2U chassis, six AIC RAID controllers are required, so six different RAID domains are required. This configuration adds cost and complexity to managing six RAID domains. Note that each AIC RAID controller currently costs approximately $400. Therefore, before considering the cost of NVMe SSD, the entire RAID solution in a single 2U chassis is over $2400 only for the AIC RAID controller.

The lack of cost-effective RAID data protection for large datasets has limited the adoption of NVMe SSDs in large business markets. Software RAID solutions can be used with relatively small data sets, but not with big data.

There are also other problems when using AIC RAID controllers.

1) As mentioned above, having multiple RAID domains within a chassis increases management complexity.

2) As a result of the complexity of RAID domain management, the chassis does not have a single RAID domain (which is preferred).

3) The CPU (Central Processing Unit) needs to support a large number of PCIe lanes. That is, (16 PCIe lanes per AIC RAID controller) x (6 AIC RAID controllers per chassis) = (96 PCIe lanes only for AIC RAID controllers). Only advanced and much more expensive CPUs currently support such a large number of PCIe lanes.

4) Each AIC RAID controller potentially consumes 25 watts, so 6 AIC RAID controllers increases power consumption to 150 watts per chassis.

5) Chassis tend to have only a few PCIe slots, potentially limiting the number of AIC RAID controllers that can be added and indirectly reducing the number of NVMe SSDs eligible for RAID protection from the chassis.

6) Software RAID solutions tend to support a relatively small number of RAID levels and tend to increase overhead on the CPU.

7) When used over a network, SSD access can be slow due to the time required to transmit the data access over the network. Note that in some examples, network storage may require implementation of software RAID, increasing overhead on the CPU.

What is needed is a way to utilize a large number of storage devices to support erasure coding without the limitations imposed by AIC RAID controllers and software RAID solutions.

US Registered Patent No. 9223734B2 US Registered Patent No. 9298648B2 US Published Patent No. 20170052916A1 US Published Patent No. 20170322898A1

The present invention is intended to solve the above-mentioned technical problem, and an object of the present invention is to provide a system that protects data using erasure codes.

A system according to an embodiment of the present invention includes a non-volatile memory express (NVMe) solid state drive (SSD), an FPGA that implements one or more functions supporting the NVMe SSD, and a peripheral component interface (PCIe). rconnect Express) switch Equipped with One or more features that support NVMe SSDs are taken from a collection that includes data acceleration, data deduplication, data integrity, data encryption, and data compression. PCIe switches can communicate with FPGAs and NVMe SSDs.

A system according to another embodiment of the present invention includes an NVMe (Non-Volatile Memory Express) SSD (Solid State Drive), an FPGA (Field Programmable Gate Arra) including a first FPGA part, and a second FPGA part. y) and Provides a system with The first FPGA portion implements one or more functions that support NVMe SSDs. The second FPGA part implements a Peripheral Component Interconnect Express (PCIe) switch. The one or more features supporting NVMe SSD are taken from a set including data acceleration, data deduplication, data integrity, data encryption, and data compression. PCIe switches can communicate with FPGAs and NVMe SSDs. The FPGA and NVMe SSD can be inside a common housing.

A system according to another embodiment of the present invention includes a NVMe (Non-Volatile Memory Express) SSD (Solid State Drive) and a PCIe (Peripheral Component Interconnect Express) including erasure coding logic. ) switch. The PCIe switch includes an external connector that allows the PCIe switch to communicate with the processor, at least one connector that allows the PCIe switch to communicate with the NVMe SSD, a PPU (Power Processing Unit) that makes up the PCIe switch, and an external connector that allows the PCIe switch to communicate with the NVMe SSD. and an erasure coding controller including circuitry for applying an erasure coding scheme to the data.

According to embodiments of the present invention, using a PCIe switch that includes Look-Aside erasure coding logic can reduce the time required to move data because it moves erasure coding closer to the storage device. Furthermore, by co-locating the erasure coding controller with a PCIe switch, expensive RAID add-in cards are not required and larger arrays can be used across multiple chassis.

1 illustrates a machine including a PCIe switch with Look-Aside erasure coding logic in accordance with one embodiment of the present invention; FIG. 2 shows additional details of the machine of FIG. 1; FIG. 2 illustrates additional details of the machine of FIG. 1, including a midplane and switch board that couples a PCIe switch containing the Look-Aside erasure coding logic of FIG. 1 to a storage device; FIG. 4 is a diagram illustrating the storage device of FIG. 3 for achieving different erasure coding schemes; FIG. 2 is a diagram illustrating details of a PCIe switch including Look-Aside erasure coding logic of FIG. 1; FIG. FIG. 3 is a diagram illustrating details of a PCIe switch including look-through erasure coding logic in accordance with another embodiment of the present invention. FIG. 2 illustrates a first topology for using a PCIe switch including the look-aside erasure coding logic of FIG. 1 in accordance with an embodiment of the present invention. 2 illustrates a second topology for using a PCIe switch including the Look-Aside erasure coding logic of FIG. 1 in accordance with another embodiment of the invention; FIG. 2 illustrates a third topology for using a PCIe switch including the Look-Aside erasure coding logic of FIG. 1 in accordance with another embodiment of the invention; FIG. 2 illustrates a fourth topology for using a PCIe switch including the Look-Aside erasure coding logic of FIG. 1 in accordance with another embodiment of the present invention. FIG. 2 is a flowchart of an example procedure for a PCIe switch that includes the look-aside erasure coding logic of FIG. 1 to support erasure coding schemes, in accordance with one embodiment of the present invention. 2 is a flowchart of an example procedure for a PCIe switch that includes the look-aside erasure coding logic of FIG. 1 to support erasure coding schemes, in accordance with one embodiment of the present invention. 2 is a flowchart of an example procedure for a PCIe switch that includes the look-aside erasure coding logic of FIG. 1 to support erasure coding schemes, in accordance with one embodiment of the present invention. 2 is a flowchart of an example procedure for a PCIe switch that includes the look-aside erasure coding logic of FIG. 1 to support erasure coding schemes, in accordance with one embodiment of the present invention. 2 is a flowchart illustrating an example procedure for a PCIe switch including the look-aside erasure coding logic of FIG. 1 to perform initialization in accordance with an embodiment of the present invention. 2 is a flowchart illustrating an example procedure for a PCIe switch including the look-aside erasure coding logic of FIG. 1 to perform initialization in accordance with an embodiment of the present invention. 2 is a flowchart of an example procedure for a PCIe switch that includes the look-aside erasure coding logic of FIG. 1 to include a new storage device in an erasure coding scheme, in accordance with one embodiment of the present invention. 2 is a flowchart of an example procedure for a PCIe switch including the look-aside erasure coding logic of FIG. 1 to handle a failed storage device, in accordance with one embodiment of the present invention.

Reference will now be made to embodiments of the inventive idea, examples of which are illustrated in the accompanying drawings. In the detailed description that follows, various specific details are provided to facilitate a thorough understanding of the technical concept of the invention. However, a person having ordinary skill in this field can implement the technical idea of the present invention without such specific details. In other instances, well-known methods, procedures, components, circuits, and networks are not described in detail so as not to unnecessarily obscure the embodiments.

Although terms such as first and second are used herein to describe various components, it will be understood that these components are not limited by these terms. These terms are used to distinguish one component from another. For example, a first module can be named a second module, and likewise a second module can be named a first module without departing from the scope of the inventive concept.

The terms used in the description of the inventive concept are used only for the purpose of describing particular embodiments and do not limit the inventive concept. As used in the description of the technical concept of the invention and the appended claims, the singular expressions also include the plural expressions unless the context clearly dictates otherwise. The term "and/or" includes any and all possible combinations of one or more of the associated items. The words "comprise" and/or "comprising", when used in the detailed description, indicate the presence of the recited features, integers, steps, acts, elements and/or components. , does not exclude the presence or addition of one or more other features, integers, steps, acts, elements, components and/or groups thereof. The components in the drawings are not necessarily to scale.

FPGAs (Field Programmable Gate Arrays) have sufficient intelligence, computing resources, and high-speed input/output (I/O) connectivity to generate RAID (Redundant Array of Independent Disks)/erasure code parity and retrieve data when required. have sex. FPGA+SSD (Solid State Drives) is an embedded PCIe (P peripheral component interconnect express ) can request a switch. A large number of coprocessors also requires more channels of NAND flash memory.

Embodiments of the present invention support erasure code in PCIe switches internal to FPGAs. Embodiments of the present invention also allow users to configure the RAID engine remotely (within the FPGA) via Baseboard Management Controllers (BMCs).
These standard interfaces, such as PCIe (used in the control plane) or SMBus (System Management Bus), allow users to preconfigure RoC (RAID-on-a-Chip) and erasure code controllers. used for. The ability to configure storage devices in this manner is useful to users who lease computing resources. In other words, once storage is finished with use, users want to quickly destroy the data before the next user uses the same computing resources. In this case, the BMC can send a delete command to all embedded PCIe switches inside multiple FPGA+SSDs. When the RoC/erasure code controller of the FPGA receives the deletion command, it can delete all the data and parity data specified by the range of the command LBA (Logical Block Address).

Today, PCIe switches expose virtual switches or groupings where one or more switches are exposed to a manager. Such a configuration is useful in virtualized environments when networks, CPU-GPUs, FPGAs, and storage behind these virtual domains are grouped together. Such virtual groups, in one embodiment, generate RAID subgroups that are exposed to user groups for the virtualized environment or used for RAID groupings such as RAID 10, RAID 50, RAID 60, etc. This applies to storage. Such layered RAID groups create smaller groups and apply additional RAID layers to create larger RAID solutions. Virtual switches manage smaller RAID groups, while main switches manage the overall RAID configuration.

As data protection schemes are enabled and management is kept close to the storage unit, this solution offers benefits that are a major differentiator in enterprise and data-centric environments. Embodiments of the invention provide higher density and performance with lower power consumption.

This solution consists of an embedded single PCIe switch with an integrated RoC or erasure code controller in the data path between the host and the SSD. The PCIe switch+RoC components can be managed by the BMC for configuration and control, and interfaces for specific configurations can be exposed to software before provisioning to new users.

When operating in this erasure code/RAID mode, all NVMe (Non-Volatile Memory Express) or NVMe-oF (NVMe over Fabric) traffic exiting or entering the embedded PCIe switch is RoC Or it can be snooped by an erasure code controller (also referred to as Look-Aside RoC or erasure code controller). The RoC or erasure code controller can determine whether data in the traffic results in a cache hit in its local cache. If there is a cache hit, there is no need to propagate the transaction (read or write) to the appropriate SSD. The requested read data is served directly by the RoC's cache. Write data is updated directly to the RoC's local cache and can be displayed as "modified" or "dirty" data.

In the case of SSDs, parity is distributed among the concatenated SSDs. For example, if RAID 4 is selected, the last SSD is used to store only parity and the remaining SSDs are used to store data.

Virtual I/O addresses are supported by the presence of an external PCIe switch between the host and the SSD device. In this case, the primary RoC, which is part of the host PCIe switch, can virtualize the addresses of all SSDs. That is, addresses and devices are not visible to the host operating system (OS). In embodiments of the present invention, peer-to-peer transactions between at least two SSDs of peers are allowed and supported. This option can be striped across one or more SSDs to improve some form of SSD redundancy and/or availability. In this mode, the RoC or erasure code controller (if present) embedded within the FPGA is disabled. The only enabled RoC/erasure code controller is in the host PCIe switch.

When the storage device operates in single device mode, all incoming NVMe/PCIe traffic can be delivered to the SSD with the requested data.

When pairing mode is enabled, the RoC/Erasure Code Controller determines whether the address of the requested data belongs to its own BAR domain. In this case, the transaction is completed by the local RoC. For write transactions, a posted write buffer or write cache (using some embedded SRAM or DRAM) is used. If there is a write cache hit (a previous write occurred and the data is still stored in the write cache buffer), the handling depends on the write cache policy. For example, if the cache policy is write-back, the write command can be completed and terminated by the RoC cache. If the cache policy is continuous write-through, the write command can be completed if the write data is successfully transmitted to the drive. In this case, the RoC can finish the write command to the host as soon as the write data is successfully updated to the local cache.

The RoC can virtualize a large number of devices that the RoC requires, represented as a single device or a smaller number of devices, protecting the large number of devices against data and device failures. Data protection schemes can be naturally distributed into groups so that if there is a device with data loss, the data can be reconstructed from other devices. RAID and erasure coding EC are commonly adopted data protection methods that use distributed algorithms to protect against these losses.

To virtualize a device with RoC, the device can be terminated with RoC and may not be visible to the host. That is, the PCIe switch is coupled to all known devices and the RoC is coupled to the switch. To manage devices, RoC discovers and configures individual devices via PCIe switches. The RoC is also passed through in basic/factory mode and the host software configures the RoC. The host software is specifically designed to work with the PCIe switch + RoC hardware. Once configured, the RoC terminates the device and makes it invisible to the host.

The PCIe switch+RoC can be configured in various ways for RAID and EC modes. It may have additional PCIe switches downstream to support more devices and create larger fan-out configurations. Additionally, combinations of one or more of these hardware may be associated together to form larger configurations. For example, two PCIe switches + RoC can work together to form an alternative configuration. Alternatively, such two PCIe switches + RoC operate separately.

If the PCIe switch+RoC operates separately, each RoC and PCIe switch combination is instantiated into a separate device by the host. The host here has standard OS drivers that can see all SSDs virtualized by RoC. For example, let's assume 6 SSDs are installed under a PCIe switch and 1 SSD is exposed to the host by RoC. The second RoC and PCIe switch combination can expose a similar configuration to the host. Two SSDs are discovered by the host for every RoC controller device (one for each). Each RoC controller can expose a separate device space for each exposed SSD. This exposed SSD and all supporting devices behind it are not visible to the host. RoC manages hardware input/output paths via PCIe switches.

This method is used in an active-passive setting. Here, the second controller is a backup path in case the first controller path fails. The host now actively uses only the first controller, and no input/output (I/O) is transmitted to the second RoC controller. When an active-passive configuration is used, the two RoC controllers can internally replicate data. This is done by the first active controller transmitting all writes to the second RoC controller, as in a RAID1 data protection setup.

The second RoC and PCIe switch may have a second active-passive configuration with no SSD of its own behind it, just a path to a backup controller. In this case, since the two RoC controllers refer to the same set of SSDs, input/output (I/O) may not be transmitted between the two RoC controllers. This is a standard active-passive setup.

The SSDs behind each RoC may also be uncoordinated with each other, in which case the two SSDs are treated as separate SSDs with no protection shared between them.

Another use is for both paths to be used in an all-active-active configuration. This setting is used for load balancing purposes. Here, the host uses all of both paths in such a way that special software layers are used to distribute the I/O workload. The two RoC controllers coordinate writes between them to keep both of the SSDs in a synchronized state. That is, each SSD of each RoC controller contains the same data as in the RAID1 setting.

In another configuration, two RoC controllers communicate in a manner that distributes their inputs and outputs with a custom configuration. Here, only one RoC controller is used by the host. Other RoC controllers are coupled to the first RoC controller. The first RoC controller can expose one or more virtual NVMe SSDs to the host. Two RoCs are configured to divide the odd and even LBA space between them. Since NVMe uses a pull model for data from the device side, only commands are transmitted by the host to the SSD exposed by the first RoC controller. The RoC controller transmits a copy of the message to the second RoC controller via its side channel connection. The RoC controller is configured to service only odd or even LBAs, stripes, areas, etc. This configuration provides internal load balancing that does not need to be managed from the host and is managed transparently by a combination of RoC and PCIe switches. A separate RoC controller can handle only odd or even LBA ranges and satisfy the host buffer requirements. Since the two RoC controllers have access to the host, they can fill in the data for their odd or even pair.

For example, the host can transmit a command to read four consecutive LBAs (0-3) to a first RoC controller and a copy to a second RoC controller. The first RoC controller then reads data for LBA (0, 2) from the first two SSDs on its PCIe switch, while the second RoC controller reads the data for LBA (0, 2) from the first two SSDs on its PCIe switch. Read data from LBAs (1 to 3) of the first two SSDs. The second RoC controller can then report to the first RoC controller that it has completed its operation, and the first RoC controller can then report to the host that the transaction is completed.

Odd/even LBA/stripe/area pairs can be applied for other load balancing applications.

Embodiments of the present invention may support failure, removal, and hot addition of SSDs. If an SSD malfunctions or is removed from its slot, the RoC in the PCIe switch needs to detect the condition. If the PCIe switch detects such a condition, the RoC initiates a reconfiguration operation for the failed or removed SSD. The RoC also handles all I/O work during the reconfiguration period by prioritizing data from the associated stripes.

There are at least two ways in which failure or removal of an SSD is reported to the RoC within a PCIe switch. In one embodiment of the present invention, all SSDs have a Present pin coupled to the BMC. If the SSD is removed from the chassis, the BMC detects the removal. The BMC reports the affected slot number to the RoC in the PCIe switch. Additionally, the BMC can periodically monitor the status of the SSD. When the BMC detects a fatal error condition reported by the SSD, the BMC may decide not to use the SSD. The BMC can then report the failed slot number to the RoC so that a new SSD can be reconfigured.

In other embodiments of the invention, a PCIe switch can support hot plug as long as all SSDs are connected via PCIe sideband signals and can detect certain error conditions. . A PCIe switch removes or adds an SSD, or detects that a PCIe link coupled to an SSD is no longer operational. In these error situations, the RoC in the PCIe switch can isolate the failed SSD, or the BMC can immediately initiate device reconfiguration by disabling power to the failed device. A failed SSD can be isolated.

When enabled, each U. The Presence (PRSNT#) pin on the 2 connector can indicate the presence of a new device in the chassis. The signals are coupled to a PCIe switch and/or a BMC. The RoC can properly configure the new drive into its existing domain with the current data protection policy.

All incoming traffic from the host is required to be communicated to the snooping P2P and address translation logic (physical to logical). During PCIe enumeration, all configuration cycles from all ports are required to be communicated to the snooping P2P logic. Depending on the selected operating mode, the operation of a PCIe switch with RoC is defined as follows.

The RoC is also placed in-line between the PCIe switch and the host processor. In such embodiments of the invention, the RoC is referred to as Look-Through RoC. When using Look-Through RoC, the RoC in which the PCIe switch acts as a general PCIe switch is disabled and becomes a re-timer for all ports. In this case, all upstream ports are connected as in normal usage.

When RoC is enabled, a small number of non-transparent bridge (NTB) ports are connected to the host. In this case, the RoC can virtualize the received address into a logical address based on the selected RAID or erasure coding level.

Regardless of whether the RoC is a Look-Aside RoC or a Look-Through RoC, all incoming read/write memory requests are sent to the RoC to determine cache hits or misses. is checked against the local cache. If there is a cache hit, the requested read data is served by the RoC local cache memory instead of the SSD. In case of a memory write hit, the write data is immediately updated to the cache memory. The same write data will be updated to the SSD later. This implementation can reduce the overall latency for memory writes and improve system performance.

If there is a cache miss, the RoC controller determines which SSD is a suitable drive to access the data.

To address a PCIe device, the PCIe device should be enabled to be mapped into the system's input/output port address space or memory mapped address space. The system's firmware, device driver, or operating system programs Base Address Registers (BARs) to inform the device of the address mapping by logging configuration commands to the PCIe controller. Because all PCIe devices are in an inactive state when the system is reconfigured, the PCIe device may not have an address assigned to the PCIe device so that the operating system or device driver can communicate with the PCIe device. be. The BIOS or operating system uses a per-slot IDSEL (Initialization Device Select) signal to select a PCIe slot (e.g., a first PCIe slot, a second PCIe slot, or a third PCIe slot on the motherboard) via the PCIe controller. ) can be addressed by region.

The PCI buses are enumerated because there is no direct way for the BIOS or operating system to determine which PCIe slot a device is installed in (or determine what function the device embodies). Ru. Bus enumeration is accomplished by attempting to read the vendor ID (VID) and device ID (DID) registers for each bus number and device number combination in the device's function 15. A device number different from the DID may simply be the serial number of the device on that bus. Additionally, after a new bridge is detected, a new bus number is defined and device enumeration is restarted from device number 0.

If no response is received from the device function 15, the bus master performs an abort and returns an all-bits-on value (FFFFFFFFFF in hexadecimal), which is an invalid VID/DID value. In this manner, the device driver knows that the specified combination of bus/device_number/function (B/D/F) does not exist. Therefore, if a read to function ID 0 of a given bus/device causes the master (initiator) to abort, the device driver can conclude that there is no active device on that bus (device is function ID 0). ). In this case, it is not necessary to read the remaining function numbers (1 to 7) because the remaining function numbers (1 to 7) do not exist.

If the read for the specified B/D/F combination for the vendor ID register is successful, the device driver knows that the device is present. The device driver can record all ones in the BAR and read the requested memory size of the device back into encoded form. The design implies that all address spaces are powers of two in size and naturally aligned.

At this point, the BIOS or operating system can program memory mapping and input/output port addresses into the configuration registers of the device's BAR. This address remains valid as long as the system is powered on. All these settings are lost when the power is turned off, and this procedure is repeated when the system is turned on again. This entire process is fully automated, so users do not have to passively configure newly added hardware by changing DIP switches on the card itself. This automatic device detection and address space allocation is implemented in a plug-and-play manner.

When a PCIe-to-PCIe bridge is discovered, the system assigns a non-zero bus number to the secondary PCI bus across the bridge and then enumerates the devices on that secondary bus. As more PCIe bridges are searched, the search continues recursively until all possible domain/bus/device combinations have been scanned.

The functionality of each non-bridge PCIe device can implement up to six BARs, and each BAR can respond to an input/output port and a different address in a memory-mapped address space. Each BAR describes an area.

The PCIe device also has an option ROM that contains driver code or configuration information.

BMC directly configures R,oC settings. The BMC has a hard-coded path or configurable settings to which specific data protection schemes are applied. The latter exposes the interface to this configuration as a BIOS option or additionally to software via a hardware exposed interface. A hard-coded scheme is configured within the BIOS firmware to provide options for enabling/disabling protection features.

To handle device failures, the BMC over the control path can detect when a drive is damaged or removed. Also, the BMC can predict that the device will be damaged soon through SMART (Self-Monitoring Analysis and Reporting Technology). In this case, the BMC reconfigures the RoC hardware to enable the failed scenario or alert the user about the situation. BMC is only allowed on the control path, not the data path. If a new drive is inserted, the BMC can intervene again and configure the new drive as part of the protected group or initiate a reconfiguration operation. The RoC hardware handles the actual reconfiguration and recovery path in this setting to provide the lowest possible performance impact while providing lower latency in the data access path.

FIG. 1 is a diagram illustrating a machine including a PCIe switch with look-aside erasure coding logic in accordance with one embodiment of the present invention. Machine 105 is illustrated in FIG. Machine 105 may include a processor 110. Processor 110 may be any of a variety of processors, such as, for example, an Intel Xeon, Celeron, Itanium, or Atom processor, an AMD Opteron processor, an ARM processor, and the like. FIG. 1 shows a single processor 110 in machine 105, which may include any number of processors, each being a single core or multi-core processor, in any desired combination. Can be mixed.

Machine 105 may also include memory 115 managed by memory controller 120. The memory 115 includes flash memory, DRAM (Dynamic Random Access Memory), SRAM (Static Random Access Memory), persistent RAM (Persistent Random Access Memory), and FRAM. (registered trademark) (Ferroelectric Random Access Memory), MRAM (Magnetoresistive Random Access Memory) The memory may be any variety of memory such as NVRAM (Non-Volatile Random Access Memory) such as NVRAM (Non-Volatile Random Access Memory). Memory 115 may also be any desired combination of different memory types.

Machine 105 may also include a PCIe switch 125 with look-aside erasure coding logic. PCIe switch 125 is any desired PCIe switch that supports look-aside erasure coding logic.

Machine 105 also includes a storage device 130 that is controlled by device driver 135. Storage device 130 may be any desired shape of storage device that can communicate with PCIe switch 125. For example, storage device 130 is an NVMe SSD.

Although FIG. 1 illustrates machine 105 as a server (either a standalone server or a rack server), embodiments of the invention include any desired type of machine 105 without limitation. For example, machine 105 may be replaced with a desktop or laptop computer, or any other machine that would benefit from embodiments of the present invention. Machines 105 also include specialized portable computing machines, tablet computers, smartphones, and other computing machines.

FIG. 2 shows additional details of the machine of FIG. 1; In FIG. 2, machine 105 typically includes one or more processors 110 that are used to coordinate the operation of the components of machine 105 and include a memory controller 120 and a clock 205. For example, processor 110 is coupled to memory 115 that includes random access memory (RAM), read-only memory (ROM), or other state storage medium. Processor 110 is also coupled to a network connector 210, for example an Ethernet connector or a wireless connector, and coupled to storage device 130. Processor 110 is also coupled to user interface 220 and bus 215 attached to input/output interface ports, which can be managed using input/output engine 225, among other configurations.

FIG. 3 is a diagram illustrating additional details of the machine 105 of FIG. 1, including a midplane and switch board that couples the PCIe switch 125 containing the Look-Aside erasure coding logic of FIG. 1 to a storage device. In FIG. 3, machine 105 includes a midplane 305 and switch boards 310, 315. Switchboards 310, 315 include PCIe switches 125, 320 and baseboard management controllers 325, 330, each containing look-aside erasure coding logic. (Switchboards 310, 315) may also include additional components not shown in FIG. That is, FIG. 3 focuses on the elements most suitable for embodiments of the invention).

In some embodiments of the invention, each of the PCIe switches 125, 320 including look-aside erasure coding logic supports up to a total of 96 PCIe lanes. U. Each U. 2 connectors support up to 4 PCIe lanes per device. Using two X4 lanes (one X4 lane for each communication direction), each PCIe switch supports a maximum of (96÷8=12) devices. Accordingly, FIG. 3 shows 12 storage devices 130-1 through 130-3 communicating with PCIe switch 125 containing look-aside erasure coding logic and 12 storage devices 130-1 through 130-3 communicating with PCIe switch 320 containing look-aside erasure coding logic. Storage devices 130-4 to 130-6 are illustrated. However, the number of storage devices communicating with a PCIe switch 125, 320 that includes Look-Aside erasure coding logic is dependent on the number of PCIe lanes provided by the PCIe switch 125, 320 that includes Look-Aside erasure coding logic and the storage device 130- Limited only by the number of PCIe lanes used by each of 1 to 130-6.

In some embodiments of the invention, PCIe switches 125, 320 that include look-aside erasure coding logic are implemented using custom circuitry. In other embodiments of the invention, the PCIe switch 125, 320 containing the Loo-Aside erasure coding logic is a suitably programmed Field Programmable Gate Array (FPGA) or Application-Specific Integrated Circuit (ASIC). Implemented using t) be done.

BMCs 325 and 330 are used to configure storage devices 130-1 to 130-6. For example, the BMC 325, 330 can initialize the storage devices 130-1 to 130-6 and delete any data existing on the storage devices 130-1 to 130-6. That is, when the storage devices 130-1 to 130-6 are added to the erasure coding method at startup, initialization and deletion of arbitrary data are performed. Alternatively, this functionality is supported by a processor (processor 110 in FIG. 1) (or by a local processor present but not shown on the switchboards 310, 315). The BMC 325, 330 (or a local processor not shown on the processor 110 or switch board 310, 315 of FIG. 1) also implements the look-aside erasure coding logic of the PCIe switch 125, 320, which also includes look-aside erasure coding logic. May be responsible for initial configuration.

FIG. 3 illustrates an example overall setup for data protection utilizing two PCIe switches 125, 320 that include Look-Aside erasure coding logic. BMCs 325, 330 can directly configure look-aside erasure coding logic. BMCs 325, 330 may have hard-coded paths or configurable settings to which specific data protection schemes are applied. The latter exposes the interface to this configuration as a BIOS (Basic Input/Output System) option or to additional software via a hardware exposed interface. Hard-coded methods can be configured within the BIOS firmware to provide options for enabling/disabling protection features.

In the event of a storage device failure, the BMC 325, 330 can detect when a storage device fails or is removed through the control path. Thereafter, the BMC 325, 330 reconfigures the Look-Aside erasure coding logic to enable failed scenarios. BMCs 325, 330 are coupled to a control path, not a data path. Similarly, when a new storage device is inserted, the BMC 325, 330 can intervene to configure the new storage device as part of an existing group or initiate a reconfiguration operation. Look-Aside erasure coding logic can handle the actual reconfiguration. A recovery path within this setup should ideally minimize the performance impact on data access, as well as reconstruct the data on the reconfigured storage device from the remaining storage device.

At this point, it is worth defining the term "erasure coding". Erasure coding is intended to describe all the methods required to encode data on multiple storage devices. At least two storage devices or at least two parts of a storage device (eg, a single shell or housing containing two or more NAND flash channels) are required for erasure coding. This is because if only one storage device is used, the data is stored using common (existing) data access techniques suitable for the storage device. In other words, erasure coding is a method of using storage devices more efficiently and/or providing data redundancy. , or any combination of these approaches.

RAID (Redundant Array of Independent Disks) indicates a subset of erasure coding. That is, RAID levels indicate specific implementations of various erasure coding schemes. However, there may be other erasure coding schemes defined beyond traditional RAID levels.

Implementing erasure coding (or RAID) often uses two or more physically partitioned storage devices. However, in some embodiments of the invention, a single shell or housing comprises multiple portions of a storage device that are treated as separate storage devices for erasure coding purposes. For example, a single NVMe SSD shell or housing contains multiple NAND flash channels. Each NAND Flash channel can be considered a separate storage device for erasure coding purposes, with data striping (or other encoding) across the various NAND Flash channels. This allows erasure coding to be implemented using a single storage device in some embodiments of the present invention. Moreover, the PCIe switch 125 containing the look-aside erasure coding logic may be configured with error correction code (either configured elsewhere on the PCIe switch 125 containing the look-aside erasure coding logic, or by additional logic) or a single storage device. can support other functions used in

FIG. 4 is a diagram illustrating storage devices 130-1 to 130-6 of FIG. 3 to achieve different erasure coding schemes. In FIG. 4, storage devices 130-1 to 130-6 are used in a RAID 0 (zero) configuration, as illustrated in erasure coding scheme 405. RAID0 stripes data across various storage devices. That is, data is divided into logical units appropriate for storage devices, and each logical unit is recorded on a different storage device, up to the number of storage devices in the array. After all the storage devices have one logical unit of data recorded on them, data continues to be recorded on the first storage device again.

RAID0 is more efficient than using a single storage device alone or an unorganized group of disks (such as JBOD (Just a Bunch of Disks) or JBOF (Just a Bunch of Flash)). also offers benefits. Because data is stored in multiple storage devices, data can be read or recorded more quickly, with each storage device operating in parallel. Therefore, for example, as shown in FIG. 4, by dividing the data among 12 storage devices 130-1 to 130-6, each storage device 130-1 to 130-6 can read the entire data. Alternatively, it is necessary to read and write only 1/12 of the total data faster than writing. The total capacity of an array is calculated as the number of storage devices in the array multiplied by the capacity of the smallest storage device in the array. Thus, in FIG. 4, the array includes 12 storage devices for data, so the total capacity of the array is 12 times the smallest storage capacity of the array.

The drawback of RAID0 is that it has a protection function even if the storage device fails. This means that if any storage device in the array fails, data will be lost. In fact, RAID0 can be more dangerous than JBOD or JBOF. This means that by striping data across many storage devices, if any individual storage device fails, all data is lost (whereas with JBOD or JBOF, files are generally Therefore, if a single storage device fails in a JBOD or JBOF configuration, some data may be lost, but not all data will necessarily be lost. isn't it.).

RAID0 is not technically a redundant array of independent disks because it does not include any redundancy. However, RAID0 is typically considered a RAID level, and RAID0 is certainly considered an erasure coding scheme.

Erasure coding scheme 410 illustrates RAID5, a common RAID scheme. In RAID 5, parity blocks are computed relative to data stored on other storage devices in that stripe. Therefore, in FIG. 4, since the RAID5 array includes a total of 12 storage devices, 11 storage devices are used as data drives, and one storage device is used as a parity drive (in RAID5, parity data is distributed across storage devices like any data, but not limited to parity drives (RAID4, which is not used much anymore, stores all parity information on a single drive). The total capacity of an array with n storage devices in the array is calculated as (n-1) times the capacity of the smallest storage device. Because each stripe includes a single parity block, erasure coding scheme 410 can tolerate the failure of at most one storage device and still have access to all data (the failed storage device data can be recovered using data on functional storage devices combined with parity blocks).

RAID5 provides less overall storage than RAID0, but provides protection against storage device failure. This is an important trade-off to make between RAID levels. Depending on overall storage capacity and relative importance of redundancy.

Other RAID levels not shown in FIG. 4 may also be used as erasure coding schemes. For example, RAID6 uses two storage devices and stores parity information, so it reduces the overall storage capacity to (n-2) times the capacity of the smallest storage device, while at the same time failing up to two storage devices. is allowed. A hybrid method is also possible. For example, RAID0+1, RAID1+0, RAID5+0, RAID6+0, and other RAID schemes are all possible, each offering a variety of overall storage capacities and tolerances for storage device failure. For example, five of the storage devices 130-1 to 130-6 are used to form one RAID5 array, and five or more storage devices 130-1 to 130-6 are used to form a second RAID5 array. Two such groups used to form and combined with the remaining two storage devices are used to form a larger RAID5 array. Alternatively, the storage devices 130-1 to 130-6 are divided into two groups, each group embodying a RAID0 array, and the two groups operate as an array larger than RAID1 (therefore, the RAID0+1 configuration ). It should be noted that RAID and erasure coding techniques use fixed codes or rotating codes, and the fixed code/parity driven driver notation is for illustrative purposes only.

Erasure coding scheme 415 represents a more general description that is applicable to all RAID levels and any desired erasure coding scheme. Given an array of storage devices 130-1 through 130-6, these storage devices are divided into two groups. That is, one group used to store data, and another group used to store code. The code may be parity information or any desired coding information that authorizes the restoration of data missing from a subset of data within a data group or some code within a coding group. As shown in FIG. 4, erasure coding scheme 415 includes up to X data storage devices and Y code storage devices. Given any combination of the X storage devices in the array, it is expected that data from all X data storage devices can be accessed and reconstructed. Thus, erasure coding scheme 415 can generally tolerate the failure of up to Y storage devices in the array and still be able to access all data stored in the array. In terms of capacity, the total capacity of erasure coding scheme 415 is X times the capacity of the smallest storage device.

Note that in the discussion above, the total capacity of any erasure coding scheme is described in relation to the "minimum storage device capacity." For some erasure coding schemes, the storage device has variable capacity and can still be fully utilized. However, some erasure coding schemes, such as RAID 0 or RAID 1, expect all storage devices to have the same capacity and will throw away all the capacity that larger storage devices contain. Therefore, the expression "smallest storage device capacity" should be understood as a relative term, and the total capacity provided by an array using any particular erasure coding scheme is greater than the formula above. there is a possibility.

Returning to FIG. 3, regardless of the particular erasure coding scheme used, the look-aside erasure coding logic of PCIe switches 125, 320 effectively removes new storage devices from physical storage devices 130-1 through 130-6. generate. This new storage device can be considered as a virtual storage device since the storage device presented by this erasure coding scheme does not physically exist. Since this virtual storage device uses the physical storage devices 130-1 to 130-6, the physical storage devices 130-1 to 130-6 must be hidden from the host. Finally, if the stored data is encoded and stored in a manner unknown to the host, the problem is that the host attempts to directly access blocks on the storage devices 130-1 to 130-6. Become.

To support the use of these virtual storage devices, PCIe switches 125 and/or 320 that include look-aside erasure coding logic can notify processor 110 of FIG. 1 of the capacity of the virtual storage devices. For example, if storage devices 130-1 through 130-6 include five NVMe SSDs each storing 1 TB of data (for mathematical simplicity, 1 TB is considered as 240 bytes instead of 1012 bytes), erasure When the coding scheme implements a RAID5 array, the effective storage capacity of the virtual storage device is 4TB. (Other implementations of erasure coding that use fewer or more storage devices can store less or more than 1 TB, respectively, resulting in virtual storage devices with different capacities.) Look-Aside Erasure Coding PCIe switches 125 and/or 320 containing logic may notify processor 110 that they are coupled to a virtual storage device that provides a total of 4 TB (or 242 bytes) of storage capacity. Processor 110 of FIG. 1 may then record data to the block within this virtual storage device, as described further below with reference to FIG. can handle the actual storage of For example, if the blocks on the NVMe SSD are each 4KB in size, processor 110 may request that data be recorded in logical blocks numbered from 0 to (230-1).

Alternatively, a PCIe switch 125 and/or 320 that includes look-aside erasure coding logic requests a block of host memory addresses from the processor 110 of FIG. 1, which illustrates how to communicate with a virtual storage device. When the processor 110 of FIG. 1 wishes to read or record data, the transmission is sent to a PCIe switch 125 and/or 320 that has look-aside erasure coding logic that includes the appropriate address within a block of host memory addresses. Sent. This block of host memory addresses must be at least the size of the virtual storage device implemented using the erasure coding scheme (and additional storage devices are expected to be added to the erasure coding scheme while in use). (It may be larger than the initial capacity of the virtual storage device).

FIG. 5 is a diagram illustrating details of PCIe switch 125 including the look-aside erasure coding logic of FIG. In FIG. 5, the PCIe switch 125 including look-aside erasure coding logic includes a connector 505, a PCIe-to-PCIe stack 510-1 to 510-6), a PCIe switch core 515, and a PPU (Power Processing Unit) 520. including a wide variety of configurations. Connector 505 allows PCIe switch 125 containing look-aside erasure coding logic to connect to various other devices within machine 105 of FIG. 1, such as processor 110 of FIG. 1 and storage devices 130-1 through 130-6 of FIG. Allows communication with the configuration. One or more connectors 505 are referred to as "external" connectors in that they are coupled to an upstream component (such as processor 110 of FIG. 1). The remaining connectors 505 are referred to as internal or downstream "connectors" because they are coupled to downstream devices (such as storage devices 130-1 through 130-6 in FIG. 3). The PCIe-to-PCIe stacks 510-1 to 416 allow data exchange between PCIe devices. For example, storage device 130-1 in FIG. 3 transmits data to storage device 130-3 in FIG. Alternatively, the processor 110 of FIG. 1 requests one or more of the storage devices 130-1 to 130-6 of FIG. 3 to perform a read or write request. PCIe-to-PCIe stacks 510-1 to 510-6 include buffers that temporarily store data. For example, if the destination device for a particular transmission is currently busy, the PCIe-to-PCIe stacks 510-1 to 510-6 will transmit data until the destination device becomes free. Store. PPU 520 acts as a configuration center that processes any configuration requests for PCIe switch 125, including look-aside erasure coding logic. Although FIG. 5 illustrates six PCIe-to-PCIe stacks 510-1 through 510-6, embodiments of the invention include any number of PCIe-to-PCIe stacks. PCIe switch core 515 operates to route data from one PCIe port to another PCIe port.

Before entering into the operation of snooping logic 525 and erasure coding controller 530, it is understood that at least two different "addresses" are used for data stored on storage devices 130-1 through 130-6 of FIG. It helps to do that. In any storage device, data is recorded at a specific address associated with a hardware structure. This address is considered a "physical" address. In the context of an NVMe SSD, a "physical" address is commonly denoted as a PBA (Physical Block Address).

Flash memory used in NVMe SSDs generally does not allow data to be overwritten in-place. Instead, if data needs to be overwritten, the previous data is invalidated and the new data is recorded somewhere in another new block on the NVMe SSD. Therefore, the PBA in which data associated with a particular data structure (be it a file, object, or other data structure) is recorded changes over time.

Note that there is another reason why data must be relocated to flash memory. Data is typically deleted from flash memory in units larger than those used when writing data to flash memory. If valid data is stored anywhere on the unit to be deleted, the valid data should be recorded elsewhere in the flash memory before the unit is deleted. This deletion process is commonly referred to as Garbage Collection, and the process of copying valid data in the unit being deleted is referred to as programming. And Wear Leveling, a process that attempts to use cells of flash memory approximately the same, can also rearrange data within flash memory.

The host may be notified each time a particular data block is moved or may be notified of a new storage location for the data. However, reporting to the host in this way places a considerable burden on the host. Therefore, most flash memory devices maintain a table that informs the host of the logical block address (LBA) where data is stored and maps the LBA to a PBA (often a flash translation hierarchy (FTL)). Thereafter, each time the data is moved to a new PBA, the flash memory updates the LBA-PBA mapping table in the FTL instead of informing the host of the new address. Therefore, for each storage device there may be a PBA and LBA associated with the data.

Adding the concept of virtual storage devices, as presented by Look-Aside erasure coding logic, introduces another level to this structure. Referring to FIG. 3, recall the example presented above where the erasure coding scheme includes five 1TB NVMe SSDs, each NVMe SSD using 4KB size blocks. Each NVMe SSD includes LBAs numbered from 0 to (228-1). However, the virtual storage device provided to the host includes LBAs numbered from 0 to (230-1).

Therefore, the LBA ranges seen by the host represent many combinations of LBA ranges for various storage devices. To distinguish between the LBA range used by hosts and the LBA range of individual storage devices, LBAs used by hosts are referred to as "host LBAs," "global LBAs," or "operating system (O/S)-aware LBAs." while the LBA used by a storage device is referred to as a “device LBA,” “local LBA,” or “post-RoC LBA.” The host LBA range is divided among the various storage devices in any desired manner. For example, a host LBA range can be divided into contiguous blocks, with each block assigned to a particular storage device. If such a method is used, host LBA0 to (228-1) will be mapped to device LBA0 to (228-1) for storage device 130-1, and host LBA228 to 229-1 will be mapped to device LBA0 to (228-1) for storage device 130-1. It is mapped to LBA0 to 228-1, etc. of the device. Alternatively, individual bits in the host LBA are used to determine the appropriate storage device and device LBA to store the data. For example, the lower bits of the host LBA are used to identify the device and the bits are striped to generate the device LBA used by the storage device. However, regardless of how the host LBA is mapped to the device's LBA, there may be two, three, or potentially much different addresses representing the location where the data is stored.

Of course, it is not required that the storage devices be of the same type. That is, they may have different sizes and may have different numbers of LBAs. For example, they may be different device types, including a mix of SSDs and hard disk drives.

For simplicity of explanation, the term "device LBA" is used even though the address provided to the storage device is not a logical block address (eg, a hard disk drive). If the "device LBA" is the actual address at which the data is stored on the storage device, the storage device may not map the device LBA to another address before accessing the data. There is.

Returning to FIG. 5, snooping logic 525 and erasure coding controller 530 operate as look-aside erasure coding logic for PCIe switch 125, which includes look-aside erasure coding logic. Snooping logic 525 may “snoop” the transmission (e.g., by intercepting the request before it is communicated to its destination) and is communicated to snooping logic 525 via multiplexer 540. Use 535-1 through 535-6 for the capture interface to determine the appropriate destination. As mentioned above, processor 110 simply "looks" at a virtual storage device of a specified capacity (or a block of host memory addresses of a particular size) and determines the virtual storage device based on the host LBA (associated with the virtual storage device). , issues a data read or write command. Snooping logic 525 converts these host LBAs to device LBAs on one or more particular physical storage devices and thereby modifies the transmission to direct the request. Snooping logic 525 manages this conversion in any desired manner. For example, snooping logic 525 may detect factors related to the manner in which the look-aside erasure coding logic operates (e.g., its own erasure coding scheme (such as RAID level), stripe size, number of storage devices, etc.). The first range of host LBA is mapped to storage device 130-1 in FIG. 3, the second range of host LBA is mapped to storage device 130-2 in FIG. Equipped with a table. Snooping logic 525 also uses specific bits in the host LBA to determine whether to store the data in question in any of the storage devices 130-1 to 130-6 of FIG. For example, if the array includes only two storage devices, snooping logic 525 uses the lower bits (or other bits of the logical block address) to determine whether the data is on either the first or second storage device. Decide what will be recorded. (Obviously, as more storage devices are included in the array, more bits will be used; be careful not to include bit combinations that "identify" storage devices that are not present in the logical block address.) For example, FIG. 3 illustrates a total of 24 storage devices 130-1 to 130-6 that use bit values 00000 to 10111. Therefore, the value of the bits between 11000 and 11111 is (Should be avoided.) Embodiments of the present invention use any other desired approach of mapping logical block addresses received from the host to block addresses on the (appropriate) storage device.

For example, a diagram of sending a write request with enough data to fill an entire stripe across storage devices 130-1 through 130-6 (after factoring erasure coding) Consider one processor 110. Snooping logic 525 can partition the data into individual logical units, and erasure coding controller 530 provides or modifies the data, as described in more detail below. Snooping logic 525 may generate one transmission scheduled to each of storage devices 130-1 through 130-6 with appropriate data.

It should be noted that when snooping logic 525 replaces the original host LBA with a device LBA appropriate for the storage device, the device LBA does not need to be the address of a physical block. In other words, the LBA of a device used by the snooping logic may itself be an address of another logical block. Such a structure allows the physical storage device to continue to properly manage its own data storage. For example, if the physical storage device is an NVMe SSD, the SSD uses its flash translation layer to manage the concatenation of the device's LBA and PBA provided on any one of the NAND flash memory chips. Move data and perform garbage collection and wear leveling. These operations occur without the knowledge of snooping logic 525. However, the LBA of the device provided by snooping logic 525 is a physical address on the storage device, provided that the storage device does not relocate data unless instructed to do so by the host.

As described above, erasure coding controller 530 performs erasure coding. The erasure coding scheme allows erasure coding controller 530 to easily generate appropriate parity data (e.g., when using a RAID 5 or RAID 6 erasure coding scheme) and the original The data is left untouched. However, in some embodiments of the invention erasure coding controller 530 may also modify the original data. For example, the erasure coding controller 530 implements an error correction code in the original data so that the blocks stored in the individual storage devices 130-1 to 130-6 in FIG. 3 can be read properly even if an error occurs. do. Alternatively, the erasure coding controller 530 encrypts the data recorded in the storage devices 130-1 to 130-6 in FIG. 3 so that the data recorded in the storage devices 130-1 to 130-6 in FIG. (or worse, if processor 110 of FIG. 1 directly records data, erasure coding controller 530 may is considered to have been damaged). Alternatively, the erasure coding controller may introduce parity information (or similar type of information) into the data recorded on each storage device 130-1 through 130-6 in FIG. The particular operations on the data, as performed by erasure coding controller 530, depend on the erasure coding scheme used.

Snooping logic 525 and erasure coding controller 530 are implemented in any desired manner. For example, snooping logic 525 and erasure coding controller 530 are implemented using a processor with appropriate software stored thereon. However, PCIe switches are generally implemented as hardware circuits (which are typically faster than software running on the processor of devices such as PCIe switches, which do not need to implement numerous functions). As such, snooping logic 525 and erasure coding controller 530 are implemented using appropriate circuitry. The circuitry includes a suitably programmed field programmable gate array (FPGA), application-specific integrated circuit (ASIC), or other desired hardware implementation.

In the most basic embodiment, look-aside erasure coding logic is implemented using only snooping logic 525 and erasure coding controller 530. However, including cache 545 and/or write buffer 550 with look-aside erasure coding logic can provide considerable advantages.

Cache 545 may store a subset of data stored on the virtual storage device. Cache 545 typically has a smaller capacity than the overall virtual storage device, but is faster to access. Therefore, by storing some data in cache 545, a cache hit to cache 545 can result in faster performance of the virtual storage device than accessing data from the underlying physical storage device. be. For example, the cache 545 may store the most recently accessed data from the virtual storage device, and as stale data increases, the cache 545 may store the most recently accessed data from the virtual storage device, and as stale data increases, the cache 545 may store the most recently accessed data from the virtual storage device; Alternative data can be identified using the most frequently used algorithms. Cache 545 is implemented using any desired memory structure, such as DRAM, SRAM, MRAM, or any other desired memory structure. Cache 545 may also be implemented using memory structures that are faster than conventional memory, such as those used in L1 or L2 caches within processors. Finally, although cache 545 is illustrated as part of PCIe switch 125 containing look-aside erasure coding logic, cache 545 may also be stored in memory 115 of FIG. PCIe switch 125 including the PCIe switch 125.

Write buffer 550 provides a mechanism for quickly processing write requests. Using erasure coding, the time required to accomplish a write operation to a virtual storage device that extends (encompasses) a large number of physical storage devices is slower than a similar write request to a single physical storage device. there is a possibility. Performing a recording (write) operation can involve reading data from other storage devices in the same block, after which new data can be merged, and then the merged data is written back to the appropriate storage device. Performing the merge may also include calculating parity or other code information. Write requests may also be delayed if the underlying physical storage device is busy performing other operations (eg, processing read requests). It may be undesirable to delay software running on processor 110 of FIG. 1 while waiting for a write request to complete. Therefore, instead of blocking the software running on processor 110 of FIG. 1, the write Buffer 550 can temporarily store data until recording to the underlying physical storage device is complete. This approach is compared to a write-back cache policy in which write operations are completed before notifying the software running on processor 110 that the write is complete. This is similar to the write-through cache policy. Like cache 545, write buffer 550 is implemented using any desired memory structure, such as DRAM, SRAM, MRAM, or an L1 or L2 cache structure.

As part of performing the write operation, the look-aside erasure coding logic can check whether any data needed to complete the write operation is present in the current cache 545. For example, when the processor 110 of FIG. 1 transmits a write request to a virtual storage device, the erasure coding scheme may require that the entire stripe be read in order to calculate parity or other code information. . If some (or all) of that data resides in cache 545, the data can be accessed from cache 545 rather than reading the data from the underlying physical storage device. Additionally, the cache policy may suggest that the recorded data should also be cached in cache 545 if the data is requested again in a short period of time.

On the other hand, although FIG. 5 depicts cache 545 and write buffer 550 as separate elements, embodiments of the present invention may combine both into a single element (sometimes simply referred to as a "cache"). . In embodiments of the invention, the cache includes a bit that indicates whether the data stored in the cache is "clean" or "dirty." "Clean" data indicates data that has been read but has not been modified since it was last recorded on the underlying physical storage device. Data that is "dirty" is data that has been modified since it was last recorded on the underlying physical storage device. If the cache contains "dirty" data, the Look-Aside erasure coding logic requires that the "dirty" data be written back to the underlying storage device when the data is removed from the cache according to the cache policy. . It is noted that embodiments of the present invention may include both cache 545, write buffer 550 (separately or combined into a single element), or neither.

As discussed above, the look-aside erasure coding logic of the PCIe switch 125, which includes look-aside erasure coding logic, can "spawn" a virtual storage device from the underlying physical storage device, which is similar to the processor of FIG. This becomes a problem when 110 gains direct access to the physical storage devices 130-1 to 130-6 in FIG. Thus, when the machine 105 of FIG. 1 initially boots (i.e., starts or powers on) and attempts to enumerate the various PCIe devices it has access to, the PCIe switch 125 containing the Look-Aside erasure coding logic , decides to use Look-Aside erasure coding logic with the storage device coupled thereto. In this case, the PCIe switch 125 that includes Look-Aside erasure coding logic must prevent enumeration of any PCIe devices downstream from the PCIe switch 125 that includes Look-Aside erasure coding logic. By preventing these enumerations, the PCIe switch 125 that includes Look-Aside erasure coding logic can access data on the storage devices 130-1 through 130-6 of FIG. 3 without regard to the processor 110 of FIG. ``Create'' a virtual storage device (which can damage data used in erasure coding schemes). However, as described below with reference to FIGS. 9 and 10, there may be situations in which a PCIe switch 125 that includes Look-Aside erasure coding logic must allow downstream enumeration of PCIe devices.

Snooping logic 525 communicates configuration commands to PPU 520. In this way, snooping logic 525 also operates as a PCIe-to-PCIe stack for the purpose of coupling PCIe switch core 515 with PPU 520.

Finally, snooping logic 525 receives erasure coding enable signal 555 from processor 110 of FIG. 1 (perhaps via a pin on PCIe switch 125 that includes look-aside erasure coding logic). Erasure coding enable signal 555 is used to enable erasure coding logic to be enabled and disabled in PCIe switch 125 that includes look-aside erasure coding logic.

FIG. 6 is a diagram illustrating details of a PCIe switch including look-through erasure coding logic according to another embodiment of the invention. As can be seen by comparing with FIGS. 5 and 6, the PCIe switch 125 including Look-Aside erasure coding logic in FIG. 5 and the PCIe switch 605 including Look-Through erasure coding logic in FIG. 6 perform Look-Aside erasure coding. The main difference between logic and Look-Through erasure coding logic is where the erasure coding logic is placed. In the PCIe switch 125 with Look-Aside erasure coding logic of FIG. The coding logic is "inline" with the PCIe switch.

Using Look-Aside erasure coding logic compared to Look-Through erasure coding logic has technical advantages and disadvantages. The Look-Aside erasure coding logic of FIG. 5 is a more complex implementation because it requires snooping logic 525 to intercept and manage data redirection from the host. In contrast, all data between the host and the storage devices 130-1 to 130-6 in FIG. 3 passes through the erasure coding controller 530, making it easier to implement the Look-Through erasure coding logic in FIG. It is. On the other hand, when the erasure coding logic is disabled, the Look-Aside erasure coding logic does not introduce any additional delay into the operation of the PCIe switch 125. In contrast, the Look-Through erasure coding logic of FIG. 6 operates as a PCIe endpoint. The Look-Through erasure coding logic of FIG. 6 may buffer data between the host and the storage devices 130-1 to 130-6 of FIG. 3, which increases communication delay. In the Look-Through erasure coding logic of FIG. 6, erasure coding controller 530 also includes elements such as a frame buffer, a path table, port adjustment logic, and a scheduler (not shown in FIG. 6). That is, it is an element generally included within the PCIe switch core 515.

Note that PCIe switches generally use the same number of ports for upstream (to hosts) and downstream (to storage devices and other connected devices) traffic. For example, if PCIe switch 605 includes a total of 96 ports, 48 are typically used for upstream traffic and 48 are used for downstream traffic. However, when the Look-Through erasure coding logic of FIG. 6 is enabled, laser coding controller 530 virtualizes all downstream devices. In such situations, communication with the host typically requires only 16 or perhaps 32 upstream ports. If the PCIe switch 605 includes more than 32 or 64 ports, the additional ports may be used to connect additional downstream devices that are used to increase the capacity of the virtual storage device. Ru. To this end, erasure coding controller 530 of FIG. 6 uses a opaque bridge (NTB) port to communicate with the host.

FIG. 6 shows a PCIe switch 605 that includes Look-Through erasure coding logic. However, embodiments of the present invention separate the Look-Through erasure coding logic from the PCIe switch 605. For example, look-through erasure coding logic may be implemented as a separate component from PCIe switch 605 using an FPGA or ASIC.

However, while there are practical and technical differences between the look-aside erasure coding logic illustrated in FIG. 5 and the look-through erasure coding logic illustrated in FIG. The logic all achieve similar results. Accordingly, the Look-Aside erasure coding logic illustrated in FIG. 5 and the Look-Through erasure coding logic illustrated in FIG. 6 may be interchanged as needed. Any reference herein to Look-Aside erasure coding logic is intended to include Look-Through erasure coding logic.

7-10 illustrate various topologies for using PCIe switch 125 including the look-aside erasure coding logic of FIG. However, regardless of the topology in use, the operation of PCIe switch 125, including the look-aside erasure coding logic of FIG. 1, is the same. That is, to provide connectivity to various connected storage devices and to support erasure coding on the storage devices.

FIG. 7 is a diagram illustrating a first topology for using PCIe switch 125 including the look-aside erasure coding logic of FIG. 1, in accordance with one embodiment of the invention. 7, a PCIe switch 125 is illustrated that includes erasure coding logic implemented as a separate component of machine 105 of FIG. That is, PCIe switch 125 containing erasure coding logic can be manufactured and sold separately from any other components, such as processor 110 or storage device 130 of FIG.

A PCIe switch 125 containing look-aside erasure coding logic is coupled to a storage device 130. In FIG. 7, a PCIe switch 125 containing look-aside erasure coding logic is illustrated as being connected to only a single storage device, which may not support erasure coding. That is, erasure coding requires at least two storage devices or at least two portions of a storage device to perform striping, chunking, grouping, and use of parity or code information. However, a single storage device PCIe switch 125 with look-aside erasure coding logic also provides several advantages. For example, the PCIe switch 125 that includes Look-Aside erasure coding logic may use error correction codes with the storage device 130 or encrypt data stored on the storage device 130 if the service is not naturally provided by the storage device 130. support what you do.

Storage device 130 is also coupled to FPGA 705. FPGA 705 supports acceleration. That is, there may be situations where data needs to be processed and then discarded. Loading all the data into the processor 110 of FIG. 1 to perform processing is time consuming and expensive. Calculations that are closer to the data are easier to perform. FPGA 705 supports performing such calculations closer to the storage device, while avoiding the need for data to be loaded into processor 110 of FIG. 1 to perform the calculations. In other words, this concept is called "acceleration." FPGA-based acceleration is described in U.S. Provisional Application No. 62/642,568, filed March 13, 2018, and U.S. Provisional Application No. 62/642, filed March 9, 2018, which are incorporated herein by reference. No. 641,267, and U.S. Provisional Application No. 62/638,904, filed on March 5, 2018, and U.S. Application No. 16/122, filed on September 5, 2018, No. 16/124,179, filed September 6, 2018, and US Application No. 16/124, filed September 6, 2018, disclosed in US Pat. , 183, and U.S. Provisional Application No. 16/122,865, filed September 5, 2018. Because the purpose of acceleration is to process data without transmitting it to the processor 110 of FIG. 1, FIG. 7 illustrates the FPGA 705 closer to the storage device 130. Note, however, that the specific enumeration shown in FIG. 7 is not required. That is, the FPGA 705 is located between the PCIe switch 125 that includes Look-Aside erasure coding logic and the storage device 130.

In addition to data acceleration, FPGA 705 provides other functionality to support storage device 130. For example, the FPGA 705 implements data deduplication functions for the storage device 130 that reduce the number of times the same data is stored on the storage device 130. The FPGA 705 determines whether more than one particular piece of data is stored on the storage device 130 and configures associations between the various logical block addresses (or other information used by the host to identify the data). , the additional copies are deleted if stored on the data storage device 130.

Alternatively, the FPGA 705 inserts a data correction code (Error Correcting Code) to prevent data loss due to errors in the operation of the storage device 130, or inserts a CRC (Cyclic Code) for end-to-end protection. The storage device 130 implements data integrity functions, such as a data integrity field (T10DIF) using redundancy correction. In this manner, the FPGA 705 can restore the original data and detect that there are errors while reading or writing the data during transmission or on the storage device 130. The FPGA 705 implements data integrity (reliability) functionality without the host knowing that data integrity functionality is provided. The host can only identify the data itself and cannot identify error correcting codes.

Alternatively, the FPGA 705 can implement data encryption functions on the storage device 130 to protect data on the storage device 130 from unauthorized users who can access it. Without the appropriate encryption key provided, the data returned from the FPGA 705 may be meaningless to the requestor. The host provides an encryption key to use when reading and writing data. Additionally, the FPGA 705 automatically performs data encryption and decryption. The FPGA 705 can store cryptographic keys and determine the appropriate cryptographic keys to use based on who requests the data (and can generate them on behalf of the host).

Alternatively, FPGA 705 can implement data compression functions on storage device 130 that reduce the total space required to store data on storage device 130. When writing data to the storage device 130, the FPGA 705 compresses the data provided by the host into a smaller storage size, and then compresses the compressed data (when reading data from the storage device 130, the original data (along with the information necessary to restore it). When reading data from the storage device 130, the FPGA 705 reads the compressed data (along with the information necessary to restore the original data from the compressed data) and decompresses it to restore the original data. I can do it.

Suitable implementations of data deduplication, data integrity, data encryption, and data compression may be used. Embodiments of the invention are not limited to specific implementations of these features.

FPGA 705 embodies the combination of functionality on storage device 130 as intended. The FPGA 705 provides data compression and data reliability (this is because data compression can increase the error sensitivity of the data, so a single error in data stored in the storage device 130 can lead to unusable data). (the size can be increased). The FPGA 705 can implement both data encryption and data compression (to protect the data while using possible small storage of data). Other combinations of two or more functions are provided by FPGA 705.

In implementing these functions in the context of overall operation, FPGA 705 reads data from appropriate sources. Although the term "source" is singular, embodiments of the invention may read data from multiple sources (such as multiple storage devices) where appropriate. FPGA 705 performs appropriate operations on the data, such as data acceleration, data aggregation, data encryption, and/or data compression. FPGA 705 may use the results of the operations to take appropriate actions, such as transmitting the results to host 105 of FIG. 1 or writing data to storage device 130.

Although the functionalities have been described with reference to FPGA 705 in FIG. 7, embodiments of the invention include these functions in any part of the system that includes an FPGA. Additionally, embodiments of the invention may include an FPGA 705 that accesses data from a "distant" storage device. For example, returning to FIG. 3, assume that storage device 130-1 includes an FPGA similar to FPGA 705, but storage device 130-2 does not include these storage devices. The FPGA included in storage device 130-1 is used to apply those functionalities to storage device 130-2 by transmitting requests to storage device 130-2. For example, if the FPGA of storage device 130-1 provides data acceleration, the FPGA of storage device 130-1 transmits a request to read data from storage device 130-2, performs the appropriate acceleration, and then , the results may then be transmitted to an appropriate destination (such as machine 105 in FIG. 1).

In FIG. 7 (and in the topology illustrated in FIGS. 8-10), a PCIe switch 125 containing look-aside erasure coding logic is attached to a device that is not suitable for erasure coding. For example, the PCIe switch 125 containing Look-Aside erasure coding logic may be attached to a device such as the FPGA 705 or GPU of FIG. 7 that is not a storage device or other storage device that has built-in erasure coding functionality. . All these devices are described as devices that are not eligible for erasure coding (or at least erasure coding by PCIe switch 125 with Look-Aside erasure coding logic).

When a PCIe switch 125 with Look-Aside erasure coding logic is connected to a device that is not eligible for erasure coding, the system has various alternatives that can be used. In one embodiment of the invention, inclusion of any device that is not eligible for erasure coding disables the look-aside erasure coding logic of the PCIe switch 125 that has look-aside erasure coding logic. Thus, for example, when the PCIe switch 125 containing look-aside erasure coding logic is coupled to the FPGA 705 of FIG. Any one of the storage devices connected to switch 125 cannot be used with erasure coding. A decision to disable look-aside erasure coding logic in a PCIe switch 125 that contains look-aside erasure coding logic necessarily translates into other PCIe switches containing look-aside erasure coding logic in the same chassis or in other chassis. Please note that this does not mean that For example, FIG. 3 illustrates two PCIe switches 125, 320 that include look-aside erasure coding logic, one of which has look-aside erasure coding logic enabled and the other Has Look-Aside erasure coding logic disabled.

Other embodiments of the present invention disable devices that are not eligible for erasure coding and treat them as if they were not coupled at all to the PCIe switch 125 containing look-aside erasure coding logic. In embodiments of the invention, the PCIe switch 125 that includes look-aside erasure coding logic enables the look-aside erasure coding logic for the storage device 130, and any other storage device eligible for erasure coding is configured as if it were a look-aside erasure coding logic. It is disabled as if it were not coupled to the PCIe switch 125 containing the Aside erasure coding logic.

In other embodiments of the invention, a PCIe switch 125 that includes look-aside erasure coding logic may enable look-aside erasure coding logic for storage devices covered by the look-aside erasure coding logic, but with no erasure coding logic. Enable other devices that are not eligible for access to the coding. These embodiments of the invention are the most complex implementations. A PCIe switch 125 that includes look-aside erasure coding logic must determine whether a device is compliant with erasure coding or not, and the next traffic is directed to the virtual storage device. Analyze the traffic to determine whether it is specified (in this case, the traffic is intercepted by the Look-Aside erasure coding logic) or unspecified (in this case, the traffic is delivered to its existing destination). There is a need to.

In embodiments of the invention in which machine 105 does not provide the full functionality of the installed device (i.e., a device that is not eligible for erasure coding or that is disabled by PCIe switch 125 that contains Look-Aside erasure coding logic) In embodiments of the present invention where erasure coding is disabled due to the presence of an error code, the machine 105 can notify the user of this fact. This notification is provided by the processor 110 of FIG. 1, the BMC 325 of FIG. 3, or the PCIe switch 125 that includes Look-Aside erasure coding logic. In addition to informing the user that some functionality has been disabled, the notification may also inform the user how to reconfigure the machine 105 to allow for the added functionality. . For example, the notification may indicate that a device that is not eligible for erasure coding is coupled to a particular slot in the midplane 305 of FIG. Storage devices eligible for coding may be offered to be coupled to other slots, such as slots coupled to PCIe switch 125 that contain Look-Aside erasure coding logic. In such a manner, at least some storage devices that are eligible for erasure coding can benefit from the erasure coding scheme without blocking access to other devices that are not eligible for erasure coding.

FIG. 8 is a diagram illustrating a second topology for using PCIe switch 125 including the look-aside erasure coding logic of FIG. 1, in accordance with another embodiment of the invention. In FIG. 8, PCIe switch 125 containing look-aside erasure coding logic is located within FPGA 705. That is, FPGA 705 also embodies PCIe switch 125 that includes look-aside erasure coding logic. A PCIe switch 125 including an FPGA 705 and look-aside erasure coding logic is coupled to storage devices 130-1 to 130-4. On the other hand, although FIG. 8 illustrates a PCIe switch 125 including an FPGA 705 and Look-Aside erasure coding logic coupled to four storage devices 130-1 through 130-4, embodiments of the present invention may include any number of Includes storage devices 130-1 to 130-6.

Typically, the topology illustrated in FIG. (could be flash memory). That is, rather than being sold as a separate configuration, the entire structure illustrated in FIG. 8 is sold as a single unit. However, embodiments of the invention include a riser card coupled to the machine 105 of FIG. 1 (possibly to the midplane 305 of FIG. 3) at one end and a storage card at the other end. The U. 2.M. 3 or SFF-TA-1008 connectors. And, while FIG. 8 shows a PCIe switch 125 that includes Look-Aside erasure coding logic as part of the FPGA 705, the PCIe switch 125 that includes Look-Aside erasure coding logic may also be implemented as part of a smart SSD. Ru.

FIG. 9 is a diagram illustrating a third topology for using PCIe switch 125 including the Look-Aside erasure coding logic of FIG. FIG. 9 illustrates two PCIe switches 125, 320 that include look-aside erasure coding logic that couples up to 24 storage devices 130-1 through 130-6 therebetween. Referring to FIG. 3, as described above, each of the PCIe switches 125, 320 including Look-Aside erasure coding logic has a It has 96 PCIe lanes with 4 PCIe lanes used in That is, each PCIe switch 125, 320 that includes Look-Aside erasure coding logic supports up to 12 storage devices. To support erasure coding across storage devices supported by multiple PCIe switches 125, 320 containing look-aside erasure coding logic, one PCIe switch containing look-aside erasure coding logic can perform erasure coding across all devices. It may have Look-Aside erasure coding logic designated and enabled to be responsible for coding. Other PCIe switches 320 that include Look-Aside erasure coding logic operate completely as PCIe switches that include Look-Aside erasure coding logic disabled. The selection of which PCIe switch should be selected to handle erasure coding may be accomplished in any desired manner. For example, two PCIe switches can negotiate this between themselves, or the first listed PCIe switch is designated to handle erasure coding. PCIe switches selected to handle erasure coding can report virtual storage devices (spanning both PCIe switches), while PCIe switches that do not handle erasure coding can report erasure coding (where processor 110 in FIG. may not report downstream devices (to prevent attempts to access storage devices that are part of the coding scheme).

Note that both PCIe switches 125, 320 containing look-aside erasure coding logic can be in the same chassis, but PCIe switches 125, 320 containing look-aside erasure coding logic can be in different chassis. In other words, the erasure coding method allows storage devices to be expanded (included) in a large number of chassis. The requirement is that PCIe switches in various chassis must be able to negotiate with each other for where the storage devices that are part of the erasure coding scheme are located. Also, embodiments of the present invention are not limited to two PCIe switches 125, 320 that include look-aside erasure coding logic. That is, a storage device included in the erasure coding scheme is coupled to any number of PCIe switches 125, 320 that include look-aside erasure coding logic.

The host LBA is partitioned into PCIe switches 125, 320 containing look-aside erasure coding logic in any desired manner. For example, the least significant bit of the host LBA is used to identify whether the PCIe switch 125 or 320 that includes any look-aside erasure coding logic includes a storage device that stores data that includes the host LBA. If you use two or more PCIe switches containing look-aside erasure coding logic, a number of bits determine which PCIe switches containing look-aside erasure coding logic manage the storage devices that store the data. used to make decisions. Once the appropriate PCIe switch containing look-aside erasure coding logic is identified (and snooping logic 525 of FIG. 5 modifies the transmission), the transmission is routed to the appropriate PCIe switch containing look-aside erasure coding logic. routed (assuming the transmission is not to a storage device coupled to a PCIe switch that has Look-Aside erasure coding logic that includes Look-Aside erasure coding logic enabled).

In another embodiment of the invention, a single PCIe switch containing look-aside erasure coding logic is responsible for virtualizing all storage devices coupled to both PCIe switches containing look-aside erasure coding logic. Instead, each PCIe switch that Look-Aside includes erasure coding logic can create a separate virtual storage device (including a separate erasure coding domain). In this manner, different erasure coding domains are generated for different customers, but the capacity may be less.

FIG. 9 can also show another embodiment of the invention. Although FIG. 9 means that only storage devices 130-1 to 130-6 are coupled to PCIe switches 125, 320 that include Look-Aside erasure coding logic, as mentioned above, the present invention Without limitation, it is meant that all storage devices 130-1 to 130-6 are used with the erasure coding method. PCIe switches 125, 320 that include look-aside erasure coding logic may have devices connected to them that are not eligible for erasure coding. Contains storage devices eligible for erasure coding that are grouped under different PCIe switches 125 that contain look-aside erasure coding logic; these devices are connected to a single PCIe switch that contains look-aside erasure coding logic. Grouped under. In this manner, the optimal functionality of machine 105 of FIG. A single (or partial) PCIe switch containing erasure coding logic is achieved by disabling the Look-Aside erasure coding logic.

FIG. 10 is a diagram illustrating a fourth topology for using PCIe switch 125 including the look-aside erasure coding logic of FIG. 1, in accordance with another embodiment of the present invention. In FIG. 10, compared to FIG. 9, PCIe switches 125, 320, 1005 including look-aside erasure coding logic are hierarchically configured. A PCIe switch 125 containing look-aside erasure coding logic at the top of the hierarchy can manage erasure coding for all storage devices below the PCIe switch 125 containing look-aside erasure coding logic in the hierarchy. , may have Look-Aside erasure coding logic enabled. On the other hand, PCIe switches 320, 1005 that include Look-Aside erasure coding logic (because their storage devices are managed by the Look-Aside erasure coding logic of PCIe switch 125 that includes Look-Aside erasure coding logic) May have Look-Aside erasure coding logic disabled.

Whereas FIG. 10 shows PCIe switches 125, 320, 1005 that include look-aside erasure coding logic configured in a two-level hierarchical structure, embodiments of the present invention is not limited to a hierarchical structure. Accordingly, embodiments of the present invention can support any number of PCIe switches containing look-aside erasure coding logic arranged in any desired hierarchy.

With reference to FIGS. 1-10, the embodiments of the invention described above concentrate on single-port storage devices. However, embodiments of the invention can be extended to dual-port storage devices that are one (or more) storage devices that communicate with multiple PCIe switches that include look-aside erasure coding logic. In these embodiments of the invention, if the PCIe switch 125 containing the look-aside erasure coding logic of FIG. A transmission may be sent to a PCIe switch 320 that includes Look-Aside erasure coding logic to attempt to communicate with the device. PCIe switch 320 containing look-aside erasure coding logic effectively operates like a bridge that allows PCIe switch 125 containing look-aside erasure coding logic to communicate with storage devices.

Embodiments of the present invention also support detecting and handling storage device failures. For example, referring again to FIG. 4, assume that storage device 130-1 has failed. Storage device 130-1 can fail for various reasons. That is, power surges can damage electronic devices. Wiring (inside storage device 130-1 or in the link between PCIe switch 125 containing look-aside erasure coding logic and storage device 130-1) may fail, and storage device 130-1 may The storage device 130-1 may also detect many errors and terminate itself, and the storage device 130-1 may have failed for other reasons. Storage device 130-1 may also be removed from its slot by a user (perhaps to replace it with a newer, more reliable, or larger storage device). For whatever reason, storage device 130-1 may become unavailable.

PCIe switch 125, which includes Look-Aside erasure coding logic, detects failure of storage device 130-1 via the Presence pin on the connector to storage device 130-1. When storage device 130-1 is removed from the chassis or when storage device 130-1 is terminated, this occurs via a Presence pin on the connector that triggers an interrupt from PCIe switch 125, which contains Look-Aside erasure coding logic. , I can no longer assert (exercise) my own existence. Alternatively, the PCIe switch 125 (or BMC 325 in FIG. 3) containing look-aside erasure coding logic may transmit a temporary message to the storage device 130-1 to indicate that it is still active (sometimes a "heartbeat"). )” can be checked. In other words, if storage device 130-1 does not respond to such a message, PCIe switch 125 or BMC 325 of FIG. 3, which includes look-aside erasure coding logic, may conclude that storage device 130-1 has failed. can.

If storage device 130-1 fails, PCIe switch 125 containing Look-Aside erasure coding logic can access any data that is normally requested from storage device 130-1 using other means. Manage the situation. For example, if there is a mirror of storage device 130-1, PCIe switch 125 including look-aside erasure coding logic requests data from the mirror of storage device 130-1. Alternatively, the PCIe switch 125 containing Look-Aside erasure coding logic may request the remaining stripes containing the desired data from other storage devices in the array, using the erasure coding information to Restore data from. PCIe switch 125 that includes look-aside erasure coding logic may have another mechanism to access data stored on failed storage device 130-1.

Embodiments of the invention also support detecting and processing the insertion of new storage devices into the array. The PCIe switch 125 (or BMC 325 in FIG. 3) containing Look-Aside erasure coding logic may sometimes detect storage device failures, verify what is coupled, or other desired mechanism. The insertion of a new storage device can be detected via the Presence pin on the connector by pinging the device (just like detecting a failed storage device, the Presence pin can be used to detect a new storage Detecting the device triggers an interrupt at the PCIe switch 125, which includes Look-Aside erasure coding logic). When a new storage device is detected, the PCIe switch 125, which includes Look-Aside erasure coding logic, adds the new storage device to the array. Adding new storage devices to an array does not necessarily involve changing erasure coding schemes. That is, these changes may require that all data stored on the storage device must be changed (for example, consider changing from RAID 5 to RAID 6. Each stripe is (Requires two parity blocks that need to be rotated between devices and requires a large amount of data computation and movement). However, adding new storage devices to existing erasure coding schemes may not require moving large amounts of data. Therefore, although adding a new storage device does not increase the array's tolerance to storage device failure, adding a new storage device may nevertheless increase the capacity of the virtual storage device.

If there is a failed storage device in the array, insertion of a new storage device is used to reconfigure the failed storage device. Erasure coding controller 530 of FIG. 5 calculates the data stored on the failed storage device and stores the data in the appropriate block address of the replacement storage device. For example, the original data that was in the failed storage device is calculated from the data (original data and all of the parity or code information) in other storage devices. The parity or code information stored on the failed storage device is recomputed from the original data on the other storage device (of course, if the failed storage device is mirrored, erasure coding controller 530 of FIG. be copied from the mirror to an alternate storage device).

Reconfiguring a failed storage device can take time. In some embodiments of the invention, reconfiguration occurs as soon as a replacement storage device is installed. In other embodiments of the invention, as long as the storage device is reconfigured within a slack period, erasure coding controller 530 of FIG. 5 does so. However, if the virtual storage device is busy, erasure coding controller 530 of FIG. Reconfigure data from failed storage devices as necessary. (These reconstructed data, of course, are recorded on an alternate storage device without waiting for complete reconstruction, so there is no need to recalculate the data again later.)

Embodiments of the invention also support initialization of storage devices. When a new storage device is added to the array - either as a replacement storage device for a failed storage device or to increase the capacity of a virtual storage device - the new storage device is initialized. Initialization includes preparing the storage device for erasure coding schemes.

Initializing a new storage device may also include deleting existing data from the new storage device. For example, consider a situation where a particular storage device is leased to a customer. That customer's lease is terminated and the storage device can be reused by a new customer. However, the storage device may still have the customer's original data. Any desired mechanism can be used to delete the data on the storage device to prevent the next customer from accessing the previous customer's data. For example, a table that stores information about where data is stored can be erased. or overwriting the data itself with new data (to prevent later attempts to recover any erased information). The new data uses a pattern designed to ensure that the original data cannot be recovered. For example, the US Department of Defense (DOD) has published standards for how to delete data to prevent recovery. These standards are used to remove obsolete data from a storage device before reusing it for a new client.

Initialization is not limited only when a new storage device is hot-added to an existing array. Initialization also occurs when the storage device, PCIe switch 125 containing look-aside erasure coding logic, or the machine 105 of FIG. 1 as a whole, first powers up.

11A-11D are examples for PCIe switch 125 that includes the look-aside erasure coding logic of FIG. 1 to support coding schemes 405, 410, 415 of FIG. 4, in accordance with one embodiment of the present invention. It is a flowchart of the procedure. At block 1103 of FIG. 11A, the PCIe switch 125 containing the look-aside erasure coding logic of FIG. 3 is initialized by the processor 110 of FIG. 1 or the BMC 325 of FIG. At block 1106, PCIe switch 125, which includes the Look-Aside erasure coding logic of FIG. 3, receives the transmission. Such transmissions may be in response to a read or write request from processor 110 of FIG. 1, a transfer of control from processor 110 of FIG. 1 or BMC 325 of FIG. 3, or a read or write request from processor 110 of FIG. may be the transmissions sent by the storage devices 130-1 to 130-6 in FIG.

At block 1109, snooping logic 525 of FIG. 5 determines whether the transmission is a control transmission from processor 110 of FIG. If so, at block 1112, PCIe switch 125 including Look-Aside erasure coding logic of FIG. 3 may communicate a control transmission to PPU 520 of FIG. The process then ends.

If the transmission is not a control transmission from processor 110 of FIG. 1, in step S1115 (FIG. 11B), snooping logic 525 of FIG. 5 may determine whether the transmission is a read or write request from the host. Otherwise, in step S1118, the snooping logic 525 of FIG. 5 may substitute the device LBA in the transmission with a host LBA compatible with the host. Snooping logic 525 of FIG. 5 can also modify the transmission to mean that it came from a virtual storage device rather than the physical storage device that stored the actual data. In step S1121, PCIe switch 125 including look-aside erasure coding logic of FIG. 3 may communicate the transmission to processor 110 of FIG. 1. The process then ends.

On the other hand, if the transmission is a read or write request from processor 110 of FIG. 1, in step S1124 snooping logic 525 of FIG. availability can be determined. If the data is available from cache 545 of FIG. 5 or write buffer 550 of FIG. 5, erasure coding controller 530 of FIG. 5 may access the data at the appropriate location in step S1127 (FIG. 11C).

If the data is not available from cache 545 of FIG. 5 or write buffer 550 of FIG. 5, in step S1130 snooping logic 525 of FIG. Transmissions can be modified to substitute for device LBAs that are not available. Snooping logic 525 of FIG. 5 may also modify the transmission to identify the appropriate storage device to receive the transmission. Next, in step S1133, snooping logic 525 may route the transmission to the appropriate storage device.

At this point, the PCIe switch 125 containing the Look-Aside erasure coding logic of FIG. 3 has the necessary data, whether the data in question is accessible from cache or read from a storage device. . At this point, the process branches. If the transmission is a read request from processor 110 of FIG. 1, PCIe switch 125 including look-aside erasure coding logic of FIG. 3 returns the data to processor 110 of FIG. 1 in step S1136. Snooping logic 525 of FIG. 5 may also store data in cache 545 of FIG. 5, as illustrated in step S1139. Step S1139 is optional and is omitted as illustrated by dotted line 1142. At this point, the process ends.

On the other hand, if the transmission from processor 110 in FIG. 1 is a write request, erasure coding controller 530 in FIG. 5 reads stripes across storage devices 130-1 to 130-6 in FIG. 3 in step S1145. Step S1145 is a re-expression of steps S1127, 1130, and 1133 and may not be necessary. Accordingly, step S1145 is included in FIG. 11C to emphasize that recording data to a virtual storage device can include reading data from an entire stripe across storage devices 130-1 through 130-6. . In step S1148, erasure coding controller 530 of FIG. 5 may merge data received from processor 110 of FIG. 1 with data stripes accessed from storage devices 130-1 to 130-6 or cache.

At this point, the procedure branches depending on whether the PCIe switch 125 that includes the Look-Aside erasure coding logic of FIG. 3 includes the write buffer 550 of FIG. Once the PCIe switch 125 including the Look-Aside erasure coding logic of FIG. 3 includes the write buffer 550 of FIG. 5, the erasure coding controller 530 of FIG. 5 (marking the data as "dirty" and needing to be flushed to storage devices 130-1 through 130-6). In the next step S1154, the PCIe switch 125 including the Look-Aside erasure coding logic of FIG. 3 reports to the processor 110 that the write request has been completed. Note that step S1154 is appropriate if write buffer 550 of FIG. 5 uses a write-back cache policy. If write buffer 550 of FIG. 5 uses a write-through cache policy, step S1154 is omitted, as illustrated by dotted line 1157.

After all, the PCIe switch 125 including the Look-Aside erasure coding logic in FIG. 3 does not include the write buffer 550 in FIG. 5, or the data in the write buffer 550 in FIG. 130-6, the erasure coding controller 530 of FIG. 5 can re-record the updated stripe in the storage devices 130-1 to 130-6 of FIG. 3 in step S1160. Next, in step S1163, the PCIe switch 125 including the Look-Aside erasure coding logic of FIG. 3 may report to the one processor 110 that the write request has been completed. Note that if the merged data is stored in write buffer 550 of FIG. 5 and write buffer 550 of FIG. 5 uses a write-back cache policy, step S1163 is not necessary. That is, the PCIe switch 125 including the Look-Aside erasure coding logic of FIG. 3 has reported that the write request has already been completed (in step S1154). In such a situation, step S1163 is omitted as illustrated by dotted line 1166. At this point, the process ends.

12A and 12B are flowcharts illustrating an example procedure for PCIe switch 125 including the look-aside erasure coding logic of FIG. 1 to perform initialization in accordance with an embodiment of the present invention. . In step S1205 of FIG. 12A, the PCIe switch 125 including the Look-Aside erasure coding logic of FIG. It can be determined whether the PCIe switch 125 has erasure coding managed by the PCIe switch 125 that includes Look-Aside erasure coding logic of FIG. There may be a device coupled to the PCIe switch 125 that includes the look-aside erasure coding logic of FIG. 3 that is not a storage device, or a device that is managed by the PCIe switch 125 that includes the look-aside erasure coding logic of FIG. If there is a storage device that does not exist, in step S1210, the PCIe switch 125 including the look-aside erasure coding logic of FIG. 3 disables the look-aside erasure coding logic. I can do it. The procedure then ends.

However, in other embodiments of the invention, the PCIe switch 125 including the look-aside erasure coding logic of FIG. Erasure coding can be managed even with connected devices. In these embodiments of the invention, or if the only storage device eligible for this erasure coding is coupled to the PCIe switch 125 that includes the Look-Aside erasure coding logic of FIG. PCIe switch 125, which includes look-aside erasure coding logic, enables the look-aside erasure coding logic. Next, in step S1220 (FIG. 12B), the PCIe switch 125 including Look-Aside erasure coding logic of FIG. 3 is configured to use an erasure coding scheme (perhaps the BMC 325 of FIG. 3 or the by 110)

In step S1225, the PCIe switch 125 including Look-Aside erasure coding logic of FIG. 3 disables devices that are not suitable for erasure coding. Step S1225 is optional, as illustrated by dotted line 1230. There may be no PCIe switch 125 containing the look-aside erasure coding logic of FIG. 3 that is not eligible for erasure coding, or the PCIe switch 125 containing the look-aside erasure coding logic of FIG. , or using erasure coding, authorizes the processor 110 of FIG. 1 to access devices that are not eligible for erasure coding.

In step S1235, the PCIe switch 125 including the look-aside erasure coding logic of FIG. 3 performs downstream enumeration of the PCIe switch 125 including the look-aside erasure coding logic of FIG. finish. In step S1240, the PCIe switch 125 including the Look-Aside erasure coding logic in FIG. 3 creates a virtual storage device in FIG. The information may be reported to processor 110. PCIe switch 125 that includes Look-Aside erasure coding logic of FIG. 3 may also report any other enumerated PCIe devices to processor 110 of FIG. 1. At this point, the process ends.

FIG. 13 shows an example of a PCIe switch 125 that includes the look-aside erasure coding logic of FIG. 1 to include new storage devices in the erasure coding schemes 405, 410, 415 of FIG. 4, in accordance with one embodiment of the present invention. It is a flowchart of the procedure as follows. In step S1305 of FIG. 13, the PCIe switch 125 (or BMC 325 of FIG. 3) including the Look-Aside erasure coding logic of FIG. 3 checks for a new storage device. Once a new storage device is detected, in the next step S1310, erasure coding controller 530 of FIG. 5 adds the new storage device to the array behind the virtual storage device. Finally, in step S1315, the PCIe switch 125 (BMC 325 of FIG. 3 or processor 110 of FIG. 1) including the Look-Aside erasure coding logic of FIG. 3 may initialize the new storage device. At this point, the procedure ends or returns to step S1305 to check for additional new storage devices, as illustrated by dotted line 1320.

FIG. 14 is a flowchart of an example procedure for PCIe switch 125 including the look-aside erasure coding logic of FIG. 1 to handle a failed storage device, in accordance with one embodiment of the present invention. In step S1405 of FIG. 14, the PCIe switch 125 (or BMC 325 of FIG. 3) including Look-Aside erasure coding logic of FIG. 3 may check for failed (or removed) storage devices. Once a failed storage device is detected, in step S1410, the erasure coding controller 530 of FIG. 5 performs erasure coding restoration of data stored on the failed storage device when a read request to access the failed data arrives. carry out. Restoring these erasure codes may include reading data from the stripe containing the requested data from other storage devices and computing the requested data from the remaining data in the stripe.

In step S1415, the PCIe switch 125 (or BMC 325 of FIG. 3) that includes Look-Aside erasure coding logic of FIG. 3 determines whether a replacement storage device has been added to the array behind the virtual storage device. If so, in step S1420 erasure coding controller 530 of FIG. 5 reconfigures the failed storage device using a replacement storage device. At this point, the procedure ends or returns to step S1405 to check for additional new storage devices, as illustrated by dotted line 1425.

Several embodiments of the invention are illustrated in FIGS. 11A-14. However, one skilled in the art will recognize that other embodiments of the invention are possible by changing the order of steps, omitting steps, or including connections not shown in the drawings. . All such variations to the flowcharts, whether explicitly described or not, are considered embodiments of the invention.

Embodiments of the present invention provide technical advantages over the prior art. Using a PCIe switch that includes Look-Aside erasure coding logic moves erasure coding closer to the storage device, reducing the time required to move data. Removing erasure coding on a processor reduces the load on the processor and allows the processor to execute more commands for applications. With a configurable erasure coding controller, any desired erasure coding scheme can be used instead of the limited set of methods supported by hardware and software erasure coding vendors. Placing the erasure coding controller with the PCIe switch avoids the need for expensive RAID add-in cards and allows larger arrays to be used up to multiple chassis.

The following description is intended to provide a concise and general description of a suitable machine or machines in which certain aspects of the inventive concept may be implemented. The machine or machines may be configured to operate, at least in part, on instructions received from other machines, interaction with a virtual reality (VR) environment, biofeedback, or other input signals, such as a keyboard, mouse, etc. It is controlled by input from ordinary input devices. As used herein, the term "machine" is intended to broadly include a single machine, a virtual machine, multiple machines, multiple virtual machines, or systems coupled in communication with devices operating together. do. Exemplary machines include, for example, transportation devices such as personal or public transport such as cars, trains, taxis, etc., as well as computing devices such as personal computers, workstations, servers, portable computers, handheld devices, mobile phones, tablets, etc. equipped with a marking device.

The machine or machines include an embedded controller, such as a programmable or non-programmable logic device or array, an application specific integrated circuit (ASIC), an embedded computer, a smart card, and the like. The machine or machines may utilize one or more connections to one or more remote machines, such as through a network interface, modem, or other communication coupling. A plurality of machines are interconnected by means of physical and/or logical networks such as an intranet, the Internet, local area networks (LAN), and wide area networks (WAN). Those skilled in the art will appreciate that network communications can be implemented using a variety of wired and/or wireless short-range or long-range carriers and radio frequency (RF), satellite, microwave, IEEE 802.11, Bluetooth, optical, You will understand the use of protocols including infrared, cable, laser, etc.

Embodiments of the technical idea of the present invention provide functions, procedures, which, when accessed by a machine, cause the machine to perform work or define types of abstract data or low-level hardware context. Described with reference to associated data, including data structures, applications, etc. The associated data may include, for example, volatile and/or non-volatile memories such as RAM, ROM, etc., other storage devices, hard disk drives, floppy disks, optical storage, tapes, flash memory, memory sticks, digital video disks, etc. stored on an associated storage medium, including biometric storage and the like. The associated data may be conveyed in the form of packets, serial data, parallel data, transmission signals, etc. through a transmission environment including physical and/or logical networks and used in a compressed or encrypted format. . The associated data is used in a distributed environment and stored locally and/or remotely for machine access.

Embodiments of the inventive concept are executable by one or more processors and include an actual non-transitory system that includes commands that cause the elements of the inventive concept to be performed as described herein. machine-readable media.

The various operations of the methods described above may be performed by any suitable means for performing the operations, such as various hardware and/or software configurations, circuits, and/or modules. Software includes an ordered list of commands that can be executed to implement logical functionality using or with a command execution system, mechanism or device such as a processor, including single or multi-core processors or systems. Relatedly, it may be embodied in any "processor-readable medium."

The steps or stages of the methods or algorithms and functions associated with the embodiments disclosed herein may be directly implemented in hardware and software modules running on a processor, or a combination thereof. If implemented in software, the functions may be stored on or transmitted over as one or more commands or code on non-transitory computer-readable media. A software module may be random access memory (RAM), flash memory, ROM, EPROM, EEPROM, registers, hard disk, removable disk, CD ROM, or any other form of storage medium known in the art.

Although the principles of the technical idea of the invention have been explained and illustrated with reference to the illustrated embodiments, the illustrated embodiments may be modified in arrangement and detail in any necessary manner without departing from these principles. It will be understood that it can be combined with Although the foregoing discussion has focused on specific embodiments, other configurations are also contemplated. Specifically, notwithstanding statements such as "according to embodiments of the technical idea of the invention" and similar expressions used herein, these phrases generally refer to possible embodiments. However, it is not intended that the technical idea of the present invention be limited to the configuration of specific embodiments. These terms can refer to the same or different embodiments that can be combined into other embodiments.

The embodiments described above do not limit the technical idea of the present invention. Although only a few embodiments have been described, those skilled in the art will appreciate that many modifications can be made to these embodiments without departing substantially from the novelty and advantages of this description. will do. Accordingly, all such modifications are intended to be included within the spirit of the invention as defined in the claims.

Embodiments of the invention extend without limitation to the following description.

Explanation 1.
Embodiments of the inventive concept include a PCIe switch that includes erasure coding logic.
The PCIe switch is
an external connector that allows the PCIe switch to communicate with a processor;
at least one connector that allows the PCIe switch to communicate with at least one storage device;
a PPU that processes the configuration of the PCIe switch;
an erasure coding controller including a circuit that applies an erasure coding method to the data stored in the at least one storage device;
snooping logic including circuitry to intercept data transmissions received at the PCIe switch and modify the data transmissions in response to the erasure coding scheme.

Explanation 2.
In the PCIe switch including erasure coding logic according to Description 1, the erasure coding logic is obtained from a set including Look-Aside erasure coding logic and Look-Through erasure coding logic.

Explanation 3.
In the PCIe switch including erasure coding logic according to Description 1, the at least one storage device includes at least one NVMe SSD.

Explanation 4.
In a PCIe switch that includes erasure coding logic according to Description 3, the snooping logic operates to intercept control transmissions received at the PCIe switch and forward the control transmissions to the PPU.

Explanation 5.
In a PCIe switch including erasure coding logic according to Description 3, the snooping logic intercepts a transmission of data received at the PCIe switch from a host and determines the host LBA used by the host in the data transmission by the at least one NVMe SSD. It operates to replace the device LBA used.

Explanation 6.
In a PCIe switch including erasure coding logic according to Description 5, the snooping logic operates to direct data transmission to at least one NVMe SSD.

Explanation 7.
In a PCIe switch that includes erasure coding logic according to Description 3, the snooping logic intercepts a transmission of data received at the PCIe switch from one of the at least one NVMe SSD, and in the transmission of data, the at least one NVMe SSD The device LBA used by one of the hosts is replaced by the host LBA used by the host.

Explanation 8.
A PCIe switch including erasure coding logic according to Description 3 further includes a cache.

Explanation 9.
In a PCIe switch including erasure coding logic according to Description 8, the snooping logic is configured to return a response to a requested data transmission from a host based at least in part on data requested in the transmission of data present in the cache. Operate.

Explanation 10.
In a PCIe switch including erasure coding logic according to Explanation 3,
A PCIe switch located in the chassis,
a chassis including memory used as an external cache by the erasure coding controller;

Explanation 11.
A PCIe switch including erasure coding logic according to Description 3 further includes a write buffer.

Explanation 12.
In a PCIe switch including erasure coding logic according to Description 11,
the data transmission including a write operation from a host;
an erasure coding controller operative to complete the write operation after transmitting a response to the data transmission to the host.

Explanation 13.
In a PCIe switch including erasure coding logic according to Description 11,
The erasure coding controller is operative to store data in a write operation in a write buffer.

Explanation 14.
In a PCIe switch including erasure coding logic according to Description 3, the PCIe switch is operative to enable the erasure coding controller and snooping logic based at least in part on at least one NVMe SSD used with the erasure coding controller. do.

Explanation 15.
In the PCIe switch including erasure coding logic according to Statement 3, the PCIe switch includes an erasure coding controller and snooping logic based at least in part on at least one NVMe SSD that includes built-in erasure coding functionality. It works to disable it.

Explanation 16.
In the PCIe switch including erasure coding logic according to Description 15, the PCIe switch provides a user with an erasure coding controller and snooping based at least in part on at least one NVMe SSD that includes built-in erasure coding functionality. Acts to notify that logic has been disabled.

Explanation 17.
In a PCIe switch including erasure coding logic according to Description 3, the at least one connector is used to disable an erasure coding controller and snooping logic based in part on a non-storage device coupled to the PCIe switch. works.

Explanation 18.
A PCIe switch including erasure coding logic according to Description 17 provides a user with an erasure coding controller and nooping logic based in part on a non-storage device coupled to the PCIe switch using the at least one connector. operates to notify that it has been disabled.

Explanation 19.
In a PCIe switch including erasure coding logic according to Description 3, the at least one connector is used to enable an erasure coding controller and snooping logic based in part on a non-storage device coupled to the PCIe switch. works.

Explanation 20.
In the PCIe switch including erasure coding logic according to Explanation 19, the PCIe switch operates to notify the user that access to a non-storage device connected to the PCIe switch is blocked.

Explanation 21.
In a PCIe switch including erasure coding logic according to Description 3, the PCIe switch includes an erasure coding controller and snooping logic to manage erasure coding schemes on at least one additional NVMe SSD coupled to the second PCIe switch. It works like using .

Explanation 22.
In the PCIe switch including erasure coding logic according to Description 21, the second PCIe switch is operative to disable the second erasure coding controller and the third snooping logic in the second PCIe switch.

Explanation 23.
In a PCIe switch including erasure coding logic according to Description 22,
a PCIe switch located within the first chassis;
a second PCIe switch located within the second chassis.

Explanation 24.
In a PCIe switch including erasure coding logic according to Explanation 3, the PCIe switch is implemented using an FPGA.

Explanation 25.
In a PCIe switch including erasure coding logic according to Explanation 3,
the at least one NVMe SSD comprising at least two NVMe SSDs;
the PCIe switch and the at least two NVMe SSDs within a common housing.

Explanation 26.
In the PCIe switch including erasure coding logic according to Description 3, the PCIe switch and the at least two NVMe SSDs are in separate housings.

Explanation 27.
In a PCIe switch including erasure coding logic according to Explanation 3,
the PCIe switch operative to detect a failed NVMe SSD of the at least one NVMe SSD;
and an erasure coding controller operative to process data transmissions to account for a failed NVMe SSD.

Explanation 28.
In a PCIe switch that includes erasure coding logic according to Description 27, the erasure coding controller operates to perform erasure coding restoration of data stored on a failed NVMe SSD.

Explanation 29.
In a PCIe switch that includes erasure coding logic according to Description 28, the erasure coding controller operates to reconfigure a replacement NVMe SSD for a failed NVMe SSD.

Explanation 30.
In a PCIe switch including erasure coding logic according to Explanation 3,
the PCIe switch operative to detect a new NVMe SSD;
Includes an erasure coding controller operative to use the new NVMe SSD as part of the erasure coding scheme.

Explanation 31.
In a PCIe switch that includes erasure coding logic according to Description 30, the erasure coding controller operates to accomplish capacity additions using new NVMe SSDs.

Explanation 32.
In a PCIe switch including erasure coding logic according to Description 30, the PCIe switch is operative to detect a new NVMe SSD coupled to one of the at least one connector.

Explanation 33.
In a PCIe switch that includes erasure coding logic according to Description 30, the PCIe switch operates to detect new NVMe SSDs via messages from a second PCIe switch.

Explanation 34.
In a PCIe switch that includes erasure coding logic according to Description 33, a new NVMe SSD is coupled to a second connector on a second PCIe switch.

Explanation 35.
In a PCIe switch including erasure coding logic according to Description 3, at least one connector includes a presence pin for detecting both a failed NVMe SSD and a new NVMe SSD.

Explanation 36.
In a PCIe switch that includes erasure coding logic according to Description 3, the PCIe switch operates to present itself to the host as a single device and prevent enumeration of at least one downstream PCIe bus.

Explanation 37.
In a PCIe switch that includes erasure coding logic according to Description 36, the PCIe switch operates to prevent enumeration of a downstream PCIe bus from a PCIe switch to a second PCIe switch.

Explanation 38.
In a PCIe switch that includes erasure coding logic according to Description 36, the PCIe switch operates to virtualize at least one NVMe SSD.

Explanation 39.
In a PCIe switch including erasure coding logic according to Description 3, the erasure coding controller is operative to initialize a new NVMe SSD coupled to one of the at least one connector.

Explanation 40.
In a PCIe switch that includes erasure coding logic according to Description 39, the erasure coding controller operates to initialize a new NVMe SSD after a hot insertion event.

Explanation 41.
In a PCIe switch that includes erasure coding logic according to Description 39, the erasure coding controller is operative to initialize at least one NVMe SSD upon power-up.

Explanation 42.
In a PCIe switch including erasure coding logic according to Description 3, the PCIe switch is part of a system including a BMC operative to initialize a new NVMe SSD coupled to one of the at least one connector.

Explanation 43.
In a PCIe switch that includes erasure coding logic according to Description 42, the BMC operates to initialize at least one NVMe SSD at power-up.

Explanation 44.
In a PCIe switch including erasure coding logic according to Description 3, the erasure coding controller includes a stripe manager that stripes data across at least one NVMe SSD.

Explanation 45.
Embodiments of the inventive concept include methods. The method includes:
receiving a transmission from a PCIe switch utilizing erasure coding logic;
processing the transmission using snooping logic in the erasure coding logic;
and communicating the transmission to its destination via the PCIe switch.

Explanation 46.
In the method according to statement 45, the erasure coding logic includes one of Look-Aside erasure coding logic and Look-Through erasure coding logic.

Explanation 47.
In the method according to Explanation 45,
Processing the transmission using snooping logic with the erasure coding logic includes determining that the transmission includes a controlled transmission by the snooping logic;
Delivering the transmission to its destination via the PCIe switch includes transmitting the transmission to a PPU (Power Processing Unit).

Explanation 48.
In the method according to Explanation 45,
Processing the transmission using snooping logic with the erasure coding logic includes processing the transmission using snooping logic based at least in part on activated erasure coding logic.

Explanation 49.
In the method according to Explanation 45,
Receiving a transmission from a PCIe switch including erasure coding logic includes receiving a read request from a host;
Processing the transmission using snooping logic in the erasure coding logic includes substituting a host LBA with an LBA of a device in the read request;
Communicating the transmission to its destination via the PCIe switch includes communicating a read request to an NVMe SSD.

Explanation 50.
In the method according to Statement 49, processing the transmission using snooping logic in the erasure coding logic further includes identifying an NVMe SSD to which the read request should be communicated.

Explanation 51.
In the method according to Explanation 49,
Processing the transmission using snooping logic in the erasure coding logic includes accessing data requested by a host in a read request from the cache based at least in part on data present in the cache. including,
Substituting the host LBA with the LBA of the device in the read request includes substituting the host LBA with the LBA of the device in the read request based at least in part on data not present in the cache;
Communicating the transmission to its destination via the PCIe switch includes communicating a read request to an NVMe SSD based at least in part on data not present in cache.

Explanation 52.
In the method according to Explanation 45,
Receiving a transmission at a PCIe switch including erasure coding logic includes receiving a read request from a host;
Processing the transmission using snooping logic in the erasure coding logic includes substituting a host LBA with an LBA of a device in the read request;
Communicating the transmission to its destination via the PCIe switch includes communicating a read request to an NVMe SSD.

Explanation 53.
In the method according to statement 52,
Processing the transmission using snooping logic in the erasure coding logic further includes identifying an NVMe SSD to which the write request is to be transmitted.

Explanation 54.
In the method according to statement 52,
reading a stripe of blocks from at least one NVMe SSD;
merging the stripe of the block with the data in the write request to form an updated stripe of the block;
The method further includes recording the updated stripe of blocks on the at least one NVMe SSD.

Explanation 55.
In the method according to statement 54,
Merging the data in the write request includes calculating additional data to be recorded on at least one NVMe SSD in addition to the data in the write request.

Explanation 56.
In the method according to statement 54,
reading a stripe of blocks from the cache based at least in part on a stripe of blocks present in the cache;
Reading a stripe of blocks from the at least one NVMe SSD comprises reading a stripe of blocks from the at least one NVMe SSD based at least in part on a stripe of blocks that are not present in the cache.

Explanation 57.
In the method according to statement 54,
Recording the updated stripe of the block to the at least one NVMe SSD includes recording the updated stripe of the block to a write buffer.

Explanation 58.
In the method according to Explanation 57,
After the updated stripe of the block is written to the write buffer and before the updated stripe of the block is recorded to the at least one NVMe SSD, the method further includes responding to the host that the recording is complete.

Explanation 59.
In the method according to Explanation 45,
Receiving the transmission at the PCIe switch including erasure coding logic includes receiving a response from the host;
Processing the transmission using snooping logic in the erasure coding logic includes substituting a device LBA in a read request with a host LBA;
Conveying the transmission to its destination via the PCIe switch includes conveying the response to a host.

Explanation 60.
In the method according to Explanation 59,
Processing the transmission using snooping logic in the erasure coding logic further includes replacing an identifier of an NVMe SSD with an identifier of a virtual storage device.

Explanation 61.
In the method according to Explanation 45,
Conveying the transmission to its destination via the PCIe switch includes conveying the transmission to a second PCIe switch to which an NVMe SSD is coupled;
The NVMe SSD is the destination.

Explanation 62.
In the method according to explanation 61,
A PCIe switch is in the first chassis and a second PCIe switch is in the second chassis.

Explanation 63.
In the method according to Explanation 45,
The method further includes initializing at least one NVMe SSD coupled to the PCIe switch for use with erasure coding.

Explanation 64.
In the method according to Explanation 45,
detecting whether a new NVMe SSD is connected to the PCIe switch;
adding a new NVMe SSD to the capacity of the virtual storage device.

Explanation 65.
In the method according to statement 64,
The method further includes initializing a new NVMe SSD for use with erasure coding.

Explanation 66.
In the method according to Explanation 45,
detecting a failed NVMe SSD coupled to the PCIe switch;
and performing erasure coding recovery of data stored on the failed NVMe SSD.

Explanation 67.
In the method according to statement 66,
detecting a replacement NVMe SSD for the failed NVMe SSD;
reconfiguring the failed NVMe SSD using the replacement NVMe SSD.

Explanation 68.
In the method according to Explanation 45,
detecting only NVMe SSDs without erasure coding functionality connected to the PCIe switch;
The method further includes enabling erasure coding logic at the PCIe switch.

Explanation 69.
In the method according to statement 68,
terminating enumeration of downstream PCIe buses of the PCIe switch.

Explanation 70.
In the method according to statement 68,
and reporting to the host a virtual storage device having a capacity based at least in part on the capacity of an NVMe SSD coupled to the PCIe switch and an erasure coding scheme.

Explanation 71.
In the method according to Explanation 45,
detecting that at least one non-storage device or at least one NVMe SSD with erasure coding functionality is coupled to the PCIe switch;
disabling the erasure coding logic within the PCIe switch.

Explanation 72.
In the method according to Explanation 45,
detecting that at least one non-storage device or at least one NVMe SSD with erasure coding functionality is coupled to the PCIe switch;
enabling the erasure coding logic in the PCIe switch;
and disabling at least one non-storage device or at least one NVMe SSD with erasure coding functionality.

Explanation 73.
In the method according to statement 72,
terminating downstream of PCIe bus enumeration from the PCIe switch.

Explanation 74.
In the method according to statement 72,
and reporting to the host a virtual storage device having a capacity based at least in part on the capacity of an NVMe SSD coupled to the PCIe switch and an erasure coding scheme.

Explanation 75.
In the method according to Explanation 45,
The method further includes configuring a PCIe switch including erasure coding logic using an erasure coding scheme.

Explanation 76.
In the method according to statement 75,
Configuring a PCIe switch including erasure coding logic using an erasure coding scheme includes configuring a PCIe switch including erasure coding logic using an erasure coding scheme using a BMC.

Explanation 77.
Embodiments of the inventive concept include an article that includes a non-transitory storage medium, the non-transitory storage medium stores a command, and when the command is executed by a machine,
receiving the transmission at the PCIe switch utilizing erasure coding logic;
processing the transmission using snooping logic in the erasure coding logic;
transmitting the transmission to the destination via the PCIe switch.

Explanation 78.
In the article according to Description 77,
Embodiments of the inventive concept include an article according to description 77, wherein the erasure coding logic includes one of Look-Aside erasure coding logic and Look-Through erasure coding logic.

Explanation 79.
In the article according to Description 77,
Processing the transmission using snooping logic in the erasure coding logic includes determining that the transmission includes a transmission controlled by the snooping logic;
Delivering the transmission to its destination via the PCIe switch includes transmitting the transmission to a PPU (Power Processing Unit).

Explanation 80.
In the article according to Description 77,
Processing the transmission using snooping logic with the erasure coding logic includes processing the transmission using snooping logic based at least in part on activated erasure coding logic.

Explanation 81.
In the article according to Description 77,
Receiving a transmission at a PCIe switch including erasure coding logic includes receiving a read request from a host;
Processing the transmission using snooping logic in the erasure coding logic includes reading a host's LBA and substituting a device's LBA in the request;
Communicating the transmission to its destination via the PCIe switch includes communicating a read request to an NVMe SSD.

Explanation 82.
In the article according to description 81,
Processing the transmission using snooping logic in the erasure coding logic further includes identifying an NVMe SSD to which the read request is to be transmitted.

Explanation 83.
In the article according to description 81,
Processing the transmission using snooping logic in the erasure coding logic includes accessing data requested by a host in a read request from the cache based at least in part on data present in the cache. including,
Substituting the LBA of the host with the LBA of the device in the read request includes substituting the LBA of the host with the LBA of the device in the read request based at least in part on data not present in the cache;
Communicating the transmission to its destination via the PCIe switch includes communicating a read request to an NVMe SSD based at least in part on data not present in cache.

Explanation 84.
In the article according to Description 77,
Receiving the transmission at the PCIe switch including erasure coding logic includes receiving a read request from the host;
Processing the transmission using snooping logic in the erasure coding logic includes substituting a host LBA with an LBA of a device in the read request;
Communicating the transmission to its destination via the PCIe switch includes communicating a read request to an NVMe SSD.

Explanation 85.
In an article according to description 84,
Processing the transmission using snooping logic in the erasure coding logic further includes identifying an NVMe SSD to which the read request is to be transmitted.

Explanation 86.
In the article according to description 84, the non-transitory storage medium stores commands, and when the commands are executed by a machine:
reading a stripe of blocks from at least one NVMe SSD;
merging the stripe of the block with the data in the write request to form an updated stripe of the block;
recording the updated stripe of blocks to the at least one NVMe SSD.

Explanation 87.
In an article according to description 86,
Merging the data in the write request includes calculating additional data to be recorded on at least one NVMe SSD in addition to the data in the write request.

Explanation 88.
In an article according to description 86,
reading a stripe of blocks from the cache based at least in part on a stripe of blocks present in the cache;
Reading a stripe of blocks from the at least one NVMe SSD includes reading a stripe of blocks from the at least one NVMe SSD based at least in part on a stripe of blocks that are not present in the cache.

Explanation 89.
In an article according to description 86,
Recording the updated stripe of the block to the at least one NVMe SSD includes recording the updated stripe of the block to a write buffer.

Explanation 90.
The article according to Description 89, wherein the non-transitory storage medium stores commands, and when the commands are executed by a machine:
After the updated stripe of the block is written to the write buffer, a step of responding to the host that the recording is completed is processed before the updated stripe of the block is recorded to the at least one NVMe SSD.

Explanation 91.
In the article according to Description 77,
Receiving the transmission at the PCIe switch including erasure coding logic includes receiving a response from the host;
Processing the transmission using snooping logic in the erasure coding logic includes substituting a device LBA in a read request with a host LBA;
Conveying the transmission to its destination via the PCIe switch includes conveying the response to a host.

Explanation 92.
In the article according to description 91,
Processing the transmission using snooping logic in the erasure coding logic further includes replacing an identifier of an NVMe SSD with an identifier of a virtual storage device.

Explanation 93.
In the article according to Description 77,
Communicating the transmission to its destination via the PCIe switch includes communicating the transmission to a second PCIe switch to which an NVMe SSD is coupled;
The NVMe SSD is the destination.

Explanation 94.
In the article according to description 93,
A PCIe switch is in the first chassis and a second PCIe switch is in the second chassis.

Explanation 95.
The article according to Description 77, wherein the non-transitory storage medium stores commands, and when the commands are executed by a machine:
Initializing at least one NVMe SSD coupled to a PCIe switch for use with erasure coding is processed.

Explanation 96.
The article according to Description 77, wherein the non-transitory storage medium stores commands, and when the commands are executed by a machine:
detecting whether a new NVMe SSD is connected to the PCIe switch;
adding a new NVMe SSD to the capacity of the virtual storage device.

Explanation 97.
The article according to statement 96, wherein the non-transitory storage medium stores commands, and when the commands are executed by a machine:
A step of initializing a new NVMe SSD for use with erasure coding is processed.

Explanation 98.
The article according to Description 77, wherein the non-transitory storage medium stores commands, and when the commands are executed by a machine:
detecting a failed NVMe SSD coupled to the PCIe switch;
performing erasure coding recovery of data stored on the failed NVMe SSD.

Explanation 99.
In the article according to Description 98, the non-transitory storage medium stores commands, and when the commands are executed by a machine:
detecting a replacement NVMe SSD for the failed NVMe SSD;
reconfiguring the failed NVMe SSD using the replacement NVMe SSD.

Explanation 100.
The article according to Description 77, wherein the non-transitory storage medium stores commands, and when the commands are executed by a machine:
detecting only NVMe SSDs without erasure coding functionality connected to the PCIe switch;
Enabling erasure coding logic at the PCIe switch.

Explanation 101.
In the article according to illustration 100, the non-transitory storage medium stores commands, and when the commands are executed by a machine:
A step of terminating enumeration of a downstream PCIe bus of a PCIe switch is processed.

Explanation 102.
In the article according to illustration 100, the non-transitory storage medium stores commands, and when the commands are executed by a machine:
Reporting to a host a virtual storage device having a capacity based at least in part on a capacity of an NVMe SSD coupled to a PCIe switch and an erasure coding scheme is processed.

Explanation 103.
The article according to Description 77, wherein the non-transitory storage medium stores commands, and when the commands are executed by a machine:
detecting that at least one non-storage device or at least one NVMe SSD with erasure coding functionality is coupled to the PCIe switch;
disabling the erasure coding logic in the PCIe switch.

Explanation 104.
The article according to Description 77, wherein the non-transitory storage medium stores commands, and when the commands are executed by a machine:
detecting that at least one non-storage device or at least one NVMe SSD with erasure coding functionality is coupled to the PCIe switch;
enabling the erasure coding logic in the PCIe switch;
disabling at least one non-storage device or at least one NVMe SSD having erasure coding capability.

Explanation 105.
In the article according to description 104, the non-transitory storage medium stores commands, and when the commands are executed by a machine:
A step of terminating downstream of PCIe bus enumeration from a PCIe switch is processed.

Explanation 106.
In the article according to description 104, the non-transitory storage medium stores commands, and when the commands are executed by a machine:
Reporting to a host a virtual storage device having a capacity based at least in part on a capacity of an NVMe SSD coupled to a PCIe switch and an erasure coding scheme is processed.

Explanation 107.
The article according to Description 77, wherein the non-transitory storage medium stores commands, and when the commands are executed by a machine:
A step of configuring a PCIe switch including erasure coding logic using an erasure coding scheme is processed.

Explanation 108.
In the article according to Description 107,
Configuring a PCIe switch that includes erasure coding logic that uses an erasure coding scheme includes configuring a PCIe switch that includes erasure coding logic that uses an erasure coding scheme using a BMC.

Explanation 109.
Embodiments of the inventive concept encompass systems. The system includes:
NVMe SSD and
an FPGA that implements a function to support the NVMe SSD;
a PCIe switch;
The capabilities are obtained from a collection including data acceleration, data deduplication, data integrity, data encryption and data compression;
The PCIe switch communicates with the FPGA and the NVMe SSD.

Explanation 110.
In the system according to Description 109,
The FPGA and the NVMe SSD are inside a common housing.

Explanation 111.
In a system according to description 110,
The PCIe switch is external to a common housing containing the FPGA and the NVMe SSD.

Explanation 112.
In the system according to Description 109,
the PCIe switch is coupled to the FPGA;
The FPGA is coupled to the NVMe SSD.

Explanation 113.
In the system according to Description 109,
the PCIe switch is coupled to the NVMe SSD;
The NVMe SSD is coupled to an FPGA.

Explanation 114.
In the system according to Description 109,
the PCIe switch includes erasure coding logic;
The erasure coding logic includes an erasure coding controller.

Explanation 115.
In a system according to description 114,
The erasure coding logic includes one of Look-Aside erasure coding logic and Look-Through erasure coding logic.

Explanation 116.
In a system according to description 114,
The erasure coding logic is operative to return a response to a read request from the host based at least in part on the data requested by the read request residing in the cache.

Explanation 117.
In a system according to description 116,
The erasure coding logic further includes a cache.

Explanation 118.
In a system according to description 116,
the PCIe switch is located within the chassis;
The chassis includes memory used as a cache by erasure coding logic.

Explanation 119.
In a system according to description 114,
The rager coding logic operates to return a response to the write request to the host before completing the write request.

Explanation 120.
In the system according to description 119,
The PCIe switch further includes a write buffer,
The erasure coding controller is operative to store data in a write request in a write buffer.

Explanation 121.
In a system according to description 114,
The rage coding logic includes Look-Aside erasure coding logic and Look-Through erasure coding logic, including snooping logic.

Explanation 122.
In a system according to description 114,
Erasure coding logic operates to intercept control transmissions received at the PCIe switch and transmit the control transmissions to the PPU.

Explanation 123.
In a system according to description 114,
Erasure coding logic is operative to intercept data transmissions received at a PCIe switch from a host and to substitute in the data transmission a host LBA used by the host with a device LBA used in the NVMe SSD. do.

Explanation 124.
In the system according to description 123,
Erasure coding logic operates to direct the NVMe SSD to the data transmission.

Explanation 125.
In a system according to description 114,
The erasure coding logic is operative to intercept data transmissions received at the PCIe switch from the NVMe SSD and to substitute the device LBA used by the NVMe SSD with the host LBA used by the host in the data transmission. .

Explanation 126.
In a system according to description 114,
Erasure coding logic defines a virtual storage device across the NVMe SSD and the second NVMe SSD.

Explanation 127.
In a system according to description 114,
The PCIe switch is operative to enable erasure coding logic based at least in part on an NVMe SSD that can be used with the erasure coding logic.

Explanation 128.
In a system according to description 114,
The device further includes a second device coupled to the PCIe switch that includes erasure coding logic.

Explanation 129.
In a system according to description 128,
The second device includes at least one of a storage device, an SSD including an FPGA, and a GPU.

Explanation 130.
In a system according to description 128,
the second device is unusable with erasure coding logic;
The PCIe switch is operative to disable erasure coding logic based at least in part on a second device being unavailable for use with the erasure coding logic.

Explanation 131.
In a system according to description 128,
the second device is unusable with erasure coding logic;
The PCIe switch is operative to enable the erasure coding logic based at least in part on the NVMe SSD that is usable with the erasure coding logic and to enable access to the second device without using the erasure coding logic. .

Explanation 132.
In a system according to description 128,
the second device is unusable with erasure coding logic;
The PCIe switch is operative to enable the erasure coding logic and disable access to the second device based at least in part on the NVMe SSD available with the erasure coding logic.

Explanation 133.
Embodiments of the inventive concept encompass systems. The system includes:
NVMe SSD and
an FPGA including a first FPGA portion implementing one or more functions supporting the NVMe SSD and a second FPGA portion implementing a PCIe switch;
the one or more functions are obtained from a set including at least one of data acceleration, data deduplication, data integrity, data encryption and data compression;
the PCIe switch communicates with the FPGA and the NVMe SSD;
The FPGA and the NVMe SSD are inside a common housing.

Explanation 134.
In the system according to description 133,
The PCIe switch includes erasure coding logic including an erasure coding controller.

Explanation 135.
In a system according to description 134,
The erasure coding logic defines a virtual storage device that spans at least two parts of the NVMe SSD.

Explanation 136.
In a system according to description 134,
The erasure coding logic defines a virtual storage device spanning an NVMe SSD and a second NVMe SSD.

Explanation 137.
In a system according to description 136,
The second NVMe SSD is inside a common housing.

Explanation 138.
In a system according to description 136,
The second NVMe SSD is external to the common housing.

Explanation 139.
In a system according to description 134,
The erasure coding logic includes at least one of look-aside erasure coding logic and look-through erasure coding logic.

Explanation 140.
In a system according to description 134,
The erasure coding logic is operative to return a response to a read request from a host based at least in part on the data requested by the read request residing in the cache.

Explanation 141.
In a system according to description 140,
The FPGA further includes the cache.

Explanation 142.
In a system according to description 140,
the common housing is located within the chassis;
The chassis includes memory used as the cache by the erasure coding logic.

Explanation 143.
In a system according to description 134,
The system wherein the erasure coding logic is operative to return a response to the write request to a host before completing the write request.

Explanation 144.
In the system according to description 143,
The FPGA further includes a write buffer;
The erasure coding controller is operative to store data in a write request in a write buffer.

Explanation 145.
In a system according to description 134,
The erasure coding logic includes look-aside erasure coding logic and look-through erasure coding logic including snooping logic.

Explanation 146.
In a system according to description 145,
The snooping logic is operative to intercept control transmissions received at the PCIe switch and communicate the control transmissions to the PPU.

Explanation 147.
In a system according to description 134,
The erasure coding logic operates to intercept the transmission of data received by the PCIe switch from the host and replace the host LBA used by the host with the device LBA used by the NVMe SSD in the data transmission. .

Explanation 148.
In the system according to description 147,
Erasure coding logic operates to instruct the NVMe SSD to transmit data.

Explanation 149.
In a system according to description 134,
The erasure coding logic operates to intercept data transmissions received by the PCIe switch from the NVCe SSD and substitute the device LBA used by the NVMe SSD for data transmission with the host LBA used by the host.

Explanation 150.
In a system according to description 134,
The PCIe switch including erasure coding logic is operative to enable the erasure coding logic based at least in part on NVMe SSDs available with the erasure coding logic.

Explanation 151.
In a system according to description 134,
The PCIe switch including erasure coding logic is operative to disable the erasure coding logic based at least in part on NVMe SSDs that cannot be used with erasure coding logic.

Explanation 152.
Embodiments of the inventive concept encompass systems. The system includes:
NVMe SSD and
a PCIe switch including erasure coding logic;
The PCIe switch is
an external connector that allows the PCIe switch to communicate with a processor;
at least one connector that allows the PCIe switch to communicate with the NVMe SSD;
a PPU that constitutes the PCIe switch;
an erasure coding controller including a circuit for applying an erasure coding scheme to data stored in the NVMe SSD.

Explanation 153.
In a system according to description 152,
The system further includes a second NVMe SSD;
The PCIe switch including erasure coding logic includes a second connector that allows the PCIe switch including erasure coding logic to communicate with a second NVMe SSD.

Explanation 154.
In a system according to description 152,
The system includes:
a second NVMe SSD;
further comprising a second PCIe switch;
The second PCIe switch is
a second external connector that allows the second PCIe switch to communicate with the processor;
a second connector that allows the second PCIe switch to communicate with the second NVMe SSD;
a third connector that allows the second PCIe switch to communicate with the PCIe switch that includes the erasure coding logic;
the PCIe switch including the erasure coding logic includes a fourth connector that allows the PCIe switch including the erasure coding logic to communicate with the second PCIe switch;
The erasure coding scheme is applied to data stored on the NVMe SSD and the second NVMe SSD.

Explanation 155.
In a system according to description 154,
The second PCIe switch further includes disabled second laser coding logic.

Explanation 156.
In a system according to description 152,
The erasure coding logic includes at least one of look-aside erasure coding logic and look-through erasure coding logic.

Explanation 157.
In a system according to description 152,
The erasure coding logic is operative to return a response to a read request from a host based at least in part on the data requested by the read request residing in the cache.

Explanation 158.
In the system according to description 157,
The erasure coding logic further includes a cache.

Explanation 159.
In the system according to description 157,
the PCIe switch including erasure coding logic is located within the chassis;
The chassis includes memory used as a cache by erasure coding logic.

Explanation 160.
In a system according to description 152,
The erasure coding logic operates to return a response to the read request to the host before completing the read request.

Explanation 161.
In a system according to description 160,
The PCIe switch including erasure coding logic further includes a write buffer;
The erasure coding controller is operative to store data in the write request in the write buffer.

Explanation 162.
In a system according to description 152,
The erasure coding logic includes Look-Aside erasure coding logic and Look-Through erasure coding logic, including snooping logic.

Explanation 163.
In a system according to description 152,
The erasure coding logic is operative to intercept control transmissions received at the PCIe switch and communicate the control transmissions to a PPU.

Explanation 164.
In a system according to description 152,
The erasure coding logic is configured to intercept data transmissions received by the PCIe switch from a host and substitute the host LBA used by the host with the device LBA used by the NVMe SSD in the data transmission. works.

Explanation 165.
In a system according to description 164,
The erasure coding logic operates to direct the data transmission to the NVMe SSD.

Explanation 166.
In a system according to description 152,
The erasure coding logic intercepts data transmissions received by the PCIe switch from the NVMe SSD and substitutes a host LBA used by the host for a device LBA used by the NVMe SSD in the data transmission. It works like that.

Explanation 167.
In a system according to description 152,
The erasure coding logic defines a virtual storage device spanning the NVMe SSD and a second NVMe SSD.

Explanation 168.
In a system according to description 152,
The PCIe switch with erasure coding logic is operative to enable erasure coding logic based at least in part on NVMe SSDs available with erasure coding logic.

Explanation 169.
In a system according to description 152,
The device further includes a second device coupled to the PCIe switch that includes erasure coding logic.

Explanation 170.
In the system according to description 169,
The second device includes at least one of a storage device, an SSD with an FPGA, and a GPU.

Explanation 171.
In the system according to description 169,
a second device is unusable with the erasure coding logic;
The PCIe switch including erasure coding logic is operative to disable the erasure coding logic based at least in part on the second device being unavailable for use with erasure coding logic.

Explanation 172.
In the system according to description 169,
the second device is unusable with erasure coding logic;
The PCIe switch including erasure coding logic enables erasure coding logic based at least in part on an NVMe SSD usable with erasure coding logic, and enables erasure coding logic to provide data to the second device without using erasure coding logic. Operates to enable access.

Explanation 173.
In the system according to description 169,
the second device is unusable with erasure coding logic;
The PCIe switch having erasure coding logic is operative to enable erasure coding logic and disable access to the second device based at least in part on an NVMe SSD available with erasure coding logic.

In conclusion, in view of various modifications of the embodiments described herein, this detailed description and the attached material are intended by way of example only and should not be considered as limiting the scope of the invention. must not. Therefore, what is claimed as the present invention is all possible modifications within the scope and spirit of the following claims and their equivalents.

15 Device functions 105 Machine 110 Processor 115 Memory 120 Memory controller 125, 320, 605, 1005 PCIe switch 130, 130-1, 130-2, 130-3, 130-5, 130-6 Storage device 135 Device driver 205 Clock 210 Network connector 215 Bus 220 User interface 225 I/O engine 305 Midplane 310, 315 Switch board 325, 330 Baseboard management controller (BMC)
405, 410 Erasure coding method 505 Connector 510-1, 510-2, 510-3, 510-4, 510-5, 510-6 PCIe-to-PCIe stack 515 PCIe switch core 520 PPU (Power Processing Unit)
525 Snooping logic 530 Erasure coding controller 535-1, 535-2, 535-3, 535-4, 535-5, 535-6 Capture interface 540 Multiplexer 545 Cache 550 Write buffer 555 Erasure coding enable signal 705 FPGA

Claims (18)

A system,
NVMe (Non-Volatile Memory Express) SSD (Solid State Drive) and
A PCIe (Peripheral Component Interconnect Express) switch including erasure coding logic;
a second NVMe SSD;
a second PCIe switch;
The PCIe switch is
an external connector that enables the PCIe switch to communicate with a processor;
at least one connector that enables the PCIe switch to communicate with the NVMe SSD;
a power processing unit (PPU) that configures the PCIe switch;
an erasure coding controller having a circuit for applying an erasure coding scheme to data stored in the NVMe SSD;
The second PCIe switch is
a second external connector that enables the second PCIe switch to communicate with the processor;
a second connector that enables the second PCIe switch to communicate with the second NVMe SSD;
a third connector that enables the second PCIe switch to communicate with the PCIe switch that includes the erasure coding logic;
the PCIe switch including the erasure coding logic includes a fourth connector that enables the PCIe switch including the erasure coding logic to communicate with the second PCIe switch;
The system is characterized in that the erasure coding method is applied to data stored in the NVMe SSD and the second NVMe SSD. 2. The erasure coding logic is operative to return a response to a read request from a host based at least in part on data requested by the read request residing in a cache. system. The system of claim 1, wherein the erasure coding logic is operative to return a response to the write request to a host before completing the write request. The erasure coding logic intercepts data transmissions received by the PCIe switch from a host and converts the host LBA (Logical Block Address) used by the host in the data transmission to the device LBA used by the NVMe SSD. 2. The system of claim 1, wherein the system operates to replace. The system of claim 1, further comprising a second device coupled to the PCIe switch that includes the erasure coding logic. The second device includes at least one of a storage device and a non-storage device having native erasure coding logic;
6. The system of claim 5, wherein the PCIe switch including the erasure coding logic is operative to disable the erasure coding logic based at least in part on the second device. The second device includes at least one of a storage device and a non-storage device having native erasure coding logic;
6. The system of claim 5, wherein the PCIe switch including the erasure coding logic is operative to enable the erasure coding logic and disable access to the second device. The system of claim 1, wherein the second PCIe switch further includes disabled second erasure coding logic. 3. The system of claim 2, wherein the erasure coding logic further includes the cache. the PCIe switch including the erasure coding logic is located within a chassis;
3. The system of claim 2, wherein the chassis includes memory used as the cache by the erasure coding logic. The PCIe switch including the erasure coding logic further includes a write buffer;
4. The system of claim 3, wherein the erasure coding controller is operative to store data in the write request in the write buffer. The erasure coding logic includes Look-Aside erasure coding logic;
The system of claim 1, wherein the Look-Aside erasure coding logic includes snooping logic. 5. The system of claim 4, wherein the erasure coding logic is further operative to direct the data transmission to the NVMe SSD. The erasure coding logic is configured to intercept data transmissions received at the PCIe switch from the NVMe SSD and substitute a device LBA used by the NVMe SSD with a host LBA used by a host in the data transmission. 2. The system of claim 1, wherein the system operates to: The system of claim 1, wherein the erasure coding logic defines a virtual storage device across the NVMe SSD and a second NVMe SSD. The PCIe switch including the erasure coding logic is operative to enable the erasure coding logic based at least in part on the NVMe SSD being a storage device that does not include native erasure coding logic. The system of claim 1. 6. The system according to claim 5, wherein the second device includes at least one of a storage device, an SSD including a field programmable gate array (FPGA), and a graphics processing unit (GPU). The second device includes at least one of a storage device and a non-storage device having native erasure coding logic;
6. The system of claim 5, wherein the PCIe switch including the erasure coding logic is operative to enable the erasure coding logic and disable access to the second device.

Images