TW202020675A - Computuer system - Google Patents

Abstract

A topology is disclosed. The topology may include at least one Non-Volatile Memory Express (NVMe) Solid State Drive (SSD), a Field Programmable Gate Array (FPGA) to implement one or more functions supporting the NVMe SSD, such as data acceleration, data deduplication, data integrity, data encryption, and data compression, and a Peripheral Component Interconnect Express (PCIe) switch. The PCIe switch may communicate with both the FPGA and the NVMe SSD.

Description

FPGA+SSD embedded PCIE valve to support erasure code data protection method

The concept of the present invention relates generally to computer systems, and more specifically, to erasing codes in peripheral component interconnect (Express Peripheral Interconnect Express, PCIe) switches.

At present, most solid state drives (SSD) based on Non-Volatile Memory Express (NVMe) protected by Redundant Array of Independent Disk (RAID) are protected by external PCIe plug-in card (Add-In-Card, AIC). To optimize the bus bandwidth between the central processing unit (CPU) of the host and the AIC RAID controller, the bus usually supports X16 PCIe lanes. However, due to the physical limitations of the standard form factor of PCIe cards, each AIC RAID controller only supports a small number of U.2 connectors (currently the preferred connector for NVMe SSDs): usually only supports two or four U.2 connector.

In order to support up to 24 NVMe SSDs in a 2U chassis, 6 AIC RAID controllers are needed, which will form 6 different RAID domains. This configuration increases the cost and complexity of managing 6 RAID domains. In addition, the current cost of each AIC RAID controller is close to $400. Therefore, even for the entire RAID solution of a single 2U mainframe, the AIC RAID controller alone exceeds $2,400, which does not include the cost of the NVMe SSD.

Due to the lack of cost-effective RAID data protection for large data sets, the adoption of NVMe SSDs in the enterprise market is limited. Software RAID solutions are suitable for relatively small data sets, but not for big data.

There are other problems when using the AIC RAID controller:

1) As mentioned above, having multiple RAID domains inside the mainframe increases management complexity.

2) As an inevitable result of the complexity of RAID domain management, the main chassis does not have a single RAID domain, but it would be preferable to have a single RAID domain.

3) The central processing unit (CPU) needs to support a large number of PCIe channels: 16 PCIe channels per AIC RAID controller × 6 AIC RAID controllers per chassis = 96 PCIe channels only for the AIC RAID controller. Only the much more expensive high-end CPUs currently support so many PCIe channels.

4) Since each AIC RAID controller may consume 25 watts, 6 AIC RAID controllers will increase the power consumption per chassis by up to 150 watts.

5) The mainframe often has only a few PCIe slots, which may limit the number of AIC RAID controllers that can be added and indirectly reduce the number of NVMe SSDs in the mainframe that can be protected by RAID.

6) Software RAID solutions often support relatively few RAID levels and increase the CPU overhead.

7) When used over a network, SSD access may be slower due to the time required to send data access between networks. In addition, in some examples, networked memory may require a software RAID implementation, which will increase the burden on the CPU.

There is still a need for a way to support erasure coding of mass storage devices without being restricted by AIC RAID controllers and software RAID solutions. [Object of the invention]

Exemplary embodiments of the present disclosure may provide a system that uses erasure codes to support data protection.

Exemplary embodiments provide a system that may include a non-volatile storage express (NVMe) solid-state drive (SSD), a field programmable gate array (FPGA) that implements functions that support NVMe SSD, and a peripheral component connection express (PCIe )switch. The functions that support NVMe SSD come from a set of functions including data acceleration, data deduplication, data integrity, data encryption, and data compression. PCIe switch communicates with FPGA and NVMe SSD.

Another exemplary embodiment provides a system that may include a non-volatile storage flash (NVMe) solid-state drive (SSD) and a field programmable gate array (FPGA). The field programmable gate array (FPGA) includes a first One FPGA part and the second FPGA part. The first FPGA part implements functions that support NVMe SSD. The second FPGA part implements peripheral component connection express (PCIe) switches. Support for NVMe SSD comes from a set of functions including data acceleration, deduplication, data integrity, data encryption and data compression. PCIe switch communicates with FPGA and NVMe SSD. FPGA and NVMe SSD are located inside the common housing.

Yet another exemplary embodiment provides a system that may include a non-volatile storage express (NVMe) solid-state drive (SSD) and a peripheral component-connected express (PCIe) switch with erasure coding logic. The PCIe switch may include an external connector that enables the PCIe switch to communicate with the processor, at least one connector that enables the PCIe switch to communicate with the NVMe SSD, a power processing unit (PPU) for configuring the PCIe switch, and erasure The coding controller, the erasure coding controller includes circuitry for applying the erasure coding scheme to the data stored on the NVMe SSD. [Effect of invention]

According to an embodiment of the present invention, a PCIe switch including Look-Aside Eraser Coding logic (Look-Aside Eraser Coding logic) is used to move the erasure code closer to the storage device, which can reduce the time required to move data back and forth. In addition, by placing the erasure code controller with the PCIe switch, expensive RAID plug-in cards are no longer required, and a larger array can be used (even spanning multiple mainframes).

Reference will now be made in detail to embodiments of the inventive concept, examples of which are shown in the drawings. In the following detailed description, many specific details are set forth to enable a thorough understanding of the inventive concept. However, it should be understood that those of ordinary skill in the art can practice the inventive concept without these specific details. In other instances, well-known methods, procedures, components, circuits, and networks have not been described in detail so as not to unnecessarily obscure aspects of the embodiments.

It will be understood that although the terms first, second, etc. may be used herein to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish each element. For example, without departing from the scope of the inventive concept, the first module may be referred to as the second module, and similarly, the second module may be referred to as the first module.

Herein, the terminology used in the description of the inventive concept is only used to illustrate specific embodiments, and is not intended to limit the inventive concept. Unless the context clearly indicates otherwise, the singular forms "a/an" and "the" as used in the description of the concept of the invention and the accompanying patent application are intended to also include the plural form . It will also be understood that the term "and/or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items. It will further be understood that when the term "comprises and/or comprising" is used in this specification, it indicates the presence of the claimed features, integers, steps, operations, elements and/or elements, but does not exclude one or more The presence or addition of other features, integers, steps, operations, elements, elements, and/or groups thereof. The elements and features shown in the drawings are not necessarily drawn to scale.

Field programmable gate array (FPGA) has sufficient intelligence, computing resources, and high-speed input/output (I/O) connections to implement redundant array of independent disks (RAID)/erase codes when necessary Erasure Code parity generation and data discovery. FPGA+Solid State Drive (SSD) may require embedded peripheral components connected to a PCIe switch to support more co-controllers/auxiliary processors, such as one or more SSDs, Graphical Processing Unit (GPU), Zhang Quantity processing unit (Tensor Processing Unit, TPU), etc. Multiple auxiliary processors also require more NAND flash memory channels.

Embodiments of the present invention support erasure codes in PCIe switches inside FPGAs. Embodiments of the inventive concept may also enable a user to remotely configure a RAID engine (internal to the FPGA) through a Baseboard Management Controller (BMC). Users can use these standard interfaces (such as PCIe (used as a control plane) or System Management Bus (System Management Bus, SMBus)) to pre-configure on-chip RAID (RAID-on-a-chip, RoC) or erase code Controller. For users who lease computing resources, it can be useful to be able to configure the storage device in this way: when completed, the user may want to quickly destroy the data before the next user may use the same computing resources. In this case, the BMC can send an erase command to all embedded PCIe switches inside multiple FPGA+SSDs. Once the erase command is received, the RoC/Erase Code Controller of the FPGA will erase both the data specified by the command logic block address (LBA) range and the parity data.

Today, PCIe switches expose virtual switches or virtual grouping, where more than one switch is exposed to managers. When the network, CPU-GPU, FPGA, and memory behind these virtual domains can be grouped together, these configurations are useful in a virtualized environment. Such virtual grouping can be applied to memory in one embodiment by creating a RAID subgroup exposed to the user group for the virtualized environment, or as an alternative for RAID grouping (eg RAID 10, RAID 50, RAID 60 etc.). These layered RAID groups create small groups and apply additional RAID layers on top to create larger RAID solutions. The virtual switch manages smaller RAID groups, while the master switch manages the overall RAID configuration.

Since the data protection scheme is enabled and management is brought closer to the storage unit, the solution provides beneficial effects with important distinguishing characteristics in the enterprise environment and the data center environment. Embodiments of the inventive concept provide higher density and performance with lower power consumption.

The solution can consist of an embedded PCIe switch with integrated RoC or an erasure code controller in the data path between the host and SSD. The PCIe switch + RoC component can be managed and configured and controlled by the BMC, and the interface can be exposed to the software for specific configuration before being released to new users.

When running in erasure code/RAID mode, all new non-volatile storage express (NVMe) or fabric-based NVMe (NVMe over Fabric, NVMe-oF) traffic to and from embedded PCIe switches may be affected by RoC or Erasing code controller (which may be referred to as bystander RoC or erasing code controller) detection. The RoC or erasure code controller can determine whether the data in the traffic causes a cache hit on its local cache. If there is a cache hit, there is no need to forward the transaction (read or write) to the appropriate SSD. The requested reading data can be provided directly by RoC cache. The written data will be directly updated to RoC's local cache and marked as "modified" or "dirty" data.

For SSDs, parity can be distributed between connected SSDs. For example, if RAID 4 is selected, the last SSD can only be used to store parity, while the other SSDs are used to store data.

By having an external PCIe switch between the host and the SSD device, virtual I/O addresses can be supported. In this case, a primary RoC that is part of the host PCIe switch can virtualize all SSD addresses. In other words, the address and device are not visible to the host operating system (OS). In such an embodiment of the inventive concept, peer-to-peer transactions between at least two SSDs that are peers are allowed and supported. This option can enhance some forms of SSD redundancy and/or availability by striping across more than one SSD. In this mode, the embedded RoC or erasure code controller in the FPGA can be disabled (if present). The only RoC/Erase Code Controller that is enabled is located in the host PCIe switch.

If the storage device operates in single-device mode, all new NVMe/PCIe traffic may be forwarded to the SSD with the requested data.

If the pairing mode is enabled, the RoC/erase code controller can determine whether the address of the requested data belongs to its own base address register (BAR) field. In this case, the transaction can be completed by the local RoC. For write transactions, you can use a posted write buffer (posted write buffer) or write cache (using some embedded static random access memory (static random access memory, SRAM) or dynamic random access memory ( dynamic random access memory, DRAM)). If there is a write cache hit (the previous write has occurred and the data is still stored in the write cache buffer), the processing depends on the write cache strategy. For example, if the cache strategy is write-back, the write command will be completed and terminated by the RoC cache. If the cache strategy is write-through, the write command will be completed when the written data has been successfully transferred to the drive. In this case, once the written data has been successfully updated to its local cache, RoC can terminate the write command to the host.

RoC can virtualize a bunch of devices it claims and present the devices as a single device or fewer devices as a protection scheme against data or device failures. Data protection schemes can be distributed across a bunch of devices in essence, so that when data is lost on any device, data can be reconstructed from other devices. RAID and erasure coding (EC) are common data protection schemes that use distributed algorithms to protect against such losses.

To virtualize the device under RoC, the device can be terminated at RoC and not visible to the host. That is, the PCIe switch can be connected to all known devices, and the RoC can be connected to the switch. To manage devices, RoC can discover and configure individual devices through PCIe switches. Alternately, RoC can be transparent in the default/factory mode and allow the host software to configure RoC. The host software can be customized to work with PCIe switch + RoC hardware. Once configured, RoC can terminate the device and make it invisible to the host.

PCIe switch + RoC can be configured for RAID mode and EC mode in a variety of ways. There can be additional PCIe switches downstream to create a larger fan-out configuration to support more devices. In addition, more than one such hardware combination can be associated together to form a larger setup. For example, 2 PCIe switches + RoC can work together to form an alternative configuration. Alternately, the two PCIe switches + RoC can work independently.

When the PCIe switch + RoC works separately, the host combines each RoC and PCIe switch to generate an entity as a separate device. Here, the host may have a standard OS driver, and the standard OS driver will see all SSDs virtualized by RoC. For example, suppose there are 6 SSDs clustered under the PCIe switch and 1 SSD is exposed to the host by RoC; the second RoC and PCIe switch combination can also expose similar settings to the host. The host discovers 2 SSDs (one for each device) for all RoC controller devices. Each RoC controller can expose a separate device space for each exposed SSD. The host may not see all devices that support the exposed SSD and are behind the exposed SSD. RoC manages hardware I/O paths through PCIe switches.

This method can be used in an active-passive setup (active-passive setup), where the second controller is a backup path used to prevent a failure of the path of the first controller. The host only actively uses the first controller here, and no I/O is sent to the second RoC controller. If the active-passive setting is used, the 2 RoC controllers can copy data internally. As in the RAID 1 data protection setup, this can be done by the first active controller sending all writes to the second RoC controller.

There may be a second active-passive setting, where the second RoC and PCIe switch may not have any SSD of its own and may only be a backup controller path. In this case, since the two RoC controllers involve the same set of SSDs, no I/O may be sent between them. This is the standard active-passive setting.

The SSDs behind each RoC may also be incompatible with each other. In this case, the two SSDs are regarded as separate SSDs, and the protection scheme is not shared between them.

In yet another usage, both paths can be used in active-active setup. This type of setting can be used for load-balancing purposes. Here, the host can use a specific software layer to distribute I/O workload to use these two paths. The two RoC controllers can coordinate their write operations between them to keep the two SSDs in sync. In other words, each SSD from each RoC controller can contain the same data as in the RAID 1 setup.

In yet another configuration, the 2 RoC controllers communicate in a manner that keeps their I/O distributed in a custom setup. Here, the host uses only one RoC controller: another RoC controller is connected to the first RoC controller. The first RoC controller may expose one or more virtual NVMe SSDs to the host. The 2 RoCs may be set to divide an odd LBA space and an even LBA space therebetween. Since NVMe uses a pull model for data from the device side, the host only sends commands to the SSD exposed by the first RoC controller. The RoC controller may send a copy of the message to the second RoC controller via its side channel connection. The RoC controller can be set to serve only odd or even LBAs, stripes, zones, etc. This setup provides internal load balancing that does not need to be managed by the host and can be transparently managed by a combination of RoC and PCIe switches. Individual RoC controllers can handle only odd LBA ranges or even LBA ranges and satisfy requests for host buffers. Since both RoC controllers access the host, they can fill in data for odd or even pairs.

For example, the host may send a command to read four consecutive LBA 0, LBA 1, LBA 2, LBA 3 to the first RoC controller, and the first RoC controller sends a copy to the second RoC controller. Next, the first RoC controller reads the data of LBA 0 and LBA 2 from the first two SSDs on its PCIe switch, while the second RoC controller reads the data from LBA 1 and LBA from the first two SSDs on its PCIe switch 3. Information. The second RoC controller may then report that it has completed its operation to the first RoC controller, and the first RoC controller may then report the transaction completion to the host.

Odd/even LBA/strip/zone pairs are examples that can be applied to other load distribution usages.

Embodiments of the inventive concept can support SSD failure, removal, and hot addition. When the SSD does not work properly or is removed from its slot, the RoC in the PCIe switch needs to detect this situation. When the PCIe switch detects this situation, RoC can start a rebuild operation on the SSD that has failed or was removed. RoC can also handle any I/O operations during the reconstruction cycle by prioritizing the data from the associated stripes.

There are at least two ways to report SSD failure or removal to RoC in PCIe switches. In one embodiment of the inventive concept, all SSDs have a presence pin (Present pin) connected to the BMC. When the SSD is pulled out of the case, the BMC will detect the removal. Next, BMC reports the affected slot number to the RoC in the PCIe switch. BMC can also periodically monitor the health of SSDs. If the BMC detects any fatal error condition reported by the SSD, the BMC may decide to stop the SSD from serving. Then, the BMC can report the failed slot number to the RoC, so that a new SSD can be rebuilt.

In another embodiment of the inventive concept, the PCIe switch may be able to support hot plugs, in which all SSDs are connected via PCIe sideband signals and can detect specific error conditions. The PCIe switch can detect when the SSD is pulled out or added in, or when the PCIe link to the SSD is no longer connected. In this error scenario, the RoC in the PCIe switch can isolate the faulty SSD, or the BMC can isolate the faulty SSD by disabling the power supply of the faulty drive and immediately starting to rebuild the drive.

When asserted, the presence (PRSNT#) pin of each U.2 connector can indicate the presence of a new device in the main chassis. The signal is connected to the PCIe switch and/or BMC. RoC can appropriately configure the new drive into its existing domain based on the current data protection strategy.

All incoming traffic from the host needs to be forwarded to probe P2P and address conversion logic (physical to logical). During PCIe enumeration, all configuration loops of all ports need to be forwarded to the probe P2P logic. Depending on the selected operating mode, the behavior of the PCIe switch with RoC is defined as follows:

RoC can also be located in a line between the PCIe switch and the main processor. In such an embodiment of the inventive concept, RoC may be referred to as Look-Through RoC. When using perspective RoC, if the PCIe switch operates like a normal PCIe switch, RoC is disabled and becomes a re-timer for all ports. In this case, all upstream ports are allowed to connect as in normal use.

If RoC is enabled, a small number of non-transparent bridge (NTB) ports will be connected to the host. In this case, RoC can virtualize the new address into a logical address according to the selected RAID or erasure coding level.

Regardless of whether RoC is watching RoC or looking through RoC, all incoming read/write memory requests can be checked against RoC's local cache to determine whether the cache hit or cache miss. If there is a cache hit, the requested read data can be provided by the RoC local cache memory instead of the SSD. For memory write hits, the write data can be immediately updated to the cache memory. The same written data can be updated to SSD later. Such an implementation can reduce the total latency of memory writing, thereby improving system performance.

If there is a cache miss, the RoC controller can determine which SSD is the correct drive to access the data.

To address PCIe devices, the PCIe device must be enabled by mapping to the system's I/O port address space or memory mapped address space. The system's firmware, device driver, or operating system program programs the base address register (BAR) to inform the device of its address mapping by writing configuration commands to the PCI controller. Since all PCIe devices are in an inactive state when the system is reset, they will not be assigned an address that the operating system or device driver can use to communicate with it. The basic input/output system (basic input/output system, BIOS) or operating system uses the PCIe controller to initialize the PCIe slots geographically (for example, using the Initialization Device Select (IDSEL) signal for each slot). Address the first PCIe slot, second PCIe slot, or third PCIe slot on the motherboard.

Since the BIOS or the operating system does not have a direct method to determine which PCIe slots have devices installed (nor does it have a direct method to determine which functions are implemented by the devices), the PCI bus will be enumerated. Bus enumeration can be implemented by attempting to read each combination of bus number and device number from the vendor identification (ID) and device identification (VID/DID) registers at function 15 of the device. Note that the device number different from DID is only the serial number of the device on this bus. In addition, after detecting a new bridge, a new bus number is defined, and device enumeration restarts at the device number zero.

If no response is received from function 15 of the device, the bus master can perform an abort and return an all-bits-on value (hexadecimal FFFFFFFF), which is invalid VID/DID value. In this way, the device driver can understand that the specified bus/device_number/function (B/D/F) does not exist. Therefore, when the reading of a function ID of zero for a given bus/device causes the main controller (starter) to abort, the device driver can conclude that there is no working device on this bus (requires the device to implement Function number zero). In this case, there is no need to read the remaining function numbers (1 to 7), because it will also not exist.

When the reading of the specified B/D/F combination is successful for the vendor ID register, the device driver will know that the device exists. The device driver can write all 1s to its BAR and read back the requested memory size of the device in coded form. The design implies that all address space sizes are powers of 2 and are naturally aligned.

At this time, the BIOS or the operating system can program the memory mapped address and the I/O port address into the BAR configuration register of the device. As long as the system remains on, these addresses will remain valid. Once the power is turned off, all these settings will be lost, and the process will be repeated the next time the system is powered on again. Because this entire process is completely automatic, users do not need to manually configure any newly added hardware by replacing the DIP switch on the card. Such automatic device discovery and address space allocation are plug and play implementations.

If a PCIe to PCIe bridge is found, the system can assign a non-zero bus number to the secondary PCI bus outside the bridge, and then enumerate the devices on this secondary bus. If more PCIe bridges are found, the discovery can continue recursively until all possible domain/bus/device combinations have been scanned.

Each non-bridge PCIe device function can implement up to 6 BARs, each of which can respond to different addresses in the I/O port and memory mapped address space. Each BAR describes a region.

PCIe devices can also have optional read only memory (ROM) that can contain drive codes or configuration information.

BMC can directly configure RoC settings. The BMC may have a hard-coded path where specific data protection schemes are to be applied or configurable settings. The latter can expose the interface as a BIOS option to this configuration, or it can be additionally exposed to the software via a hardware exposed interface. A hard-coded scheme can be built into the BIOS firmware, and can still provide options to enable/disable protection.

To handle device failures, the BMC can detect when the drive has deteriorated or been removed through the control path. BMC can also use Self-Monitoring Analysis and Reporting Technology (SMART) to determine that the device is expected to deteriorate soon. In these situations, the BMC can reconfigure RoC hardware to enable failed scenarios or warn the user of the situation. BMC only enters the control path, not the data path. When a new drive is inserted, the BMC can intervene again and configure the new drive as part of a protected group, or initiate a rebuild operation. RoC hardware can handle the actual reconstruction, the recovery path in this setup, to provide the smallest possible performance impact, while providing less latency in the data access path.

FIG. 1 illustrates a machine according to an embodiment of the inventive concept, the machine including a peripheral component connection express (PCIe) switch with side-by-side erasure coding logic. In FIG. 1, the machine 105 is shown. The machine 105 may include a processor 110. The processor 110 may be any kind of processor: for example, Intel Xeon, Celeron, Itanium or Atom processor, Advanced Micro Devices, AMD) Opteron processor, advanced reduced instruction set computer (advanced RSIC machine, ARM) processor, etc. Although FIG. 1 shows a single processor 110 in the machine 105, the machine 105 may include any number of processors, each of which may be a single-core processor or a multi-core processor, and may be any Mix as desired.

The machine 105 may further include a memory 115, which may be managed by the memory controller 120. The memory 115 may be any kind of memory, such as flash memory, dynamic random access memory (DRAM), static random access memory (SRAM), permanent random access memory (Persistent Random Access Memory), Ferroelectric Random Access Memory (FRAM) or non-volatile random access memory (Non-Volatile Random Access Memory, such as Magnetoresistive Random Access Memory, MRAM) , NVRAM). The memory 115 can also be any desired combination of different memory types.

The machine 105 may also include a peripheral component connection express (PCIe) switch 125 with side-by-side erasure coding logic. The PCIe switch 125 may be any desired PCIe switch that supports side-by-side erasure coding logic.

The machine 105 may further include a storage device 130, which may be controlled by the device driver 135. The storage device 130 may be any desired form of storage device capable of communicating with the PCIe switch 125. For example, the storage device 130 may be a non-volatile storage fast (NVMe) solid state drive (SSD).

Although FIG. 1 illustrates the machine 105 as a server (which may be a stand-alone server or a rack server), embodiments of the inventive concept may include any desired type of machine 105 without limitation. . For example, the machine 105 may be replaced with a desktop computer or a laptop computer or any other machine that may benefit from embodiments of the inventive concept. The machine 105 may also include a dedicated portable computer device, a tablet computer (tablet computer), a smartphone, and other computer devices.

Fig. 2 shows additional details of the machine shown in Fig. 1. In FIG. 2, generally, the machine 105 includes one or more processors 110, which may include a memory controller 120 and a clock 205, which may be used to coordinate the operation of elements of the device 105 . The processor 110 may also be coupled to a memory 115, which may include, for example, random access memory (RAM), read-only memory (ROM), or other state retention media. The processor 110 may also be coupled to the storage device 130 and the network connector 210. The network connector 210 may be, for example, an Ethernet connector or a wireless connector. The processor 110 can also be connected to a bus 215, which can be attached to a user interface 220 and an input/output interface port, which can be managed using an input/output engine 225 and other components.

FIG. 3 shows additional details of the machine 105 shown in FIG. 1, including a power distribution board and an intermediate plane connecting the PCIe switch 125 with the side-by-side erasure coding logic shown in FIG. 1 to the storage device. In FIG. 3, the machine 105 may include an intermediate plane 305 and distribution boards 310 and 315. Respectively, the power distribution board 310 may include a PCIe switch 125 with a side-by-side erasure coding logic and a baseboard management controller 325, and the power distribution board 315 may include a PCIe switch 320 with a side-by-side erasure coding logic and a baseboard management controller 330. (Distribution boards 310 and 315 may also include additional components not shown in FIG. 3: FIG. 3 focuses on the elements most relevant to embodiments of the inventive concept.)

In some embodiments of the inventive concept, each PCIe switch 125 and 320 with side-by-side erasure coding logic can support up to a total of 96 PCIe lanes. By using U.2 connectors to connect PCIe switches 125 and 320 with side-by-side erasure coding logic to storage devices 130-1 to 130-6, each U.2 connector supports up to 4 PCIe channels per device . Two X4 channels are used (one X4 channel per communication direction), which means that each PCIe switch can support up to 96 ÷ 8 = 12 devices. Therefore, FIG. 3 shows that 12 storage devices 130-1 to 130-3 communicate with the PCIe switch 125 with side-by-side erasure coding logic, and 12 storage devices 130-4 to 130-6 have side-by-side erasure coding The logical PCIe switch 320 communicates. However, the number of storage devices that communicate with PCIe switches 125 and 320 with bypass erasure coding logic is only provided by the number of PCIe channels provided by PCIe switches 125 and 320 with bypass erasure coding logic and each storage device 130- The number of PCIe channels used from 1 to 130-6 is limited.

In some embodiments of the inventive concept, PCIe switches 125 and 320 with side-by-side erasure coding logic can be implemented using custom circuitry. In other embodiments of the inventive concept, PCIe switches 125 and 320 with side-by-side erasure coding logic can use field programmable gate arrays (FPGAs) or application-specific integrated circuits (Application-Specific Integrated Circuits) with appropriate programming , ASIC) to implement.

The BMCs 325 and 330 can be used to configure the storage devices 130-1 to 130-6. For example, the BMCs 325 and 330 can initialize the storage devices 130-1 to 130-6, thereby erasing any data present on the storage devices 130-1 to 130-6: at startup, when the storage devices 130-1 to 130 -6 is added to the erasure coding scheme or when both occur at the same time. Alternatively, such a function may be supported by a processor (the processor 110 shown in FIG. 1 or a local processor existing (but not shown) on the power distribution boards 310 and 315). BMCs 325 and 330 (or processor 110 shown in FIG. 1 or local processors present (but not shown) on power distribution boards 310 and 315) can also be responsible for PCIe switches 125 and 320 with side-by-side erasure coding logic Depending on the initial configuration of erasure coding logic.

FIG. 3 shows an exemplary complete setup of data protection for two PCIe switches 125 and 320 with side-by-side erasure coding logic: BMC 325 and 330 can be directly configured with side-by-side erasure coding logic. BMC 325 and 330 may have hard-coded paths in which specific data protection schemes are applied or configurable settings. The latter can expose the interface to this configuration as a basic input/output system (BIOS) option, or to additional software via a hardware exposed interface. A hard-coded scheme can be built into the BIOS firmware, and can still provide options to enable/disable protection.

In the event of a storage device failure, the BMCs 325 and 330 can detect when the storage device has deteriorated or was removed via the control path. The BMCs 325 and 330 can then reconfigure the sidetrack erasure coding logic to enable the failure scenario. BMC 325 and 330 can be connected to the control path, but not to the data path. Similarly, when a new storage device is inserted, BMCs 325 and 330 can intervene and configure the new storage device as part of an established group, or initiate a rebuild operation. Bypass erasure coding logic can handle the actual reconstruction; ideally, the recovery path in this setting should minimize the performance impact on data access, and reconstruct the data on the storage device from the remaining storage devices.

At this time, it makes sense to define the term "erasure coding". Erasure coding is intended to illustrate any desired method for encoding data on multiple storage devices. Erasure coding may require at least two storage devices or at least two parts of the storage device (eg, a single shell or housing containing two or more NAND flash memory channels), which is Because if only one storage device is used, data can be stored using conventional data access techniques suitable for the storage device. In other words, erasure coding is defined to mean spanning two or more storage devices, two or more parts of a single storage device in a more efficient use of storage devices and/or providing data redundancy or It can store data in any combination.

A redundant array of independent disks (RAID) represents a subset of erasure coding; or in other words, a RAID level represents a specific implementation of various erasure coding schemes. However, there may be other erasure coding schemes that can be defined beyond traditional RAID levels.

Generally, the implementation of erasure coding (or RAID) uses two or more physically different storage devices. However, in some embodiments of the inventive concept, a single shell or housing may include multiple parts of the storage device, which may be considered separate storage devices for the purpose of erasure coding. For example, a single NVMe SSD case or housing may include multiple NAND flash memory channels. For the purpose of erasure coding, each NAND flash memory channel can be regarded as a separate storage device, and data is striped (or encoded) across various NAND flash memory channels. In some embodiments of the inventive concept, this makes it possible to implement erasure coding using a single storage device. In addition, the PCIe switch 125 with side-view erasure coding logic may support Error Correcting Code (built-in somewhere in the PCIe switch 125 with side-view erasure coding logic, or through additional logic) or Other functions that can be used with a single storage device.

FIG. 4 shows the storage devices 130-1 to 130-6 shown in FIG. 3 for implementing different erasure coding schemes. In FIG. 4, as shown in the erasure coding scheme 405, storage devices 130-1 to 130-6 can be used in a RAID 0 configuration. RAID 0 stripes data across various storage devices. In other words, the data is divided into logical units suitable for storage devices, and each logical unit is written to different storage devices up to the number of storage devices in the array; one data logical unit is written on all storage devices Then, write the data on the first storage device again, and so on.

Compared with the use of a single storage device alone or even an unorganized group of diskettes (such as a diskette cabinet (Just a Bunch of Disks, JBOD) or a flash memory cabinet (Just a Bunch of Flash, JBOF)), RAID 0 has an advantage. Since the data is stored on multiple storage devices, the data can be read and written faster, with each storage device operating in parallel. Therefore, for example, by dividing data across 12 storage devices 130-1 to 130-6 as shown in FIG. 4, each storage device 130-1 to 130-6 only needs to read or write twelve points of the total data One, this is faster than reading or writing the entire data. The total capacity of the array can be calculated as the number of storage devices in the array multiplied by the capacity of the smallest storage device in the array. Therefore, in FIG. 4, since the array includes 12 data storage devices, the total capacity of the array is 12 times the capacity of the smallest storage device in the array.

The disadvantage of RAID 0 is that there is protection against storage device failure: if any storage device in the array fails, data will be lost. In fact, RAID 0 can be seen as a higher risk than JBOD or JBOF: by striping data across multiple storage devices, if any individual storage device fails, all data will be lost. (In contrast, for JBOD or JBOF, files are usually written to only one storage device. Therefore, although in the JBOD or JBOF settings, a single storage device failure may cause some data to be lost, but not all data will necessarily be lost.)

RAID 0 does not include any redundancy, and therefore is not technically a redundant array of independent disks. But traditionally, RAID 0 is regarded as a RAID level, and RAID 0 can undoubtedly be regarded as an erasure coding scheme.

The erasure coding scheme 410 shows RAID 5, which is a common RAID scheme. In RAID 5, parity blocks can be calculated for data stored on other storage devices in this stripe. Therefore, in FIG. 4, since the RAID 5 array includes a total of 12 storage devices, 11 storage devices are used as data drives, and one storage device is used as a co-location drive. (In RAID 5, co-located data is not limited to co-located drives, but is distributed across storage devices like any data. RAID 4, which is no longer used, stores all co-located information on a single drive.) Array (where the array There are n storage devices) The total capacity can be calculated as n-1 times the capacity of the smallest storage device. Since each stripe includes a co-located block, the erasure coding scheme 410 can tolerate the failure of up to one storage device and still be able to access all data (the data on the faulty storage device can be used in conjunction with the co-located block on the functional storage device Data to recover).

Note that compared to RAID 0, RAID 5 provides less total storage, but provides some protection against storage device failures. When determining the RAID level, this is an important trade-off: the relative importance of total storage capacity and redundancy.

Other RAID levels not shown in FIG. 4 can also be used as the erasure coding scheme. For example, RAID 6 uses two storage devices to store parity information, thereby reducing the total storage capacity to n-2 times the capacity of the minimum storage device, but tolerating up to two storage device failures at the same time. Hybrid schemes are also possible: for example, RAID 0+1, RAID 1+0, RAID 5+0, RAID 6+0 and other RAID schemes are possible, each scheme provides different total storage capacity and storage device fault tolerance degree. For example, five of the storage devices 130-1 to 130-6 can be used to form a RAID 5 array, and the other five of the storage devices 130-1 to 130-6 can be used to form a second RAID 5 array, and these two The group combined with the remaining two storage devices can be used to form a larger RAID 5 array. Alternatively, the storage devices 130-1 to 130-6 can be divided into two groups, each group implementing a RAID 0 array, wherein the two groups act as a larger RAID 1 array (thus implementing a RAID 0+1 setup ). It should be noted that RAID and erasure coding techniques use fixed codes or rotating codes, and the above fixed code/co-located drive symbols are for illustrative purposes only.

The erasure coding scheme 415 represents a more general description, which applies to all RAID levels and any other desired erasure coding scheme. Considering the array of storage devices 130-1 to 130-6, these storage devices can be divided into two groups: one group is used to store data, and the other group is used to store codes. The code may be parity information or any other desired encoding information that allows the recovery of lost data from a subset of data in the data group and some encodings in the encoding group. As shown in FIG. 4, the erasure coding scheme 415 may include up to X data storage devices and Y code storage devices. Considering any combination of X storage devices from the array, it is expected that it is possible to access or reconstruct data from all X data storage devices. Therefore, the erasure coding scheme 415 can generally tolerate up to Y storage device failures in the array and still be able to access all data stored in the array. In terms of capacity, the total capacity of the erasure coding scheme 415 is X times the capacity of the smallest storage device.

Note that in the above discussion, the total capacity of any erasure coding scheme is stated relative to "the capacity of the smallest storage device". For some erasure coding schemes, storage devices may have different capacities and still be fully utilized. But some erasure coding schemes (such as RAID 0 or RAID 1) expect all storage devices to have the same capacity, and will discard any capacity that larger storage devices may include. Therefore, the phrase "minimum storage device capacity" should be understood as a relative phrase, and the total capacity provided by an array using any particular erasure coding scheme may be greater than the above formula.

Returning to FIG. 3, regardless of the specific erasure coding scheme used, the bypass erasure coding logic of the PCIe switches 125 and 320 will effectively create new storage devices from the physical storage devices 130-1 to 130-6. Since the storage device presented by the erasure coding scheme does not physically exist, this new storage device can be regarded as a virtual storage device. And since this virtual storage device uses physical storage devices 130-1 to 130-6, the physical storage devices 130-1 to 130-6 should be hidden by the host. After all, when the data stored on the storage devices 130-1 to 130-6 may have been encoded in a manner unknown to the host, the host's attempt to directly access the blocks on the storage devices 130-1 to 130-6 will be a problem.

To support the use of this virtual storage device, PCIe switches 125 and/or 320 with side-by-side erasure coding logic can notify the processor 110 shown in FIG. 1 of the capacity of the virtual storage device. For example, if storage devices 130-1 to 130-6 include five NVMe SSDs (each NVMe SSD stores 1 terabyte (TB) of data (for mathematical simplicity, 1 TB is regarded as 240 bits Tuples instead of 1012 bytes) and the erasure coding scheme implements a RAID 5 array, the effective storage capacity of the virtual storage device is 4 TB. (Other implementations of erasure coding use fewer or more storage devices (Each storage device can store less than or more than 1 TB.) It may result in virtual storage devices with different capacities.) PCIe switches 125 and/or 320 with side-by-side erasure coding logic can be connected to provide a total of 4 The virtual storage device of TB (or 242 bytes) storage capacity is notified to the processor 110. As explained further below with reference to FIG. 5, the processor 110 shown in FIG. 1 can then write data to the blocks in this virtual storage device , And the side-by-side erasure coding logic can handle the actual storage of the data. For example, if the block size on the NVMe SSD is 4 kilobytes (KB), the processor 110 may request that the data be written to the number from 0 to Logic blocks between 230-1.

Alternatively, PCIe switches 125 and/or 320 with side-by-side erasure coding logic may request a host memory address block from the processor 110 shown in FIG. 1, which represents a method for communicating with a virtual storage device. When the processor 110 shown in FIG. 1 wants to read or write data, the transmission including the appropriate address in the address block of the host memory can be sent to the PCIe switch 125 with sidetrack erasure coding logic and/or Or 320. This host memory address block should be at least as large as the virtual storage device implemented using the erasure coding scheme (and if it is expected that additional storage devices can be added to the erasure coding scheme during use, it can be larger than the initial capacity of the virtual storage device ).

FIG. 5 shows the details of the PCIe switch 125 with the bypass erasure coding logic shown in FIG. In FIG. 5, the PCIe switch 125 with side-by-side erasure coding logic may include various elements, such as a connector 505, a PCIe-to-PCIe stack (510-1 to 510-6), a PCIe switch core 515 And power processing unit (PPU) 520. The connector 505 enables the PCIe switch 125 with bypass erasure coding logic and various other elements in the machine 105 shown in FIG. 1 (such as the processor 110 shown in FIG. 1 and the storage devices 130-1 to 130- shown in FIG. 3 6) Communication. One or more of the connectors 505 may be referred to as "external" connectors because they are connected to upstream elements (such as the processor 110 shown in FIG. 1); the remaining connectors 505 may be referred to as internal or downstream" "Connector" because it is connected to a downstream device (for example, storage devices 130-1 to 130-6 shown in FIG. 3). PCIe to PCIe stacks 510-1 to 510-6 allow data exchange between PCIe devices. For example, the storage device 130-1 shown in FIG. 3 may send data to the storage device 130-3 shown in FIG. Alternatively, the processor 110 shown in FIG. 1 may be requesting one or more of the storage devices 130-1 to 130-6 shown in FIG. 3 to perform a read or write request. The PCIe to PCIe stacks 510-1 to 510-6 may include buffers to temporarily store data: for example, if the destination device of a particular transfer is currently busy, the buffers in the PCIe to PCIe stacks 510-1 to 510-6 may be The transmission is stored until the destination device is idle. The PPU 520 may act as a configuration center to handle any configuration requests for the PCIe switch 125 with side-by-side erasure coding logic. Although FIG. 5 shows six PCIe to PCIe stacks 510-1 to 510-6, embodiments of the inventive concept may include any number of PCIe to PCIe stacks. The PCIe switch core 515 operates to route data from one PCIe port to another PCIe port.

Before entering the operation of the detection logic 525 and the erasure coding controller 530, it is understood that there are at least two different "addresses" for storing the data stored on the storage devices 130-1 to 130-6 shown in FIG. 3 of. On any storage device, data is written to a specific address associated with the hardware structure: this address can be regarded as a "physical" address: in the context of NVMe SSDs, a "physical" address is often called Physical block address (Physical Block Address, PBA).

The flash memory used in NVMe SSDs usually does not allow data to be rewritten in place. Conversely, when data needs to be rewritten, the old data will be invalidated, and the new data will be written to new data blocks at other locations on the NVMe SSD. Therefore, the PBA written to the data associated with a particular data structure (whether it is a document, object, or any other data structure) may change over time.

In addition, there are other reasons for relocating data in flash memory. The data is usually erased from the flash memory in a unit larger than the unit used when writing the data to the flash memory. If valid data is stored elsewhere in the cell to be erased, the valid data must be written to another location in the flash memory before the cell can be erased. This erasing process is usually called Garbage Collection, and the process of copying valid data from the unit to be erased is called programming. And Wear Levelling (a process of trying to use cells in flash memory at roughly the same level) can also relocate data in the flash memory.

Each time a specific data block is moved, the host can receive a notification and be informed of the new storage location of the data. But notifying the host in this way will bring a significant burden to the host. Therefore, most flash memory devices notify the host of the logical block address (LBA) where the data is stored, and maintain a table that maps the LBA to the PBA (usually located in the Flash Translation Layer (FTL) ). Then, whenever the data in question is moved to a new PBA, the flash memory can update the LBA to PBA mapping table in the FTL instead of notifying the host of the new address. Therefore, for each storage device, there may be both PBA and LBA associated with the data.

By adding the concept of virtual storage devices presented by side-by-side erasure coding logic, a further level is introduced for this structure. Recall the example presented above with reference to FIG. 3, where the erasure coding scheme includes five 1 TB NVMe SSDs, each of which uses blocks of 4 KB in size. Each NVMe SSD may include LBAs numbered from 0 to 228-1. But the virtual storage device presented to the host includes LBAs with numbers between 0 and 230-1.

Therefore, the LBA range seen by the host may represent a combination of multiple LBA ranges of various storage devices. To distinguish between the LBA range used by the host and the LBA range of each storage device, the LBA used by the host can be called "host LBA (host LBA)", "global LBA (global LBA)" or "Operating System (O/S) Aware LBA (operating system-aware LBA)", and the LBA used by the storage device may be called "device LBA (device LBA)", "local LBA (local LBA)" or " LBA behind RoC (LBA behind RoC)". The host LBA range can be divided between various storage devices in any desired manner. For example, the host LBA range can be divided into consecutive blocks, where each individual block is allocated to a specific storage device. By using this scheme, the host LBA 0 to LBA 228-1 can be mapped to the device LBA 0 to LBA 228-1 of the storage device 130-1, and the host LBA 228 to LBA 229-1 can be mapped to the device of the storage device 130-2 LBA 0 to LBA 228-1, and so on. Alternatively, the individual bits in the host LBA can be used to determine the appropriate storage device and the device LBA that stores this data: for example, use the lower-order bits in the host LBA to identify the device and strip the bits To generate the device LBA used by the storage device. However, no matter how the host LBA is mapped to the device LBA, there may be two, three, or even more different addresses that represent data storage locations.

Of course, storage devices are not required to be homogeneous: they can have different sizes and therefore different numbers of LBAs: they can even be different device types such as mixing SSDs with hard drives.

Note that for simplicity of explanation, even if the address provided to the storage device is not a logical block address (for example, a hard disk drive), the term "device LBA" may be used. If the "device LBA" is the actual address of the data stored on the storage device, the storage device may not map the device LBA to a different address before accessing the data.

Returning now to FIG. 5, the detection logic 525 and the erasure coding controller 530 act as the sidetrack erasure coding logic of the PCIe switch 125 with the sidetrack erasure coding logic. The detection logic 525 can “probe” (for example, by intercepting the request before it is delivered to its destination) transmission, and use capture interfaces 535-1 to 535-6 to determine the appropriate destination, capture interface 535-1 to 535-6 may be passed to the detection logic 525 via the multiplexer 540. As discussed above, the processor 110 only "sees" a virtual storage device of a given capacity (or a block of host memory address of a certain size) and issues a read or write based on the host LBA (associated with the virtual storage device) Information orders. The detection logic 525 may convert these host LBAs into device LBAs on one or more specific physical storage devices, and change the transmission accordingly to direct the request. The detection logic 525 can manage this conversion in any desired manner. For example, the detection logic 525 may include mapping the first range of host LBAs to the storage device 130-1 shown in FIG. 3, and mapping the second range of host LBAs to the storage device 130-2 shown in FIG. 3 (and so on) Table, where the device LBA depends on factors that can be related to how the side-by-side erasure coding logic operates: for example, the erasure coding scheme itself (eg RAID level), stripe size, number of storage devices, etc. Alternatively, the detection logic 525 may use specific bits in the host LBA to determine which of the storage devices 130-1 to 130-6 shown in FIG. 3 stores the data in question: for example, if the array includes only two storage devices, Then, the detection logic 525 may use low-order bits (or some other bits in the logical block address) to determine whether the data is to be written to the first storage device or the second storage device. (Obviously, as more storage devices are included in the array, more bits can be used, taking care to ensure that logical block addresses do not include bit combinations that "identify" nonexistent storage devices. For example, Figure 3 shows A total of 24 storage devices 130-1 to 130-6 are available, and storage devices 130-1 to 130-6 can use bit values from 00000 to 10111; bit values between 11000 and 11111 should be avoided.) Embodiments of the inventive concept may Use any other desired method to map the logical block address received from the host to the (suitable) block address on the storage device.

As an example, consider that the processor 110 shown in FIG. 1 sends a write request with sufficient data to fill the entire stripe across all storage devices in the storage devices 130-1 to 130-6 (in the calculation After erasure coding). The detection logic 525 may divide the data into separate logical units, and as discussed below, the erasure coding controller 530 may provide or modify the data. The detection logic 525 may then generate a transmission with suitable data, the destination of the one transmission being each of the storage devices 130-1 to 130-6.

Note that when probe logic 525 replaces the original host LBA with a device LBA suitable for the storage device in question, this device LBA does not have to be a physical block address. In other words, the device LBA used by the detection logic itself can be another logical block address. This structure enables the physical storage device to continue to manage its own data storage when appropriate. For example, if the physical storage device is a NVMe SSD, the SSD can move data around for garbage collection or wear leveling, and use its flash memory conversion layer to manage one NAND of the provided device LBA and NAND flash memory chip The association of PBA on the flash memory chip. Such operations may occur without knowing the detection logic 525. However, if the storage device in question does not relocate data, the device LBA provided by the probe logic 525 can be a physical address on the storage device unless the host instructs so.

As described above, the erasure coding controller 530 may implement an erasure coding scheme. Depending on the erasure coding scheme, the erasure coding controller 530 can simply generate suitable co-located data (for example, when using a RAID 5 or RAID 6 erasure coding scheme), while allowing the original data (as shown in Figure 1 Provided by the processor 110) remains unchanged. However, in some embodiments of the inventive concept, the erasure coding controller 530 may also modify the original material. For example, the erasure coding controller 530 may implement an error correction code on the original data, so that even in the case of an error, the respective storage devices 130-1 to 130-6 stored in FIG. 3 can be properly read On the block. Alternatively, the erasure coding controller 530 may encrypt the data written to the storage devices 130-1 to 130-6 shown in FIG. 3 so that the data written to the storage devices 130-1 to 130-6 shown in FIG. 3 Cannot be read without an encryption key (or worse, cause the erasure coding controller 530 to think that if the processor 110 shown in FIG. 1 wants to directly write data, the storage device 130-1 to 130-6 was destroyed. Alternatively, the erasure coding controller 530 may introduce parity information (or similar types of information) into the data written into each of the storage devices 130-1 to 130-6 shown in FIG. The specific operation performed by the erasure coding controller 530 on the material depends on the erasure coding scheme used.

The detection logic 525 and the erasure coding controller 530 can be implemented in any desired manner. For example, the detection logic 525 and the erasure code controller 530 may be implemented using a processor on which suitable software is stored. However, since PCIe switches are generally implemented as hardware circuitry (which is usually faster than software running on the processor of devices such as PCIe switches, which generally do not need to implement a large number of functions), detection logic 525 and erasure coding The controller 530 may be implemented using suitable circuitry. Such circuitry may include FPGA, ASIC, or any other desired hardware implementation programmed in a suitable manner.

In the most basic embodiment, only the probing logic 525 and the erasure coding controller 530 may be used to implement the sidetrack erasure coding logic. But including cache 545 and/or write buffer 550 in the side-by-side erasure coding logic may provide significant benefits.

The cache 545 may store a subset of data stored in the virtual storage device. In general, the capacity of cache 545 is less than the total virtual storage device, but the access is faster. Therefore, compared to accessing data from an underlying physical storage device, by storing some data in the cache 545, a cache hit on the cache 545 can result in faster performance of the virtual storage device . For example, the cache 545 may store the latest data accessed from the virtual storage device, and use any desired algorithm to identify the data to be replaced as it becomes older (eg, Least Recently Used algorithm or the most recently used algorithm). Least Frequently Used algorithm). The cache 545 may be implemented using any desired memory structure (eg, DRAM, SRAM, MRAM, or any other desired memory structure). The cache 545 may even be implemented using a memory structure that is faster than traditional memory, such as can be used in the L1 or L2 cache in the processor. Finally, although cache 545 is shown as part of PCIe switch 125 with side-by-side erasure coding logic, cache 545 may also be stored in memory 115 shown in FIG. The PCIe switch 125 accesses from the memory 115.

The write buffer 550 provides a mechanism to accelerate write requests. The time required to perform a write operation on a virtual storage device that uses erasure coding to span multiple physical storage devices may be slower than a similar write request to a single physical storage device. Performing a write operation may involve reading data from other storage devices in the same block, after which new data may be merged, and then the merged data may be written back to the appropriate storage device. The implementation of consolidation may also involve calculation of parity information or other code information. And if the underlying physical storage device is busy performing other operations (eg, processing read requests), write requests may also be delayed. It may be undesirable to delay software running on the processor 110 shown in FIG. 1 while waiting for the write request to complete. Therefore, the write buffer 550 can temporarily store data until the write to the underlying physical storage device is completed, rather than blocking the software running on the processor 110 shown in FIG. 1; meanwhile, the detection logic 525 can change the write request The completion is notified to the software running on the processor 110 shown in FIG. Compared with the write-back cache strategy, this method is similar to the write-through cache policy (write-through cache policy). In the write-back cache strategy, the write operation is completed by the software running on the processor 110 Before being told that the write is complete. Like cache 530, write buffer 550 can be implemented using any desired memory structure, such as DRAM, SRAM, MRAM, or L1 or L2 cache structure, among other possibilities.

As part of performing the write operation, the side-by-side erasure coding logic may check whether any of the data required to complete the write operation is currently in the cache 545. For example, when the processor 110 shown in FIG. 1 sends a write request to the virtual storage device, the erasure coding scheme may need to read the entire stripe to calculate parity information or other code information. If some (or all) of the data resides in the cache 545, the data can be accessed from the cache 545 instead of reading the data from the underlying physical storage device. In addition, the cache strategy may suggest that the data to be written should also be cached in the cache 545 in case the data may be requested again in the near future.

Although FIG. 5 shows the cache 545 and the write buffer 550 as separate elements, embodiments of the inventive concept may combine the two into a single element (which may be simply referred to as "cache"). In such an embodiment of the inventive concept, the cache may include a bit indicating whether the data stored thereon is "clean" or "dirty". "Clean" data means data that has only been read but not modified since it was last written to the underlying physical storage device; "dirty" data has been modified since it was last written to the underlying physical storage device. If the cache includes "dirty" data, when the data is removed from the cache according to the cache strategy, the sidetrack erasure coding logic may need to write the "dirty" data back to the underlying storage device. In addition, embodiments of the inventive concept may include cache 545, write buffer 550, both (either individually or combined into a single element), or neither.

As discussed above, the side-by-side erasure coding logic in the PCIe switch 125 with side-by-side erasure coding logic can "create" the virtual storage device from the underlying physical storage device, and if the processor 110 shown in FIG. The direct access of the physical storage devices 130-1 to 130-6 shown in 3 will be a problem. Therefore, when the machine 105 shown in FIG. 1 initially boots (ie, starts or powers up) and attempts to enumerate the various PCIe devices that are accessible, the PCIe switch 125 with side-by-side erasure coding logic can determine that it wants to use the bypass Depending on the erasure coding logic and the attached storage device. In such a situation, the PCIe switch 125 with side-by-side erasure coding logic should prevent enumeration of any PCIe devices downstream of the PCIe switch 125 with side-by-side erasure coding logic. By preventing such enumeration, the PCIe switch 125 with side-by-side erasure coding logic can "create" a virtual storage device without worrying that the processor 110 shown in FIG. 1 may be able to directly access the storage device 130-1 shown in FIG. To the data on 130-6 (this may destroy the data used in the erasure coding scheme). But as discussed below with reference to FIGS. 9-10, there may be scenarios where the PCIe switch 125 with side-by-side erasure coding logic should allow downstream enumeration of PCIe devices.

The probe logic 525 may also pass configuration commands to the PPU 520. In this way, the detection logic 525 can also operate as a PCIe to PCIe stack to achieve the purpose of connecting the PCIe switch core 515 and the PPU 520.

Finally, the detection logic 525 may receive the erasure coding enable signal 555 from the processor 110 shown in FIG. 1 (possibly through a pin on the PCIe switch 125 with side-by-side erasure coding logic). The erasure coding enable signal 555 can be used to enable to disable erasure coding logic in the PCIe switch 125 with bypass erasure coding logic.

6 shows details of a PCIe switch with perspective erasure coding logic according to another embodiment of the inventive concept. It can be seen by comparing FIGS. 5 and 6 that in the PCIe switch 125 with bypass erasure coding logic shown in FIG. 5 and the PCIe switch 605 with perspective erasure coding logic shown in FIG. 6, the bypass erasure coding logic The main difference from perspective erasure coding logic is where the erasure coding logic is placed. In the PCIe switch 125 with side-by-side erasure coding logic shown in FIG. 5, the erasure coding logic is located on the “side” of the PCIe switch, while the PCIe switch 605 with perspective erasure coding logic shown in FIG. 6 In the erasing coding logic and PCIe switch "inline".

Compared with perspective erasure coding logic, the use of side-by-side erasure coding logic has technical advantages and disadvantages. The side-by-side erasure coding logic shown in FIG. 5 is a more complex implementation because probing logic 525 is required to intercept and manage the redirection of data from the host. In contrast, the perspective erasure coding logic shown in FIG. 6 is easier to implement because all data between the host and the storage devices 130-1 to 130-6 shown in FIG. 3 pass through the erasure coding controller 530. On the other hand, when erasure coding logic is disabled, including side-by-side erasure coding logic does not introduce additional latency to the operation of PCIe switch 125. In contrast, the perspective erasure coding logic shown in FIG. 6 can serve as a PCIe endpoint. The perspective erasure coding logic shown in FIG. 6 may buffer data between the host and the storage devices 130-1 to 130-6 shown in FIG. 3, which may increase the latency of communication. In the perspective erasure coding logic shown in FIG. 6, the erasure coding controller 530 may further include, for example, a frame buffer (Frame Buffer), a routing table (Route Table), a port arbitration logic (Port Arbitration logic) and a scheduler (Scheduler ) (Not shown in FIG. 6) and other elements: elements generally included in the PCIe switch core 515.

In addition, PCIe switches usually use the same number of ports for upstream (to host) traffic and downstream (to storage devices and other connected devices) traffic. For example, if the PCIe switch 605 includes a total of 96 ports, usually 48 are used for upstream traffic and 48 are used for downstream traffic. However, with the perspective erasure coding logic shown in FIG. 6 enabled, the erasure coding controller 530 can virtualize all downstream devices. In this case, communication with the host usually requires only 16 or possibly 32 upstream ports. If the PCIe switch 605 includes more ports than 32 or 64 ports, the additional ports can be used to connect additional downstream devices, and the additional downstream devices can be used to increase the capacity of the virtual storage device. To this end, the erasure code controller 530 shown in FIG. 6 may use a non-transparent bridge (NTB) port to communicate with the host.

Figure 6 shows a PCIe switch 605 that includes perspective erasure coding logic. However, embodiments of the inventive concept may separate the perspective erasure coding logic from the PCIe switch 605. For example, the perspective erasure coding logic may be implemented as a separate component from the PCIe switch 605 using FPGA or ASIC.

However, although there are implementation and technical differences between the side-by-side erasure coding logic shown in FIG. 5 and the perspective erasure coding logic shown in FIG. 6, the two erasure coding logics will be functionally different. Achieve similar results. Therefore, the side-by-side erasure coding logic as shown in FIG. 5 and the perspective-erasure coding logic as shown in FIG. 6 may be interchanged as necessary. Any reference to by-pass erasure coding logic in this document is intended to also include perspective erasure coding logic.

7 to 10 illustrate various topologies using the PCIe switch 125 shown in FIG. 1 with side-by-side erasure coding logic. Regardless of the topology in use, the operation of the PCIe switch 125 with side-by-side erasure coding logic shown in FIG. 1 is the same: it not only provides connections to various additional storage devices, but also supports erasure coding across these storage devices.

FIG. 7 shows a first topology using the PCIe switch 125 with bypass code erasure coding logic shown in FIG. 1 according to an embodiment of the inventive concept. In FIG. 7, a PCIe switch 125 with side-by-side erasure coding logic is shown, which may be implemented as a separate component of the machine 105 shown in FIG. That is, the PCIe switch 125 with side-by-side erasure coding logic can be manufactured and sold separately from any other components such as the processor 110 or the storage device 130 shown in FIG. 1.

A PCIe switch 125 with side-by-side erasure coding logic may be connected to the storage device 130. In FIG. 7, the PCIe switch 125 with side-by-side erasure coding logic is shown as being connected only to a single storage device, which may not support erasure coding: erasure coding requires at least two storage devices or storage devices At least two parts of it to implement striping, chunking, grouping and using parity information or code information. But even a single storage device PCIe switch 125 with side-by-side erasure coding logic can provide some advantages. For example, the PCIe switch 125 with side-by-side erasure coding logic can support the use of error correction codes with the storage device 130, or if the storage device 130 does not provide these services locally, encrypt the data stored on the storage device 130 .

The storage device 130 may also be connected to the FPGA 705. FPGA 705 can support acceleration. In short, there may be situations where data needs to be processed and then discarded. Loading all such data into the processor 110 shown in FIG. 1 to perform processing can be expensive and time-consuming: calculations can be performed more easily at locations closer to the data. FPGA 705 can support such calculations at a location closer to the memory, so that it is no longer necessary to load data into the processor 110 shown in FIG. 1 to perform calculations: this concept is called "acceleration". FPGA-based acceleration is discussed more in U.S. Patent Application No. 16/122,865 filed on September 5, 2018, which asserts U.S. Provisional Patent Application No. 62 filed on March 13, 2018 /642,568, U.S. Provisional Patent Application No. 62/641,267 filed on March 9, 2018, and U.S. Provisional Patent Application No. 62/638,904 filed on March 5, 2018 (all applications in the application (Incorporated herein for reference) and the rights and interests of US Patent Application No. 16/124,179 filed on September 6, 2018, US Patent Application No. 16/124,182 filed on September 6, 2018, and September 6, 2018 U.S. Patent Application No. 16/124,182 filed on the Japanese filing date (all applications in the application are continuations of U.S. Patent Application No. 16/122,865 filed on September 5, 2018 and are incorporated herein by reference) Rights. Since the purpose of acceleration is to process the data without transferring the data to the processor 110 shown in FIG. 1, FIG. 7 shows the FPGA 705 closer to the storage device 130. Note, however, that the specific arrangement shown in FIG. 7 is not required: FPGA 705 may be located between PCIe switch 125 with side-by-side erasure coding logic and storage device 130.

In addition to data acceleration, FPGA 705 can provide other functions to support storage device 130. For example, the FPGA 705 may implement a deduplication function on the storage device 130 in an attempt to reduce the number of times the same data is stored on the storage device 130. FPGA 705 can determine whether specific data is stored on storage device 130 more than once, established between various logical block addresses (or other information used by the host to identify the data) and the location where the data is stored on storage device 130 Associate and delete additional copies.

Alternatively, FPGA 705 can implement data integrity functions on storage device 130, such as adding error correction codes to prevent data loss due to operational errors of storage device 130 or using cyclic redundancy correction (Cyclic Redundancy) Correction (CRC) T10DIF (Data Integrity Field) for end-to-end protection. In this way, FPGA 705 may be able to detect when there is an erroneous writing and reading of data on storage device 130 or data in transmission, and restore the original data. Note that FPGA 705 can implement the data integrity function without the host being aware that the data integrity function is being provided: the host may only see the data itself, but not any of the error correction codes.

Alternatively, FPGA 705 can implement a data encryption function on storage device 130 to prevent unauthorized parties from accessing the data on storage device 130: Without providing an appropriate encryption key, the data returned from FPGA 705 is The requester may be meaningless. The host can provide the encryption key to be used when writing and reading data. Alternatively, FPGA 705 can automatically perform data encryption and decryption: FPGA 705 can store the encryption key (and can even generate the encryption key on behalf of the host) and determine the appropriate encryption key to use based on who requested the data.

Alternatively, FPGA 705 may implement a data compression function on storage device 130 to reduce the amount of space required to store data on storage device 130. When writing data to the storage device 130, the FPGA 705 can implement the following functions: compress the data provided by the host into a smaller amount of storage, and then store the compressed data (and restore the original data when reading data from the storage device 130 Any information needed). When reading data from the storage device 130, the FPGA 705 can read the compressed data (and any information needed to restore the original data from the compressed data) and remove the compression to restore the original data.

Any desired implementation of deduplication, data integrity, data encryption, and data compression can be used. Embodiments of the inventive concept are not limited to specific implementations of any of these functions.

The FPGA 705 may also implement any combination of functions on the storage device 130 as needed. For example, FPGA 705 may implement both data compression and data integrity (because data compression may increase the sensitivity of data to errors: a single error in the data stored on storage device 130 may render a large amount of data unusable). Or FPGA 705 can implement both data encryption and data compression (to protect data while using as little memory as possible). FPGA 705 may also provide other combinations of two or more functions.

In terms of overall operation, when any of these functions is implemented, the FPGA 705 can read data from a suitable source. Note that although the term "source" is a singular noun, when appropriate, embodiments of the inventive concept can read data from multiple sources (eg, multiple storage devices). The FPGA 705 can then perform appropriate operations on the data: data acceleration, data integration, data encryption, and/or data compression. The FPGA 705 may then take appropriate action on the result of the operation: for example, sending the result to the host 105 shown in FIG. 1 or writing the data to the storage device 130.

Although the above functions are explained with reference to the FPGA 705 shown in FIG. 7, embodiments of the inventive concept may include these functions at any place in a system including an FPGA. In addition, embodiments of the inventive concept allow FPGA 705 to access data from a "remote" storage device. For example, returning to FIG. 3 for the time being, and assuming that the storage device 130-1 includes an FPGA similar to the FPGA 705, but the storage device 130-2 lacks such a storage device. The FPGA included in the storage device 130-1 may be used to apply its function to the storage device 130-2 by sending a request to the storage device 130-2. For example, if the FPGA in the storage device 130-1 provides data acceleration, the FPGA in the storage device 130-1 may send a request to read the data from the storage device 130-2, perform the appropriate acceleration, and then send the result to the appropriate Destination (for example, host 105 shown in FIG. 1).

In FIG. 7 (and in the topology shown in FIGS. 8 to 10 below), a PCIe switch 125 with side-by-side erasure coding logic may be attached to a device that does not qualify for erasure coding. For example, a PCIe switch 125 with side-by-side erasure coding logic can be attached to other storage devices with built-in erasure coding functions, or devices such as FPGA 705 or graphics processing unit (GPU) shown in FIG. 7 that are not storage devices . All such devices may be described as devices that are not eligible for erasure coding (or at least not eligible for erasure coding provided by PCIe switch 125 with side-by-side erasure coding logic).

When a PCIe switch 125 with bypass erasure coding logic is connected to a device that does not qualify for erasure coding, the system has various alternatives available. In one embodiment of the inventive concept, including any device that does not qualify for erasure coding may cause the bypass erasure coding logic of the PCIe switch 125 with the bypass erasure coding logic to be disabled. Therefore, for example, if the PCIe switch 125 with bypass erasure coding logic is to be connected to the FPGA 705 or GPU shown in FIG. 7 or a storage device with native erasure coding logic, it is connected to the None of the storage devices of PCIe switch 125 can be used with erasure coding. Note that the decision to disable the side-by-side erasure coding logic of the PCIe switch 125 with side-by-side erasure coding logic does not necessarily translate to other PCIe with side-by-side erasure coding logic in the same mainframe or other mainframes switch. For example, FIG. 3 shows two PCIe switches 125 and 320 with side-by-side erasure coding logic, one of which can enable the side-by-side erasure coding logic and the other PCIe switch can disable the side-by-side erasure coding logic.

Another embodiment of the inventive concept can disable a device that does not qualify for erasure coding as if it were not connected to the PCIe switch 125 with bypass erasure coding logic at all. In this embodiment of the inventive concept, the PCIe switch 125 with side-by-side erasure coding logic can enable the side-by-side erasure coding logic to the storage device 130, and can disable any other storage devices that qualify for erasure coding, as if It is not connected to the PCIe switch 125 with side-by-side erasure coding logic.

In yet another embodiment of the inventive concept, the PCIe switch 125 with side-by-side erasure coding logic can enable the side-by-side erasure coding logic for storage devices that can be overwritten by the side-by-side erasure coding logic, but still enable non-compliance Other devices that are eligible for erasure coding are accessed. This embodiment of the inventive concept is the most complex implementation: PCIe switch 125 with bypass erasure coding logic needs to determine which devices are eligible for erasure coding and which do not, and then analyze the traffic to determine the destination of the traffic Whether it is a virtual storage device (in this case, the traffic is intercepted by sidetrack erasure coding logic) or not a virtual storage device (in this case, the traffic is delivered to its original destination).

In the embodiment of the inventive concept where the machine 105 does not ultimately provide all the functions of the installed device (ie, where the erasure coding is disabled due to the existence of a device that does not qualify for erasure coding, or such a device is Depending on the erasure coding PCIe switch 125 disabled embodiment), the machine 105 may notify the user of this fact. This notification may be provided by the processor 110 shown in FIG. 1, the BMC 325 shown in FIG. 3, or the PCIe switch 125 with side-by-side erasure coding logic. In addition to informing the user that some functions have been disabled, the notification may also inform the user how to reconfigure the machine 105 to allow added functions. For example, the notification may suggest that devices that are not eligible for erasure coding are connected to specific slots in the middle plane 305 shown in FIG. 3 (probably those connected to PCIe switches 320 with bypass erasure coding logic) It is also recommended to connect storage devices that are eligible for erasure coding to other slots (such as those connected to PCIe switches 125 with side-by-side erasure coding logic). In this way, at least some storage devices that qualify for erasure coding can benefit from the erasure coding scheme without blocking access to other devices that are not eligible for erasure coding.

FIG. 8 illustrates a second topology using the PCIe switch 125 with side-by-side erasure coding logic shown in FIG. 1 according to another embodiment of the inventive concept. In FIG. 8, the PCIe switch 125 with side-by-side erasure coding logic may be located within the FPGA 705: that is, the FPGA 705 may also implement the PCIe switch 125 with side-by-side erasure coding logic. The FPGA 705 and the PCIe switch 125 with side-by-side erasure coding logic may then be connected to the storage devices 130-1 to 130-4. Although FIG. 8 shows that FPGA 705 and PCIe switch 125 with bypass erasure coding logic are connected to four storage devices 130-1 to 130-4, embodiments of the inventive concept may include any number of storage devices 130-1 To 130-6.

In general, the topology shown in FIG. 8 can be implemented in a single shell or housing and contains all of the components shown (SSD 130-1 to 130-4 can be separate flash memories, not self-contained SSDs) . That is, the entire structure shown in FIG. 8 can be sold as a single unit rather than as a separate element. However, embodiments of the inventive concept may also include a riser card connected to the machine 105 shown in FIG. 1 (possibly connected to the intermediate plane 305 shown in FIG. 3) at one end and having a connection for storage to the storage at the other end. Connectors for devices 130-1 to 130-4 (eg U.2, M.3 or SFF-TA-1008 connectors). And although FIG. 8 shows the PCIe switch 125 with side-by-side erasure coding logic as part of the FPGA 705, the PCIe switch 125 with side-by-side erasure coding logic may also be implemented as part of the smart SSD.

FIG. 9 illustrates a third topology for using the PCIe switch 125 with bypass code erasure coding logic shown in FIG. 1 according to yet another embodiment of the inventive concept. In FIG. 9, two PCIe switches 125 and 320 with bypass erasure coding logic are shown, and up to 24 storage devices 130-1 to 130-6 are connected between the two PCIe switches 125 and 320. As described above with reference to FIG. 3, each PCIe switch 125 and 320 with side-by-side erasure coding logic may include 96 PCIe lanes, using four PCIe lanes in each direction to communicate with the storage devices 130-1 to 130- One of 6 communications: each PCIe switch 125 and 320 with side-by-side erasure coding logic can then support up to 12 storage devices. To support erasure coding across storage devices supported by multiple PCIe switches 125 and 320 with side-by-side erasure coding logic, a PCIe switch with side-by-side erasure coding logic can be designated to be responsible for erasure coding across all devices , And can enable side-by-side erasure coding logic. Another PCIe switch 320 with side-by-side erasure coding logic may operate purely as a PCIe switch, with side-by-side erasure coding logic disabled. The choice of which PCIe switch should be selected to handle erasure coding can be done in any desired manner: for example, the two PCIe switches can negotiate between them, or the PCIe switch that is enumerated first can be designated Used to deal with erasure codes. PCIe switches that are selected to handle erasure coding may then report virtual storage devices (across two PCIe switches), and PCIe switches that do not handle erasure coding may not report to downstream devices (to prevent processor 110 shown in FIG. 1 from attempting access Storage devices as part of the erasure coding scheme).

Note that although PCIe switches 125 and 320 with side-by-side erasure coding logic can both be located in the same mainframe, PCIe switches 125 and 320 with side-by-side erasure coding logic can be located in different mainframes. That is, the erasure coding scheme can span storage devices between multiple main chassis. All that is required is that the PCIe switches in the various mainframes can negotiate with each other the location of the storage device to be part of the erasure coding scheme. The embodiments of the inventive concept are also not limited to two PCIe switches 125 and 320 with side-by-side erasure coding logic: the storage devices included in the erasure coding scheme can be connected to any number of PCIe with side-by-side erasure coding logic Switches 125 and 320.

The host LBA can be split across PCIe switches 125 and 320 with side-by-side erasure coding logic in any desired manner. For example, the least significant bit in the host LBA can be used to identify which PCIe switch 125 or 320 with side-by-side erasure coding logic includes a storage device that stores data with this host LBA. By using more than two PCIe switches with bypass erasure coding logic, multiple bits can be used to determine which PCIe switch with bypass erasure coding logic manages the storage device that stores the data. Once a suitable PCIe switch with side-by-side erasure coding logic has been identified (and the detection logic 525 shown in FIG. 5 has been modified for transmission), the transmission can be routed to the appropriate PCIe switch with side-by-side erasure coding logic (assuming The destination of the transmission is not a storage device connected to a PCIe switch with bypass erasure coding logic enabled and bypass erasure coding logic enabled).

In another embodiment of the inventive concept, a single PCIe switch with bypass erasure coding logic is no longer responsible for virtualizing all storage devices connected to two PCIe switches with bypass erasure coding logic, but Each PCIe switch with side-by-side erasure coding logic can create a separate virtual storage device (with a separate erasure coding domain). In this way, different erasure coding fields can be created for different customers, but with smaller capacities.

Figure 9 may also represent another embodiment of the inventive concept. Although FIG. 9 implies that only storage devices 130-1 to 130-16 are connected to PCIe switches 125 and 320 with side-by-side erasure coding logic and all storage devices 130-1 to 130-6 can be used with the erasure coding scheme, But as discussed above, embodiments of the inventive concept are not limited to this: PCIe switches 125 and 320 with side-by-side erasure coding logic may have devices that are not eligible for erasure coding connected to them. Such devices can be grouped under a single PCIe switch with side-by-side erasure coding logic, and storage devices that qualify for erasure coding are grouped under different PCIe switches 125 with side-by-side erasure coding logic. In this way, the best function of the machine 105 shown in FIG. 1 can be achieved. One (or some) PCIe switches with side-by-side erasure coding logic enable the side-by-side erasure coding logic, and one (or some) have side-by-side The PCIe switch with erasure coding logic disables the sidetrack erasure coding logic.

FIG. 10 illustrates a fourth topology for using the PCIe switch 125 with bypass erasure coding logic shown in FIG. 1 according to yet another embodiment of the inventive concept. In FIG. 10, compared with FIG. 9, PCIe switches 125, 320, and 1005 with side-by-side erasure coding logic may be constructed in a hierarchy. At the top of the hierarchical structure, the PCIe switch 125 with side-by-side erasure coding logic can manage the erasure coding of all storage devices in the hierarchical structure below the PCIe switch 125 with side-by-side erasure coding logic, and therefore the side Depending on erasure coding logic. On the other hand, PCIe switches 320 and 1005 with side-by-side erasure coding logic can disable their side-by-side erasure coding logic (because their storage device is by side-by-side erasure coding logic of PCIe switch 125 with side-by-side erasure coding logic management).

Although FIG. 10 shows three PCIe switches 125, 320, and 1005 with side-by-side erasure coding logic configured in a two-tier hierarchy, embodiments of the inventive concept are included in the included PCIe switches The number of or its hierarchical arrangement is not limited. Therefore, embodiments of the inventive concept can support any number of PCIe switches with side-by-side erasure coding logic arranged in any desired hierarchical structure.

The embodiments of the inventive concept explained above with reference to FIGS. 1 to 10 focus on the dual-port storage device. However, embodiments of the inventive concept can be extended to dual-port storage devices, where one (or more) storage devices communicate with multiple PCIe switches with side-by-side erasure coding logic. In such an embodiment of the inventive concept, if the PCIe switch 125 with bypass code erasure coding logic shown in FIG. 3 cannot communicate with the dual-port storage device, the PCIe switch 125 with bypass code erasure coding logic can The PCIe switch 320 with side-by-side erasure coding logic sends a transmission in an attempt to communicate with the storage device. The PCIe switch 320 with side-view erasure coding logic effectively acts as a bridge to enable the PCIe switch 125 with side-view erasure coding logic to communicate with the storage device.

Embodiments of the inventive concept can also support the detection and handling of storage device failures. For example, consider FIG. 4 again, and assume that the storage device 130-1 fails. The storage device 130-1 may fail for any number of reasons: power surges may have damaged electronics, wiring (inside the storage device 130-1 or the storage device 130-1 and the Except in the connection between the encoded logic PCIe switches 125), a failure may have occurred, the storage device 130-1 may have detected too many errors and shut itself down, or the storage device 130-1 may have failed for other reasons. The storage device 130-1 may also have been removed from its slot by the user (perhaps to replace it with a newer, more reliable or larger storage device). Whatever the reason, the storage device 130-1 may become unusable.

The PCIe switch 125 with bypass code erasure coding logic can detect the failure of the storage device 130-1 through the presence pin on the connector of the storage device 130-1. If the storage device 130-1 is removed from the main case, or if the storage device 130-1 has been shut down, it may no longer assert its presence through the presence pin on the connector, which may An interrupt is triggered in the PCIe switch 125 except for coding logic. Alternatively, the PCIe switch 125 (or BMC 325 shown in FIG. 3) with side-by-side erasure coding logic can send an occasional message to the storage device 130-1 to check whether it is still active (sometimes The process called "heartbeat": If the storage device 130-1 does not respond to this kind of message, then the PCIe switch 125 with the bypass erasure coding logic or the BMC 325 shown in FIG. 3 can conclude the storage device 130-1 has failed.

If (and when) the storage device 130-1 fails, the PCIe switch 125 with side-by-side erasure coding logic can manage all data by using other means to access any data normally requested from the storage device 130-1 Describe the situation. For example, if there is a mirror of the storage device 130-1, the PCIe switch 125 with bypass erasure coding logic may request data from the mirror of the storage device 130-1. Alternatively, PCIe switch 125 with side-by-side erasure coding logic can request the rest of the stripe containing the desired data from other storage devices in the array and use erasure coding information to reconstruct the data from storage device 130-1 . There may be other mechanisms by which the PCIe switch 125 with bypass erasure coding logic can access the data stored on the fault storage device 130-1.

Embodiments of the inventive concept can also support the detection and handling of the insertion of new storage devices in the array. As with the detection of storage device failures, PCIe switch 125 (or BMC 325 shown in Figure 3) with side-by-side erasure coding logic can accidentally inspect the device to see what is connected or any other desired mechanism, using the The presence pin detects the insertion of a new storage device (as with the detection of a failed storage device, using the presence pin to detect a new storage device may trigger an interrupt in the PCIe switch 125 with side-by-side erasure coding logic). When a new storage device is detected, the PCIe switch 125 with side-by-side erasure coding logic can add this new storage device to the array. Adding new storage devices to the array does not necessarily involve changing the erasure coding scheme: such changes may require changes to all the data stored on the storage devices. (For example, consider changing from RAID 5 to RAID 6: each stripe will now require two co-located blocks (which will need to be rotated between storage devices), requiring large amounts of data to be calculated and moved.) But new storage devices are added It may not be necessary to move large amounts of data around to existing erasure coding schemes. Therefore, although adding a new storage device may not improve the array's tolerance to storage device failure, adding a new storage device can still increase the capacity of the virtual storage device.

If a faulty storage device already exists in the array, the insertion of the new storage device can be used to rebuild the faulty storage device. The erasure coding controller 530 shown in FIG. 5 can calculate the data stored on the faulty storage device and store the data in the appropriate block address on the replacement storage device. For example, the original data on the faulty storage device can be calculated based on the data on other storage devices (both the original data and the parity or code information); the parity or code information stored on the faulty storage device can be based on the original data on the other storage device Data to recalculate. (Of course, if the faulty storage device has an image, the erasure code controller 530 shown in FIG. 5 can simply instruct to copy the data from the image to the replacement storage device.)

Rebuilding a failed storage device can be a time-consuming process. In some embodiments of the inventive concept, once a replacement storage device is installed, it can be rebuilt. In other embodiments of the inventive concept, as far as the storage device can be rebuilt in the idle time period, the erasure coding controller 530 shown in FIG. 5 can rebuild the storage device in the idle time period. However, if the virtual storage device is busy, the erasure coding controller 530 shown in FIG. 5 can postpone the reconstruction of the replacement storage device until the idle time occurs, and can be reconstructed as needed based on the request from the processor 110 shown in FIG. 1 Data from the faulty storage device. (Of course, such reconstructed data can be written to a replacement storage device without waiting for the complete reconstruction, so that it is no longer necessary to recalculate this data again later.)

Embodiments of the inventive concept can also support the initialization of storage devices. When a new storage device is added to the array (as a replacement storage device for a faulty storage device, or to increase the capacity of a virtual storage device), the new storage device can be initialized. Initialization may include preparing the storage device for an erasure coding scheme.

The initialization of the new storage device may also involve erasing existing data from the new storage device. For example, consider the scenario of renting a specific storage device to a customer. This customer's lease has ended, and the storage device can be reused for new customers. However, the data from the original customer may still be stored on the storage device. To prevent future customers from gaining access to the data of earlier customers, any desired mechanism can be used to erase the data on the storage device. For example, a table storing information about where data is stored can be erased. Or the data itself can be overwritten with new data (to prevent later attempts to recover any information that may have been deleted): new data can use a model designed to illustrate the way to ensure that the original data cannot be recovered. For example, the U.S. Department of Defense (DOD) has issued standards on how to erase data to prevent recovery: these standards can be used to erase old data on storage devices and then reuse the storage devices for new clients.

Initialization may not be limited to when a new storage device is hot added to an existing array. It can also be initialized when the storage device or the PCIe switch 125 with side-by-side erasure coding logic or the machine 105 shown in FIG. 1 as a whole is first powered together.

11A to 11D show a flowchart of an example process of the PCIe switch 125 with bypass code erasure coding logic shown in FIG. 1 to support the erasure coding schemes 405, 410, and 415. In FIG. 11A, at block 1103, the PCIe switch 125 with bypass code erasure coding logic shown in FIG. 3 may be initialized (perhaps through the BMC 325 shown in FIG. 3 or the processor 110 shown in FIG. 1). At block 1106, the PCIe switch 125 with side-by-side erasure coding logic shown in FIG. 3 may receive the transmission. This transmission may be a read or write request from the processor 110 shown in FIG. 1, a control transmission from the processor 110 shown in FIG. 1 or the BMC 325 shown in FIG. 3, or from the storage device 130-1 shown in FIG. 130-6 transmits in response to a read or write request from the processor 110 shown in FIG.

At block 1109, the detection logic 525 shown in FIG. 5 may determine whether the transmission is a control transmission from the processor 110 shown in FIG. If so, at block 1112, the PCIe switch 125 with side-by-side erasure coding logic shown in FIG. 3 may deliver the control transmission to the PPU 520 shown in FIG. 5, after which the process ends.

If the transmission is not a control transmission from the processor 110 shown in FIG. 1, at block 1115 (FIG. 11B), the detection logic 525 shown in FIG. 5 can determine whether the transmission is a read or write request from the host. If the transmission is not a read or write request from the host, at block 1118, the detection logic 525 shown in FIG. 5 may replace the device LBA with a host LBA suitable for the host during the transmission. The probe logic 525 shown in FIG. 5 can also modify the transmission to imply that the transmission comes from a virtual storage device, rather than a physical storage device that stores actual data. At block 1121, the PCIe switch 125 with bypass erasure coding logic shown in FIG. 3 may deliver the transmission to the processor 110 shown in FIG. 1, after which the process ends.

On the other hand, if the transmission is a read or write request from the processor 110 shown in FIG. 1, then at block 1124, the detection logic 525 shown in FIG. 5 may determine that the data in question is cached 545 shown in FIG. Or whether the write buffer 550 shown in FIG. 5 is available. If the data is available in cache 545 shown in FIG. 5 or write buffer 550 shown in FIG. 5, then at block 1127 (FIG. 11C), erasure coding controller 530 shown in FIG. 5 can be accessed from a suitable location data.

If the data is not available in the cache 545 shown in FIG. 5 or the write buffer 550 shown in FIG. 5, then at block 1130, the detection logic 525 shown in FIG. 5 may modify the transmission so that the storage device should read the data from it The device LBA replaces the host LBA provided by the host. The detection logic 525 shown in FIG. 5 may also modify the transmission to identify the appropriate storage device to receive the transmission. Next, at block 1133, the probe logic 525 may deliver the transmission to the appropriate storage device.

Regardless of whether the data in question can be accessed from the cache or from the storage device, at this time, the PCIe switch 125 with the bypass erasure coding logic shown in FIG. 3 has the required data. At this time, the processing may diverge. If the transmission is a read request from the processor 110 shown in FIG. 1, then at block 1136, the PCIe switch 125 with the bypass erasure coding logic shown in FIG. 3 may return the data to the processor 110 shown in FIG. As shown in block 1139, the detection logic 525 shown in FIG. 5 can also store data in the cache 545 shown in FIG. 5; block 1139 is optional and can be omitted as shown by the dashed line 1142. At this point, the process ends.

On the other hand, if the transfer from the processor 110 shown in FIG. 1 is a write request, then at block 1145, the erasure coding controller 530 shown in FIG. 5 can read the storage devices 130-1 through 3 shown in FIG. 130-6 strips. Block 1145 is actually a restatement of blocks 1127, 1130, and 1133, and may not be required; block 1145 is included in FIG. 11C to emphasize that writing data to the virtual storage device may involve cross storage devices 130-1 to 130 The entire strip of -6 reads the data. At block 1148, the erasure coding controller 530 shown in FIG. 5 may merge the data received from the processor 110 shown in FIG. 1 with the data stripe accessed from the cache or from the storage devices 130-1 to 130-6 .

At this time, depending on whether the PCIe switch 125 with bypass erasure coding logic shown in FIG. 3 includes the write buffer 550 shown in FIG. 5, the process may be diverged again. If the PCIe switch 125 with the bypass erasure coding logic shown in FIG. 3 includes the write buffer 550 shown in FIG. 5, then at block 1151 (FIG. 11D), the erasure coding controller 530 shown in FIG. 5 may merge The data stripe is written to the write buffer 550 shown in FIG. 5 (this data is marked as dirty and needs to be cleared to the storage devices 130-1 to 130-6). Next, at block 1154, the PCIe switch 125 with the bypass erasure coding logic shown in FIG. 3 may report the write request completion to the processor 110 shown in FIG. Note that if the write buffer 550 shown in FIG. 5 uses a write-back cache strategy, block 1154 is appropriate; if the write buffer 550 shown in FIG. 5 uses a write-through cache strategy, then block 1154 may be as follows The dotted line 1157 is omitted.

Finally, since the PCIe switch 125 with bypass erasure coding logic shown in FIG. 3 does not include the write buffer 550 shown in FIG. 5, or because the data in the write buffer 550 shown in FIG. 5 will be cleared to FIG. 3 The storage devices 130-1 to 130-6 are shown, so at block 1160, the erasure coding controller 530 shown in FIG. 5 can write the updated stripe back to the storage devices 130-1 to 130-6 shown in FIG. . Next, at block 1163, the PCIe switch 125 with the bypass erasure coding logic shown in FIG. 3 may report the write request completion to the processor 110 shown in FIG. Note that if the merged data has been stored in the write buffer 550 shown in FIG. 5 and the write buffer 550 shown in FIG. 5 uses a write-back cache strategy, the block 1163 is not necessary: The PCIe switch 125 depending on the erasure coding logic has reported that the write request is complete (at block 1154). In such a scenario, the block 1163 may be omitted as shown by the dashed line 1166. At this point, the process ends.

FIGS. 12A to 12B illustrate an exemplary process of the PCIe switch 125 with bypass erasure coding logic shown in FIG. 1 performing initialization according to an embodiment of the inventive concept. In FIG. 12A, at block 1205, the PCIe switch 125 with bypass code erasure coding logic shown in FIG. 3 determines whether the device connected to the PCIe switch 125 with bypass code erasure coding logic shown in FIG. 3 is only a storage device And it may have an erasure code managed by the PCIe switch 125 with the bypass erasure code logic shown in FIG. 3. If there is a device that is not a storage device or a device that may not have an erasure-coded storage device managed by the PCIe switch 125 with bypass erasure coding logic shown in FIG. 3, connect to the side-view erasure code shown in FIG. 3. Logical PCIe switch 125, in some embodiments of the inventive concept, at block 1210, the PCIe switch 125 with bypass erasure coding logic shown in FIG. 3 may disable the bypass erasure coding logic, after which, Processing ends.

However, in other embodiments of the inventive concept, even if a device that does not meet the qualification for erasure coding is connected to the PCIe switch 125 with the bypass erasure coding logic shown in FIG. 3, the device with the bypass erasure coding logic shown in FIG. 3 The PCIe switch 125 can also manage erasure coding. In these embodiments of the inventive concept, or if only storage devices eligible for erasure coding are connected to the PCIe switch 125 with bypass code erasure coding logic shown in FIG. 3, at block 1215, The PCIe switch 125 of the side-view erasure coding logic can enable the side-view erasure coding logic. Next, at block 1220 (FIG. 12B), the PCIe switch 125 with side-by-side erasure coding logic shown in FIG. 3 may be configured to use an erasure coding scheme (possibly through the BMC 325 shown in FIG. 3 or the processing shown in FIG. 1 110).

At block 1225, the PCIe switch 125 shown in FIG. 3 with bypass erasure coding logic may disable devices that do not qualify for erasure coding. Note that, as indicated by the dashed line 1230, block 1225 is optional: there may not be any devices that do not qualify for erasure coding connected to the PCIe switch 125 with bypass code erasure coding logic shown in FIG. 3, or Erasure coding is used, however, the PCIe switch 125 shown in FIG. 3 with bypass erasure coding logic may allow the processor 110 shown in FIG. 1 to access those devices that are not eligible for erasure coding.

At block 1235, for any device that has undergone erasure coding, the PCIe switch 125 with bypass code erasure coding logic shown in FIG. 3 may terminate the enumeration downstream of the PCIe switch 125 with bypass code erasure coding logic shown in FIG. . At block 1240, based on the storage devices 130-1 to 130-6 shown in FIG. 3 undergoing erasure coding, the PCIe switch 125 with bypass code erasure coding logic shown in FIG. 3 can report the virtual storage device to FIG. 1 Processor 110. The PCIe switch 125 with side-by-side erasure coding logic shown in FIG. 3 may also report any other PCIe devices that may be enumerated to the processor 110 shown in FIG. 1. At this point, the process ends.

FIG. 13 shows an exemplary process of incorporating the new storage device into the erasure coding schemes 405, 410, and 415 shown in FIG. 4 by the PCIe switch 125 with side-by-side erasure coding logic shown in FIG. 1 according to an embodiment of the inventive concept. Flow chart. In FIG. 13, at block 1305, the PCIe switch 125 (or the BMC 325 shown in FIG. 3) shown in FIG. 3 with side-by-side erasure coding logic can check for new storage devices. If a new storage device is detected, at block 1310, the erasure coding controller 530 shown in FIG. 5 may add the new storage device to the array behind the virtual storage device. Finally, at block 1315, the PCIe switch 125 (or the BMC 325 shown in FIG. 3, or the processor 110 shown in FIG. 1) with the bypass erasure coding logic shown in FIG. 3 can initialize the new storage device. At this point, the process may end, or may return to block 1305 as indicated by the dashed line 1320 to check for additional new storage devices.

FIG. 14 shows a flowchart of an exemplary process of the PCIe switch 125 with bypass erasure coding logic shown in FIG. 1 to handle a faulty storage device according to an embodiment of the inventive concept. In FIG. 14, at block 1405, the PCIe switch 125 (or BMC 325 shown in FIG. 3) shown in FIG. 3 with side-by-side erasure coding logic can check for failed (or removed) storage devices. If a faulty storage device is detected, then at block 1410, when a read request that would otherwise access data from the faulty storage device arrives, the erasure code controller 530 shown in FIG. 5 can implement the storage on the faulty storage device The erasure coding of the data is restored. Such erasure code recovery may involve reading data from a strip that includes the requested data from other storage devices and calculating the requested data based on the remaining data in the strip.

At block 1415, the PCIe switch 125 (or BMC 325 shown in FIG. 3) with the bypass erasure coding logic shown in FIG. 3 can determine whether the replacement storage device has been added to the array behind the virtual storage device. If the storage device has been added to the array behind the virtual storage device, at block 1420, the erasure coding controller 530 shown in FIG. 5 may use the replacement storage device to rebuild the failed storage device. At this point, the process may end, or may return to block 1405 as indicated by the dashed line 1425 to check for additional new storage devices.

In FIGS. 11A to 14, some embodiments of the inventive concept are shown. However, those skilled in the art will recognize that other embodiments of the inventive concept may be implemented by changing the order of the blocks, by omitting the blocks, or by including links not shown in the drawings. Regardless of whether it is explicitly stated or not, all such variations of the flowchart are considered as embodiments of the inventive concept.

Compared with the prior art, embodiments of the inventive concept provide technical advantages. Using a PCIe switch with side-by-side erasure coding logic will move the erasure code closer to the storage device, which will reduce the time required to move data around. Removing the erasure code from the processor reduces the load on the processor, allowing the processor to execute more instructions for the application. By using a configurable erasure coding controller, any desired erasure coding scheme can be used instead of the limited set of schemes supported by hardware and software erasure coding vendors. By placing the erasure coding controller with the PCIe switch, expensive RAID plug-in cards are no longer required, and larger arrays can be used even across multiple mainframes.

The following discussion is intended to provide a brief general description of a suitable machine or machines in which certain aspects of the inventive concept can be implemented. The one or more machines may be controlled, at least in part, by input from traditional input devices (eg, keyboard, mouse, etc.) and by instructions received from another machine, and virtual reality (VR) environment Interactive, biometric feedback or other input signals to control. The term "machine" as used herein is intended to broadly encompass a single machine, a virtual machine, or a system of communicatively coupled machines, virtual machines, or devices operating together. Exemplary machines include computing devices such as personal computers, workstations, servers, portable computers, handheld devices, telephones, tablets, and transportation devices such as private or public transportation vehicles (such as cars, trains, taxis, etc.).

The one or more machines may include an embedded controller, such as a programmable logic device or array or a non-programmable logic device or array, an application specific integrated circuit (ASIC), an embedded computer, a smart card, and so on. The one or more machines may utilize one or more connections to one or more remote machines (eg, via a network interface, modem, or other communication coupling method). The machines can be interconnected through physical and/or logical networks such as intranet, Internet, local area network, wide area network, etc. Those skilled in the art will understand that network communications can utilize a variety of wired and/or wireless short-range or remote carriers and protocols, including radio frequency (RF), satellite, microwave, electrical and electronic engineering institutes (Institute of Electrical and Electronics Engineers, IEEE) 802.11, Bluetooth, optical devices, infrared devices, cables, lasers, etc.

Embodiments of the inventive concept can be described with reference to or in conjunction with related data including functions, processes, data structures, applications, etc. When the related data is accessed by the machine, the machine can perform tasks or define abstract data types or Low-level hardware context. The associated data may be stored in, for example, volatile memory and/or non-volatile memory (eg, RAM, ROM, etc.) or stored in hard drives, floppy disks, optical memory, magnetic tape, flash memory, Other storage devices including memory stick, digital video disk, biological memory, etc. and their associated storage media. Related data can be delivered in a transmission environment including a physical network and/or a logical network in the form of packets, serial data, parallel data, and propagation signals, and can be used in a compressed or encrypted format. The associated data can be used in a distributed environment and stored locally and/or remotely for machine access.

Embodiments of the inventive concept may include a tangible, non-transitory machine-readable medium that includes instructions executable by one or more processors, the instructions including instructions for practicing described herein Instruction of elements of the inventive concept.

The various operations of the above method can be performed by any suitable tool capable of performing the operations (eg, various hardware and/or software components, circuits, and/or modules). The software may include an ordered list of executable instructions for implementing logical functions, and may be implemented in any "processor-readable medium" for instruction execution systems, devices, or devices (eg, single-core or multi-core processors) Or a system with a processor) or used in conjunction with an instruction execution system, device, or device.

The blocks or steps of the methods or algorithms and functions described in conjunction with the embodiments disclosed herein may be directly implemented in hardware, in a software module executed by a processor, or in a combination of the two. If implemented in software, the functions may be stored on or transmitted over as tangible, non-transitory computer-readable medium as one or more instructions or codes. Software modules can reside in random access memory (RAM), flash memory, read only memory (ROM), electrically programmable read only memory (EPROM), electrically erasable programmable read only memory (EEPROM), scratchpad, hard drive, removable disk, CD-ROM, or any other form of storage medium known in the art.

Given that the principles of the inventive concept have been explained and explained with reference to the illustrated embodiments, it will be appreciated that the illustrated embodiments can be modified in arrangement and detail without departing from such principles, and can be modified in any desired manner Way. Also, although the foregoing discussion has focused on specific embodiments, other configurations are also contemplated. In particular, even if expressions such as “embodiments according to the inventive concept” are used herein, these phrases are intended to roughly mention the possibility of the embodiments, and are not intended to limit the inventive concept to specific embodiment configurations . These terms used herein may refer to the same embodiment or different embodiments that can be combined into other embodiments.

The foregoing illustrative embodiments should not be construed as limiting the inventive concepts of the present invention. Although several embodiments have been described, those skilled in the art will readily understand that many modifications can be made to these embodiments without substantially departing from the novel teachings and advantages of the present disclosure. Therefore, all such modifications are intended to include the scope of the inventive concept as defined in the scope of the patent application.

Embodiments of the inventive concept can be extended to the following statements, but are not limited thereto:

Statement 1. Embodiments of the inventive concept include a peripheral element connection express (PCIe) switch with erasure coding logic, and a peripheral element connection express (PCIe) switch with erasure coding logic including:

External connector, which enables the PCIe switch to communicate with the processor;

At least one connector that enables the PCIe switch to communicate with at least one storage device;

Power processing unit (PPU), which handles the configuration of PCIe switches;

An erasure coding controller, including circuitry for applying an erasure coding scheme to data stored on the at least one storage device; and

The detection logic includes circuitry for intercepting data transmissions received at the PCIe switch and modifying the data transmission in response to the erasure coding scheme.

Statement 2. Embodiments of the inventive concept include a PCIe switch with erasure coding logic according to statement 1, wherein the erasure coding logic includes at least one of side-by-side erasure coding logic and perspective erasure coding logic.

Statement 3. An embodiment of the inventive concept includes a PCIe switch with erasure coding logic according to Statement 1, wherein the at least one storage device includes at least one non-volatile storage express (NVMe) solid-state drive (SSD).

Statement 4. An embodiment of the inventive concept includes a PCIe switch with erasure coding logic according to statement 3, where the probe logic is operable to intercept control transmissions received at the PCIe switch and forward the control transmissions to the PPU.

Statement 5. An embodiment of the inventive concept includes a PCIe switch with erasure coding logic according to statement 3, wherein the detection logic is operable to intercept data transmissions received from the host at the PCIe switch, and the data transmission At least one device LBA used by the NVMe SSD replaces the host logical block address (LBA) used by the host.

Statement 6. An embodiment of the inventive concept includes a PCIe switch with erasure coding logic according to Statement 5, wherein the probe logic is further operable to direct data transfer to the at least one NVMe SSD.

Statement 7. An embodiment of the inventive concept includes a PCIe switch with erasure coding logic according to Statement 3, wherein the detection logic is operable to intercept data transmissions received from one of the at least one NVMe SSD at the PCIe switch In the data transmission, the host LBA used by the host replaces the device LBA used by the one NVMe SSD in the at least one NVMe SSD.

Statement 8. An embodiment of the inventive concept includes a PCIe switch with erasure coding logic according to statement 3, and a PCIe switch with erasure coding logic further includes a cache.

Statement 9. Embodiments of the inventive concept include a PCIe switch with erasure coding logic according to Statement 8, wherein the probe logic is operable to return data to the data based at least in part on the presence of data requested in the data transfer from the host in the cache The transmitted response.

Statement 10. An embodiment of the inventive concept includes a PCIe switch with erasure coding logic according to Statement 3, wherein:

The PCIe switch is located in the mainframe; and

The main case includes memory used by the erasure code controller as an external cache.

Statement 11. An embodiment of the inventive concept includes a PCIe switch with erasure coding logic according to statement 3, and a PCIe switch with erasure coding logic further includes a write buffer.

Statement 12. An embodiment of the inventive concept includes a PCIe switch with erasure coding logic according to Statement 11, where:

Data transmission includes write operations from the host; and

The erasure coding controller is operable to complete the write operation after sending a response to the data transmission to the host.

Statement 13. An embodiment of the inventive concept includes a PCIe switch with erasure coding logic according to statement 11, wherein the erasure coding controller is operable to store data in a write operation in a write buffer.

Statement 14. Embodiments of the inventive concept include a PCIe switch with erasure coding logic according to statement 3, wherein the PCIe switch is operable to be based at least in part on all NVMe SSDs in the at least one NVMe SSD with erasure coding controllers Used together to enable erasure coding controller and detection logic.

Statement 15. Embodiments of the inventive concept include a PCIe switch with erasure coding logic according to statement 3, wherein the PCIe switch is operable to disable erasure based at least in part on the at least one NVMe SSD including a built-in erasure coding function Encoding controller and detection logic.

Statement 16. Embodiments of the inventive concept include a PCIe switch with erasure coding logic according to statement 15, wherein the PCIe switch is operable to erase based at least in part on the at least one NVMe SSD including a built-in erasure coding function The coding controller and detection logic are disabled to inform the user.

Statement 17. An embodiment of the inventive concept includes a PCIe switch with erasure coding logic according to Statement 3, wherein the PCIe switch is operable to connect to the PCIe switch based at least in part on at least one non-storage device using the at least one connector Disable erasure coding controller and detection logic.

Statement 18. Embodiments of the inventive concept include a PCIe switch with erasure coding logic according to Statement 17, wherein the PCIe switch is operable to connect to the PCIe using the at least one connector based at least in part on the at least one non-storage device The switch notifies the user that the erasure code controller and detection logic are disabled.

Statement 19. Embodiments of the inventive concept include a PCIe switch with erasure coding logic according to statement 3, wherein the PCIe switch is operable to enable erasure coding controller and detection logic together with the at least one NVMe SSD and prevent Access to non-storage devices connected to the PCIe switch using the at least one connector.

Statement 20. An embodiment of the inventive concept includes a PCIe switch with erasure coding logic according to statement 19, wherein the PCIe switch is operable to notify the user that access to non-storage devices connected to the PCIe switch is blocked.

Statement 21. Embodiments of the inventive concept include a PCIe switch with erasure coding logic according to statement 3, wherein the PCIe switch is operable to use the erasure coding controller and detection logic to manage at least one add-on connected to the second PCIe switch Erase coding scheme on NVMe SSD.

Statement 22. An embodiment of the inventive concept includes a PCIe switch with erasure coding logic according to statement 21, wherein the second PCIe switch is operable to disable the second erasure coding controller and the second detection logic in the second PCIe switch .

Statement 23. An embodiment of the inventive concept includes a PCIe switch with erasure coding logic according to Statement 22, where:

The PCIe switch is located in the first mainframe; and

The second PCIe switch is located in the second mainframe.

Statement 24. An embodiment of the inventive concept includes a PCIe switch with erasure coding logic according to Statement 3, where the PCIe switch is implemented using a field programmable gate array (FPGA).

Statement 25. An embodiment of the inventive concept includes a PCIe switch with erasure coding logic according to Statement 3, wherein:

The at least one NVMe SSD includes at least two NVMe SSDs; and

The PCIe switch and the at least two NVMe SSDs are located inside a common housing.

Statement 26. An embodiment of the inventive concept includes a PCIe switch with erasure coding logic according to statement 3, wherein the PCIe switch and the at least one NVMe SSD are located in separate housings.

Statement 27. An embodiment of the inventive concept includes a PCIe switch with erasure coding logic according to Statement 3, wherein:

The PCIe switch is operable to detect a faulty NVMe SSD of the at least one NVMe SSD; and

The erasure coding controller is operable to handle data transmission in response to a faulty NVMe SSD.

Statement 28. An embodiment of the inventive concept includes a PCIe switch with erasure coding logic according to statement 27, wherein the erasure coding controller is operable to perform erasure coding recovery of data stored on a failed NVMe SSD.

Statement 29. An embodiment of the inventive concept includes a PCIe switch with erasure coding logic according to statement 28, wherein the erasure coding controller is operable to replace the NVMe SSD for a failed NVMe SSD reconstruction.

Statement 30. An embodiment of the inventive concept includes a PCIe switch with erasure coding logic according to Statement 3, wherein:

The PCIe switch can operate to detect the new NVMe SSD; and

The erasure coding controller is operable to use the new NVMe SSD as part of the erasure coding scheme.

Statement 31. An embodiment of the inventive concept includes a PCIe switch with erasure coding logic according to statement 30, wherein the erasure coding controller is operable to perform a capacity increase using a new NVMe SSD.

Statement 32. An embodiment of the inventive concept includes a PCIe switch with erasure coding logic according to statement 30, wherein the PCIe switch is operable to detect a new NVMe SSD connected to one of the at least one connector.

Statement 33. An embodiment of the inventive concept includes a PCIe switch with erasure coding logic according to statement 30, wherein the PCIe switch is operable to detect a new NVMe SSD through a message from a second PCIe switch.

Statement 34. An embodiment of the inventive concept includes a PCIe switch with erasure coding logic according to statement 33, wherein a new NVMe SSD is connected to a second connector on a second PCIe switch.

Statement 35. An embodiment of the inventive concept includes a PCIe switch with erasure coding logic according to Statement 3, wherein the at least one connector includes presence pins for detecting both a faulty NVMe SSD and a new NVMe SSD.

Statement 36. An embodiment of the inventive concept includes a PCIe switch with erasure coding logic according to statement 3, wherein the PCIe switch is operable to present itself as a single device to the host and prevent downstream PCIe to the at least one NVMe SSD Busbar enumeration.

Statement 37. An embodiment of the inventive concept includes a PCIe switch with erasure coding logic according to statement 36, wherein the PCIe switch is further operable to prevent downstream PCIe bus enumeration of the second PCIe switch downstream of the PCIe switch.

Statement 38. An embodiment of the inventive concept includes a PCIe switch with erasure coding logic according to statement 36, wherein the PCIe switch is operable to virtualize the at least one NVMe SSD.

Statement 39. An embodiment of the inventive concept includes a PCIe switch with erasure coding logic according to statement 3, wherein the erasure coding controller is operable to connect a new NVMe SSD connected to one of the at least one connector initialization.

Statement 40. An embodiment of the inventive concept includes a PCIe switch with erasure coding logic according to statement 39, wherein the erasure coding controller is operable to initialize a new NVMe SSD after a hot insertion event.

Statement 41. An embodiment of the inventive concept includes a PCIe switch with erasure coding logic according to statement 39, wherein the erasure coding controller is further operable to initialize the at least one NVMe SSD at startup.

Statement 42. An embodiment of the inventive concept includes a PCIe switch with erasure coding logic according to Statement 3, where the PCIe switch is part of a system that includes a baseboard management controller (BMC) that is operable to connect to A new NVMe SSD of one of the at least one connector is initialized.

Statement 43. An embodiment of the inventive concept includes a PCIe switch with erasure coding logic according to statement 42, wherein the BMC is operable to initialize the at least one NVMe SSD at startup.

Statement 44. An embodiment of the inventive concept includes a PCIe switch with erasure coding logic according to statement 3, wherein the erasure coding controller includes a stripe manager for striping data across the at least one NVMe SSD.

Statement 45. An embodiment of the inventive concept includes a method, the method comprising:

Receive transmissions at peripheral component connection express (PCIe) switches with erasure coding logic;

Use probing logic in erasure coding logic to process the transmission; and

Deliver the transmission to its destination through the PCIe switch.

Statement 46. An embodiment of the inventive concept includes a method according to Statement 45, wherein the erasure coding logic includes at least one of side-view erasure coding logic and perspective erasure coding logic.

Statement 47. An embodiment of the inventive concept includes a method according to Statement 45, wherein:

Processing the transmission using the detection logic in the erasure coding logic includes determining by the detection logic that the transmission includes the control transmission; and

Delivering the transmission to its destination through the PCIe switch includes delivering the transmission to the power processing unit (PPU).

Statement 48. An embodiment of the inventive concept includes a method according to Statement 45, wherein processing the transmission using probe logic in erasure coding logic includes processing the transmission using probe logic for active use based at least in part on the erasure coding logic.

Statement 49. An embodiment of the inventive concept includes a method according to Statement 45, wherein:

Receiving transmissions at peripheral component connection express (PCIe) switches with erasure coding logic includes receiving read requests from the host;

Using probe logic in erasure coding logic to process the transmission includes replacing the host LBA with the device logical block address (LBA) in the read request; and

Delivering the transmission to its destination through the PCIe switch includes delivering the read request to a non-volatile storage express (NVMe) solid-state drive (SSD).

Statement 50. An embodiment of the inventive concept includes a method according to statement 49, wherein processing the transmission using probe logic in erasure coding logic further includes identifying the NVMe SSD to which the read request should be delivered.

Statement 51. An embodiment of the inventive concept includes a method according to Statement 49, in which:

Processing the transmission using the detection logic in the erasure coding logic further includes accessing the data requested by the host in the read request from the cache based at least in part on the data residing in the cache;

Replacing the host LBA with the device logical block address (LBA) in the read request includes replacing the host LBA with the device LBA in the read request based at least in part on the data not residing in the cache; and

Delivering the transmission to its destination through the PCIe switch includes delivering the read request to the NVMe SSD based at least in part on the material not residing in the cache.

Statement 52. An embodiment of the inventive concept includes a method according to Statement 45, wherein:

Receiving a transmission at a peripheral component connection express (PCIe) switch with erasure coding logic includes receiving a write request from the host;

Using probe logic in erasure coding logic to process the transmission includes replacing the host LBA with the device LBA in the write request; and

Delivering the transmission to its destination through the PCIe switch includes delivering the write request to the NVMe SSD.

Statement 53. An embodiment of the inventive concept includes a method according to Statement 52, wherein using probe logic in erasure coding logic to process the transmission further includes identifying the NVMe SSD to which the write request should be delivered.

Statement 54. An embodiment of the inventive concept includes a method according to Statement 52, the method further comprising:

Read block stripes from at least one NVMe SSD;

Merge the data written in the request with the block stripe to form an updated block stripe; and

Write the updated block stripe to the at least one NVMe SSD.

Statement 55. An embodiment of the inventive concept includes a method according to statement 54, wherein combining the data in the write request includes calculating additional data to be written to the at least one NVMe SSD in addition to the data in the write request .

Statement 56. An embodiment of the inventive concept includes a method according to Statement 54, wherein:

The method further includes reading the block stripe from the cache based at least in part on the block stripe residing in the cache; and

Reading the block stripes from the at least one NVMe SSD includes reading the block stripes from the at least one NVMe SSD based at least in part on the block stripes not residing in the cache.

Statement 57. An embodiment of the inventive concept includes the method according to statement 54, wherein writing the updated block stripe to the at least one NVMe SSD includes writing the updated block stripe to the write buffer.

Statement 58. An embodiment of the inventive concept includes a method according to statement 57, the method further comprising writing to the write buffer after the updated block stripe is written to and after the updated block stripe is written The response to the host is completed before the at least one NVMe SSD is completed.

Statement 59. An embodiment of the inventive concept includes a method according to Statement 45, wherein:

Receiving a transmission at a PCIe switch with peripheral code erasure coding logic includes receiving a response from the NVMe SSD;

Use the detection logic in the erasure coding logic to process the transmission including replacing the device LBA with the host LBA in the response; and

Delivering the transmission to its destination through the PCIe switch includes delivering the response to the host.

Statement 60. An embodiment of the inventive concept includes a method according to statement 59, wherein processing the transmission using probe logic in erasure coding logic further includes replacing the identifier of the NVMe SSD with an identifier of the virtual storage device.

Statement 61. An embodiment of the inventive concept includes the method according to Statement 45, wherein delivering the transmission to its destination through the PCIe switch includes delivering the transmission to the second PCIe switch to which the NVMe SSD is connected, the NVMe SSD being the destination.

Statement 62. An embodiment of the inventive concept includes the method according to Statement 61, wherein the PCIe switch is located in the first mainframe and the second PCIe switch is located in the second mainframe.

Statement 63. An embodiment of the inventive concept includes the method according to Statement 45, the method further comprising initializing at least one NVMe SSD connected to the PCIe switch for use with erasure coding.

Statement 64. An embodiment of the inventive concept includes a method according to Statement 45, the method further comprising:

A new NVMe SSD is detected connected to the PCIe switch; and

Add the new NVMe SSD to the capacity of the virtual storage device.

Statement 65. An embodiment of the inventive concept includes a method according to statement 64, the method further comprising initializing a new NVMe SSD for use with erasure coding.

Statement 66. An embodiment of the inventive concept includes a method according to Statement 45, the method further comprising:

Detect a faulty NVMe SSD connected to a PCIe switch; and

Perform erasure code recovery on the data stored on the faulty NVMe SSD.

Statement 67. An embodiment of the inventive concept includes a method according to Statement 66, the method further comprising:

Replacement NVMe SSD for faulty NVMe SSD detection; and

Use the replacement NVMe SSD to rebuild the faulty NVMe SSD.

Statement 68. An embodiment of the inventive concept includes a method according to Statement 45, the method further comprising:

It is detected that only NVMe SSD without erasure coding is connected to the PCIe switch; and

Enable the erasure coding logic in the PCIe switch.

Statement 69. An embodiment of the inventive concept includes a method according to Statement 68, the method further comprising terminating the PCIe bus enumeration downstream of the PCIe switch.

Statement 70. An embodiment of the inventive concept includes a method according to Statement 68, the method further comprising reporting a virtual storage device to a host, the capacity of the virtual storage device is based at least in part on the capacity of an NVMe SSD connected to a PCIe switch and Erase coding scheme.

Statement 71. An embodiment of the inventive concept includes a method according to Statement 45, the method further comprising:

It is detected that at least one non-storage device or at least one NVMe SSD with erasure coding function is connected to the PCIe switch; and

Disable the erasure coding logic in the PCIe switch.

Statement 72. An embodiment of the inventive concept includes a method according to Statement 45, the method further comprising:

It is detected that at least one non-storage device or at least one NVMe SSD with erasure coding function is connected to the PCIe switch;

Enable erasure coding logic in PCIe switches; and

Disable the at least one non-storage device or the at least one NVMe SSD with erasure coding function.

Statement 73. An embodiment of the inventive concept includes a method according to Statement 72, the method further comprising terminating the PCIe bus enumeration downstream of the PCIe switch.

Statement 74. An embodiment of the inventive concept includes a method according to Statement 72, the method further comprising reporting a virtual storage device to a host, the capacity of the virtual storage device is based at least in part on the capacity of an NVMe SSD connected to a PCIe switch and Erase coding scheme.

Statement 75. An embodiment of the inventive concept includes a method according to Statement 45, the method further comprising configuring a PCIe switch with erasure coding logic to use an erasure coding scheme.

Statement 76. An embodiment of the inventive concept includes a method according to Statement 75, wherein configuring a PCIe switch with erasure coding logic to use an erasure coding scheme includes using a baseboard management controller (BMC) to configure a PCIe with erasure coding logic The switch is configured to use an erasure coding scheme.

Statement 77. An embodiment of the inventive concept includes an article that includes a non-transitory storage medium that has instructions stored thereon that when executed by a machine cause:

Receive transmissions at peripheral component connection express (PCIe) switches with erasure coding logic;

Use the detection logic in the erasure coding logic to process the transmission; and

Deliver the transmission to its destination through the PCIe switch.

Statement 78. An embodiment of the inventive concept includes an article according to Statement 77, wherein the erasure coding logic includes at least one of side-view erasure coding logic and perspective erasure coding logic.

Statement 79. An embodiment of the inventive concept includes an article according to Statement 77, in which:

Processing the transmission using the detection logic in the erasure coding logic includes determining by the detection logic that the transmission includes the control transmission; and

Delivering the transmission to its destination through the PCIe switch includes delivering the transmission to the power processing unit (PPU).

Statement 80. An embodiment of the inventive concept includes an article according to statement 77, wherein using probe logic in erasure coding logic to process the transmission includes using probe logic to process the transmission for active use based at least in part on the erasure coding logic.

Statement 81. An embodiment of the inventive concept includes an article according to Statement 77, in which:

Receiving transmissions at peripheral component connection express (PCIe) switches with erasure coding logic includes receiving read requests from the host;

Using probe logic in erasure coding logic to process the transmission includes replacing the host LBA with the device logical block address (LBA) in the read request; and

Delivering the transmission to its destination through the PCIe switch includes delivering the read request to a non-volatile storage express (NVMe) solid-state drive (SSD).

Statement 82. An embodiment of the inventive concept includes an item according to statement 81, wherein processing the transmission using probe logic in erasure coding logic further includes identifying the NVMe SSD to which the read request should be delivered.

Statement 83. An embodiment of the inventive concept includes an article according to Statement 81, in which:

Processing the transmission using the detection logic in the erasure coding logic further includes accessing the data requested by the host in the read request from the cache based at least in part on the data residing in the cache;

Replacing the host LBA with the device logical block address (LBA) in the read request includes replacing the host LBA with the device LBA in the read request based at least in part on the data not residing in the cache; and

Delivering the transmission to its destination through the PCIe switch includes delivering the read request to the NVMe SSD based at least in part on the material not residing in the cache.

Statement 84. An embodiment of the inventive concept includes an article according to Statement 77, in which:

Receiving a transmission at a peripheral component connection express (PCIe) switch with erasure coding logic includes receiving a write request from the host;

Using probe logic in erasure coding logic to process the transmission includes replacing the host LBA with the device LBA in the write request; and

Delivering the transmission to its destination through the PCIe switch includes delivering the write request to the NVMe SSD.

Statement 85. An embodiment of the inventive concept includes an article according to statement 84, wherein processing the transmission using probe logic in erasure coding logic further includes identifying the NVMe SSD to which the write request should be delivered.

Statement 86. An embodiment of the inventive concept includes an article according to Statement 84, on which non-transitory storage media stores further instructions that when executed by a machine cause:

Read block stripes from at least one NVMe SSD;

Merge the data written in the request with the block stripe to form an updated block stripe; and

Write the updated block stripe to the at least one NVMe SSD.

Statement 87. An embodiment of the inventive concept includes an item according to statement 86, wherein combining the data in the write request includes calculating additional data to be written to the at least one NVMe SSD in addition to the data in the write request .

Statement 88. An embodiment of the inventive concept includes an article according to statement 86, in which:

Further instructions are stored on the non-transitory storage medium, which when executed by the machine causes the block stripe to be read from the cache based at least in part on the block stripe residing in the cache; and

Reading the block stripes from the at least one NVMe SSD includes reading the block stripes from the at least one NVMe SSD based at least in part on the block stripes not residing in the cache.

Statement 89. An embodiment of the inventive concept includes the article according to statement 86, wherein writing the updated block stripe to the at least one NVMe SSD includes writing the updated block stripe to the write buffer.

Statement 90. An embodiment of the inventive concept includes an article according to statement 89, on which a non-transitory storage medium stores further instructions which, when executed by the machine, cause the block stripes that have been updated to be written After writing to the buffer and before the updated block stripe is written to the at least one NVMe SSD, it responds to the host.

Statement 91. An embodiment of the inventive concept includes an article according to Statement 77, in which:

Receiving a transmission at a PCIe switch with peripheral code erasure coding logic includes receiving a response from the NVMe SSD;

Use the detection logic in the erasure coding logic to process the transmission including replacing the device LBA with the host LBA in the response; and

Delivering the transmission to its destination through the PCIe switch includes delivering the response to the host.

Statement 92. An embodiment of the inventive concept includes an item according to Statement 91, wherein processing the transmission using probe logic in erasure coding logic further includes replacing the identification word of the NVMe SSD with the identification word of the virtual storage device.

Statement 93. An embodiment of the inventive concept includes an article according to Statement 77, wherein delivering the transmission to its destination through the PCIe switch includes delivering the transmission to the second PCIe switch to which the NVMe SSD is connected, the NVMe SSD being the destination.

Statement 94. An embodiment of the inventive concept includes the article according to statement 93, wherein the PCIe switch is located in the first main chassis and the second PCIe switch is located in the second main chassis.

Statement 95. An embodiment of the inventive concept includes the item according to Statement 77, further instructions are stored on the non-transitory storage medium, which when executed by the machine initializes at least one NVMe SSD connected to the PCIe switch, to Used with erasure coding.

Statement 96. An embodiment of the inventive concept includes an article according to Statement 77, on which non-transitory storage media stores further instructions which when executed by a machine cause:

It is detected that the new NVMe SSD is connected to the PCIe switch; and

Add the new NVMe SSD to the capacity of the virtual storage device.

Statement 97. An embodiment of the inventive concept includes the article according to statement 96, where further instructions are stored on the non-transitory storage medium that when executed by the machine causes the new NVMe SSD to be initialized for use with erasure coding .

Statement 98. An embodiment of the inventive concept includes an article according to Statement 77, on which non-transitory storage media stores further instructions that when executed by a machine cause:

Detect a faulty NVMe SSD connected to a PCIe switch; and

Perform erasure code recovery on the data stored on the faulty NVMe SSD.

Statement 99. An embodiment of the inventive concept includes an article according to Statement 98, on which non-transitory storage media stores further instructions that when executed by a machine cause:

Detect the replacement NVMe SSD for the faulty NVMe SSD; and

Use the replacement NVMe SSD to rebuild the faulty NVMe SSD.

Statement 100. An embodiment of the inventive concept includes an article according to statement 77, on which non-transitory storage media stores further instructions that when executed by a machine cause:

It is detected that only the NVMe SSD without erasure coding is connected to the PCIe switch; and

Enable the erasure coding logic in the PCIe switch.

Statement 101. An embodiment of the inventive concept includes an item according to Statement 100, on which non-transitory storage media stores further instructions that when executed by a machine causes termination of PCIe bus enumeration downstream of the PCIe switch.

Statement 102. An embodiment of the inventive concept includes an item according to Statement 100, on which non-transitory storage media stores further instructions that when executed by a machine causes a virtual storage device to be reported to the host, the virtual storage device The capacity is based at least in part on the capacity of the NVMe SSD connected to the PCIe switch and the erasure coding scheme.

Statement 103. An embodiment of the inventive concept includes an article according to statement 77, on which non-transitory storage media stores further instructions, which when executed by a machine cause:

It is detected that at least one non-storage device or at least one NVMe SSD with erasure coding function is connected to the PCIe switch; and

Disable the erasure coding logic in the PCIe switch.

Statement 104. An embodiment of the inventive concept includes an article according to Statement 77, on which non-transitory storage media stores further instructions that when executed by a machine cause:

It is detected that at least one non-storage device or at least one NVMe SSD with erasure coding function is connected to the PCIe switch;

Enable the erasure coding logic in the PCIe switch; and

Disable the at least one non-storage device or the at least one NVMe SSD with erasure coding function.

Statement 105. An embodiment of the inventive concept includes the item according to statement 104, on which non-transitory storage media stores further instructions that when executed by the machine cause termination of the PCIe bus enumeration downstream of the PCIe switch.

Statement 106. An embodiment of the inventive concept includes an article according to statement 104, on which non-transitory storage media stores further instructions that when executed by a machine causes the virtual storage device to be reported to the host, the virtual storage device The capacity is based at least in part on the capacity of the NVMe SSD connected to the PCIe switch and the erasure coding scheme.

Statement 107. An embodiment of the inventive concept includes an item according to statement 77, on which non-transitory storage media stores further instructions that when executed by a machine causes the PCIe switch with erasure coding logic to be configured to use erasure Except coding scheme.

Statement 108. An embodiment of the inventive concept includes an article according to statement 107, wherein configuring a PCIe switch with erasure coding logic to use an erasure coding scheme includes using a baseboard management controller (BMC) to configure a PCIe with erasure coding logic The switch is configured to use an erasure coding scheme.

Statement 109. An embodiment of the inventive concept includes a system including:

Non-volatile storage fast (NVMe) solid state drive (SSD);

Field programmable gate array (FPGA) that implements one or more functions that support NVMe SSD, the functions including at least one of data acceleration, deduplication, data integrity, data encryption, and data compression; and

Peripheral components connected to Express (PCIe) switches

PCIe switch communicates with FPGA and NVMe SSD.

Statement 110. An embodiment of the inventive concept includes a system according to statement 109, wherein the FPGA and NVMe SSD are located inside a common housing.

Statement 111. An embodiment of the inventive concept includes a system according to Statement 110, in which the PCIe switch is located outside of a common housing that includes an FPGA and an NVMe SSD.

Statement 112. An embodiment of the inventive concept includes a system according to Statement 109, in which:

The PCIe switch is connected to the FPGA; and

The FPGA is connected to the NVMe SSD.

Statement 113. An embodiment of the inventive concept includes a system according to Statement 109, in which:

The PCIe switch is connected to the NVMe SSD; and

The NVMe SSD is connected to the FPGA.

Statement 114. An embodiment of the inventive concept includes a system according to statement 109, wherein the PCIe switch includes erasure coding logic, and the erasure coding logic includes an erasure coding controller.

Statement 115. An embodiment of the inventive concept includes a system according to statement 114, wherein the erasure coding logic includes at least one of side-view erasure coding logic and perspective erasure coding logic.

Statement 116. An embodiment of the inventive concept includes a system according to statement 114, wherein the erasure coding logic is operable to return to the read based at least in part on the presence of the data requested in the read request from the host in the cache Requested response.

Statement 117. Embodiments of the inventive concept include a system according to statement 116, wherein the erasure coding logic further includes cache.

Statement 118. An embodiment of the inventive concept includes a system according to statement 116, wherein:

The PCIe switch is located in the mainframe; and

The main case includes memory used as cache by erasure coding logic.

Statement 119. Embodiments of the inventive concept include a system according to statement 114, wherein the erasure coding logic is operable to return a response to the write request to the host before completing the write request.

Statement 120. An embodiment of the inventive concept includes a system according to statement 119, in which:

The PCIe switch further includes a write buffer; and

The erasure coding controller is operable to store the data in the write request in the write buffer.

Statement 121. An embodiment of the inventive concept includes a system according to statement 114, wherein the erasure coding logic includes side-by-side erasure coding logic, and the side-view erasure-coding logic includes detection logic.

Statement 122. An embodiment of the inventive concept includes a system according to statement 114, wherein the erasure coding logic is operable to intercept control transmissions received at the PCIe switch and forward the control transmissions to the power processing unit (PPU).

Statement 123. An embodiment of the inventive concept includes a system according to statement 114, wherein the erasure coding logic is operable to intercept data transmissions received from the host at the PCIe switch, and in the data transmission with device logic used by the NVMe SSD Block address (LBA) replaces the host LBA used by the host.

Statement 124. An embodiment of the inventive concept includes a system according to statement 123, wherein the erasure coding logic is further operable to direct data transfer to the NVMe SSD.

Statement 125. An embodiment of the inventive concept includes a system according to statement 114, wherein the erasure coding logic is operable to intercept data transfers received from the NVMe SSD at the PCIe switch, and use the host LBA used by the host in the data transfer Replace the device LBA used by NVMe SSD.

Statement 126. An embodiment of the inventive concept includes a system according to statement 114, wherein the erasure coding logic defines a virtual storage device that spans the NVMe SSD and the second NVMe SSD.

Statement 127. Embodiments of the inventive concept include a system according to statement 114, wherein the PCIe switch is operable to enable erasure coding logic based at least in part on the NVMe SSD being able to be used with erasure coding logic.

Statement 128. An embodiment of the inventive concept includes a system according to statement 114, the system further comprising a second device connected to a PCIe switch with erasure coding logic.

Statement 129. An embodiment of the inventive concept includes a system according to statement 128, wherein the second device includes at least one of a storage device, an SSD with a field programmable gate array (FPGA), and a graphics processing unit (GPU).

Statement 130. An embodiment of the inventive concept includes a system according to statement 128, in which:

The second device cannot be used with erasure coding logic; and

The PCIe switch is operable to disable erasure coding logic based at least in part on the inability of the second device to be used with erasure coding logic.

Statement 131. An embodiment of the inventive concept includes a system according to Statement 128, in which:

The second device cannot be used with erasure coding logic; and

The PCIe switch is operable to enable erasure coding logic based at least in part on the NVMe SSD being able to be used with erasure coding logic, and enable access to the second device without using erasure coding logic.

Statement 132. An embodiment of the inventive concept includes a system according to statement 128, in which:

The second device cannot be used with erasure coding logic; and

The PCIe switch is operable to enable erasure coding logic based at least in part on the NVMe SSD can be used with erasure coding logic, and disable access to the second device.

Statement 133. Embodiments of the inventive concept include a system, the system including:

Non-volatile storage fast (NVMe) solid-state drive (SSD); and

Field programmable gate array (FPGA). The FPGA includes a first FPGA part and a second FPGA part. The first FPGA part implements one or more functions that support NVMe SSD. The functions include data acceleration, deduplication, At least one of data integrity, data encryption and data compression, and the second FPGA part implements peripheral component connection express (PCIe) switches,

PCIe switch communicates with FPGA and NVMe SSD, and

Among them, FPGA and NVMe SSD are located inside the common housing.

Statement 134. An embodiment of the inventive concept includes a system according to statement 133, wherein the PCIe switch includes erasure coding logic, and the erasure coding logic includes an erasure coding controller.

Statement 135. Embodiments of the inventive concept include a system according to statement 134, wherein the erasure coding logic defines a virtual storage device that spans at least two parts of the NVMe SSD.

Statement 136. An embodiment of the inventive concept includes a system according to statement 134, wherein the erasure coding logic defines a virtual storage device that spans the NVMe SSD and the second NVMe SSD.

Statement 137. An embodiment of the inventive concept includes the system according to statement 136, wherein the second NVMe SSD is located inside the common housing.

Statement 138. An embodiment of the inventive concept includes the system according to statement 136, wherein the second NVMe SSD is located outside the common housing.

Statement 139. Embodiments of the inventive concept include a system according to statement 134, wherein the erasure coding logic includes at least one of side-view erasure coding logic and perspective erasure coding logic.

Statement 140. An embodiment of the inventive concept includes a system according to statement 134, wherein the erasure coding logic is operable to return to the read based at least in part on the presence of the data requested in the read request from the host in the cache Requested response.

Statement 141. An embodiment of the inventive concept includes a system according to statement 140, wherein the FPGA further includes a cache.

Statement 142. An embodiment of the inventive concept includes a system according to statement 140, wherein:

The common casing is located in the main casing; and

The main case includes memory used as cache by erasure coding logic.

Statement 143. An embodiment of the inventive concept includes a system according to statement 134, wherein the erasure coding logic is operable to return a response to the write request to the host before completing the write request.

Statement 144. An embodiment of the inventive concept includes a system according to statement 143, in which:

The FPGA further includes a write buffer; and

The erasure coding controller is operable to store the data in the write request in the write buffer.

Statement 145. Embodiments of the inventive concept include a system according to statement 134, wherein the erasure coding logic includes side-by-side erasure coding logic, and the side-view erasure-coding logic includes detection logic.

Statement 146. An embodiment of the inventive concept includes a system according to statement 145, wherein the probe logic is operable to intercept control transmissions received at the PCIe switch and forward the control transmissions to the power processing unit (PPU).

Statement 147. An embodiment of the inventive concept includes a system according to statement 134, wherein the erasure coding logic is operable to intercept data transmissions received from the host at the PCIe switch, and in the data transmission with device logic used by the NVMe SSD Block address (LBA) replaces the host LBA used by the host.

Statement 148. Embodiments of the inventive concept include a system according to statement 147, wherein the erasure coding logic is further operable to direct data transfer to the NVMe SSD.

Statement 149. An embodiment of the inventive concept includes a system according to statement 134, where the erasure coding logic is operable to intercept data transfers received from the NVMe SSD at the PCIe switch, and use the host LBA used by the host in the data transfer Replace the device LBA used by NVMe SSD.

Statement 150. An embodiment of the inventive concept includes a system according to statement 134, wherein a PCIe switch with erasure coding logic is operable to enable erasure coding logic based at least in part on NVMe SSDs that can be used with erasure coding logic.

Statement 151. Embodiments of the inventive concept include a system according to statement 134, wherein a PCIe switch with erasure coding logic is operable to disable erasure coding logic based at least in part on the NVMe SSD cannot be used with erasure coding logic.

Statement 152. Embodiments of the inventive concept include a system that includes:

Non-volatile storage fast (NVMe) solid-state drive (SSD); and

Peripheral components with erasure-coded logic connect to PCIe switches, including:

External connector, which enables the PCIe switch to communicate with the processor;

At least one connector that enables the PCIe switch to communicate with the NVMe SSD;

Power Processing Unit (PPU) with PCIe switches; and

The erasure coding controller includes circuitry for applying the erasure coding scheme to the data stored on the NVMe SSD.

Statement 153. Embodiments of the inventive concept include a system according to statement 152, wherein:

The system further includes a second NVMe SSD; and

The PCIe switch with erasure coding logic includes a second connector to enable the PCIe switch with erasure coding logic to communicate with the second NVMe SSD.

Statement 154. An embodiment of the inventive concept includes a system according to statement 152, wherein:

The system further includes:

Second NVMe SSD; and

The second PCIe switch includes:

The second external connector enables the second PCIe switch to communicate with the processor;

A second connector that enables the second PCIe switch to communicate with the second NVMe SSD; and

The third connector enables the second PCIe switch to communicate with the PCIe switch with erasure coding logic; and

The PCIe switch with erasure coding logic includes a fourth connector to enable the PCIe switch with erasure coding logic to communicate with the second PCIe switch,

The erasure coding scheme is applied to the data stored on the NVMe SSD and the second NVMe SSD.

Statement 155. Embodiments of the inventive concept include a system according to statement 154, wherein the second PCIe switch further includes disabled second erasure coding logic.

Statement 156. Embodiments of the inventive concept include a system according to statement 152, wherein the erasure coding logic includes at least one of side-view erasure coding logic and perspective erasure coding logic.

Statement 157. Embodiments of the inventive concept include a system according to statement 152, wherein the erasure coding logic is operable to return to the read based at least in part on the presence of data requested in the read request from the host in the cache Requested response.

Statement 158. Embodiments of the inventive concept include a system according to statement 157, wherein the erasure coding logic further includes cache.

Statement 159. An embodiment of the inventive concept includes a system according to statement 157, in which:

The PCIe switch with erasure coding logic is located in the mainframe; and

The main case includes memory used as cache by erasure coding logic.

Statement 160. An embodiment of the inventive concept includes a system according to statement 152, wherein the erasure coding logic is operable to return a response to the write request to the host before completing the write request.

Statement 161. An embodiment of the inventive concept includes a system according to statement 160, in which:

The PCIe switch with erasure coding logic further includes a write buffer; and

The erasure coding controller is operable to store the data in the write request in the write buffer.

Statement 162. An embodiment of the inventive concept includes a system according to statement 152, wherein the erasure coding logic includes side-by-side erasure coding logic, and the side-view erasure-coding logic includes detection logic.

Statement 163. Embodiments of the inventive concept include a system according to statement 152, where the erasure coding logic is operable to intercept control transmissions received at the PCIe switch and forward the control transmissions to the power processing unit (PPU).

Statement 164. Embodiments of the inventive concept include a system according to statement 152, wherein the erasure coding logic is operable to intercept data transmissions received from the host at the PCIe switch, and in the data transmission with device logic used by the NVMe SSD Block address (LBA) replaces the host LBA used by the host.

Statement 165. Embodiments of the inventive concept include a system according to statement 164, wherein the erasure coding logic is further operable to direct data transfer to the NVMe SSD.

Statement 166. An embodiment of the inventive concept includes a system according to statement 152, where the erasure coding logic is operable to intercept data transmissions received from the NVMe SSD at the PCIe switch, and use the host LBA used by the host in the data transmission Replace the device LBA used by NVMe SSD.

Statement 167. An embodiment of the inventive concept includes a system according to statement 152, wherein the erasure coding logic defines a virtual storage device that spans the NVMe SSD and the second NVMe SSD.

Statement 168. Embodiments of the inventive concept include a system according to statement 152, wherein a PCIe switch with erasure coding logic is operable to enable erasure coding logic based at least in part on the NVMe SSD being able to be used with erasure coding logic.

Statement 169. An embodiment of the inventive concept includes a system according to statement 152, the system further comprising a second device connected to a PCIe switch with erasure coding logic.

Statement 170. An embodiment of the inventive concept includes a system according to statement 169, wherein the second device includes at least one of a storage device, an SSD with a field programmable gate array (FPGA), and a graphics processing unit (GPU).

Statement 171. An embodiment of the inventive concept includes a system according to statement 169, in which:

The second device cannot be used with erasure coding logic; and

The PCIe switch with erasure coding logic is operable to disable erasure coding logic based at least in part on the inability of the second device to be used with erasure coding logic.

Statement 172. Embodiments of the inventive concept include a system according to statement 169, in which:

The second device cannot be used with erasure coding logic; and

A PCIe switch with erasure coding logic is operable to enable erasure coding logic based at least in part on the NVMe SSD can be used with erasure coding logic, and enable storage of the second device without using erasure coding logic take.

Statement 173. An embodiment of the inventive concept includes a system according to statement 169, in which:

The second device cannot be used with erasure coding logic; and

The PCIe switch with erasure coding logic is operable to enable erasure coding logic based at least in part on the NVMe SSD being able to be used with the erasure coding logic and disable access to the second device.

Therefore, with regard to the various arrangements of the embodiments described herein, this detailed description and accompanying materials are intended to be illustrative only and should not be considered as limiting the scope of the inventive concept. Therefore, the concept of the present invention claims that all such modifications can fall within the scope and spirit of the above patent application scope and equivalent clauses.

105: machine/host 110: processor 115: Memory 120: memory controller 125, 320, 605, 1005: Peripheral components connected to Express (PCIe) switches 130: storage device 130-1, 130-2, 130-3, 130-4, 130-5, 130-6: solid state drive (SSD)/storage device/physical storage device 135: device driver 205: Clock 210: network connector 215: busbar 220: user interface 225: input/output engine 305: middle plane 310, 315: distribution board 325, 330: Baseboard Management Controller (BMC) 405, 410, 415: erasure coding scheme 505: connector 510-1, 510-2, 510-3, 510-4, 510-5, 510-6: PCIe to PCIe stack 515: PCIe switch core 520: Power Processing Unit (PPU) 525: Detection logic 530: Erase code controller 535-1, 535-2, 535-3, 535-4, 535-5, 535-6: capture interface 540: Multiplexer 545: Cache 550: Write buffer 555: Erase code enable signal 705: Field programmable gate array (FPGA) 1103, 1106, 1109, 1112, 1115, 1118, 1121, 1124, 1127, 1130, 1133, 1136, 1139, 1145, 1148, 1151, 1154, 1160, 1163, 1205, 1210, 1215, 1220, 1225, 1235, 1240, 1305, 1310, 1315, 1405, 1410, 1415, 1420: square

FIG. 1 illustrates a machine according to an embodiment of the inventive concept, the machine including a peripheral component connection express (PCIe) switch with side-by-side erasure coding logic. Fig. 2 shows additional details of the machine shown in Fig. 1. FIG. 3 shows additional details of the machine shown in FIG. 1, the additional details including a power distribution board and a mid-plane connecting the PCIe switch with the bypass erasure coding logic shown in FIG. 1 to the storage device. FIG. 4 shows the storage device shown in FIG. 3 for implementing different erasure coding schemes. FIG. 5 shows details of the PCIe switch shown in FIG. 1 with side-by-side erasure coding logic. 6 shows details of a PCIe switch with perspective erasure coding logic according to another embodiment of the inventive concept. FIG. 7 illustrates a first topology using the PCIe switch with side-by-side erasure coding logic shown in FIG. 1 according to an embodiment of the inventive concept. FIG. 8 illustrates a second topology using the PCIe switch with bypass code erasure coding logic shown in FIG. 1 according to another embodiment of the inventive concept. FIG. 9 shows a third topology using the PCIe switch with side-by-side erasure coding logic shown in FIG. 1 according to yet another embodiment of the inventive concept. FIG. 10 illustrates a fourth topology using the PCIe switch with side-by-side erasure coding logic shown in FIG. 1 according to yet another embodiment of the inventive concept. FIGS. 11A to 11D show a flowchart of an exemplary process of the PCIe switch with bypass erasure coding logic shown in FIG. 1 supporting the erasure coding scheme according to an embodiment of the inventive concept. FIGS. 12A-12B illustrate an exemplary process for the PCIe switch with bypass erasure coding logic shown in FIG. 1 to perform initialization according to an embodiment of the inventive concept. 13 shows a flowchart of an exemplary process of the PCIe switch shown in FIG. 1 with side-by-side erasure coding logic incorporating a new storage device into an erasure coding scheme according to an embodiment of the inventive concept. 14 shows a flowchart of an exemplary process of the PCIe switch with bypass erasure coding logic shown in FIG. 1 to handle a faulty storage device according to an embodiment of the inventive concept.

125: PCIe switch

505: connector

510-1~510-6: PCIe to PCIe stack

515: PCIe switch core

520: PPU

525: Detection logic

530: Erase code controller

535-1~535-6: Capture interface

540: Multiplexer

545: Cache

550: Write buffer

555: Erase code enable signal

Claims (20)

A system including: Non-volatile storage fast solid-state drive; Field-programmable gate array that implements one or more functions that support the non-volatile storage fast solid-state drive, including data acceleration, deduplication, data integrity, and data encryption And at least one of data compression; and The peripheral components are connected to the fast switch; The peripheral device is connected to a fast switch to communicate with the field programmable gate array and the non-volatile storage fast solid-state drive. The system according to item 1 of the patent application scope, wherein the peripheral element connection fast switch includes erasure coding logic, and the erasure coding logic includes an erasure coding controller. The system of claim 2 of the patent application scope, wherein the erasure coding logic operates to return a response to the read request based on the presence of at least part of the data requested in the read request from the host in the cache . The system of claim 2 of the patent application scope, wherein the erasure coding logic operates to return a response to the write request to the host before completing the write request. The system according to item 2 of the patent application scope, wherein the erasure coding logic includes side-view erasure coding logic, and the side-view erasure coding logic includes detection logic. The system according to item 2 of the patent application scope, wherein the erasure coding logic operates to intercept the data transmission received from the host where the peripheral element is connected to the fast switch, and replace the host in the data transmission. The host logical block address used is the logical block address of the device of the non-volatile storage fast solid-state drive. The system of claim 2 of the patent application scope, wherein the peripheral element is connected to a fast switch operation to enable the erasure based on at least a portion of the non-volatile storage fast solid-state drive that does not include native erasure coding logic Coding logic. The system as described in item 2 of the scope of the patent application further includes a second device connected to the peripheral element connected to the fast switch with erasure coding logic. The system as described in item 8 of the patent application scope, in which: The second device includes at least one of a non-storage device and a storage device having local erasure coding logic; and The peripheral element is connected to a fast switch operation to disable the erasure coding logic based at least in part on the second device. The system as described in item 8 of the patent application scope, in which: The second device includes at least one of a non-storage device and a storage device having local erasure coding logic; and At least one of the peripheral elements including a non-storage device and a storage device with a local erasure code is connected to a fast switch operation to enable the at least partly based on the non-volatile storage fast solid-state drive not including the local erasure code logic Erasure coding logic, and enable access to the second device without using the erasure coding logic. A system including: Non-volatile storage fast solid-state drives; and On-site programmable gate array, the on-site programmable gate array includes a first on-site programmable gate array part and a second on-site programmable gate array part, the first on-site programmable gate array part implements one or Multiple functions to support the non-volatile storage fast solid-state drive, the functions include at least one of data acceleration, deduplication, data integrity, data encryption, and data compression, and the second field programmable gate The array part implements peripheral components to connect fast switches, Wherein the peripheral components are connected to the fast switch to communicate with the field programmable gate array and the non-volatile storage fast solid-state drive, and The on-site programmable gate array and the non-volatile storage fast solid-state drive are located inside a common housing. The system as recited in item 11 of the patent application range, wherein the peripheral element connection fast switch includes erasure coding logic, and the erasure coding logic includes an erasure coding controller. The system according to item 12 of the patent application scope, wherein the erasure coding logic includes at least one of side-view erasure coding logic and perspective erasure coding logic. The system of claim 12 of the patent application scope, wherein the erasure coding logic operates to return a response to the read request based at least in part on the presence of data requested in the read request from the host in the cache . The system of claim 12 of the patent application scope, wherein the erasure coding logic operates to return a response to the write request to the host before completing the write request. The system according to item 12 of the patent application scope, wherein the erasure coding logic includes side-view erasure coding logic, and the side-view erasure coding logic includes detection logic. The system of claim 12 of the patent application scope, wherein the erasure coding logic operates to intercept the data transmission received from the host where the peripheral element is connected to the fast switch, and in the data transmission The device logical block address used by the volatile storage fast solid-state drive replaces the host logical block address used by the host. The system of claim 12 of the patent application scope, wherein the erasure coding logic operates to intercept the data transmission received from the non-volatile storage fast solid-state drive at the peripheral element connected to the fast switch, and in the In the data transmission, the logical block address of the device used by the non-volatile storage fast solid-state drive is replaced with the logical block address of the host used by the host. The system of claim 12 of the patent application scope, wherein the peripheral element with erasure coding logic is connected to a fast switch to operate based on at least part of the non-volatile storage fast solid-state drive does not include native erasure coding logic Instead, the erasure coding logic is enabled. The system of claim 12 of the patent application scope, wherein the peripheral element with erasure coding logic is connected to a fast switch to operate based on at least part of the non-volatile storage fast solid-state drive including local erasure coding logic Disable the erasure coding logic.

Images