CN110247860B

CN110247860B - Multi-mode and/or multi-speed NVMe-oF device

Info

Publication number: CN110247860B
Application number: CN201910159821.2A
Authority: CN
Inventors: 颂蓬·保罗·奥莱瑞兹; 弗雷德·沃利
Original assignee: Samsung Electronics Co Ltd
Current assignee: Samsung Electronics Co Ltd
Priority date: 2018-03-09
Filing date: 2019-03-04
Publication date: 2022-09-27
Anticipated expiration: 2039-03-04
Also published as: CN110247860A; US11018444B2; KR20190106680A; US20230198182A1; US11588261B2; KR102483969B1; US20210265755A1; US20190280411A1; KR20230008678A

Abstract

A multi-mode and/or multi-speed NVMe-af device is provided. In an example, an apparatus includes: a Printed Circuit Board (PCB); at least one Solid State Drive (SSD) connected on a first side of the PCB via at least one SSD connector; at least one Field Programmable Gate Array (FPGA) mounted on the PCB at a second side of the PCB; at least one connector attached to the PCB on a third side of the PCB, wherein the apparatus is configured to operate at a first speed of a plurality of operating speeds based on a first input received via the at least one connector.

Description

Multi-mode and/or multi-speed NVMe-oF device

This patent application claims the benefit of U.S. provisional patent application No. 62/641250 entitled "method and apparatus for supporting single FPGA + multiple NF1 SSDS" filed on 2018, 3, 9, the entire contents of which are expressly incorporated herein by reference.

Technical Field

One or more aspects oF embodiments according to the invention relate to a network attached device, e.g., a multimode and/or multi-speed fabric-based non-volatile storage express (NVMe-af) device.

Background

With NVM-af configurations (e.g., configurations using ethernet-attached non-volatile storage high speed (NVMe) Solid State Drives (SSDs)), it can be challenging to improve the cost and performance oF ethernet and SSDs. For example, with the advent of 50G/100G technology and beyond, ethernet speed has increased, while SSD performance may depend on the peripheral component interconnect express (PCIe) interface and NAND technology. Because each device can provide a point-to-point connection, a fabric-attached SSD presents additional unique design challenges for supporting erasure code data protection.

Accordingly, a storage device capable oF supporting both NVMe and NVMe-af protocols and operating at different ethernet speeds would be desirable.

The above information disclosed in this background section is only for enhancement of understanding of the background of the invention and therefore it may contain information that does not form the prior art.

Disclosure of Invention

Aspects of embodiments of the present disclosure relate to an apparatus, comprising: a Printed Circuit Board (PCB); at least one Solid State Drive (SSD) connected on a first side of the PCB via at least one SSD connector; at least one Field Programmable Gate Array (FPGA) mounted on the PCB at a second side of the PCB; and at least one connector attached to the PCB at a third side of the PCB, wherein the apparatus is configured to operate at a first speed of a plurality of operating speeds based on a first input received via the at least one connector.

In an example, the at least one SSD includes a first SSD and a second SSD, and the at least one SSD connector includes a first SSD connector and a second SSD connector, the first SSD is connected with the PCB at a first side of the PCB via the first SSD connector, and the second SSD is connected with the PCB at the first side of the PCB via the second SSD connector, each of the first SSD and the second SSD is a new size specification 1(NF1) SSD, wherein the at least one connector is a U.2 connector, and each of the first SSD connector and the second SSD connector is an m.2 connector.

In an example, the at least one SSD connector is attached to and electrically connected with the at least one SSD at a first side of the at least one SSD, and the at least one SSD connector is attached to and perpendicular to the first side of the PCB at a first side of the PCB.

In an example, the apparatus further comprises: a structural support between the second side of the PCB and the second side of the at least one SSD, wherein the structural support is a plastic support or a bracket. In an example, the length of the at least one SSD is 110mm and the length of the PCB is between 110mm and 142.2 mm. In an example, a first side of the at least one FPGA is attached to a second side of the PCB, the at least one FPGA being electrically connected with the PCB.

In an example, the apparatus further comprises: a first interface layer attached at a third side of the at least one SSD; and a second interface layer attached to a second side of the at least one FPGA, wherein the first and second interface layers are used to transfer heat generated by the at least one SSD and the at least one FPGA during operation of the at least one SSD and the at least one FPGA.

In an example, the at least one connector is an SFF-TA-1008 connector. In an example, the PCB has a length equal to or longer than a length of the at least one SSD. In an example, the length of the PCB is equal to or longer than the length of the at least one FPGA, which is 80 mm. In an example, the at least one SSD operates at a second speed of the plurality of operating speeds, wherein the plurality of operating speeds of the device are two or more speeds above 10G. In an example, a first input is received from the midplane via the at least one connector, wherein the first input is controlled by using two general purpose input/output (GPIO) pins or one or more internal registers internal to the at least one FPGA controlled by a Baseboard Management Controller (BMC) or a local Central Processing Unit (CPU) of the motherboard.

In an example, a system includes: a first interface layer; at least one Solid State Drive (SSD) attached to a first interface layer on a first side of the at least one SSD; a Printed Circuit Board (PCB) connected with the at least one SSD on a first side of the PCB via at least one SSD connector; at least one Field Programmable Gate Array (FPGA) mounted on the PCB at a second side of the PCB, wherein a first side of the at least one FPGA is attached to the PCB; a second interface layer attached to the at least one FPGA on a second side of the at least one FPGA; and at least one connector attached to the PCB on a third side of the PCB, wherein the system is configured to operate at a first speed of a plurality of operating speeds based on a first input received via the at least one connector.

In an example, the at least one connector is an U.2 connector or an SFF-TA-1008 connector and the at least one SSD connector is an m.2 connector, the at least one SSD connector being perpendicular to the first side of the PCB.

In an example, the system further comprises: a structural support between the second side of the PCB and the second side of the at least one SSD, wherein the structural support is a plastic support or a bracket. In an example, the length of the at least one SSD is 110mm, the length of the PCB is between 110mm and 142.2mm, and the length of the at least one FPGA is 80 mm. In an example, a length of the PCB is equal to or longer than a length of the at least one SSD, wherein the length of the PCB is equal to or longer than a length of the at least one FPGA.

In an example, a method includes: attaching a first interface layer to a first side of at least one Solid State Drive (SSD); connecting the at least one SSD on a first side of a Printed Circuit Board (PCB) via at least one SSD connector; mounting at least one Field Programmable Gate Array (FPGA) on a second side of the PCB, wherein the first side of the at least one FPGA is attached to the PCB; attaching a second interface layer to a second side of the at least one FPGA; and attaching at least one connector to a third side of the PCB, wherein the at least one SSD is configured to operate at a first speed of a plurality of operating speeds based on a first input received via the at least one connector.

In an example, the method further comprises including a structural support between the second side of the PCB and the second side of the at least one SSD, wherein the structural support is a plastic support or bracket and the at least one SSD connector is perpendicular to the first side of the PCB. In an example, a length of the PCB is equal to or longer than a length of the at least one SSD, wherein the length of the PCB is equal to or longer than a length of the at least one FPGA.

Drawings

These and other features of some example embodiments of the invention will be appreciated and understood with reference to the specification, claims and drawings, in which:

fig. 1 illustrates a top view oF an NVMe-af device according to some example embodiments oF the invention;

fig. 2 illustrates a cross-sectional view oF an NVMe-af device according to some example embodiments oF the invention;

fig. 3A illustrates a configuration oF an NVMe-af device according to some example embodiments oF the invention;

FIG. 3B shows a block diagram of an example SSD, according to some example embodiments of the present invention;

fig. 4 illustrates a block diagram of a midplane or Complex Programmable Logic Device (CPLD) according to some example embodiments of the invention;

fig. 5 illustrates an example table showing example uses oF U.2 connectors according to an example multimode NVMe-af device configuration, according to some example embodiments oF the invention;

fig. 6A (fig. 6a.1 and 6a.2) illustrates a block diagram oF an example NVMe-af device operating with non-high availability (non-HA) according to some example embodiments oF the invention;

FIG. 6B (FIGS. 6B.1 and 6B.2) illustrates a block diagram oF an example NVMe-oF device operating in High Availability (HA) mode, according to some example embodiments oF the invention;

fig. 7A illustrates a block diagram of an example Field Programmable Gate Array (FPGA) device connected with one or more SSDs, according to some example embodiments of the invention;

fig. 7B illustrates another block diagram of an example FPGA device connected with one or more SSDs according to some example embodiments of the present invention;

fig. 8 shows a block diagram of an example switch, according to some example embodiments of the invention;

FIG. 9 illustrates a flow diagram of a method for operating a storage device, according to some example embodiments of the invention;

fig. 10A-10B illustrate top and bottom views, respectively, of an assembly according to some example embodiments of the invention.

Detailed Description

The detailed description set forth below in connection with the appended drawings is intended as a description oF some exemplary embodiments oF systems and methods for supporting multi-mode and/or multi-speed NVMe-af devices provided in accordance with the present invention and is not intended to represent the only forms in which the present invention may be constructed or utilized. This description sets forth the features of the invention in connection with the illustrated embodiments. It is to be understood, however, that the same or equivalent functions and structures may be accomplished by different embodiments that are also intended to be encompassed within the scope of the invention. As noted elsewhere herein, like element numbers are intended to indicate like elements or features.

NVMe is a standard that defines a register level interface for host software to communicate with a non-volatile storage subsystem (e.g., SSD) over a PCIe bus. NVMe is an alternative to the Small Computer System Interface (SCSI) standard for connecting and transferring data between a host and a peripheral target storage device or system. PCIe connected NVMe SSDs allow applications to communicate directly with memory.

The physical connection of NVMe is based on the PCIe bus. A typical ethernet SSD (essd) has a standard SSD (e.g., U.2) connector to interface with the system over a PCIe bus via a midplane. U.2(SFF-8639) is a computer interface for connecting an SSD to a computer. The U.2 connector can support two Serial Advanced Technology Attachment (SATA) ports or two serial attached scsi (sas) ports and support up to four parallel I/O channels in PCIe SSDs (X4). If not used, the two

PCIe lanes

1 and 2 may optionally be used as additional SAS ports as desired. The U.2 connector is standardized for NVMe and supports PCIe 3.0X4, where PCIe 3.0X4 provides five times the speed of a typical SATA SSD.

NVM-ofs are extensions to the NVMe standard that can run over various fabrics (or connecting wires) other than PCIe. Here, the term "fabric" denotes a network topology in which network nodes can communicate data with each other through various interconnection protocols, ports, and switches. For example, an ethernet-attached SSD may be directly attached to the fabric, and in this case, the fabric is an ethernet.

NVMe-af enables the use oF alternate transport for PCIe, extending the distance that NVMe host devices can connect to NVMe storage drives or subsystems. Thus, NVMe-af is a specification designed to enable fast non-volatile storage message-based commands to transfer data directly between a host computer and a target solid-state storage device (e.g., an eSSD or NVMe-af device) or system over a network such as ethernet, Fibre Channel (FC), or InfiniBand. When configured to support the NVMe-af standard, the system may support a variety oF fabrics including not only ethernet, but fibre channel, InfiniBand, and other network fabrics. For example, the eSSD may be directly attached to the fabric, and in this case, the fabric is an ethernet. The eSD may refer to an SSD that may support the NVMe-oF protocol. For ease oF illustration, the following examples and embodiments may show an ethernet-attached NVMe-af device. However, it is noted that any other type oF NVMe-af device may be used without departing from the scope oF the present disclosure.

As described above, the physical connection of NVMe is based on the PCIe bus. Recently, with the advent of PCIe4.0, bandwidth mismatch may be higher than PCIe 3.0. A single 25G ethernet does not have enough bandwidth to keep up with the end of X4PCIe4.0 of the SSD (up to 8 GB/s). 50G or 100G Ethernet provides a better match for X4PCIe4.0 of SSD. Thus, embodiments oF the invention may include a storage device (e.g., eSSD) capable oF supporting both NVMe protocol and NVMe-af protocol, and when in NVMe-af mode, the storage device may be capable oF operating at different ethernet speeds from 10G up to 100G or higher without any changes to its hardware.

In one configuration, to support device (e.g., NVMe-af device) speeds oF 100G or higher, example embodiments oF the present disclosure may have two small form factor (2.5 inch hard drive form factor) SSDs packaged with a single Field Programmable Gate Array (FPGA). The single FPGA would be able to support multiple SSDs to improve performance while maintaining the same FPGA cost. Furthermore, example embodiments oF NVMe-af devices having at least one FPGA and a plurality oF SSDs oF the present disclosure may fit existing and/or new SSD size specifications (e.g., small size Specification (SFF) or standard PCIe card size specification, e.g., full-height full-length (FH-FL) card profile or full-height half-length (FH-HL) profile). The performance and capacity oF example embodiments oF NVMe-af devices oF the present disclosure may also be linearly extended by adding more similar devices and/or circuit board motherboards.

Further, embodiments oF the present invention may enable any platform system to support different types oF NVMe-af devices from different vendors. By using common building blocks (such as switcher boards, midplane, and ethernet SSDs), ecosystem providers will be able to enter the market and offer various price/performance products to meet various customers faster than existing systems. Some example embodiments may also enable a common system platform capable oF supporting both NVMe-af devices with standard U.2 (such as PM1725a or PM1735) and emerging devices (such as NF1 SSDs). In one configuration, the NF1SSD may also be referred to as an NGSFF or M.3-based SSD.

Different embodiments oF the present system provide a single platform and common building block that can support both NVMe devices and NVMe-af devices, and when in NVMe-af mode, the system would be able to run at different ethernet speeds from 10G and beyond 10G without any change to its hardware. Common building blocks that can support both NVMe and NVMe-af devices can include switch boards, midplanes, and esds. The present system may also provide a common system platform capable oF supporting NVMe-af devices with standard SSD connectors (such as PM1725a or PM1735) or SSDs based on M.3 or NGSFF. Different configurations of the present system may also be compatible with technological advancements such as 50G and 100G or higher speed ethernet and PCIe 4.0. In particular, the different configurations oF the present application provide a system that can support both NVMe protocol and NVMe-af protocol, as well as SSDs (e.g., esds) oF various types oF attached chassis. Further, the different configurations of the present application can provide disclosure as to how a circuit board and a device (e.g., eSSD) coordinate for the device to understand where the device has been deployed and what characteristics the device should assume as a result of the device being deployed in a particular location of a data set.

The fabric-attached ssd (essd) disclosed herein is a single generic device that can be used in multiple systems that are compatible with the NVMe standard and the NVMe-af standard. Ethernet SSDs may use standard SSD connectors, such as U.2, to interface with a host device via the middle plane. The U.2 connector can support 25Gbps (100G-KR4) Ethernet. The esds disclosed herein may be devices that can be used in a number oF systems that are compatible with the NVMe standard and the NVMe-af standard. Accordingly, the fabric-attached ssd (essd) may also be referred to as a multimode NVMe-af device. The multimode NVMe-af device can support the NVMe standard or NVMe-af standard by detecting product information from a known location (e.g., circuit board type pin E6 oF U.2 which can be driven directly from the motherboard or midplane). For example, a defined signal (e.g., a mode signal) on the U.2 connector may indicate to the drive whether to operate in NVMe mode or NVMe-af mode. If present as an NVMe circuit board, the X4 channel PCIe of the U.2 connector would be driven by the PCIe engine of the eSSD controller. In this case, the device will disable the fabric attached port (e.g., ethernet port, fibre channel port, or InfiniBand port) and all NVMe protocols and functions are supported or enabled. If presented as an NVMe-oF circuit board, the fabric attached port will be enabled by using only unused and available SAS pins.

Fig. 1 shows a top view oF NVMe-af device 100. The NVMe-af device 100 oF fig. 1 includes at least one SSD 102, an FPGA Printed Circuit Board (PCB)104, at least two standard SSD connectors (such as m.2106 and 108), and at least one U.2 connector 110. The at least two m.2

connectors

106 and 108 may be vertical or 90 degree connectors to connect one or more SSDs (e.g., 102) with the FPGA PCB 104. The vertical m.2

connectors

106 and 108 are perpendicular to a first side (e.g., the top side of fig. 1) 112 of the FPGA PCB 104. At least one SSD 102 may be a new size specification 1(NF1) SSD. At least one SSD 102 is connected with FPGA PCB104 at a first side 112 of FPGA PCB104 via a vertical m.2 connector 106. In one configuration, the length oF the at least one SSD 102 may be 110mm and the length oF the FPGA PCB104 may be between 110mm and 142.2mm, otherwise the NVMe-af device 100 may not be plugged into a standard circuit board. In an example, the m.2 standard allows for modules or circuit boards having widths of 12mm, 16mm, 22mm, and 30mm and lengths of 16mm, 26mm, 30mm, 38mm, 42mm, 60mm, 80mm, and 110 mm. In an example, the length of the FPGA PCB104 may be equal to or longer than the length of the at least one SSD 102.

In an example, additional SSDs may also be connected on the first side 112 of the FPGA PCB104 via the vertical m.2 connector 108. In one configuration, the NVMe-af device 100 may support multiple SAS ports (e.g., SAS0 and SAS1) and up to four PCIe X4 lanes via U.2 connector 110. Two SAS ports (SAS0 and SAS1) may be used as ethernet ports by NVMe-af device 100. The NVMe-af device 100 would be able to operate at different ethernet speeds from 10G up to 100G or higher without any change to the hardware. In an example, an SFF-TA-1008 connector may be used in place oF the U.2 connector 110 to connect the NVMe-af device 100 with the midplane via a SFF-TA-1002 connector connectable with the midplane.

In one example, an FPGA (e.g., shown in fig. 2) may be connected with the FPGA PCB104 on a second side opposite the first side 112 of the FPGA PCB104, wherein the at least one SSD 102 is connected with the first side 112 of the FPGA PCB104 via the vertical m.2 connector 106. In one configuration, one or more layers oF thermally conductive interface material (TIM) may be placed between the housing and the NVMe-af device when the NVMe-af device is placed in the housing. In an example, the length of the FPGA PCB104 may be equal to or longer than the length of the at least one SSD 102 and the FPGA. In one configuration, the length of the FPGA may be 80 mm.

In one example, power loss protection components (such as supercapacitors and/or controller ASICs in the NF1SSD 102) may be transferred from the NF1SSD 102 to the FPGA PCB104 to provide additional space available in the NF1SSD 102 for additional NAND flash packages that extend the storage capacity of the NF1SSD 102.

Fig. 2 shows a cross-sectional view oF NVMe-af device 200 in metal housing 202. The NVMe-af device 200 may be the NVMe-af device 100 oF fig. 1. The NVMe-af device 200 includes a metal housing 202, a first TIM layer 240, at least one NF1SSD204, at least one vertical m.2 connector 210, a plastic or metal support or bracket 212, an FPGA PCB 216, an FPGA222, a second TIM layer 236, and a standard SSD connector (such as U.2 connector 214).

In the example embodiment of fig. 2, a first side 246 of the first TIM layer 240 is attached to (or in thermal contact with) a first side 242 of a metallic or thermally conductive casing 202. The second side 248 of the first TIM layer 240 is attached to the first side 224 of the NF1SSD204 (or in thermal contact with the first side 224). The NF1SSD204 is connected with the FPGA PCB 216 via a vertical m.2 connector 210, wherein the vertical m.2 connector 210 is attached to the NF1SSD204 at the second side 206 of the NF1SSD204 and is electrically connected with the NF1SSD 204. The vertical m.2 connector 210 is mounted on the FPGA PCB 216. In an example, the vertical m.2 connector 210 and the FPGA PCB 216 are separated by an air gap. In one configuration, the length of the NF1SSD204 may be a standard length (e.g., 110mm), and the length of the FPGA PCB 216 may be longer than 110 mm. In an example, the length of the FPGA PCB 216 may be equal to or longer than the length of the NF1SSD 204.

In an example, a plastic (or other suitable insulating material) support or bracket 212 that provides structural support for the NF1SSD204 may be located between the third side 230 of the FPGA PCB 216 and the third side 226 of the NF1SSD 204. A plastic support or bracket 212 may be located near the fourth side 220 of the FPGA PCB 216 and the fourth side 208 of the NF1SSD 204.

In an example, the U.2 connector 214 is mounted on the FPGA PCB 216 and electrically connected with the FPGA PCB 216. In one configuration, NVMe-af device 200 may support up to four PCIe X4 lanes for multiple SAS ports (e.g., SAS0 and SAS1) and PCIe X4 buses via U.2 connector 214. Two SAS ports (SAS0 and SAS1) are unused by NVMe devices and may be used as ethernet ports by NVMe-af device 200. The NVM-af device 200 may be able to operate at different ethernet speeds from 10G up to 100G or higher without any change to the hardware.

In the NVMe-af device oF fig. 2, a third side 230 oF the FPGA PCB 216 is attached to a first side 232 oF the FPGA 222. In an example, the FPGA222 can be mounted on a third side 230 of the FPGA PCB 216 and electrically connected with the FPGA PCB 216. The second side 234 of the FPGA222 is attached to the second TIM layer 236 (or in thermal contact with the second TIM layer 236) at the first side 250 of the second TIM layer 236. The second side 252 of the second TIM layer 236 is attached to (or in thermal contact with) the second side 244 of the metal casing 202. The first TIM layer 240 and the second TIM layer 236 may be used to spread and dissipate heat generated by the NF1SSD204 and the FPGA222 during normal operation oF the electronic device (such as the NF1SSD 204) and the FPGA222, preventing the NVMe-af device 200 from overheating by transferring the generated heat to the outside oF the NVMe-af device 200 via the housing 202. In an example, the length of the second TIM layer 236 may be equal to or longer than the length of the FPGA 222. Other electronic devices, such as DDR4 memory (as shown in fig. 3A), may require additional strips of TIM.

Fig. 3A shows a configuration oF NVMe-af device 300. The NVM-af device 300 can be the NVMe-af device 100 oF fig. 1 or the NVMe-af device 200 oF fig. 2. The NVMe-af device 300 oF fig. 3A includes an FPGA302, a first NF1SSD 314, a second NF1SSD316, two double data rate fourth generation (DDR4) memories (322 and 324), and four capacitors (326, 328, 330, and 332). NVMe-af device 300 is also connected with U.2 connector 312, and via U.2 connector 312, NVMe-af device 300 can support up to four PCIe X4 lanes oF two SAS ports (SAS0 and SAS1) and a PCIe X4 bus. Two SAS ports (SAS0 and SAS1) may be used as ethernet ports by NVMe-af device 300. The NVM-af device 300 would be able to operate at different ethernet speeds from 10G up to 100G or higher without any change to the hardware.

FIG. 3B illustrates a block diagram of an example NF1SSD 334, according to one embodiment. The NF1SSD 334 may be SSD 314 or SSD316 of fig. 3A. New size specification 1(NF1) of NF1SSD 334 may accommodate NAND flash devices of multiple rows or channels, maximizing the capacity of the modular SSD. The NF1SSD 334 may be connected with the FPGA via a PCIe X4 bus 336. The FPGA may be the FPGA302 of fig. 3A and the PCIe X4 bus 336 may be one of the

PCIe X4 bus

318 or 320 of fig. 3A. The NF1SSD 334 may also accommodate at least one optional ethernet port 338 (ethernet port a and/or ethernet port B) using unused or reserved (e.g., high speed) pins of an m.2 connector used to connect with an FPGA (e.g., FPGA302 of fig. 3A). At least one ethernet port 338 may be used when NF1SSD 334 is operating in NVMe-af mode.

In some embodiments, the NF1SSD 334 may conform to the 2.5 inch hard disk drive size specification (or Small Form Factor (SFF)) standard but have an extended length. In other embodiments, NF1SSD 334 may conform to standard PCIe card size specifications, such as a full-height full-length (FH-FL) card profile or a full-height half-length (FH-HL) profile. The NF1SSD 334 may also include a controller, buffer memory, and flash memory. The controller may execute software, NVMe commands, and/or firmware stored, for example, in a buffer memory or read only memory of the controller (or a buffer memory or read only memory separate from the controller). In fig. 3B, the eSSD ASIC 340 is an enhanced SSD controller that includes a Flash Translation Layer (FTL) and a flash controller. In an example, the eSSD ASIC 340 may be used as a power loss protection component in the NF1SSD 334.

Returning to fig. 3A, the NVMe-af device 300 may be configured to operate in NVMe mode or NVMe-af mode. In NVMe-af mode, NVMe-af device 300 may configure two PCIe lanes (PCIe1 and PCIe 2) as ethernet port 306 (ethernet port B) and ethernet port 310 (ethernet port D). The NVMe-af device 300 may also configure two SAS ports (SAS0 and SAS1) as ethernet port 304 (ethernet port a) and ethernet port 308 (ethernet port C). The first 25G ethernet port 304 may be connected at pins S2, S3, S5, and S6 of the U.2 connector 312, the second 25G ethernet port 306 may be connected at pins S17, S18, S20, and S21 of the U.2 connector 312, the third 25G ethernet port 308 may be connected at pins S9, S10, S12, and S13 of the U.2 connector 312, and the fourth 25G ethernet port 310 may be connected at pins S23, S24, S26, and S27 of the U.2 connector 312. When in NVMe-af mode, the NVMe-af device 300 may be capable oF operating at different ethernet speeds from 10G up to 100G or higher.

In fig. 3A, the FPGA302 may provide an interface between four 25Gbps ethernet ports (304, 306, 308, 310) and two NF1SSDs (314, 316) when the NVMe-if device 300 is configured in NVMe-if mode. Four 25G ethernet ports (304, 306, 308, 310) may be connected to the motherboard through the midplane according to the mode oF operation oF the NVMe-af device 300. In one configuration, the motherboard may include one or more switching elements, one or more storage elements, one or more I/O elements, and the like. The FPGA302 interfaces with the first NF1SSD 314 via a first PCIe X4 bus 318, and the FPGA302 interfaces with the second NF1SSD316 via a second PCIe X4 bus 320. The first PCIe X4 bus 318 and the second PCIe X4 bus 320 may be connected with the first NF1SSD 314 and the second NF1SSD316 through their respective m.2/M.3 connectors. In this case, unused or reserved (e.g., high speed) pins of the m.2 connector attached to SSD 314 and SSD316 may be used for ethernet connections. In this mode, the FPGA302 can be used as an NVMe-oF target. The NVMe-af target implemented on the eSSD ASIC/FPGA 302 provides all network and storage protocol processing that eliminates the need for an X86-based Central Processing Unit (CPU) on the motherboard in the target device. For NVMe-af based systems, the X86 based CPU on the motherboard no longer needs to move data between the initiator (e.g., host software) and the target device (i.e., NVMe-af device) because the target device can move the data itself.

In one configuration, when the multimode NVMe-af device 300 is in NVMe-af mode, the NVMe-af device 300 may have two X1PCIe lanes (PCIe 0 and PCIe 3) for the control plane oF the first (port a) and second (port B) ethernet ports. Such a configuration may make two PCIe lanes (PCIe1 and PCIe 2) available for use by the additional 25G ethernet ports (ethernet ports B and D). In some configurations, when NVMe-af device 300 is operating in NVMe-af mode at 10G (single or dual port) or 25G (single or dual port) speed, SAS port 0 is used for ethernet port a (first port) and SAS port 1 is unused. PCIe lane 0 and PCIe lane 3 are used as control planes for NVMe-af controllers that attach first (port a) and second (port B) ethernet, PCIe lane 1 is used for ethernet port B (second port), and PCIe lane 2 is unused. In some other configurations, SAS port 0 is used for ethernet port a (first port) and SAS port 1 is used as ethernet port C (third port) when NVMe-af device 300 is operating in NVMe-af mode at 50G (single or dual port) or 100G (single port only). PCIe lanes 0 and 3 are used as control planes for NVMe-af controllers that attach first (port a) and second (port B) ethernet, PCIe lane 1 is used for ethernet port B (second port), and PCIe lane 2 is used for ethernet port D (fourth port). When the NVMe-af device 300 operates at 100G speed in NVMe-af mode, PCIe lane 3 is not used because 100G speed does not support dual port.

In one configuration, the NVMe-af device 300 may be configured in NVMe mode. In NVMe mode, NVMe-af device 300 may use all four PCIe X4 lanes (in single port mode) to carry PCIe signals over the PCIe X4 bus. The PCIe X4 bus is connected to the midplane, and the PCIe bus is used as a data plane and a control plane. In an aspect, at a given time, both the first NF1SSD 314 and the second NF1SSD316 oF the NVMe-if device 300 may operate in NVMe mode or NVMe-if mode. In another aspect, at a given time, the first NF1SSD 314 may operate in NVMe mode and the second NF1SSD316 may operate in NVMe-af mode. In another aspect, at a given time, the first NF1SSD 314 may operate in NVMe-af mode and the second NF1SSD316 may operate in NVMe mode.

The operational mode oF the NVMe-af device 300 may be self-configuring or set externally using physical pins (e.g., pin E6 oF the U.2 connector 312 may be used as a board type pin indicating the protocol (NVMe or NVMe-af) that may be used on the board) or by in-band commands from the motherboard's Baseboard Management Controller (BMC). Manageability information retrieved over the Ethernet is referred to as "in-band" information, while manageability information retrieved over the PCIe bus is referred to as "out-of-band" information. When configured as an NVMe-af device, the multimode NVMe-af device 300 (circuit board type pin E6 high) may be configured in a single port NVMe-af mode or in a dual port NVMe-af mode. In the single port NVMe-af mode, pin E25 oF the U.2 connector 312 may be high (not asserted or inactive), while in the dual port NVMe-af mode, pin E25 oF the U.2 connector 312 may be asserted low or active.

In one configuration, as shown in fig. 4, different operating speeds oF the NVMe-af device 300 may be achieved by using two additional general purpose input/output (GPIO) (asserted [ 1: 0]) pins 402 located on the midplane or by the CPLD 404 during the reset #407 signal is asserted, where the CPLD 404 may be controlled by the BMC or local CPU oF the motherboard or external pins on the midplane. The output of the MUX is connected to pins E23 and E24 of U.2 connector 312. In the former option, the ESpeed pin 402 is multiplexed with the u.2i2c pin inside the CPLD 404 and may be latched after the reset 407 has been asserted (high to low). In one configuration, the inputs of the MUX are driven by the CPLD 404 or BMC of the motherboard or local CPU. However, in some configurations, the location of the MUX is internal to the CPLD 404. The number oF MUXs is equal to the maximum number oF NVME-af devices or slots supported in the circuit board. In this way, each device can be individually and independently reset and can operate at different ethernet speeds. The method may support hot plug events (hot add and hot remove). If there is one MUX shared for all devices, existing devices in the circuit board may be affected when a new device is hot plugged, due to reset cycles that may be undesirable.

Where the CPLD 404 is controlled by the motherboard's BMC or local CPU, a control signal from the BMC or local CPU may be received at the CPLD/BMC GPIO pin 406 of the CPLD 404. Table 1 below shows the status oF the ESpeed pin during different operating speeds oF NVMe-af device 300.

Ethernet speed	ESpeed pin	1	ESped pin 0
				10G	High (a)	Height of
25G	Height of	Is low with
			50G	Is low in	Height of
100G	Is low in	Is low in

TABLE 1

Further, the SMBus 408 is used to select ethernet speed during a power-on reset or PCIe reset. In one configuration, the multiplexer is selected by a power-on or reset signal 407, and when reset 407 is active low, the multiplexer may select anticipated (0: 1)402 to the FPGA302 via the external register chip, and when reset 407 is active high, the multiplexer may connect the SMBus 408 of each slot to the BMC on the motherboard. In an example, the FPGA302 may not be fully functional during a reset. The FPGA302 can take several minutes to completely download the bit file (image). During the download of the bit file, the FPGA302 may select an appropriate ethernet-related bit file based on the ethernet speed input pins from the registered buffer chips.

In one configuration, the multi-mode NVMe-af device 300 may be configured as a single port or dual port in NVMe mode or in NVMe-af mode.

Fig. 5 illustrates an example use oF the U.2 connector 312 according to the configuration oF the multimode NVMe-af device 300. When configured as an NVMe device (circuit board type pin E6 low), the multimode NVMe-af device 300 may be configured as a single port NVMe mode or a dual port NVMe mode according to the dual port EN # pin E25 oF the U.2 connector 312. In the single port NVMe mode, the dual port EN # pin E25 of the U.2 connector 312 may be asserted high (inactive). In the single port NVMe mode, the PCIe lanes 0-3 of the U.2 connector 312 are used as a single 4-lane. In dual port NVMe mode, the dual port EN # pin E25 of the U.2 connector 312 may be asserted low (active). In dual port NVMe mode, PCIe lanes 0-3 are divided into 2 by 2 lanes;

PCIe lanes

0 and 1 are used for the first port (port a) and PCIe lanes 2 and 3 are used for the second port (port B).

In some configurations, if the product information is stored in the circuit board (e.g., via an EEPROM or CPLD), the selection of a2 by 2 lane (in dual port mode) or a single quad lane (in single port mode) of the PCIe bus on U.2 connector 312 may be driven by the EEPROM. When operating in NVMe mode, the multimode NVMe-af device 300 may disable the ethernet engine and the NVMe protocols and functions are supported or enabled. When operating as an NVME-af device, the product information is used to configure the appropriate SAS and PCIe ports according to the selected ethernet speed and dual port functionality.

In some configurations, product information used for self-configuration is stored in the circuit board as part of the Vital Product Data (VPD). During startup, the multimode NVMe-af device 300 can retrieve VPD information from the circuit board and configure itself based on the VPD. In some configurations, the multimode NVMe-af device 300 may be configured in various ways without departing from the scope oF the present disclosure. For example, the multimode NVMe-af device 300 may be configured through control commands issued by a BMC oF a motherboard connected with the multimode NVMe-af device 300 via a PCIe bus. The present system provides a platform that can support various types oF NVMe and NVMe-af devices in non-high availability (non-HA) mode (i.e., single path input/output (I/O)) or HA mode (i.e., multi-path I/O) with minimal hardware changes.

Further details regarding the method oF operating the multimode NVMe-af device 300 can be found in united states non-provisional patent application No. 16/007949 entitled "system and method for supporting multimode and/or multi-speed fabric-based high speed non-volatile storage NVMe-oF devices," filed on 13/6.2018, the entire contents oF which are expressly incorporated herein by reference.

Fig. 6A (fig. 6a.1 and 6a.2) illustrates a block diagram oF an example NVMe-af device 600 operating in a non-HA mode according to one embodiment. The NVMe-af device 600 may be the NVMe-af device 100 oF fig. 1 or the NVMe-af device 200 oF fig. 2 or the NVMe-af device 300 oF fig. 3. In this example, NVMe-af device 600 may support single path I/O and may be able to operate at different ethernet speeds oF 10G and 25G without any change to the hardware. NVMe-af device 600 may include FPGA 602, first NF1SSD 614, second NF1SSD 616, and two DDR4 memories (622 and 624). The FPGA 602 interfaces with the first NF1SSD 614 via a first PCIe X4 bus 618, and the FPGA 602 interfaces with the second NF1SSD 616 via a second PCIe X4 bus 620. NVMe-af device 600 is also connected with U.2 connector 612, and via U.2 connector 612, NVMe-af device 600 can support up to four PCIe X4 lanes oF two SAS ports (SAS0 and SAS1) and PCIe X4 buses. Two SAS ports (SAS0 and SAS1) may be used as ethernet ports. NVMe-af device 600 may configure two PCIe lanes (PCIe1 and PCIe 2) as ethernet ports 606 (ethernet port B) and 610 (ethernet port D). NVMe-af device 600 may also configure two SAS ports (SAS0 and SAS1) as ethernet ports 604 (ethernet port a) and 608 (ethernet port C). The first 25G ethernet port 604 may be connected at pins S2, S3, S5, and S6 of the U.2 connector 612, the second 25G ethernet port 606 may be connected at pins S17, S18, S20, and S21 of the U.2 connector 612, the third 25G ethernet port 608 may be connected at pins S9, S10, S12, and S13 of the U.2 connector 612, and the fourth 25G ethernet port 610 may be connected at pins S23, S24, S26, and S27 of the U.2 connector 612.

In one configuration, the

ethernet ports

604, 606, 608, and 610 may be connected with the motherboard 601 through the midplane 626 via a number of high-speed Molex connectors 628, where the high-speed Molex connectors 628 may collectively carry all of the

ethernet ports

604, 606, 608, and 610, as well as other non-high-speed control signals (such as SMBus, reset, clock, etc.). The motherboard 601 may include a local CPU 634, a BMC 632, an ethernet switch controller 636, a PCIe switch 638, and two DDR4 memories (640 and 642). In some configurations, the motherboard 601 may push various signals to the NVMe-af device 600 through the midplane 626 and perform various services to the NVMe-af device 600 through the

ethernet ports

604, 606, 608, and 610. For example, the motherboard 601 may receive device specific information from the NVMe-af device 600 via

ethernet ports

604, 606, 608, and 610, wherein the device specific information includes, but is not limited to, health status information, Field Replaceable Unit (FRU) information, and sensor information oF the NVMe-af device 600. Motherboard 601 may also perform various services through

ethernet ports

604, 606, 608, and 610, including, but not limited to, discovery services to BMCs (e.g., BMC 632) or local host CPUs (e.g., CPU 634) and download services to new eSSD firmware for performing firmware upgrades.

Fig. 6B (fig. 6b.1 and 6b.2) illustrates a block diagram oF the example NVMe-af device 600 oF fig. 6A operating in HA mode according to one embodiment. In this example, NVMe-af device 600 may support multi-path I/O and may be able to operate at different ethernet speeds oF 10G and 25G without any change to hardware. In HA mode, the NVMe-af device 600 may be connected via an intermediary 626 with two

motherboards

601 and 603 supporting multiple I/O in a dual port configuration (in HA mode). Depending on the system configuration, signal integrity may need to be tested to ensure that the common midplane 626 can support both configurations. If signal integrity is insufficient, the system may have a midplane that includes two versions of a first midplane for HA mode and a second midplane for non-HA mode. U.2 the E25 pin of connector 612 can be used to enable a dual port configuration. The NVMe-af device 600 may configure its operating mode using physical pins (e.g., vendor defined pins (pin E6) on the board oF the motherboard (601 or 603)) or through in-band commands from the BMC (632 or 644) oF the motherboard (601 or 603, not shown).

In one configuration, when operating in HA mode, the ethernet port 604 (ethernet port a) and the ethernet port 606 (ethernet port B) oF the NVMe-af device 600 may be connected with the motherboard 601 through the midplane 626 via high-speed Molex connectors 628, where the high-speed Molex connectors 628 may collectively carry the

ethernet ports

604 and 606 and other non-high-speed control signals (such as SMBus, reset, clock, etc.). Further, in HA mode, the ethernet port 608 (ethernet port C) and the ethernet port 610 (ethernet port D) oF the NVMe-od device 600 may be connected with the second motherboard 603 through the midplane 626 via several high-speed Molex connectors 656, wherein the high-speed Molex connectors 656 may collectively carry the

ethernet ports

608 and 610 as well as other non-high speed control signals such as SMBus, reset, clock, etc. The second motherboard 603 may include a local CPU 646, a BMC 644, an ethernet switch controller 648, a PCIe switch 650, and two DDR4 memories (652 and 654).

Because the multimode NVMe-af device 600 can operate in both NVMe mode and NVMe-af mode, the cost oF developing and deploying the device can be reduced because the same device can be used in both NVMe mode and NVMe mode. For similar reasons, the multimode NVMe-af device 600 may enter the market faster. The multimode NVMe-af device can be used in various products and circuit boards. Two lanes of the PCIe bus are reserved for standard features through the control plane. The local CPU, BMC and other devices may use two channels oF the PCIe bus as control planes to communicate with each NVMe-af device inside the circuit board without additional cost. The NVMe midplane can be used unmodified and no new connectors on the NVMe-af device 600 are required due to the additional GPIO pins located on the midplane (e.g., ESpeed [ 1: 0]402 oF fig. 4). Further, any platform system may wish to support different types oF NVMe-af devices from different vendors. By using common building blocks (such as switch boards, midplane, and ethernet SSDs), different embodiments of the present application may enable existing ecosystem providers to enter the market faster and offer various price/performance products to meet different customers. It may also be desirable to have a common system platform capable oF supporting NVMe-af devices with standard U.2 (such as PM1725a or PM1735) and emerging devices (such as SSD based on M.3 or NGSFF).

Fig. 7A is a block diagram illustrating an example FPGA device 700 connected with two NF1SSDs (718A, 718B) according to one embodiment. The FPGA device 700 may include an FPGA 701. FPGA701 can provide an interface between U.2 connector 706 and multiple flash drives (e.g., 703A and 703B). FPGA701 may interface with NF1SSD718A via port 707 connected to PCIe X4 bus 716A through m.2 connector 709A. FPGA701 may interface with NF1SSD 718B via port 708 connected to PCIe X4 bus 716B through m.2 connector 709B. In one configuration, m.2

connectors

709A and 709B may be vertical or 90 degree connectors to connect NF1SSD718A and NF1SSD 718B laterally with FPGA device 700. The FPGA701 may also be connected with multiple DDR4 memories (e.g., 702A-702J). FPGA701 is also coupled to flash drives 703A-703B and clock circuit 714.

The U.2 connector 706 may be connected to the FPGA701 through two

multiplexers

710 and 712 via two

PCIe X4 buses

704 and 705. The PCIe X4 bus 704 may be used to send signals or packets to the motherboard through the midplane through the multiplexer 710 via the U.2 connector 706, and the PCIe X4 bus 705 may be used to receive packets from the motherboard through the midplane through the multiplexer 712 via the U.2 connector 706. In some configurations, different operating speeds of the plurality of flash drives (e.g., NF1SSDs (718A, 718B)) connected with FPGA701 may be achieved by using two additional GPIO (expected [ 1: 0]) pins (e.g., pins 402 of fig. 4) located on a midplane or CPLD (e.g., 404 of fig. 4), which may be controlled by one or more internal registers internal to FPGA 701. Table 1 shows the state of the ESpeed pin during different operating speeds of the

flash drives

703A and 703B connected to the FPGA 701.

Fig. 7B illustrates physical compression of the FPGA device 700 of fig. 7A. In fig. 7B, the FPGA device 700 is connected with a first NF1SSD718A via a first vertical m.2 connector 709A and with a second NF1SSD 718B via a second vertical m.2 connector 709B. The top view of the assembly of fig. 7B (FPGA device 700 and

NF1SSDs

718A and 718B) may be presented when

NF1SSDs

718A and 718B are arranged on the upper side of FPGA device 700. In one configuration, the assemblies oF fig. 7B (FPGA device 700 and NF1SSD718A and 718B) may be NVMe-af device 100 oF fig. 1 or NVMe-af device 200 oF fig. 2, and may support up to four PCIe X4 channels oF multiple SAS ports (e.g., SAS0 and SAS1) and PCIe X4 bus via U.2 connector 706. Two SAS ports (SAS0 and SAS1) may be used as ethernet ports through the assembly of fig. 7B (FPGA device 700 and NF1SSD718A and 718B). The assembly of fig. 7B (FPGA device 700 and

NF1SSDs

718A and 718B) would be able to run at different ethernet speeds of 10G up to 100G or higher without any change to the hardware.

In one example, the FPGA701 may be connected with the FPGA PCB 720 at a second side (not shown) opposite the side of the FPGA PCB 720, wherein the first NF1SSD718A is connected with the side of the FPGA PCB 720 via a first vertical m.2 connector 709A, and the second NF1SSD 718B is connected with the side of the FPGA PCB 720 via a second vertical m.2 connector 709B.

Fig. 8 is a block diagram illustrating an example switch including two main boards according to one embodiment. Switch 800 includes two

motherboards

801A and 801B to support multiple I/O in a dual port configuration (in HA mode) via midplane 861. Motherboard 801A includes ethernet switch 804A and PCIe switch 805A, and motherboard 801B includes ethernet switch 804B and PCIe switch 805B. As shown in the example motherboard 601 shown in FIG. 6A, each of

motherboards

801A and 801B may include other components and modules, such as local CPUs (806A, 806B), BMCs (807A, 807B), and so forth.

Several esds can be inserted into the device ports of the switch 800. For example, each of the essds is connected to switch 800 using an U.2 connector. Each eSSD may be connected with both motherboard 801A and motherboard 801B. In this example, the eSSD inserted into switch 800 is configured as an NVMe-af device, which needs to be connected with switch 800 through midplane 861 via PCIe bus and ethernet port.

FIG. 9 is a flow diagram of a method for operating a memory device, according to an embodiment of the invention. The storage device may be the NVMe-af device 600 oF fig. 6A.

At 901, a storage device receives a first input. The storage device may receive a first input from the motherboard or BMC through the midplane via the plurality of device ports. For example, the NVMe-af device 600 may receive a first input from the motherboard 601 or BMC 632 through the device ports 604-606 through the midplane 626 oF fig. 6A.

In one configuration, the first input may be controlled by using a physical pin on a circuit board of the motherboard or by an in-band command from the BMC. For example, the first input received at the NVMe-af device 600 may be controlled by using a physical pin (E6) on the circuit board oF the motherboard 601 oF fig. 6A.

At 902, based on a first input, the storage device determines whether to operate in a first mode of operation or a second mode of operation. In one configuration, the first and second modes oF operation oF the storage device may be NVMe and NVMe-af modes. For example, based on the first input, NVMe-af device 600 determines whether to operate in NVMe mode or NVMe-af mode.

At 902, based on the first input, if the storage device determines to operate in a first mode of operation, at 903, the storage device performs a storage operation according to the first mode of operation. For example, if based on the first input, the NVMe-if device 600 determines to operate in NVMe mode, the NVMe-if device 600 performs storage operations according to the NVMe protocol.

However, if at 902, the memory device determines to operate in a second mode of operation based on the first input, then at 904, the memory device performs a memory operation according to the second mode of operation. For example, if based on the first input, the NVMe-if device 600 determines to operate in NVMe-if mode, the NVMe-if device 600 performs storage operations according to the NVMe-if protocol.

At 905, the storage device receives a second input when operating in a second mode of operation. The storage device may receive a second input from the midplane via the plurality of device ports. For example, when operating in NVMe-af mode, NVMe-af device 600 may receive a second input from midplane 626 via plurality oF device ports 604-606 oF fig. 6A.

In one configuration, the second input is controlled by using two GPIO pins located on the midplane and controlled by the motherboard's BMC or local CPU or one or more internal registers internal to the FPGA of the memory device. For example, the second input may be controlled by using two GPIO pins (e.g., (anticipated [ 1: 0]) pin 402 oF fig. 4) located on the midplane 626, controlled by the BMC 632 or local CPU 634 oF the motherboard 601, or one or more internal registers internal to the FPGA 602 oF the NVMe-af device 600 oF fig. 6A.

At 906, based on the second input, the storage device selects an operating speed from a plurality of operating speeds for the storage device. For example, based on the second input, the NVMe-af device 600 oF fig. 6A selects an operating speed from a plurality oF operating speeds for the storage device. For example, when operating in NVMe-af mode, based on the second input, the NVMe-af device 600 oF fig. 6A selects an operating speed oF 100G or higher. In one configuration, the plurality of operating speeds of the storage device may be any value of speed between 10G and 100G or higher.

At 907, the storage device is operated at the selected operating speed while operating in the second mode. For example, the NVMe-af device 600 oF fig. 6A operates at an operating speed oF 100G or higher when operating in NVMe-af mode.

Fig. 10A to 10B show top and bottom views, respectively, of the assembly 1000. The assembly includes SFF SSD 1002 in standard drive tray 1004. In an example, the length of the assembly 1000 may be 7 "long. In one example, SFF SSD 1002 may be 5.75 "or 146mm in length and SFF SSD 1002 may fit a 2.5 inch SFF standard circuit board, since a 2.5 inch SFF standard circuit board provides space for devices up to 5.75".

In an example, SFF SSD 1002 may be NF1SSD 102 of fig. 1. The length of the PCB (e.g., FPGA PCB 104) on which SFF SSD 1002 may be mounted may be between 110mm and 142.2 mm. Such a length may enable a PCB on which SFF SSD 1002 may be mounted to support an SFF-TA-1008 connector (instead of U.2 connector 110) that connects SFF SSD 1002 with a midplane via an SFF-TA-1002 connector, where the SFF-TA-1002 connector may connect with the midplane.

It will be understood that, although the terms "first," "second," "third," etc. may be used herein to describe various elements, components, regions, layers and/or sections, these elements, components, regions, layers and/or sections should not be limited by these terms. These terms are only used to distinguish one element, component, region, layer or section from another element, component, region, layer or section. Thus, a first element, component, region, layer or section discussed herein could be termed a second element, component, region, layer or section without departing from the spirit and scope of the present inventive concept.

Spatially relative terms (such as "under," "below," "bottom," "under," "over," "top," and the like) may be used herein for ease of description to describe one element or feature's relationship to another element or feature as illustrated in the figures. It will be understood that such spatially relative terms are intended to encompass different orientations of the device in use or operation in addition to the orientation depicted in the figures. For example, if the device in the figures is turned over, elements described as "below" or "beneath" other elements or features would then be oriented "above" the other elements or features. Thus, the example terms "below" and "beneath" can encompass both an orientation of above and below. The device may be otherwise positioned (e.g., rotated 90 degrees or at other orientations) and the spatially relative descriptors used herein interpreted accordingly. Further, it will also be understood that when a layer is referred to as being "between" two layers, it can be the only layer between the two layers, or one or more intervening layers may also be present.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the inventive concept. As used herein, the terms "substantially," "about," and the like are used as approximations, not terms of degree, and are intended to describe the inherent deviation of a measured or calculated value that will be recognized by one of ordinary skill in the art.

As used herein, the singular forms are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and/or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof. As used herein, the term "and/or" includes any and all combinations of one or more of the associated listed items. Expressions such as "at least one of …" when following a column of elements modify the entire column of elements rather than a single element of the column. Furthermore, the use of "may" when describing embodiments of the inventive concept refers to "one or more embodiments of the invention". Moreover, the term "exemplary" is intended to mean exemplary or illustrative. As used herein, the terms "use" and "used" may be considered synonymous with the terms "utilize" and "utilized," respectively.

It will be understood that when an element or layer is referred to as being "on," "connected to," "coupled to," or "adjacent to" another element or layer, it can be directly on, "connected to," "coupled to," or "adjacent to" the other element or layer, or one or more intervening elements or layers may be present. In contrast, when an element or layer is referred to as being "directly on," "directly connected to," "directly coupled to," or "directly adjacent to" another element or layer, there are no intervening elements or layers present.

Any numerical range recited herein is intended to include all sub-ranges subsumed within that range with the same numerical precision. For example, a range of "1.0 to 10.0" is intended to include all sub-ranges between (and including) the minimum value of 1.0 and the maximum value of 10.0, i.e., having a minimum value equal to or greater than 1.0 and a maximum value of equal to or less than 10.0, such as, for example, 2.4 to 7.6. Any maximum numerical limitation recited herein is intended to include all lower numerical limitations subsumed therein, and any minimum numerical limitation recited in this specification is intended to include all higher numerical limitations subsumed therein.

Although exemplary embodiments oF multi-mode and/or multi-speed fabric-based high speed non-volatile memory (NVM) (NVMe-af) devices have been specifically described and illustrated herein, many modifications and variations will be apparent to those skilled in the art. It will thus be appreciated that the systems and methods for supporting multi-mode and/or multi-speed NVMe-af devices constructed in accordance with the principles oF the present invention may be implemented differently than as specifically described herein. The invention is also defined in the following claims and equivalents thereof.

Claims

1. A memory device, comprising:

a Printed Circuit Board (PCB);

at least one Solid State Drive (SSD) connected on a first side of the PCB via at least one SSD connector;

at least one Field Programmable Gate Array (FPGA) mounted on the PCB at a second side of the PCB;

at least one connector attached to the PCB at a third side of the PCB,

wherein the storage device is configured to: determining whether to operate in a first mode oF operation or a second mode oF operation based on a first input received via the at least one connector, wherein the first mode oF operation is a non-volatile memory express, NVMe, mode and the second mode oF operation is a network-based non-volatile memory express, NVMe-af, mode.

2. The storage device of claim 1, wherein the at least one SSD comprises a first SSD and a second SSD, and the at least one SSD connector comprises a first SSD connector and a second SSD connector, the first SSD connected with the PCB at a first side of the PCB via the first SSD connector, and the second SSD connected with the PCB at the first side of the PCB via the second SSD connector, each of the first SSD and the second SSD being a New size Specification 1NF 1SSD, wherein the at least one connector is a U.2 connector, and each of the first SSD connector and the second SSD connector is an M.2 connector.

3. The storage device of claim 1, wherein the at least one SSD connector is attached to and electrically connected with the at least one SSD on a first side of the at least one SSD, the at least one SSD connector is attached to and perpendicular to the first side of the PCB on a first side of the PCB.

4. The storage device of claim 1, further comprising: a structural support between the second side of the PCB and the second side of the at least one SSD, wherein the structural support is a plastic support or a bracket.

5. The storage device of claim 1, wherein a length of the at least one SSD is 110mm and a length of the PCB is between 110mm and 142.2 mm.

6. The storage device of claim 1, wherein a first side of the at least one FPGA is attached to a second side of the PCB, the at least one FPGA being electrically connected with the PCB.

7. The storage device of claim 6, further comprising: a first interface layer attached at a third side of the at least one SSD; and a second interface layer attached to a second side of the at least one FPGA, wherein the first and second interface layers are used to transfer heat generated by the at least one SSD and the at least one FPGA during operation of the at least one SSD and the at least one FPGA.

8. The storage device of claim 1, wherein the at least one connector is an SFF-TA-1008 connector.

9. The storage device of claim 1, wherein a length of the PCB is equal to or longer than a length of the at least one SSD.

10. The storage device of claim 1, wherein the length of the PCB is equal to or longer than the length of the at least one FPGA, which is 80 mm.

11. The storage device of claim 1, wherein when it is determined that the storage device is operating in the second mode of operation, the at least one SSD operates at a selected operating speed of a plurality of operating speeds of the storage device, wherein the plurality of operating speeds of the storage device are two or more speeds above 10G, and the selected operating speed is selected from the plurality of operating speeds based on a second input received by the storage device.

12. The memory device of claim 1, wherein the first input is received from the midplane via the at least one connector, wherein the first input is controlled by using two general purpose input/output (GPIO) pins or one or more internal registers internal to the at least one FPGA controlled by a Baseboard Management Controller (BMC) of the motherboard or a local Central Processing Unit (CPU).

13. A storage system, comprising:

a first interface layer;

at least one Solid State Drive (SSD), wherein a first interface layer is attached to a first side of the at least one SSD;

a Printed Circuit Board (PCB) connected with the at least one SSD on a first side of the PCB via at least one SSD connector;

at least one Field Programmable Gate Array (FPGA) mounted on the PCB at a second side of the PCB, wherein a first side of the at least one FPGA is attached to the PCB;

a second interface layer attached to the at least one FPGA on a second side of the at least one FPGA;

at least one connector attached to the PCB at a third side of the PCB,

wherein the storage system is configured to: determining whether to operate in a first mode oF operation or a second mode oF operation based on a first input received via the at least one connector, wherein the first mode oF operation is a non-volatile memory express (NVMe) mode and the second mode oF operation is a network-based non-volatile memory express (NVMe-oF) mode.

14. The storage system of claim 13, wherein the at least one connector is an U.2 connector or an SFF-TA-1008 connector and the at least one SSD connector is an m.2 connector, the at least one SSD connector being perpendicular to the first side of the PCB.

15. The storage system of claim 13, further comprising: a structural support between the second side of the PCB and the second side of the at least one SSD, wherein the structural support is a plastic support or a bracket.

16. The storage system of claim 13, wherein the at least one SSD is 110mm in length, the PCB is between 110mm and 142.2mm in length, and the at least one FPGA is 80mm in length.

17. The storage system of claim 13, wherein the length of the PCB is equal to or longer than the length of the at least one SSD, and the length of the PCB is equal to or longer than the length of the at least one FPGA.

18. A method of operation of a storage system, comprising:

attaching a first interface layer to a first side of at least one Solid State Drive (SSD);

connecting the at least one SSD on a first side of a printed circuit board PCB via at least one SSD connector;

mounting at least one Field Programmable Gate Array (FPGA) on a second side of the PCB, wherein the first side of the at least one FPGA is attached to the PCB;

attaching a second interface layer to a second side of the at least one FPGA;

attaching at least one connector to a third side of the PCB,

wherein the at least one SSD is configured to: determining whether to operate in a first mode oF operation or a second mode oF operation based on a first input received via the at least one connector, wherein the first mode oF operation is a non-volatile memory express, NVMe, mode and the second mode oF operation is a network-based non-volatile memory express, NVMe-af, mode.

19. The method of operation of claim 18, further comprising: adding a structural support between the second side of the PCB and the second side of the at least one SSD, wherein the structural support is a plastic support or bracket and the at least one SSD connector is perpendicular to the first side of the PCB.

20. The operating method of claim 18, wherein the length of the PCB is equal to or longer than the length of the at least one SSD, and the length of the PCB is equal to or longer than the length of the at least one FPGA.