US20190317908A1 - Device hot plug system - Google Patents
Device hot plug system Download PDFInfo
- Publication number
- US20190317908A1 US20190317908A1 US15/953,208 US201815953208A US2019317908A1 US 20190317908 A1 US20190317908 A1 US 20190317908A1 US 201815953208 A US201815953208 A US 201815953208A US 2019317908 A1 US2019317908 A1 US 2019317908A1
- Authority
- US
- United States
- Prior art keywords
- hot
- reference clock
- connector
- plug
- reset state
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F13/00—Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
- G06F13/14—Handling requests for interconnection or transfer
- G06F13/16—Handling requests for interconnection or transfer for access to memory bus
- G06F13/1668—Details of memory controller
- G06F13/1689—Synchronisation and timing concerns
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F13/00—Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
- G06F13/38—Information transfer, e.g. on bus
- G06F13/40—Bus structure
- G06F13/4063—Device-to-bus coupling
- G06F13/4068—Electrical coupling
- G06F13/4081—Live connection to bus, e.g. hot-plugging
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F13/00—Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
- G06F13/38—Information transfer, e.g. on bus
- G06F13/40—Bus structure
- G06F13/4063—Device-to-bus coupling
- G06F13/409—Mechanical coupling
Definitions
- the present disclosure relates generally to information handling systems, and more particularly to the hot plugging of devices in an information handling system.
- An information handling system generally processes, compiles, stores, and/or communicates information or data for business, personal, or other purposes thereby allowing users to take advantage of the value of the information.
- information handling systems may also vary regarding what information is handled, how the information is handled, how much information is processed, stored, or communicated, and how quickly and efficiently the information may be processed, stored, or communicated.
- the variations in information handling systems allow for information handling systems to be general or configured for a specific user or specific use such as financial transaction processing, airline reservations, enterprise data storage, or global communications.
- information handling systems may include a variety of hardware and software components that may be configured to process, store, and communicate information and may include one or more computer systems, data storage systems, and networking systems.
- Information handling systems sometime utilize devices that may be “hot-plugged”, “hot-swapped”, “hot-inserted”, and/or otherwise communicatively coupled to the system without shutting down or stopping the system.
- server devices often utilize hot-plug devices for storage.
- the conventional hot-plugging of some single communication lane storage devices such as, for example, Serial Attached Small Computer System Interface (SCSI) (SAS) storage devices and Serial AT Attachment (SATA) storage drives, is relatively stable.
- SAS Serial Attached Small Computer System Interface
- SAS Serial AT Attachment
- SATA Serial AT Attachment
- the hot-plugging of multiple communication lane storage devices has been found to raise some issues.
- PCIe Peripheral Component Interconnect express
- NVMe storage devices Non-Volatile Memory storage devices
- Such NVMe storage devices typically include securing latches that are rotatably coupled to one end of the front face of the NVMe storage device, with those securing latches also coupled to a securing subsystem on the NVMe storage device that is configured to engage the server chassis and/or connector to which the NVMe storage device is being connected.
- the securing latch when the securing latch is rotated away from the front face of the NVMe storage device such that the securing subsystem is unactivated, the NVMe storage device may be positioned in the storage device housing that is defined by the server system and located adjacent a server system connector.
- the securing latch may then be rotated towards the front face of the NVMe storage device in order to activate the securing subsystem to engage the server system and/or the sever system connector and mate the storage device connector on the NVMe storage device with the server system connector in the server system.
- the server device connector includes four sets of communication lane pins that are distributed across its length, with the fourth communication lane pins located closest to that point of contact, and the first communication lane pins located further away from the point of contact between the securing subsystem and the server system/server system connector. Furthermore, the server system connector includes reset pin (e.g. a PERst pin) that is located approximately midway along the length of the server system connector, and a pair of reference clock pins that are located adjacent the end of the server system connector that is opposite the point of contact between the securing subsystem and the server system/server system connector.
- reset pin e.g. a PERst pin
- the engagement of the storage device connector with the server system connector results in engagement of the storage device connector with power pins on the server system connector, along with the sequential engagement of the fourth communication lane pins, followed by engagement of the third communication lane pins, followed by engagement of the second communication lane pins, and followed by engagement of the first communication lane pins.
- the NVMe storage device is configured with a 70 millisecond reset time that starts when the NVME storage device receives power from the server system connector.
- the reset time for the NVMe storage device may expire, causing problems with the operation of the NVMe storage device.
- the closing of the securing latch may cause the NVMe storage device to engage the fourth communication lane pins and the third communication lane pins, followed by engagement of the NVMe storage device with the reset pin after the expiration of the reset time for the NVMe storage device, which results in the reset of the NVMe storage device being driven by the engagement of the reset pin.
- the reset pin is being de-asserted prior to the NVMe storage device engaging each of the second communication lane pins, the first communication lane pins, and the reference clock pins.
- NVMe storage devices that are configured to perform “lane reversal” to convert the fourth communication lane to a first communication lane (and the third communication lane to a second communication lane in some embodiments), and ignore any other communication lanes that would otherwise be enabled by the subsequent engagement of the first and second communication lane pins.
- the NVMe storage device will not operate at its maximum throughput (i.e., the NVMe storage device is capable of communicating via four communication lanes, but only one or two communication lanes are enabled.) Furthermore, in NVMe storage devices that are not configured to perform “lane reversal”, the NVMe storage device may not enable any communication lanes if an unexpected communication lane is detected (e.g., the fourth communication lane is detected when the first communication lane is expected), causing the NVMe storage device to be completely unusable (e.g., having no communication lanes available.)
- the sequential engagement of the storage device connector with the server system connector results in the sequential engagement of the NVMe storage device with the reset pin, followed by engagement with the reference clock pins as discussed above.
- the engagement of the NVMe storage device with the reset pin occurs prior to the reference clock (which is connected to the reference clock pins) reaching stability (e.g., 100 ⁇ s after engagement of the reference clock pins), and can cause the NVMe storage device to exit reset and enter its operational/functional state (i.e., the NVMe storage device will exit reset when the reset pin is engaged because the server system is not in reset.)
- the provisioning of the de-assertion signal on the reset pin while the reference clock is in the unstable state causes a violation (e.g., a Tperst-clk violation) of the PCIe electromechanical specification, and puts the NVMe storage device into a non-deterministic state due to the pre-reference-clock-stability de-assertion signal preventing
- an Information Handling System includes a processing system; and a memory system that is coupled to the processing system and that includes instructions that, when executed by the processing system, cause the processing system to provide a hot-plug engine that is configured, in response to a hot-plug event with a connector, to: enter a reset state; monitor, while in the reset state, a reference clock; determine, while in the reset state, that the reference clock has been stable for a predetermined time period; exit, in response to determining that the reference clock has been stable for the predetermined time period, the reset state; and provide, in response to exiting the reset state, a de-assertion signal on a reset pin included on the connector.
- a hot-plug engine that is configured, in response to a hot-plug event with a connector, to: enter a reset state; monitor, while in the reset state, a reference clock; determine, while in the reset state, that the reference clock has been stable for a predetermined time period; exit, in response to determining that the reference clock has been stable for
- FIG. 1 is a schematic view illustrating an embodiment of an information handling system.
- FIG. 2 is a schematic view illustrating an embodiment of a server system.
- FIG. 3 is a schematic view illustrating an embodiment of a connector that may be provided in the server system of FIG. 2 .
- FIG. 4 is a schematic view illustrating an embodiment of a hot-plug device.
- FIG. 5 is a schematic view illustrating an embodiment of registers that may be provided in the hot-plug device of FIG. 4 .
- FIG. 6 is a schematic view illustrating an embodiment of a circuit that may be provided in the hot-plug device of FIG. 4 .
- FIG. 7 is a flow chart illustrating an embodiment of a method for hot-plugging a device.
- FIG. 8A is a perspective view illustrating an embodiment of the hot-plug device of FIG. 4 being coupled to the server system of FIG. 2 .
- FIG. 8B is a schematic view illustrating an embodiment of the hot-plug device of FIG. 4 being coupled to the server system of FIG. 2 .
- FIG. 8C is a schematic view illustrating an embodiment of the registers of FIG. 5 with particular values during the method of FIG. 7 .
- FIG. 8D is a chart illustrating an embodiment of signal generated in response to the hot-plugging of the hot-plug device of FIG. 4 to the server system of FIG. 2 .
- an information handling system may include any instrumentality or aggregate of instrumentalities operable to compute, calculate, determine, classify, process, transmit, receive, retrieve, originate, switch, store, display, communicate, manifest, detect, record, reproduce, handle, or utilize any form of information, intelligence, or data for business, scientific, control, or other purposes.
- an information handling system may be a personal computer (e.g., desktop or laptop), tablet computer, mobile device (e.g., personal digital assistant (PDA) or smart phone), server (e.g., blade server or rack server), a network storage device, or any other suitable device and may vary in size, shape, performance, functionality, and price.
- the information handling system may include random access memory (RAM), one or more processing resources such as a central processing unit (CPU) or hardware or software control logic, ROM, and/or other types of nonvolatile memory. Additional components of the information handling system may include one or more disk drives, one or more network ports for communicating with external devices as well as various input and output (I/O) devices, such as a keyboard, a mouse, touchscreen and/or a video display. The information handling system may also include one or more buses operable to transmit communications between the various hardware components.
- RAM random access memory
- processing resources such as a central processing unit (CPU) or hardware or software control logic
- ROM read-only memory
- Additional components of the information handling system may include one or more disk drives, one or more network ports for communicating with external devices as well as various input and output (I/O) devices, such as a keyboard, a mouse, touchscreen and/or a video display.
- I/O input and output
- the information handling system may also include one or more buses operable to transmit communications between the various
- IHS 100 includes a processor 102 , which is connected to a bus 104 .
- Bus 104 serves as a connection between processor 102 and other components of IHS 100 .
- An input device 106 is coupled to processor 102 to provide input to processor 102 .
- Examples of input devices may include keyboards, touchscreens, pointing devices such as mice, trackballs, and trackpads, and/or a variety of other input devices known in the art.
- Programs and data are stored on a mass storage device 108 , which is coupled to processor 102 . Examples of mass storage devices may include hard discs, optical disks, magneto-optical discs, solid-state storage devices, and/or a variety other mass storage devices known in the art.
- IHS 100 further includes a display 110 , which is coupled to processor 102 by a video controller 112 .
- a system memory 114 is coupled to processor 102 to provide the processor with fast storage to facilitate execution of computer programs by processor 102 .
- Examples of system memory may include random access memory (RAM) devices such as dynamic RAM (DRAM), synchronous DRAM (SDRAM), solid state memory devices, and/or a variety of other memory devices known in the art.
- RAM random access memory
- DRAM dynamic RAM
- SDRAM synchronous DRAM
- solid state memory devices solid state memory devices
- a chassis 116 houses some or all of the components of IHS 100 . It should be understood that other buses and intermediate circuits can be deployed between the components described above and processor 102 to facilitate interconnection between the components and the processor 102 .
- the server system 200 may be provided by the IHS 100 discussed above with reference to FIG. 1 , and/or may include some or all of the components of the IHS 100 .
- the server system 200 includes a chassis 202 that houses the components of the server system 200 , only some of which are illustrated in FIG. 2 .
- the chassis 202 may house a circuit board 204 such as, for example, a motherboard and/or other boards that would be apparent to one of skill in the art in possession of the present disclosure.
- the chassis 202 may also house a processing system (not illustrated, but which may include the processor 102 discussed above with reference to FIG.
- a memory system (not illustrated, but which may include the memory 114 discussed above with reference to FIG. 1 and may be mounted to the circuit board 204 ) that is coupled to the processing system and that includes instruction that, when executed by the processing system, cause the processing system to provide a server engine 206 that is configured to perform the functions of the server engines and server systems discussed below.
- an optional reference clock 208 may be housed in the chassis 208 .
- FIG. 2 illustrates the optional reference clock 208 that is included as part of the server engine 206 and that is coupled to the circuit board 204 , although other locations for a server-system-reference-clock will fall within the scope of the present disclosure as well.
- a connector 210 is mounted to the circuit board 204 and coupled to the server engine 206 (e.g., via a trace included on the circuit board 204 .)
- the connector 210 is described as being a female Peripheral Component Interconnect express (PCIe) Non-Volatile Memory (NVMe) connector having a first end 210 a and a second end 210 b that is located opposite the connector 210 from the first end 210 a , although other multi-communication lane connectors may benefit from the teachings of the present disclosure and thus are envisioned as falling within its scope as well.
- PCIe Peripheral Component Interconnect express
- NVMe Non-Volatile Memory
- the chassis 202 houses a device housing 212 that extends into the chassis 202 adjacent the connector 210 and that is configured to house the hot-plug device of the present disclosure.
- a specific server system 200 has been illustrated and described for purposes of discussion herein, one of skill in the art in possession of the present disclosure will recognize that a variety of systems may include a variety of components, features, and/or configurations for providing conventional functionality, as well as for utilizing the hot-plug device of the present disclosure, while remaining within the scope of the present disclosure.
- the connector 300 includes a base 302 having a first end 302 a that corresponds to the first end 210 a of the connector 210 in the examples below, and a second end 302 b that is located opposite the base 302 from the first end 302 a and that corresponds to the second end 210 b of the connector 210 in the examples below.
- a plurality of pins may be provided on the connector 300 .
- a first set of communication lane pins 304 a - 304 e are included on the connector 300 adjacent the second end 302 b of the base 302 , and may include an ePCIe lane 0 positive transmit pin 304 a (“PETp0”, pin # E10), an ePCIe lane 0 negative transmit pin 304 b (“PETn0”, pin # E11), a ground pin 304 c (“GND”, pin # E12), an ePCIe lane 0 negative receive pin 304 d (“PERn0”, pin # E13), and an ePCIe lane 0 positive receive pin 304 e (“PERp0”, pin # E14).
- a second set of communication lane pins 306 a - 306 e are included on the connector 300 between the first set of communication lane pins 304 a - e and the first end 302 a of the base 302 , and may include an ePCIe lane 1 positive transmit pin 306 a (“PETp1”, pin # E19), an ePCIe lane 1 negative transmit pin 306 b (“PETn1”, pin # E20), a ground pin 306 c (“GND”, pin # E21), an ePCIe lane 1 negative receive pin 306 d (“PERn1”, pin # E22), and an ePCIe lane 1 positive receive pin 306 e (“PERp1”, pin # E23).
- a third set of communication lane pins 308 a - 308 e are included on the connector 300 between the second set of communication lane pins 306 a - e and the first end 302 a of the base 302 , and may include an ePCIe lane 2 positive transmit pin 308 a (“PETp2”, pin # E25), an ePCIe lane 2 negative transmit pin 308 b (“PETn2”, pin # E26), a ground pin 308 c (“GND”, pin # E27), an ePCIe lane 2 negative receive pin 308 d (“PERn2”, pin # E28), and an ePCIe lane 2 positive receive pin 308 e (“PERp2”, pin # E29).
- a fourth set of communication lane pins 310 a - 310 e are included on the connector 300 between the third set of communication lane pins 308 a - e and the first end 302 a of the base 302 , and may include an ePCIe lane 3 positive transmit pin 310 a (“PETp3”, pin # E31), an ePCIe lane 3 negative transmit pin 310 b (“PETn3”, pin # E32), a ground pin 310 c (“GND”, pin # E33), an ePCIe lane 3 negative receive pin 310 d (“PERn3”, pin # E34), and an ePCIe lane 3 positive receive pin 310 e (“PERp3”, pin # E35).
- a fifth set of communication lane pins 311 a - 311 e are included on the connector 300 adjacent the second end 302 b of the base 302 and opposite the base from the first set of communication lane pins 304 a - e , and may include a SAS/SATA/SATAe lane 0 positive transmit pin 311 a (“SOT+(A+)”, pin # S2), a SAS/SATA/SATAe lane 0 negative transmit pin 311 b (“SOT ⁇ (A ⁇ )”, pin # S3), a ground pin 311 c (“GND”, pin # S4), a SAS/SATA/SATAe lane 0 negative receive pin 311 d (“SOR ⁇ (B ⁇ )”, pin # S5), and a SAS/SATA/SATAe lane 0 positive receive pin 311 e (“SOR+(B+)”, pin # S6).
- hot-plug devices such as x4 NVMe storage devices may enable the use of four communication lanes on the connector 300 that are provided by the first, second, third, and fourth set of communication lane pins discussed above, while hot-plug devices such as x1 SAS/SATA storage devices may enable the use of a single communication lane on the connector 300 that is provided by the fifth set of communication lane pins discussed above.
- a plurality of reference clock pins 312 a and 312 b are included on the connector 300 between the first set of communication lane pins 304 ae and the second side 302 b of the base 302 , and may include an ePCIe positive reference clock pin 312 a (“RefClk0+”, pin # E7) and an ePCIe negative reference clock pin 312 b (“RefClk0 ⁇ ”, pin # E8).
- the reference clock pins 312 a and 312 b on the connector 300 may be coupled to the reference clock 208 included in the server system 200 (e.g., via the traces between the server engine 206 and the connector 210 in FIG.
- the connector 300 also include a reset pin 314 a that located approximately midway along the length of the connector 300 between the fifth set of communication lane pins 311 a - e and the first side 302 a of the base 302 , and may include an ePCIe port A reset pin 314 a (“ePERst0#”, pin # E5).
- the connector 300 also include a plurality of power pins 315 a , 315 b , and 315 c , each of which are located adjacent the first side 302 a of the base 302 , and that may each include ePCIe SSD power pins (pins # P13, P14, and P15). While a specific connector has been illustrated and described, one of skill in the art in possession of the present disclosure will recognize that a variety of connectors utilized in hot-plug-enabled systems may benefit from the teachings of the present disclosure and thus fall within its scope as well.
- the hot-plug device 400 may be provided by the IHS 100 discussed above with reference to FIG. 1 , and/or may include some or all of the components of the IHS 100 .
- the hot-plug device 400 is an NVMe device (and specifically, an NVMe storage device), but one of skill in the art in possession of the present disclosure will recognize that a variety of other hot plug devices may benefit from the teachings of the present disclosure and thus will fall within its scope as well.
- the hot-plug device 400 includes a chassis 402 that houses the components of the hot plug-device 400 , only some of which are illustrated in FIG. 4 .
- the chassis 402 may house a processing system (not illustrated, but which may include the processor 102 discussed above with reference to FIG. 1 ), and a memory system (not illustrated, but which may include the memory 114 discussed above with reference to FIG. 1 ) that is coupled to the processing system and that includes instructions that, when executed by the processing system, cause the processing system to provide a hot-plug engine 404 that is configured to perform the functions of the hot-plug engines and hot-plug devices discussed below.
- a processing system not illustrated, but which may include the processor 102 discussed above with reference to FIG. 1
- a memory system not illustrated, but which may include the memory 114 discussed above with reference to FIG. 1
- the hot-plug engine 404 may be provided by an Application Specific Integrated Circuit (ASIC) in the hot-plug device 400 (e.g., the storage device controller ASIC in an NVMe storage device), although other components (e.g., firmware, CPU/memory combinations, etc.) that may be configured to provide the functionality of the hot-plug engine 404 discussed below are envisioned as falling within the scope of the present disclosure as well.
- ASIC Application Specific Integrated Circuit
- a plurality of registers 406 are housed in the chassis 402 .
- FIG. 4 illustrates the registers included as part of the server engine 206 (e.g., as registers included in the ASIC discussed above), although other locations for the registers will fall within the scope of the present disclosure as well.
- an optional reference clock 408 may be housed in the chassis 402 , and FIG. 4 illustrates the optional reference clock 408 included as part of the hot-plug engine 404 , although other locations for a hot-plug-device-reference-clock will fall within the scope of the present disclosure as well.
- the reference clock 408 may be provided in the hot-plug device 400 in possible future implementations of NVMe storage devices that provide Separate Reference clocks with Independent Spread Spectrum Clocking (SSC) (SRIS).
- SRIS implementations may omit the reference clock 208 from the server system 200 and the reference clock pins 312 a and 312 b on the connector 300 , and instead provide that any two ends of a communication link (e.g., including the hot-plug device 400 ) include a separate reference clock (e.g., the reference clock 408 ).
- a connector 410 is included on the chassis 402 and is coupled to the hot-plug engine 404 .
- the connector 410 is described as being a male Peripheral Component Interconnect express (PCIe) Non-Volatile Memory (NVMe) connector having a first end 410 a and a second end 410 b that is located opposite the connector 410 from the first end 410 a , although other multi-communication lane connectors may benefit from the teachings of the present disclosure and thus are envisioned as falling within its scope as well.
- PCIe Peripheral Component Interconnect express
- NVMe Non-Volatile Memory
- the connector 410 includes pins that are configured to engage corresponding pins on the connector 300 (e.g., the communication lane pins, reference clock pins, and reset pin discussed with reference to FIG. 3 ).
- the chassis 402 include a securing subsystem having a securing latch 412 .
- the securing latch 412 includes a securing subsystem activation arm 412 a that is rotatably coupled to the chassis 402 via a rotatable coupling 412 b that allows the securing subsystem activation arm 412 a to move between a securing orientation in which the securing subsystem activation arm 412 a is located immediately adjacent the chassis 402 (illustrated in solid lines in FIG. 4 ), and a deactivated orientation in which the securing subsystem activation arm 412 a is rotated away from the chassis 402 (illustrated in dashed lines in FIG. 4 ).
- the securing latch 412 may be coupled to other components of the securing subsystem that are located on the chassis 402 and that are configured to engage the chassis 202 of the server system 200 and/or the connector 210 / 300 to couple the hot-plug device 400 the server system 200 , discussed in further detail below. While a hot-plug device 400 has been illustrated and described for purposes of discussion herein, one of skill in the art in possession of the present disclosure will recognize that hot-plug devices may include a variety of components, features, and/or configurations for providing conventional hot-plug device functionality, as well as the functionality discussed below, while remaining within the scope of the present disclosure.
- FIG. 5 an embodiment of a plurality of registers 500 are illustrated that may be the registers 406 discussed above with reference to FIG. 4 .
- the registers 500 may be provided in an ASIC included in the hot-plug device 400 .
- the registers 500 include a reference clock implementation register 502 that may be configured to indicate how a reference clock is implemented in the device hot-plug system of the present disclosure.
- the reference clock implementation register 502 may include a value of “0” to indicate the system reference clock is implemented in the device hot-plug system by the reference clock 208 included in the server system 200 (i.e., a reference clock that is external to the hot-plug device 400 ), and the reference clock implementation register 502 may include a value of “1” to indicate that the system reference clock is implemented in the device hot-plug system by the reference clock 408 included in the hot-plug device 400 (i.e., a reference clock that is internal to the hot-plug device 400 ).
- the registers 500 also include a clock counter value register 504 that may be configured to indicate an amount of time the hot-plug device 400 should remain in a device reset state (e.g., “100” in the example below), discussed in further detail below.
- the registers 500 also include a clock counter time unit register 506 that may be configured to indicate time units for the amount of time that the hot-plug device 400 should remain in the device reset state (e.g., “ ⁇ s” in the example below), discussed in further detail below.
- registers 500 While a specific examples of registers have been illustrated and described, one of skill In the art in possession of the present disclosure will recognize that the information included in the registers 500 , and/or the functionality resulting from the use of that information, may be provided in a variety of manners that will fall within the scope of the present disclosure as well.
- the circuit 600 includes a multiplexer 602 that receives inputs from the reference clock implementation register 502 , and may receive inputs from either of the reference clocks 208 and 408 (when present in the hot-plug system).
- the multiplexer 602 outputs to a decision block 604 that receives inputs from the clock counter value register 504 and the clock counter time unit register 506 , and that provides for the determination of whether a counter has expired according to those registers.
- the circuit 600 also includes decision blocks 606 , 608 , 610 , and 611 that each provide for the determination of whether a respective communication lane has been detected.
- An AND gate 612 a receives the YES outputs of each of the decision blocks 610 and 611 , and will output a signal when each of those decision blocks have detected their respective communication lanes.
- An AND gate 612 b receives the output of the AND gate 612 a , and the YES output of decision block 608 , and will output a signal when decision block 608 detects its respective communication lane, and the AND gate 612 a indicates each of the decision blocks 610 and 611 have detected their respective communication lanes
- a multiplexer 612 c receives inputs from decision block 608 , AND gate 612 b , and a register 613 in the hot-plug device 400 that indicates the number of communication lanes available for the hot-plug device 400 (e.g., four communications lanes in this example.) The multiplexer 612 c outputs to an AND gate 612 d that receives an input from decision block 606 , and the AND gate 612 d will output a signal when the decision block 606 detect
- An AND gate 614 receives the YES output of the decision block 604 , and the output of the AND gate 612 d , and will output a device reset signal to an AND gate 616 when the decision block 604 determines that that a counter has expired according to the registers 504 and 506 , and a signal is received from the AND gate 612 d .
- the AND gate 616 also receives an input from the reset pin on the connector 210 / 300 (e.g., “PERst #”, pin # E5), and will output the reset signal used to reset the hot-plug device in response to receiving the device reset signal from the AND gate 614 and detecting a connection to the reset pin on the connector 210 / 300 . While a specific circuit has been illustrated and described, one of skill in the art in possession of the present disclosure will recognize that the functionality of the hot-plug engine 404 may be implemented in a variety of manners that will fall within the scope of the present disclosure as well
- a hot-plug device such as, for example, an NVMe storage device, that is configured, when being hot-plugged to a connector, to hold itself in a device reset state for a predetermined amount of time that is designated in register(s) included in the hot-plug device and that is known to allow a reference clock to reach a stable reference clock state.
- the hot-plug device then performs a device reset that results in the provisioning of a de-assertion signal on a reset pin included on the connector that ensures that the hot-plug device is put into a deterministic state with associated state machines in their default state.
- the hot-plug device may also be configured, when being hot-plugged to the connector, to hold itself in the device reset state until each of the communication pins on the connector have been engaged in order to ensure that each communication lane on the hot-plug device is utilized in its subsequent operation.
- securing subsystems on such hot-plug devices that include securing latches (and/or other features) that provide for sequential engagement of the hot-plug device with pins on the connector do not cause issues associated with conventional hot-plug systems such as, for example, the communication lane issues and reference clock instability/non-deterministic device state issues discussed above.
- the method 700 begins at block 702 where a hot-plug device is hot-plugged to a connector and enters a device reset state.
- the hot-plug device of FIG. 4 is hot-plugged to the connector 210 / 300 in the server system 200 .
- the hot-plug device 400 is positioned adjacent the server system 200 such that connector 410 on the hot-plug device 400 is located adjacent the device housing 212 on the server system 200 , with the securing latch 412 in the deactivated orientation (illustrated by the dashed lines in FIG. 4 ).
- the hot-plug device 400 is then moved in a direction A such that the hot-plug device 400 enters the device housing 212 and moves through the device housing 212 until the connector 410 on the hot-plug device 400 is located immediately adjacent the connector 210 / 300 in the server system 200 , as illustrated in FIG. 8B .
- the hot-plugging of the hot-plug device 400 to the connector 210 / 300 on the server system 200 occurs when the securing latch 412 is moved from the deactivated orientation in which the securing subsystem activation arm 412 a is rotated away from the chassis 402 of the hot-plug device 400 (illustrated by the dashed lines in FIG.
- Positioning of the securing latch 412 in the securing orientation activates the securing subsystem on the hot-plug device 400 by, for example, causing that securing subsystem to engage the chassis 202 of the server system 200 and/or the connector 210 / 300 to mate the connector 410 on the hot-plug device 400 with the connector 210 / 300 in the server system 200 , while securing the hot-plug device 400 in the chassis 202 of the server system 200 .
- “hot-plugging”, “hot-swapping”, “hot-inserting”, and/or otherwise communicatively coupling the hot-plug device 400 to the server system 200 without shutting down or stopping the server system 200 may be accomplished in a variety of manners.
- the server engine 206 may include an operating system that is in a runtime state and that is controlling the operation of the server system 200 when the hot-plug device 400 is connected to the connector 210 / 300 in order to provide a hot-plug event.
- the server engine 206 may include a Basic Input Output System (BIOS) that has been currently performing a boot of the server system 200 when the hot-plug device 400 is connected to the connector 210 / 300 in order to provide the hot-plug event.
- BIOS Basic Input Output System
- the use of the securing latch 412 and securing subsystem on the hot-plug device 400 to mate the connector 410 on the hot-plug device 400 to the connector 210 / 300 causes a sequential engagement of the connector 410 on the hot-plug device 400 with the pins on the connector 210 / 300 in the server system 200 . As can be seen in FIG.
- the first end 410 a of the connector 410 is located adjacent the first end 210 a / 302 a of the connector 210 / 300
- the second end 410 b of the connector 410 is located adjacent the second end 210 b / 302 b of the connector 210 / 300 .
- the movement of the securing latch 412 to the securing orientation cause the securing subsystem on the hot-plug device 400 to engage the engage the chassis 202 of the server system 200 and/or the connector 210 / 300 at a point of contact that is adjacent or on the first end 210 a / 302 a of the connector 210 / 300 such that the connector 410 on the hot-plug device 400 first engages the pins on the connector 210 / 300 adjacent its first end 210 a / 302 a , and then engages adjacent pins along the length of the connector 210 / 300 in sequence until the pins on the connector 210 / 300 adjacent its second end 210 b / 302 b are engaged.
- the sequential engagement of the connector 410 on the hot-plug device 400 with the connector 300 in this manner results in engagement of the power pins 315 a - c , along with the sequential engagement of the fourth set of communication lane pins 310 a - 310 e , followed by engagement of the third set of communication lane pins 318 a - 308 e , followed by engagement of the second set of communication lane pins 306 a - 306 e , and followed by engagement of the first set of communication lane pins 304 a - 304 e .
- the engagement of the power pins 315 a - c may start a reset time in the NVMe storage device and, prior to all of the sets of communication lane pins engaging, the reset time for the NVMe storage device may expire, causing problems with the operation of the NVMe storage device due to the reset of the NVMe storage device being driven by the engagement of the reset pin.
- Those problems may include the NVMe storage device performing a “lane reversal” to convert the a subset of the communication lanes (provided by the communication lane pins) to active communication lane(s) (and ignoring any other communication lanes that are enabled by subsequent engagement of the other sets of communication lane pins) and preventing the NVMe storage device from operating at its maximum throughput, or not enabling any communication lanes due to the detection of an unexpected communication lane and making the NVMe storage device unusable.
- the sequential engagement of the connector 410 on the hot-plug device 400 with the connector 300 in the manner discussed above may result in the sequential engagement of the reset pin 314 a followed by engagement of the reference clock pins 312 a and 312 b .
- the NVMe storage device may reset and provide a de-assertion signal on the reset pin 314 a , and the provisioning of the de-assertion signal on the reset pin 314 a while the reference clock is in the unstable state causes a violation (e.g., a Tperst-clk violation) of the PCIe electromechanical specification, and puts the NVMe storage device into a non-deterministic state due to the pre-reference-clock-stability de-assertion signal preventing its state machines from entering their default state.
- a violation e.g., a Tperst-clk violation
- the device hot-plug system of the present disclosure may operate to prevent these issues by having the hot-plug device “hold” itself in its device reset state until the reference clock is stable for a predetermined time period and, in some embodiments, “hold” itself in its device reset state until each of the communication lanes have been activated.
- the method 700 then proceeds to decision block 704 where it is determined whether a reference clock has been stable for a predetermined amount of time.
- server systems that include the reference clock 208 coupled to the reference clock pins 312 a and 312 b on the connector 210 / 300 , at block 702 and during the hot-plugging of the hot-plug device 400 to the connector 210 / 300 on the server system 200 , the connector 410 on the hot-plug device 400 will engage the reset pin 314 a , followed by its engagement with the reference clocks pins 312 a and 312 b .
- the engagement of the connector 410 on the hot-plug device 400 with the reference clocks pins 312 a and 312 b allows the hot-plug engine 404 to begin monitoring the reference clock 208 at decision block 704 .
- the hot-plug engine 404 may access the registers 406 / 500 and retrieve the values included therein to determine whether the reference clock 208 has been stable for the predetermined amount of time.
- registers 500 an example of the registers 500 is illustrated with the reference clock implementation register 502 including a “0” to indicate that the reference clock 208 is included in the server system 200 , and the clock counter value register 504 including a “100” and the clock counter time unit register 506 including a “ ⁇ s” to indicate that the predetermined amount of time is 100 microseconds.
- the reference clock implementation register 502 including a “0” to indicate that the reference clock 208 is included in the server system 200
- the clock counter value register 504 including a “100”
- the clock counter time unit register 506 including a “ ⁇ s” to indicate that the predetermined amount of time is 100 microseconds.
- the reference clock implementation register 502 value of “0” may cause the hot-plug engine 404 to pass the signal from reference clock 208 to the decision block 604
- the clock counter value register 504 and clock counter time unit register 506 values of “100” and “ ⁇ s” may cause the hot-plug engine 404 to monitor the signal from the reference clock 208 for 100 microseconds at decision block 604 .
- a counter in the hot-plug engine 404 may be incremented based on the reference clock 208 (accessed via the engaged reference clock pins 312 a and 312 b ) until that counter exceeds 100 microseconds.
- a reset pin signal 802 e.g., “PERST #”
- a reference clock signal 804 e.g., “REFCLK”
- a 3.3 V aux signal e.g., a 3.3/12 volt signal
- a PCI express link signal e.g., a system management bus (SMBus) signal
- JTAG Joint Test Action Group
- the reset pin signal 802 is asserted (which, in the illustrated example, causes the signal to go “low”), and at time 1 , the 3.3/12 volt signal begins ramping up while the SMBus signal goes active.
- the 3.3/12 volt signal is stable, and the reference clock signal 804 begins but is unstable.
- the reference clock signal 804 becomes stable, and at time 4 , the reset pin signal 802 is de-asserted.
- the PCI express link signal is active, and at time 6 , the JTAG signal is active.
- the time period between time 3 and time 4 corresponds to the time period between the time at which the reference clock signal becomes stable, and the time at which the reset pin signal 802 should be de-asserted (as de-asserting the reset pin signal 802 between time 2 and time 3 while the reference clock signal 804 is unstable can cause the issues discussed above).
- the time period between time 3 and time 4 may be set at 100 microseconds to ensure that the reference clock signal 804 is stable before de-asserting the reset pin signal 802 .
- hot-plug devices that include the reference clock 408 (e.g., when the reference clock 208 in the server system 200 and the reference clock pins 312 a and 312 b on the connector 210 / 300 may be omitted), at block 702 and during the hot-plugging of the hot-plug device 400 to the connector 210 / 300 on the server system 200 , the connector 410 on the hot-plug device 400 will engage the reset pin 314 a .
- the hot-plug engine 404 may begin monitoring the reference clock 408 .
- the hot-plug engine 404 may access the registers 406 / 500 and retrieve the values included therein to determine whether the reference clock 408 has been stable for the predetermined amount of time.
- the reference clock implementation register 502 may include a “1” to indicate that the reference clock 408 is included in the hot-plug device 400 , with the clock counter value register 504 including a “100” and the clock counter time unit register 506 including a “ ⁇ s” to indicate that the predetermined amount of time is 100 microseconds.
- the reference clock implementation register 502 value of “1” may cause the hot-plug engine 404 to pass the signal from reference clock 408 to the decision block 604
- the clock counter value register 504 and clock counter time unit register 506 values of “100” and “ ⁇ s” may cause the hot-plug engine 404 to monitor the signal from the reference clock 408 for 100 microseconds.
- a counter in the hot-plug engine 404 may be incremented based on the reference clock 408 until that counter exceeds 100 microseconds.
- the method 700 returns to decision block 704 and continues to loop through decision block 704 until the reference clock has been stable for the predetermined amount of time (e.g., that counter based on the reference clock reaches 100 microseconds).
- the method 700 may proceed to optional decision block 706 where it is determined whether each communication lane pin on the connector has been engaged.
- the connector 410 on the hot-plug device 400 will first engage the fourth set of communication lane pins 310 a - 310 e , followed by engagement of the third set of communication lane pins 318 a - 308 e , followed by engagement of the second set of communication lane pins 306 a - 306 e , and followed by engagement of the first set of communication lane pins 304 a - 304 e.
- the hot-plug engine 404 will monitor each of the sets of communications lane pins via the decision blocks 606 , 608 , 610 , and 611 .
- the decision blocks 606 , 608 , 610 , and 611 operate to determine whether each of the first, second, third, and fourth communication lanes (i.e., provided by respective sets of communication lane pins), respectively, have been detected and, if so, pass a respective signal to the AND gates 6121 , 612 b , 612 d , and the multiplexer 612 c .
- the AND gate 612 d may be configured to only pass a signal to the AND gate 614 when each of the communication lanes have been detected via the decision blocks 606 , 608 , 610 , and 611 , and the detected number of communication lanes corresponds to an expected number of communication lanes in the register 613 .
- the hot-plug engine 404 will monitor the set of communications lane pins via a single decision block 606 , and that decision block 606 will operate to determine whether the fifth communication lane (i.e., provided by respective sets of communication lane pins 311 a - e ) has been detected (and is the only communication lane that is expected according to the register 613 ) and, if so, pass a respective signal to the AND gate 612 d.
- the fifth communication lane i.e., provided by respective sets of communication lane pins 311 a - e
- the method 700 returns to decision block 706 and continues to loop through decision block 706 until each communication lane pin on the connector has been engaged.
- the method 700 proceeds to block 708 where the hot-plug device exits the device reset state. While decision blocks 704 and 706 have been illustrated and described as being performed separately with decision block 704 being performed before decision block 706 , those decision blocks may be performed simultaneously or in a different order while remaining within the scope of the present disclosure.
- decision block 704 may be performed (e.g., a determination that reference clock is stable) prior to decision block 706 being performed (e.g., a determination that all communication lanes have been detected).
- decision block 706 may be performed (e.g., a determination that all communication lanes have been detected) prior to decision block 704 being performed (e.g., a determination that reference clock is stable.)
- the hot-plug engine 404 causes the hot-plug device 400 to exit its device reset state.
- the hot-plug engine 404 may be configured to cause the hot-plug device 400 to exit its device reset state (e.g., the “device_reset #) output of the AND gate 614 ).
- the method 700 then proceeds to block 710 where the hot-plug device enters an operational/functional state by providing a de-assertion of the hot-plug device function/operation reset, taking the hot-plug device out of the reset state and into the operational/functional state.
- the decision block 604 passing its signal to the AND gate 614 , and the AND gate 612 d passing its signal to the AND gate 614 causes the AND gate 614 to provide a signal (device_reset #) to the AND gate 616 .
- the engagement of the hot-plug device 400 with the connector 210 / 300 will cause a reset-pin signal (server_PERst #) to be provided to the AND gate 616 .
- the AND gate 616 Upon receiving those signals, the AND gate 616 will provide a signal that causes a de-assertion of the hot-plug device function/operation reset. As discussed above, due to decision block 604 , that de-assertion signal is only provided once the time period between time 3 and time 4 in FIG.
- the hot-plug device 400 is configured to hold itself in its device reset state until the reference clock has become stable, and in some embodiments may also hold itself in its device reset state until each of the communication lanes for the hot-plug device 400 have been detected.
- an NVMe storage device that is configured, when being hot-plugged to a connector, to hold itself in a reset state for a predetermined amount of time until a reference clock has reached a stable reference clock state. Following that predetermined amount of time and associated stability of the reference clock, the NVMe storage device then performs a device reset that results in the provisioning of a de-assertion signal on a reset pin included on the connector that ensures that the NVMe storage device is put into a deterministic state with associated state machines in their default state.
- the NVMe storage device may also be configured, when being hot-plugged to the connector, to hold itself in the reset state until each of the communication pins on the connector have been engaged in order to ensure that each communication lane on the NVMe storage device is utilized in its subsequent operation.
- securing subsystems on such NVMe storage device that include securing latches (and/or other features) that provide for sequential engagement of the NVMe storage device with pins on the connector do not cause issues associated with conventional hot-plug systems.
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Hardware Design (AREA)
- Stored Programmes (AREA)
Abstract
Description
- The present disclosure relates generally to information handling systems, and more particularly to the hot plugging of devices in an information handling system.
- As the value and use of information continues to increase, individuals and businesses seek additional ways to process and store information. One option available to users is information handling systems. An information handling system generally processes, compiles, stores, and/or communicates information or data for business, personal, or other purposes thereby allowing users to take advantage of the value of the information. Because technology and information handling needs and requirements vary between different users or applications, information handling systems may also vary regarding what information is handled, how the information is handled, how much information is processed, stored, or communicated, and how quickly and efficiently the information may be processed, stored, or communicated. The variations in information handling systems allow for information handling systems to be general or configured for a specific user or specific use such as financial transaction processing, airline reservations, enterprise data storage, or global communications. In addition, information handling systems may include a variety of hardware and software components that may be configured to process, store, and communicate information and may include one or more computer systems, data storage systems, and networking systems.
- Information handling systems sometime utilize devices that may be “hot-plugged”, “hot-swapped”, “hot-inserted”, and/or otherwise communicatively coupled to the system without shutting down or stopping the system. For example, server devices often utilize hot-plug devices for storage. The conventional hot-plugging of some single communication lane storage devices such as, for example, Serial Attached Small Computer System Interface (SCSI) (SAS) storage devices and Serial AT Attachment (SATA) storage drives, is relatively stable. However, the hot-plugging of multiple communication lane storage devices has been found to raise some issues. For example, Peripheral Component Interconnect (PCI) express (PCIe) Non-Volatile Memory storage devices (NVMe storage devices) experience several issues when utilizing conventional hot-plugging techniques.
- Such NVMe storage devices typically include securing latches that are rotatably coupled to one end of the front face of the NVMe storage device, with those securing latches also coupled to a securing subsystem on the NVMe storage device that is configured to engage the server chassis and/or connector to which the NVMe storage device is being connected. Conventionally, when the securing latch is rotated away from the front face of the NVMe storage device such that the securing subsystem is unactivated, the NVMe storage device may be positioned in the storage device housing that is defined by the server system and located adjacent a server system connector. The securing latch may then be rotated towards the front face of the NVMe storage device in order to activate the securing subsystem to engage the server system and/or the sever system connector and mate the storage device connector on the NVMe storage device with the server system connector in the server system.
- It has been found that the use of such securing latches to mate the storage device connector with the server system connector results in a sequential engagement of the pins on those connectors that can cause issues with the subsequent use of the NVMe storage device. For example, as the securing latch is rotated towards the front face of the NVMe storage device, the pins on the server system connector that are closest to the point of contact between the securing subsystem and the server system/server system connector (e.g., adjacent the fulcrum/rotatable coupling of the securing latch with the front face of the NVMe storage device) engage corresponding pins on the storage device connector first, followed sequentially by the pins that are further away from that point of contact. The server device connector includes four sets of communication lane pins that are distributed across its length, with the fourth communication lane pins located closest to that point of contact, and the first communication lane pins located further away from the point of contact between the securing subsystem and the server system/server system connector. Furthermore, the server system connector includes reset pin (e.g. a PERst pin) that is located approximately midway along the length of the server system connector, and a pair of reference clock pins that are located adjacent the end of the server system connector that is opposite the point of contact between the securing subsystem and the server system/server system connector.
- In one example of issues resulting from such device hot-plug systems, the engagement of the storage device connector with the server system connector results in engagement of the storage device connector with power pins on the server system connector, along with the sequential engagement of the fourth communication lane pins, followed by engagement of the third communication lane pins, followed by engagement of the second communication lane pins, and followed by engagement of the first communication lane pins. In conventional systems, the NVMe storage device is configured with a 70 millisecond reset time that starts when the NVME storage device receives power from the server system connector. However, in some embodiments, prior to all of the sets of communication lane pins engaging, the reset time for the NVMe storage device may expire, causing problems with the operation of the NVMe storage device.
- For example, in some situations the closing of the securing latch may cause the NVMe storage device to engage the fourth communication lane pins and the third communication lane pins, followed by engagement of the NVMe storage device with the reset pin after the expiration of the reset time for the NVMe storage device, which results in the reset of the NVMe storage device being driven by the engagement of the reset pin. Because the server system is in normal operating mode upon that engagement (i.e., the NVMe storage device is being hot-plugged to the server system), the reset pin is being de-asserted prior to the NVMe storage device engaging each of the second communication lane pins, the first communication lane pins, and the reference clock pins. This may cause NVMe storage devices that are configured to perform “lane reversal” to convert the fourth communication lane to a first communication lane (and the third communication lane to a second communication lane in some embodiments), and ignore any other communication lanes that would otherwise be enabled by the subsequent engagement of the first and second communication lane pins. As such, in this example, the NVMe storage device will not operate at its maximum throughput (i.e., the NVMe storage device is capable of communicating via four communication lanes, but only one or two communication lanes are enabled.) Furthermore, in NVMe storage devices that are not configured to perform “lane reversal”, the NVMe storage device may not enable any communication lanes if an unexpected communication lane is detected (e.g., the fourth communication lane is detected when the first communication lane is expected), causing the NVMe storage device to be completely unusable (e.g., having no communication lanes available.)
- In another example of issues resulting from such device hot-plug systems, the sequential engagement of the storage device connector with the server system connector results in the sequential engagement of the NVMe storage device with the reset pin, followed by engagement with the reference clock pins as discussed above. In some situations, the engagement of the NVMe storage device with the reset pin occurs prior to the reference clock (which is connected to the reference clock pins) reaching stability (e.g., 100 μs after engagement of the reference clock pins), and can cause the NVMe storage device to exit reset and enter its operational/functional state (i.e., the NVMe storage device will exit reset when the reset pin is engaged because the server system is not in reset.) The provisioning of the de-assertion signal on the reset pin while the reference clock is in the unstable state causes a violation (e.g., a Tperst-clk violation) of the PCIe electromechanical specification, and puts the NVMe storage device into a non-deterministic state due to the pre-reference-clock-stability de-assertion signal preventing its state machines from entering their default state (i.e., the state machines utilize the asynchronous de-assertion of the reset pin following reference clock stability to return to their default states.)
- Accordingly, it would be desirable to provide an improved device hot-plug system.
- According to one embodiment, an Information Handling System (IHS) includes a processing system; and a memory system that is coupled to the processing system and that includes instructions that, when executed by the processing system, cause the processing system to provide a hot-plug engine that is configured, in response to a hot-plug event with a connector, to: enter a reset state; monitor, while in the reset state, a reference clock; determine, while in the reset state, that the reference clock has been stable for a predetermined time period; exit, in response to determining that the reference clock has been stable for the predetermined time period, the reset state; and provide, in response to exiting the reset state, a de-assertion signal on a reset pin included on the connector.
-
FIG. 1 is a schematic view illustrating an embodiment of an information handling system. -
FIG. 2 is a schematic view illustrating an embodiment of a server system. -
FIG. 3 is a schematic view illustrating an embodiment of a connector that may be provided in the server system ofFIG. 2 . -
FIG. 4 is a schematic view illustrating an embodiment of a hot-plug device. -
FIG. 5 is a schematic view illustrating an embodiment of registers that may be provided in the hot-plug device ofFIG. 4 . -
FIG. 6 is a schematic view illustrating an embodiment of a circuit that may be provided in the hot-plug device ofFIG. 4 . -
FIG. 7 is a flow chart illustrating an embodiment of a method for hot-plugging a device. -
FIG. 8A is a perspective view illustrating an embodiment of the hot-plug device ofFIG. 4 being coupled to the server system ofFIG. 2 . -
FIG. 8B is a schematic view illustrating an embodiment of the hot-plug device ofFIG. 4 being coupled to the server system ofFIG. 2 . -
FIG. 8C is a schematic view illustrating an embodiment of the registers ofFIG. 5 with particular values during the method ofFIG. 7 . -
FIG. 8D is a chart illustrating an embodiment of signal generated in response to the hot-plugging of the hot-plug device ofFIG. 4 to the server system ofFIG. 2 . - For purposes of this disclosure, an information handling system may include any instrumentality or aggregate of instrumentalities operable to compute, calculate, determine, classify, process, transmit, receive, retrieve, originate, switch, store, display, communicate, manifest, detect, record, reproduce, handle, or utilize any form of information, intelligence, or data for business, scientific, control, or other purposes. For example, an information handling system may be a personal computer (e.g., desktop or laptop), tablet computer, mobile device (e.g., personal digital assistant (PDA) or smart phone), server (e.g., blade server or rack server), a network storage device, or any other suitable device and may vary in size, shape, performance, functionality, and price. The information handling system may include random access memory (RAM), one or more processing resources such as a central processing unit (CPU) or hardware or software control logic, ROM, and/or other types of nonvolatile memory. Additional components of the information handling system may include one or more disk drives, one or more network ports for communicating with external devices as well as various input and output (I/O) devices, such as a keyboard, a mouse, touchscreen and/or a video display. The information handling system may also include one or more buses operable to transmit communications between the various hardware components.
- In one embodiment, IHS 100,
FIG. 1 , includes aprocessor 102, which is connected to abus 104.Bus 104 serves as a connection betweenprocessor 102 and other components of IHS 100. Aninput device 106 is coupled toprocessor 102 to provide input toprocessor 102. Examples of input devices may include keyboards, touchscreens, pointing devices such as mice, trackballs, and trackpads, and/or a variety of other input devices known in the art. Programs and data are stored on amass storage device 108, which is coupled toprocessor 102. Examples of mass storage devices may include hard discs, optical disks, magneto-optical discs, solid-state storage devices, and/or a variety other mass storage devices known in the art. IHS 100 further includes adisplay 110, which is coupled toprocessor 102 by avideo controller 112. Asystem memory 114 is coupled toprocessor 102 to provide the processor with fast storage to facilitate execution of computer programs byprocessor 102. Examples of system memory may include random access memory (RAM) devices such as dynamic RAM (DRAM), synchronous DRAM (SDRAM), solid state memory devices, and/or a variety of other memory devices known in the art. In an embodiment, achassis 116 houses some or all of the components ofIHS 100. It should be understood that other buses and intermediate circuits can be deployed between the components described above andprocessor 102 to facilitate interconnection between the components and theprocessor 102. - Referring now to
FIG. 2 , an embodiment of aserver system 200 is illustrated. In an embodiment, theserver system 200 may be provided by theIHS 100 discussed above with reference toFIG. 1 , and/or may include some or all of the components of theIHS 100. In the illustrated embodiment, theserver system 200 includes achassis 202 that houses the components of theserver system 200, only some of which are illustrated inFIG. 2 . For example, thechassis 202 may house acircuit board 204 such as, for example, a motherboard and/or other boards that would be apparent to one of skill in the art in possession of the present disclosure. In the illustrated embodiment, thechassis 202 may also house a processing system (not illustrated, but which may include theprocessor 102 discussed above with reference toFIG. 1 and may be mounted to the circuit board 204), and a memory system (not illustrated, but which may include thememory 114 discussed above with reference toFIG. 1 and may be mounted to the circuit board 204) that is coupled to the processing system and that includes instruction that, when executed by the processing system, cause the processing system to provide aserver engine 206 that is configured to perform the functions of the server engines and server systems discussed below. - In some embodiments, an
optional reference clock 208 may be housed in thechassis 208. For example,FIG. 2 illustrates theoptional reference clock 208 that is included as part of theserver engine 206 and that is coupled to thecircuit board 204, although other locations for a server-system-reference-clock will fall within the scope of the present disclosure as well. Aconnector 210 is mounted to thecircuit board 204 and coupled to the server engine 206 (e.g., via a trace included on thecircuit board 204.) In the examples below, theconnector 210 is described as being a female Peripheral Component Interconnect express (PCIe) Non-Volatile Memory (NVMe) connector having afirst end 210 a and asecond end 210 b that is located opposite theconnector 210 from thefirst end 210 a, although other multi-communication lane connectors may benefit from the teachings of the present disclosure and thus are envisioned as falling within its scope as well. In the illustrated embodiment, thechassis 202 houses adevice housing 212 that extends into thechassis 202 adjacent theconnector 210 and that is configured to house the hot-plug device of the present disclosure. While aspecific server system 200 has been illustrated and described for purposes of discussion herein, one of skill in the art in possession of the present disclosure will recognize that a variety of systems may include a variety of components, features, and/or configurations for providing conventional functionality, as well as for utilizing the hot-plug device of the present disclosure, while remaining within the scope of the present disclosure. - Referring now to
FIG. 3 , an embodiment of aconnector 300 is illustrated that may be theconnector 210 discussed above with reference toFIG. 2 . As such, theconnector 300 is illustrated and described as a female NVMe connector, although other multi-communication lane connectors may benefit from the teachings of the present disclosure and thus are envisioned as falling within its scope as well. Theconnector 300 includes a base 302 having afirst end 302 a that corresponds to thefirst end 210 a of theconnector 210 in the examples below, and asecond end 302 b that is located opposite the base 302 from thefirst end 302 a and that corresponds to thesecond end 210 b of theconnector 210 in the examples below. - A plurality of pins may be provided on the
connector 300. In the illustrated embodiment, a first set of communication lane pins 304 a-304 e are included on theconnector 300 adjacent thesecond end 302 b of thebase 302, and may include anePCIe lane 0 positive transmitpin 304 a (“PETp0”, pin # E10), anePCIe lane 0 negative transmitpin 304 b (“PETn0”, pin # E11), aground pin 304 c (“GND”, pin # E12), anePCIe lane 0 negative receivepin 304 d (“PERn0”, pin # E13), and anePCIe lane 0 positive receivepin 304 e (“PERp0”, pin # E14). Furthermore, a second set of communication lane pins 306 a-306 e are included on theconnector 300 between the first set of communication lane pins 304 a-e and thefirst end 302 a of thebase 302, and may include anePCIe lane 1 positive transmitpin 306 a (“PETp1”, pin # E19), anePCIe lane 1 negative transmitpin 306 b (“PETn1”, pin # E20), aground pin 306 c (“GND”, pin # E21), anePCIe lane 1 negative receivepin 306 d (“PERn1”, pin # E22), and anePCIe lane 1 positive receivepin 306 e (“PERp1”, pin # E23). Further still, a third set of communication lane pins 308 a-308 e are included on theconnector 300 between the second set of communication lane pins 306 a-e and thefirst end 302 a of thebase 302, and may include anePCIe lane 2 positive transmitpin 308 a (“PETp2”, pin # E25), anePCIe lane 2 negative transmitpin 308 b (“PETn2”, pin # E26), aground pin 308 c (“GND”, pin # E27), anePCIe lane 2 negative receivepin 308 d (“PERn2”, pin # E28), and anePCIe lane 2 positive receive pin 308 e (“PERp2”, pin # E29). Finally, a fourth set of communication lane pins 310 a-310 e are included on theconnector 300 between the third set of communication lane pins 308 a-e and thefirst end 302 a of thebase 302, and may include anePCIe lane 3 positive transmitpin 310 a (“PETp3”, pin # E31), anePCIe lane 3 negative transmitpin 310 b (“PETn3”, pin # E32), aground pin 310 c (“GND”, pin # E33), anePCIe lane 3 negative receivepin 310 d (“PERn3”, pin # E34), and anePCIe lane 3 positive receivepin 310 e (“PERp3”, pin # E35). - In the some embodiments, a fifth set of communication lane pins 311 a-311 e are included on the
connector 300 adjacent thesecond end 302 b of thebase 302 and opposite the base from the first set of communication lane pins 304 a-e, and may include a SAS/SATA/SATAe lane 0 positive transmitpin 311 a (“SOT+(A+)”, pin # S2), a SAS/SATA/SATAe lane 0 negative transmitpin 311 b (“SOT−(A−)”, pin # S3), aground pin 311 c (“GND”, pin # S4), a SAS/SATA/SATAe lane 0 negative receivepin 311 d (“SOR−(B−)”, pin # S5), and a SAS/SATA/SATAe lane 0 positive receivepin 311 e (“SOR+(B+)”, pin # S6). As discussed in the examples provided below, hot-plug devices such as x4 NVMe storage devices may enable the use of four communication lanes on theconnector 300 that are provided by the first, second, third, and fourth set of communication lane pins discussed above, while hot-plug devices such as x1 SAS/SATA storage devices may enable the use of a single communication lane on theconnector 300 that is provided by the fifth set of communication lane pins discussed above. - Furthermore, a plurality of reference clock pins 312 a and 312 b are included on the
connector 300 between the first set of communication lane pins 304 ae and thesecond side 302 b of thebase 302, and may include an ePCIe positivereference clock pin 312 a (“RefClk0+”, pin # E7) and an ePCIe negativereference clock pin 312 b (“RefClk0−”, pin # E8). One of skill in the art in possession of the present disclosure will recognize that the reference clock pins 312 a and 312 b on theconnector 300 may be coupled to thereference clock 208 included in the server system 200 (e.g., via the traces between theserver engine 206 and theconnector 210 inFIG. 2 ) in situations where theoptional reference clock 208 is included in theserver system 200, and may be omitted or otherwise not utilized by the systems and methods of the present disclosure when the reference clock is included in the hot-plug device of the present disclosure (discussed in further detail below.) Theconnector 300 also include areset pin 314 a that located approximately midway along the length of theconnector 300 between the fifth set of communication lane pins 311 a-e and thefirst side 302 a of thebase 302, and may include an ePCIe portA reset pin 314 a (“ePERst0#”, pin # E5). Theconnector 300 also include a plurality of power pins 315 a, 315 b, and 315 c, each of which are located adjacent thefirst side 302 a of thebase 302, and that may each include ePCIe SSD power pins (pins # P13, P14, and P15). While a specific connector has been illustrated and described, one of skill in the art in possession of the present disclosure will recognize that a variety of connectors utilized in hot-plug-enabled systems may benefit from the teachings of the present disclosure and thus fall within its scope as well. - Referring now to
FIG. 4 , an embodiment of a hot-plug device 400 is illustrated. In some embodiment, the hot-plug device 400 may be provided by theIHS 100 discussed above with reference toFIG. 1 , and/or may include some or all of the components of theIHS 100. In the examples below, the hot-plug device 400 is an NVMe device (and specifically, an NVMe storage device), but one of skill in the art in possession of the present disclosure will recognize that a variety of other hot plug devices may benefit from the teachings of the present disclosure and thus will fall within its scope as well. In the illustrated embodiment, the hot-plug device 400 includes achassis 402 that houses the components of the hot plug-device 400, only some of which are illustrated inFIG. 4 . For example, thechassis 402 may house a processing system (not illustrated, but which may include theprocessor 102 discussed above with reference toFIG. 1 ), and a memory system (not illustrated, but which may include thememory 114 discussed above with reference toFIG. 1 ) that is coupled to the processing system and that includes instructions that, when executed by the processing system, cause the processing system to provide a hot-plug engine 404 that is configured to perform the functions of the hot-plug engines and hot-plug devices discussed below. In a specific example, the hot-plug engine 404 may be provided by an Application Specific Integrated Circuit (ASIC) in the hot-plug device 400 (e.g., the storage device controller ASIC in an NVMe storage device), although other components (e.g., firmware, CPU/memory combinations, etc.) that may be configured to provide the functionality of the hot-plug engine 404 discussed below are envisioned as falling within the scope of the present disclosure as well. - In the illustrated embodiment, a plurality of
registers 406 are housed in thechassis 402. For example,FIG. 4 illustrates the registers included as part of the server engine 206 (e.g., as registers included in the ASIC discussed above), although other locations for the registers will fall within the scope of the present disclosure as well. In some embodiments, anoptional reference clock 408 may be housed in thechassis 402, andFIG. 4 illustrates theoptional reference clock 408 included as part of the hot-plug engine 404, although other locations for a hot-plug-device-reference-clock will fall within the scope of the present disclosure as well. For example, thereference clock 408 may be provided in the hot-plug device 400 in possible future implementations of NVMe storage devices that provide Separate Reference clocks with Independent Spread Spectrum Clocking (SSC) (SRIS). As would be appreciated by one of skill in the art in possession of the present disclosure, SRIS implementations may omit thereference clock 208 from theserver system 200 and the reference clock pins 312 a and 312 b on theconnector 300, and instead provide that any two ends of a communication link (e.g., including the hot-plug device 400) include a separate reference clock (e.g., the reference clock 408). Aconnector 410 is included on thechassis 402 and is coupled to the hot-plug engine 404. In the examples below, theconnector 410 is described as being a male Peripheral Component Interconnect express (PCIe) Non-Volatile Memory (NVMe) connector having afirst end 410 a and asecond end 410 b that is located opposite theconnector 410 from thefirst end 410 a, although other multi-communication lane connectors may benefit from the teachings of the present disclosure and thus are envisioned as falling within its scope as well. In the examples below, while not illustrated in detail, one of skill in the art in possession of the present disclosure will appreciate that theconnector 410 includes pins that are configured to engage corresponding pins on the connector 300 (e.g., the communication lane pins, reference clock pins, and reset pin discussed with reference toFIG. 3 ). - In the illustrated embodiment, the
chassis 402 include a securing subsystem having a securinglatch 412. In the illustrated embodiment, the securinglatch 412 includes a securingsubsystem activation arm 412 a that is rotatably coupled to thechassis 402 via arotatable coupling 412 b that allows the securingsubsystem activation arm 412 a to move between a securing orientation in which the securingsubsystem activation arm 412 a is located immediately adjacent the chassis 402 (illustrated in solid lines inFIG. 4 ), and a deactivated orientation in which the securingsubsystem activation arm 412 a is rotated away from the chassis 402 (illustrated in dashed lines inFIG. 4 ). While not illustrated, one of skill in the art in possession of the present disclosure will recognize that the securinglatch 412 may be coupled to other components of the securing subsystem that are located on thechassis 402 and that are configured to engage thechassis 202 of theserver system 200 and/or theconnector 210/300 to couple the hot-plug device 400 theserver system 200, discussed in further detail below. While a hot-plug device 400 has been illustrated and described for purposes of discussion herein, one of skill in the art in possession of the present disclosure will recognize that hot-plug devices may include a variety of components, features, and/or configurations for providing conventional hot-plug device functionality, as well as the functionality discussed below, while remaining within the scope of the present disclosure. - Referring now to
FIG. 5 , an embodiment of a plurality ofregisters 500 are illustrated that may be theregisters 406 discussed above with reference toFIG. 4 . As such, in some embodiments, theregisters 500 may be provided in an ASIC included in the hot-plug device 400. Theregisters 500 include a referenceclock implementation register 502 that may be configured to indicate how a reference clock is implemented in the device hot-plug system of the present disclosure. In the examples below, the referenceclock implementation register 502 may include a value of “0” to indicate the system reference clock is implemented in the device hot-plug system by thereference clock 208 included in the server system 200 (i.e., a reference clock that is external to the hot-plug device 400), and the referenceclock implementation register 502 may include a value of “1” to indicate that the system reference clock is implemented in the device hot-plug system by thereference clock 408 included in the hot-plug device 400 (i.e., a reference clock that is internal to the hot-plug device 400). Theregisters 500 also include a clock counter value register 504 that may be configured to indicate an amount of time the hot-plug device 400 should remain in a device reset state (e.g., “100” in the example below), discussed in further detail below. Theregisters 500 also include a clock countertime unit register 506 that may be configured to indicate time units for the amount of time that the hot-plug device 400 should remain in the device reset state (e.g., “μs” in the example below), discussed in further detail below. While a specific examples of registers have been illustrated and described, one of skill In the art in possession of the present disclosure will recognize that the information included in theregisters 500, and/or the functionality resulting from the use of that information, may be provided in a variety of manners that will fall within the scope of the present disclosure as well. - Referring now to
FIG. 6 , aschematic circuit 600 is illustrated that may be implemented by the hot-plug engine 404. In the illustrated embodiment, thecircuit 600 includes amultiplexer 602 that receives inputs from the referenceclock implementation register 502, and may receive inputs from either of the reference clocks 208 and 408 (when present in the hot-plug system). Themultiplexer 602 outputs to adecision block 604 that receives inputs from the clockcounter value register 504 and the clock countertime unit register 506, and that provides for the determination of whether a counter has expired according to those registers. Thecircuit 600 also includes decision blocks 606, 608, 610, and 611 that each provide for the determination of whether a respective communication lane has been detected. An ANDgate 612 a receives the YES outputs of each of the decision blocks 610 and 611, and will output a signal when each of those decision blocks have detected their respective communication lanes. An ANDgate 612 b receives the output of the ANDgate 612 a, and the YES output ofdecision block 608, and will output a signal whendecision block 608 detects its respective communication lane, and the ANDgate 612 a indicates each of the decision blocks 610 and 611 have detected their respective communication lanes Amultiplexer 612 c receives inputs fromdecision block 608, ANDgate 612 b, and aregister 613 in the hot-plug device 400 that indicates the number of communication lanes available for the hot-plug device 400 (e.g., four communications lanes in this example.) Themultiplexer 612 c outputs to an ANDgate 612 d that receives an input fromdecision block 606, and the ANDgate 612 d will output a signal when thedecision block 606 detects its respective communication lane, and themultiplexer 612 c indicates that the expected number of communication lanes have been detected. An ANDgate 614 receives the YES output of thedecision block 604, and the output of the ANDgate 612 d, and will output a device reset signal to an ANDgate 616 when thedecision block 604 determines that that a counter has expired according to theregisters gate 612 d. The ANDgate 616 also receives an input from the reset pin on theconnector 210/300 (e.g., “PERst #”, pin # E5), and will output the reset signal used to reset the hot-plug device in response to receiving the device reset signal from the ANDgate 614 and detecting a connection to the reset pin on theconnector 210/300. While a specific circuit has been illustrated and described, one of skill in the art in possession of the present disclosure will recognize that the functionality of the hot-plug engine 404 may be implemented in a variety of manners that will fall within the scope of the present disclosure as well - Referring now to
FIG. 7 , an embodiment of amethod 700 for hot-plugging a device is illustrated. As discussed herein, the systems and methods of the present disclosure provide a hot-plug device such as, for example, an NVMe storage device, that is configured, when being hot-plugged to a connector, to hold itself in a device reset state for a predetermined amount of time that is designated in register(s) included in the hot-plug device and that is known to allow a reference clock to reach a stable reference clock state. Following that predetermined amount of time and associated stability of the reference clock, the hot-plug device then performs a device reset that results in the provisioning of a de-assertion signal on a reset pin included on the connector that ensures that the hot-plug device is put into a deterministic state with associated state machines in their default state. In addition, the hot-plug device may also be configured, when being hot-plugged to the connector, to hold itself in the device reset state until each of the communication pins on the connector have been engaged in order to ensure that each communication lane on the hot-plug device is utilized in its subsequent operation. As such, securing subsystems on such hot-plug devices that include securing latches (and/or other features) that provide for sequential engagement of the hot-plug device with pins on the connector do not cause issues associated with conventional hot-plug systems such as, for example, the communication lane issues and reference clock instability/non-deterministic device state issues discussed above. - The
method 700 begins atblock 702 where a hot-plug device is hot-plugged to a connector and enters a device reset state. In an embodiment, atblock 702, the hot-plug device ofFIG. 4 is hot-plugged to theconnector 210/300 in theserver system 200. For example, with reference toFIG. 8A , the hot-plug device 400 is positioned adjacent theserver system 200 such thatconnector 410 on the hot-plug device 400 is located adjacent thedevice housing 212 on theserver system 200, with the securinglatch 412 in the deactivated orientation (illustrated by the dashed lines inFIG. 4 ). The hot-plug device 400 is then moved in a direction A such that the hot-plug device 400 enters thedevice housing 212 and moves through thedevice housing 212 until theconnector 410 on the hot-plug device 400 is located immediately adjacent theconnector 210/300 in theserver system 200, as illustrated inFIG. 8B . The hot-plugging of the hot-plug device 400 to theconnector 210/300 on theserver system 200 occurs when the securinglatch 412 is moved from the deactivated orientation in which the securingsubsystem activation arm 412 a is rotated away from thechassis 402 of the hot-plug device 400 (illustrated by the dashed lines inFIG. 8B ) to the securing orientation in which the securingsubsystem activation arm 412 a is positioned immediately adjacent thechassis 402 of the hot-plug device 400 (illustrated by the solid lines inFIG. 8B ). Positioning of the securinglatch 412 in the securing orientation activates the securing subsystem on the hot-plug device 400 by, for example, causing that securing subsystem to engage thechassis 202 of theserver system 200 and/or theconnector 210/300 to mate theconnector 410 on the hot-plug device 400 with theconnector 210/300 in theserver system 200, while securing the hot-plug device 400 in thechassis 202 of theserver system 200. - In an embodiment, “hot-plugging”, “hot-swapping”, “hot-inserting”, and/or otherwise communicatively coupling the hot-
plug device 400 to theserver system 200 without shutting down or stopping theserver system 200 may be accomplished in a variety of manners. For example, theserver engine 206 may include an operating system that is in a runtime state and that is controlling the operation of theserver system 200 when the hot-plug device 400 is connected to theconnector 210/300 in order to provide a hot-plug event. However, in some embodiments, theserver engine 206 may include a Basic Input Output System (BIOS) that has been currently performing a boot of theserver system 200 when the hot-plug device 400 is connected to theconnector 210/300 in order to provide the hot-plug event. While a few examples have been provided, one of skill in the art in possession of the present disclosure will recognize that hot-plug events may occur and/or be performed in a variety of manners that will fall within the scope of the present disclosure. - As discussed above, the use of the securing
latch 412 and securing subsystem on the hot-plug device 400 to mate theconnector 410 on the hot-plug device 400 to theconnector 210/300 causes a sequential engagement of theconnector 410 on the hot-plug device 400 with the pins on theconnector 210/300 in theserver system 200. As can be seen inFIG. 8B , with the hot-plug device 400 positioned in thedevice housing 212 in thechassis 202 of theserver system 200, thefirst end 410 a of theconnector 410 is located adjacent thefirst end 210 a/302 a of theconnector 210/300, and thesecond end 410 b of theconnector 410 is located adjacent thesecond end 210 b/302 b of theconnector 210/300. The movement of the securinglatch 412 to the securing orientation cause the securing subsystem on the hot-plug device 400 to engage the engage thechassis 202 of theserver system 200 and/or theconnector 210/300 at a point of contact that is adjacent or on thefirst end 210 a/302 a of theconnector 210/300 such that theconnector 410 on the hot-plug device 400 first engages the pins on theconnector 210/300 adjacent itsfirst end 210 a/302 a, and then engages adjacent pins along the length of theconnector 210/300 in sequence until the pins on theconnector 210/300 adjacent itssecond end 210 b/302 b are engaged. - For example, with reference to
FIG. 3 , the sequential engagement of theconnector 410 on the hot-plug device 400 with theconnector 300 in this manner results in engagement of the power pins 315 a-c, along with the sequential engagement of the fourth set of communication lane pins 310 a-310 e, followed by engagement of the third set of communication lane pins 318 a-308 e, followed by engagement of the second set of communication lane pins 306 a-306 e, and followed by engagement of the first set of communication lane pins 304 a-304 e. As discussed above for conventional NVMe storage device hot-plug systems, the engagement of the power pins 315 a-c may start a reset time in the NVMe storage device and, prior to all of the sets of communication lane pins engaging, the reset time for the NVMe storage device may expire, causing problems with the operation of the NVMe storage device due to the reset of the NVMe storage device being driven by the engagement of the reset pin. Those problems may include the NVMe storage device performing a “lane reversal” to convert the a subset of the communication lanes (provided by the communication lane pins) to active communication lane(s) (and ignoring any other communication lanes that are enabled by subsequent engagement of the other sets of communication lane pins) and preventing the NVMe storage device from operating at its maximum throughput, or not enabling any communication lanes due to the detection of an unexpected communication lane and making the NVMe storage device unusable. - As also discussed above for conventional NVMe storage device hot-plug systems, the sequential engagement of the
connector 410 on the hot-plug device 400 with theconnector 300 in the manner discussed above may result in the sequential engagement of thereset pin 314 a followed by engagement of the reference clock pins 312 a and 312 b. However, prior to the reference clock (which is connected to the reference clock pins) reaching stability, the NVMe storage device may reset and provide a de-assertion signal on thereset pin 314 a, and the provisioning of the de-assertion signal on thereset pin 314 a while the reference clock is in the unstable state causes a violation (e.g., a Tperst-clk violation) of the PCIe electromechanical specification, and puts the NVMe storage device into a non-deterministic state due to the pre-reference-clock-stability de-assertion signal preventing its state machines from entering their default state. As discussed in further detail below, the device hot-plug system of the present disclosure may operate to prevent these issues by having the hot-plug device “hold” itself in its device reset state until the reference clock is stable for a predetermined time period and, in some embodiments, “hold” itself in its device reset state until each of the communication lanes have been activated. - The
method 700 then proceeds to decision block 704 where it is determined whether a reference clock has been stable for a predetermined amount of time. In server systems that include thereference clock 208 coupled to the reference clock pins 312 a and 312 b on theconnector 210/300, atblock 702 and during the hot-plugging of the hot-plug device 400 to theconnector 210/300 on theserver system 200, theconnector 410 on the hot-plug device 400 will engage thereset pin 314 a, followed by its engagement with the reference clocks pins 312 a and 312 b. The engagement of theconnector 410 on the hot-plug device 400 with the reference clocks pins 312 a and 312 b allows the hot-plug engine 404 to begin monitoring thereference clock 208 atdecision block 704. For example, atdecision block 704, the hot-plug engine 404 may access theregisters 406/500 and retrieve the values included therein to determine whether thereference clock 208 has been stable for the predetermined amount of time. With reference toFIG. 8C , an example of theregisters 500 is illustrated with the referenceclock implementation register 502 including a “0” to indicate that thereference clock 208 is included in theserver system 200, and the clock counter value register 504 including a “100” and the clock countertime unit register 506 including a “μs” to indicate that the predetermined amount of time is 100 microseconds. With reference toFIG. 6 , the referenceclock implementation register 502 value of “0” may cause the hot-plug engine 404 to pass the signal fromreference clock 208 to thedecision block 604, and the clockcounter value register 504 and clock countertime unit register 506 values of “100” and “μs” may cause the hot-plug engine 404 to monitor the signal from thereference clock 208 for 100 microseconds atdecision block 604. For example, atdecision block 704, a counter in the hot-plug engine 404 may be incremented based on the reference clock 208 (accessed via the engaged reference clock pins 312 a and 312 b) until that counter exceeds 100 microseconds. - With reference to
FIG. 8D , an embodiment of the hot-plugging of a hot-plug device (an NVMe storage device in this example) to a server system is illustrated. In the illustrated embodiment, a reset pin signal 802 (e.g., “PERST #”) and a reference clock signal 804 (e.g., “REFCLK”), as well as a 3.3 Vaux signal, a 3.3/12 volt signal, a PCI express link signal, a system management bus (SMBus) signal, and a Joint Test Action Group (JTAG) signal, provided during the hot-plug event are illustrated. As can be seen, attime 0, thereset pin signal 802 is asserted (which, in the illustrated example, causes the signal to go “low”), and attime 1, the 3.3/12 volt signal begins ramping up while the SMBus signal goes active. Attime 2, the 3.3/12 volt signal is stable, and thereference clock signal 804 begins but is unstable. Attime 3, thereference clock signal 804 becomes stable, and at time 4, thereset pin signal 802 is de-asserted. Attime 5, the PCI express link signal is active, and attime 6, the JTAG signal is active. As would be understood by one of skill in the art, the time period betweentime 3 and time 4 corresponds to the time period between the time at which the reference clock signal becomes stable, and the time at which thereset pin signal 802 should be de-asserted (as de-asserting thereset pin signal 802 betweentime 2 andtime 3 while thereference clock signal 804 is unstable can cause the issues discussed above). As such, using the example provided above, the time period betweentime 3 and time 4 may be set at 100 microseconds to ensure that thereference clock signal 804 is stable before de-asserting thereset pin signal 802. - In hot-plug devices that include the reference clock 408 (e.g., when the
reference clock 208 in theserver system 200 and the reference clock pins 312 a and 312 b on theconnector 210/300 may be omitted), atblock 702 and during the hot-plugging of the hot-plug device 400 to theconnector 210/300 on theserver system 200, theconnector 410 on the hot-plug device 400 will engage thereset pin 314 a. Atdecision block 704, the hot-plug engine 404 may begin monitoring thereference clock 408. For example, atdecision block 704, the hot-plug engine 404 may access theregisters 406/500 and retrieve the values included therein to determine whether thereference clock 408 has been stable for the predetermined amount of time. In such an example, the referenceclock implementation register 502 may include a “1” to indicate that thereference clock 408 is included in the hot-plug device 400, with the clock counter value register 504 including a “100” and the clock countertime unit register 506 including a “μs” to indicate that the predetermined amount of time is 100 microseconds. With reference toFIG. 6 , the referenceclock implementation register 502 value of “1” may cause the hot-plug engine 404 to pass the signal fromreference clock 408 to thedecision block 604, and the clockcounter value register 504 and clock countertime unit register 506 values of “100” and “μs” may cause the hot-plug engine 404 to monitor the signal from thereference clock 408 for 100 microseconds. For example, atdecision block 704, a counter in the hot-plug engine 404 may be incremented based on thereference clock 408 until that counter exceeds 100 microseconds. - If at
decision block 704 it is determined that the reference clock has not been stable for the predetermined amount of time, themethod 700 returns to decision block 704 and continues to loop throughdecision block 704 until the reference clock has been stable for the predetermined amount of time (e.g., that counter based on the reference clock reaches 100 microseconds). When it is determined atblock 704 that the reference clock has been stable for the predetermined amount of time (and thedecision block 604 inFIG. 6 passes a signal to the AND gate 614), themethod 700 may proceed tooptional decision block 706 where it is determined whether each communication lane pin on the connector has been engaged. In an embodiment, atblock 702 and during the hot-plugging of the hot-plug device 400 to theconnector 210/300 on theserver system 200, theconnector 410 on the hot-plug device 400 will first engage the fourth set of communication lane pins 310 a-310 e, followed by engagement of the third set of communication lane pins 318 a-308 e, followed by engagement of the second set of communication lane pins 306 a-306 e, and followed by engagement of the first set of communication lane pins 304 a-304 e. - With reference to
FIG. 6 , the hot-plug engine 404 will monitor each of the sets of communications lane pins via the decision blocks 606, 608, 610, and 611. As can be seen, the decision blocks 606, 608, 610, and 611 operate to determine whether each of the first, second, third, and fourth communication lanes (i.e., provided by respective sets of communication lane pins), respectively, have been detected and, if so, pass a respective signal to the ANDgates multiplexer 612 c. As will be appreciated by one of skill in the art, the ANDgate 612 d may be configured to only pass a signal to the ANDgate 614 when each of the communication lanes have been detected via the decision blocks 606, 608, 610, and 611, and the detected number of communication lanes corresponds to an expected number of communication lanes in theregister 613. In embodiments of hot-plug devices with a single communication lane (e.g., SAS/SATA/SATAe storage devices), atblock 706 the hot-plug engine 404 will monitor the set of communications lane pins via asingle decision block 606, and thatdecision block 606 will operate to determine whether the fifth communication lane (i.e., provided by respective sets of communication lane pins 311 a-e) has been detected (and is the only communication lane that is expected according to the register 613) and, if so, pass a respective signal to the ANDgate 612 d. - If at
decision block 706 it is determined that each communication lane pin on the connector has not been engaged, themethod 700 returns to decision block 706 and continues to loop throughdecision block 706 until each communication lane pin on the connector has been engaged. When it is determined atblock 706 that each communication lane pin on the connector has been engaged, or following a determination that the reference clock has been stable for the predetermined amount of time (e.g., whenoptional decision block 706 is omitted), themethod 700 proceeds to block 708 where the hot-plug device exits the device reset state. While decision blocks 704 and 706 have been illustrated and described as being performed separately withdecision block 704 being performed beforedecision block 706, those decision blocks may be performed simultaneously or in a different order while remaining within the scope of the present disclosure. For example, in embodiments in which the reference clock is included in theserver system 200,decision block 704 may be performed (e.g., a determination that reference clock is stable) prior to decision block 706 being performed (e.g., a determination that all communication lanes have been detected). In another example, in embodiments in which the reference clock is included in the hot-plug device 400,decision block 706 may be performed (e.g., a determination that all communication lanes have been detected) prior to decision block 704 being performed (e.g., a determination that reference clock is stable.) - In an embodiment, at
block 708, the hot-plug engine 404 causes the hot-plug device 400 to exit its device reset state. For example, upon thedecision block 604 passing its signal to the ANDgate 614, and the ANDgate 612 d passing its signal to the ANDgate 614, the hot-plug engine 404 may be configured to cause the hot-plug device 400 to exit its device reset state (e.g., the “device_reset #) output of the AND gate 614). Themethod 700 then proceeds to block 710 where the hot-plug device enters an operational/functional state by providing a de-assertion of the hot-plug device function/operation reset, taking the hot-plug device out of the reset state and into the operational/functional state. In an embodiment, at block 710, thedecision block 604 passing its signal to the ANDgate 614, and the ANDgate 612 d passing its signal to the ANDgate 614, causes the ANDgate 614 to provide a signal (device_reset #) to the ANDgate 616. In addition, the engagement of the hot-plug device 400 with theconnector 210/300 will cause a reset-pin signal (server_PERst #) to be provided to the ANDgate 616. Upon receiving those signals, the ANDgate 616 will provide a signal that causes a de-assertion of the hot-plug device function/operation reset. As discussed above, due to decision block 604, that de-assertion signal is only provided once the time period betweentime 3 and time 4 inFIG. 8D (which corresponds to the time designated by the clockcounter value register 504 and the clock counter time unit register 506) has elapsed. As such, the hot-plug device 400 is configured to hold itself in its device reset state until the reference clock has become stable, and in some embodiments may also hold itself in its device reset state until each of the communication lanes for the hot-plug device 400 have been detected. - Thus, systems and methods have been described that provide for an NVMe storage device that is configured, when being hot-plugged to a connector, to hold itself in a reset state for a predetermined amount of time until a reference clock has reached a stable reference clock state. Following that predetermined amount of time and associated stability of the reference clock, the NVMe storage device then performs a device reset that results in the provisioning of a de-assertion signal on a reset pin included on the connector that ensures that the NVMe storage device is put into a deterministic state with associated state machines in their default state. In addition, the NVMe storage device may also be configured, when being hot-plugged to the connector, to hold itself in the reset state until each of the communication pins on the connector have been engaged in order to ensure that each communication lane on the NVMe storage device is utilized in its subsequent operation. As such, securing subsystems on such NVMe storage device that include securing latches (and/or other features) that provide for sequential engagement of the NVMe storage device with pins on the connector do not cause issues associated with conventional hot-plug systems.
- Although illustrative embodiments have been shown and described, a wide range of modification, change and substitution is contemplated in the foregoing disclosure and in some instances, some features of the embodiments may be employed without a corresponding use of other features. Accordingly, it is appropriate that the appended claims be construed broadly and in a manner consistent with the scope of the embodiments disclosed herein.
Claims (20)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US15/953,208 US10437751B1 (en) | 2018-04-13 | 2018-04-13 | Device hot plug system |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US15/953,208 US10437751B1 (en) | 2018-04-13 | 2018-04-13 | Device hot plug system |
Publications (2)
Publication Number | Publication Date |
---|---|
US10437751B1 US10437751B1 (en) | 2019-10-08 |
US20190317908A1 true US20190317908A1 (en) | 2019-10-17 |
Family
ID=68101984
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US15/953,208 Active US10437751B1 (en) | 2018-04-13 | 2018-04-13 | Device hot plug system |
Country Status (1)
Country | Link |
---|---|
US (1) | US10437751B1 (en) |
Families Citing this family (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
TWI687790B (en) * | 2018-06-25 | 2020-03-11 | 緯創資通股份有限公司 | Electronic system capable of detecting number of times for hot plugs |
Family Cites Families (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
KR100259841B1 (en) * | 1997-07-31 | 2000-06-15 | 윤종용 | A hot plug of pci bus using single chip |
US6286066B1 (en) * | 1998-12-15 | 2001-09-04 | Dell U.S.A., L.P. | Hot-plug interface for detecting adapter card insertion and removal |
US7493438B2 (en) * | 2001-10-03 | 2009-02-17 | Intel Corporation | Apparatus and method for enumeration of processors during hot-plug of a compute node |
TWI322569B (en) * | 2006-09-26 | 2010-03-21 | Optoma Corp | Reset circuit and method for high definition multimedia interface |
US20120092044A1 (en) * | 2010-10-13 | 2012-04-19 | Ability Enterprise Co., Ltd. | Circuit for swapping a memory card in an electronic device |
-
2018
- 2018-04-13 US US15/953,208 patent/US10437751B1/en active Active
Also Published As
Publication number | Publication date |
---|---|
US10437751B1 (en) | 2019-10-08 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US8332669B2 (en) | Information handling system port security | |
US10846159B2 (en) | System and method for managing, resetting and diagnosing failures of a device management bus | |
US10317973B2 (en) | Peripheral device expansion card system | |
US10852352B2 (en) | System and method to secure FPGA card debug ports | |
US11782089B2 (en) | Detecting and remediating unauthorized debug sessions | |
US20170293341A1 (en) | Power supply unit mismatch detection system | |
JP6864718B2 (en) | Hybrid power supply system and method | |
US8880747B2 (en) | Endpoint device discovery system | |
US10146704B2 (en) | Volatile/non-volatile memory device access provisioning system | |
US20120095719A1 (en) | Memory thermal management based on slot information | |
US10846256B2 (en) | Multi-endpoint device sideband communication system | |
US10140235B2 (en) | Server | |
US9946552B2 (en) | System and method for detecting redundant array of independent disks (RAID) controller state from baseboard management controller (BMC) | |
US11321246B2 (en) | Support information provisioning system | |
US10437751B1 (en) | Device hot plug system | |
US11341076B2 (en) | Hot-plugged PCIe device configuration system | |
CN113448489B (en) | Computer readable storage medium, method and apparatus for controlling access to flash memory card | |
US20200210367A1 (en) | External serial at attachment device | |
US10855523B2 (en) | Intention-based device component tracking system | |
US20170192917A1 (en) | Systems and methods for hardware arbitration of a communications bus | |
US11163343B1 (en) | Flexible power supply unit (PSU) bay | |
US6697954B1 (en) | Method/apparatus for preserving state of an event during powerup reset sequence based on state of an event signal immediately prior to the reset | |
US11334359B2 (en) | Systems and methods for management of dynamic devices | |
US10747706B2 (en) | Server event log storage and retrieval system | |
US10936037B2 (en) | Multi-power rail power supply unit mismatch detection system |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: DELL PRODUCTS L.P., TEXAS Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:NORDEN, HAHN;ARZOLA, CHRISTOPHER;BOLEN, AUSTIN;SIGNING DATES FROM 20180327 TO 20180328;REEL/FRAME:045539/0406 |
|
FEPP | Fee payment procedure |
Free format text: ENTITY STATUS SET TO UNDISCOUNTED (ORIGINAL EVENT CODE: BIG.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY |
|
AS | Assignment |
Owner name: CREDIT SUISSE AG, CAYMAN ISLANDS BRANCH, AS COLLAT Free format text: PATENT SECURITY AGREEMENT (CREDIT);ASSIGNORS:DELL PRODUCTS L.P.;EMC CORPORATION;EMC IP HOLDING COMPANY LLC;REEL/FRAME:046286/0653 Effective date: 20180529 Owner name: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., A Free format text: PATENT SECURITY AGREEMENT (NOTES);ASSIGNORS:DELL PRODUCTS L.P.;EMC CORPORATION;EMC IP HOLDING COMPANY LLC;REEL/FRAME:046366/0014 Effective date: 20180529 Owner name: CREDIT SUISSE AG, CAYMAN ISLANDS BRANCH, AS COLLATERAL AGENT, NORTH CAROLINA Free format text: PATENT SECURITY AGREEMENT (CREDIT);ASSIGNORS:DELL PRODUCTS L.P.;EMC CORPORATION;EMC IP HOLDING COMPANY LLC;REEL/FRAME:046286/0653 Effective date: 20180529 Owner name: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS COLLATERAL AGENT, TEXAS Free format text: PATENT SECURITY AGREEMENT (NOTES);ASSIGNORS:DELL PRODUCTS L.P.;EMC CORPORATION;EMC IP HOLDING COMPANY LLC;REEL/FRAME:046366/0014 Effective date: 20180529 |
|
AS | Assignment |
Owner name: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., T Free format text: SECURITY AGREEMENT;ASSIGNORS:CREDANT TECHNOLOGIES, INC.;DELL INTERNATIONAL L.L.C.;DELL MARKETING L.P.;AND OTHERS;REEL/FRAME:049452/0223 Effective date: 20190320 Owner name: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., TEXAS Free format text: SECURITY AGREEMENT;ASSIGNORS:CREDANT TECHNOLOGIES, INC.;DELL INTERNATIONAL L.L.C.;DELL MARKETING L.P.;AND OTHERS;REEL/FRAME:049452/0223 Effective date: 20190320 |
|
STCF | Information on status: patent grant |
Free format text: PATENTED CASE |
|
AS | Assignment |
Owner name: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., TEXAS Free format text: SECURITY AGREEMENT;ASSIGNORS:CREDANT TECHNOLOGIES INC.;DELL INTERNATIONAL L.L.C.;DELL MARKETING L.P.;AND OTHERS;REEL/FRAME:053546/0001 Effective date: 20200409 |
|
AS | Assignment |
Owner name: EMC IP HOLDING COMPANY LLC, TEXAS Free format text: RELEASE OF SECURITY INTEREST AT REEL 046286 FRAME 0653;ASSIGNOR:CREDIT SUISSE AG, CAYMAN ISLANDS BRANCH;REEL/FRAME:058298/0093 Effective date: 20211101 Owner name: EMC CORPORATION, MASSACHUSETTS Free format text: RELEASE OF SECURITY INTEREST AT REEL 046286 FRAME 0653;ASSIGNOR:CREDIT SUISSE AG, CAYMAN ISLANDS BRANCH;REEL/FRAME:058298/0093 Effective date: 20211101 Owner name: DELL PRODUCTS L.P., TEXAS Free format text: RELEASE OF SECURITY INTEREST AT REEL 046286 FRAME 0653;ASSIGNOR:CREDIT SUISSE AG, CAYMAN ISLANDS BRANCH;REEL/FRAME:058298/0093 Effective date: 20211101 |
|
AS | Assignment |
Owner name: EMC IP HOLDING COMPANY LLC, TEXAS Free format text: RELEASE OF SECURITY INTEREST IN PATENTS PREVIOUSLY RECORDED AT REEL/FRAME (046366/0014);ASSIGNOR:THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT;REEL/FRAME:060450/0306 Effective date: 20220329 Owner name: EMC CORPORATION, MASSACHUSETTS Free format text: RELEASE OF SECURITY INTEREST IN PATENTS PREVIOUSLY RECORDED AT REEL/FRAME (046366/0014);ASSIGNOR:THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT;REEL/FRAME:060450/0306 Effective date: 20220329 Owner name: DELL PRODUCTS L.P., TEXAS Free format text: RELEASE OF SECURITY INTEREST IN PATENTS PREVIOUSLY RECORDED AT REEL/FRAME (046366/0014);ASSIGNOR:THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT;REEL/FRAME:060450/0306 Effective date: 20220329 |
|
MAFP | Maintenance fee payment |
Free format text: PAYMENT OF MAINTENANCE FEE, 4TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1551); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY Year of fee payment: 4 |