CN111078600B - PCIe Switch-based RSSD large-capacity storage system - Google Patents
PCIe Switch-based RSSD large-capacity storage system Download PDFInfo
- Publication number
- CN111078600B CN111078600B CN201911119077.XA CN201911119077A CN111078600B CN 111078600 B CN111078600 B CN 111078600B CN 201911119077 A CN201911119077 A CN 201911119077A CN 111078600 B CN111078600 B CN 111078600B
- Authority
- CN
- China
- Prior art keywords
- pcie switch
- rssd
- pcie
- switch card
- card
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000013461 design Methods 0.000 description 13
- 230000005540 biological transmission Effects 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 238000000034 method Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000011144 upstream manufacturing Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F13/00—Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
- G06F13/14—Handling requests for interconnection or transfer
- G06F13/16—Handling requests for interconnection or transfer for access to memory bus
- G06F13/1605—Handling requests for interconnection or transfer for access to memory bus based on arbitration
- G06F13/1652—Handling requests for interconnection or transfer for access to memory bus based on arbitration in a multiprocessor architecture
- G06F13/1657—Access to multiple memories
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F13/00—Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
- G06F13/14—Handling requests for interconnection or transfer
- G06F13/16—Handling requests for interconnection or transfer for access to memory bus
- G06F13/1668—Details of memory controller
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F13/00—Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
- G06F13/38—Information transfer, e.g. on bus
- G06F13/42—Bus transfer protocol, e.g. handshake; Synchronisation
- G06F13/4282—Bus transfer protocol, e.g. handshake; Synchronisation on a serial bus, e.g. I2C bus, SPI bus
Abstract
The invention discloses a PCIe Switch-based RSSD (remote secure digital) mass storage system, which comprises: the first PCIE Switch card, the second PCIE Switch card, the third PCIE Switch card, the plurality of RSSD discs arranged on the first backboard and the plurality of RSSD discs arranged on the second backboard are arranged on the mainboard; the first CPU is respectively connected with the uplink ports of a first PCIE Switch card and a second PCIE Switch card, and the second CPU is connected with the uplink port of a third PCIE Switch card; the downstream ports of the first PCIE Switch card are connected to a part of RSSD disks disposed on the first backplane, the downstream ports of the third PCIE Switch card are connected to a part of RSSD disks disposed on the second backplane, and the downstream ports of the second PCIE Switch card are respectively connected to the remaining RSSD disks disposed on the first backplane and the remaining RSSD disks disposed on the second backplane.
Description
Technical Field
The invention relates to the technical field of server system design, in particular to a PCIe Switch-based RSSD (remote secure digital) mass storage system.
Background
In the server system, a main board and different types of sub cards are matched together to realize all functions of the system together; at the motherboard and daughter card, PCIe high-speed bus is generally used to complete the transmission of high-speed signals. PCIe belongs to high-speed serial point-to-point double-channel high-bandwidth transmission, connected devices distribute independent channel bandwidth and do not share bus bandwidth, and the PCIe mainly supports functions of active power management, error reporting, end-to-end reliable transmission, hot plug, quality of service (QOS) and the like.
PCIe is used as a high-speed bus, different speeds can be achieved under different bandwidths, and a PCIe Switch chip is used as a commonly-used bandwidth expansion controller, so that the expansion of uplink and downlink PCIe channels can be realized, and more downlink devices can be met.
Ruler is a code number of a brand-new solid state disk manufactured by Intel, is specially manufactured for a server data center, and is called to provide a remarkable capacity of 1PB (1000TB) in a 1U space; the Ruler SSD design operates with reference to the PCIe protocol, with a bandwidth of x4, and can reduce bandwidth usage.
In the prior art, due to bandwidth limitation, a multi-disk SSD cannot be directly supported, and a PCIe Switch based RSSD design is not enough for a system, and a storage design with more disks cannot be realized. The specific problems are as follows:
1. the number of direct connection hard disks is limited, and a larger storage space cannot be realized;
2. the RSSD design based on PCIe Switch is not enough, and the PCIe Switch card has compatibility and backboard management problems when mixed-assembling a backboard;
3. when the same CPLD FW version is used for 2 backplanes, the problem that the two backplanes cannot be distinguished exists.
Disclosure of Invention
The invention aims to provide a PCIe Switch-based RSSD (secure storage device) mass storage system, which solves the system design problem of multi-disk RSSD.
In order to achieve the purpose, the invention adopts the following technical scheme:
a PCIe Switch based RSSD mass storage system, comprising: the first PCIE Switch card, the second PCIE Switch card, the third PCIE Switch card, the plurality of RSSD discs arranged on the first backboard and the plurality of RSSD discs arranged on the second backboard are arranged on the mainboard;
the first CPU is respectively connected with the uplink ports of a first PCIE Switch card and a second PCIE Switch card, and the second CPU is connected with the uplink port of a third PCIE Switch card; the downstream ports of the first PCIE Switch card are connected to a part of RSSD disks disposed on the first backplane, the downstream ports of the third PCIE Switch card are connected to a part of RSSD disks disposed on the second backplane, and the downstream ports of the second PCIE Switch card are respectively connected to the remaining RSSD disks disposed on the first backplane and the remaining RSSD disks disposed on the second backplane.
Preferably, the first back plate and the second back plate are respectively connected with a CPLD, and the CPLD is connected with a status indicator lamp set to control on and off of the status indicator lamp set; the CPLD is also connected with the backboard indicating unit, the output end of the backboard indicating unit is at low level, the CPLD is identified as the first backboard, the output end of the backboard indicating unit is at high level, and the CPLD is identified as the second backboard.
Preferably, the first CPU is connected to a front-end PCIE port of the first riser card, a first rear-end PCIE port of the first riser card is connected to the first PCIE Switch card, and a second rear-end PCIE port of the first riser card is connected to the second PCIE Switch card.
Preferably, the second CPU is connected to a front-end PCIE port of the second riser card, and a first rear-end PCIE port of the second riser card is connected to a third PCIE Switch card.
Preferably, the downstream port of the first PCIE Switch card is connected to an RSSD disk with a disk number of 1 to 23, which is disposed on the first backplane; and the downlink port of the second PCIE Switch card is connected with an RSSD (remote secure disk) with the disk number of 24-29 arranged on the first backboard.
Preferably, the downstream port of the third PCIE Switch card is connected to an RSSD disk with a disk number of 6 to 29, which is disposed on the second backplane; and the downlink port of the second PCIE Switch card is connected with an RSSD disk with a disk number of 0-5, which is arranged on the second backboard.
The effect provided in the summary of the invention is only the effect of the embodiment, not all the effects of the invention, and one of the above technical solutions has the following advantages or beneficial effects:
the invention provides a design scheme of an RSSD system based on PCIe Switch, which realizes the design of a large-capacity storage system which expands PCIe through the PCIe Switch and supports 60 disk positions, effectively ensures correct disk sequence, hard disk state management and normal lighting of a state indicator lamp, effectively increases storage capacity and reduces cost. In the invention, the lighting information is analyzed in-band by the PCIe Switch chip and is provided for the back plate CPLD for lighting. In-band analysis does not need to judge the state of the CPU VMD, thereby reducing the code complexity of the CPLD.
Drawings
FIG. 1 is a schematic diagram of a system according to an embodiment of the present invention.
Detailed Description
In order to clearly explain the technical features of the present invention, the following detailed description of the present invention is provided with reference to the accompanying drawings. The following disclosure provides many different embodiments, or examples, for implementing different features of the invention. To simplify the disclosure of the present invention, the components and arrangements of specific examples are described below. Furthermore, the present invention may repeat reference numerals and/or letters in the various examples. This repetition is for the purpose of simplicity and clarity and does not in itself dictate a relationship between the various embodiments and/or configurations discussed. It should be noted that the components illustrated in the figures are not necessarily drawn to scale. Descriptions of well-known components and processing techniques and procedures are omitted so as to not unnecessarily limit the invention.
As shown in fig. 1, the present invention is implemented by a motherboard, Riser cards, 3 PCIe Switch cards, and 2 30-disk backplanes, where the number of Riser cards is determined by the PCIe expansion mode and the bandwidth number of the motherboard. PCIe resources on the mainboard are switched into PCIe standard slots through the riser card, the PCIe Switch card is in a standard card form, the PCIe Switch card used in the invention is an upstream x16 bandwidth and a downstream x48 bandwidth, and the RSSD used in the invention is an x2 bandwidth, so that 1 PCIe Switch card can expand an x48 bandwidth at most and supports RSSDs with 24 x2 bandwidths. The invention is a system design of 60-disk RSSD, 2 backplanes are used for management, and 1 backplane supports 30 disks, so that 60 disks need to be supported by 3 PCIe Switch cards.
In order to control the disk order of the hard disk, PCIe Switch cards 1 and 2 are connected to the slot expanded by the CPU0, the disk is in the front, PCIe Switch card 3 is connected to the slot expanded by the CPU1, and the disk is in the back. At the rear end of the PCIe Switch card, the PCIe Switch card 1 is connected with 0-23 disks of the backplane 1, the PCIe Switch card 2 is connected with 24-29 disks of the backplane 1 and 0-5 disks of the backplane 2, and the PCIe Switch card 3 is connected with 6-29 disks of the backplane 2. In order to control the disk sequence, the hard disks on the two backplanes have different sequences, the front 24 disks of the backplane 1 are from 1 PCIe Switch card, and the back 24 disks of the backplane 2 are from 1 PCIe Switch card.
The backboard is characterized in that the hard disk state management and the indication lamp display are the main design points, the general scheme is that the backboard state monitoring and lighting are realized on the backboard through a CPLD, and common led has an active/locate/error lamp. The general lighting of the direct-connected backboard is realized by VPP I2C which is directly output by an Intel CPU, VPP I2C is serial data which contains information of a plurality of bays, the core function of the CPLD on the backboard is to analyze the VPP I2C, analyze the information of the corresponding bays and light a hard disk indicator lamp. The CPLD under the direct connection needs to preferentially judge the state of the CPU VMD, and the CPLD can be lighted when the CPLD is enabled and the lighting information is ignored when the CPLD is closed.
In the system design scheme of the invention, VPP I2C straight out from a CPU on a mainboard does not need to be connected, and because PCIe Switch is expanded, the expanded downlink lane cannot correspond to PCIe ports of the CPU one by one, therefore, in the invention, the lighting information is analyzed out from the PCIe Switch chip in band and is provided for a back panel CPLD to be used for lighting. In the scheme of the invention, the in-band analysis does not need to judge the state of the CPU VMD, thereby reducing the code complexity of the CPLD. Under the condition that the backboard 1 and the backboard 2 share the CPLD, in order to distinguish the two backplates and different VPP I2C signals, the invention uses Jumper to inform the CPLD on the backboard, when the Jumper is set to be 0, the Jumper corresponds to the backboard 1, and when the Jumper is set to be 1, the Jumper corresponds to the backboard 2.
The invention provides a design scheme of an RSSD system based on PCIe Switch, which realizes the design of a large-capacity storage system which expands PCIe through the PCIe Switch and supports 60 disk positions, effectively ensures correct disk sequence, hard disk state management and normal lighting of a state indicator lamp, effectively increases storage capacity and reduces cost.
Although the embodiments of the present invention have been described with reference to the accompanying drawings, it is not intended to limit the scope of the present invention, and it should be understood by those skilled in the art that various modifications and variations can be made without inventive efforts by those skilled in the art based on the technical solution of the present invention.
Claims (6)
1. A PCIe Switch based RSSD mass storage system, comprising: the first PCIE Switch card, the second PCIE Switch card, the third PCIE Switch card, the plurality of RSSD discs arranged on the first backboard and the plurality of RSSD discs arranged on the second backboard are arranged on the mainboard;
the first CPU is respectively connected with the uplink ports of a first PCIE Switch card and a second PCIE Switch card, and the second CPU is connected with the uplink port of a third PCIE Switch card; the downstream ports of the first PCIE Switch card are connected to a part of RSSD disks disposed on the first backplane, the downstream ports of the third PCIE Switch card are connected to a part of RSSD disks disposed on the second backplane, and the downstream ports of the second PCIE Switch card are respectively connected to the remaining RSSD disks disposed on the first backplane and the remaining RSSD disks disposed on the second backplane.
2. The PCIe Switch based RSSD mass storage system of claim 1, wherein said first and second back plates are connected to CPLD respectively, said CPLD is connected to status indicator lamp set to control on/off of the status indicator lamp set; the CPLD is also connected with the backboard indicating unit, the output end of the backboard indicating unit is at low level, the CPLD is identified as the first backboard, the output end of the backboard indicating unit is at high level, and the CPLD is identified as the second backboard.
3. The PCIe Switch-based RSSD mass storage system of claim 2, wherein the first CPU is connected to a front-end PCIe port of a first riser card, a first back-end PCIe port of the first riser card is connected to the first PCIe Switch card, and a second back-end PCIe port of the first riser card is connected to the second PCIe Switch card.
4. The PCIe Switch-based RSSD mass storage system of claim 3, wherein the second CPU is connected to a front-end PCIe port of a second riser card, and a first back-end PCIe port of the second riser card is connected to a third PCIe Switch card.
5. The PCIe Switch based RSSD mass storage system of claim 4, wherein the downstream port of said first PCIE Switch card is connected to the RSSD disk with disk number 0-23 set on the first backplane; and the downlink port of the second PCIE Switch card is connected with an RSSD (remote secure disk) with the disk number of 24-29 arranged on the first backboard.
6. The PCIe Switch based RSSD mass storage system of claim 4, wherein the downstream port of said third PCIE Switch card is connected to the RSSD disk with disk number 6-29 set on the second backplane; and the downlink port of the second PCIE Switch card is connected with an RSSD disk with a disk number of 0-5, which is arranged on the second backboard.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911119077.XA CN111078600B (en) | 2019-11-15 | 2019-11-15 | PCIe Switch-based RSSD large-capacity storage system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911119077.XA CN111078600B (en) | 2019-11-15 | 2019-11-15 | PCIe Switch-based RSSD large-capacity storage system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111078600A CN111078600A (en) | 2020-04-28 |
| CN111078600B true CN111078600B (en) | 2021-04-27 |
Family
ID=70311160
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911119077.XA Active CN111078600B (en) | 2019-11-15 | 2019-11-15 | PCIe Switch-based RSSD large-capacity storage system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111078600B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113110978B (en) * | 2021-04-07 | 2023-02-28 | 山东英信计算机技术有限公司 | Hard disk backboard lamp control device and method |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN203858630U (en) * | 2014-05-16 | 2014-10-01 | 昆达电脑科技(昆山)有限公司 | Pcie interface switching device |
| CN206684730U (en) * | 2017-04-20 | 2017-11-28 | 郑州云海信息技术有限公司 | The system that a kind of PCIE of storage server extends direct-connected hard disk |
| CN107766213A (en) * | 2017-09-29 | 2018-03-06 | 郑州云海信息技术有限公司 | A kind of method and system for realizing NVME hard disk lightings |
| CN107908586A (en) * | 2017-12-22 | 2018-04-13 | 郑州云海信息技术有限公司 | A kind of NVME hard-disc storages structure and its implementation |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101719051B (en) * | 2009-11-09 | 2012-06-13 | 中兴通讯股份有限公司 | Multi-control disk array and implementation method thereof |
| CN104049915B (en) * | 2014-06-20 | 2017-01-25 | 华为技术有限公司 | Storage system and communication method |
| KR101631461B1 (en) * | 2014-09-30 | 2016-06-17 | 주식회사 네오셈 | Memory Device Test Apparatus and Method |
| US10418121B2 (en) * | 2017-03-27 | 2019-09-17 | Sk Hynix Memory Solutions Inc. | Memory system with diagnose command and operating method thereof |
| CN109032855B (en) * | 2018-07-24 | 2021-12-03 | 郑州云海信息技术有限公司 | Double-control storage device |
-
2019
- 2019-11-15 CN CN201911119077.XA patent/CN111078600B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN203858630U (en) * | 2014-05-16 | 2014-10-01 | 昆达电脑科技(昆山)有限公司 | Pcie interface switching device |
| CN206684730U (en) * | 2017-04-20 | 2017-11-28 | 郑州云海信息技术有限公司 | The system that a kind of PCIE of storage server extends direct-connected hard disk |
| CN107766213A (en) * | 2017-09-29 | 2018-03-06 | 郑州云海信息技术有限公司 | A kind of method and system for realizing NVME hard disk lightings |
| CN107908586A (en) * | 2017-12-22 | 2018-04-13 | 郑州云海信息技术有限公司 | A kind of NVME hard-disc storages structure and its implementation |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111078600A (en) | 2020-04-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10896113B2 (en) | Method for implementing backplane lighting for multiple NVMe hard disks | |
| US10452576B2 (en) | NVMe drive detection from a SAS/SATA connector | |
| CN107357753B (en) | Method and system for realizing automatic matching of PCIE port and hard disk address | |
| CN105824366A (en) | Large-capacity high-speed recording board card on basis of Rapid IO (Input-Output) | |
| US10545901B2 (en) | Memory card expansion | |
| CN109164884B (en) | Design method of hard disk backboard, hard disk backboard and server | |
| CN213365380U (en) | Server mainboard and server | |
| CN112286857A (en) | Server I2C bus allocation method and system supporting multi-backplane cascade | |
| CN210639614U (en) | NVME hard disk backboard system supporting various bandwidths | |
| CN108280036A (en) | A kind of storage hard disk backboard for supporting NVME | |
| CN111078600B (en) | PCIe Switch-based RSSD large-capacity storage system | |
| CN103036958A (en) | Server cluster high-speed memory system | |
| CN100541387C (en) | A kind of server system based on the Opteron processor | |
| CN102521194A (en) | Method and system capable of realizing high-speed interconnection between devices | |
| CN104460857A (en) | Peripheral component interconnect-express card and method and device for using same | |
| CN215219678U (en) | PCIE dual junction test fixture device | |
| CN213276462U (en) | Two-way server mainboard and two-way server | |
| CN107704403B (en) | Device and method for optimizing signal transmission of main back plate | |
| CN109582620B (en) | UART interface conversion device and method | |
| CN210328065U (en) | Driving system of hard disk signal lamp | |
| CN112069108A (en) | Flexible server configuration system and method based on PCIE Switch | |
| CN114116564A (en) | PCIE (peripheral component interface express) bandwidth resource expansion method and device for bridging PCIE bus and Raid card | |
| CN201628957U (en) | Host and server | |
| CN112306920A (en) | Method for reducing hard disk logic controllers and server | |
| CN114528236B (en) | Hard disk backboard communication device, communication method and server |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |