CN113515478B - PCIe Switch system expansion management method - Google Patents

PCIe Switch system expansion management method Download PDF

Info

Publication number
CN113515478B
CN113515478B CN202110885427.4A CN202110885427A CN113515478B CN 113515478 B CN113515478 B CN 113515478B CN 202110885427 A CN202110885427 A CN 202110885427A CN 113515478 B CN113515478 B CN 113515478B
Authority
CN
China
Prior art keywords
pcie switch
switch device
pcie
port
expansion
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202110885427.4A
Other languages
Chinese (zh)
Other versions
CN113515478A (en
Inventor
苏海亮
宁佐林
宁丽霞
马乐
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Wuxi Zhongxing Microsystem Technology Co ltd
Original Assignee
Wuxi Zhongxing Microsystem Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Wuxi Zhongxing Microsystem Technology Co ltd filed Critical Wuxi Zhongxing Microsystem Technology Co ltd
Priority to CN202110885427.4A priority Critical patent/CN113515478B/en
Publication of CN113515478A publication Critical patent/CN113515478A/en
Application granted granted Critical
Publication of CN113515478B publication Critical patent/CN113515478B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F13/00—Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
    • G06F13/38—Information transfer, e.g. on bus
    • G06F13/382—Information transfer, e.g. on bus using universal interface adapter
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F13/00—Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
    • G06F13/38—Information transfer, e.g. on bus
    • G06F13/42—Bus transfer protocol, e.g. handshake; Synchronisation
    • G06F13/4204—Bus transfer protocol, e.g. handshake; Synchronisation on a parallel bus
    • G06F13/4221—Bus transfer protocol, e.g. handshake; Synchronisation on a parallel bus being an input/output bus, e.g. ISA bus, EISA bus, PCI bus, SCSI bus
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2213/00—Indexing scheme relating to interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
    • G06F2213/0026—PCI express

Abstract

The invention provides a system expansion management method of PCIe Switch, which comprises the following steps: determining the number of the downstream ports DP of the PCIe Switch equipment to be expanded, and if the number of the DP exceeds a preset threshold, connecting the first PCIe Switch equipment and the second PCIe Switch equipment by adopting a first expansion mode; and if the number of the DPs does not exceed the preset threshold, connecting the first PCIe Switch device and the second PCIe Switch device by adopting a second expansion mode, wherein the second expansion mode is different from the first expansion mode. The invention flexibly supports PCIe Switch expansion configuration of a plurality of modes, and can realize adaptation of different modes by updating software firmware according to different application scenes. Particularly, when the number of the DP ports is small, the internal ports are used for interconnection, Bus numbers are saved, and the method is suitable for PCIe system scenes with limited Bus number resources.

Description

PCIe Switch system expansion management method
Technical Field
The invention belongs to the field of bus design, and particularly relates to a system expansion management method for PCIe Switch.
Background
The PCIe Switch is an expansion device in the PCIe topology, and may be used to expand the number of lanes (lanes) for PCIe, so as to enhance the expansion capability of the PCIe device of the main CPU. PCIe Switch is widely applied to a board card device of a large storage system or a server platform. A conventional PCIe expansion mode is as shown in fig. 1, one PCIe Switch chip is connected to another PCIe Switch chip, each device is an independent PCIe Switch chip, and no matter how many devices can be connected to Downstream Ports (DP) of a lower PCIe Switch chip, how many functions of the devices are, an UP port of the lower PCIe Switch always needs to occupy one Bus number in the whole PCIe topology. As shown in fig. 1, there are three PCIe switches, PCIe Switch 1, PCIe Switch 2, and PCIe Switch 3. Each Switch's Upstream Port (UP) occupies a Bus number in the PCIe topology. It can be seen that the prior art has the following problems in the practical application of PCIe extension: the extension of the PCIe Switch chip is fixed and lacks flexibility. When the device is extended through the PCIe Switch chip, each PCIe Switch is an individual chip, and occupies a larger space on an actual product board, and device extension is performed through the PCIe Switch chip, which is performed by adding the PCIe Switch individual chip, so that the BOM cost of the board is inevitably increased.
Disclosure of Invention
The invention aims to provide a system expansion management method of PCIe Switch equipment, which comprises the following steps:
determining the number of downlink ports DP of PCIe Switch equipment to be expanded;
if the number of the DPs exceeds a preset threshold value, connecting a first PCIe Switch device and a second PCIe Switch device by adopting a first expansion mode;
and if the number of the DPs does not exceed the preset threshold, connecting the first PCIe Switch equipment and the second PCIe Switch equipment by adopting a second expansion mode, wherein the second expansion mode is different from the first expansion mode.
Preferably, the connecting the first PCIe Switch device and the second PCIe Switch device in the first extended mode further includes:
connecting a downstream port DP of the first PCIe Switch device with an upstream port UP of the second PCIe Switch device; and configuring a downstream port DP of the first PCIe Switch device as a virtual downstream port VDP, and configuring an upstream port UP of the second PCIe Switch device as a virtual upstream port VUP.
Preferably, in the first extended mode, Bus numbers of the first and second PCIe Switch devices are different.
Preferably, the connecting the first PCIe Switch device and the second PCIe Switch device in the second expansion mode further includes:
connecting the first internal port of the first PCIe Switch device with the second internal port of the second PCIe Switch device.
Preferably, in the second extended mode, all the downstream ports DP in the first PCIe Switch device and the second PCIe Switch device share the same Bus number. When the first PCIe Switch device is connected with a host device, the internal port is transparent to the host device.
Preferably, the preset threshold is the maximum number of devices that can be hooked by a single Bus number in the PCIe device.
Preferably, the first PCIe Switch device and the second PCIe Switch device are connected through an interconnection bus, where the interconnection bus includes a high speed I/O pin interconnection interface or a high speed Serdes interconnection interface.
Preferably, the first PCIe Switch device and the second PCIe Switch device are a first PCIe Switch chip and a second PCIe Switch chip, respectively.
Preferably, the first PCIe Switch device and the second PCIe Switch device are a first PCIe Switch Die and a second PCIe Switch Die, respectively.
Compared with the prior art, the scheme of the invention selects different modes according to the number of DP ports of Die or PCIe Switch extension. For the Die or PCIe Switch chip of the same PCIe Switch, configurations of multiple modes can be flexibly supported, so that Die or PCIe Switch chip extensions of multiple forms are supported. The configuration is carried out in the initialization stage of PCIe Switch or Die, and the adaptation of different modes can be realized by updating software and firmware according to different application scenes. When the number of the DP ports is small, the internal ports are used for interconnection, Bus numbers are saved, and the method is particularly suitable for PCIe system scenes with limited Bus number resources.
Additional features and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by the practice of the invention. The objectives and other advantages of the invention will be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
Fig. 1 is a schematic diagram of the structure of an extended mode of PCIe Switch according to the prior art.
Fig. 2 is a schematic diagram of a first extended mode of PCIe Switch according to the present invention.
Fig. 3 is a schematic diagram of a second extended mode of PCIe Switch according to the present invention.
Fig. 4 is a schematic diagram of an interconnect structure of more than two PCIe switches according to the second expansion mode of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The invention discloses an expansion scheme of PCIe Switch, which comprises the steps of firstly judging the number of expanded PCIe downlink ports DP, determining an expansion mode to be used according to the difference of the number of the expanded PCIe downlink ports DP, and if the number of the DPs exceeds a preset threshold value, selecting the same mode as the general PCIe Switch expansion as a first expansion mode. If the number of DPs does not exceed the preset threshold, the way in which multiple PCIe switches share the Bus number may be selected as the second expansion mode. The preset threshold is the maximum number of devices that a single Bus number can be hooked in the PCIe Switch.
The PCIe Switch is connected with the PCIe Switch through an interconnection bus, the interconnection bus is a self-defined interconnection interface, and the interconnection bus can be configured into ports such as VUP (virtual private protocol), VDP (virtual private protocol) and the like according to application scenes. However, for the Host, the Host end does not need to distinguish the interconnect interface from the common PCIe port. For example, the interconnect interface may be a high speed I/O pin interconnect or a high speed Serdes interconnect.
In one embodiment, the preset threshold is 32. In the first expansion mode, namely when the number of the downstream ports DP exceeds 32, the mode of the common PCIe Switch expansion is utilized. As shown in fig. 2, since a Bus number can be attached with 32 devices at most in the PCIe protocol, when the number of PCIe DPs exceeds 32, it is necessary to connect the UP port of the lower PCIe Switch to one of the DP ports of the upper PCIe Switch through a new Bus number by expanding the Bus number, and attach another device under the new Bus number. And by analogy, hanging 32 devices on the new Bus number at most, and if the devices which are not hung still exist, continuing to expand the new Bus number.
The port of the interconnection interface between PCIe switches is also a port of the Bus internal switching network, and the port may be configured into multiple roles, for example, three roles, such as a VUP (virtual UP port), a VDP (virtual DP port), and an internal port. Referring to fig. 2, in a preferred embodiment, if two switches PCIe Switch 1 and PCIe Switch 2 are interconnected, and the number of extended DP ports exceeds 32, the ports of the interconnect interface of PCIe Switch 1 are configured as VDP ports, fig. 2 exemplarily shows DP1_0 as VDP, and the ports of the interconnect interface of PCIe Switch 2 are configured as VUP ports, and fig. 2 exemplarily shows UP2 as VUP.
For example, if the VDP port is number DP1_0 in PCIe Switch 1, the VUP port is UP2 throughout the PCIe topology. The Host detects the presence of a DP1_0 device on Bus64 during the enumeration phase, but the DP1_0 appears to the Host to be identical to the other DPs. And Host will detect that a UP2, i.e. a UP under Bus65, is hooked under the DP1_0 device, and the UP is VUP. But from the Host's perspective, the UP2 is identical to the ordinary PCIe UP port.
The VUPs and VDPs comprise at least PCIe 4KB configuration space registers, other PCIe functions can be extended as needed, and all the VUPs and VDPs conform to PCIe protocol specifications.
In one embodiment, the preset threshold is 32. In the second expansion mode, that is, when the downstream port DP does not exceed the maximum number of devices that can be attached by a single Bus number, two PCIe Switch expansion devices may be used, but all the downstream ports DP in the two PCIe switches share one Bus number, as shown in fig. 3.
If the two switches of the PCIe Switch 1 and the PCIe Switch 2 are interconnected, and the number of the expanded DP ports is less than 32, the interconnection is performed in the manner shown in fig. 3, at this time, the ports of the two switches of the PCIe Switch 1 and the PCIe Switch 2 are both configured as internal ports, and the interconnection port at this time is only one port of the Switch internal switching network. One Bus number can be saved. Because of the internal port, the Host end can not sense the existence of the connection of PCIe Switch 2, and can not sense the port of VUP/VDP in the PCIe Switch 2, and can only detect that a maximum of 32 DPs are connected under Bus 64.
Those skilled in the art will appreciate that the topology of the elements and the number of functional blocks described in the above embodiments are by way of example only. Those skilled in the art can apply the method of the present invention to the interconnection of two PCIe Switch devices or to the interconnection of a plurality of PCIe Switch devices as needed. Fig. 4 shows a case where PCIe Switch 1 and PCIe Switch 2 and PCIe Switch 3 are respectively interconnected.
As shown in fig. 4, three PCIe switches share one Bus number, and ports of two switches of PCIe Switch 1, PCIe Switch 2, and PCIe Switch 3 are configured as internal ports, and the interconnect port at this time is only one port of the Switch internal switching network. Two Bus numbers can be saved. The Host end can not sense the existence of the connection of PCIe switches 2 and 3, and can only detect that a maximum of 32 DPs are connected under Bus 64.
According to alternative embodiments, the PCIe Switch 1 and PCIe Switch 2 may be two PCIe Switch Die, or may be two independent Switch chips.
When the PCIe Switch Die 1 and Die 2 need to be connected in an expansion mode, the expansion mode used is determined according to the number of the DP of the downstream ports, and if the number of the DP exceeds a preset threshold, the same mode as that of the general PCIe expansion can be selected. If the number of DPs does not exceed the preset threshold, a way of sharing the Bus number by multiple Die may be selected. The preset threshold is the maximum number of devices which can be hooked by a single Bus number in the Die. The above ports can be configured in various roles, for example, three roles of VUP (virtual UP port), VDP (virtual DP port), internal port, and the like.
In one embodiment, the preset threshold is 32. Namely, when the number of the downstream ports DP exceeds 32, the mode of the universal PCIe expansion is utilized. Since a Bus number can be hooked with 32 devices at most, when the number of the DPs exceeds 32, the Bus number needs to be expanded. Connecting the UP port of the lower-level Die with one of the DP ports of the upper-level Die through a new Bus number, and additionally hanging other equipment under the new Bus number. And by analogy, the new Bus number is hooked with 32 devices at most, and if the un-hooked devices still exist, the new Bus number is continuously expanded.
When the number of the downlink ports DP does not exceed the maximum equipment number which can be hooked by a single Bus number, two Dies can share one Bus number. At this time, the ports of both switches of Die 1 and Die 2 are configured as internal ports, and the interconnect port at this time is only one port of the Die internal switching network. Thus saving one Bus number. Because of the internal ports, the Host end cannot sense the existence of the connection of Die 2, and cannot sense the port of VUP/VDP in Die 2, and only up to 32 DPs connected under Bus64 can be detected.
When PCIe Switch 1 and PCIe Switch 2 are Die, a chip may be sealed for use, and the BOM cost of a board may be saved.
Therefore, by adopting the technical scheme of the invention, the Die or PCIe Switch chip of the same PCIe Switch can flexibly support the configuration of a plurality of modes, thereby supporting the expansion of the Die or PCIe Switch chip of a plurality of forms. The invention can select different modes according to the number of DP ports of Die or PCIe Switch expansion. The configuration is carried out in the initialization stage of PCIe Switch or Die, and the adaptation of different modes can be realized by updating software and firmware according to different application scenes. When the number of the DP ports is excessive, the data transmission method is interconnected by using a VUP (virtual private protocol) and VDP (virtual private protocol) mode, and is compatible with the expansion mode of the current PCIe system. When the number of the DP ports is small, the internal ports are used for interconnection, Bus numbers are saved, and the method is particularly suitable for PCIe system scenes with limited Bus number resources.
Although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.

Claims (6)

1. A system expansion management method of PCIe Switch equipment is characterized by comprising the following steps:
determining the number of downlink ports DP of PCIe Switch equipment to be expanded;
if the number of the downlink ports DP exceeds a preset threshold value, connecting a first PCIe Switch device and a second PCIe Switch device by adopting a first expansion mode;
if the number of the downstream ports DP does not exceed the preset threshold, connecting the first PCIe Switch device and the second PCIe Switch device by adopting a second expansion mode, wherein the second expansion mode is different from the first expansion mode;
the connecting the first PCIe Switch device and the second PCIe Switch device in the first expansion mode further includes:
connecting a downstream port DP of the first PCIe Switch device with an upstream port UP of the second PCIe Switch device;
configuring a downstream port DP of the first PCIe Switch device as a virtual downstream port VDP, and configuring an upstream port UP of the second PCIe Switch device as a virtual upstream port VUP; in the first extended mode, the Bus numbers of the first PCIe Switch device and the second PCIe Switch device are different;
the connecting the first PCIe Switch device and the second PCIe Switch device in the second expansion mode further includes:
connecting a first internal port of the first PCIe Switch device with a second internal port of the second PCIe Switch device; in the second extended mode, all the downstream ports DP in the first PCIe Switch device and the second PCIe Switch device share the same Bus number.
2. The system expansion management method for PCIe Switch devices of claim 1, wherein when the first PCIe Switch device is connected with a host device, the internal port is transparent to the host device.
3. The method of claim 1, wherein the predetermined threshold is a maximum number of devices that can be hooked by a single Bus number in the PCIe Switch device.
4. The system expansion management method for PCIe Switch devices according to claim 1, wherein the first PCIe Switch device and the second PCIe Switch device are connected through an interconnection bus, and the interconnection bus comprises a high-speed I/O pin interconnection interface or a high-speed Serdes interconnection interface.
5. The system expansion management method for the PCIe Switch devices according to claim 1, wherein the first PCIe Switch device and the second PCIe Switch device are a first PCIe Switch chip and a second PCIe Switch chip, respectively.
6. The system expansion management method for PCIe Switch devices according to claim 1, wherein the first PCIe Switch device and the second PCIe Switch device are a first PCIe Switch Die and a second PCIe Switch Die, respectively.
CN202110885427.4A 2021-08-03 2021-08-03 PCIe Switch system expansion management method Active CN113515478B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110885427.4A CN113515478B (en) 2021-08-03 2021-08-03 PCIe Switch system expansion management method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110885427.4A CN113515478B (en) 2021-08-03 2021-08-03 PCIe Switch system expansion management method

Publications (2)

Publication Number Publication Date
CN113515478A CN113515478A (en) 2021-10-19
CN113515478B true CN113515478B (en) 2022-09-09

Family

ID=78068726

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110885427.4A Active CN113515478B (en) 2021-08-03 2021-08-03 PCIe Switch system expansion management method

Country Status (1)

Country Link
CN (1) CN113515478B (en)

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114253625B (en) * 2021-11-29 2023-07-14 山东云海国创云计算装备产业创新中心有限公司 PCIe Switch chip working mode switching method, system, storage medium and device
CN114238000B (en) * 2021-12-21 2022-11-22 无锡众星微系统技术有限公司 Port test management method and device for PCIe Switch
CN114968885A (en) * 2022-05-31 2022-08-30 山东云海国创云计算装备产业创新中心有限公司 Multi-fusion expansion equipment
CN115114219B (en) * 2022-07-22 2023-10-20 深圳星云智联科技有限公司 PCI-E topology method, device, equipment and storage medium

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111538693A (en) * 2020-04-27 2020-08-14 中国科学院自动化研究所 PCIE bus expansion system and method

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105553886A (en) * 2015-12-25 2016-05-04 山东海量信息技术研究院 PCIE switch capable of flexibly extending port number
CN111586331A (en) * 2019-02-18 2020-08-25 浙江宇视科技有限公司 Video processing equipment

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111538693A (en) * 2020-04-27 2020-08-14 中国科学院自动化研究所 PCIE bus expansion system and method

Also Published As

Publication number Publication date
CN113515478A (en) 2021-10-19

Similar Documents

Publication Publication Date Title
CN113515478B (en) PCIe Switch system expansion management method
US7099969B2 (en) Dynamic reconfiguration of PCI Express links
US10585832B2 (en) Apparatus and methods for dynamic role switching among USB hosts and devices
CN102045608B (en) Network device for optical communication and method thereof for automatically configuring exchange interface
US20060168377A1 (en) Reallocation of PCI express links using hot plug event
JP2510080B2 (en) System device and method for connecting multiple protocol terminals
US7120711B2 (en) System and method for communicating over intra-hierarchy and inter-hierarchy links
KR100334857B1 (en) Variable slot configuration for multi-speed bus
CN104657317B (en) Server
US10673693B2 (en) Ring based network interconnect
CN1909559B (en) Interface board based on rapid periphery components interconnection and method for switching main-control board
US20140280960A1 (en) Methods and apparatus for dynamically allocating devices between multiple controllers
CN109002411B (en) Method and system for automatically configuring GPU expansion box and GPU expansion box capable of being automatically configured
CN101557379B (en) Link reconfiguration method for PCIE interface and device thereof
US8244948B2 (en) Method and system for combining multiple SAS expanders into a SAS switch
CN101610192A (en) A kind of communication slave, bus cascade method and system
CN108259246A (en) A kind of circuit and server that NCSI signal managements function is supported to automatically switch
US20180276167A1 (en) Memory card expansion
CN105205025A (en) Chip interconnection method, chips and device
US8015341B2 (en) Communications device and method for configuring a communication port depending on a direction condition of a remote device communicatively coupled to the communications port
JP4800607B2 (en) Universal controller for peripheral devices in computing systems
CN209248436U (en) A kind of expansion board clamping and server
CN109684255A (en) A kind of FPGA pin multiplexing circuit and control method
CN210573287U (en) Expansion circuit
US7143216B1 (en) System for configuring expandable buses in a multi-device storage container and related method

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant