WO2013030910A1 - Mra switch, computer system using same, and method for changing setting of mra switch - Google Patents

Mra switch, computer system using same, and method for changing setting of mra switch Download PDF

Info

Publication number
WO2013030910A1
WO2013030910A1 PCT/JP2011/069354 JP2011069354W WO2013030910A1 WO 2013030910 A1 WO2013030910 A1 WO 2013030910A1 JP 2011069354 W JP2011069354 W JP 2011069354W WO 2013030910 A1 WO2013030910 A1 WO 2013030910A1
Authority
WO
WIPO (PCT)
Prior art keywords
switch
mra
bridge
register
identifier
Prior art date
Application number
PCT/JP2011/069354
Other languages
French (fr)
Japanese (ja)
Inventor
渡辺　憲一
Original Assignee
株式会社日立製作所
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 株式会社日立製作所 filed Critical 株式会社日立製作所
Priority to PCT/JP2011/069354 priority Critical patent/WO2013030910A1/en
Publication of WO2013030910A1 publication Critical patent/WO2013030910A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F13/00—Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
    • G06F13/38—Information transfer, e.g. on bus
    • G06F13/40—Bus structure
    • G06F13/4004—Coupling between buses
    • G06F13/4022—Coupling between buses using switching circuits, e.g. switching matrix, connection or expansion network

Definitions

  • the present invention relates to an MRA switch (Multi-Root Aware (MRA) Switch), a computer system using the switch, and an MRA switch setting changing method.
  • MRA switch Multi-Root Aware (MRA) Switch
  • the PCI Express and its successor specification, the PCI Express specification are defined and used as general-purpose I / O standards that allow devices provided by different vendors to exchange data and operate.
  • the Multi-Root I / O Virtualization and Sharing Specification (MR-IOV) specifications are defined as a standard specification for sharing a PCI Express device among a plurality of computers, for example, by extending the PCI Express specification. Used in computers such as blade servers.
  • Root Complex is the vertex of the I / O tree for connecting the CPU and memory to the I / O.
  • End Point is a device having functions such as a graphic controller, LAN, and fiber channel.
  • Switch is a component for connecting Root Complex, a plurality of End points, and another Switch. The connection between Root Complex and Switch, Switch and End Point, or Root Complex and End Point is particularly called Link.
  • Link In the I / O tree, the side closer to RootRComplex is called Upstream, and the side closer to End Point is called Downstream.
  • PCIe-Switch a switch that complies with the PCI-Express standard is referred to as PCIe-Switch, and is distinguished from MRA-Switch described later.
  • MRA-Switch Multi-RootRAware Switch
  • MRA device End Point corresponding to MR-IOV specification
  • MRA-Switch internally configures a virtual PCIe-Switch, and connects PCI Express devices and MRA devices to each other.
  • the virtual PCIe-Switch in the MRA-Switch is particularly referred to as Virtual (Switch (VS).
  • MRA-Switch is generally implemented as an LSI.
  • PCIe-Switch has a plurality of relay units called PCI-PCI Bridges and a plurality of external connection units called Ports. There is one PCI-PCI Bridge of PCIe-Switch, one on the Upstream side and multiple on the Downstream side. PCIe-Switch belongs to one I / O tree, and PCI-PCI Bridge and Port have a one-to-one correspondence. Note that PCIe-Switch is described in Non-Patent Document 1.
  • MRA-Switch has a plurality of virtual switches and a plurality of ports, and each virtual switch has a plurality of VS bridges.
  • Each Virtual Switch has one VS Bridge on the Upstream side and multiple VS Bridges on the Downstream side, as in PCIe-Switch.
  • a plurality of VS Bridges share a single Port and a link with a connection destination.
  • the VS Bridge has a PCI-PCI Bridge function and a function added in the MR-IOV specification, and is handled in the same manner as the PCI-PCI Bridge by the OS (Operating System).
  • Patent Document 1 MRA-Switch and an electronic computer using the MRA-Switch are described in Patent Document 1 and Non-Patent Document 2.
  • the connection between PCI Express Link and PCI-PCI Bridge of Upstream in PCIe-Switch and PCI-PCI Bridge of Downstream is recognized as PCI bus of PCI specification by the OS.
  • Fig. 11 shows a typical MRA-Switch configuration.
  • the MRA-Switch 110 has a management unit 1101 for managing the entire MRA-Switch 110.
  • the software for setting the MRA-Switch 110 is called Multi-Root-PCI Manager (MR-PCIM).
  • MR-PCIM 1110 performs VS setting in the MRA-Switch 110 via the management unit 1101.
  • Tables set by the MR-PCIM 111 include tables such as VS Bridge Table 1102, Port Table 1103, and VS Table 1104 defined in the MR-IOV specification. Among these tables, VS
  • Table 1102 is a table for setting for every VS
  • PCI-SIG issued, PCI Express (R) Base Specification Revision 2.0, 2006, 1.3.3 (P.39) Published by PCI-SIG, Multi-Root I / O Virtualization and Sharing Specification Revision 1.0, 2008, 1.2.4.4 (P.29)
  • a system in which a plurality of RCs and a plurality of EPs are connected by MRA-Switch is greatly influenced by the flexibility of the configuration of Virtual Switch in MRA-Switch.
  • which Virtual Switch the VS Bridge belongs to is determined at the time of designing the MRA-Switch LSI, so changing the setting of the MRA-Switch during the operation of the computer is not possible. Impossible. For this reason, the configuration flexibility of the entire computer system may not be supported. Therefore, since the configuration of the MRA-Switch determines the flexibility of the configuration of the entire system, it is desired to improve the flexibility of the configuration of the MRA-Switch.
  • connection relationship between VS Bridge and Port can be changed in VS, but it is difficult to mount this on LSI. This is because all-to-all wiring must be performed between VS Bridge and Port, and the amount of wiring increases. Although the amount of wiring can be reduced by fixing the correspondence between VS Bridge and Port, this greatly reduces the flexibility of the MRA-Switch configuration.
  • An object of the present invention is to enable the movement of VS Bridge between different virtual switches (VS) to improve the flexibility of the configuration of the MRA switch, a computer system using the MRA switch, and a method for changing the setting of the MRA switch Is to provide.
  • VS virtual switches
  • the MRA switch according to the present invention is preferably an MRA switch based on the MR-IOV specification, A plurality of ports each connected with an identifier connected to an I / O device; One or more virtual switches (VS) each having a unique identifier; A plurality of VS Bridges constituting the virtual switch and connecting the virtual switch and the port; A storage unit for storing the identifier of the virtual switch to which each VS Bridge belongs; The VS Bridge can be moved between different virtual switches by changing the identifier of the virtual switch stored in the storage unit according to an instruction from the outside.
  • VS virtual switches
  • the storage unit is a register that is mounted in each of the VS Bridges and sets an identifier of the virtual switch for each VS Bridge.
  • the storage unit includes a P # register that sets an identifier of the port to which the VS Bridge is connected, and a Virtual Hierarchy for the port to which the VS Bridge is connected to identify the VS Bridge.
  • the MRA switch stores a VS Bridge table including a VH # register for setting (VH) and a VS # register for setting an identifier of a virtual switch to which the VS Bridge belongs.
  • the MRA switch includes a management unit that manages the MRA switch, and the contents of the storage unit or the register are rewritten by an instruction from a management terminal connected to the management unit.
  • the computer system according to the present invention is preferably based on the MR-IOV specification, one or a plurality of MRA switches having a connection relationship and one or a plurality of one or a plurality of MRA switches connected to ports of the one or a plurality of MRA switches.
  • a computer system comprising a computer and one or more I / O devices connected to a port of any one or more of the MRA switches,
  • the MRA switch is connected to an I / O device and includes a plurality of ports each assigned an identifier, one or more virtual switches (VS) each assigned a unique identifier, and the virtual switch
  • VS virtual switches
  • a plurality of VS Bridges that connect the virtual switch and the port, and a storage unit that stores an identifier of the virtual switch to which each VS Bridge belongs, By changing the identifier of the virtual switch stored in the storage unit according to an instruction from the outside, the computer system is configured to enable the movement of the VS Bridge between different virtual switches.
  • the storage unit is a register that is mounted in each of the VS Bridges and sets an identifier of the virtual switch for each VS Bridge.
  • the storage unit includes a P # register that sets an identifier of the port to which the VS Bridge is connected, and a Virtual Hierarchy for the port to which the VS Bridge is connected to identify the VS Bridge.
  • the computer system stores a VS Bridge table including a VH # register for setting (VH) and a VS # register for setting an identifier of a virtual switch to which the VS Bridge belongs.
  • each of the MRA switches includes a management unit that manages the MRA switch, and further includes a management terminal connected to the management unit. According to an instruction from the management terminal, the storage unit or The computer system in which the contents of the register are rewritten.
  • the MRA switch setting changing method is preferably an MRA switch setting changing method based on MR-IOV specifications
  • the MRA switch is connected to an I / O device and includes a plurality of ports each assigned an identifier, one or more virtual switches (VS) each assigned a unique identifier, and the virtual switch
  • VS virtual switches
  • a plurality of VS Bridges that connect the virtual switch and the port, a storage unit that stores an identifier of the virtual switch to which each VS Bridge belongs, and a management unit that manages the MRA switch, Changing the setting of the MRA switch that enables movement of the VS Bridge between different virtual switches by changing the identifier of the virtual switch stored in the storage unit according to an instruction from the management terminal connected to the management unit Configured as a method.
  • the storage unit includes a P # register that sets an identifier of the port to which the VS Bridge is connected, and a Virtual Hierarchy (for identifying the VS Bridge by the port to which the VS Bridge is connected).
  • VH # register for setting VH
  • a VS Bridge table including a VS # register for setting an identifier of a virtual switch to which the VS Bridge belongs, From the management terminal, an identifier to be set in any of the P # register, the VH # register, and the VS # register is input,
  • the management unit of the MRA switch writes the identifier input from the management terminal to the corresponding P # register, the VH # register, and the VS # register. Composed.
  • the present invention it is possible to move and set the VS Bridge between different virtual switches (VS) even after the computer is operating.
  • VS Bridge of VS Virtual Switch
  • the flexibility of the MRA switch can be improved.
  • VS Bridge belonging to the same Port can be moved between a plurality of Virtual Switches. Independently, it is possible to ensure flexibility.
  • FIG. 3 is a diagram showing a modified MRA-Switch configuration in the first embodiment.
  • FIG. 10 is a diagram showing a computer system in which a computer and I / O are connected by MRA-Switch in the third embodiment.
  • FIG. 10 is a diagram showing a computer system in which a computer and I / O are connected by MRA-Switch in the third embodiment.
  • FIG. 10 is a diagram showing a computer system in which a computer and I / O are connected by MRA-Switch in the third embodiment.
  • FIG. 10 is a diagram showing a configuration of MRA-Switch in the third embodiment.
  • FIG. 10 is a diagram illustrating a configuration example of a computer system having MRA-Switch in the third embodiment.
  • FIG. 10 is a diagram illustrating a configuration example of a computer system for explaining setting change of a configuration of MRA-Switch in the fourth embodiment.
  • FIG. 10 is a diagram showing a configuration of MRA-Switch 101 in the fourth embodiment.
  • FIG. 14 is an operation flowchart of MRA-Switch setting change according to the fourth embodiment.
  • FIG. 10 is a flowchart of MRA-Switch setting operation in the fourth embodiment.
  • FIG. 1 shows the configuration of MRA-Switch according to an embodiment.
  • MRA-Switch 1 has a plurality of Ports (ports) 120 and 120 to which external devices are connected.
  • the identifiers P # 0 and P # 1 are assigned to the Ports 120 and 121, respectively.
  • MRA-Switch1 has two VSs (Virtual Switch) 191 and VS192 formed therein. Identifiers VS # 0 and VS # 1 are assigned to VS191 and 192, respectively.
  • MRA-Switch1 also has three VS Bridges (VSBr) 140-142. Of these VS Bridges, VSBrs 140 and 141 belong to VS # 0, and VSBr142 belongs to VS # 1.
  • Each VSBr 140-142 has a VS Bridge Bridge table.
  • the VS Bridge table is formed in a memory (not shown) mounted on each VSBr 140-142.
  • This VS Bridge table includes a P # register 150 for setting the Port to which the VSBr is connected, and a VH # register for setting the Virtual Hierarchy (VH) for identifying the VSBr by the Port to which the VSBr 140 is connected. 151 and a VS # register 152 for setting the Virtual Switch to which the VSBr belongs.
  • the VS # register 152 is a configuration characteristic of the present invention, and the function or operation realized by the register 152 makes it possible to move (rearrange) VS Bridge between different Virtual Switches. It is possible to change the configuration of the switch flexibly. In that sense, this VSBr can be said to be Relocatable (relocatable) VS Bridge (RVSBr).
  • VS # 0 is set in the VS # register 152 of the VSBr140, and is connected to the VS191 having the identifier VS # 0.
  • VS # 1 is set in the VS # register 152 of the VSBr 142 and is connected to the VS192.
  • VSBr 141 is connected to VS 191.
  • P # 0 is set in the P # register 150 of the VSBr 140
  • VH # 0 is set in the VH # register 151, which is connected to the Virtual Hierarchy 160 of the Port 120.
  • VSBr 141 is connected to VH 161 of Port 121
  • VSBr 142 is connected to VH 162 of Port 121.
  • FIG. 2 shows the configuration of the MRA-Switch in a state where the connection of the VSBr 142 is changed from the state of FIG. That is, in the state of FIG. 1, VS # 1 in VSBr142 is set to VS # 1 and connected to VS192, but in the state of FIG.2, VS # 0 in VSBr142 is set to VS # 0. And connected to the VS 191.
  • the Virtual Switch connected as viewed from the VS Bridge is fixed.
  • the Virtual Switch connected as viewed from the VS Bridge can be changed by the VS # register 152, and the VS Bridge can be moved between different VSs. In this way, by moving the VSBr 142 between the VS 191 and the VS 192, the degree of freedom in the configuration of the Virtual Switch can be increased.
  • the Virtual ⁇ Switch to which the VS Bridge belongs can be set to be changeable.
  • various methods of mounting the VS # register and methods of recognition from the MR-PCIM are conceivable.
  • the mounting position of the VS # register in the LSI it is not always mounted in the corresponding VS Bridge as shown in FIGS.
  • all VS # Bridge VS # registers in MRA-Switch can be mounted in one place. Since the VS # register is a register that is set for each VSgBridge, the VS # register is mounted in the VS Bridge in the examples of FIGS.
  • the VS # register does not need to be included in the VS Bridge Table when viewed from the MR-PCIM.
  • a register space called a VS # register Table in which the VS # registers are consolidated in one place. is there.
  • the VS # register is a register that is set for each VS Bridge.
  • the size of VSVBridge Table for each 1VS Bridge is freely determined. Therefore, adding the VS # register to VS Bridge Table can be realized in a form that minimizes the amount of change from the MR-IOV specification. For this reason, in the example of FIGS. 1 and 2, it is preferable to mount the VS # register by including it in VS Bridge Table.
  • the VS # register for setting the VS configuration in the MRA-Switch is installed in the VSVBridge Table similarly to the P # register and the VH # register.
  • the VS # register When configuring the VS at the time of input, it is possible to set the VS # register in the same manner as the P # register and VH # register.
  • the Device number is one of identifiers assigned to the PCI-PCI Bridge, and is used as information for distinguishing the PCI-PCI Bridge on the Downstream side in the PCIe-Switch.
  • the device number is a unique identifier for each VS Bridge on the Downstream side.
  • the PCI Express specification states that “the designer may determine the device number of the PCI-PCI Bridge on the downstream side of the PCIe-Switch by some method”. Therefore, when setting the MRA-Switch configuration, the device number of the VS Bridge on the Downstream side is arbitrarily set so as to be unique in each VS, and this is the same when the present invention is implemented. It is. As described above, according to the present invention, the flexibility of the MRA-Switch configuration can be realized by extending the MR-IOV specification.
  • connection relationship between VS Bridge and Port can be changed, but it is not always easy to mount on LSI due to the wiring relationship.
  • fixing the correspondence between VS Bridge and Port can be an effective means to reduce the amount of wiring.
  • the present invention is effective even when such means is applied.
  • FIG. 3 shows a configuration example of MRA-Switch in which the connection relationship between VS Bridge and Port of MRA-Switch 1 ′ is fixed.
  • the VSBr 140 ′ is connected to the Port 120, and the VSBr 141 ′ and the VSBr 142 ′ are connected to the Port 121, and the connection relationship is fixed.
  • VS 192 cannot use VS Bridge connected to Port 120 ′.
  • FIG. 4 shows a configuration example of MRA-Switch in which the present invention is applied to the MRA-Switch of FIG. Similar to the relationship between VS Bridge and Port in FIG. 3, VSBr 140 is connected to Port 120, and VSBr 141 and VSBr 142 are connected to Port 121, and the relationship is fixed. However, since the virtual switch to which the VSBr 140 belongs is determined by setting the VS # register 152, it is possible to set whether to connect to the VS 191 or the VS 192. When the VSBr 140 is connected to the VS 192, the VSBr 140 connected to the Port 120 can be used from the VS 192. As described above, according to the present invention, even when the correspondence between VS Bridge and Port is fixed, a degree of freedom can be obtained independently of the MR-IOV specification.
  • Migration An example of a computer system to which MRA-Switch according to the present invention is applied will be described. Moving an OS, application, or service running on a computer to another computer is called migration. Migration involves stopping the computer for hardware maintenance or replacement, but wants to migrate by running applications and services.
  • the OS recognizes not only an I / O-specific identifier (such as Fiber Channel's World Wide Name) when recognizing I / O, but also information on the route from the root complex to the I / O in the PCI tree structure. Some are used as identification information.
  • I / O-specific identifier such as Fiber Channel's World Wide Name
  • FIG. 10 shows the overall configuration of a computer system having MRA-Switches 101 to 103.
  • the plurality of computers 5 are connected to the MRA-Switch 101.
  • the MRA-Switch 102 is connected to the I / O 591, and the MRA-Switch 103 is connected to the I / O 592.
  • migration is performed between the computer 500 and the other computers 600 and 700 in this computer system.
  • FIG. 5 to 7 show configuration examples of the computer and the MRA-Switch used in the computer system shown in FIG.
  • the computer 500 is connected to I / O 591 and 592
  • the computer 600 is connected to I / O 691 and 692
  • the computer 700 is connected to I / O 791 and 792.
  • migration can be performed between the computer 500 and the computer 700 on which the information on the route from the root complex to the I / O matches, but the computer 500 on which the information on the route does not match. And computer 600 cannot be migrated.
  • PCI Bus is simply referred to as Bus.
  • a computer 500 includes a CPU 5011 and an RC 5012.
  • the RC 5012 is connected to the Port 511 and Bus 550 of the MRA-Switch 101.
  • the Bus 550 is assigned an identifier B # 0.
  • the identifier assigned to the Bus is determined by searching for Switch and End Point connected to the OS under Root Complex.
  • MRA-Switch 101 is composed of Virtual Switch having VSBr 514 as Upstream PCI-PCI Bridge and VSBr 515 and VSBr 516 as Downstream PCI-PCI Bridge (1 is connected between VSBr 514 and 55B and 515Br at 15B and 515Br). ing.
  • the Port 511 is connected to the VSBr 514, the Port 512 is connected to the VSBr 515, and the Port 513 is connected to the VSBr 516.
  • the connection between Port and VS Bridge is not Bus and is not given an identifier.
  • Port 512 of MRA-Switch 101 and Port 521 of MRA-Switch 102 are connected by Bus 552 (B # 2), and Port 513 of MRA-Switch 101 and Port 531 of MRA-Switch 103 are connected by Bus 555 (B # 5).
  • MRA-Switch 102 and MRA-Switch 103 are set in the same manner as MRA-Switch 101.
  • Port 522 of MRA-Switch 102 is connected by I / O 591 and Bus 554 (B # 4)
  • Port 532 of MRA-Switch 103 is I / O 592.
  • the path information used when the OS operating on the computer 500 recognizes I / O includes an identifier assigned to Bus.
  • the OS recognizes the I / O 591 in the form of B # 0-B # 1-B # 2-B # 3-B # 4-I / O591.
  • I / O 592 is recognized in the form of B # 0-B # 1-B # 5-B # 6-B # 7-I / O592.
  • FIG. 6 shows the configuration of the computer 500 and MRA-Switch that cannot be migrated with the computer 500 of FIG.
  • the computer 600 is connected to the I / O 691 and 692.
  • the OS operating on the computer 600 recognizes the I / O 691 as B # 0-B # 1-B # 2-B # 3-B # 4-I / O691.
  • I / O 692 is recognized in the form of B # 0-B # 1-B # 2-B # 3-B # 5-I / O692.
  • FIG. 7 shows the configuration of the computer 500 and MRA-Switch that can be migrated with the computer 500 of FIG.
  • the computer 700 is connected to the I / O 791 and the I / O 792.
  • VSBr 714 and VSBr 715 of MRA-Switch 7101 are connected to Port 712.
  • VSBr 724 and VSBr 726 of MRA-Switch 7102 are connected to Port 721.
  • VSBr 714 and VSBr 724 are connected by Bus 752 (B # 2)
  • VSBr 715 and VSBr 726 are connected by Bus 755 (B # 5).
  • a link can be shared by a plurality of buses, and the bus 752 (B # 2) and the bus 755 (B # 5) share the link.
  • the OS operating on the computer 700 recognizes the I / O 791 in the form of B # 0-B # 1-B # 2-B # 3-B # 4-I / O791.
  • I / O 792 is recognized in the form of B # 0-B # 1-B # 5-B # 6-B # 7-I / O792. If the I / O 591 and I / O 791 and the I / O 592 and I / O 792 are the same device, the I / O 592 and the I / O 792 are recognized in the same way from the OS. It is possible to perform migration with
  • MRA-Switch 80 having two Virtual Switches inside and having a fixed correspondence between VS Bridge and Port.
  • Port 801 and VSBr 810 are fixedly connected to Port 802 and VSBr 813, respectively.
  • Port 803 and VSBr 811 and VSBr 814 are fixedly connected, and Port 804 and VSBr 812 and VSBr 815 are fixedly connected.
  • the virtual switches configured inside are VS820 and VS821.
  • this MRA-Switch 80 unlike the MRA-Switch 7101, it is not possible to connect two VS Bridges on one Downstream side of one VS to the same Port.
  • VS 820 cannot use two VS Bridges connected to Port 803.
  • FIG. 9 shows the configuration of MRA-Switch according to the embodiment of the present invention.
  • MRA-Switch 90 Port 901 and VSBr 910 and Port 902 and VSBr 913 are fixedly connected to each other. Further, Port 903 and VSBr 911 and VSBr 914 are fixedly connected, and Port 904 and VSBr 912 and VSBr 915 are fixedly connected.
  • Virtual Switch configured internally is VS920 and VS921.
  • VSBr911 and VSBr914 can be connected to VS920. With such a configuration, as in MRA-Switch 7101, it is possible to connect two VS Bridges on one Downstream side of one VS to the same Port.
  • VS Bridge can be moved between VSs by applying the present invention, so it is possible to save a necessary route for migration. It becomes.
  • the flexibility of the connection relationship between the computer and the I / O is added to the MR-IOV specification, so that the degree of freedom of migration can be improved.
  • FIG. 12 shows a computer system connected by three MRA-Switches 101-103.
  • Computers 500 and 1500 are connected to the Ports 1250 and 1251 of the MRA-Switch 101.
  • an I / O 91 is connected to the Port 1252 of the MRA-Switch 101 via the MRA-Switch 102.
  • an I / O 92 is connected to the Port 1253 of the MRA-Switch 101 via the MRA-Switch 103.
  • the illustration of Ports of MRA-Switches 102 and 103 is omitted.
  • the MRA-Switch 101 includes VSVBridge Tables 1301 to 1331 provided for each of the VSBrs 130 to 133 connected to the Ports 1250 to 1253, and the Port Table 135 and VS Table 136, respectively.
  • the computer 500 is connected to the I / O 91 via the MRA-Switch 101, 102, and the computer 1500 connects to the I / O 92 via the MRA-Switch 101, 103. You can see that they are connected.
  • a management terminal 180 is connected to the management units 1240, 1241, and 1242 that manage the MRA-Switches 101 to 103.
  • the management terminal 180 includes a processor that executes a program, a memory that stores the program and various data, and an input / output device that is operated by an administrator (detailed illustration is omitted).
  • MR-PCIM Multi-Root PCI Manager
  • VS Bridge Tables 1311 to 1341, Port Table 135, and VS Table 136 defined in the MR-IOV specification are set.
  • the MR-PCIM may not be executed by the processor of the management terminal 180, but may be executed by, for example, a processor included in the MRA-Switch and included in the management unit.
  • the management units 1240 to 1242 notify the MR-PCIM of events related to the entire MRA-Switch, and acquire statistics such as the flow rate of packets passing through.
  • the MR-PCIM is notified of an event that requires intervention of MR-PCIM in the arbitration of processing for the card.
  • the administrator operates the input / output device of the management terminal 180 to sequentially input the setting information of the MRA-Switches 101, 102, and 103 (S1401 to S1403). That is, for each MRA-Switch, the port number for identifying the port, the VH number, and identification information (VS number) indicating to which Virtual Switch the VS Bridge belongs are input from the management terminal 180.
  • each piece of information is input for each MRA-Switch from the management terminal 180
  • the MR-PCIM 181 is executed by the processor of the management terminal 180
  • each VS Bridge Table 1301 is executed by the management units 1240 to 1242 of each MRA-Switch.
  • ⁇ 1331, Port ⁇ ⁇ ⁇ Table 135, VS Table 136, the input VS number, port number, and VH number are registered and set.
  • the VH number is registered in VS Table 136 (S1501). Further, VS numbers are registered in the VSBridge Tables 1301 to 1331 of the VSBrs 130 to 133, respectively (S1502 to S1505). Further, the Port number is registered in the Port table 135 (S1506).
  • the tables 136, 1301 to 1331, 135 are changed from the previously set VS number, port number, and VH number to the newly set VS number, port number, and VH number.
  • the configuration of each MRA-Switch is changed. In this setting change, it is not necessary to change all of the VS number, the port number, and the VH number, and only one or two can be changed according to the request of the computer system.

Abstract

The present invention makes VS bridges movable between different virtual switches and improves the flexibility of the configuration of an MRA switch. This MRA switch compliant with the MR-IOV specification comprises: a plurality of ports to be connected to I/O devices and each having an identifier assigned thereto; one or a plurality of virtual switches (VSs) each having a unique identifier assigned thereto; a plurality of VS bridges constituting the virtual switches and connecting the virtual switches and the ports; and a storage unit that stores the identifiers of the respective virtual switches to which the VS bridges belong. By changing the identifiers of the virtual switches stored in the storage unit by an instruction from outside, the VS bridges can be moved between different virtual switches.

Description

ＭＲＡスイッチ、それを用いた計算機システム、及びＭＲＡスイッチの設定変更方法MRA switch, computer system using the same, and method for changing MRA switch settings
　本発明は、ＭＲＡスイッチ（Ｍｕｌｔｉ－Ｒｏｏｔ Ａｗａｒｅ（ＭＲＡ） Ｓｗｉｔｃｈ）、それを用いた計算機システム、及びＭＲＡスイッチの設定変更方法に関するものである。 The present invention relates to an MRA switch (Multi-Root Aware (MRA) Switch), a computer system using the switch, and an MRA switch setting changing method.
　ＰＣＩ仕様およびその後継仕様であるＰＣＩ Ｅｘｐｒｅｓｓ仕様は、異なるベンダが提供するデバイスが相互にデータをやりとりして動作するための、汎用のＩ／Ｏ規格として定義され、使用されている。また、Ｍｕｌｔｉ－Ｒｏｏｔ Ｉ／Ｏ Ｖｉｒｔｕａｌｉｚａｔｉｏｎ ａｎｄ Ｓｈａｒｉｎｇ Ｓｐｅｃｉｆｉｃａｔｉｏｎ（ＭＲ－ＩＯＶ）仕様は、複数の計算機間でＰＣＩ Ｅｘｐｒｅｓｓのデバイスを共有するための標準仕様として、ＰＣＩ Ｅｘｐｒｅｓｓ仕様を拡張する形で定義され、例えばブレードサーバなど計算機で用いられている。 The PCI Express and its successor specification, the PCI Express specification, are defined and used as general-purpose I / O standards that allow devices provided by different vendors to exchange data and operate. In addition, the Multi-Root I / O Virtualization and Sharing Specification (MR-IOV) specifications are defined as a standard specification for sharing a PCI Express device among a plurality of computers, for example, by extending the PCI Express specification. Used in computers such as blade servers.
　ＰＣＩ Ｅｘｐｒｅｓｓ仕様では、Ｒｏｏｔ Ｃｏｍｐｌｅｘ（ＲＣ）、Ｓｗｉｔｃｈ、Ｅｎｄ Ｐｏｉｎｔ（ＥＰ）の３つの構成要素が定義されている。Ｒｏｏｔ ＣｏｍｐｌｅｘはＣＰＵやメモリとＩ／Ｏを接続するためのＩ／Ｏのツリーの頂点である。Ｅｎｄ ＰｏｉｎｔはグラフィックのコントローラやＬＡＮ、ファイバーチャネルといった機能を持ったデバイスである。そして、ＳｗｉｔｃｈはＲｏｏｔ Ｃｏｍｐｌｅｘや複数のＥｎｄ Ｐｏｉｎｔ、別のＳｗｉｔｃｈを接続するための構成要素である。Ｒｏｏｔ ＣｏｍｐｌｅｘとＳｗｉｔｃｈ、ＳｗｉｔｃｈとＥｎｄ Ｐｏｉｎｔ、あるいはＲｏｏｔ ＣｏｍｐｌｅｘとＥｎｄ Ｐｏｉｎｔの接続のことを特にＬｉｎｋという。また、Ｉ／Ｏのツリーにおいて、Ｒｏｏｔ Ｃｏｍｐｌｅｘに近い側をＵｐｓｔｒｅａｍ、Ｅｎｄ Ｐｏｉｎｔに近い側をＤｏｗｎｓｔｒｅａｍと呼ぶ。ここでは、ＰＣＩ Ｅｘｐｒｅｓｓ規格に準拠したＳｗｉｔｃｈのことをＰＣＩｅ－Ｓｗｉｔｃｈと表し、後述のＭＲＡ－Ｓｗｉｔｃｈと区別する。 In the PCI Express specification, three components of Root Complex (RC), Switch, and End Point (EP) are defined. Root Complex is the vertex of the I / O tree for connecting the CPU and memory to the I / O. End Point is a device having functions such as a graphic controller, LAN, and fiber channel. Switch is a component for connecting Root Complex, a plurality of End points, and another Switch. The connection between Root Complex and Switch, Switch and End Point, or Root Complex and End Point is particularly called Link. In the I / O tree, the side closer to RootＲComplex is called Upstream, and the side closer to End Point is called Downstream. Here, a switch that complies with the PCI-Express standard is referred to as PCIe-Switch, and is distinguished from MRA-Switch described later.
　ＭＲ－ＩＯＶ仕様に対応したＳｗｉｔｃｈを、Ｍｕｌｔｉ－Ｒｏｏｔ Ａｗａｒｅ Ｓｗｉｔｃｈ（ＭＲＡ－Ｓｗｉｔｃｈ）といい、ＭＲ－ＩＯＶ仕様に対応したＥｎｄ ＰｏｉｎｔをＭＲＡデバイスという。ＭＲＡ－Ｓｗｉｔｃｈは、仮想的なＰＣＩｅ－Ｓｗｉｔｃｈを内部的に構成し、ＰＣＩ ＥｘｐｒｅｓｓデバイスやＭＲＡデバイスを相互に接続する。ＭＲＡ－Ｓｗｉｔｃｈ内の仮想的なＰＣＩｅ－Ｓｗｉｔｃｈのことを特にＶｉｒｔｕａｌ Ｓｗｉｔｃｈ（ＶＳ）という。ＭＲＡ－Ｓｗｉｔｃｈは一般的にＬＳＩとして実装される。 Switch corresponding to MR-IOV specification is called Multi-RootＲAware Switch (MRA-Switch), and End Point corresponding to MR-IOV specification is called MRA device. MRA-Switch internally configures a virtual PCIe-Switch, and connects PCI Express devices and MRA devices to each other. The virtual PCIe-Switch in the MRA-Switch is particularly referred to as Virtual （Switch (VS). MRA-Switch is generally implemented as an LSI.
　ＰＣＩｅ－Ｓｗｉｔｃｈは、複数のＰＣＩ－ＰＣＩ Ｂｒｉｄｇｅとよばれる中継部と複数のＰｏｒｔと呼ばれる外部との接続部を持つ。ＰＣＩｅ－Ｓｗｉｔｃｈの持つＰＣＩ－ＰＣＩ ＢｒｉｄｇｅはＵｐｓｔｒｅａｍ側に１つ、Ｄｏｗｎｓｔｒｅａｍ側に複数である。そしてＰＣＩｅ－Ｓｗｉｔｃｈは、１つのＩ／Ｏのツリーに属し、ＰＣＩ－ＰＣＩ ＢｒｉｄｇｅとＰｏｒｔは１対１に対応する。なお、ＰＣＩｅ－Ｓｗｉｔｃｈは非特許文献１に記載されている。 PCIe-Switch has a plurality of relay units called PCI-PCI Bridges and a plurality of external connection units called Ports. There is one PCI-PCI Bridge of PCIe-Switch, one on the Upstream side and multiple on the Downstream side. PCIe-Switch belongs to one I / O tree, and PCI-PCI Bridge and Port have a one-to-one correspondence. Note that PCIe-Switch is described in Non-Patent Document 1.
　一方、ＭＲＡ－Ｓｗｉｔｃｈは複数のＶｉｒｔｕａｌ Ｓｗｉｔｃｈと複数のＰｏｒｔを持ち、各Ｖｉｒｔｕａｌ Ｓｗｉｔｃｈが複数のＶＳ Ｂｒｉｄｇｅを持つ。各Ｖｉｒｔｕａｌ Ｓｗｉｔｃｈの持つＶＳ ＢｒｉｄｇｅはＰＣＩｅ－Ｓｗｉｔｃｈと同様に、Ｕｐｓｔｒｅａｍ側に１つ、Ｄｏｗｎｓｔｒｅａｍ側に複数である。ＭＲＡ－Ｓｗｉｔｃｈにおいては、複数のＶＳ Ｂｒｉｄｇｅで１つのＰｏｒｔや、接続先とのＬｉｎｋを共有する。ＶＳ ＢｒｉｄｇｅはＰＣＩ－ＰＣＩ Ｂｒｉｄｇｅの機能とＭＲ－ＩＯＶ仕様で追加された機能を持ち、ＯＳ（Ｏｐｅｒａｔｉｎｇ Ｓｙｓｔｅｍ）からはＰＣＩ－ＰＣＩ Ｂｒｉｄｇｅと同様に扱われる。なお、ＭＲＡ－Ｓｗｉｔｃｈ及びそれを用いた電子計算機については、特許文献１及び非特許文献２に記載されている。
ＰＣＩ ＥｘｐｒｅｓｓのＬｉｎｋや、ＰＣＩｅ－Ｓｗｉｔｃｈ内のＵｐｓｔｒｅａｍのＰＣＩ－ＰＣＩ ＢｒｉｄｇｅとＤｏｗｎｓｔｒｅａｍのＰＣＩ－ＰＣＩ Ｂｒｉｄｇｅの間の接続は、ＯＳからはＰＣＩ仕様のＰＣＩ Ｂｕｓとして認識される。 On the other hand, MRA-Switch has a plurality of virtual switches and a plurality of ports, and each virtual switch has a plurality of VS bridges. Each Virtual Switch has one VS Bridge on the Upstream side and multiple VS Bridges on the Downstream side, as in PCIe-Switch. In MRA-Switch, a plurality of VS Bridges share a single Port and a link with a connection destination. The VS Bridge has a PCI-PCI Bridge function and a function added in the MR-IOV specification, and is handled in the same manner as the PCI-PCI Bridge by the OS (Operating System). MRA-Switch and an electronic computer using the MRA-Switch are described in Patent Document 1 and Non-Patent Document 2.
The connection between PCI Express Link and PCI-PCI Bridge of Upstream in PCIe-Switch and PCI-PCI Bridge of Downstream is recognized as PCI bus of PCI specification by the OS.
　一般的なＭＲＡ－Ｓｗｉｔｃｈの構成を図１１に示す。ＭＲＡ－Ｓｗｉｔｃｈ１１０は、ＭＲＡ－Ｓｗｉｔｃｈ１１０全体を管理するための管理ユニット１１０１を持つ。またＭＲＡ－Ｓｗｉｔｃｈ１１０を設定するためのソフトウェアのことをＭｕｌｔｉ－Ｒｏｏｔ ＰＣＩ Ｍａｎａｇｅｒ（ＭＲ－ＰＣＩＭ）という。ＭＲ－ＰＣＩＭ１１１０は管理ユニット１１０１経由でＭＲＡ－Ｓｗｉｔｃｈ１１０内部のＶＳの設定などを行う。ＭＲ－ＰＣＩＭ１１１が設定するテーブルには、ＭＲ－ＩＯＶ仕様で定義されているＶＳ Ｂｒｉｄｇｅ Ｔａｂｌｅ１１０２、Ｐｏｒｔ Ｔａｂｌｅ１１０３、ＶＳ Ｔａｂｌｅ１１０４のようなテーブルがある。これらのテーブルのうち、ＶＳ Ｂｒｉｄｇｅ Ｔａｂｌｅ１１０２はＶＳ Ｂｒｉｄｇｅごとの設定を行うためのテーブルである。これらのテーブルを設定することで、ＭＲＡ－Ｓｗｉｔｃｈ内のＶｉｒｔｕａｌ Ｓｗｉｔｃｈの構造が決定される。 Fig. 11 shows a typical MRA-Switch configuration. The MRA-Switch 110 has a management unit 1101 for managing the entire MRA-Switch 110. The software for setting the MRA-Switch 110 is called Multi-Root－PCI Manager (MR-PCIM). The MR-PCIM 1110 performs VS setting in the MRA-Switch 110 via the management unit 1101. Tables set by the MR-PCIM 111 include tables such as VS Bridge Table 1102, Port Table 1103, and VS Table 1104 defined in the MR-IOV specification. Among these tables, VS | Bridge | Table 1102 is a table for setting for every VS | Bridge. By setting these tables, the structure of Virtual Switch in MRA-Switch is determined.
ＷＯ２０１１/００１５０８（公開）WO2011 / 001508 (open)
　複数のＲＣと複数のＥＰがＭＲＡ－Ｓｗｉｔｃｈによって接続されるシステムは、ＭＲＡ－Ｓｗｉｔｃｈ内のＶｉｒｔｕａｌ Ｓｗｉｔｃｈの構成の柔軟性に大きく左右される。然るに、従来のＭＲＡ－Ｓｗｉｔｃｈでは、ＶＳ ＢｒｉｄｇｅがどのＶｉｒｔｕａｌ Ｓｗｉｔｃｈに属しているかは、ＭＲＡ－ＳｗｉｔｃｈのＬＳＩを設計する時に決定されてしまうので、計算機の稼動時にＭＲＡ－Ｓｗｉｔｃｈの設定を変更することは不可能である。そのため、計算機システム全体の構成の柔軟性に対応できないことがあった。そこで、ＭＲＡ－Ｓｗｉｔｃｈの構成はシステム全体の構成の柔軟性を左右するため、ＭＲＡ－Ｓｗｉｔｃｈの構成の柔軟性を向上させることが望まれる。 A system in which a plurality of RCs and a plurality of EPs are connected by MRA-Switch is greatly influenced by the flexibility of the configuration of Virtual Switch in MRA-Switch. However, in the conventional MRA-Switch, which Virtual Switch the VS Bridge belongs to is determined at the time of designing the MRA-Switch LSI, so changing the setting of the MRA-Switch during the operation of the computer is not possible. Impossible. For this reason, the configuration flexibility of the entire computer system may not be supported. Therefore, since the configuration of the MRA-Switch determines the flexibility of the configuration of the entire system, it is desired to improve the flexibility of the configuration of the MRA-Switch.
　更に、ＭＲ－ＩＯＶ仕様では、ＶＳの中で、ＶＳ ＢｒｉｄｇｅとＰｏｒｔの接続関係を変更できるようになっているが、これをＬＳＩに実装するのが困難である。何故なら、ＶＳ ＢｒｉｄｇｅとＰｏｒｔの間に全対全の配線を行わなければならず、配線量が増加するからである。ＶＳ ＢｒｉｄｇｅとＰｏｒｔの対応を固定にすることで、配線量を減らすことは可能であるが、これではＭＲＡ－Ｓｗｉｔｃｈの構成の柔軟性は大幅に低下してしまう。 Furthermore, in the MR-IOV specification, the connection relationship between VS Bridge and Port can be changed in VS, but it is difficult to mount this on LSI. This is because all-to-all wiring must be performed between VS Bridge and Port, and the amount of wiring increases. Although the amount of wiring can be reduced by fixing the correspondence between VS Bridge and Port, this greatly reduces the flexibility of the MRA-Switch configuration.
　本発明の目的は、異なる仮想スイッチ（ＶＳ）の間でＶＳ Ｂｒｉｄｇｅを移動可能として、ＭＲＡスイッチの構成の柔軟性を向上させたＭＲＡスイッチ、それを用いた計算機システム、及びＭＲＡスイッチの設定変更方法を提供することにある。 An object of the present invention is to enable the movement of VS Bridge between different virtual switches (VS) to improve the flexibility of the configuration of the MRA switch, a computer system using the MRA switch, and a method for changing the setting of the MRA switch Is to provide.
　本発明に係るＭＲＡスイッチは、好ましくは、ＭＲ－ＩＯＶ仕様に基づくＭＲＡスイッチであって、
Ｉ／Ｏデバイスに接続される、それぞれ識別子が付与された複数のポートと、
それぞれ固有の識別子が付与された、１または複数の仮想スイッチ（ＶＳ）と、
該仮想スイッチを構成し、該仮想スイッチと該ポートを接続する、複数のＶＳ Ｂｒｉｄｇｅと、
各該ＶＳ Ｂｒｉｄｇｅが属する該仮想スイッチの識別子を記憶する記憶部を有し、
外部からの指示によって、該記憶部に記憶される該仮想スイッチの識別子を変更することにより、異なる仮想スイッチ間で該ＶＳ Ｂｒｉｄｇｅの移動を可能とすることを特徴とするＭＲＡスイッチとして構成される。 The MRA switch according to the present invention is preferably an MRA switch based on the MR-IOV specification,
A plurality of ports each connected with an identifier connected to an I / O device;
One or more virtual switches (VS) each having a unique identifier;
A plurality of VS Bridges constituting the virtual switch and connecting the virtual switch and the port;
A storage unit for storing the identifier of the virtual switch to which each VS Bridge belongs;
The VS Bridge can be moved between different virtual switches by changing the identifier of the virtual switch stored in the storage unit according to an instruction from the outside.
　好ましい例では、前記記憶部は、各々の該ＶＳ Ｂｒｉｄｇｅ内に実装され、該ＶＳ Ｂｒｉｄｇｅごとに、該仮想スイッチの識別子を設定するレジスタである。
また、好ましくは、前記記憶部は、該ＶＳ Ｂｒｉｄｇｅの接続されている該ポートの識別子を設定するＰ＃レジスタと、該ＶＳ Ｂｒｉｄｇｅの接続されている該ポートがＶＳ Ｂｒｉｄｇｅを識別するためのＶｉｒｔｕａｌ Ｈｉｅｒａｒｃｈｙ（ＶＨ）を設定するＶＨ＃レジスタと、該ＶＳ Ｂｒｉｄｇｅが属する仮想スイッチの識別子を設定するＶＳ＃レジスタとを含むＶＳ Ｂｒｉｄｇｅテーブルを記憶する前記ＭＲＡスイッチとして構成される。
また、好ましくは、該ＭＲＡスイッチは、該ＭＲＡスイッチの管理を行う管理ユニットを備え、該管理ユニットに接続される管理端末からの指示によって、前記記憶部又は前記レジスタの内容が書き換えられる。 In a preferred example, the storage unit is a register that is mounted in each of the VS Bridges and sets an identifier of the virtual switch for each VS Bridge.
Preferably, the storage unit includes a P # register that sets an identifier of the port to which the VS Bridge is connected, and a Virtual Hierarchy for the port to which the VS Bridge is connected to identify the VS Bridge. The MRA switch stores a VS Bridge table including a VH # register for setting (VH) and a VS # register for setting an identifier of a virtual switch to which the VS Bridge belongs.
Preferably, the MRA switch includes a management unit that manages the MRA switch, and the contents of the storage unit or the register are rewritten by an instruction from a management terminal connected to the management unit.
　本発明に係る計算機システムは、好ましくは、ＭＲ－ＩＯＶ仕様に基づく、１又は接続関係にある複数のＭＲＡスイッチと、１又は複数のいずれかの該ＭＲＡスイッチのポートに接続された１又は複数のコンピュータと、１又は複数のいずれかの該ＭＲＡスイッチのポートに接続された１又は複数のＩ／Ｏデバイスを含む計算機システムであって、
該ＭＲＡスイッチは、Ｉ／Ｏデバイスに接続される、それぞれ識別子が付与された複数のポートと、それぞれ固有の識別子が付与された、１または複数の仮想スイッチ（ＶＳ）と、該仮想スイッチを構成し、該仮想スイッチと該ポートを接続する、複数のＶＳ Ｂｒｉｄｇｅと、各該ＶＳ Ｂｒｉｄｇｅが属する該仮想スイッチの識別子を記憶する記憶部を有し、
外部からの指示によって、該記憶部に記憶される該仮想スイッチの識別子を変更することにより、異なる仮想スイッチ間で該ＶＳ Ｂｒｉｄｇｅの移動を可能とすることを特徴とする計算機システムとして構成される。 The computer system according to the present invention is preferably based on the MR-IOV specification, one or a plurality of MRA switches having a connection relationship and one or a plurality of one or a plurality of MRA switches connected to ports of the one or a plurality of MRA switches. A computer system comprising a computer and one or more I / O devices connected to a port of any one or more of the MRA switches,
The MRA switch is connected to an I / O device and includes a plurality of ports each assigned an identifier, one or more virtual switches (VS) each assigned a unique identifier, and the virtual switch A plurality of VS Bridges that connect the virtual switch and the port, and a storage unit that stores an identifier of the virtual switch to which each VS Bridge belongs,
By changing the identifier of the virtual switch stored in the storage unit according to an instruction from the outside, the computer system is configured to enable the movement of the VS Bridge between different virtual switches.
　好ましい例では、前記記憶部は、各々の該ＶＳ Ｂｒｉｄｇｅ内に実装され、該ＶＳ Ｂｒｉｄｇｅごとに、該仮想スイッチの識別子を設定するレジスタである。
また、好ましくは、前記記憶部は、該ＶＳ Ｂｒｉｄｇｅの接続されている該ポートの識別子を設定するＰ＃レジスタと、該ＶＳ Ｂｒｉｄｇｅの接続されている該ポートがＶＳ Ｂｒｉｄｇｅを識別するためのＶｉｒｔｕａｌ Ｈｉｅｒａｒｃｈｙ（ＶＨ）を設定するＶＨ＃レジスタと、該ＶＳ Ｂｒｉｄｇｅが属する仮想スイッチの識別子を設定するＶＳ＃レジスタとを含むＶＳ Ｂｒｉｄｇｅテーブルを記憶する前記計算機システムである。 In a preferred example, the storage unit is a register that is mounted in each of the VS Bridges and sets an identifier of the virtual switch for each VS Bridge.
Preferably, the storage unit includes a P # register that sets an identifier of the port to which the VS Bridge is connected, and a Virtual Hierarchy for the port to which the VS Bridge is connected to identify the VS Bridge. The computer system stores a VS Bridge table including a VH # register for setting (VH) and a VS # register for setting an identifier of a virtual switch to which the VS Bridge belongs.
　また、好ましくは、各前記ＭＲＡスイッチは、該ＭＲＡスイッチの管理を行う管理ユニットを備え、更に、該管理ユニットに接続される管理端末を有し、該管理端末からの指示に従って、前記記憶部又は前記レジスタの内容が書き換えられる前記計算機システムである。 Preferably, each of the MRA switches includes a management unit that manages the MRA switch, and further includes a management terminal connected to the management unit. According to an instruction from the management terminal, the storage unit or The computer system in which the contents of the register are rewritten.
　本発明に係るＭＲＡスイッチの設定変更方法は、好ましくは、ＭＲ－ＩＯＶ仕様に基づくＭＲＡスイッチの設定変更方法であって、
該ＭＲＡスイッチは、Ｉ／Ｏデバイスに接続される、それぞれ識別子が付与された複数のポートと、それぞれ固有の識別子が付与された、１または複数の仮想スイッチ（ＶＳ）と、該仮想スイッチを構成し、該仮想スイッチと該ポートを接続する、複数のＶＳ Ｂｒｉｄｇｅと、各該ＶＳ Ｂｒｉｄｇｅが属する該仮想スイッチの識別子を記憶する記憶部と、該ＭＲＡスイッチの管理を行う管理ユニットを有し、
該管理ユニットに接続される管理端末から指示によって、該記憶部に記憶される該仮想スイッチの識別子を変更することにより、異なる仮想スイッチ間で該ＶＳ Ｂｒｉｄｇｅの移動を可能とするＭＲＡスイッチの設定変更方法として構成される。 The MRA switch setting changing method according to the present invention is preferably an MRA switch setting changing method based on MR-IOV specifications,
The MRA switch is connected to an I / O device and includes a plurality of ports each assigned an identifier, one or more virtual switches (VS) each assigned a unique identifier, and the virtual switch A plurality of VS Bridges that connect the virtual switch and the port, a storage unit that stores an identifier of the virtual switch to which each VS Bridge belongs, and a management unit that manages the MRA switch,
Changing the setting of the MRA switch that enables movement of the VS Bridge between different virtual switches by changing the identifier of the virtual switch stored in the storage unit according to an instruction from the management terminal connected to the management unit Configured as a method.
　好ましい例では、前記記憶部は、該ＶＳ Ｂｒｉｄｇｅの接続されている該ポートの識別子を設定するＰ＃レジスタと、該ＶＳ Ｂｒｉｄｇｅの接続されている該ポートがＶＳ Ｂｒｉｄｇｅを識別するためのＶｉｒｔｕａｌ Ｈｉｅｒａｒｃｈｙ（ＶＨ）を設定するＶＨ＃レジスタと、該ＶＳ Ｂｒｉｄｇｅが属する仮想スイッチの識別子を設定するＶＳ＃レジスタとを含むＶＳ Ｂｒｉｄｇｅテーブルを記憶しており、
前記管理端末から、該Ｐ＃レジスタ、該ＶＨ＃レジスタ、該ＶＳ＃レジスタのいずれかにセットすべき識別子を入力し、
前記ＭＲＡスイッチの前記管理ユニットは、該管理端末から入力された識別子を該当する、該Ｐ＃レジスタ、該ＶＨ＃レジスタ、該ＶＳ＃レジスタに書き込むことを特徴とする前記ＭＲＡスイッチの設定変更方法として構成される。 In a preferred example, the storage unit includes a P # register that sets an identifier of the port to which the VS Bridge is connected, and a Virtual Hierarchy (for identifying the VS Bridge by the port to which the VS Bridge is connected). VH # register for setting VH) and a VS Bridge table including a VS # register for setting an identifier of a virtual switch to which the VS Bridge belongs,
From the management terminal, an identifier to be set in any of the P # register, the VH # register, and the VS # register is input,
As the MRA switch setting changing method, the management unit of the MRA switch writes the identifier input from the management terminal to the corresponding P # register, the VH # register, and the VS # register. Composed.
　本発明によれば、コンピュータの稼働後でも、異なる仮想スイッチ（ＶＳ）の間でＶＳ Ｂｒｉｄｇｅを移動して設定することが可能である。これにより従来のＭＲＡスイッチに比べて、ＶＳ（Ｖｉｒｔｕａｌ Ｓｗｉｔｃｈ）の持つＶＳ Ｂｒｉｄｇｅを増減可能にでき、ＭＲＡスイッチの柔軟性を向上させることが可能である。
また、ＶＳ ＢｒｉｄｇｅとＰｏｒｔの関係を固定にして、配線の問題を回避した場合でも、同一のＰｏｒｔに属しているＶＳ Ｂｒｉｄｇｅを複数のＶｉｒｔｕａｌ Ｓｗｉｔｃｈの間で移動させることができるため、ＭＲ－ＩＯＶ仕様とは独立に柔軟性を確保することが可能である。 According to the present invention, it is possible to move and set the VS Bridge between different virtual switches (VS) even after the computer is operating. As a result, the VS Bridge of VS (Virtual Switch) can be increased or decreased as compared with the conventional MRA switch, and the flexibility of the MRA switch can be improved.
In addition, even when the relationship between VS Bridge and Port is fixed and the wiring problem is avoided, VS Bridge belonging to the same Port can be moved between a plurality of Virtual Switches. Independently, it is possible to ensure flexibility.
本発明の一実施例（実施例１）におけるＭＲＡ－Ｓｗｉｔｃｈの構成を示す図。The figure which shows the structure of MRA-Switch in one Example (Example 1) of this invention. 実施例１における、変更されたＭＲＡ－Ｓｗｉｔｃｈの構成を示す図。FIG. 3 is a diagram showing a modified MRA-Switch configuration in the first embodiment. ＶＳ ＢｒｉｄｅとＰｏｒｔの接続関係が固定であるＭＲＡ－Ｓｗｉｔｃｈの例を示す図。The figure which shows the example of MRA-Switch in which the connection relation of VS Bride and Port is fixed. 実施例２における、ＶＳ ＢｒｉｄｅとＰｏｒｔの接続関係が固定であるＭＲＡ－Ｓｗｉｔｃｈを示す図。The figure which shows MRA-Switch in which the connection relation of VS Bride and Port in Example 2 is fixed. 実施例３における、ＭＲＡ－ＳｗｉｔｃｈによってコンピュータとＩ／Ｏが接続される計算機システムを示す図。FIG. 10 is a diagram showing a computer system in which a computer and I / O are connected by MRA-Switch in the third embodiment. 実施例３における、ＭＲＡ－ＳｗｉｔｃｈによってコンピュータとＩ／Ｏが接続される計算機システムを示す図。FIG. 10 is a diagram showing a computer system in which a computer and I / O are connected by MRA-Switch in the third embodiment. 実施例３における、ＭＲＡ－ＳｗｉｔｃｈによってコンピュータとＩ／Ｏが接続される計算機システムを示す図。FIG. 10 is a diagram showing a computer system in which a computer and I / O are connected by MRA-Switch in the third embodiment. 固定的な接続関係を有するＭＲＡ－Ｓｗｉｔｃｈの構成を示す図。The figure which shows the structure of MRA-Switch which has a fixed connection relationship. 実施例３における、ＭＲＡ－Ｓｗｉｔｃｈの構成を示す図。FIG. 10 is a diagram showing a configuration of MRA-Switch in the third embodiment. 実施例３における、ＭＲＡ－Ｓｗｉｔｃｈを有する計算機システムの構成例を示す図。FIG. 10 is a diagram illustrating a configuration example of a computer system having MRA-Switch in the third embodiment. 一般的なＭＲＡ－Ｓｗｉｔｃｈの構成を示す図。The figure which shows the structure of general MRA-Switch. 実施例４におけるＭＲＡ－Ｓｗｉｔｃｈの構成の設定変更の説明に供する計算機システムの構成例を示す図。FIG. 10 is a diagram illustrating a configuration example of a computer system for explaining setting change of a configuration of MRA-Switch in the fourth embodiment. 実施例４におけるＭＲＡ－Ｓｗｉｔｃｈ１０１の構成を示す図。FIG. 10 is a diagram showing a configuration of MRA-Switch 101 in the fourth embodiment. 実施例４におけるＭＲＡ－Ｓｗｉｔｃｈの設定変更の操作動作フロー図。FIG. 14 is an operation flowchart of MRA-Switch setting change according to the fourth embodiment. 実施例４におけるＭＲＡ－Ｓｗｉｔｃｈの設定動作のフロー図。FIG. 10 is a flowchart of MRA-Switch setting operation in the fourth embodiment.
　以下、図面を参照して、本発明の好ましい実施例について説明する。 Hereinafter, preferred embodiments of the present invention will be described with reference to the drawings.
図１は、一実施例によるＭＲＡ－Ｓｗｉｔｃｈの構成を示す。
ＭＲＡ－Ｓｗｉｔｃｈ１は、外部のデバイスが接続される複数のＰｏｒｔ（ポート）１２０，１２０を有する。ここで、Ｐｏｒｔ１２０及び１２１にはそれぞれ、Ｐ＃０及びＰ＃１の識別子が付与されている。ＭＲＡ－Ｓｗｉｔｃｈ１はその内部に形成される２つのＶＳ（Ｖｉｒｔｕａｌ Ｓｗｉｔｃｈ）１９１、ＶＳ１９２を有する。ＶＳ１９１，１９２にはそれぞれ、識別子ＶＳ＃０、ＶＳ＃１が付与されている。ＭＲＡ－Ｓｗｉｔｃｈ１はまた、３つのＶＳ Ｂｒｉｄｇｅ（ＶＳＢｒ）１４０～１４２を有する。これらのＶＳ Ｂｒｉｄｇｅのうち、ＶＳＢｒ１４０及び１４１はＶＳ＃０に属し、ＶＳＢｒ１４２はＶＳ＃１に属している。 FIG. 1 shows the configuration of MRA-Switch according to an embodiment.
MRA-Switch 1 has a plurality of Ports (ports) 120 and 120 to which external devices are connected. Here, the identifiers P # 0 and P # 1 are assigned to the Ports 120 and 121, respectively. MRA-Switch1 has two VSs (Virtual Switch) 191 and VS192 formed therein. Identifiers VS # 0 and VS # 1 are assigned to VS191 and 192, respectively. MRA-Switch1 also has three VS Bridges (VSBr) 140-142. Of these VS Bridges, VSBrs 140 and 141 belong to VS # 0, and VSBr142 belongs to VS # 1.
　各ＶＳＢｒ１４０～１４２は、ＶＳ Ｂｒｉｄｇｅ テーブルを有する。ＶＳ Ｂｒｉｄｇｅ テーブルは、各ＶＳＢｒ１４０～１４２に実装されたメモリ（図示せず）内に形成される。このＶＳ Ｂｒｉｄｇｅテーブルは、そのＶＳＢｒの接続されているＰｏｒｔを設定するためのＰ＃レジスタ１５０、そのＶＳＢｒ１４０の接続されているＰｏｒｔがＶＳＢｒを識別するためのＶｉｒｔｕａｌ Ｈｉｅｒａｒｃｈｙ（ＶＨ）を設定するＶＨ＃レジスタ１５１、及びそのＶＳＢｒが所属するＶｉｒｔｕａｌ Ｓｗｉｔｃｈを設定するためのＶＳ＃レジスタ１５２を有する。 Each VSBr 140-142 has a VS Bridge Bridge table. The VS Bridge table is formed in a memory (not shown) mounted on each VSBr 140-142. This VS Bridge table includes a P # register 150 for setting the Port to which the VSBr is connected, and a VH # register for setting the Virtual Hierarchy (VH) for identifying the VSBr by the Port to which the VSBr 140 is connected. 151 and a VS # register 152 for setting the Virtual Switch to which the VSBr belongs.
　ＶＳ＃レジスタ１５２は本発明に特徴的な構成であり、このレジスタ１５２によって実現される機能ないし作用によって、異なるＶｉｒｔｕａｌ Ｓｗｉｔｃｈの間でＶＳ Ｂｒｉｄｇｅの移動（再配置）を行うことが可能となり、ＭＲＡ－Ｓｗｉｔｃｈの構成の柔軟に変更することが可能となる。その意味では、このＶＳＢｒをＲｅｌｏｃａｔａｂｌｅ（再配置可能な） ＶＳ Ｂｒｉｄｇｅ（ＲＶＳＢｒ）と言うことができる。 The VS # register 152 is a configuration characteristic of the present invention, and the function or operation realized by the register 152 makes it possible to move (rearrange) VS Bridge between different Virtual Switches. It is possible to change the configuration of the switch flexibly. In that sense, this VSBr can be said to be Relocatable (relocatable) VS Bridge (RVSBr).
　ＶＳＢｒ１４０のＶＳ＃レジスタ１５２にはＶＳ＃０が設定されており、識別子ＶＳ＃０を持つＶＳ１９１に接続されている。また、ＶＳＢｒ１４２のＶＳ＃レジスタ１５２にはＶＳ＃１が設定されており、ＶＳ１９２に接続されている。同様に、ＶＳＢｒ１４１はＶＳ１９１に接続されている。
また、ＶＳＢｒ１４０のＰ＃レジスタ１５０にはＰ＃０が設定され、ＶＨ＃レジスタ１５１にはＶＨ＃０が設定されて、Ｐｏｒｔ１２０のＶｉｒｔｕａｌ Ｈｉｅｒａｒｃｈｙ１６０と接続されている。同様に、ＶＳＢｒ１４１はＰｏｒｔ１２１のＶＨ１６１と接続され、ＶＳＢｒ１４２はＰｏｒｔ１２１のＶＨ１６２と接続されている。 VS # 0 is set in the VS # register 152 of the VSBr140, and is connected to the VS191 having the identifier VS # 0. Also, VS # 1 is set in the VS # register 152 of the VSBr 142 and is connected to the VS192. Similarly, VSBr 141 is connected to VS 191.
Further, P # 0 is set in the P # register 150 of the VSBr 140, and VH # 0 is set in the VH # register 151, which is connected to the Virtual Hierarchy 160 of the Port 120. Similarly, VSBr 141 is connected to VH 161 of Port 121, and VSBr 142 is connected to VH 162 of Port 121.
　図２は、図１の状態からＶＳＢｒ１４２の接続が変更された状態のＭＲＡ－Ｓｗｉｔｃｈの構成を示す。即ち、図１の状態では、ＶＳＢｒ１４２におけるＶＳ＃レジスタ１５２にはＶＳ＃１が設定されてＶＳ１９２に接続されていたが、図２の状態では、ＶＳＢｒ１４２のＶＳ＃レジスタ１５２にはＶＳ＃０が設定され、ＶＳ１９１に接続されている。
因みに、従来のＭＲＡ－Ｓｗｉｔｃｈ１では、ＶＳ Ｂｒｄｉｇｅから見て接続されるＶｉｒｔｕａｌ Ｓｗｉｔｃｈが固定であった。しかし、本発明によるＭＲＡ－Ｓｗｉｔｃｈでは、ＶＳ Ｂｒｄｉｇｅから見て接続されるＶｉｒｔｕａｌ Ｓｗｉｔｃｈは、ＶＳ＃レジスタ１５２によって変更可能となり、ＶＳ Ｂｒｉｄｇｅを異なるＶＳの間で移動することが可能となる。このように、ＶＳ１９１とＶＳ１９２との間でＶＳＢｒ１４２を移動させることで、Ｖｉｒｔｕａｌ Ｓｗｉｔｃｈの構成の自由度を上げることができる。 FIG. 2 shows the configuration of the MRA-Switch in a state where the connection of the VSBr 142 is changed from the state of FIG. That is, in the state of FIG. 1, VS # 1 in VSBr142 is set to VS # 1 and connected to VS192, but in the state of FIG.2, VS # 0 in VSBr142 is set to VS # 0. And connected to the VS 191.
Incidentally, in the conventional MRA-Switch1, the Virtual Switch connected as viewed from the VS Bridge is fixed. However, in MRA-Switch according to the present invention, the Virtual Switch connected as viewed from the VS Bridge can be changed by the VS # register 152, and the VS Bridge can be moved between different VSs. In this way, by moving the VSBr 142 between the VS 191 and the VS 192, the degree of freedom in the configuration of the Virtual Switch can be increased.
　本発明では、ＶＳ Ｂｒｉｄｇｅごとに、所属するＶｉｒｔｕａｌ Ｓｗｉｔｃｈを変更可能に設定できることが重要である。そのためのＶＳ＃レジスタの実装の仕方やＭＲ－ＰＣＩＭからの認識の方法は種々考えられる。例えば、ＶＳ＃レジスタのＬＳＩ内での実装位置について言えば、必ずしも図１や図２のように、対応するＶＳ Ｂｒｉｄｇｅ内にそれぞれ実装するとは限らない。他の例として例えば、ＭＲＡ－Ｓｗｉｔｃｈ内のＶＳ Ｂｒｉｄｇｅ全てのＶＳ＃レジスタを１箇所に纏めて実装することも可能である。なお、ＶＳ＃レジスタはＶＳ Ｂｒｉｄｇｅごとに設定するレジスタであるため、図１及び図２の例では、ＶＳ Ｂｒｉｄｇｅ内に実装されている。 In the present invention, for each VS Bridge, it is important that the Virtual 設定 Switch to which the VS Bridge belongs can be set to be changeable. For this purpose, various methods of mounting the VS # register and methods of recognition from the MR-PCIM are conceivable. For example, as for the mounting position of the VS # register in the LSI, it is not always mounted in the corresponding VS Bridge as shown in FIGS. As another example, for example, all VS # Bridge VS # registers in MRA-Switch can be mounted in one place. Since the VS # register is a register that is set for each VSｇBridge, the VS # register is mounted in the VS Bridge in the examples of FIGS.
　また本発明では、ＭＲ－ＰＣＩＭから見てＶＳ＃レジスタがＶＳ Ｂｒｉｄｇｅ Ｔａｂｌｅに含まれている必要もなく、例えばＶＳ＃レジスタを１箇所に固めたＶＳ＃レジスタＴａｂｌｅというレジスタ空間を持つことも可能である。ＶＳ＃レジスタはＶＳ Ｂｒｉｄｇｅごとに設定するレジスタである。また、ＭＲ－ＩＯＶ仕様ではＶＳ Ｂｒｉｄｇｅ Ｔａｂｌｅの１ＶＳ Ｂｒｉｄｇｅごとの大きさが自由に定められる。そのため、ＶＳ＃レジスタをＶＳ Ｂｒｉｄｇｅ Ｔａｂｌｅに追加することは、ＭＲ－ＩＯＶ仕様からの変更量を最小限にする形で実現可能である。このような理由により、図１及び図２の例においては、ＶＳ＃レジスタをＶＳ Ｂｒｉｄｇｅ Ｔａｂｌｅに含ませて実装するのが好ましい。 In the present invention, the VS # register does not need to be included in the VS Bridge Table when viewed from the MR-PCIM. For example, it is possible to have a register space called a VS # register Table in which the VS # registers are consolidated in one place. is there. The VS # register is a register that is set for each VS Bridge. In the MR-IOV specifications, the size of VSＶBridge Table for each 1VS Bridge is freely determined. Therefore, adding the VS # register to VS Bridge Table can be realized in a form that minimizes the amount of change from the MR-IOV specification. For this reason, in the example of FIGS. 1 and 2, it is preferable to mount the VS # register by including it in VS Bridge Table.
　本発明において、ＭＲＡ－Ｓｗｉｔｃｈ内のＶＳの構成を設定するためのＶＳ＃レジスタは、Ｐ＃レジスタ及びＶＨ＃レジスタと同様に、ＶＳ Ｂｒｉｄｇｅ Ｔａｂｌｅ内に設置されるので、例えば、ＭＲＡ－Ｓｗｉｔｃｈの電源投入時にＶＳの構成を行う時には、Ｐ＃レジスタやＶＨ＃レジスタと同様に、ＶＳ＃レジスタを設定することが可能である。 In the present invention, the VS # register for setting the VS configuration in the MRA-Switch is installed in the VSＶBridge Table similarly to the P # register and the VH # register. When configuring the VS at the time of input, it is possible to set the VS # register in the same manner as the P # register and VH # register.
　ここで、デバイス（Ｄｅｖｉｃｅ）番号について述べる。Ｄｅｖｉｃｅ番号はＰＣＩ－ＰＣＩ Ｂｒｉｄｇｅに付与される識別子の１つであり、ＰＣＩｅ－ＳｗｉｔｃｈにおいてはＤｏｗｎｓｔｒｅａｍ側のＰＣＩ－ＰＣＩ Ｂｒｉｄｇｅを区別するための情報として用いられる。ＭＲＡ－Ｓｗｉｔｃｈの各ＶＳにおいても、Ｄｅｖｉｃｅ番号はＤｏｗｎｓｔｒｅａｍ側のＶＳ Ｂｒｉｄｇｅごとに一意な識別子である。これに対して、ＰＣＩ Ｅｘｐｒｅｓｓ仕様では「ＰＣＩｅ－ＳｗｉｔｃｈのＤｏｗｎｓｔｒｅａｍ側のＰＣＩ－ＰＣＩ ＢｒｉｄｇｅのＤｅｖｉｃｅ番号は設計者が何らかの方法で決めてよい」とされている。そのため、ＭＲＡ－Ｓｗｉｔｃｈの構成を設定する際には、Ｄｏｗｎｓｔｒｅａｍ側のＶＳ ＢｒｉｄｇｅのＤｅｖｉｃｅ番号を各ＶＳにおいて一意であるように任意に設定するものであり、これは本発明を実施する際にも同様である。
このように、本発明によれば、ＭＲ－ＩＯＶ仕様を拡張する形でＭＲＡ－Ｓｗｉｔｃｈの構成の柔軟性を実現出来る。 Here, the device number will be described. The Device number is one of identifiers assigned to the PCI-PCI Bridge, and is used as information for distinguishing the PCI-PCI Bridge on the Downstream side in the PCIe-Switch. In each VS of MRA-Switch, the device number is a unique identifier for each VS Bridge on the Downstream side. In contrast, the PCI Express specification states that “the designer may determine the device number of the PCI-PCI Bridge on the downstream side of the PCIe-Switch by some method”. Therefore, when setting the MRA-Switch configuration, the device number of the VS Bridge on the Downstream side is arbitrarily set so as to be unique in each VS, and this is the same when the present invention is implemented. It is.
As described above, according to the present invention, the flexibility of the MRA-Switch configuration can be realized by extending the MR-IOV specification.
　ＭＲ－ＩＯＶ仕様では、ＶＳ ＢｒｉｄｇｅとＰｏｒｔの接続関係を変更できるようになっているが、配線の関係からＬＳＩに実装することは必ずしも容易ではない。この課題に対して、ＶＳ ＢｒｉｄｇｅとＰｏｒｔの対応を固定にすることで、配線量を減らすことが有力な手段となり得る。このような手段を適用する場合でも本発明は有効となる。 In the MR-IOV specification, the connection relationship between VS Bridge and Port can be changed, but it is not always easy to mount on LSI due to the wiring relationship. For this problem, fixing the correspondence between VS Bridge and Port can be an effective means to reduce the amount of wiring. The present invention is effective even when such means is applied.
　図３は、ＭＲＡ－Ｓｗｉｔｃｈ１´のＶＳ ＢｒｉｄｅとＰｏｒｔの接続関係が固定であるＭＲＡ－Ｓｗｉｔｃｈの構成例を示す。ＶＳＢｒ１４０´はＰｏｒｔ１２０に接続され、ＶＳＢｒ１４１´とＶＳＢｒ１４２´はＰｏｒｔ１２１に接続されており、その接続関係は固定である。このＭＲＡ－Ｓｗｉｔｃｈ１では、ＶＳ１９２はＰｏｒｔ１２０´に接続されているＶＳ Ｂｒｉｄｇｅを利用できない。 FIG. 3 shows a configuration example of MRA-Switch in which the connection relationship between VS Bridge and Port of MRA-Switch 1 ′ is fixed. The VSBr 140 ′ is connected to the Port 120, and the VSBr 141 ′ and the VSBr 142 ′ are connected to the Port 121, and the connection relationship is fixed. In this MRA-Switch1, VS 192 cannot use VS Bridge connected to Port 120 ′.
　図４は、図３のＭＲＡ－Ｓｗｉｔｃｈに対して、本発明を適用したＭＲＡ－Ｓｗｉｔｃｈの構成例を示す。
図３のＶＳ ＢｒｉｄｇｅとＰｏｒｔの関係と同様に、ＶＳＢｒ１４０はＰｏｒｔ１２０に接続され、ＶＳＢｒ１４１とＶＳＢｒ１４２はＰｏｒｔ１２１に接続されており、その関係は固定となる。しかし、ＶＳＢｒ１４０はＶＳ＃レジスタ１５２を設定することで、所属するＶｉｒｔｕａｌ Ｓｗｉｔｃｈが決定されるので、ＶＳ１９１とＶＳ１９２の何れかに接続するかを設定することが可能である。ＶＳＢｒ１４０をＶＳ１９２に接続すると、ＶＳ１９２からはＰｏｒｔ１２０に接続されているＶＳＢｒ１４０が利用できることになる。
このように、本発明によれば、ＶＳ ＢｒｉｄｇｅとＰｏｒｔの対応関係を固定にした場合でも、ＭＲ－ＩＯＶ仕様とは独立に自由度を得ることが出来る。 FIG. 4 shows a configuration example of MRA-Switch in which the present invention is applied to the MRA-Switch of FIG.
Similar to the relationship between VS Bridge and Port in FIG. 3, VSBr 140 is connected to Port 120, and VSBr 141 and VSBr 142 are connected to Port 121, and the relationship is fixed. However, since the virtual switch to which the VSBr 140 belongs is determined by setting the VS # register 152, it is possible to set whether to connect to the VS 191 or the VS 192. When the VSBr 140 is connected to the VS 192, the VSBr 140 connected to the Port 120 can be used from the VS 192.
As described above, according to the present invention, even when the correspondence between VS Bridge and Port is fixed, a degree of freedom can be obtained independently of the MR-IOV specification.
　本発明によるＭＲＡ－Ｓｗｉｔｃｈを適用した計算機システムの例について述べる。
あるコンピュータ上で動作しているＯＳやアプリケーション、サービスを別のコンピュータ上に移動させることをマイグレーションという。マイグレーションは、ハードウェアの保守や交換等ためにコンピュータを停止させなければならないが、アプリケーションやサービスは稼動させてマイグレーションを行いたい。 An example of a computer system to which MRA-Switch according to the present invention is applied will be described.
Moving an OS, application, or service running on a computer to another computer is called migration. Migration involves stopping the computer for hardware maintenance or replacement, but wants to migrate by running applications and services.
　マイグレーションの際には、移動前のコンピュータと移動後のコンピュータでＯＳの認識するハードウェアの情報が一致していなければならない。ハードウェア情報が一致していない場合は、ハードウェア構成が変化したことになり、ＯＳが正しく立ち上がらないといった問題や、ＯＳが立ち上がってもネットワークのＩＰアドレスが変わってしまうといった問題が生じる。
ＯＳには、Ｉ／Ｏを認識する際にＩ／Ｏ固有の識別子（Ｆｉｂｒｅ ＣｈａｎｎｅｌのＷｏｒｌｄ Ｗｉｄｅ Ｎａｍｅなど）だけでなく、ＰＣＩのツリー構造でＲｏｏｔ ＣｏｍｐｌｅｘからＩ／Ｏに至る経路上の情報も、識別情報として利用するものがある。このようなＯＳをマイグレーションする際には、経路上の情報も移動前と移動後で一致していなければならない。 At the time of migration, hardware information recognized by the OS on the computer before movement and the computer after movement must match. If the hardware information does not match, the hardware configuration has changed, and problems such as the OS not starting up correctly and the network IP address changing even when the OS starts up occur.
The OS recognizes not only an I / O-specific identifier (such as Fiber Channel's World Wide Name) when recognizing I / O, but also information on the route from the root complex to the I / O in the PCI tree structure. Some are used as identification information. When migrating such an OS, the information on the route must match before and after the movement.
　図１０は、ＭＲＡ－Ｓｗｉｔｃｈ１０１～１０３を有する計算機システムの全体構成を示す。複数のコンピュータ５はＭＲＡ－Ｓｗｉｔｃｈ１０１と接続している。また、ＭＲＡ－Ｓｗｉｔｃｈ１０２はＩ／Ｏ５９１と接続し、ＭＲＡ－Ｓｗｉｔｃｈ１０３はＩ／Ｏ５９２と接続している。この計算機システムにおいて、コンピュータ５００と他のコンピュータ６００、７００の間でマイグレーションを行う場合を考える。 FIG. 10 shows the overall configuration of a computer system having MRA-Switches 101 to 103. The plurality of computers 5 are connected to the MRA-Switch 101. The MRA-Switch 102 is connected to the I / O 591, and the MRA-Switch 103 is connected to the I / O 592. Consider a case where migration is performed between the computer 500 and the other computers 600 and 700 in this computer system.
　図５～図７は、図１０の計算機システムに使用される、コンピュータ及びＭＲＡ－Ｓｗｉｔｃｈの構成例を示す。コンピュータ５００はＩ／Ｏ５９１及び５９２と接続され、コンピュータ６００はＩ／Ｏ６９１及び６９２と、コンピュータ７００はＩ／Ｏ７９１及び７９２と接続されている。
この例では、Ｒｏｏｔ ＣｏｍｐｌｅｘからＩ／Ｏに至るまでの経路上の情報が一致しているコンピュータ５００とコンピュータ７００の間ではマイグレーションを行うことが出来るが、経路上の情報が一致していないコンピュータ５００とコンピュータ６００の間ではマイグレーションが行えない。以下、これについて説明する。なお、本実施例においてＰＣＩ Ｂｕｓを単にＢｕｓ（バス）と表す。 5 to 7 show configuration examples of the computer and the MRA-Switch used in the computer system shown in FIG. The computer 500 is connected to I / O 591 and 592, the computer 600 is connected to I / O 691 and 692, and the computer 700 is connected to I / O 791 and 792.
In this example, migration can be performed between the computer 500 and the computer 700 on which the information on the route from the root complex to the I / O matches, but the computer 500 on which the information on the route does not match. And computer 600 cannot be migrated. This will be described below. In this embodiment, PCI Bus is simply referred to as Bus.
　図５において、コンピュータ５００はＣＰＵ５０１１とＲＣ５０１２を有する。ＲＣ５０１２は、ＭＲＡ－Ｓｗｉｔｃｈ１０１のＰｏｒｔ５１１とＢｕｓ５５０で接続されている。Ｂｕｓ５５０にはＢ＃０という識別子が付与されている。Ｂｕｓに付与される識別子は、ＯＳがＲｏｏｔ Ｃｏｍｐｌｅｘ以下に接続されているＳｗｉｔｃｈやＥｎｄ Ｐｏｉｎｔを探索して決定する。ＭＲＡ－Ｓｗｉｔｃｈ１０１は、ＶＳＢｒ５１４をＵｐｓｔｒｅａｍ ＰＣＩ－ＰＣＩ Ｂｒｉｄｇｅとし、ＶＳＢｒ５１５とＶＳＢｒ５１６をＤｏｗｎｓｔｒｅａｍ ＰＣＩ－ＰＣＩ ＢｒｉｄｇｅとするＶｉｒｔｕａｌ Ｓｗｉｔｃｈが構成されており、ＶＳＢｒ５１４とＶＳＢｒ５１５及び５１６の間はＢｕｓ５５１（Ｂ＃１）で接続されている。ＭＲＡ－Ｓｗｉｔｃｈ１０１において、Ｐｏｒｔ５１１はＶＳＢｒ５１４と、Ｐｏｒｔ５１２はＶＳＢｒ５１５と、Ｐｏｒｔ５１３はＶＳＢｒ５１６とそれぞれ接続されている。ＰｏｒｔとＶＳ Ｂｒｉｄｇｅの間の接続はＢｕｓではなく、識別子も付与されない。ＭＲＡ－Ｓｗｉｔｃｈ１０１のＰｏｒｔ５１２とＭＲＡ－Ｓｗｉｔｃｈ１０２のＰｏｒｔ５２１はＢｕｓ５５２（Ｂ＃２）で接続され、ＭＲＡ－Ｓｗｉｔｃｈ１０１のＰｏｒｔ５１３とＭＲＡ－Ｓｗｉｔｃｈ１０３のＰｏｒｔ５３１はＢｕｓ５５５（Ｂ＃５）で接続されている。ＭＲＡ－Ｓｗｉｔｃｈ１０２、ＭＲＡ－Ｓｗｉｔｃｈ１０３もＭＲＡ－Ｓｗｉｔｃｈ１０１と同様に設定が行われており、ＭＲＡ－Ｓｗｉｔｃｈ１０２のＰｏｒｔ５２２はＩ／Ｏ５９１とＢｕｓ５５４（Ｂ＃４）で接続され、ＭＲＡ－Ｓｗｉｔｃｈ１０３のＰｏｒｔ５３２はＩ／Ｏ５９２とＢｕｓ５５７（Ｂ＃７）で接続されている。 In FIG. 5, a computer 500 includes a CPU 5011 and an RC 5012. The RC 5012 is connected to the Port 511 and Bus 550 of the MRA-Switch 101. The Bus 550 is assigned an identifier B # 0. The identifier assigned to the Bus is determined by searching for Switch and End Point connected to the OS under Root Complex. MRA-Switch 101 is composed of Virtual Switch having VSBr 514 as Upstream PCI-PCI Bridge and VSBr 515 and VSBr 516 as Downstream PCI-PCI Bridge (1 is connected between VSBr 514 and 55B and 515Br at 15B and 515Br). ing. In the MRA-Switch 101, the Port 511 is connected to the VSBr 514, the Port 512 is connected to the VSBr 515, and the Port 513 is connected to the VSBr 516. The connection between Port and VS Bridge is not Bus and is not given an identifier. Port 512 of MRA-Switch 101 and Port 521 of MRA-Switch 102 are connected by Bus 552 (B # 2), and Port 513 of MRA-Switch 101 and Port 531 of MRA-Switch 103 are connected by Bus 555 (B # 5). MRA-Switch 102 and MRA-Switch 103 are set in the same manner as MRA-Switch 101. Port 522 of MRA-Switch 102 is connected by I / O 591 and Bus 554 (B # 4), and Port 532 of MRA-Switch 103 is I / O 592. And Bus 557 (B # 7).
　コンピュータ５００上で動作するＯＳがＩ／Ｏを認識する際に利用する経路情報にＢｕｓに付与されている識別子も含まれるとする。この場合、ＯＳは、Ｉ／Ｏ５９１を、Ｂ＃０－Ｂ＃１－Ｂ＃２－Ｂ＃３－Ｂ＃４－Ｉ／Ｏ５９１という形で認識する。同様に、Ｉ／Ｏ５９２を、Ｂ＃０－Ｂ＃１－Ｂ＃５－Ｂ＃６－Ｂ＃７－Ｉ／Ｏ５９２という形で認識する。 Assume that the path information used when the OS operating on the computer 500 recognizes I / O includes an identifier assigned to Bus. In this case, the OS recognizes the I / O 591 in the form of B # 0-B # 1-B # 2-B # 3-B # 4-I / O591. Similarly, I / O 592 is recognized in the form of B # 0-B # 1-B # 5-B # 6-B # 7-I / O592.
　図６は、図５のコンピュータ５００とマイグレーションが出来ないコンピュータ６００及びＭＲＡ－Ｓｗｉｔｃｈの構成を示す。
コンピュータ６００はＩ／Ｏ６９１及び６９２と接続されている。コンピュータ６００上で動作するＯＳはＩ／Ｏ６９１をＢ＃０－Ｂ＃１－Ｂ＃２－Ｂ＃３－Ｂ＃４－Ｉ／Ｏ６９１と認識する。同様に、Ｉ／Ｏ６９２を、Ｂ＃０－Ｂ＃１－Ｂ＃２－Ｂ＃３－Ｂ＃５－Ｉ／Ｏ６９２という形で認識する。 FIG. 6 shows the configuration of the computer 500 and MRA-Switch that cannot be migrated with the computer 500 of FIG.
The computer 600 is connected to the I / O 691 and 692. The OS operating on the computer 600 recognizes the I / O 691 as B # 0-B # 1-B # 2-B # 3-B # 4-I / O691. Similarly, I / O 692 is recognized in the form of B # 0-B # 1-B # 2-B # 3-B # 5-I / O692.
　Ｉ／Ｏ５９１とＩ／Ｏ６９１、Ｉ／Ｏ５９２とＩ／Ｏ６９２が同一のデバイスであっても、Ｉ／Ｏ５９２とＩ／Ｏ６９２のＯＳからの認識のされ方が異なるため、コンピュータ５００とコンピュータ６００ではマイグレーションを行うことが出来ない。このＯＳの認識するＩ／Ｏの経路情報の差異は、Ｂｕｓの接続形態が異なるために生じているものである。 Even if the I / O 591 and I / O 691 and the I / O 592 and I / O 692 are the same device, the I / O 592 and the I / O 692 are recognized differently from the OS. Can not be done. This difference in I / O path information recognized by the OS is caused by the difference in the bus connection form.
　図７は、図５のコンピュータ５００とマイグレーションが可能なコンピュータ７００及びＭＲＡ－Ｓｗｉｔｃｈの構成を示す。
コンピュータ７００はＩ／Ｏ７９１、Ｉ／Ｏ７９２と接続されている。図７の構成では、ＭＲＡ－Ｓｗｉｔｃｈ７１０１のＶＳＢｒ７１４とＶＳＢｒ７１５はＰｏｒｔ７１２に接続されている。同様に、ＭＲＡ－Ｓｗｉｔｃｈ７１０２のＶＳＢｒ７２４とＶＳＢｒ７２６はＰｏｒｔ７２１に接続されている。ＶＳＢｒ７１４とＶＳＢｒ７２４はＢｕｓ７５２（Ｂ＃２）で接続されており、ＶＳＢｒ７１５とＶＳＢｒ７２６はＢｕｓ７５５（Ｂ＃５）で接続されている。ＭＲ－ＩＯＶ仕様ではＬｉｎｋを複数のＢｕｓで共有することが可能であり、Ｂｕｓ７５２（Ｂ＃２）とＢｕｓ７５５（Ｂ＃５）はＬｉｎｋを共有している状態である。コンピュータ７００上で動作するＯＳはＩ／Ｏ７９１を、Ｂ＃０－Ｂ＃１－Ｂ＃２－Ｂ＃３－Ｂ＃４－Ｉ／Ｏ７９１という形で認識する。同様にＩ／Ｏ７９２を、Ｂ＃０－Ｂ＃１－Ｂ＃５－Ｂ＃６－Ｂ＃７－Ｉ／Ｏ７９２という形で認識する。
Ｉ／Ｏ５９１とＩ／Ｏ７９１、Ｉ／Ｏ５９２とＩ／Ｏ７９２が同一デバイスであれば、Ｉ／Ｏ５９２とＩ／Ｏ７９２のＯＳからの認識のされ方が同一であるため、コンピュータ５００とコンピュータ７００の間でマイグレーションを行うことが可能である。 FIG. 7 shows the configuration of the computer 500 and MRA-Switch that can be migrated with the computer 500 of FIG.
The computer 700 is connected to the I / O 791 and the I / O 792. In the configuration of FIG. 7, VSBr 714 and VSBr 715 of MRA-Switch 7101 are connected to Port 712. Similarly, VSBr 724 and VSBr 726 of MRA-Switch 7102 are connected to Port 721. VSBr 714 and VSBr 724 are connected by Bus 752 (B # 2), and VSBr 715 and VSBr 726 are connected by Bus 755 (B # 5). In the MR-IOV specification, a link can be shared by a plurality of buses, and the bus 752 (B # 2) and the bus 755 (B # 5) share the link. The OS operating on the computer 700 recognizes the I / O 791 in the form of B # 0-B # 1-B # 2-B # 3-B # 4-I / O791. Similarly, I / O 792 is recognized in the form of B # 0-B # 1-B # 5-B # 6-B # 7-I / O792.
If the I / O 591 and I / O 791 and the I / O 592 and I / O 792 are the same device, the I / O 592 and the I / O 792 are recognized in the same way from the OS. It is possible to perform migration with
　ここで、図８のように、内部にＶｉｒｔｕａｌ Ｓｗｉｔｃｈを２つ持つ、ＶＳ ＢｒｉｄｇｅとＰｏｒｔの対応が固定になっているＭＲＡ－Ｓｗｉｔｃｈ８０を考える。ＭＲＡ－Ｓｗｉｔｃｈ８０において、Ｐｏｒｔ８０１とＶＳＢｒ８１０が、Ｐｏｒｔ８０２とＶＳＢｒ８１３がそれぞれ固定的に接続されている。また、Ｐｏｒｔ８０３とＶＳＢｒ８１１及びＶＳＢｒ８１４が、Ｐｏｒｔ８０４とＶＳＢｒ８１２及びＶＳＢｒ８１５がそれぞれ固定的に接続されている。また、内部で構成されているＶｉｒｔｕａｌ Ｓｗｉｔｃｈは、ＶＳ８２０とＶＳ８２１である。このＭＲＡ－Ｓｗｉｔｃｈ８０では、ＭＲＡ－Ｓｗｉｔｃｈ７１０１のように、１つのＶＳの２個のＤｏｗｎｓｔｒｅａｍ側のＶＳ Ｂｒｉｄｇｅを同一のＰｏｒｔに接続する構成を行うことは出来ない。
このように、ＶＳ ＢｒｉｄｇｅとＰｏｒｔの対応が固定になっているため、ＶＳ８２０がＰｏｒｔ８０３に接続されているＶＳ Ｂｒｉｄｇｅを２つ利用できない。 Here, as shown in FIG. 8, consider an MRA-Switch 80 having two Virtual Switches inside and having a fixed correspondence between VS Bridge and Port. In MRA-Switch 80, Port 801 and VSBr 810 are fixedly connected to Port 802 and VSBr 813, respectively. Further, Port 803 and VSBr 811 and VSBr 814 are fixedly connected, and Port 804 and VSBr 812 and VSBr 815 are fixedly connected. Also, the virtual switches configured inside are VS820 and VS821. In this MRA-Switch 80, unlike the MRA-Switch 7101, it is not possible to connect two VS Bridges on one Downstream side of one VS to the same Port.
As described above, since the correspondence between VS Bridge and Port is fixed, VS 820 cannot use two VS Bridges connected to Port 803.
　図９は、本発明の実施例によるＭＲＡ－Ｓｗｉｔｃｈの構成を示す。ＭＲＡ－Ｓｗｉｔｃｈ９０において、Ｐｏｒｔ９０１とＶＳＢｒ９１０、及びＰｏｒｔ９０２とＶＳＢｒ９１３がそれぞれ固定的に接続されている。また、Ｐｏｒｔ９０３とＶＳＢｒ９１１及びＶＳＢｒ９１４が、Ｐｏｒｔ９０４とＶＳＢｒ９１２及びＶＳＢｒ９１５がそれぞれ固定的に接続されている。また、内部で構成されているＶｉｒｔｕａｌ Ｓｗｉｔｃｈは、ＶＳ９２０とＶＳ９２１である。本発明においては、Ｖｉｒｔｕａｌ ＳｗｉｔｃｈとＶＳ Ｂｒｉｄｇｅの接続関係を変更可能に設定できるので、ＶＳＢｒ９１１及びＶＳＢｒ９１４をＶＳ９２０に接続することが出来る。このような構成により、ＭＲＡ－Ｓｗｉｔｃｈ７１０１のように、１つのＶＳの２個のＤｏｗｎｓｔｒｅａｍ側のＶＳ Ｂｒｉｄｇｅを同一のＰｏｒｔに接続する構成を行うことが可能となる。 FIG. 9 shows the configuration of MRA-Switch according to the embodiment of the present invention. In MRA-Switch 90, Port 901 and VSBr 910 and Port 902 and VSBr 913 are fixedly connected to each other. Further, Port 903 and VSBr 911 and VSBr 914 are fixedly connected, and Port 904 and VSBr 912 and VSBr 915 are fixedly connected. In addition, Virtual Switch configured internally is VS920 and VS921. In the present invention, since the connection relationship between Virtual Switch and VS Bridge can be set to be changeable, VSBr911 and VSBr914 can be connected to VS920. With such a configuration, as in MRA-Switch 7101, it is possible to connect two VS Bridges on one Downstream side of one VS to the same Port.
　以上のように、ＶＳ ＢｒｉｄｇｅとＰｏｒｔの対応を固定にしたＭＲＡ－Ｓｗｉｔｃｈでも、本発明を適用することによりＶＳ間でＶＳ Ｂｒｉｄｇｅを移動できるため、マイグレーションの際に必要な経路を保存することが可能となる。本発明によれば、コンピュータとＩ／Ｏの接続関係の柔軟性がＭＲ－ＩＯＶ仕様に追加されるので、マイグレーションの自由度を向上させることができる。 As described above, even in MRA-Switch in which the correspondence between VS Bridge and Port is fixed, VS Bridge can be moved between VSs by applying the present invention, so it is possible to save a necessary route for migration. It becomes. According to the present invention, the flexibility of the connection relationship between the computer and the I / O is added to the MR-IOV specification, so that the degree of freedom of migration can be improved.
図１２～図１５を参照して、ＭＲＡ－Ｓｗｉｔｃｈの構成の設定変更について説明する。
図１２は、３つのＭＲＡ－Ｓｗｉｔｃｈ１０１～１０３によって接続された計算機システムを示す。ＭＲＡ－Ｓｗｉｔｃｈ１０１のＰｏｒt１２５０，１２５１にはコンピュータ５００、１５００が接続されている。また、ＭＲＡ－Ｓｗｉｔｃｈ１０１のＰｏｒt１２５２には、ＭＲＡ－Ｓｗｉｔｃｈ１０２を介してＩ／Ｏ９１が接続されている。同様に、ＭＲＡ－Ｓｗｉｔｃｈ１０１のＰｏｒt１２５３には、ＭＲＡ－Ｓｗｉｔｃｈ１０３を介してＩ／Ｏ９２が接続されている。なお、ＭＲＡ－Ｓｗｉｔｃｈ１０２、１０３のＰｏｒｔの図示は省略されている。 With reference to FIGS. 12 to 15, setting change of the MRA-Switch configuration will be described.
FIG. 12 shows a computer system connected by three MRA-Switches 101-103. Computers 500 and 1500 are connected to the Ports 1250 and 1251 of the MRA-Switch 101. Further, an I / O 91 is connected to the Port 1252 of the MRA-Switch 101 via the MRA-Switch 102. Similarly, an I / O 92 is connected to the Port 1253 of the MRA-Switch 101 via the MRA-Switch 103. The illustration of Ports of MRA- Switches 102 and 103 is omitted.
　ＭＲＡ－Ｓｗｉｔｃｈ１０１は、図１３に示すように、各Ｐｏｒｔ１２５０～１２５３に接続されるＶＳＢｒ１３０～１３３毎に備えられたＶＳ Ｂｒｉｄｇｅ Ｔａｂｌｅ１３０１～１３３１、及びＰｏｒt Ｔａｂｌｅ１３５、ＶＳ Ｔａｂｌｅ１３６を備えて構成される。図１２の構成及び図１３のＭＲＡ－Ｓｗｉｔｃｈ１０１の構成から、コンピュータ５００は、ＭＲＡ－Ｓｗｉｔｃｈ１０１、１０２を介してＩ／Ｏ９１に接続され、コンピュータ１５００は、ＭＲＡ－Ｓｗｉｔｃｈ１０１、１０３を介してＩ／Ｏ９２に接続されていることがわかる。 As shown in FIG. 13, the MRA-Switch 101 includes VSＶBridge Tables 1301 to 1331 provided for each of the VSBrs 130 to 133 connected to the Ports 1250 to 1253, and the Port Table 135 and VS Table 136, respectively. From the configuration of FIG. 12 and the configuration of MRA-Switch 101 of FIG. 13, the computer 500 is connected to the I / O 91 via the MRA- Switch 101, 102, and the computer 1500 connects to the I / O 92 via the MRA- Switch 101, 103. You can see that they are connected.
　各ＭＲＡ－Ｓｗｉｔｃｈ１０１～１０３の管理を行う管理ユニット１２４０、１２４１，１２４２には、管理端末１８０が接続される。管理端末１８０は、プログラムを実行するプロセッサ、プログラムや種々のデータを格納するメモリ、及び管理者が操作する入出力器を有している（詳細図示省略）。プロセッサでは、接続されたＭＲＡ－Ｓｗｉｔｃｈ１０１～１０３の内部の構成を設定するためのソフトウェアである、ＭＲ－ＰＣＩＭ（Ｍｕｌｔｉ－Ｒｏｏｔ ＰＣＩ Ｍａｎａｇｅｒ）が実行され、管理者が管理端末１８０の入出力器から設定情報を入力することにより、ＭＲ－ＩＯＶ仕様で定義される、ＶＳ Ｂｒｉｄｇｅ Ｔａｂｌｅ１３１１～１３４１、Ｐｏｒｔ Ｔａｂｌｅ１３５、ＶＳ Ｔａｂｌｅ１３６の設定が行われる。 A management terminal 180 is connected to the management units 1240, 1241, and 1242 that manage the MRA-Switches 101 to 103. The management terminal 180 includes a processor that executes a program, a memory that stores the program and various data, and an input / output device that is operated by an administrator (detailed illustration is omitted). In the processor, MR-PCIM (Multi-Root PCI Manager), which is software for setting the internal configuration of the connected MRA-Switch 101 to 103, is executed and set by the administrator from the input / output device of the management terminal 180 By inputting information, VS Bridge Tables 1311 to 1341, Port Table 135, and VS Table 136 defined in the MR-IOV specification are set.
　なお、他の例として、ＭＲ－ＰＣＩＭは、管理端末１８０のプロセッサで実行しないで、ＭＲＡ－Ｓｗｉｔｃｈに搭載された、例えば管理ユニットが有するプロセッサで実行するようにしてもよい。
また、管理ユニット１２４０～１２４２は、各Ｔａｂｌｅの設定の他に、ＭＲＡ－Ｓｗｉｔｃｈ全体に係わるイベントをＭＲ－ＰＣＩＭへ通知したり、通過するパケットの流量などの統計の取得を行う。ＭＲ－ＩＯＶ仕様では複数のコンピュータが1枚のカードを共有できるため、カードに対する処理の調停でＭＲ－ＰＣＩＭの介入が必要になるイベントの通知をＭＲ－ＰＣＩＭに対して行う。 As another example, the MR-PCIM may not be executed by the processor of the management terminal 180, but may be executed by, for example, a processor included in the MRA-Switch and included in the management unit.
In addition to setting each table, the management units 1240 to 1242 notify the MR-PCIM of events related to the entire MRA-Switch, and acquire statistics such as the flow rate of packets passing through. In the MR-IOV specification, since a plurality of computers can share one card, the MR-PCIM is notified of an event that requires intervention of MR-PCIM in the arbitration of processing for the card.
　次に、図１４及び図１５を参照して、ＭＲＡ－Ｓｗｉｔｃｈ１０１～１０３の設定変更について説明する。
まず、管理者は、管理端末１８０の入出力器を操作して、ＭＲＡ－Ｓｗｉｔｃｈ１０１，１０２，１０３の設定情報を順次入力する（Ｓ１４０１～Ｓ１４０３）。
即ち、各ＭＲＡ－Ｓｗｉｔｃｈごとに、Ｐｏｒｔ識別するポート番号と、ＶＨ番号と、ＶＳ ＢｒｉｄｇｅがどのＶｉｒｔｕａｌ Ｓｗｉｔｃｈに所属しているかを示す識別情報（ＶＳ番号）がそれぞれ管理端末１８０より入力される。 Next, referring to FIG. 14 and FIG. 15, the setting change of the MRA-Switches 101 to 103 will be described.
First, the administrator operates the input / output device of the management terminal 180 to sequentially input the setting information of the MRA- Switches 101, 102, and 103 (S1401 to S1403).
That is, for each MRA-Switch, the port number for identifying the port, the VH number, and identification information (VS number) indicating to which Virtual Switch the VS Bridge belongs are input from the management terminal 180.
　管理端末１８０から、各ＭＲＡ－Ｓｗｉｔｃｈごとに各情報が入力されると、管理端末１８０のプロセッサでＭＲ－ＰＣＩＭ１８１が実行されて、各ＭＲＡ－Ｓｗｉｔｃｈの管理ユニット１２４０～１２４２によって、それぞれのＶＳ Ｂｒｉｄｇｅ Ｔａｂｌｅ１３０１～１３３１、Ｐｏｒｔ Ｔａｂｌｅ１３５、ＶＳ Ｔａｂｌｅ１３６に、入力されたＶＳ番号と、ポート番号と、ＶＨ番号が登録されて設定が行われる。 When each piece of information is input for each MRA-Switch from the management terminal 180, the MR-PCIM 181 is executed by the processor of the management terminal 180, and each VS Bridge Table 1301 is executed by the management units 1240 to 1242 of each MRA-Switch. ˜1331, Port ポ ー ト Table 135, VS Table 136, the input VS number, port number, and VH number are registered and set.
　例えば、ＭＲＡ－Ｓｗｉｔｃｈ１０１では、図１５に示すように、ＶＳ Ｔａｂｌｅ１３６にはＶＨ番号が登録される（Ｓ１５０１）。更に、ＶＳＢｒ１３０～１３３のＶＳＢｒｉｄｇｅ Ｔａｂｌｅ１３０１～１３３１には、それぞれＶＳ番号が登録される（Ｓ１５０２～Ｓ１５０５）。また、Ｐｏｒｔ Ｔａｂｌｅ１３５には、Ｐｏｒｔ番号が登録される（Ｓ１５０６）。 For example, in MRA-Switch 101, as shown in FIG. 15, the VH number is registered in VS Table 136 (S1501). Further, VS numbers are registered in the VSBridge Tables 1301 to 1331 of the VSBrs 130 to 133, respectively (S1502 to S1505). Further, the Port number is registered in the Port table 135 (S1506).
　このように、各テーブル１３６，１３０１～１３３１、１３５は、従前に設定されたＶＳ番号、ポート番号、ＶＨ番号から、新たに設定されたＶＳ番号、ポート番号、ＶＨ番号に変更され、計算機システムにおける各ＭＲＡ－Ｓｗｉｔｃｈの構成の設定変更が行われる。なお、この設定変更では、ＶＳ番号、ポート番号、ＶＨ番号の全てを変更する必要はなく、計算機システムの要請に応じていずれか１つ又は２つのみの変更も可能である。
In this way, the tables 136, 1301 to 1331, 135 are changed from the previously set VS number, port number, and VH number to the newly set VS number, port number, and VH number. The configuration of each MRA-Switch is changed. In this setting change, it is not necessary to change all of the VS number, the port number, and the VH number, and only one or two can be changed according to the request of the computer system.
１、１０１～１０３：ＭＲＡ－Ｓｗｉｔｃｈ　　１２０、１２１：Ｐｏｒｔ
１４０、１４１、１４２：ＶＳ Ｂｒｉｄｇｅ
１５０：Ｐ＃レジスタ　　１５１：ＶＨ＃レジスタ　　１５２：ＶＳ＃レジスタ
１６０、１６１、１６２：Ｖｉｒｔｕａｌ Ｈｉｅｒａｒｃｈｙ
１９１、１９２：Ｖｉｒｔｕａｌ Ｓｗｉｔｃｈ
５、５００、６００、７００：コンピュータ 1, 101-103: MRA-Switch 120, 121: Port
140, 141, 142: VS Bridge
150: P # register 151: VH # register 152: VS # register 160, 161, 162: Virtual Hierarchy
191, 192: Virtual Switch
5, 500, 600, 700: Computer

Claims (10)

  1. 　ＭＲ－ＩＯＶ仕様に基づくＭＲＡスイッチであって、
    Ｉ／Ｏデバイスに接続される、それぞれ識別子が付与された複数のポートと、
    それぞれ固有の識別子が付与された、１または複数の仮想スイッチ（ＶＳ）と、
    該仮想スイッチを構成し、該仮想スイッチと該ポートを接続する、複数のＶＳ Ｂｒｉｄｇｅと、
    各該ＶＳ Ｂｒｉｄｇｅが属する該仮想スイッチの識別子を記憶する記憶部を有し、
    外部からの指示によって、該記憶部に記憶される該仮想スイッチの識別子を変更することにより、異なる仮想スイッチ間で該ＶＳ Ｂｒｉｄｇｅの移動を可能とすることを特徴とするＭＲＡスイッチ。     MRA switch based on MR-IOV specifications,
    A plurality of ports each connected with an identifier connected to an I / O device;
    One or more virtual switches (VS) each having a unique identifier;
    A plurality of VS Bridges constituting the virtual switch and connecting the virtual switch and the port;
    A storage unit for storing the identifier of the virtual switch to which each VS Bridge belongs;
    An MRA switch that enables movement of the VS Bridge between different virtual switches by changing an identifier of the virtual switch stored in the storage unit according to an instruction from the outside.
  2. 前記記憶部は、各々の該ＶＳ Ｂｒｉｄｇｅ内に実装され、該ＶＳ Ｂｒｉｄｇｅごとに、該仮想スイッチの識別子を設定するレジスタであることを特徴とする請求項１のＭＲＡスイッチ。 2. The MRA switch according to claim 1, wherein the storage unit is a register that is mounted in each VSＶBridge and sets an identifier of the virtual switch for each VS Bridge. 3.
  3. 前記記憶部は、
    該ＶＳ Ｂｒｉｄｇｅの接続されている該ポートの識別子を設定するＰ＃レジスタと、該ＶＳ Ｂｒｉｄｇｅの接続されている該ポートがＶＳ Ｂｒｉｄｇｅを識別するためのＶｉｒｔｕａｌ Ｈｉｅｒａｒｃｈｙ（ＶＨ）を設定するＶＨ＃レジスタと、該ＶＳ Ｂｒｉｄｇｅが属する仮想スイッチの識別子を設定するＶＳ＃レジスタとを含むＶＳ Ｂｒｉｄｇｅテーブルを記憶すること
    を特徴とする請求項１又は２のＭＲＡスイッチ。     The storage unit
    A P # register that sets an identifier of the port to which the VS Bridge is connected, and a VH # register that sets a Virtual Hierarchy (VH) for identifying the VS Bridge by the port to which the VS Bridge is connected The MRA switch according to claim 1 or 2, further comprising a VS Bridge table including a VS # register for setting an identifier of a virtual switch to which the VS Bridge belongs.
  4. 該ＭＲＡスイッチは、該ＭＲＡスイッチの管理を行う管理ユニットを備え、該管理ユニットに接続される管理端末からの指示によって、前記記憶部又は前記レジスタの内容が書き換えられること
    を特徴とする請求項１乃至３のいずれかの請求項記載のＭＲＡスイッチ。     The MRA switch includes a management unit that manages the MRA switch, and the contents of the storage unit or the register are rewritten by an instruction from a management terminal connected to the management unit. The MRA switch according to any one of claims 1 to 3.
  5. 　ＭＲ－ＩＯＶ仕様に基づく、１又は接続関係にある複数のＭＲＡスイッチと、
    １又は複数のいずれかの該ＭＲＡスイッチのポートに接続された１又は複数のコンピュータと、
    １又は複数のいずれかの該ＭＲＡスイッチのポートに接続された１又は複数のＩ／Ｏデバイスを含む計算機システムであって、
    該ＭＲＡスイッチは、
    Ｉ／Ｏデバイスに接続される、それぞれ識別子が付与された複数のポートと、
    それぞれ固有の識別子が付与された、１または複数の仮想スイッチ（ＶＳ）と、
    該仮想スイッチを構成し、該仮想スイッチと該ポートを接続する、複数のＶＳ Ｂｒｉｄｇｅと、
    各該ＶＳ Ｂｒｉｄｇｅが属する該仮想スイッチの識別子を記憶する記憶部を有し、
    外部からの指示によって、該記憶部に記憶される該仮想スイッチの識別子を変更することにより、異なる仮想スイッチ間で該ＶＳ Ｂｒｉｄｇｅの移動を可能とすることを特徴とする計算機システム。     One or more MRA switches in connection relationship based on MR-IOV specifications;
    One or more computers connected to a port of any one or more of the MRA switches;
    A computer system including one or more I / O devices connected to a port of any one or more of the MRA switches,
    The MRA switch is
    A plurality of ports each connected with an identifier connected to an I / O device;
    One or more virtual switches (VS) each having a unique identifier;
    A plurality of VS Bridges constituting the virtual switch and connecting the virtual switch and the port;
    A storage unit for storing the identifier of the virtual switch to which each VS Bridge belongs;
    A computer system which enables movement of the VS Bridge between different virtual switches by changing the identifier of the virtual switch stored in the storage unit in accordance with an instruction from the outside.
  6. 前記記憶部は、各々の該ＶＳ Ｂｒｉｄｇｅ内に実装され、該ＶＳ Ｂｒｉｄｇｅごとに、該仮想スイッチの識別子を設定するレジスタであること
    を特徴とする請求項５の計算機システム。     6. The computer system according to claim 5, wherein the storage unit is a register that is mounted in each of the VS Bridges and sets an identifier of the virtual switch for each VS Bridge.
  7. 前記記憶部は、
    該ＶＳ Ｂｒｉｄｇｅの接続されている該ポートの識別子を設定するＰ＃レジスタと、該ＶＳ Ｂｒｉｄｇｅの接続されている該ポートがＶＳ Ｂｒｉｄｇｅを識別するためのＶｉｒｔｕａｌ Ｈｉｅｒａｒｃｈｙ（ＶＨ）を設定するＶＨ＃レジスタと、該ＶＳ Ｂｒｉｄｇｅが属する仮想スイッチの識別子を設定するＶＳ＃レジスタとを含むＶＳ Ｂｒｉｄｇｅテーブルを記憶することを特徴とする請求項５又は６の計算機システム。     The storage unit
    A P # register that sets an identifier of the port to which the VS Bridge is connected, and a VH # register that sets a Virtual Hierarchy (VH) for identifying the VS Bridge by the port to which the VS Bridge is connected 7. The computer system according to claim 5, wherein a VS Bridge table including a VS # register for setting an identifier of a virtual switch to which the VS Bridge belongs is stored.
  8. 各前記ＭＲＡスイッチは、該ＭＲＡスイッチの管理を行う管理ユニットを備え、更に、
    該管理ユニットに接続される管理端末を有し、
    該管理端末からの指示に従って、前記記憶部又は前記レジスタの内容が書き換えられることを特徴とする請求項５乃至７のいずれかの請求項記載の計算機システム。     Each MRA switch includes a management unit for managing the MRA switch, and
    A management terminal connected to the management unit;
    8. The computer system according to claim 5, wherein the contents of the storage unit or the register are rewritten in accordance with an instruction from the management terminal.
  9. 　ＭＲ－ＩＯＶ仕様に基づくＭＲＡスイッチの設定変更方法であって、
    該ＭＲＡスイッチは、
    Ｉ／Ｏデバイスに接続される、それぞれ識別子が付与された複数のポートと、
    それぞれ固有の識別子が付与された、１または複数の仮想スイッチ（ＶＳ）と、
    該仮想スイッチを構成し、該仮想スイッチと該ポートを接続する、複数のＶＳ Ｂｒｉｄｇｅと、
    各該ＶＳ Ｂｒｉｄｇｅが属する該仮想スイッチの識別子を記憶する記憶部と、
    該ＭＲＡスイッチの管理を行う管理ユニットを有し、
    該管理ユニットに接続される管理端末から指示によって、該記憶部に記憶される該仮想スイッチの識別子を変更することにより、異なる仮想スイッチ間で該ＶＳ Ｂｒｉｄｇｅの移動を可能とするＭＲＡスイッチの設定変更方法。     An MRA switch setting changing method based on MR-IOV specifications,
    The MRA switch is
    A plurality of ports each connected with an identifier connected to an I / O device;
    One or more virtual switches (VS) each having a unique identifier;
    A plurality of VS Bridges constituting the virtual switch and connecting the virtual switch and the port;
    A storage unit that stores an identifier of the virtual switch to which each VS Bridge belongs;
    A management unit for managing the MRA switch;
    Changing the setting of the MRA switch that enables movement of the VS Bridge between different virtual switches by changing the identifier of the virtual switch stored in the storage unit according to an instruction from the management terminal connected to the management unit Method.
  10. 前記記憶部は、該ＶＳ Ｂｒｉｄｇｅの接続されている該ポートの識別子を設定するＰ＃レジスタと、該ＶＳ Ｂｒｉｄｇｅの接続されている該ポートがＶＳ Ｂｒｉｄｇｅを識別するためのＶｉｒｔｕａｌ Ｈｉｅｒａｒｃｈｙ（ＶＨ）を設定するＶＨ＃レジスタと、該ＶＳ Ｂｒｉｄｇｅが属する仮想スイッチの識別子を設定するＶＳ＃レジスタとを含むＶＳ Ｂｒｉｄｇｅテーブルを記憶しており、
    前記管理端末から、該Ｐ＃レジスタ、該ＶＨ＃レジスタ、該ＶＳ＃レジスタのいずれかにセットすべき識別子を入力し、
    前記ＭＲＡスイッチの前記管理ユニットは、該管理端末から入力された識別子を該当する、該Ｐ＃レジスタ、該ＶＨ＃レジスタ、該ＶＳ＃レジスタに書き込むことを
    特徴とする請求項９のＭＲＡスイッチの設定変更方法。     The storage unit sets a P # register for setting an identifier of the port to which the VS Bridge is connected, and sets a Virtual Hierarchy (VH) for the port to which the VS Bridge is connected to identify the VS Bridge And a VS Bridge table including a VS # register for setting an identifier of a virtual switch to which the VS Bridge belongs,
    From the management terminal, an identifier to be set in any of the P # register, the VH # register, and the VS # register is input,
    10. The MRA switch setting according to claim 9, wherein the management unit of the MRA switch writes the identifier input from the management terminal to the corresponding P # register, the VH # register, and the VS # register. Modification method.
PCT/JP2011/069354 2011-08-26 2011-08-26 Mra switch, computer system using same, and method for changing setting of mra switch WO2013030910A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/JP2011/069354 WO2013030910A1 (en) 2011-08-26 2011-08-26 Mra switch, computer system using same, and method for changing setting of mra switch

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2011/069354 WO2013030910A1 (en) 2011-08-26 2011-08-26 Mra switch, computer system using same, and method for changing setting of mra switch

Publications (1)

Publication Number Publication Date
WO2013030910A1 true WO2013030910A1 (en) 2013-03-07

Family

ID=47755465

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2011/069354 WO2013030910A1 (en) 2011-08-26 2011-08-26 Mra switch, computer system using same, and method for changing setting of mra switch

Country Status (1)

Country Link
WO (1) WO2013030910A1 (en)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2010191814A (en) * 2009-02-19 2010-09-02 Hitachi Ltd Computer system, management method and management server
WO2011001508A1 (en) * 2009-06-30 2011-01-06 株式会社日立製作所 Electronic apparatus connection device, computer, electronic apparatus connection device storage unit allocation method
JP2011081462A (en) * 2009-10-05 2011-04-21 Hitachi Ltd Computer device, computer system, adapter inheritance method

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2010191814A (en) * 2009-02-19 2010-09-02 Hitachi Ltd Computer system, management method and management server
WO2011001508A1 (en) * 2009-06-30 2011-01-06 株式会社日立製作所 Electronic apparatus connection device, computer, electronic apparatus connection device storage unit allocation method
JP2011081462A (en) * 2009-10-05 2011-04-21 Hitachi Ltd Computer device, computer system, adapter inheritance method

Similar Documents

Publication Publication Date Title
US10210120B2 (en) Method, apparatus and system to implement secondary bus functionality via a reconfigurable virtual switch
US8412863B2 (en) Storage apparatus and virtual port migration method for storage apparatus
US9792240B2 (en) Method for dynamic configuration of a PCIE slot device for single or multi root ability
US8352665B2 (en) Computer system and bus assignment method
CN107409116B (en) Computing device, method, medium, and apparatus for managing virtual network functions
US9858102B2 (en) Data path failover method for SR-IOV capable ethernet controller
US8645605B2 (en) Sharing multiple virtual functions to a host using a pseudo physical function
US8521941B2 (en) Multi-root sharing of single-root input/output virtualization
US8103810B2 (en) Native and non-native I/O virtualization in a single adapter
US10333865B2 (en) Transformation of peripheral component interconnect express compliant virtual devices in a network environment
US20180189109A1 (en) Management system and management method for computer system
US20180239725A1 (en) Persistent Remote Direct Memory Access
US20070136458A1 (en) Creation and management of ATPT in switches of multi-host PCI topologies
CN107181679A (en) A kind of port binding implementation method and device
CN104426814A (en) Numa node peripheral switch
US8972611B2 (en) Multi-server consolidated input/output (IO) device
JP2008152786A (en) Method for migrating virtual function from first to second physical function of one or more end points in data processing system, program, and system (system and method for migration of single root stateless virtual function)
US20180321964A1 (en) Computer, device allocation management method, and program recording medium
US11005968B2 (en) Fabric support for quality of service
CN105320628A (en) Adaptation device, system and method for enabling single I/O device to be shared by multiple root nodes
US11683372B2 (en) Virtualized fabric management server for storage area network
US10261935B1 (en) Monitoring excessive use of a peripheral device
US10089267B2 (en) Low latency efficient sharing of resources in multi-server ecosystems
JP2012146105A (en) Computer system
JP5813699B2 (en) Communication system, management apparatus, management method, and management program

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 11871539

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 11871539

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: JP