WO2010055791A1 - I/oバスシステム - Google Patents
I/oバスシステム Download PDFInfo
- Publication number
- WO2010055791A1 WO2010055791A1 PCT/JP2009/068789 JP2009068789W WO2010055791A1 WO 2010055791 A1 WO2010055791 A1 WO 2010055791A1 JP 2009068789 W JP2009068789 W JP 2009068789W WO 2010055791 A1 WO2010055791 A1 WO 2010055791A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- network
- pci express
- packet
- host
- bridge
- Prior art date
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F13/00—Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
- G06F13/38—Information transfer, e.g. on bus
- G06F13/40—Bus structure
- G06F13/4004—Coupling between buses
- G06F13/4022—Coupling between buses using switching circuits, e.g. switching matrix, connection or expansion network
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F13/00—Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
- G06F13/38—Information transfer, e.g. on bus
- G06F13/40—Bus structure
- G06F13/4004—Coupling between buses
- G06F13/4027—Coupling between buses using bus bridges
- G06F13/404—Coupling between buses using bus bridges with address mapping
Definitions
- the present invention relates to an I / O bus, and more particularly to an I / O bus system, an I / O bus system control method, and a program recording medium for controlling an I / O bus system.
- I / O bus for example, a PCI bus
- technologies related to a new I / O bus standard changing to the conventional I / O bus standard have been proposed.
- a technique related to an I / O bus system designed according to the new I / O bus standard as described in Japanese Patent Application Laid-Open No. 2007-219873 is known.
- FIG. 1 is a block diagram showing the configuration of the I / O bus system described in Japanese Patent Laid-Open No. 2007-219873.
- the I / O bus system includes a host 101, a PCI express switch 105, and an I / O 103.
- the host 101 includes a CPU 111, a memory 113, and a root complex 112.
- the route complex 112 connects the CPU 111, the memory 113, and the PCI express switch 105 to each other.
- the right to use the I / O 103 is assigned to one of the hosts 101. Also, the assignment of the I / O 103 to the host 101 can be changed.
- the PCI express switch 105 includes an upstream PCI express-network bridge 121, a network switch 122, a downstream PCI express-network bridge 125, and a system manager 151.
- the upstream PCI express-network bridge 121 bridges the PCI express bus and the network on the host 101 side.
- the network switch 122 transfers network packets.
- the downstream PCI express-network bridge 125 bridges the network and the PCI express bus on the I / O 103 side.
- the system manager 151 controls the connection between the upstream PCI express-network bridge 121 and the downstream PCI express-network bridge 125 and sets the allocation of the I / O 103 to the host 101.
- the upstream PCI express-network bridge 121 is connected to a plurality of downstream PCI express-network bridges 125.
- the upstream PCI express-network bridge 121 receives an I / O packet addressed to the I / O 103 from the route complex 112, encapsulates the received I / O packet into a network packet, and transfers the packet to the network switch 122.
- the upstream PCI express-network bridge 121 assigns the network address of the downstream PCI express-network bridge 125 connected to the I / O 103 that is the destination of the original I / O packet to the destination of the encapsulated packet. Describe.
- the upstream PCI express-network bridge 121 receives a network packet in which an I / O packet addressed to the host 101 is encapsulated from the network switch 122.
- the upstream PCI express-network bridge 121 decapsulates the received network packet and transmits it to the root complex 112.
- the downstream PCI express-network bridge 125 is connected to one upstream PCI express-network bridge 121.
- the downstream PCI express-network bridge 125 receives an I / O packet addressed to the host 101 corresponding to the upstream PCI express-network bridge 121 connected from the I / O 103, encapsulates the received I / O packet into a network packet,
- the network address of the upstream PCI Express-network bridge 121 connected to the destination host 101 of the I / O packet is written and transferred to the network switch 122.
- the network switch 122 receives a network packet in which an I / O packet addressed to the I / O 103 is encapsulated, decapsulates the I / O packet, and transmits it to the I / O 103.
- FIG. 2 is a diagram showing the address space 115-1 of the host 101-1.
- the address space 115 of the host 101 includes an ID number space 1151 that is an ID number space that is a combination of a bus number, a device number, and a function number, and a physical memory space 1152 that has a power of two. .
- the address of the I / O 103 used by the host 101 is mapped to the ID number space 1151 and the physical memory space 1152.
- the I / O 103-1 to I / O 103-M are respectively the I / O 103-1 to I / O 103-M maps 1511-1 to 1511-M in the ID number space 1151-1 and the physical memory space. 1152-1 I / O 103-1 to I / O 103-M maps 1521-1 to 1521-M.
- the conventional I / O bus system having such a configuration operates as follows.
- the system manager 151 controls which upstream PCI express-network bridge 121 the downstream PCI express-network bridge 125 is connected to by transmitting a control packet to the downstream PCI express-network bridge 125.
- the I / O 103 is assigned to the host 101 connected to the upstream PCI express-network bridge 121 to which the downstream PCI express-network bridge 125 is connected.
- the upstream PCI express-network bridge 121 and the downstream PCI express-network bridge 125 encapsulate an I / O packet transmitted / received between the host 101 and the I / O 103 assigned to the host 101 into a network packet. Tunneling is performed between the PCI express-network bridge 121 and the downstream PCI express-network bridge 125.
- the host 101 is an upstream PCI-PCI bridge inside the PCI express switch in which the upstream PCI express-network bridge 121 conforms to the standard, and the downstream PCI express-network bridge 125 is a downstream PCI-PCI bridge inside the PCI express switch. And thereby recognize that the region between the upstream PCI express-network bridge 25 and the downstream PCI express-network bridge 21 is a PCI express switch compliant with the standard. Therefore, in the conventional I / O bus system, the assignment of the I / O 103 to the host 101 can be freely changed without requiring special software for the host 101.
- the related I / O bus system is designed on the assumption that a downstream PCI express-network bridge is connected to one upstream PCI express-network bridge and tunnels I / O packets. In addition, it is designed on the assumption that I / O resources are occupied by one host. Therefore, I / O resources may not be shared from two or more hosts at the same time.
- the downstream PCI express-network bridge may not be able to tunnel I / O packets by connecting to two or more upstream PCI express-network bridges at the same time.
- An object of the present invention is to provide an I / O bus system in which I / O resources can be shared from two or more hosts at the same time.
- An I / O bus system comprising a connection bridge disposed between an express network bridge and swapping network addresses described in network packets transmitted between the plurality of hosts and the I / O resources.
- the present invention can configure an I / O bus system in which a downstream PCI Express-network bridge can simultaneously connect to two or more upstream PCI Express-network bridges to tunnel I / O packets.
- the effect of the present invention is that I / O resources can be shared from two or more hosts at the same time.
- the reason is that one downstream PCI Express-Network Bridge can be connected to multiple upstream PCI Express-Network Bridges by swapping the addresses described in the header of the network packet encapsulating the I / O packet.
- I / O resources are configured in advance, I / O resources configured in the address space of each host are mapped in units of functions, and I / O encapsulated in network packets using the mapping information This is because a function held by a single I / O resource is assigned to an arbitrary host by swapping the header information described in the O packet.
- downstream PCI express-network bridge can simultaneously connect to two or more upstream PCI express-network bridges to tunnel I / O packets.
- FIG. 1 is a block diagram showing a configuration of a related I / O bus system.
- FIG. 2 is a diagram showing the address space of the host.
- FIG. 3 is a block diagram illustrating the configuration of the I / O bus system according to the first embodiment of this invention.
- FIG. 4 is a diagram showing the relationship between the host 1 and the address space of the I / O resource.
- FIG. 5A is a block diagram illustrating the configuration of the target host search table.
- FIG. 5B is a block diagram illustrating the configuration of the host mapping table.
- FIG. 6 is a diagram illustrating an operation when configuring a downstream PCI Express network bridge and I / O resources.
- FIG. 1 is a block diagram showing a configuration of a related I / O bus system.
- FIG. 2 is a diagram showing the address space of the host.
- FIG. 3 is a block diagram illustrating the configuration of the I / O bus system according to the first embodiment of this invention.
- FIG. 4 is a
- FIG. 7 is a diagram illustrating an operation when the host 1 transmits an I / O packet other than the configuration packet to the I / O resource.
- FIG. 8 is a diagram illustrating an operation when an I / O packet is transmitted from an I / O resource to a host.
- FIG. 9 is a block diagram showing a configuration of the second exemplary embodiment of the present invention.
- FIG. 3 is a block diagram illustrating the configuration of the I / O bus system 10 according to the first embodiment of this invention.
- the I / O bus system 10 according to the first embodiment of the present invention includes a plurality of hosts 1, a PCI express switch 2, and an I / O resource 3.
- Each of the plurality of hosts 1 includes a CPU 11, a root complex 12, and a memory 13.
- the I / O resource 3 includes a plurality of functions 31 (first function 31-1 to N-th function 31-N) and is simultaneously shared by the host 1.
- the I / O resource 3 provides the function 31 individually to each of the plurality of hosts 1, thereby realizing simultaneous sharing of the I / O resource 3 by the plurality of hosts 1.
- each of the first function 31-1 to the Nth function 31-N is assigned to the first host 1-1 to the Nth host 1-N on a one-to-one basis.
- I / O resource 3 holding a plurality of functions 31, multi-function I / O and I / O corresponding to software virtualization within a single host (I / O corresponding to I / O virtualization) O) and the like are exemplified.
- the PCI express switch 2 includes an upstream PCI express network bridge 21 connected to the host 1, a network switch 22, a system manager 23, a multi-root connection bridge 24, and a downstream PCI express network bridge 25.
- the multi-root connection bridge 24 connects the downstream PCI express network bridge 25 with the plurality of upstream PCI express network bridges 21 to share the I / O resource 3 among the plurality of hosts 1 at the same time.
- the specific upstream PCI express network bridge 21 operates assuming that no other upstream PCI express network bridge 21 exists and is connected to the downstream PCI express network bridge 25 on a one-to-one basis.
- the first upstream PCI express network bridge 21-1 operates independently without depending on the behavior of the second upstream PCI express network bridge 21-2 to the Nth upstream PCI express network bridge 21-N.
- the downstream PCI express network bridge 25 can only be connected to one upstream PCI express network bridge 21 by design. Therefore, the downstream PCI express network bridge 25 in the present embodiment performs the connection that is originally performed with the upstream PCI express network bridge 21 to the multi-root connection bridge 24.
- the system manager 23 transmits a control packet to the multi-root connection bridge 24, controls a plurality of upstream PCI express network bridges 21 connected to the downstream PCI express network bridge 25, and assigns each function 31 of the I / O resource 3. Specify host 1.
- the multi-route connection bridge 24 includes a packet transfer unit 241, an I / O configuration unit 242, an I / O information storage unit 243, a downstream PCI express network bridge pseudo register 244, and an I / O function pseudo register 245. It is out.
- the packet transfer unit 241 swaps and transfers the network packet header of the network packet in which the I / O packet is encapsulated and the I / O packet header.
- the I / O configuration unit 242 configures the I / O resource 3.
- the I / O information storage unit 243 holds information required when the packet transfer unit 241 swaps information described in the packet.
- the downstream PCI express network bridge pseudo register 244 provides a pseudo register function of the downstream PCI express network bridge 25.
- the I / O function pseudo register 245 provides a pseudo register function of each function 31.
- the multi-root connection bridge 24 includes a plurality of downstream PCI express network bridge pseudo registers 244 (first downstream PCI express network bridge pseudo register 244-1 to Nth downstream PCI express network bridge pseudo register 244-N). Shall be included.
- the multi-root connection bridge 24 includes a plurality of I / O function pseudo registers 245 (first I / O function pseudo register 245-1 to NI / O function pseudo register 245-N).
- the first downstream PCI express network bridge pseudo register 244-1 to the Nth downstream PCI express network bridge pseudo register 244-N and the first I / O function pseudo register 245-1 to the NI / O function pseudo register 245-N are: Each access from the first host 1-1 to the Nth host 1-N is accepted.
- the I / O configuration unit 242 configures the downstream PCI express network bridge 25 and the I / O resource 3 before the I / O resource 3 is used from the host 1.
- the I / O configuration unit 242 transmits a configuration packet encapsulated in a network packet to the downstream PCI express network bridge 25 and the I / O resource 3, and configures the downstream PCI express network bridge 25 and the I / O resource 3. Perform At this time, the destination of the network packet is set to the downstream PCI express network bridge 25. Further, the I / O configuration unit 242 records configuration information of the downstream PCI express network bridge 25 and the I / O resource 3 in the I / O information storage unit 243.
- FIG. 4 is a diagram showing the relationship between the address space of the host 1 and the I / O resource 3.
- the I / O resource address space 32 of the I / O resource 3 includes an ID number space 321 and a physical memory space 322.
- the ID number space 321 is an ID number space including a group of “bus number, device number, function number”.
- the I / O configuration unit 242 configures the I / O resource address space 32 of the I / O resource 3.
- a host map 3211 of the host 1 to be assigned to the host 1 later is configured, and in the physical memory space 322, a host map 3221 of the host 1 is configured.
- the host map 3211-1 to host map 3211 -N of the host 1 and the host map 3221-1 to host map 3221 -N of the host 1 are the first function 31-1 to N-th function 31-of the I / O resource 3. Corresponds to N.
- the I / O configuration unit 242 also receives a configuration packet issued by the host 1 to the downstream PCI express network bridge 25 and the I / O resource 3 from the packet transfer unit 241.
- the configuration packet is a write
- the configuration for the downstream PCI express network bridge 25 is recorded in the downstream PCI express network bridge pseudo register 244, and the configuration for the I / O resource 3 is recorded in the I / O function pseudo register 245.
- the configuration packet is a read, the information requested by the packet is read from the corresponding address of the downstream PCI express network bridge pseudo register 244 or the I / O function pseudo register 245 and is sent to the host 1 via the packet transfer unit 241. Send back.
- the I / O information storage unit 243 includes a downstream PCI express network bridge 25 performed by the I / O configuration unit 242, configuration information for the I / O resource 3, and the host 1 includes a downstream PCI express network bridge pseudo register 244.
- the information written in the I / O function pseudo register 245 is held, and the packet transfer unit 241 swaps the information described in the header of the network packet and the I / O packet in the network packet in which the I / O packet is encapsulated.
- Providing information necessary for The I / O information storage unit 243 holds a target host search table 2431 and a host mapping table 2432.
- FIG. 5A is a block diagram illustrating the configuration of the target host search table 2431.
- the target host search table 2431 associates each function 31 of the I / O resource 3 with the host 1 to which the function 31 is assigned and the network address (HA) of the upstream PCI express network bridge 21 to which the host 1 is connected.
- FIG. 5B is a block diagram illustrating the configuration of the host mapping table 2432.
- the host mapping table 2432 is prepared for each individual host 1, and the ID number given to the root complex 12 by the host 1, the ID number given to the function 31 of the I / O resource 3 to which the host 1 is assigned, The memory space lower limit and the upper limit are made to correspond to values configured by the I / O configuration unit 242 of the I / O resource 3.
- the host mapping table 2432 shows the bus number, device number, and function number, which are ID numbers, as B, D, and F, respectively.
- 5B shows the mapping relationship between the host-side address space 14 of the host 1 held in the host mapping table 2432 and the I / O resource address space 32 of the I / O resource 3.
- the ID number space 321 and the physical memory space 322 are divided into the ID number space 141 and the physical memory space of the host-side address space 14 of the host 1 with the function 31 as a unit. 142.
- downstream PCI express network bridge pseudo register 244 holds the information that the host 1 gives to the downstream PCI express network bridge 25 by the configuration write for each host 1.
- the I / O function pseudo register 245 holds configuration information that the host 1 gives to the function 31 of the I / O resource 3. Further, when the host 1 configures the function 31, it asks about the capacity of the memory space allocated to the function 31, and the I / O function pseudo register 245 provides this information. This information is recorded in the I / O information recording unit 243 when the I / O configuration unit 242 configures the I / O resource 3, and this information is recorded in the I / O information recording unit 243. This is reflected in the function pseudo register 245.
- the packet transfer unit 241 receives from the network switch 22 a network packet in which an I / O packet other than the configuration packet issued by the host 1 to the I / O resource 3 is encapsulated, and transmits the network packet representing the host 1 The address is swapped with the network address of the multi-root connection bridge 24. Further, the host mapping table 2432 corresponding to the transmission source host 1 is searched, and the transmission source address of the I / O packet is determined from the ID number of the route complex 12 in the host side address space 14 of the host 1 and the I / O resource 3 I Swap to the ID number of the root complex 12 in the / O resource address space 32.
- the destination address of the I / O packet is determined from the ID number of the function 31 in the host-side address space 14 of the host 1 and the I / O resource 3 I Swap to the ID number of the function 31 in the / O resource address space 32.
- the destination address of the I / O packet is represented in the memory space, the destination address of the I / O packet is swapped to (destination memory ⁇ Mem lower limit IxR + Mem lower limit IxI).
- x 1 to N is a value corresponding to the host 1 that transmitted the I / O packet.
- the packet transfer unit 241 also receives from the network switch 22 a network packet that encapsulates the configuration packet sent from the host 1 to the downstream PCI express network bridge 25 and the I / O resource 3, decapsulates the configuration packet, and To / O configuration unit 242
- the packet transfer unit 241 also receives a network packet in which the I / O packet issued by the I / O resource 3 to the host 1 is encapsulated, and transmits the destination address of the network packet from the network address of the multi-root connection bridge 241. Swap to the network address of the upstream PCI Express network bridge 21 to which the host 1 to which the original function 31 is assigned is connected.
- the network address of the upstream PCI express network bridge 21 is searched by using the target host search table 2431 held by the I / O information storage unit 243 using the transmission source function 31 described in the I / O packet as a key.
- the destination address and the source address of the encapsulated I / O packet are swapped with reference to the host mapping table 2432 corresponding to the destination host 1.
- the correspondence of the address to be swapped is the reverse of the swap reference performed when the host 1 transmits an I / O packet to the I / O resource 3, and the host 1 from the I / O resource address space 32 of the I / O resource 3 To the host-side address space 14.
- the packet transfer unit 241 also receives the response of the configuration packet transmitted from the I / O configuration unit 242 by the host 1 to the downstream PCI express network bridge pseudo register 244 and the I / O function pseudo register 245, and receives the response packet destination.
- the response packet is encapsulated using the network address of the upstream PCI Express network bridge 21 to which the host 1 is connected, and transmitted to the host 1.
- the packet transfer unit 241 also receives a broadcast control packet transmitted from the downstream PCI express network bridge 25, copies the broadcast control packet by the number of hosts 1, and transmits it to each host.
- the broadcast control packet transmitted by the downstream PCI express network bridge 25 describes the information of the multi-root connection bridge 24 as the connection destination of the downstream PCI express network bridge 25.
- the packet transfer unit 241 The destination information is rewritten to the upstream PCI express-network bridge to which the host 1 is connected, and a copy of the broadcast control packet is transmitted.
- FIG. 6 is a diagram illustrating an operation when the downstream PCI express network bridge 25 and the I / O resource 3 are configured.
- the I / O configuration unit 242 issues a configuration packet before the I / O resource 3 is used from the host 1, and configures the downstream PCI express network bridge 25 and the I / O resource 3 (Step S1). S401).
- the packet transfer unit 241 encapsulates the configuration packet passed from the I / O configuration unit 242 into a network packet, and tunnels the configuration packet with the downstream PCI express network bridge 25, whereby the configuration packet is transferred to the downstream PCI express network. Transmit to the bridge 25 and the I / O resource 3.
- the I / O configuration unit 242 records the configuration information given to the I / O resource 3 in the host mapping table 2432 held by the I / O information storage unit 243 (step S402).
- the capacity of the memory space requested by each function 31 of the I / O resource 3 recorded in the I / O information storage unit 243 is reflected in the I / O function pseudo register 245 (step S403).
- the system manager 23 issues a control packet to the multi-root connection bridge 24, and the first function 31-1 to the Nth function 31-N of the I / O resource 3 are set to the first host 1-1 to the first host respectively.
- Assign to N hosts 1-N step S404). By this assignment, an entry related to the host in the target host search table 2431 held by the I / O information storage unit 243 is described.
- each host 1 starts its configuration cycle (step S405).
- the configuration packet issued by the host 1 to the downstream PCI express network bridge 25 is sent to the downstream PCI express network bridge pseudo register 244 assigned to the host 1 and to the function 31 included in the I / O resource 3 assigned to the host 1.
- the configuration packet to be issued is processed by the I / O configuration unit 242 as an access to the I / O function pseudo register 245 corresponding to the function 31.
- information necessary for describing an entry in the host mapping table 2432 of the I / O information storage unit 243 is extracted from the configuration access of the host, and an entry is formed (step S406).
- FIG. 7 is a diagram illustrating an operation when the host 1 transmits an I / O packet other than the configuration packet to the I / O resource 3.
- the route complex 12 transmits an I / O packet to the function 31 of the I / O resource 3 assigned to the host 1 in accordance with a command of software operating on the CPU 11 (step S501).
- the upstream PCI express network bridge 21 receives the I / O packet issued by the route complex 12, encapsulates the I / O packet using the network address of the downstream PCI express network bridge 25 to which the I / O resource 3 is connected,
- the data is transmitted to the network switch 22 (step S502).
- the network switch 22 transfers the network packet in which the I / O packet is encapsulated to the multi-root connection bridge 24 (step S503).
- the packet transfer unit 241 receives the network packet in which the I / O packet is encapsulated, and swaps the transmission source address of the network packet representing the host 1 with the network address of the multi-root connection bridge 24 (step S504). Further, the host mapping table 2432 corresponding to the transmission host is searched, and the transmission source address of the I / O packet is determined from the ID number of the route complex 12 in the host-side address space 14 of the host 1 and the I / O of the I / O resource 3. Swap to the ID number of the root complex 12 in the resource address space 32.
- the destination address of the I / O bucket is expressed by an ID number
- the destination address of the I / O packet is determined from the ID number of the function 31 in the address space of the host 1 and the I / O resource of the I / O resource 3 Swap to the ID number of the function 31 in the address space 32.
- the destination address of the I / O packet is represented in the memory space
- the destination address of the I / O packet is swapped to (destination memory ⁇ Mem lower limit IxR + Mem lower limit IxI).
- x 1 to N is a value corresponding to the host 1 that transmitted the I / O packet.
- the packet transfer unit 241 transmits the network packet to the downstream PCI express network bridge 25.
- the downstream PCI express network bridge 25 decapsulates the I / O packet (step S505) and transmits it to the I / O resource 3.
- the function 31 of the I / O resource 3 receives the I / O packet transmitted by the host 1 to which the function 31 is assigned (step S506).
- FIG. 8 is a diagram illustrating an operation when an I / O packet is transmitted from the I / O resource 3 to the host 1.
- the function 31 of the I / O resource 3 transmits an I / O packet to the assigned host (step S601).
- the downstream PCI express network bridge 25 receives the I / O packet issued by the function 31, encapsulates the I / O packet using the network address of the multi-root connection bridge 24 as the destination address, and transmits the packet to the multi-root connection bridge 24 ( Step S602).
- the packet transfer unit 241 receives the network packet in which the I / O packet is encapsulated, and swaps the destination address of the network packet representing the multi-root connection bridge 24 with the network address of the host 1 that is the destination of the I / O packet. (Step S603).
- the network address of the destination host 1 is searched in the target host search table 2431 using the source function number described in the I / O packet encapsulated in the network packet as a key. Further, the host mapping table 2432 corresponding to the destination host is searched, and the source address of the I / O packet is determined from the ID number of the function 31 in the I / O resource address space 32 of the I / O resource 3 on the host side of the host 1.
- the destination address of the I / O packet is determined from the ID number of the root complex 12 in the I / O resource address space 32 of the I / O resource 3 by the host. Swap to the ID number of the root complex 12 in the one host-side address space 14.
- the destination address of the I / O packet is expressed in the memory space, the destination address of the I / O packet is swapped to (destination memory ⁇ Mem lower limit IxI + Mem lower limit IxR).
- the packet transfer unit 241 transmits the network packet to the network switch 22.
- the network switch 22 transfers the network packet in which the I / O packet is encapsulated to the upstream PCI express network bridge 21 (step S604).
- the upstream PCI express network bridge 21 decapsulates the I / O packet (step S605) and transmits it to the route complex 12.
- the route complex 12 receives the I / O packet issued by the function 31 assigned to the host 1 (step S606).
- the number of network switches 22 connected between the upstream PCI express network bridge 21 and the multi-root connection bridge 24 is not limited. Further, another network switch 22 may be connected between the multi-root connection bridge 24 and the downstream PCI express network bridge 25.
- I / O resource 3 the case where there is one I / O resource 3 is shown, but the number of I / O resources 3 that are simultaneously shared by a plurality of hosts 1 is not limited. A plurality of I / O resources 3 may be connected using the network switch 22.
- the first function 31-1 to the Nth function 31-N of the I / O resource are assigned to the first host 1-1 to the Nth host 1-N, respectively.
- the combination of how to assign 31 to the host 1 is arbitrary. There may also be a host 1 to which a plurality of functions 31 are assigned, or a host 1 to which no function 31 is assigned.
- the number of functions 31 held by the I / O resource 3 and the number of hosts 1 may be different.
- a downstream PCI express network bridge that is originally connected to a single upstream PCI express-network bridge by a multi-root connection bridge is connected to a plurality of upstream PCI express-network bridges.
- / O resource function is assigned to each host in units of functions.
- the multi-root connection bridge simulates that all network packets sent by each host are sent from the multi-root connection bridge, and the destination address of the network packet sent by the downstream PCI Express-network bridge is multi-route connected. Swap from the bridge to the network address of each host.
- the multi-root connection bridge configures I / O resources in advance and maps the configuration information to the configuration that each host provides to the I / O resources. Then, the information described in the header of the I / O packet transmitted / received between the host and the I / O is swapped using the mapping information. Thereby, an I / O function can be assigned to each host in units of functions, and I / O resources can be shared simultaneously from a plurality of hosts.
- FIG. 9 is a block diagram showing a configuration of the second exemplary embodiment of the present invention.
- the PCI express switch 2 in the first embodiment shown in FIG. It is different in point.
- the multi-root connection bridge 41 in the second embodiment includes a processor 411 that processes a network packet in which an I / O packet is encapsulated, and a memory that holds a packet processing program and configuration information of the I / O bus system 10. 412.
- the memory 412 has the same information as the multi-route connection bridge program 4121 and the I / O information storage unit 243 that cause the processor 411 to perform the functions of the packet transfer unit 241 and the I / O configuration unit 242 according to the first embodiment.
- the I / O information storage unit 4122 that holds the same information as the downstream PCI express network bridge pseudo register 244, and the same information as the I / O function pseudo register 245.
- an I / O function register information storage unit 4124 to be held.
- the multi-root connection bridge program 4121 is read by the processor 411, and causes the processor 411 to perform the functions of the packet transfer unit 241 and the I / O configuration unit 242 in the first embodiment.
- access to the I / O information storage unit 243, the downstream PCI express network bridge pseudo register 244, and the I / O function pseudo register 245 generated in the first embodiment is the I / O stored in the memory 412. Generated for corresponding addresses in the information storage unit 4122, the downstream PCI express-network bridge register information storage unit 4123, and the I / O function register information storage unit 4124.
- the processor performs the same function as the multi-root connection bridge in the first embodiment using a program held in the memory. Therefore, it is possible to realize simultaneous I / O sharing from a plurality of hosts by operating the designed program on a general-purpose processor without developing special hardware.
- the present invention is an application in which an I / O device is shared among information processing apparatuses including a plurality of computers or a CPU or a CPU-like arithmetic device in a computer apparatus, a network apparatus, an industrial apparatus, or a consumer apparatus. Applicable to.
- the I / O bus system described above is characterized in that a network address described in a network packet is swapped, and a single downstream PCI express-network bridge and a plurality of upstream PCI express-network bridges are connected.
- the I / O bus system described above swaps the transmission source address of the network packet transmitted by the upstream PCI express-network bridge with the network address of the bridge that mediates the connection and transmits it to the downstream PCI express-network bridge. Then, the upstream PCI Express-Network Bridge is swapped by swapping the destination address of the network packet transmitted by the downstream PCI Express-Network Bridge from the network address of the bridge that mediates the connection to the network address of the upstream PCI Express-Network Bridge. It is characterized by transmitting to.
- the I / O bus system described above is characterized in that I / O resources are configured in advance and the I / O resources are mapped in units of functions to the configuration performed by the host.
- the I / O bus system pre-configures I / O resources and stores configuration information given to the I / O resources by the host, and the I / O resource configuration information and the host stores the I / O resources. Mapping information of configuration information to be given to a resource is stored, the stored mapping information is referenced, and addresses described in headers of I / O packets transmitted / received between the host and the I / O resource are swapped And
- an I / O bus system control program recording medium for storing an I / O bus system control program includes a step of connecting a plurality of upstream PCI express-network bridges and a plurality of hosts on a one-to-one basis, and a single downstream Connecting the PCI Express-network bridge and the I / O resource between the upstream PCI express-network bridge and the downstream PCI express-network bridge between the plurality of hosts and the I / O resource.
- an I / O bus system control program showing a procedure for realizing an operation including a step of swapping a network address described in a network packet that transmits the I / O bus.
- the swapping step swaps the source address of the network packet transmitted by the upstream PCI express-network bridge with the network address of the bridge that mediates the connection, thereby downstream PCI express.
- the step of sending to the network bridge and the downstream PCI express-the destination address of the network packet sent by the network bridge is swapped from the network address of the bridge that mediates the connection to the network address of the upstream PCI express-network bridge
- An I / O bus system control program is stored, comprising transmitting to the upstream PCI express-network bridge.
- the I / O bus system control program recording medium further includes an operation of pre-configuring I / O resources and mapping the I / O resources in units of functions to configurations performed by the host.
- An I / O bus system control program indicating a procedure for realizing the above is stored.
- the mapping step includes a step of pre-configuring I / O resources, a step of storing configuration information given to the I / O resources by the host, and an I / O Storing the configuration information of the resource and the mapping information of the configuration information given to the I / O resource by the host, and the swapping step refers to the stored mapping information, and the host and the I / O resource
- An I / O bus system control program including a step of swapping addresses described in headers of I / O packets transmitted / received between the I / O packets is stored.
- the I / O bus system control program recording medium stores an I / O bus system control program indicating a procedure for realizing an operation including a step of storing a host to which a function of an I / O resource is assigned.
- the step of configuring the I / O resource and the downstream PCI express-network bridge in advance, and the step of swapping the I / O bus system control program recording medium Swap the network address of the network packet that encapsulates the I / O packet by referring to the information about the host to which the O resource function is assigned, and encapsulate it in the network packet by referring to the mapping information of the stored configuration information Storing an I / O bus system control program characterized by including a step of swapping the address of the generated I / O packet.
- the network address swaps the source address of the network packet transmitted by the upstream PCI Express-network bridge to the network address of the bridge that mediates the connection
- the downstream I / O bus system control characterized in that the destination address of a network packet transmitted by PCI Express-Network Bridge is swapped from the network address of the bridge that mediates the connection to the network address of the upstream PCI Express-Network Bridge Memorize the program.
- the I / O bus system control program recording medium is an I / O indicating a procedure for realizing an operation including a step of causing a processor to change a function of an I / O resource allocated to a host by a control packet. Stores the bus system control program.
- the I / O bus system control program recording medium is characterized in that the address swap of the I / O packet targets the memory space and the ID number described in the I / O packet. Store the control program.
Abstract
Description
この出願は、2008年11月13日に出願された日本出願特願2008-290692を基礎とする優先権を主張し、その開示の全てをここに取り込む。
Claims (12)
- 複数のホストに一対一に接続される複数の上流PCIエクスプレス-ネットワークブリッジと、
I/Oリソースに接続される単一の下流PCIエクスプレス-ネットワークブリッジと、
前記上流PCIエクスプレス-ネットワークブリッジと前記下流PCIエクスプレス-ネットワークブリッジとの間に配置され、前記複数のホストと前記I/Oリソースとの間を伝送するネットワークパケットに記載されたネットワークアドレスをスワップする接続ブリッジと
を具備する
I/Oバスシステム - 請求の範囲1に記載のI/Oバスシステムにおいて、
前記接続ブリッジは、
前記上流PCIエクスプレス-ネットワークブリッジが送信したネットワークパケットの送信元アドレスを、接続を仲介するブリッジのネットワークアドレスにスワップして下流PCIエクスプレス-ネットワークブリッジに送信し、
前記下流PCIエクスプレス-ネットワークブリッジが送信したネットワークパケットの送信先アドレスを、前記接続を仲介するブリッジのネットワークアドレスから前記上流PCIエクスプレス-ネットワークブリッジにスワップして前記上流PCIエクスプレス-ネットワークブリッジに送信する
I/Oバスシステム。 - 請求の範囲1または2に記載のI/Oバスシステムにおいて、
前記接続ブリッジは、
前記I/Oリソースを、あらかじめコンフィグレーションし、前記ホストが行うコンフィグレーションに対し前記I/Oリソースを、ファンクションを単位としてマッピングすることを特徴とする
I/Oバスシステム。 - 請求の範囲3に記載のI/Oバスシステムにおいて、
前記接続ブリッジは、
前記I/Oリソースをあらかじめコンフィグレーションするコンフィグレーション手段と、
前記ホストが前記I/Oリソースに与えるコンフィグレーション情報を記憶するコンフィグレーション情報記憶手段と、
前記I/Oリソースのコンフィグレーション情報と、前記ホストが前記I/Oリソースに与えるコンフィグレーション情報とのマッピング情報を記憶するマッピング情報記憶手段と、
前記マッピング情報を参照して前記ホストと前記I/Oリソースの間で送受信するI/Oパケットのヘッダに記載されたアドレスをスワップするスワップ手段と
を備えたことを特徴とする
I/Oバスシステム。 - 請求の範囲4に記載のI/Oバスシステムにおいて、さらに、
前記I/Oリソースのファンクションが割り当てられたホストを記憶するファンクション割り当て記憶手段を備え、
前記上流PCIエクスプレス-ネットワークブリッジは、前記ホストをネットワークに接続し、
前記下流PCIエクスプレス-ネットワークブリッジは、前記I/Oリソースをネットワークに接続し、
前記コンフィグレーション手段は、前記I/Oリソースと前記下流PCIエクスプレス-ネットワークブリッジとをあらかじめコンフィグレーションし、
前記スワップ手段は、
前記ファンクション割り当て記憶手段を参照してI/Oパケットをカプセル化したネットワークパケットのネットワークアドレスをスワップし、
前記I/Oリソースのコンフィグレーション情報と、前記ホストが前記I/Oリソースに与えるコンフィグレーション情報とのマッピング情報を参照してネットワークパケットにカプセル化されたI/Oパケットのアドレスをスワップすることを特徴とする
I/Oバスシステム。 - 請求の範囲5に記載のI/Oバスシステムにおいて、
前記スワップ手段は、
前記上流PCIエクスプレス-ネットワークブリッジが送信したネットワークパケットの送信元アドレスを、前記スワップ手段のネットワークアドレスにスワップし、
前記下流PCIエクスプレス-ネットワークブリッジが送信したネットワークパケットの送信先アドレスを、前記スワップ手段のネットワークアドレスから前記上流PCIエクスプレス-ネットワークブリッジのネットワークアドレスにスワップすることを特徴とする
I/Oバスシステム。 - 請求の範囲6に記載のI/Oバスシステムにおいて、
前記ホストに割り当てる前記I/Oリソースのファンクションは、制御パケットにより変更できることを特徴とする
I/Oバスシステム。 - 請求の範囲7に記載のI/Oバスシステムにおいて、
前記I/Oパケットのアドレスのスワップは、前記I/Oパケットに記載されたメモリ空間とID番号とを対象とすることを特徴とする
I/Oバスシステム。 - ホストを上流PCIエクスプレス-ネットワークブリッジを用いてネットワークに接続し、
I/Oリソースを下流PCIエクスプレス-ネットワークブリッジを用いてネットワークに接続し、
I/Oリソースと前記下流PCIエクスプレス-ネットワークブリッジをあらかじめコンフィグレーションし、
ホストがI/Oリソースに与えるコンフィグレーション情報を記憶し、
I/Oリソースのファンクションが割り当てられたホストを記憶し、
I/Oリソースのコンフィグレーション情報と前記ホストがI/Oリソースに与えるコンフィグレーション情報のマッピング情報を記憶し、
前記記憶したI/Oリソースのファンクションが割り当てられたホストに関する情報を参照してI/Oパケットをカプセル化したネットワークパケットのネットワークアドレスをスワップし、前記記憶したコンフィグレーション情報のマッピング情報を参照してネットワークパケットにカプセル化されたI/Oパケットのアドレスをスワップすることを特徴とする
I/Oバスシステム制御方法。 - 請求の範囲9に記載のI/Oバスシステム制御方法において、
前記ネットワークアドレスのスワップは、前記上流PCIエクスプレス-ネットワークブリッジが送信したネットワークパケットの送信元アドレスを、接続を仲介するブリッジのネットワークアドレスにスワップし、前記下流PCIエクスプレス-ネットワークブリッジが送信したネットワークパケットの送信先アドレスを、前記接続を仲介するブリッジのネットワークアドレスから前記上流PCIエクスプレス-ネットワークブリッジのネットワークアドレスにスワップすることを特徴とする
I/Oバスシステム制御方法。 - 請求の範囲10に記載のI/Oバスシステム制御方法において、
ホストに割り当てるI/Oリソースのファンクションを、制御パケットにより変更することを特徴とする
I/Oバスシステム制御方法。 - 請求の範囲11に記載のI/Oバスシステム制御方法において、
前記I/Oパケットのアドレスのスワップは、I/Oパケットに記載されたメモリとID番号を対象とすることを特徴とする
I/Oバスシステム制御方法。
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US13/121,227 US9213662B2 (en) | 2008-11-13 | 2009-11-04 | I/O bus system |
JP2010537754A JP5434929B2 (ja) | 2008-11-13 | 2009-11-04 | I/oバスシステム |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2008290692 | 2008-11-13 | ||
JP2008-290692 | 2008-11-13 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2010055791A1 true WO2010055791A1 (ja) | 2010-05-20 |
Family
ID=42169926
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/JP2009/068789 WO2010055791A1 (ja) | 2008-11-13 | 2009-11-04 | I/oバスシステム |
Country Status (3)
Country | Link |
---|---|
US (1) | US9213662B2 (ja) |
JP (1) | JP5434929B2 (ja) |
WO (1) | WO2010055791A1 (ja) |
Cited By (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2013045236A (ja) * | 2011-08-23 | 2013-03-04 | Fujitsu Ltd | 通信装置およびid設定方法 |
JP2013196593A (ja) * | 2012-03-22 | 2013-09-30 | Ricoh Co Ltd | データ処理装置、データ処理方法及びプログラム |
JP2014002545A (ja) * | 2012-06-18 | 2014-01-09 | Ricoh Co Ltd | データ転送装置、及びデータ転送方法 |
WO2017154943A1 (ja) * | 2016-03-11 | 2017-09-14 | パナソニックIpマネジメント株式会社 | 情報処理装置 |
WO2017154942A1 (ja) * | 2016-03-11 | 2017-09-14 | パナソニックIpマネジメント株式会社 | 情報処理装置 |
Families Citing this family (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP5467541B2 (ja) * | 2010-02-22 | 2014-04-09 | 日本電気株式会社 | 通信制御システム、スイッチングノード、通信制御方法、及び通信制御用プログラム |
JP5598148B2 (ja) * | 2010-08-05 | 2014-10-01 | 富士通株式会社 | スイッチング装置、情報処理装置、及びスイッチング装置の制御方法 |
US8799519B2 (en) * | 2011-06-27 | 2014-08-05 | Via Technologies, Inc. | Network-to-network bridge |
US8832331B2 (en) | 2011-08-29 | 2014-09-09 | Ati Technologies Ulc | Data modification for device communication channel packets |
US9032102B2 (en) * | 2012-03-02 | 2015-05-12 | International Business Machines Corporation | Decode data for fast PCI express multi-function device address decode |
US8917736B2 (en) | 2012-12-19 | 2014-12-23 | International Business Machines Corporation | Unified system networking with PCIE-CEE tunneling |
US8891542B2 (en) | 2012-12-19 | 2014-11-18 | International Business Machines Corporation | Unified system networking with CEE-PCIe tunneling |
Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2001282701A (ja) * | 2000-03-31 | 2001-10-12 | Aiwa Co Ltd | 情報処理装置及び情報処理方法 |
JP2007219873A (ja) * | 2006-02-17 | 2007-08-30 | Nec Corp | スイッチ及びネットワークブリッジ装置 |
Family Cites Families (15)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6934763B2 (en) * | 2000-04-04 | 2005-08-23 | Fujitsu Limited | Communication data relay system and method of controlling connectability between domains |
US20030093604A1 (en) * | 2001-11-14 | 2003-05-15 | Lee Terry Ping-Chung | Method of error isolation for shared PCI slots |
US7024510B2 (en) * | 2003-03-17 | 2006-04-04 | Hewlett-Packard Development Company, L.P. | Supporting a host-to-input/output (I/O) bridge |
US7415551B2 (en) * | 2003-08-18 | 2008-08-19 | Dell Products L.P. | Multi-host virtual bridge input-output resource switch |
US7200687B2 (en) * | 2003-09-25 | 2007-04-03 | International Business Machines Coporation | Location-based non-uniform allocation of memory resources in memory mapped input/output fabric |
US20060126612A1 (en) * | 2004-11-23 | 2006-06-15 | Sandy Douglas L | Method of transporting a PCI express packet over an IP packet network |
US7478178B2 (en) * | 2005-04-22 | 2009-01-13 | Sun Microsystems, Inc. | Virtualization for device sharing |
US7293129B2 (en) * | 2005-04-22 | 2007-11-06 | Sun Microsystems, Inc. | Flexible routing and addressing |
US8223745B2 (en) * | 2005-04-22 | 2012-07-17 | Oracle America, Inc. | Adding packet routing information without ECRC recalculation |
US7395367B2 (en) * | 2005-10-27 | 2008-07-01 | International Business Machines Corporation | Method using a master node to control I/O fabric configuration in a multi-host environment |
US7484029B2 (en) * | 2006-02-09 | 2009-01-27 | International Business Machines Corporation | Method, apparatus, and computer usable program code for migrating virtual adapters from source physical adapters to destination physical adapters |
JP4998469B2 (ja) * | 2006-08-09 | 2012-08-15 | 日本電気株式会社 | インターコネクション用スイッチおよびシステム |
WO2008018845A1 (en) * | 2006-08-09 | 2008-02-14 | Hasim Eral | Operating method for pneumatic hybrid engine (working with compressed air) |
CN101123570B (zh) * | 2006-08-09 | 2011-05-18 | 华为技术有限公司 | 多个运营商以太网之间的数据转发方法和系统 |
JP4810349B2 (ja) * | 2006-08-11 | 2011-11-09 | 日本電気株式会社 | I/o装置及び方法 |
-
2009
- 2009-11-04 US US13/121,227 patent/US9213662B2/en active Active
- 2009-11-04 JP JP2010537754A patent/JP5434929B2/ja active Active
- 2009-11-04 WO PCT/JP2009/068789 patent/WO2010055791A1/ja active Application Filing
Patent Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2001282701A (ja) * | 2000-03-31 | 2001-10-12 | Aiwa Co Ltd | 情報処理装置及び情報処理方法 |
JP2007219873A (ja) * | 2006-02-17 | 2007-08-30 | Nec Corp | スイッチ及びネットワークブリッジ装置 |
Cited By (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2013045236A (ja) * | 2011-08-23 | 2013-03-04 | Fujitsu Ltd | 通信装置およびid設定方法 |
JP2013196593A (ja) * | 2012-03-22 | 2013-09-30 | Ricoh Co Ltd | データ処理装置、データ処理方法及びプログラム |
JP2014002545A (ja) * | 2012-06-18 | 2014-01-09 | Ricoh Co Ltd | データ転送装置、及びデータ転送方法 |
WO2017154943A1 (ja) * | 2016-03-11 | 2017-09-14 | パナソニックIpマネジメント株式会社 | 情報処理装置 |
WO2017154942A1 (ja) * | 2016-03-11 | 2017-09-14 | パナソニックIpマネジメント株式会社 | 情報処理装置 |
JPWO2017154942A1 (ja) * | 2016-03-11 | 2019-01-17 | パナソニックIpマネジメント株式会社 | 情報処理装置 |
JPWO2017154943A1 (ja) * | 2016-03-11 | 2019-01-17 | パナソニックIpマネジメント株式会社 | 情報処理装置 |
JP7054769B2 (ja) | 2016-03-11 | 2022-04-15 | パナソニックIpマネジメント株式会社 | 情報処理装置 |
Also Published As
Publication number | Publication date |
---|---|
JPWO2010055791A1 (ja) | 2012-04-12 |
US9213662B2 (en) | 2015-12-15 |
JP5434929B2 (ja) | 2014-03-05 |
US20110206051A1 (en) | 2011-08-25 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
JP5434929B2 (ja) | I/oバスシステム | |
US8494833B2 (en) | Emulating a computer run time environment | |
CN108268412B (zh) | 用于扩展外围组件互连高速结构的系统和方法 | |
US20090245257A1 (en) | Network On Chip | |
US20090109996A1 (en) | Network on Chip | |
US20090271172A1 (en) | Emulating A Computer Run Time Environment | |
JP5660053B2 (ja) | ネットワーク装置、ネットワーク構成方法及びネットワーク装置のプログラムを記録したプログラム記録媒体 | |
US11418455B2 (en) | Transparent packet splitting and recombining | |
KR101003102B1 (ko) | 멀티 프로세싱 유닛에 대한 메모리 매핑방법, 및 장치 | |
US9015380B2 (en) | Exchanging message data in a distributed computer system | |
JP5469081B2 (ja) | 制御パス入出力仮想化方法 | |
US11740929B2 (en) | Registering a custom atomic operation with the operating system | |
US11829323B2 (en) | Method of notifying a process or programmable atomic operation traps | |
WO2006018753A1 (en) | Integrated circuit and method for packet switching control | |
US20220121474A1 (en) | Method of completing a programmable atomic transaction | |
US20230244416A1 (en) | Communicating a programmable atomic operator to a memory controller | |
WO2022086789A1 (en) | Method of executing programmable atomic unit resources within a multi-process system | |
CN1666185A (zh) | 支持包处理的可配置的多端口多协议网络接口 | |
JP5731108B2 (ja) | 中継手段、中継システム、中継方法およびプログラム | |
CN105630727A (zh) | 多SoC节点之间的访问方法、装置和系统 | |
US20220382557A1 (en) | On-demand programmable atomic kernel loading | |
JP6573046B1 (ja) | 情報処理装置、情報処理システムおよび情報処理プログラム | |
US20070038745A1 (en) | Processing module | |
US20220121483A1 (en) | Thread execution control in a barrel processor | |
JP6384359B2 (ja) | 分散共有メモリを有する情報処理装置、方法、および、プログラム |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 09826041 Country of ref document: EP Kind code of ref document: A1 |
|
WWE | Wipo information: entry into national phase |
Ref document number: 13121227 Country of ref document: US |
|
ENP | Entry into the national phase |
Ref document number: 2010537754 Country of ref document: JP Kind code of ref document: A |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 09826041 Country of ref document: EP Kind code of ref document: A1 |