JP2014206883A - Computing system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a network switch connecting many computing machines in order to demonstrate an expected processing performance, having small conversion cost of a protocol used for a data transfer of the computing machine, and having few computing resources during data transfer of the computing machine.SOLUTION: A network switch includes: a device emulator which each computing machine recognizes as a device; a data controller which can directly operate the data stored in a memory of each computing machine; a port memory for storing the data on the way during data transfer; and a packet router for determining a routing of the data inside the network switch in association with each computing machine. Each device emulator is connected with each other by a first network. Each data controller is connected with each other by a second network via the packet router, and session structuring and session release of the data transfer between two computing machines are performed by the two corresponding device emulators, and execution of the data transfer is performed by the two corresponding data controllers.

Description

  The present invention relates to a network switch of a computer system that connects a plurality of computers using a general-purpose bus.

  One means for improving the performance of a computer system is to connect a large number of computers such as blade servers and cluster systems. When connecting a large number of computers, the conventional general-purpose technology connects a network card typified by an Ethernet network with PCIe, which is a general-purpose bus of the computer, and performs data transfer using the PCIe protocol. Further, network expansion cards, network expansion cards and network switches, and network switches are connected in multiple stages with dedicated network cables, and data transfer is performed using a predetermined dedicated protocol.

  In conventional general-purpose technology, network switches can be connected by combining general-purpose technologies in this way, so when you want to increase the number of computers connected to the computer system, you can easily increase the number of computers. . However, for the purpose of improving the performance of a computer system, the choice of connecting a large number of computers is not always effective, and whether it is effective depends on the design of the network adopted in the computer system, There is a problem to be solved in the adopted network in order to obtain an actual performance close to the expected processing performance of a computer system that aims to improve performance by connecting computers.

For example, in data transfer, when a packet storing data passes through a network device, protocol conversion or data change may be required. If it increases, there is a problem that the cost of protocol conversion increases and the performance of the network decreases.
In addition, there are programs that increase the frequency of data transfer exponentially as the number of connected computers increases. For such programs, even if the number of connected computers is increased, every time data is transferred. Consumes computer resources such as CPU computing functions, memory space, bus bandwidth, synchronization time, etc., and computer resources cannot be allocated to programs that you want to use, and the performance of the computer system is significantly higher than expected processing performance. There is also a problem that decreases.
In order to solve such network problems, dedicated hardware is designed, and the dedicated hardware is built into the computer to install it near the CPU. As a result, the price of the computer alone increases. There is also a problem.

  As described above, even if a network is constructed with the aim of improving the performance of a computer system, the expected processing performance is not always achieved. However, InfiniBand uses a DMA (Direct Memory Access) function that reads and writes data to the computer's memory, and realizes the data transfer function on the InfiniBand network card side, thereby consuming computer resources during data transfer. To achieve high-speed data transfer and low latency. Further, by constructing a distributed bus system for an on-chip bus such as the publication number JP-A-2005-182818, devices on the distributed bus system can transfer data without going through a computer. By utilizing this, it is possible to connect a plurality of computers with a distributed bus system and use the distributed bus system itself instead of a network to reduce the data transfer protocol conversion process.

JP 2005-182818 A JP 2004-227570 A

  For the purpose of improving the performance of a computer system, the selection of connecting a large number of computers is not always effective. Whether it is effective depends on the design of the network used in the computer system, and in order to obtain the processing performance expected by a computer system that aims to improve performance by connecting many computers, There is a problem to be solved.

  The purpose of the present invention is to reduce the conversion cost of the protocol used for data transfer when connecting multiple computers in order to improve the performance of the computer system, and it is necessary for data transfer processing to extract the processing performance of the connected computers It is to realize a network switch that consumes less computer resources.

  A general-purpose bus direct-coupled network switch that has multiple ports that can be connected to the general-purpose bus of the computer is used to construct a network that connects many computers.

  The general-purpose bus direct-coupled network switch has a DMA controller that can directly read and write data in the computer's memory. When transferring data to the computer, the computer uses the address of the read data area in the memory of the sending computer. The size and the address of the write data area in the memory of the receiving computer are notified to the general-purpose bus direct-coupled network switch, the general-purpose bus direct-coupled network switch performs all data transfer processing, and the data transfer completion notification is calculated by the computer. Send to. At this time, the transmission side, the reception side, and the computer are all in a standby state for receiving a completion notification.

  The computer in the standby state for receiving the completion notification does not need to be controlled for data transfer. Therefore, it is not necessary for the computer to divide the computer resource for data transfer, it is possible to shift to a calculation unrelated to the data transfer, and the computer resource can be used efficiently. The general-purpose bus direct coupling type network switch has a memory inside. In the conventional network switch, the packet is divided into small packets and transmitted in pieces. At this time, if the data transfer path of the receiving computer is congested, for example, the data transfer path of the transmitting computer is not congested, and even if data transmission processing is possible on the transmitting computer, The transmission process of data transfer from the sending computer cannot be completed.

  However, general-purpose bus direct-coupled network switches read data from the sending computer memory all at once and write them to the memory inside the switch at once when transferring data from the sending computer memory to the switch side. Is possible. Further, when transferring data from the switch side to the memory of the receiving computer, it is possible to read the data from the memory inside the switch all at once and write it to the memory of the receiving computer all at once. In this way, the general-purpose bus direct-coupled network switch is equipped with a DMA controller that handles the data transfer function, so that the computer can minimize the computer resources required for data transfer processing, and the data at the time of data transfer Can be secured in the memory, the path for data transfer can be used efficiently. As a result, it becomes possible to assign the computer resources to the calculation to be originally executed, and to reduce the waiting time for the data transfer process, so that it is possible to improve the performance of the computer system.

  According to the general-purpose bus direct coupling type network switch of the present invention, data transfer, which becomes difficult to control as the number of computers to be connected increases, is not controlled by the computer side, but can be controlled independently on the network switch side.

It is a block diagram which shows the Example of the general purpose bus direct coupling type network switch which is one of the forms of this invention. FIG. 2 is a block diagram showing an internal configuration example of a device emulator shown in the form of FIG. 1. It is a block diagram which shows the example of an internal structure of the data controller shown in the form of FIG. FIG. 2 is a block diagram showing an internal configuration example of a port memory shown in the form of FIG. 1. It is a block diagram which shows the example of an internal structure of the packet router shown in the form of FIG. It is a figure which shows the hardware and software hierarchy concept of the general purpose bus direct coupling | bonding type | mold network switch which is one form of this invention, and the node connected to it. It is the schematic which shows the example in the mounting of FIG. It is a sequence diagram which shows the operation | movement of the session construction between nodes in the form of FIG. It is a sequence diagram which shows the operation | movement of the session cancellation | release between nodes in the form of FIG. It is a sequence diagram which shows the operation | movement of the data transfer between nodes in the form of FIG. It is a sequence diagram which shows the flow of the data in the form of FIG. It is the flowchart figure which showed the detail in the sequence diagram of session construction of FIG. It is the flowchart figure which showed the detail in the sequence diagram of session construction of FIG. FIG. 11 is the first half of the flowchart showing details in the data transfer sequence diagram of FIG. 10. FIG. 11 is an intermediate third portion of the flowchart showing details in the sequence diagram of data transfer in FIG. 10. FIG. 11 is the latter half of the flowchart showing the details in the data transfer sequence diagram of FIG. It is a protocol format table | surface of the protocol used in the Example of FIG. It is a data format table | surface of the data area in the protocol used in the Example of FIG.

  Hereinafter, embodiments for carrying out the present invention will be described with reference to the drawings.

  FIG. 1 shows an embodiment of a general-purpose bus direct coupling network switch according to the present invention. The general-purpose bus direct coupling type network switch 1000 has N ports 1022, 1042, and 1062 for processing electrical signals of PCIe, which is a general-purpose bus for connecting nodes, and the nodes 1020, 1040, and 1060 are respectively connected to the PCIe. Each port 1022, 1042, 1062 is connected via signal lines 1021, 1041, 1061.

  The general-purpose bus direct coupling type network switch 1000 has device emulators 1024, 1044, 1064, data controllers 1026, 1046, 1066, port memories 1028, 1048, 1068, packet routers 1030, 1050 for N ports 1022, 1042, 1062. 1070 respectively.

  The device emulators 1024, 1044, and 1064 are connected to the nodes 1020, 1040, and 1060 through the signal lines 1021, 1041, and 1061, the ports 1022, 1042, and 1062, and the signal lines 1023, 1043, and 1063, and the nodes 1020, 1040, and 1060 are connected. Recognizes the device emulators 1024, 1044, and 1064 as devices having a network function.

  The data controllers 1026, 1046, and 1066 have a DMA function capable of directly reading and writing data to and from the memories in the nodes 1020, 1040, and 1060, and the nodes 1020, 1040, and 1060 and the data controllers 1026, 1046, and 1066 are included. Data transfer between the data controllers 1026, 1046, and 1066.

  The port memories 1028, 1048, 1068 are connected to the data controllers 1026, 1046, 1066 via signal lines 1027, 1047, 1067 and have a function of storing data received by the data controllers 1026, 1046, 1066.

  The packet routers 1030, 1050, and 1070 are connected to the data controllers 1026, 1046, and 1066 via signal lines 1029, 1049, and 1069, and have a communication function between the data controllers 1026, 1046, and 1066.

  Device emulators 1024, 1044, and 1064 are connected to ports 1022, 1042, and 1062 through signal lines 1023, 1043, and 1063, respectively. The data controllers 1026, 1046, 1066 are connected to device emulators 1024, 1044, 1064 via signal lines 1025, 1045, 1065. The port memories 1028, 1048, 1068 are connected to data controllers 1026, 1046, 1066 via signal lines 1027, 1047, 1067. The packet routers 1030, 1050 and 1070 are connected to data controllers 1026, 1046 and 1066 for controlling data transfer via signal lines 1029, 1049 and 1069. The device emulators 1024, 1044, and 1064 and the packet routers 1030, 1050, and 1070 are connected between the device emulator networks 1031, 1051, and 1071 and the packet routers in the general-purpose bus direct coupling network switch 1000. Networks 1033, 1053, and 1073 are included.

  Node # 1 1020, which is the first node, is connected to port # 1 1022 and port # 1 1021 of general-purpose bus direct coupling network switch 1000, and port # 1 1022 is recognized by node # 1 1020 as a device. The device emulator # 1 1024 is connected to the signal line 1023. Normally, a PCIe device can be connected to only one node, but device emulators 1024, 1044, and 1064 corresponding to the number of nodes that can be connected to the general-purpose bus direct coupling type network switch 1000 are included in the general-purpose bus direct coupling type network switch 1000. Thus, a single general-purpose bus direct-coupled network switch 1000 realizes connection of a plurality of nodes via PCIe.

  Device emulator # 1 1024 transmits a control statement between the device emulators using the device emulator network 1031 that is completely coupled to all the device emulators 1024, 1048, and 1068 in the general-purpose bus direct coupling type network switch 1000. To do. Data transfer is performed with the data controller # 1 1026 through the signal line 1025.

  The data controllers 1026, 1046, and 1066 perform DMA transfer between the general-purpose bus direct coupling network switch 1000 and the nodes 1020, 1040, and 1060 in order to realize data transfer that does not load the nodes 1020, 1040, and 1060. It has a function. Also, it has a data transfer function in the general-purpose bus direct coupling type network switch 1000.

  The data controller # 1 1026 reads / writes data to / from the port memory # 1 1028 using the signal line 1027 when transmitting / receiving data to / from the device emulator # 1 1024 or the packet router # 1 1030. Whenever data is transmitted from the device emulator # 1 1024 or the packet router # 1 1030 to the data controller # 1 1026, the transmitted data is stored in the port memory # 1 1028, so that the general-purpose bus direct connection type network switch A collective amount of data transfer between the network 1000 and the nodes 1020, 1040, 1060 and in the general-purpose bus direct coupling network switch 1000 is realized. During data transfer between data controllers, the data controller # 1 uses the signal line 1029 to realize data communication between the data controllers via the packet router # 1 1030. Packet router # 1 1030 holds a fully-coupled packet router network 1033 and transmits data between the packet routers. Further, the inter-device emulator network 1031 and the inter-packet router network 1033 use the ports 1032 and 1034 to the outside, separately from the signal lines for realizing internal communication, of the plurality of general-purpose bus direct coupling network switches 1000. Realize the connection. Note that the node # 2 1040 and the node #N 1060 have the same configuration as the node # 1 1020.

  The device emulators 1024, 1044, 1064, data controllers 1026, 1046, 1066, port memories 1028, 1048, 1068, and packet routers 1030, 1050, 1070 that have appeared in the embodiment of FIG. . The figure shown is a detailed configuration diagram belonging to the i-th node and node #i. FIG. 2 shows a device emulator #i 1100, FIG. 3 shows a data controller #i 1200, FIG. 4 shows a port memory #i 1300, and FIG. The packet router #i 1400 will be described.

  The internal configuration of the device emulator #i 1100 will be described with reference to FIG. The device emulator #i 1100 includes a PCIe communication unit 1110, a transaction management unit 1111, an inter-device emulator communication unit 1112, and a network management unit 1113.

  The device emulator #i 1100 is connected to a port #i 1102 that performs electrical signal processing of PCIe via a signal line 1103, and the port #i 1102 and the node #i of the i-th node are connected via PCIe #i 1101. . The signal line 1103 is connected to the PCIe communication unit 1110 in the device emulator #i 1100. The PCIe communication unit 1110 provides an information processing function necessary for PCIe communication. The PCIe communication unit 1110 and the transaction management unit 1111 are connected by a signal line 1114. In the inter-node communication, the transaction management unit 1111 manages as a transaction from receiving a data transfer request from a node to constructing an inter-node session, transferring data, releasing the inter-node session, and notifying completion of data transfer to the node. Communication between device emulators is performed in establishing and releasing a session between nodes.

  At this time, the transaction management unit 1111 realizes inter-device emulator communication via the inter-device emulator communication unit 1112 connected by the signal line 1115. The inter-device emulator communication unit 1112 is composed of a plurality of signal lines for direct communication with all device emulators in the general-purpose bus direct coupling type network switch and communication with the outside, and these signal lines are connected between the device emulators. The network 1105 is used. At this time, the transaction management unit 1111 acquires the physical information and logical information of the network necessary for data transfer from the network management unit 1113 through the signal line 1116. Among transactions, data transfer is entrusted to the data controller #i 1200 of FIG. 3 through the signal line 1104.

  The internal configuration of the data controller #i 1200 will be described with reference to FIG. The data controller #i 1200 includes a DMA controller 1210 and a memory controller 1215. The DMA controller 1210 is connected to the transaction management unit 1111 of the device emulator #i 1100 of FIG. When the transaction management unit 1111 establishes and releases a session, the DMA controller 1210 is notified by the transaction management unit 1111 of preparation for data transfer and notification of the release. At this time, the DMA controller 1210 secures from an unused area of the buffer 1211 in the DMA controller 1210. Further, the memory controller 1215 is requested to secure the memory in the port memory 1300 via the signal line 1214. The memory controller 1215 communicates with the port memory 1300 via the signal lines 1202 and 1203. When transmitting data transfer between DMA controllers, data is read from the memory controller 1215, read / write is executed to the buffer 1211, and data is transmitted to the packet router #i 1400 via the signal line 1201.

  When receiving data transfer between DMA controllers, the packet router #i 1400 receives data via the signal line 1201, executes writing / reading to the buffer 1211, and writing data to the memory controller 1215. In data reading / writing, the top address of data is managed by the address register 1212, and the progress of data processing is managed by the index register 1213.

  The internal configuration of the port memory #i 1300 will be described with reference to FIG. The port memory #i 1300 includes a sub memory controller # 1 1310, a sub memory controller # 2 1311, a local memory # 1 inside memory 1314, and a local memory # 2 outside memory 1315.

  The sub memory controllers 1310 and 1311 communicate with the memory controller 1215 in the data controller #i 1200 of FIG. 3 via signal lines 1202 and 1203. The sub memory controllers 1310 and 1311 communicate with the local memory # 1 inside memory 1314 and the local memory # 2 outside memory 1315 via signal lines 1312 and 1313.

  When the node #i is the transmission source node, the data transmitted from the node #i to the data controller #i is written to the local memory # 1 inside memory 1314 for temporary storage of data. Thereafter, the data is read to the data controller #i 1200 and transmitted to the packet router #i 1400 via the signal line 1201 for transfer to the destination node.

  When the node #i is a destination node, the data received by the data controller #i 1200 is written in the local memory # 2 outside memory 1315 for temporary storage of data. Thereafter, the data is read to the data controller #i 1200 and transmitted to the device emulator #i 1100 via the signal line 1104 in order to transfer to the node #i which is the transmission destination node.

  The internal configuration of the packet router #i 1400 will be described with reference to FIG. The packet router #i 1400 includes a data routing unit 1410.

When the node #i is the transmission source node, the data routing unit 1410 writes / reads the data transmitted from the data controller #i 1200 via the signal line 1201 to / from the buffer 1411, and sends it to the transmission destination node. The data is transmitted to the connected data routing unit through the inter-packet router network 1401.
When the node #i is a destination node, the data routing unit 1410 writes / reads data transmitted from the data routing unit connected to the transmission source node via the inter-packet router network 1401 to / from the buffer 1411. Then, the data is transmitted to the data controller #i 1200 through the signal line 1201. In data reading / writing, the top address of data is managed by the address register 1412, and the progress of data processing is managed by the index register 1413.

  With reference to FIG. 6, the hierarchical concept of hardware and software will be described using a general-purpose bus direct coupling network switch 1000 to which two nodes are connected as an example.

  The node is connected to the general-purpose bus direct-coupled network switch 1000 by PCIe 1525 and 1545, respectively, has hardware 1520 and 1540, and includes memories 1521 and 1541 therein. On the hardware 1520 and 1540, OSs 1522 and 1542 are operating. Drivers 1523 and 1543 for controlling the general-purpose bus direct coupling network switch 1000 are installed in the OSs 1522 and 1542, respectively. The user programs 1524 and 1544 are running on the OSs 1522 and 1542, and when there is a communication request in the user programs 1524 and 1544, the request is transmitted to the drivers 1523 and 1543. The drivers 1523 and 1543 communicate with the general-purpose bus direct coupling network switch 1000 via the PCIe 1525 and 1545 in accordance with the request contents. The drivers 1523 and 1543 behave like normal network card drivers for the OSs 1522 and 1542 and the user programs 1524 and 1544. The drivers 1523 and 1543 are special in addition to the drivers 1523 and 1543 installed in the OSs 1522 and 1542 and the user programs 1524 and 1544. The configuration does not require software.

  FIG. 7 shows an example of implementation of the general-purpose bus direct coupling network switch of the present invention.

  The general-purpose bus direct-coupled network switch of the present invention, which is the general-purpose bus direct-coupled network switch 1000 in FIG. 1-6, is shown as switch # 1 1601 in FIG. 7 and installed on the backplane board 1600 of the blade server. To do.

  On the backplane board 1600 there are nodes # 1, 2, 3, N 1620, 1630, 1640, 1650, PCIe sockets # 1, 2, 3, N 1621, 1631, 1641, 1651 and PCIe signal bundle # 1, 2, 3, N 1622, 1632, 1642, 1652 and connected to the switch # 11601. The inter-switch signal line bundle 1602 is for expanding the network by expanding the switch.

8-10, a session construction process, a session release process, and a data transfer process in inter-node communication will be described with reference to FIGS. 1 and 6. FIG. The configuration of the general-purpose bus direct coupling network switch 1000 is based on FIG. 1, the nodes to be used are nodes # 1, # 2 1020, 1040, and the internal configuration is based on FIG. In the description, node # 1 is a transmission node and node # 2 is a reception node, and the following parts are used. Node # 1 driver 2000, node # 1 memory 2001, port # 1 2002, port # 1 device emulator # 1 2003, port # 1 port memory # 1 2004, port # 1 data controller # 1 2005, node # 2 driver 2010, Node # 2 memory 2011, port # 2 2012, port # 2 device emulator # 2 2013, port # 2 port memory # 2 2014, port # 2 data controller # 2 2015. Also,
The node # 1 driver 2000 and the node # 2 driver 2010 in FIG. 8-10 mean the drivers 1523 and 1543 in FIG. The node # 1 memory 2001 and the node # 2 memory 2011 in FIG. 8-10 mean the memories 1521 and 1541 in FIG. Port # 1 2002 and port # 2 2012 in FIG. 8-10 mean the ports 1022 and 1042 in FIG. The port # 1 device emulator 2003 and the port # 2 device emulator 2013 in FIG. 8-10 mean the memories 1024 and 1044 in FIG. Port # 1 port memory # 1 2004 and port # 2 port memory # 1 2014 in FIG. 8-10 mean the port memories 1028 and 1048 in FIG. Port # 1 data controller # 1 2005 and port # 2 data controller # 2 2015 in FIG. 8-10 mean the memories 1026 and 1046 in FIG.

  FIG. 8 illustrates a process of session construction before data transfer is performed via the general-purpose bus direct coupling type network switch 1000.

  When executing data transfer from node # 1 to node # 2, session construction is executed. At that time, the nodes # 1, 2 and drivers 2000, 2010 manage the processing in the nodes # 1, # 2. First, the node # 1 driver 2000 transmits a session connection request 2020 to the port # 1 device emulator # 1 2003 via the port # 1 2002. The session connection request includes information necessary for data transfer, such as an address of transmission source data, a data size, a transmission destination, and a transmission destination.

  The port # 1 device emulator # 1 2003 that has received the session connection request 2020 manages a series of operations of session construction, data transfer, and session release as one transaction viewed from the transmission node. As transaction information, information such as a transmission node, a reception node, a data size, a read source address, a write destination address, data transfer path information, and a buffer are managed.

  In data transfer, a buffer at the data transfer path or relay part is required. In order to secure these resources on the transmission node side, the port # 1 device emulator # 1 2003 transmits a data transfer preparation request 2021 to the port # 1 data controller # 1 2005. The port # 1 data controller # 1 2005 that has received the data transfer preparation request 2021 secures a buffer for a route and a relay part necessary for data transfer, and describes a data transfer preparation completion notification and information necessary for data transfer. The prepared data transfer preparation information 2022 is transmitted. Thereafter, the port # 1 device emulator # 1 2003 sends the session connection information 2020 received from the node # 1 driver 2000 and the data transfer preparation information 2022 received from the port # 1 data controller # 1 2005 to the inter-device emulator networks 1031 and 1051. Via the port 20, send 2023 to the port # 2 device emulator # 2 2013. The port # 2 device emulator # 2 2013 that has received the session connection request and the data transfer preparation information 2023 from the port # 1 data controller # 1 2005 is similar to the port # 1 device emulator # 1 2003. A series of operations for session cancellation is managed as one transaction viewed from the receiving node.

  In order to secure these resources on the receiving node side, port # 2 device emulator # 2 2013 sends a data transfer preparation request to port # 2 data controller # 2 2015 and data transfer preparation information from port # 1 data controller # 1 2005. Is transmitted 2024. Receiving the data transfer preparation request and the data transfer preparation information 2024 from the port # 1 data controller # 1 2005, the port # 2 data controller # 2 2015 secures a buffer for the route and relay part necessary for the data transfer, # 1 Data controller # 1 Based on data transfer preparation information from 2005, the route of the data transfer source is grasped. Thereafter, data transfer preparation completion notification and data transfer preparation information 2025 in which information necessary for data transfer is described are transmitted to port # 2 device emulator # 2 2013. Thereafter, the port # 2 device emulator # 2 2013 that has received the session connection request and the data transfer preparation information 2025 from the port # 2 data controller # 2 2015 connects to the node # 2 driver 2010 via the port # 2 2012. A request 2026 is transmitted.

Through the above process, the transmission of the session connection request from the node # 1 to the node # 2 is completed. Thereafter, the node # 2 driver 2010 enters a process for transmitting a session connection permission notification from the node # 2 to the node # 1. First, the node # 2 driver 2010 transmits a session connection permission notification 2027 to the port # 2 device emulator # 2 2013 via the port # 2 2012. The port # 2 device emulator # 2 2013 that has received the session connection permission notification 2027 from the node # 2 driver 2010 receives the session connection permission notification 2027 from the node # 2 driver 2010 and the data transfer preparation information from the port # 2 data controller # 2 2015. 2025 is transmitted 2028 to the port # 1 device emulator # 12003 via the inter-device emulator networks 1031 and 1051. The port # 1 device emulator # 12003 receiving the session connection permission notification 2027 and the data transfer preparation information 2025 from the port # 2 data controller # 2 2015 receives the data transfer preparation information 2025 from the port # 2 data controller # 2 2015. Send 2029 to port # 1 data controller # 1 2005. The port # 1 data controller # 1 2005 having received the data transfer preparation information 2025 from the port # 2 data controller # 2 2015 is based on the data transfer preparation information 2025 from the port # 2 data controller # 2 2015. Know the route. Thereafter, the port # 1 device emulator # 1 2003 transmits 2030 a session connection permission notification 2027 to the node # 1 driver 2000 via the port # 1 2002.
Through the above process, the node # 1 driver 2000 receives the session connection permission notification 2030, and completes the session construction between the nodes # 1 and # 2.

  FIG. 9 illustrates a process for canceling a session after data transfer is performed via the general-purpose bus direct coupling type network switch 1000. After the data transfer between the nodes # 1 and # 2 is completed, the session between the node # 1 driver 2010 and the node # 2 driver 2010 is released.

  First, a session release request 2120 is transmitted from the port # 1 device emulator # 1 2003 via the port # 2 device emulator # 2 2013 and the port # 2 2012.

  The node # 2 driver 2010 that has received the session release request 2120 transmits a session release notification 2121 to the port # 2 device emulator # 2 2013 via the port # 2 2012.

  The port # 2 device emulator # 2 2013 that has received the session release notification 2121 uses the port to release resources such as the data transfer path secured by the port # 2 data controller # 2 2015 and the buffer at the relay site in data transfer. Data transfer end processing request 2122 is transmitted to # 2 data controller # 2 2015, and after releasing these resources, port # 2 device emulator # 2 2013 sends data transfer end processing release notification 2123 from port # 2 data controller # 2 2015. Receive.

  Thereafter, the port # 2 device emulator # 2 2013 transmits a session release notification 2124 to the port # 1 device emulator # 1 2003 via the inter-device emulator networks 1031 and 1051.

  Port # 1 Device Emulator # 2 When the session release notification is received from 2013, Port # 1 Device Emulator # 1 2003 uses the data transfer path secured by Port # 1 Data Controller # 1 2005, the buffer at the relay site, etc. In order to release resources, a data transfer end processing request 2125 is transmitted to the port # 1 data controller # 1 2005, and after these resources are released, the port # 1 device emulator # 1 2003 receives from the port # 1 data controller # 1 2005. The data transfer end process release notification 2126 is received.

  Finally, the port # 1 device emulator # 1 2003 transmits a session release notification 2127 to the node # 1 driver 2000 via the port # 1 2002, and ends the session release process.

  In FIG. 10, a data transfer process via the general-purpose bus direct coupling network switch 1000 will be described.

  After the session construction is established between the nodes # 1 and # 2, the data transfer process is started. When a session is established, the source node, data read destination address, size, destination node, and data write destination address are managed by the transaction of port # 1 device emulator # 1 2003. Therefore, the node # 1 driver 2000 transmits a data transfer transaction start request 2220 to the port # 1 device emulator # 1 2003 via the port # 1 2002, and the port # 1 device emulator 1 2003 includes a DMA controller. A data transfer start request 2221 is transmitted to the port # 1 data controller # 1 2005 to be transmitted. Receiving the data transfer start request 2221, the port # 1 data controller # 1 2005 requests the DMA read 2222 from the node # 1 memory 2001, and receives the data transfer 2223 in the specified section from the node # 1 memory 2001.

  After that, the port # 1 data controller # 1 2005 writes the received data to the port # 1 port memory # 1 2004, executes 2224, and when the data disappears from the buffer on the port # 1 data controller # 1, if necessary. , Read out 2225 from port # 1 port memory # 12004.

  Thereafter, the data is transferred 2226 from the port # 1 port memory # 1 2004 to the port # 2 port memory # 2 2014 via the packet routers # 1, # 2 1030, 1050.

  After that, when the port # 2 data controller # 2 2015 executes the writing 2227 of the received data to the port # 2 port memory # 2 and the data disappears from the buffer on the port # 2 data controller # 2, if necessary, Read 2228 from port # 2 port memory # 2 2014 is executed.

  Thereafter, the port # 2 data controller # 2 2015 that has received the data executes a data transfer (DMA Write) 2229 to the node # 2 memory 2011, and after transferring the data in the specified section to the node # 2 memory 2011, the port # 2 data A notification 2230 indicating that the data transfer in the designated section is completed is transmitted from the controller # 2 2015 to the port # 2 data controller # 1 2005. In the data transfer, the processes 2222 to 2230 are repeated until the data transfer is completed.

  After the data transfer ends, the port # 1 data controller # 1 2005 sends a data transfer end notification 2232 to the node # 1 driver 2000 via the port # 1 device emulator # 1 2003 and port # 1 2002 which manage the transaction. Send. Thereafter, the port # 1 device emulator # 1 2003 transmits data transfer transaction end notifications 2233 and 2234 to the node # 1 driver 2000 and the node # 2 driver 2010, and the data transfer process ends.

  FIG. 11 illustrates a process in which the data controller 1200 executes reading / writing with respect to the port memory 1300 based on FIGS.

  The data controller 1200 is not always capable of transmitting data immediately after receiving the data, and the buffer 1211 possessed by the DMA controller 1210 built in the data controller 1200 is also finite. When data is written and data is erased from the buffer at the time of data transmission, data is read from the port memory 1300. When data is received from the device emulator 1100 and transmitted to the packet router 1400, the data is written to and read from the inside memory 1314 built in the port memory 1300, and the data is received from the packet router 1400. When transmitting to the emulator 1100, data is written to and read from the outside memory 1315 built in the port memory 1300.

  In port memory writing, the port # 1 DMA controller # 1 2300 receives a transmission permission request for inquiring permission of memory writing from the outside.

  After receiving, the port # 1 DMA controller # 1 2300 transmits a transmission permission request 2320 to the port # 1 sub memory controller # 1 2302 via the port # 1 memory controller # 1 2301, and in response, the port # 1 sub memory controller # 1 2302 The transmission permission 2321 is received from # 1 2302.

  Thereafter, the port # 1 DMA controller # 1 2300 transmits data to the port # 1 local memory # 1 2303 via the port # 1 memory controller # 1 2301 and the port # 1 sub memory controller # 1 2302, and executes writing 2322 In response, a reception completion notification 2323 is received.

  In reading the port memory, the port # 1 DMA controller # 1 2300 receives from the outside a data transmission request for making a read request to the memory.

  After reception, the port # 1 DMA controller # 1 2300 includes the start address and the specified size to the port # 1 local memory # 2 2305 via the port # 1 main controller # 1 2301 and the port # 1 sub memory controller # 2 2304. The data transmission request 2340 is transmitted, and the data transmission 2341 from the port # 1 local memory # 2 2305 is received as a response.

  The session construction process, session release process, and data transfer process in the inter-node communication described with reference to FIG.

  In FIG. 12-14, blocks are arranged in two rows on the left and right sides, meaning the node # 1 side and the # 2 side. As in FIG. 8-10, node # 1 is the transmission node and node # 2 is the reception node.

  A flowchart of the session construction process will be described with reference to FIG. In node-to-node data transfer, session construction is first performed. Step 2400 is set as the start of the session construction process, and the process proceeds to step 2401.

  In step 2401, a session connection request is transmitted 2020 from the node # 1 driver 1523, 2000 to the port # 1 device emulator # 1 1024, 2003.

  In the session connection request, in order to copy the data in the memory 1521 of the node # 1 1020 which is the transmission node to the memory 1541 of the node # 2 1040 which is the reception node, the start address and size of the data in the transmission node , Receiving node destinations are included.

  In the port # 1 device emulator # 1 1024, 2003, the session connection request is transmitted to the transaction management unit 1111 via the PCIe communication unit 1110. The transaction management unit 1111 acquires route information in the general-purpose bus direct coupling type network switch 1000 from the network management unit 1113 based on the destination information in the session connection request. Next, the process proceeds to step 2402. In step 2402, the transaction management unit 1111 of the port # 1 device emulator # 1 1024, 1100 receives data transfer preparation information from the port # 1 data controller # 1 1026, 1200 using signals 2021, 2022. This completes securing the path and buffer used on the transmission node side in the general-purpose bus direct coupling network switch 1000. Next, the process proceeds to step 2403.

  In step 2403, a session connection request and a port # 1 data controller # 1 1026 are transmitted from the port # 1 device emulator # 1 1024, 1100 to the port # 2 device emulator # 2 1044, 1100 via the device emulator network 1031, 1051. Data transfer preparation information from 1200 is transmitted. Thereafter, the port # 1 device emulator # 1 1024, 1100 transitions to step 2404 where it waits for reception of a session connection permission from the port # 2 device emulator # 2 1044, 1100.

  Next, in step 2403, port # 1 device emulator # 1 1024, 1100 sends a session connection request to port # 2 device emulator # 2 1044, 1100 and data transfer preparation information from port # 1 data controller # 1 1026, 1200. Processing step 2405 of port # 2 device emulator # 2 after transmission will be described.

  In step 2405, the port # 2 device emulator # 2 1044, 1100 sends a session connection request and the port # 1 data controller # 1 1026 from the port # 1 device emulator # 1 1024, 1100 via the device emulator network 1031, 1051. Data transfer preparation information from 1200 is received.

  Thereafter, in step 2406, the transaction management unit 1111 of the port # 2 device emulator # 2 1044, 1100 acquires data transfer preparation information from the port # 2 data controller # 2 1046, 1200 using the signals 2024, 2025. . This completes securing the path and buffer used on the transmission node side in the general-purpose bus direct coupling network switch 1000. At this time, the data transfer preparation information from the acquired port # 1 data controller # 1 1026, 1200 is used to set a path to be used for data transfer, and then the process proceeds to step 2407.

  In step 2407, a session connection request is transmitted 2026 from the port # 2 device emulator # 2 1044, 2013 to the node # 2 driver 1543, 2010. Next, the process proceeds to step 2408.

  In step 2408, the node # 2 drivers 1543 and 2010 that have received the session connection request transmit 2027 a session connection permission notification to the port # 2 device emulator # 2 1044 and 2013.

  In the session connection request, the start address of the write area in the receiving node for replicating the data in the memory 1521 of the node # 1 1020, which is the sending node, to the memory 1521 of the node # 2 1040, which is the receiving node. included.

  In the port # 2 device emulator # 2 1044 and 2013, the session connection request is transmitted to the transaction management unit 1111 via the PCIe communication unit 1110. The transaction management unit 1111 acquires route information in the general-purpose bus direct coupling type network switch 1000 from the network management unit 1113 based on the destination information in the session connection request. Next, the process proceeds to step 2409.

  In step 2409, the session connection permission notification and the port # 2 data controller # 2 1046 are transmitted from the port # 2 device emulator # 2 1044, 1100 to the port # 1 device emulator # 1 1024, 1100 via the device emulator network 1031, 1051. Data transfer preparation information from 1200 is transmitted.

  Next, in step 2409, port # 2 device emulator # 2 1044, 1100 sends a session connection permission notification to port # 1 device emulator # 1 1024, 1100 and data transfer preparation information from port # 2 data controller # 2 1046, 1200. Next, the processing step 2410 of the port # 1 device emulator # 1 after transmitting “” will be described.

  In step 2410, the port # 1 device emulator # 1 1024 and 1100 are notified of the session connection permission and the port # 2 data controller # 2 1046 from the port # 2 device emulator # 2 1044 and 1100 via the device emulator network 1031 and 1051, respectively. The data transfer preparation information from 1200 is received 2028.

  Thereafter, in step 2411, the transaction management unit 1111 of the port # 1 device emulator # 1 1024, 1100 sends a signal 2029 to the port # 1 data controller # 1 1026, 1200 to send the port # 2 data controller # 2 1046, Data transfer preparation information from 1200 is transmitted. Thus, the data transfer preparation information from the acquired port # 2 data controller # 2 1046, 1200 is used to set the path used for data transfer, and then the process proceeds to step 2412.

  In step 2412, a path and a buffer necessary for data transfer are secured in the general-purpose bus direct coupling type network switch 1000, and the port # 1 device emulator # 1 1024, 1100 to the node # 1 driver 1523, 2000. A session connection permission notification 2030 is transmitted.

  Finally, in step 2413, the above processing is confirmed, and the session construction process ends.

  A flowchart of the session release process will be described with reference to FIG. In the data transfer between nodes, the session is finally released. Step 2500 is set to start the session release process, and the process proceeds to step 2501.

  In step 2501, a session release request is transmitted 2120 from the port # 1 device emulator # 1 1024, 2003 to the port # 2 device emulator # 2 1044, 2013 via the network between device emulators.

  Since the information for identifying the session is possessed by the transaction management unit 1111 in the port # 1 device emulator # 1 1024, 2003, the session release request is transmitted from the transaction management unit 1111.

  Thereafter, the port # 1 device emulator # 1 1024, 1100 transitions to step 2502 where the reception of the session release notification from the port # 2 device emulator # 2 1044, 1100 is awaited.

  Next, processing step 2503 of port # 2 device emulator # 2 after port # 1 device emulator # 1 1024, 1100 transmits a session connection request to port # 2 device emulator # 2 1044, 1100 in step 2501 will be described. To do. In step 2503, the port # 2 device emulator # 2 1044, 1100 receives 2120 the session release request from the port # 1 device emulator # 1 1024, 1100 via the inter-device emulator network 1031, 1051.

  Thereafter, in step 2504, a session release request is transmitted 2120 from the port # 2 device emulator # 2 1044, 2013 to the node # 2 driver 1543, 2010. Next, the process proceeds to step 2505.

  The node # 2 drivers 1543 and 2010 that have received the session release request send 2121 a session release notification to the port # 2 device emulator # 2 1044 and 2013. In the port # 2 device emulator # 2 1044 and 2013, the session release notification is transmitted to the transaction management unit 1111 via the PCIe communication unit 1110.

  Thereafter, in step 2506, the transaction management unit 1111 of the port # 2 device emulator # 2 1044, 1100 sends a data transfer end processing completion notification from the port # 2 data controller # 2 1046, 1200 using the signals 2122, 2123. Receive. As a result, the path and buffer used on the receiving node side in the general-purpose bus direct coupling network switch 1000 are completely released. Next, the process proceeds to step 2507.

  In step 2507, a session release notification is transmitted 2124 from the port # 2 device emulator # 2 1044, 1100 to the port # 1 device emulator # 1 1024, 1100 via the inter-device emulator network 1031, 1051.

  Next, in step 2507, processing step 2508 of port # 1 device emulator # 1 after port # 2 device emulator # 2 1044, 1100 transmits a session connection permission notification to port # 1 device emulator # 1 1024, 1100. explain.

  In step 2508, the port # 1 device emulator # 1 1024, 1100 receives a session release notification from the port # 2 device emulator # 2 1044, 1100 via the device emulator network 1031, 1051.

  Thereafter, in step 2509, the transaction management unit 1111 of the port # 1 device emulator # 1 1024, 1100 sends a data transfer end processing completion notification from the port # 1 data controller # 1 1026, 1200 using signals 2125, 2126. Receive. As a result, the path and buffer used on the transmission node side in the general-purpose bus direct coupling type network switch 1000 are completely released. Next, the process proceeds to step 2510.

  In step 2510, finally, a session connection permission notification 2030 is transmitted from the port # 1 device emulator # 1 1024, 1100 to the node # 1 driver 1523, 2000.

  In step 2511, the above processing is confirmed, and the session cancellation process ends.

  A flowchart of the data transfer process is described with reference to FIGS. 14a, 14b and 14c. In step 2600, the data transfer process is started, and the process proceeds to step 2601 to confirm that the session connection is completed in the node # 1 driver. Next, the process proceeds to step 2602.

  In step 2602, a data transfer transaction start request is transmitted 2220 from the node # 1 driver 1523 to the port # 1 device emulator # 1 1024, 1100.

  In step 2603, a data transfer start request is transmitted 2221 from the port # 1 device emulator # 1 1024, 1100 to the port # 1 data controller # 1 1026, 1200, and the process proceeds to step 2604.

  In step 2604, a DMA Read is transmitted 2222 from the port # 1 data controller # 1 1026, 1200 to the node # 1 memory 1521 and data is received 2223 in order to transfer the data in the section that can be read / written by DMA. .

  In step 2605, it is confirmed whether the data requested by DMA Read has been acquired. If it is not completed, the process proceeds to step 2604. If completed, the process proceeds to step 2606.

  In step 2606, data is stored in the port # 1 port memory # 1 1028, 1300 using the signal 2224. Next, in step 2607, port # 1 data controller # 1 1026, 1200 to port # 2 data controller # 2 1046, 1200 port # 1 packet router # 1 1030, 1400, port # 2 packet router # 2 1050, 1400 The data transmission 2226 is started via the inter-packet router networks 1033 and 1053.

  However, in step 2608, it is confirmed whether the port # 1 data controller # 1 1026, 1200 has received a data transfer pause interrupt from the port # 1 packet router # 1 1030, 1400. If No, the process proceeds to step 2613, and the data transmission from the port # 1 data controller # 1 1026, 1200 to the port # 2 data controller # 2 1046, 1200 is completed.

  If port # 1 data controller # 1 1026, 1200 receives a data transfer pause interrupt from port # 2 data controller # 2 1046, 1200 at step 2609, port # 1 data controller # 1 1026, 1200 at step 2610 Data transmission from port # 2 to data controller # 2 1046, 1200 is suspended. Thereafter, in step 2611, the port # 1 data controller # 1 1026, 1200 receives the transfer resumption permission notification from the port # 2 data controller # 2 1046, 1200. However, if the data has disappeared from the buffer of the port # 1 data controller # 1 1026, 1200 at step 2612, the data read from the port # 1 port memory # 1 1028, 1300 is executed 2225, and the processing returns to step 2607.

  When data transmission is completed at port # 1 data controller # 1 1026, 1200, the process returns to step 2613.

  Next, port # 2 data controller # 2 1046, 1200 when port # 1 data controller # 1 1026, 1200 starts transmitting data to port # 2 data controller # 2 1046, 1200 in step 2607 will be described. .

  In step 2614, the port # 2 data controller # 2 1046, 1200 starts receiving data 2226 from the port # 1 data controller # 1 1026, 1200. However, in step 2615, it is determined whether the port # 2 data controller # 2 1046, 1200 transmits a data transfer pause interrupt from the port # 2 packet router # 2 1050, 1400. If Yes, the process proceeds to step 2616. If No, the process proceeds to step 2619, and the data reception from the port # 1 data controller # 1 1026, 1200 of the port # 2 data controller # 2 1046, 1200 is continued.

  If port # 2 data controller # 2 1046, 1200 sends a data transfer pause interrupt to port # 1 data controller # 1 1026, 1200 in step 2616, port # 2 data controller # 2 1046, 1200 is sent in step 2617. Data reception from port # 1 data controller # 1 1026, 1200 is temporarily stopped.

  Thereafter, in step 2618, the port # 2 data controller # 2 1046, 1200 transmits a transfer restart permission notice to the port # 1 data controller # 1 1026, 1200, and the process returns to step 2614. If it is determined in step 2615 that the transfer suspension interrupt has not been transmitted, the port # 2 data controller # 2 1046, 1200 continues to receive data from the port # 1 data controller # 1 1026, 1200 in step 2619.

  Thereafter, in step 2620, writing 2227 of the received data to the port # 2 port memory # 2 1048, 1300 is started.

  In step 2622, it is determined whether the port # 2 data controller # 2 1046, 1200 can DMA-write data to the node # 2 memory 1541.

  In step 2623, the operation is temporarily stopped, and then the process returns to step 2622 again. In step 2622, it is determined whether the port # 2 data controller # 2 1046, 1200 can DMA write data to the node # 2 memory 1541. If Yes, the flow proceeds to step 2624, and the port # 2 port memory # 2 1048, 1300 starts. Data read 2228 is executed.

Thereafter, in step 2625, the DMA write of the data in the section allocated from the port # 2 data controller # 2 1046, 1200 to the node # 2 memory 1541 is completed, and a data transfer end notification is sent to the port # 1 data controller # 1 1026. Next, in step 2625, the port # 2 data controller # 2 1046, 1200 transmits the data transfer end notification to the port # 1 data controller # 1 1026, 1200. The port # 1 data controller after transmission 2230 # 1 1026 and 1200 will be described.

  First, in step 2626, the port # 1 data controller # 1 1026, 1200 receives the data transfer end notification from the port # 2 data controller # 2 1046, 1200 2230, and in step 2627, the transfer of all data is completed. If it is No, the process returns to Step 2604, and the operation is repeated. If it is Yes, the process proceeds to Step 2628.

In step 2628, a data transfer end notification is transmitted 2231 from port # 1 data controller # 1 1026, 1200 to port # 2 data controller # 2 1046, 1200. In step 2629, port # 1 data controller # 1 1026, 1200 to port A data transfer end notification is transmitted 2232 to the node # 1 driver 1523 via the # 1 device emulator # 1 1024 and 1100.

  In step 2630, a data transfer transaction end notification 2234 is transmitted from the port # 1 device emulator # 1 1024, 1100 to the port # 2 device emulator # 2 1044, 1100 via the inter-device emulator networks 1031, 1051, 1105. Then, the port # 1 device emulator # 1 1024, 1100 sends a data transfer transaction end notification 2233 to the node # 1 driver 1523 in the same manner.

  Next, in step 2628, for port # 2 data controller # 2 1046, 1200 after port 2 data controller # 1 1026, 1200 sends 2231 a data transfer end notification to port # 2 data controller # 2 1046, 1200 explain.

  In step 2632, the port # 2 data controller # 2 1046, 1200 receives the data transfer end notification 2231 from the port # 1 data controller # 1 1026, 1200.

  In step 2633, a data transfer end notification is transmitted 2231 from the port # 2 data controller # 1 1046, 1200 to the node # 2 driver 1543 via the port # 2 device emulator # 1 1044, 1100.

  In step 2634, the port # 2 device emulator # 2 1044, 1100 receives the data transfer transaction end notification 2234 from the port # 1 device emulator # 1 1024, 1100 via the inter-device emulator networks 1031, 1051, 1105, and step 2635. Then, the port # 2 device emulator # 1 1044, 1100 transmits a data transfer transaction end notification 2234 to the node # 2 driver 1543 in the same manner. Finally, in step 2632, the data transfer ends with all these processes.

  In FIG. 15, the configuration of the protocol used when the nodes 1020, 104, and 1060 communicate via the general-purpose bus direct coupling type network switch 1000, and the protocol is a device emulator in the general-purpose bus direct coupling type network switch 1000. 1024, 1044, 1064, data controllers 1026, 1046, 1066, port memories 1028, 1048, 1068, packet routers 1030, 1050, 1070 will be described how they are used.

  Since the general-purpose bus direct coupling type network switch 1000 and the paths of the nodes 1020, 1040, and 1060 are connected by PCIe, the packet structure used for data transfer depends on the PCIe protocol, and the packet is connected to the PCIe packet area 1710. The non-PCIe packet area 1720 is divided into two. At this time, only the information of the PCIe packet area 1710 is referred to on the PCIe path, and data transfer between the general-purpose bus direct coupling network switch 1000 and the nodes 1020, 1040, and 1060 is realized. At this time, the non-PCIe packet area does not affect the packet behavior during data transfer on the PCIe bus.

  The general-purpose bus direct coupling type network switch transmits and receives electrical signals in accordance with the PCIe protocol at ports 1022, 1042, and 1062, and transmits and receives packets converted to logic signals to and from device emulators 1024, 1044, and 1064. Thereafter, the PCIe communication unit 1110 in the device emulators 1024, 1044, and 1064 performs signal processing in accordance with the PCIe protocol, and performs data transfer with the transaction management 1111 only in the non-PCIe packet area from which the PCIe packet area 1710 is deleted. Data transfer between the device emulators 1024, 1044, 1064, data controllers 1026, 1046, 1066, port memories 1028, 1048, 1068, and packet routers 1030, 1050, 1070 is performed by referring to the non-PCI packet area. To do.

  The non-PCIe packet area 2720 includes a packet length 2730, a local transaction area 2740, a global transaction area 2750, and a data area 2760.

  The packet length 2720 indicates the size of the non-PCIe packet area 2720.

  The local transaction area 2740 includes a packet type 2741, a status 2742, a requester ID 2743, a requester transaction identifier 2744, a completer ID 2745, a completer transaction identifier 2746, and a signal processing direction identifier 2747.

  The local transaction area 2740 includes signals of device emulators 1024, 1044, 1064, data controllers 1026, 1046, 1066, port memories 1028, 1048, 1068 and packet routers 1030, 1050, 1070 inside the general-purpose bus direct coupling type network switch 1000. Used to control data transfer between parts connected by a line. The packet type 2741 indicates what function the packet is issued for. Each device follows the packet type 2741, and the device that receives the packet determines the behavior. The requester ID 2743 and the completer ID 2745 mean a device that transmits and receives a packet. Requester transaction identifier 2744 and completer transaction identifier 2746 are used by each device to identify local transactions between devices.

  When the requester transmits a packet for the first time, the requester transaction identifier 2744 is determined so as to have a unique value in the requester, and the completer transaction identifier 2746 sets an unspecified parameter. When the completer determines that there is an unknown requester transaction identifier 2744 and an unspecified parameter completer transaction identifier 2746 in the received packet, the completer sets the completer transaction identifier 2746 so as to have a unique value in the requester. The requester ID 2743, completer ID 2745, and requester transaction identifier 2744 are determined by the requester when starting a local transaction and are not changed until the local transaction ends.

  The completer transaction identifier 2746 is an unspecified parameter when a local transaction is started, and is determined by the completer when the completer returns a signal, and is not changed until the local transaction ends.

  The signal processing direction identifier 2747 sets the direction of the signal. The global transaction area 2750 includes a packet type 2751, a status 2752, a requester ID 2753, a requester transaction identifier 2754, a completer ID 2755, a completer transaction identifier 2756, and a signal processing direction identifier 2757.

  The global transaction area 2750 is used for controlling communication between nodes.

  The usage method of the parameter IDs 2751-2757 is the same as the parameters 2741-2747 in the local transaction area. However, the requester ID 2753 and completer ID 2755 of the global transaction area 2750 mean a node that transmits and receives a packet.

  The data area 2760 is composed of a data length 2761 and data 2762.

  Data length 2761 indicates the size of data 2760. The content of the data 2762 differs depending on the 2740 packet type 2741 in the local transaction area. The contents are shown in a data format table 2800 composed of 2810 to 2826.

FIG. 16 explains the data format of the data area in the protocol used when the nodes 1020, 104, and 1060 communicate via the general-purpose bus direct coupling network switch 1000 and how the data is used. To do.
The data format table 2800 is associated with the packet type 2810 in the local transaction area and the data 2820 in the data area. An RTS (Request To Send) 2811 is a transmission permission request, and the data 2762 includes a start address of the reading destination of the memory of the transmission data and a size 2821. A CTS (Clear To Send) 2812 is a transmission permission notification, and the data 2762 has a start address 2822 of the write destination of the memory of received data.

  A DMAr (DMA read) 2813 is a direct memory access read, and the data 2762 has a start address and a size 2823 of the transmission data memory read destination.

  DMAw (DMA write) 2814 is a write by direct memory access, and data 2762 includes a data write destination head address and data 2824. ACK / NACK 2815 is an Acknowledgment / No-Acknowledgement meaning success or failure of processing, and the data 2762 has success / failure and a sequence number 2825.

  DATA 2816 is used when data is transmitted, and data 2762 includes a sequence number, a size, and data 2826.

  The ELSE 2817 is prepared for an instruction necessary for each device, and the data 2762 becomes a device-dependent 2826.

1000: General-purpose bus direct coupling network switch 1020, 1040, 1060 ... Nodes 1022, 1042, 1062, 1102 ... Ports 1024, 1044, 1064, 1100 ... Device emulators 1026, 1046, 1066, 1200 ... Data controllers 1028, 1048, 1068 DESCRIPTION OF SYMBOLS 1300 ... Port memory 1030, 1050, 1070, 1400 ... Packet router 1031 ... Network between device emulators 1033 ... Network between packet routers 1051 ... Network between device emulators 1053 ... Network between packet routers 1071 ... Network between device emulators 1073 ... Between packet routers Network 1100 ... Device emulator #i
1102 ... Port #i 1102
1105: Device emulator network 1110 ... PCIe communication unit 1111 ... Transaction management unit 1112 ... Device emulator communication unit 1113 ... Network management unit 1200 ... Data controller #i
1210: DMA controller 1211 built in data controller #i 1200 ... Buffer 1212 built in DMA controller #i 1210 ... Address register 1213 built in DMA controller #i 1210 ... Index built in DMA controller #i 1210 Register 1215 ... Memory controller 1300 ... Port memory #i
1310: Sub memory controller # 1
1311 ... Sub memory controller # 2
1314: Local memory # 1 built in the port memory #i 1300
1315: Local memory # 2 built in port memory #i 1300
1400: Packet router #i
1401 ... Network between packet routers 1410 ... Data routing unit 1411 ... Buffer 1412 ... Address register 1413 ... Index register 1520 ... Node # 1 hardware 1521 ... Memory 1522 ... OS
1523 ... Driver 1524 ... User program 1540 ... Node # 2 hardware 1600 ... Backplane board 1601 ... Switch # 1
1620 ... Node # 1
1621 ... PCIe socket # 1
1650: Node #N
1651 ... PCIe socket #N

Claims (3)

  In a computer system in which multiple computers are connected via a network switch to transfer data between the memory of each computer, the network switch directly manipulates the data in the device emulator and the memory of each computer that each computer recognizes as a device. Each device emulator is connected to each other via the first network. A data controller, a port memory for storing data during data transfer, and a packet router that determines the routing of data in the network switch are provided for each computer. Each data controller is connected to each other in a second network via a packet router, and two device emulators corresponding to the two computers for establishing and releasing a session for data transfer between the two computers are provided in the first network. 1 net A computer system in which two data controllers corresponding to the two computers execute data transfer via the second network, and execute data transfer via the second network. The compatible device emulator acquires the start address, size, and transmission destination of the memory of the transmitting computer with the transmission data from the driver of the transmitting computer, and the data controller compatible with the transmitting computer is a device emulator compatible with the transmitting computer. Reads data from the memory of the sending computer from the information notified from the data controller, the data controller compatible with the receiving computer writes the data into the memory of the receiving computer, and the device emulator corresponding to the sending computer receives the data from the sending side when the writing is completed. Confirm that the data transfer has been completed Computer system, characterized by.   In the data transfer requested by the OS of the transmitting computer, the driver of the transmitting computer notifies the device emulator corresponding to the transmitting computer of the start address and size of the data to be transferred, and the destination of the destination computer, 2. The computer system according to claim 1, wherein a notification that the data communication by the device emulator corresponding to the transmission side computer is completed is received, and the OS of the transmission side computer is notified that the data communication is completed. .   The computer system according to claim 1, wherein the port memory is connected to the data controller in the network switch and has a function of storing data received by the data controller.

Images