JP4367427B2 - Computer system, communication method and program in parallel computer system - Google Patents

Description

  The present invention relates to a computer system, a communication method in a parallel computer system, and a program, and more particularly to a communication technique between computer systems constituting a parallel computer system.

  In the field of numerical computation that requires enormous calculations such as structure / collision / fluid analysis, weather simulation, etc., in order to solve more problems or solve problems of larger models, the processing speed is increased. An expensive dedicated machine called a supercomputer has been developed. Such machines could only be used by companies and some researchers at national laboratories. On the other hand, with the recent improvement in performance of workstations and personal computers, it has become possible for non-specialists to easily construct a high-performance computer system called a PC cluster.

  A PC cluster is configured by connecting a plurality of personal computers through a commodity network such as Gigabit Ethernet. Then, the processing speed is increased by distributing the processing of the application in cooperation with each machine using communication software for parallel computation.

  However, recently, with the improvement of CPU performance, in the case of Gigabit Ethernet and the like, there is a case where the network performance becomes a bottleneck and the CPU performance cannot be sufficiently used in an application with a large communication ratio. For this reason, high-speed networks called millinets and Infiniband have begun to be used. These networks are several times better than Gigabit Ethernet. However, improving the performance of a single network, such as Millinet or InfiniBand, is very expensive and has electrical limitations.

  Therefore, a technology has been born to improve the performance by using a plurality of network interface cards (hereinafter abbreviated as NIC). As means for using such a plurality of NICs, there is a technique called network trunking. This is a technology that makes a plurality of NICs virtually appear as one NIC. Using network trunking, the theoretical performance is the sum of the performance of the NIC used. That is, if network trunking is performed using N NICs, the performance becomes N times.

  As a related technology, an interconnection network including a multistage switching network having two buffers between the processor and the memory is provided, and the traffic allocation ratio is controlled based on the detection result of the traffic state of the two multistage switching networks. Japanese Patent Application Laid-Open No. H11-228707 describes a traffic route designation technique between a processor and a memory. This technique alleviates congestion by localizing the heterogeneous part of the traffic to one switching network and allocating the remaining part of the traffic to the second switching network. In addition, the path designation method is dynamically changed by detecting a traffic imbalance so that performance is maximized under all conditions.

JP-A-7-28764

  By the way, an application operating on a parallel computing system constructed by connecting a plurality of machines via a network advances the calculation while communicating between the machines. With recent improvements in CPU performance, network communication performance is lower than CPU performance. Therefore, in applications where communication is frequently performed, network communication performance often becomes a bottleneck, and shortening the communication time becomes important in order to shorten the execution time of the application. Here, the wider the peak bandwidth of the network, the faster data of a large size can be transferred. On the other hand, in applications that transfer a lot of small size data such as one integer variable, the peak bandwidth of the network has little effect on the execution time of the application, but messages that can be transferred within a unit time Increasing the number is important for shortening the execution time.

  According to the network trunking described above, the maximum performance is improved, but the number of messages that can be handled at one time does not increase. The reason for this is that if the NIC is virtually combined and the width increases, the message size that can be transferred at a time increases. However, since there is only one NIC, the number of messages that can be transferred at one time is determined by the NIC. It is because it is not different from one time. If the application communication pattern is large for each message size, network trunking will improve performance, but if many small messages are transferred, network trunking can improve performance. Absent.

  In Patent Document 1, since the configuration is a multistage switching network, the network between the processor and the memory is shared. Therefore, if a large amount of communication occurs only between some processors and memories, a hot spot may occur in a path used for the communication, and traffic may be uneven in the entire network. Therefore, by detecting this traffic non-uniformity and changing the communication path, the traffic is made uniform throughout the network to improve transfer efficiency. However, although there are a plurality of communication paths between a specific processor and a memory, the communication performance cannot be improved by simultaneously using a plurality of communication paths because only the path is selected.

  An object of the present invention is to provide a computer system for improving communication performance, a communication method in a parallel computer system, and a program.

A computer system according to one aspect of the present invention is a computer system that constitutes a parallel computer system, and includes a plurality of network interface cards for connecting to other computer systems and a partner connected by a plurality of network interface cards. Measuring means for measuring the amount of communication with the computer system, and switching means for dynamically switching the network interface card to be used in the plurality of network interface cards according to the measured amount of communication , the measuring means, The communication amount is measured by the number of transfer messages per predetermined time, the switching unit performs dynamic switching of the network interface card every predetermined time, and the switching unit has two or more processes as communication partners, and the communication amount If there is a difference of more than a certain amount depending on the communication partner The selects to sequentially use multiple network interface cards in the message transfer, when the amount of communication is a difference of less than a predetermined amount by the correspondent shall be the fixed network interface card to be used for message transfer .

  A computer system according to another aspect of the present invention is a computer system constituting a parallel computer system, which is connected to a reception-side network interface card group of a counterpart computer system and includes a plurality of network interface cards. The number of messages sent to the other computer system from the card group and the sending side network interface card group from the sending side process is measured at regular intervals and stored as sending number information by destination, and based on the communication method list A transmission side communication unit for selecting and transmitting a network interface card included in each of the transmission side network interface card group and the reception side network interface card group of the counterpart computer system; A transmission-side communication method creation unit that creates a communication method list by referring to transmission-source-specific reception number information and transmission-destination-specific transmission number information sent from the partner computer system based on the number of messages received at the reception side of the transmission system The communication method list includes identification information of the network interface card used on the transmission side and the reception side of the partner computer system, and includes reception number information for each transmission source and transmission number information for each transmission destination. Dynamically changed by

  In the computer system of the third development form, a receiving-side network interface card group including a plurality of network interface cards connected to the transmitting-side network interface card group of the counterpart computer system, and a receiving-side network interface card group for each transmission source Receiving side communication unit that measures the number of messages received and passed to the receiving process at regular intervals and stores them as received number information for each transmission source, and creation of a receiving side communication method that transmits the received number information for each transmission source to the partner computer system It is preferable that the receiving side node further includes a unit.

  In the computer system of the fourth development form, the transmission side communication method creation unit has two or more processes as communication partners based on the number of messages measured every predetermined time, and the number of messages depends on the communication partner. If there is a difference above, select to use a plurality of network interface cards sequentially in the message transfer, and if the number of messages is less than a predetermined number depending on the communication partner, the network interface used for message transfer It is preferable to change the communication method list so that the card is fixed.

  In the computer system of the fifth development form, the transmission side communication method creation unit (1) if the number of destinations whose number of messages is equal to or greater than the threshold is less than the number of NICs per node, If the communication method is cyclic, (2) if the number of destinations whose number of messages is equal to or greater than the threshold is equal to or greater than the number of NICs per node and an even number, the communication method with these destinations is single ( 3) If the number of destinations with the number of messages equal to or greater than the threshold is equal to or greater than the number of NICs per node and an odd number, the communication method with the destination with the smallest number of messages is set cyclically to the remaining destinations. It is preferable to change the identification information in the communication method list so that the communication method with the destination is single and (4) the communication method with the other party whose number of messages is less than the threshold is cyclic.

A communication method in a parallel computer system according to one aspect of the present invention is a method in which a computer system constituting the parallel computer system includes a plurality of network interface cards for connecting to other computer systems, and performs communication. Measuring the amount of communication with the partner computer system connected by the network interface card, dynamically switching the network interface card to be used in the plurality of network interface cards according to the measured communication amount, the in the step of viewing containing, measuring, the traffic measured by the number of transferred messages per predetermined time in switching step performs a dynamic switching of network interface cards at each predetermined time, in switch step, If there are two or more processes to be the communication partner, and there is a difference of more than a predetermined amount depending on the communication partner, select to use multiple network interface cards sequentially for message transfer. If the difference is less than a predetermined amount, the network interface card used for message transfer is fixed .

  According to another aspect of the present invention, there is provided a communication method in a parallel computer system, in which a computer system constituting a parallel computer system is connected to a receiving-side network interface card group of a counterpart computer system and includes a plurality of network interface cards. A method of communicating as a sending node with an interface card group, measuring the number of messages sent from a sending process to a partner computer system via a sending network interface card group at regular intervals, and by destination. The network interface included in each of the transmitting side network interface card group and the receiving side network interface card group of the counterpart computer system based on the communication method list Refers to the step of selecting and transmitting each of the scan cards, and the number-of-receiver-by-source information and the number-of-destination-by-destination information sent from the partner computer system based on the number of messages received at the receiver of the partner computer system Creating a communication method list, and the communication method list includes identification information of network interface cards used on the transmission side and the reception side of the partner computer system, and includes reception number information by transmission source and transmission destination. It is dynamically changed according to the transmission number information.

A program according to an aspect of the present invention is a computer system of a partner connected by a plurality of network interface cards to a computer system that includes a plurality of network interface cards for connecting to other computer systems and constitutes a parallel computer system. traffic measurement process of measuring a between the switching process the network interface card dynamically switched for use in a plurality of network interface cards in accordance with the traffic, to the execution, measurement process, per predetermined time period The switching process performs dynamic switching of the network interface card every predetermined time, and the switching process has two or more processes as communication partners. If there is a difference of more than a predetermined amount, Select to sequentially use multiple network interface cards in the sage transfer, when the amount of communication is a difference of less than a predetermined amount by the communication partner, a fixed network interface card to be used for message transfer.

  A program according to another aspect of the present invention includes a transmission-side network interface card group including a plurality of network interface cards connected to a reception-side network interface card group of a partner computer system, and a parallel computer system as a transmission-side node. A processing method list in which the number of messages sent from the sending process to the partner computer system via the sending network interface card group is measured and stored as the sending number information by destination in the computer system to be configured. Processing for selecting and transmitting the network interface card included in each of the network interface card group on the transmission side and the network interface card group on the reception side of the partner computer system based on A process for creating a communication method list by referring to the received number information by transmission source and the transmitted number information by destination based on the number of messages received at the receiving side of the computer system. The process of creating the communication method list includes the communication method list including the identification information of the network interface card used on the transmission side and the reception side of the partner computer system. To change dynamically.

  According to the present invention, when there are a plurality of NICs per node, even if the communication amount between the processes is uneven depending on the communication partner, a plurality of NICs are used on an average basis. The overhead for handling the NIC can be reduced. As a result, the number of messages that can be transferred per node increases as compared with a single NIC, and communication performance is improved.

  The reason is that the number of messages transferred per unit time is measured, and if the communication volume between each process is uneven depending on the communication partner, the usage rate of multiple NICs is adjusted to be average, This is because the NIC used for message transfer is adjusted so as to be fixed in order to reduce the overhead.

  A computer system according to an embodiment of the present invention is a computer system corresponding to a node constituting a parallel computer system. In the transmission side node of the computer system, a transmission side network interface card group (3 in FIG. 1) connected to the reception side network interface card group (5 in FIG. 1) of the partner computer system and including a plurality of network interface cards; A transmission side communication unit (2 in FIG. 1) and a transmission side communication method creation unit (7 in FIG. 1) are provided. The transmission-side communication unit measures the number of messages transmitted from the transmission-side process (1 in FIG. 1) to the partner computer system via the transmission-side network interface card group at regular intervals, and transmits the transmission number information for each transmission destination (see FIG. 1) and the network interface card included in each of the transmitting-side network interface card group and the receiving-side network interface card group of the partner computer system based on the communication method list (72 in FIG. 1). To send. The transmission-side communication method creation unit is configured to receive source-specific reception number information (52 in FIG. 1) and transmission-destination-specific transmission number information sent from the partner computer system based on the number of messages received on the reception side of the partner computer system. To create a communication method list. The communication method list includes identification information of the network interface card used on the transmission side and the reception side of the partner computer system, and is dynamically changed according to the transmission number information by transmission source and the transmission number information by transmission destination.

  In the receiving node of the computer system, a receiving side network interface card group (4 in FIG. 1) connected to the transmitting side network interface card group (3 in FIG. 1) of the partner computer system and including a plurality of network interface cards. A receiving-side communication unit (5 in FIG. 1) that measures the number of messages received from the receiving-side network interface card group for each transmitting source and passes to the receiving process at regular intervals, and stores it as receiving-number-based receiving number information; A receiving-side communication method creating unit (8 in FIG. 1) that transmits the original reception number information (52 in FIG. 1) to the partner computer system.

  Based on the number of messages measured at regular time intervals, the transmission side communication method creation unit sequentially uses a plurality of network interface cards for message transfer when the number of messages differs by a predetermined number or more depending on the communication partner. If the number of messages is less than a predetermined number depending on the communication partner, the network interface card used for message transfer is fixed. More specifically, (1) when the number of destinations whose number of messages is equal to or greater than the threshold is less than the number of NICs per node, the communication method with these destinations is made cyclic, and (2) the number of messages If the number of destinations for which the number is equal to or greater than the threshold is equal to or greater than the number of NICs per node and an even number, the communication method with these destinations is set to single. (3) If the number is greater than or equal to the number of NICs per node and is an odd number, the communication method with the partner with the smallest number of messages is made cyclic, and the communication method with the other partner is made single, and (4) message The identification information in the communication method list is changed so that the communication method with the other party whose number is less than the threshold value is cyclic.

  When there are multiple network interface cards, the computer system configured as described above dynamically changes the network interface card to be used during execution according to the communication pattern of the application, and changes the network interface card to be used and the communication method. In order to make a change, the changing means is activated at regular intervals separately from the message transfer process. By dynamically switching the network interface card to be used according to the amount of communication, a plurality of network interface cards can be used on average regardless of the communication pattern, and the communication performance can be improved. Hereinafter, it will be described in detail with reference to the drawings in accordance with embodiments.

  FIG. 1 is a block diagram showing the configuration of the parallel computing system according to the first embodiment of the present invention. In FIG. 1, only a pair of processes in a process group of an application that performs computation while communicating with each other in a plurality of computing systems constituting a parallel computing system is shown. The parallel computing system includes a transmission side process 1, a transmission side communication unit 2, a transmission side NIC 3, and a transmission side communication method creation unit 7 indicating a process of the transmission side node at the time of communication in the transmission side node. The receiving side node includes a receiving side NIC 4, a receiving side communication unit 5, a receiving side process 6, and a receiving side communication method creating unit 8. In such a configuration, it is assumed that the transmission process 1 communicates with the reception process 6 via the transmission NIC 3 and the reception NIC 4.

  The transmission side communication unit 2 includes a transmission command analysis unit 21, transmission destination information 22, a used NIC selection unit 23, and transmission destination-specific transmission number information 24. The transmission command analysis means 21 analyzes the transmission destination of the transmission command issued from the transmission side process 1. The transmission destination information 22 is information indicating the transmission destination analyzed by the transmission command analysis unit 21. In the application process group, a unique process ID is assigned to each process in order to identify the process. The transmission destination information 22 corresponds to a process ID for identifying the transmission destination. Based on the destination information 22 and the used NIC / communication method list 72, the used NIC selecting means 23 includes a NIC in the transmitting NIC 3 used on the transmitting side and a NIC in the receiving NIC 4 used on the receiving side at the time of message transfer. And select. The transmission-destination-specific transmission number information 24 is information in which the transmission-destination-specific transmission message number is recorded. The information is reset each time the transmission-destination-specific transmission number information collecting unit 71 refers to the information.

  The transmission-side communication method creation unit 7 includes a transmission-destination-specific transmission number information collection unit 71, a use NIC / communication method list 72, and a use NIC / communication method list generation unit 73. The transmission-destination-by-destination information collection means 71 selects a transmission destination subject to change of the communication method based on the transmission-destination-by-destination information 24, selects a NIC that transmits a message to the transmission destination, and selects it. The information is sent to the use NIC / communication method list generation means 73. The used NIC / communication method list 72 is a list that records which NIC of the transmission side (transmission source) and the transmission destination is used at the time of message transfer and which communication method is used for each transmission destination. Here, the communication method is provided with a mode called “single” in which only a single NIC is used and a mode called “cyclic” in which a plurality of NICs are sequentially used. In the case of cyclic, a plurality of NICs are periodically used in continuous message transmission. The used NIC / communication method list generating unit 73 uses the used NIC / communication method according to the communication method and the used NIC update instruction sent from the transmission destination-specific transmission number information collection unit 71 and the transmission source-specific reception number information collection unit 81. The list 72 is updated.

  The reception-side communication unit 5 includes a transmission source-specific reception number information storage unit 51 and transmission source-specific reception number information 52. The transmission source received number information storage unit 51 analyzes the transmission source of the message received from the receiving side NIC 4 and updates the transmission source reception number information 52. The transmission source-specific reception number information 52 is information in which the number of received messages by transmission source is recorded, and the information is reset each time the transmission source-specific reception number information collecting unit 81 refers to the information.

  The reception-side communication method creation unit 8 includes a transmission source-specific reception number information collection unit 81. The number-of-reception-by-send information collection unit 81 selects a source to be changed in communication method based on the number-of-reception-by-send information 52, selects a NIC that receives a message from the source, and selects the NIC. The information is sent to the use NIC / communication method list generation means 73.

  In the above configuration, the transmission side process 1, the transmission side communication unit 2, the transmission side communication method creation unit 7, the reception side communication unit 5, the reception side communication method creation unit 8, and the reception side process 6 are programmed in the computer system. You may make it implement | achieve by performing. In this case, the transmission side process 1 and the reception side process 6 correspond to application programs for parallel calculation, and the transmission side communication unit 2 and the reception side communication unit 5 are respectively connected to the parallel calculation communication libraries on the transmission side and reception side. The transmission side communication method creation unit 7 and the reception side communication method creation unit 8 correspond to the use NIC / communication method list creation tools on the transmission side and the reception side, respectively.

  Next, the operation of the parallel computing system will be described in detail with reference to FIG. In the message transmission, the used NIC selection means 23 refers to the used NIC / communication method list 72 and selects the NIC to be used on the transmission side and the reception side to perform communication. Further, the transmission command analyzing means 21 stores the number of transmitted messages in the transmission destination-specific transmission number information 24, and the transmission destination-specific transmission number information 24 is incremented by one every time one message is transmitted. On the other hand, the transmission source reception number information storage unit 51 stores the number of received messages in the transmission source reception number information 52, and the transmission source reception number information 52 is incremented by one each time one message is received. Is done.

  The transmission-side communication method creation unit 7 activates the transmission-destination-specific transmission number information collection means 71 at regular time intervals separately from the message transmission process, refers to the transmission-destination-specific transmission number information 24, and per unit time. The used NIC / communication method list 72 is updated by identifying a process in which the number of transmitted messages is equal to or greater than the threshold. At that time, the transmission number information 24 by destination is reset. On the other hand, the receiving-side communication method creation unit 8 activates the transmission source reception number information collecting unit 81 at regular time intervals separately from the message reception processing, refers to the transmission source reception number information 52, The process in which the number of received messages is equal to or greater than the threshold value is identified, and the use NIC / communication method list generation means 73 is instructed to update the use NIC / communication method list 72. At that time, the transmission source reception number information 52 is reset.

  The used NIC selection means 23 refers to the used NIC / communication method list 72, indexes the list by the process ID of the transmission destination, and determines the transmitting NIC and receiving NIC used for communication. The transmission side NIC and the reception side NIC used for communication are determined according to each transmission destination. Further, the communication method is determined on both the transmission side and the reception side for each transmission destination. When the communication method is single, the NIC to be used does not change until instructed by the use NIC / communication method list generation means 73. When the communication method is cyclic, the communication system is updated to use the next NIC after referring to the list. For example, when there are two NICs, NIC0 and NIC1 are used in order, and when there are four NICs, NIC0, 1, 2, and 3 are repeatedly used.

  As shown in FIG. 2, the initial value of the used NIC / communication method list 72 is cyclic for all communication methods, and the used NIC on both the transmission side and the reception side is the remainder obtained by dividing the ID of the transmission destination by the number of NICs per node. It is a NIC corresponding to the number.

  FIG. 3 is a diagram illustrating an example of changing the used NIC / communication method list 72 of process ID0. The used NIC / communication method list 72 is updated as shown in FIG. 3B when a message is transmitted once from process ID 0 to process ID 1 and process ID 2 in the state of FIG. Is done. That is, the transmission side use NIC number of process ID1 and the reception side use NIC number of process ID2 are updated from 0 to 1.

Next, a method of updating the used NIC / communication method list 72 through the used NIC / communication method list generating unit 73 by the transmission-destination transmission number information collecting unit 71 activated at regular time intervals will be described. The transmission-destination-by-destination information collection means 71 is activated at regular intervals, and confirms the number of messages by transmission destination transmitted during the current activation from the last activation with the transmission-destination-specific transmission number information 24. The communication method is set according to the following conditions.
(1) When the number of destinations whose number of messages is equal to or greater than the threshold is less than the number of NICs per node, the transmission side communication method of those destinations is made cyclic.
(2) If the number of messages whose number of messages is equal to or greater than the threshold is equal to or greater than the number of NICs per node and an even number, the transmission side communication method of those destinations is updated to single.
(3) When the number of messages whose number of messages is equal to or greater than the threshold is equal to or greater than the number of NICs per node and an odd number, the number of messages with the smallest number of messages is determined by cyclically setting the transmission side communication method, Update to single.
(4) A transmission destination whose number of transmission messages is less than the threshold value makes the communication method cyclic.
Note that the transmitting side use NIC updates the ID of the transmission destination to a remainder number divided by the number of ports per node.

Next, a description will be given of a method in which the transmission source reception number information collecting unit 81 activated at regular time intervals updates the used NIC / communication method list 72 through the used NIC / communication method list generating unit 73. The transmission source reception number information collecting unit 81 is activated at regular intervals, and checks the number of transmission source messages received during the current activation from the last activation by the transmission source reception number information 52. The communication method is set according to the following conditions.
(1) When the number of messages whose number of messages is greater than or equal to the threshold is less than the number of NICs per node, the communication method on the receiving side of those sources is cyclic.
(2) When the number of messages whose number of messages is equal to or greater than the threshold is equal to or greater than the number of NICs per node and an even number, the receiving side communication method of those sources is updated to single.
(3) When the number of messages is greater than or equal to the threshold and the number of NICs per node is an odd number and the number of messages is odd, the source with the smallest number of messages makes the receiving side communication method cyclic and the remaining sources are Update to single.
(4) The transmission source whose number of received messages is less than the threshold value makes the communication method cyclic.
The receiving side use NIC updates the transmission source ID to the remainder obtained by dividing the transmission source ID by the number of ports per node.

  FIG. 3C shows an example of the used NIC / communication method list 72 when the communication method is changed as described above. In the state of FIG. 3A, in the process ID 2, the number of messages from the process ID 0 exceeds the threshold and exceeds the threshold from the time when the source-specific reception number information collecting unit 81 is activated until the current activation. Suppose that the number of transmission sources becomes equal to or greater than the number of NICs. In this case, the change of the communication method is instructed from the received number information collection means 81 of the process ID 2 to the use NIC / communication method list generation means 73 of the process ID 0, and the use NIC / communication method list 72 of the process ID 0 is updated. Is done. That is, the communication method on the receiving side of process ID 2 is changed from cyclic to single according to the condition (2).

  Next, the relationship between the communication method and the communication amount will be described using a specific example. FIG. 4 is a diagram showing a communication form, and on a machine having one NIC per node, alternate communication from process ID 0 to process ID 2 and process ID 3, and from process ID 1 to process ID 2 and process ID 3 This shows that alternate communication is being repeated simultaneously. Transmission from process ID 0 to process ID 2 and process ID 3 cannot be performed at the same time because there is only one NIC and is performed alternately. Transmission from process ID 1 to process ID 2 and process ID 3 is also performed alternately because there is only one NIC.

  When the NIC latency is 20 μs, a message of about 4 bytes, which is one integer variable, can be transferred in the same time as the latency. Accordingly, the number of messages of about 4 bytes that can be transferred per second is 50,000. At this time, if it is attempted to transmit 25 million 4-byte messages to process ID 2 and process ID 3, process ID 0 cannot be transferred at the same time and must be transferred alternately, which takes 100 seconds. Transfer of data from process ID 1 to process ID 2 and process ID 3 also takes similar time. In process ID 2 and process ID 3, since messages come from both process ID 0 and process ID 1, there is a possibility that reception processes overlap. However, since the received message is once stored in the buffer, if the CPU speed is sufficiently higher than the communication performance of the network, reception processing can be performed at the same time.

  Further, as shown in FIG. 5, on a machine having two NICs per node, transmission from process ID 0 to process ID 2 is NIC 0, transmission from process ID 0 to process ID 3 is NIC 1, and from process ID 1 If the transmission to the process ID 2 is NIC0 and the transmission from the process ID 1 to the process ID 3 is fixed as the NIC 1, the transmission to the process ID 2 and the process ID 3 can be performed at the same time. . In this case, since the message to be transmitted is once stored in the buffer, if the CPU speed is sufficiently higher than the communication performance of the network, the message can be transmitted simultaneously from process ID 0 to process ID 2 and process ID 3. . However, for example, when a large amount of messages are transmitted only from process ID 0 to process ID 2 and almost no message is transmitted from process ID 0 to process ID 3, the usage rates of the two NICs are biased, and there are a plurality of messages. NIC cannot be used on average.

  Further, in FIG. 6, a message is transmitted by alternately using the NIC on the transmission side and the NIC on the reception side on a machine having two NICs per node. When the process ID 0 first transmits a message to the process ID 2, the process ID 0 is transmitted from the transmitting NIC 0 to the receiving NIC 0. Next, when a message is transmitted to process ID 2, the message is transmitted from NIC 1 on the transmission side to NIC 1 on the reception side. Transmission from process ID 0 to process ID 3 and transmission from process ID 1 to process ID 2 and process ID 3 are performed in the same manner. In this case, since the NIC on the transmission side and the NIC on the reception side are alternately used, even if there is a bias in the communication amount depending on the communication partner, a plurality of NICs are used on average. However, when there are a plurality of NICs for the transmission destination, there is a cost for processing when the order of messages arriving at the receiving side is changed. When this cost is converted into latency and set to 10 μsec, the latency that was 20 μsec in the example of FIG. 4 becomes 30 μsec. At this time, the number of messages that can be transferred per second is 33333. If a process ID 0 attempts to send a 4-byte message of 2500000 pieces to process ID 2 and process ID 3, it can be transferred in about 70 seconds.

  In the present embodiment, the message transmission from process ID 0 to process ID 2 and process ID 3 and the message transmission from process ID 1 to process ID 2 and process ID 3 are cyclic in the initial state. Therefore, the latency is 30 μsec, and the number of messages that can be transferred per second is 33333. For example, when the transmission-destination-by-destination information collecting unit 71 is activated at intervals of 10 seconds, for example, the transmission-destination-by-destination information collecting unit 71 is activated after 10 seconds, and the transmission-destination-by-destination information 24 is checked. Since the number of messages sent to the process ID 2 and the process ID 3 is sufficiently large and exceeds the threshold, and the number of transmission destinations equal to or more than the threshold is two and an even number, the transmission side communication method is single. In addition, when the transmission source-specific reception number information collecting unit 81 is activated at intervals of 10 seconds, for example, the transmission source-specific reception number information collecting unit 81 is activated 10 seconds later, and the transmission source-specific reception number information 52 is examined. Since the number of received messages from process ID 0 and process ID 1 is sufficiently large and equal to or greater than the threshold, and the number of transmission sources equal to or greater than the threshold is two and an even number, the receiving side communication method is single.

  When the communication method on the transmission side and the reception side is single, the latency is 20 μs as described with reference to FIG. 5, so that 50000 messages can be transferred per second. In the first 10 seconds, 333330 messages are transmitted by the cyclic communication method, and the remaining are 250000000-333330 = 2166670 messages, and the time required to transmit 2500000 messages is approximately 43 seconds. Therefore, it takes about 53 seconds to send 2500000 messages. In this embodiment, the processing speed can be increased by 1.32 (= 70/53) times as compared with communication using cyclic communication for all transfers. Compared to 50 seconds when the NIC is fixed and used when the communication volume of each communication partner is balanced, the performance difference is within 10%.

  Further, even if the communication pattern changes after a certain time, the transmission number information collecting means 71 for each transmission destination and the reception number information collecting means 81 for each transmission which are activated at intervals of 10 seconds, for example, detect a change in the communication pattern. Thus, since the communication method between the transmission side and the reception side can be changed, a plurality of NICs are used in a balanced manner. That is, it is possible to reduce the overhead of handling a plurality of NICs by using a plurality of NICs on average. As a result, the number of messages that can be transferred per node increases as compared with the case where there is a single NIC, the communication performance is improved, and the execution of the application can be finished earlier.

  FIG. 7 is a block diagram showing the configuration of the parallel computing system according to the second embodiment of the present invention. 7, the parallel computing system is different from FIG. 1 in that, for example, two CPUs are provided per node, and two processes exist on each of the transmission side and the reception side. In the transmission side node, the transmission side process 1a sends a message to the transmission side NIC 3a via the transmission side communication unit 2a, and the transmission side process 1b sends a message to the transmission side NIC 3a via the transmission side communication unit 2b. The transmission side communication units 2a and 2b share the transmission side communication method creation unit 7a. On the other hand, in the receiving side node, the receiving side process 6a receives a message from the receiving side NIC 4a via the receiving side communication unit 5a, and the receiving side process 6b receives a message from the receiving side NIC 4a via the receiving side communication unit 5b. . The receiving side communication units 5a and 5b share the receiving side communication method creating unit 8a.

  In such a configuration, the transmission-by-destination transmission number information 24 shown in FIG. 1 in the transmission side communication units 2a and 2b is managed separately for each process on the transmission side, and in the reception side communication units 5a and 5b. 1 is managed separately for each process on the receiving side. There is one transmission-side communication method creation unit 7a and one reception-side communication method creation unit 8a regardless of the number of processes in the node. At the time of message transmission, the sending processes 1a and 1b refer to the common use NIC / communication method list 72 shown in FIG.

A method of updating the used NIC / communication method list 72 through the used NIC / communication method list generating unit 73 by the transmission-destination-specific transmission number information collecting unit 71 activated at regular time intervals will be described. The transmission-destination-by-destination transmission number information collection unit 71 activated at regular intervals by the transmission-side communication method creation unit 7a is a transmission destination that has been transmitted during the current activation from the last activation of all transmission-side processes in the node. The transmission-destination transmission number information 24 in which another message number is recorded is confirmed.
(1) If the number of transmission destinations with the number of messages equal to or greater than the threshold is less than the number of NICs per node, the transmission side communication method of those transmission destinations is made cyclic.
(2) If the number of transmission destinations with the number of messages equal to or greater than the threshold is equal to or greater than the number of NICs per node and an even number, the transmission side communication method of those transmission destinations is updated to single.
(3) If the number of destinations with the number of messages equal to or greater than the threshold is greater than or equal to the number of NICs per node and an odd number, the destination with the smallest number of messages makes the transmission side communication method cyclic, and the remaining destinations are single. Update.
(4) A transmission destination whose number of transmission messages is less than the threshold value makes the communication method cyclic.
Note that the transmitting side use NIC updates the ID of the transmission destination to a remainder number divided by the number of ports per node.

Next, a description will be given of a method in which the transmission source reception number information collecting unit 81 activated at regular time intervals updates the used NIC / communication method list 72 through the used NIC / communication method list generating unit 73. The reception-side received number information collection means 81 activated at regular intervals by the reception-side communication method creation unit 8a is the transmission source received during the current activation from the last activation of all the reception-side processes in the node. The number-of-reception-by-source information 52 in which the number of other messages is recorded is confirmed.
(1) If the number of transmission sources with the number of messages equal to or greater than the threshold is less than the number of NICs per node, the communication method on the receiving side of those transmission sources is made cyclic.
(2) If the number of messages whose number of messages is equal to or greater than the threshold is equal to or greater than the number of NICs per node and an even number, the receiving side communication method of those sources is updated to single.
(3) If the number of messages with the number of messages equal to or greater than the threshold is greater than or equal to the number of NICs per node and an odd number, the source with the smallest number of messages makes the receiving side communication method cyclic, and the remaining sources are single. Update.
(4) The transmission source whose number of received messages is less than the threshold value makes the communication method cyclic.
The receiving side use NIC updates the transmission source ID to the remainder obtained by dividing the transmission source ID by the number of ports per node.

  The case where two parallel computing application processes exist in the node has been described above, but it goes without saying that there may be three or more processes. According to the present invention, not only a communication pattern of a single process but also a communication pattern of all processes related to computation in a node can be analyzed, and a plurality of NICs can be averaged and overhead for handling a plurality of NICs can be used. .

  The present invention can be applied to the field of high performance computing. Especially in a system where the network transfer speed is slower than the CPU speed, such as connecting a machine with the latest CPU with a relatively slow network device such as Gigabit Ethernet, the processes running on each machine are communicating with each other. It can be applied to run parallel computing applications that solve

1 is a block diagram showing a configuration of a parallel computing system according to a first example of the present invention. It is a figure showing the initial value of a use NIC and a communication method list. It is a figure showing the example of a change of use NIC and a communication method list | wrist. It is a 1st figure showing the communication form between nodes. It is a 3rd figure showing the communication form between nodes. It is a 3rd figure showing the communication form between nodes. It is a block diagram which shows the structure of the parallel computing system which concerns on 2nd Example of this invention.

Explanation of symbols

1, 1a, 1b Transmission side process 2, 2a, 2b Transmission side communication unit 3, 3a Transmission side NIC
4, 4a Receiving side NIC
5, 5a, 5b Reception side communication unit 6, 6a, 6b Reception side process 7, 7a Transmission side communication method creation unit 8, 8a Reception side communication method creation unit 21 Transmission command analysis means 22 Transmission destination information 23 Use NIC selection means 24 Transmission number information by transmission destination 51 Reception number information by transmission source storage means 52 Transmission number information by transmission source 71 Transmission number information by transmission destination collection means 72 Used NIC / communication method list 73 Usage NIC / communication method list generation means 81 Transmission source Separately received number information collection means

Claims (16)

A computer system constituting a parallel computer system,
Multiple network interface cards to connect to other computer systems;
Measuring means for measuring the amount of communication with a partner computer system connected by the plurality of network interface cards;
Switching means for dynamically switching a network interface card to be used in the plurality of network interface cards according to the measured traffic;
Equipped with a,
The measuring means measures the communication amount by the number of transfer messages per predetermined time,
The switching means performs dynamic switching of the network interface card every predetermined time,
The switching means selects to sequentially use the plurality of network interface cards in message transfer when there are two or more processes that are communication partners and the communication amount differs by a predetermined amount or more depending on the communication partner. When the communication amount is less than a predetermined amount depending on a communication partner, a network interface card used for message transfer is fixed. A computer system constituting a parallel computer system,
A transmission side network interface card group including a plurality of network interface cards connected to the reception side network interface card group of the partner computer system; and
The number of messages transmitted from the transmitting side process to the partner computer system via the transmitting side network interface card group is measured at regular intervals and stored as transmission number information by destination, and based on the communication method list, the transmitting side A transmission side communication unit for selecting and transmitting each network interface card included in each of the network interface card group and the reception side network interface card group of the partner computer system;
The transmission for creating the communication method list with reference to the number-of-reception-by-source information and the number-of-transmission-by-destination transmission number information sent from the partner computer system based on the number of messages received at the receiving side of the partner computer system Side communication method creation part,
At the sending node,
The communication method list includes identification information of the network interface card used on the transmission side and the reception side of the partner computer system, and is dynamically changed by the reception number information by transmission source and the transmission number information by transmission destination. A computer system characterized by that. A receiving side network interface card group including a plurality of network interface cards connected to a sending side network interface card group of a partner computer system;
A receiving-side communication unit that measures the number of messages received from the receiving-side network interface card group for each transmitting source and passes to the receiving process at regular intervals, and stores the received number-by-sending-source received number information;
A communication method creation unit on the receiving side that transmits the number-of-reception received number information to the other computer system;
The computer system according to claim 2 , further comprising: a receiver node. Based on the number of messages measured at regular time intervals, the transmission side communication method creation unit has two or more processes that are communication partners, and when the number of messages has a difference of a predetermined number or more depending on the communication partner, In the message transfer, the network interface card is selected to be used sequentially, and when the number of messages is less than a predetermined number depending on the communication partner, the network interface card used for message transfer is fixed. 3. The computer system according to claim 2, wherein the communication method list is changed. The transmission side communication method creation unit
(1) When the number of destinations whose number of messages is equal to or greater than the threshold is less than the number of NICs per node, the communication method with these destinations is set cyclically,
(2) If the number of destinations whose number of messages is equal to or greater than the threshold is equal to or greater than the number of NICs per node and an even number, the communication method with these destinations is made single,
(3) If the number of destinations whose number of messages is greater than or equal to the threshold is greater than or equal to the number of NICs per node and an odd number, the communication method with the destination with the smallest number of messages is cyclically selected, and the remaining counterparts The communication method with the destination is single,
(4) To make the communication method with the other party whose number of messages is less than the threshold value cyclic,
Claim 2 or 4 computer system according and changes the identification information in the communication mode list. Parallel computer system comprising a plurality of computer system according to any one of claims 1-5. A computer system constituting a parallel computer system is provided with a plurality of network interface cards for communication with other computer systems, and communicates.
Measuring the amount of communication with a partner computer system connected by the plurality of network interface cards;
Dynamically switching a network interface card to be used in the plurality of network interface cards according to the measured traffic volume;
Only including,
In the measuring step, the communication amount is measured by the number of messages transferred per predetermined time,
In the switching step, the network interface card is dynamically switched every predetermined time;
In the step of switching, when there are two or more processes that are communication partners, and there is a difference of a predetermined amount or more depending on the communication partners, a selection is made to sequentially use the plurality of network interface cards in message transfer. A communication method in a parallel computer system, wherein a network interface card used for message transfer is fixed when the communication amount is less than a predetermined amount depending on a communication partner. A computer system constituting a parallel computer system is connected to a receiving-side network interface card group of a partner computer system and includes a transmitting-side network interface card group including a plurality of network interface cards and communicates as a transmitting-side node. ,
A step of measuring the number of messages transmitted from the transmitting side process to the other computer system via the transmitting side network interface card group at regular intervals and storing it as transmission number information by destination;
Selecting and transmitting each of the network interface cards included in each of the transmission side network interface card group and the reception side network interface card group of the counterpart computer system based on the communication method list;
Creating the communication method list with reference to the number-of-transmissions received-by-source information and the number-of-destination-by-destination sending information sent from the partner computer system based on the number of messages received at the receiving side of the partner computer system; ,
Including
The communication method list includes identification information of the network interface card used on the transmission side and the reception side of the partner computer system, and is dynamically changed by the reception number information by transmission source and the transmission number information by transmission destination. A communication method in a parallel computer system. The computer system includes a method of communicating as a receiving side node further including a receiving side network interface card group including a plurality of network interface cards connected to a transmitting side network interface card group of a counterpart computer system,
A step of measuring the number of messages received from the receiving-side network interface card group for each transmission source and passing to the reception process at regular intervals, and storing the received number of reception information for each transmission source;
A step of transmitting the number-of-reception received number information to the partner computer system;
The communication method in the parallel computer system according to claim 8 , further comprising: In the step of creating the communication method list, when there are two or more processes that are communication partners based on the number of messages measured every certain time, and there is a difference of a predetermined number or more depending on the communication partners in the number of messages In the message transfer, the plurality of network interface cards are selected to be used sequentially, and when the number of messages is less than a predetermined number depending on the communication partner, the network interface card used for the message transfer is fixed. 9. The communication method in the parallel computer system according to claim 8, wherein the communication method list is changed. In the step of creating the communication method list,
(1) When the number of destinations whose number of messages is equal to or greater than the threshold is less than the number of NICs per node, the communication method with these destinations is set cyclically,
(2) If the number of destinations whose number of messages is equal to or greater than the threshold is equal to or greater than the number of NICs per node and an even number, the communication method with these destinations is made single,
(3) If the number of destinations whose number of messages is greater than or equal to the threshold is greater than or equal to the number of NICs per node and an odd number, the communication method with the destination with the smallest number of messages is cyclically selected, and the remaining counterparts The communication method with the destination is single,
(4) To make the communication method with the other party whose number of messages is less than the threshold value cyclic,
Communication method in a parallel computer system according to claim 8 or 10, wherein changing the identification information in the communication mode list. A computer system comprising a plurality of network interface cards for connecting to other computer systems and constituting a parallel computer system,
A measurement process for measuring the amount of communication with a partner computer system connected by the plurality of network interface cards;
A switching process for dynamically switching a network interface card to be used in the plurality of network interface cards according to the communication amount;
Was executed,
The measurement process measures the communication amount by the number of transfer messages per predetermined time,
The switching process performs dynamic switching of the network interface card every predetermined time,
In the switching process, when there are two or more processes as communication partners and the communication amount differs by a predetermined amount or more depending on the communication partners, the plurality of network interface cards are selected to be used sequentially in message transfer. A program that is a process for fixing a network interface card used for message transfer when the communication amount is less than a predetermined amount depending on a communication partner. A computer system that is connected to a reception-side network interface card group of a partner computer system and includes a transmission-side network interface card group including a plurality of network interface cards and constitutes a parallel computer system as a transmission-side node.
A process of measuring the number of messages transmitted from the transmission side process to the partner computer system via the transmission side network interface card group at regular intervals and storing it as transmission number information by destination,
A process of selecting and transmitting each network interface card included in each of the transmitting network interface card group and the receiving network interface card group of the counterpart computer system based on a communication method list;
Processing for creating the communication method list with reference to the number-of-reception-by-source information and the number-of-transmission-by-destination transmission information sent from the partner computer system based on the number of messages received at the receiving side of the partner computer system; ,
And execute
The process of creating the communication method list includes the communication method list including identification information of a network interface card used on the transmission side and the reception side of the partner computer system, the transmission number information by transmission source and the transmission by transmission destination A program that is dynamically changed according to numerical information. In the process of communicating as a receiving side node, the computer system further includes a receiving side network interface card group including a plurality of network interface cards while being connected to the transmitting side network interface card group of the counterpart computer system.
A process of measuring the number of messages received from the network interface card group on the receiving side for each transmission source and passing to the reception process at regular intervals, and storing the received number of received information as a transmission source,
A process of sending the number-of-reception received number information to the other computer system;
14. The program according to claim 13 , further comprising: The process of creating the communication method list is based on the case where there are two or more processes that are communication partners based on the number of messages measured at regular intervals, and there is a difference of a predetermined number or more depending on the communication partners. In the message transfer, the plurality of network interface cards are selected to be used sequentially, and when the number of messages is less than a predetermined number depending on the communication partner, the network interface card used for the message transfer is fixed. 14. The program according to claim 13, wherein the communication method list is changed. The process of creating the communication method list includes:
(1) When the number of destinations whose number of messages is equal to or greater than the threshold is less than the number of NICs per node, the communication method with these destinations is set cyclically,
(2) If the number of destinations whose number of messages is equal to or greater than the threshold is equal to or greater than the number of NICs per node and an even number, the communication method with these destinations is made single,
(3) If the number of destinations whose number of messages is greater than or equal to the threshold is greater than or equal to the number of NICs per node and an odd number, the communication method with the destination with the smallest number of messages is cyclically selected, and the remaining counterparts The communication method with the destination is single,
(4) To make the communication method with the other party whose number of messages is less than the threshold value cyclic,
16. The program according to claim 13, wherein the identification information in the communication method list is changed.

Images