JP2007520093A - Group-to-group communication and error-tolerant symmetric multicomputing system on a single connection - Google Patents

Abstract

An extended TCD / IP, for example, is described herein in a system capable of reliable and ordered data communication between two node sets with atomic multipoint transport as well as multipoint transfer. When multiple nodes are transported with data, the transport is performed atomically. An error-tolerant and symmetric multi-computing system that utilizes a group of nodes is also described herein. Symmetrical groups on networked nodes utilizing a reliable, ordered and atomic group-to-group TCP communication system are used to provide error tolerance and single system image to client applications . Communication between the client and the group is based on a standard because an arbitrary standard TCP / IP endpoint can communicate with the group without interruption. The processing load is shared among a group of nodes by transparent task distribution to application segments. This system is error-tolerant in that even if one node may fail, the remaining replicas will continue service without interruption of service or connection.

Description

  The present invention relates to network communication between n to n points in one network where n may be any integer value.

Brief description of related technology

  For optimal resource utilization, flexibility and management cost reduction, industry can add processing and storage capabilities as needed to meet changing needs and resources are dynamically prepared There is a need for a solution based on the “utility computing” model. Traditional mainframe solutions are too expensive to reach the average enterprise. There are many high performance ones, but there are low cost “blade servers” and network technologies available on the market. However, no solution exists today that can efficiently and flexibly gather these resources and run a wide range of applications that meet the needs of utility computing.

  The client-server paradigm is popular in the industry for its ease with which clients can make requests and servers respond. To enable this paradigm, a popular communication protocol utilized between a client and server in a communication network is the transfer control protocol / Internet protocol or simply “TCP / IP”. In a communication network, a client (or client system or machine) sees a server (or client system or machine) as a single logical host or entity. A single physical server often cannot service many clients efficiently. In addition, the server that caused the error renders the client inoperable.

  In order to address the shortcomings of single physical servers, cluster configurations with many servers running in parallel or grids serving clients have been developed using load balancers. These configurations provide potential benefits such as error tolerance, low cost, efficiency and flexibility comparable to mainframes. However, most of these and other benefits remain unrealized due to their inherent limitations and the lack of a standard platform on which many applications can be built.

  In addition to physical clustering, traditional software systems have also made efforts to introduce clustering at the application and operating system level. However, the disadvantages of such software configurations include instances where clustering is embedded in the application outcomes with limited use of these applications. Similarly, although operating system level clustering is attractive, traditional efforts in these areas have not been successful so far due to the many abstractions that must be virtualized.

  Compared to physical servers and software applications and operating system clustering, network level clustering not only suffers from any problems, but also offers some attractive benefits. For example, the ability to address a cluster of server nodes as a single virtual entity is a useful requirement for client-server programming. In addition, the ability to easily create virtual clusters in a pool of nodes adds to the convenience and flexibility of the mainframe class.

  Traditional network level clustering platforms must be universal and usable by a wide range of applications. These applications range from web servers, storage servers, database servers, scientific and application grid computing. These traditional network-level clusters must be capable of combining computing power and node capacity, so applications will grow in size gradually without a seam. Existing applications must be able to run with a minimum number or changes. However, traditional network level clusters have had limited success.

  The “Symmetric Multiprocessor” (SMP) architecture is somewhat successful because of the simplicity of the bus, which makes the processing and storage location transparent to the application. With respect to clustering, the transparency of the virtual bus connecting the server nodes also provides the transparency of the node location and the transparency of the node ID. However, such conventional systems lack the ability for the bus to be directly connectable by client applications for efficiency. Similarly, buses based on “User Datagram Protocol” (“UDP”) packet broadcasts and multiplex broadcasts lack the guarantee of data transport and cause application level clustering.

  TCP / IP is the only protocol with guaranteed transport that is most widely used in the industry. This is particularly desirable for virtualization due to the guarantee of TCP data transport with ordered transport guarantees and ubiquity. However, the possibilities were limited by the support of TCP for only two endpoints per connection. Asymmetric configuration of element / node processing with pre-assigned tasks to distribute incoming requests to the cluster, not only inherently inflexible, but also difficult to manage and balance load It is. An asymmetric node is often a single point that becomes an error or bottleneck. In order for MC (multicomputing) to succeed, a symmetric configuration is required to face the asymmetric node configuration.

  Another problem with asymmetry in the client / server environment is latency. Dedicated hardware is used to reduce the waiting time for data passing through switches and routers. When data must pass through the node's UDP / TCP / IP stack, considerable latency is added by replicas and processing. Therefore, for optimal performance, the system must avoid the passage of data through a coordination node that has an asymmetric configuration. However, when the CPU of the server node has to handle a large amount of network traffic, the processing amount of the application and its processing are suffered. Thus, conventional systems must utilize a dedicated adapter card or hardware accelerator such as an “integrated circuit” chip that reduces latency at endpoints and improves application performance. This increases the cost and complexity of the system.

  Low cost error tolerance is highly desired by many enterprise applications. Solutions that utilize a fixed number of extra hardware components suffer from inflexibility, lack of easy repair capabilities, and higher costs due to complexity. Today's solutions provide a high degree of usability by quickly switching services to a standby server after an error occurs. Since the standby system is passive, its resources are only unused because of high costs. In the simplest and most powerful form of error tolerance with replicas, services over one connection can continue without interruption due to node errors.

  On a traditional cluster, the active node performs the task and the passive node updates subsequent changes. In many instances, the number of updates is less compared to other tasks such as queries. A machine is best utilized when the load is shared with all replicas while the updates are reflected in the replicas. Replica updates must be done at the same time and at the same time for consistency. With respect to atomic transport, data is guaranteed to be transported to all target endpoints before the client is sent with a TCP ACK that indicates receipt of the data. In the event of a replica error, the remaining replicas can continue service while avoiding connection interruptions that affect error tolerance. Non-atomic replicas lack usability. Specifically, when a client request is accepted by a service replica, each produces a certain response. Since the client sees the server as a single entity, it must be ensured that only one instance of the response is sent back to the client. Similarly, if a multi-client replica attempts to send the same request, it must be ensured that only one instance is sent to the server. Conventional systems often fail to provide atomicity, thus lacking ease of use and error tolerance to avoid disrupting connections.

  Another problem with conventional clustering is load balancing. For any system, the ability to balance the load evenly within the node is necessary for optimal application performance. However, conventional clustering systems provide only limited support due to the standard scheme of load balancing such as round robin, content subdivision, and priority weighting. Furthermore, many conventional clustering systems are unable to support the execution of specific load balancing scheme applications.

  Many services have very variable load levels in time-dependent clusters. The running process may need to be moved to evacuate the active server. Conventional cluster systems often lack support for adding or moving nodes / replicas to the cluster so that they are easily implemented and without service interruption.

  Numerous attempts have been made to tackle network level virtualization. However, each attempt still has some serious drawbacks. For example, one conventional solution is a load balancing device in a cluster of “web servers” popular in the industry. This load balancer, also published in US Pat. Nos. 6,006,264 and 6,449,647, switches incoming client TCP connections to a server in a pool of servers. A conventional server for this processing is “Microsoft network load balancing software that broadcasts or multiplex broadcasts client packets to all nodes via a switch or router. However, once a connection is mapped, the same server is Handles all client requests during the lifetime of a TCP connection in a conventional one-to-one relationship.

  The problem with conventional systems as described above is that a mapped server that cannot perform all the services requested by the client during one connection when the service includes different tasks that execute on multiple nodes. If there is even one, a service error occurs, so a complete solution cannot be provided. This limits the use of such a system for web page provisioning services where only one task that provides a page is replicated to many nodes. Furthermore, any mapping of external devices executed for one server becomes a bottleneck and a single point becomes an error. Additionally, replication is not supported because one connection has only two endpoints. Therefore, with respect to such a single end point TCP, the update is not reflected on the replica, so that there is a considerable limitation in usability.

  In order to address some of the shortcomings of the conventional system described above, another conventional system has attempted to distribute client requests across a connection to nodes that provide different tasks. Ravi Kok et al. Published one such system in its paper “Half Pipe Anchoring”. Half-pipe anchoring is based on backend forwarding. In this scheme, when a client request reaches a cluster of servers, a specified server accepts the request and forwards it to the optimal server after reviewing the data. After the connection status information is given, the optimum server responds directly to the client by changing the address according to the initial target address. Here, a single TCP endpoint is dynamically mapped to a node to distribute requests. This scheme is an example of an “asymmetric” method because the intermediary node disturbs the data and distributes it based on the data content.

  Another conventional system that attempts to perform an asymmetric configuration is published in two white papers written by EMIC Network. In this conventional system, the designated node not only intercepts incoming data but also captures it, and then only one node to reliably transport it to multiple nodes using a proprietary protocol. Is allowed to transfer data, and the data must then be transferred first to the server designated to retransmit it to the client. Again, the single endpoint is dynamically mapped and the TCP connection ends at the intermediary node where replication is initiated. This scheme is another example of an “asymmetric” method because the intermediary node is replicated at the same time as interfering with the data.

  In both schemes described above, endpoints can be mapped to various nodes, but the TCP definition of two endpoints is maintained. Replication in these conventional schemes is done at the application level using a proprietary protocol. In addition, these conventional schemes employ an asymmetric node configuration in which selected nodes act as application level routers that distribute requests. However, this asymmetry creates a scalability limit noted in “Scalable content-aware request distribution in clusters based on network servers” by Aaron et al. These limitations include single point errors, total data bottlenecks, sub-optimal performance with longer latency, and lack of location transparency.

  Therefore, there is a need for a method for utilizing the current definition of the two endpoints of TCP that provides a symmetric system and an m-to-n connection (where m and n can be the same or different integers).

  The requirements mentioned above as well as other requirements apply to nodes that are configured symmetrically by extending the current range of TCP for host-to-host communication to group-to-group communication, and more specifically for nodes configured symmetrically. Satisfied by extending to the end of the group. Each such endpoint is eligible to receive and forward independently and in parallel while maintaining TCP's ordered forwarding. Data is conveyed to all groups or subsets depending on the configuration. Only the required target endpoints are required to receive data before a TCP ACK is sent to the peer group.

  In one embodiment, the present invention enables addressing a cluster of nodes as a single virtual entity with one or more IP addresses. The communication between the client group and the server group is strictly compliant with the standard because any standard TCP / IP endpoint can communicate with the group without a seam. Data is transported atomically to endpoints that end in symmetrically configured nodes of the group.

  A filter installed at the connection endpoint filters out incoming data not related to the application segment. Application segment data delivery is dynamically configured and appropriately controlled by installed filters. In addition, the filter optionally performs input of incoming data directly to the target application without mediating copying.

  Input / output over one connection can be received and transferred independently and in parallel with each other, so that the connection is broken. All transfers are sequential for each TCP specification, and transfer control between nodes is ordered according to a round robin between group nodes, a round robin between transfer request factors, or an application specific scheme. Nodes can be retransmitted in parallel and no additional synchronization is required to retransmit between them.

  For scalability and load sharing, application functions are distributed among group nodes. To do this, the application is logically segmented and each subset of applications is executed. An incoming request arriving on a TCP connection is then transported to a segment in a group that efficiently distributes the load. By carrying only a certain set of requests for application instances, logical segmentation can be performed without application code changes. Application segments may be moved dynamically from node to node without interruption of connection.

  It is further noted that a node can communicate with other nodes in the group by creating a connection to the virtual entity represented by the group. This provides all the above features for communication between group nodes.

  This system is error tolerant when a node executing an application fails, because a set of remaining application replicas in the group continues to provide service without interrupting connection and service provision. Nodes can be added in a manner that is transparent to the application to maintain a constant quality of service or can be dynamically evacuated from the group. For the purpose of balancing the load between nodes or evacuating a node, the system moves the active service transparently and redistributes the tasks within the group.

  Applications running on a group of nodes can see and run the rest of the group as a single virtual entity that simplifies client / server application programming as well as resource management. According to one embodiment of the present invention, an application that is often executed independently on a group node can be divided into one or more segments in a manner that is transparent to applications that do not require code changes.

  The system distributes the processing load among a group of nodes by dynamically and easily distributing incoming tasks over one connection to various application segments. A single request arriving on a connection may be serviced by multiple segments that work together, allowing a finer distribution of computation or processing between nodes. The system enables parallel execution of 1-segment multi-instances. Requests are delivered to instances selected based on schemes such as analogy-based round robin, load minimization nodes, analogy compliance, and content hashing.

  An incoming request on one connection is transported atomically to multiple segment instances for error tolerance. The results are optionally compared and a single instance is output. If there is a segment / node error, the remaining segment instances continue to service without interrupting the connection.

  The system allows flexible and external management of the system by distributing tasks in a granular manner that controls and configures the filters at the connection endpoints. When evacuated, the load responsibility range of a node is moved to another node using round robin, load minimization or an application specific scheme. The system automatically and dynamically adds resources from pools to groups that meet changing needs. Similarly, nodes are evacuated and prepared dynamically and automatically. The system maintains the individual quality of services that automatically and dynamically add or evacuate resources.

  The features and advantages described in this specification are not necessarily all inclusive, and many additional features and advantages will be apparent to those of ordinary skill in the art, especially when looking at the drawings, specifications and claims. It will be clear. In addition, it is noted that the language used in the specification was chosen primarily for readability and intuitive purposes, and not to depict or limit the subject matter of the invention. It should be.

  The features and advantages described in this specification are not necessarily all inclusive, and many additional features and advantages will be apparent to those of ordinary skill in the art, especially when looking at the drawings, specifications and claims. It will be clear. In addition, it should be noted that the language used in the specification was selected primarily for readability and instructional purposes, and not to depict or limit the subject matter of the invention. is there.

Detailed Description of the Invention

  The present invention includes a system that has become available for reliable and ordered data communication between two node sets with, for example, atomic multipoint transport and multipoint transfer extending TCP / IP It is. The present invention extends the TCP concept of reliable host-to-host communication, including symmetric group-to-group communication that maintains a TCP specification for data traffic between groups. Furthermore, the present invention extends the definition of the two endpoints of a TCP connection that currently includes at least two endpoint groups in communication over a single connection.

  In the present invention, the connection endpoint ends at a group node. If multiple nodes must be transported with the data, the transport is performed atomically. As an option for data originating from multiple nodes, a single data instance is transferred. As will be further described below, each endpoint is composed of receiveHead and sendHead that operate independently.

Introduction In one embodiment of the present invention, a node has one connection per network, for example, a general purpose computer, or a microprocessor or software configured to function with a network-related data processor for communication. Data processing devices such as other devices are included. One group refers to a collection of one or more nodes configured symmetrically. An application segment refers to a segment of an application that may be serviced in conjunction with an application or other application segment running on various groups of nodes. One application is composed of one or more application segments, and one application segment is composed of one or more processes.

  sendHead refers to the transfer end of a TCP connection that controls data transfer and maintains the transfer state at the node. receiveHead controls the reception of data on the connection and refers to the receiving end of the TCP connection that maintains the reception state at the node. The active sendHead refers to a sendHead intended to have the latest transfer status information such as the sequence number of data and the sequence number of the latest response.

  The bus controller controls and / or references the node that coordinates the connection establishment and termination process with the peer group. One signal refers to a message exchanged within a node group across one logical bus. If the source and target of the signal are in the same node, it may result in receiving it internally, but no signal is emitted. The endpoint of one connection refers to a TCP-like stack that exchanges data with peer groups in order based on a pre-matched set of sequence numbers. An end point of a connection has at least one output data stream start point and an input data stream end point. One request refers to one selected segment of the incoming data stream, eg, a client request for a service.

System Overview Referring now to FIG. 1a, a communication system according to one embodiment of the present invention is illustrated. The communication system includes a TCP connection 130 that couples the first group 120 and the second group 160. Throughout the example, the first group 120 includes first, second and third member nodes, 100a, 100b. There are 100c and the second group 160 has first and second member nodes 150x, 150y. Both group member nodes are configured symmetrically because each node has equal access to one TCP connection and is executed independently and in parallel. The first data stream 110 and the second data stream 111 can flow between the first group 120 and the second group 160 of the communication system.

  The first application segment 135 and the second application segment 136 constitute a server application on 120. The first application segment 135 has one set of replicas 135x and 135y, and the second application segment 136 also has one set of replicas 136y and 136x. Application segment replicas 135x and 135y are executed on nodes 100a and 100b, respectively, while application segment replicas 136x and 136y are executed on nodes 100b and 100c, respectively. Client applications in group 160 are composed of application segments 151 with replicas 151a and 151b.

  Application segments 135 and 136 of the first group 120 communicate over the connection 130 with the segment 151 of the second group 160. The two data streams 110 and 111 of the connection 130 follow the TCP protocol. The connection 130 may have three different connection end points 130a, 130b, 130c in the first group 120, and two different connection end points 130x, 130y in the group 160 on the same connection.

  Each group 120, 160 is assigned to a respective group of Internet Protocol (“IP”) addresses 121, 161. Groups consider each other as a single entity while the nodes are configured. Communications between the two groups 120, 160 are addressed to each other through group IP addresses 121, 161. For example, when a request from the segment 151 arrives at the first group 120, this is seen as data coming from the group IP address 161. Similarly, the second group 160 transmits data targeted up to the group address 121.

  The endpoints 130a, 130b, 130c in the first group 120 can be set such that one or more application segment replicas 135a, 135b, 136a, 136b are carried with the incoming request. Examples of data being transported to application segments by different policies are all replicas, one replica, all application segments and selected application segments, based on request content, round robin request distribution, or individual nodes The target is determined based on the hashing scheme for mapping the map request to the priority, the priority weighting, and the like. In this scheme modification (“write”), the request is carried to all replicas of the application segment, while the “read” request is carried to only one selected replica.

  Either the endpoint 130x or 130y in the second group 160 can send a request to the server group 120. One or more sendHeads at the end points 130a, 130b, 130c of the first group 120 receive data depending on the setting. The end points 130a, 130b, and 130c of one group 120 can transmit the reply data received at the end points 130x and 130y of the second group 160. An application process that wants to receive some or all of the incoming data is guaranteed to have received it before the client responds to receipt of the data. In order to maintain a TCP sequential order for data transfer, TCP sequence numbers are assigned in sequential order before data transfer begins.

  Optionally, the duplicate data output by the replicas 151a and 151b of the second group 160 is reduced to a single instance for transfer to the first group 120 by the communication system. Similarly, the replica outputs of the application segments 135 and 136 of the first group 120 can also be reduced to the same. In many cases, only one replica is carried in accordance with the setting, so it is not always necessary that the replicas of 135a, 135b, 136a, and 136b must produce output.

  The communication system according to the present invention provides client / server requests as well as useful responses that are atomic. That is, they transmit and receive data on a single connection by allowing multiple processing across two groups and are transmitted or received as one continuous sequence of bytes.

  The protocol between groups 120 and 160 is TCP and data is guaranteed to be carried in sequential order sent as per conventional TCP. When targeted to multiple endpoints, data is guaranteed to be delivered to all target endpoints before being sent in a TCP ACK segment that directs the client to receive the data. Optionally, if the replica output must be reduced to a single copy output transfer, the output is guaranteed to be atomic because the data is transferred even if all nodes output the same data. However, if the results do not match, the application may optionally select an output to be transferred based on the majority match, correct answer or normal result.

  With respect to application segmentation, application processing is usually carried with only a selected portion of the incoming data stream for processing. For example, a request arriving on the second data stream 111 can be carried to select an application segment. The order of data transport until application processing must be guaranteed that this is the order in which it was transmitted as specified by RFC793. That is, by the time certain data is conveyed to an application segment, all preceding data that arrives in the stream must be successfully conveyed to its target endpoint.

  Referring to FIG. 1b, the first group 120 includes first, second, and third nodes 100a, 100b, and 100c. The connection 130 between the first group 120 and the second group 160 includes outgoing and incoming data streams 110, 111. Each node 100a to 100c has a group-to-group communication stack 130a to 130c, respectively. The transport of data to all of the nodes passes through switches 141a to 141c which are respectively coupled to nodes 100a to 100c. Since popular hardware technology such as Ethernet (registered trademark) is not reliable, no assumption is made regarding the transport guarantee to the switches 141a to 141c by the base hardware. The transport of data to each of the nodes 100a to 100c or any subset thereof may be selective or may not be possible at all, depending on the underlying hardware device.

  Incoming data can be either regular TCP / IP stacks 140a, 140c, or group-to-group communication stacks 130a, 130c, through switches 141a, 141c, depending on IP address and / or port. It is switched to. The application processing 142 of the node 100 communicates using the standard TCP stack 140. Application segments 135x, y, 136x, y communicate with group communication stacks 130a through 130c, respectively. The transport control signal 105 adjusts and controls the operation of the group 131. The range of signals transmitted on the control bus 105 is limited to the first group 120. Virtual bus 143 is comprised of first and second data streams 110, 111 and control signal 105 that spans group 120. This bus is connected directly to TCP connection 130 by a peer group.

  An alternative to virtual bus 143 is point-to-point communication between nodes, which has a more convenient use of bandwidth. However, this requires each node in the communication system in order not to lose sight of other nodes and their addresses and roles. In one embodiment, the logical bus model is preferred over issuing control messages due to location and ID transparency.

  Referring to FIG. 1c, a connection end point 130a according to an embodiment of the present invention is illustrated. Generally, switch 141 directs input 171 data to either a standard TCP stack or a group-to-group communication stack or “Internet Protocol” (“IP”). For fragmented IP packets, reassembly is performed before 170 is moved to 171. If the input packet is not fragmented, it can be passed directly to the input content filter 171 after some basic consistency checks. Input content filter 171 examines input content and / or packet headers to determine whether this includes data passed to an application segment (eg, 135x, 135y, or 136x).

  If the communication system decides not to pass the packet anywhere else, it discards without any action and at the same time frees any memory. Otherwise, the input content filter 171 marks the segment of the packet that is passed to the application. The packet is then passed to the IP input processing layer 172 for full authentication including checksum calculation and other consistency checks. Any invalid packet is discarded without further processing. The resulting packet is then passed to the group-to-group TCP layer 173. The group-to-group TCP layer 173 coordinates with group nodes (eg, 120, 160) and controls the receipt of data to satisfy TCP specification requirements such as responses to peer groups. The group to group TCP layer 173 maintains the input TCP state of the connection and passes data to the socket through the data path 137. Data path 138 directs the transfer data path from the socket interface to the stack.

The user socket transmits data that activates the output content filter 174. In one embodiment, the output content filter 174 is not installed, so no operation is performed. The error tolerance filter simultaneously matches data with other replica segment outputs and forwards a single output instance. The selection of the output instance to be transferred to the peer group depends on the policy set in the filter such as peer output, majority match, correct answer, or normal operation output and the like. If there is an error in the transfer of the segment instance, the replica takes over and continues the transfer without interrupting the connection. Upon receipt of a normal output instance in the peer group, the replica discards the data and releases the memory. The output content filter 174 passes the data to the group TCP output layer 175 for transfer. The group TCP output layer 175 controls data transfer and maintains a transfer state related to the group node. The group TCP output layer 175 operates with its group node that transfers data to the peer group in sequential order as defined by TCP. The group TCP output layer 175 passes through the IP output layer 176 together with the transfer data. The IP output layer 176 executes a standard IP function related to data and passes it to the device driver 177 for data transfer.

  If the output match result by the output content filter 174 indicates a result that varies depending on the node, the replica of the subset is regarded as an error, and the service is continued without interruption of the connection at the remaining endpoints. Excluded from. In some embodiments with endpoint exclusion, such exclusion is in accordance with a scheme in which the majority of endpoints are consistent with the result of excluding others. As an alternative, the exclusion of endpoints may occur if there is an error in the operation. The endpoint exclusion can also be from any application specific scheme that is programmable with the filter. If there is an end point error during data transfer, the end point of the replica may end the transfer without interrupting the connection.

  Referring to FIG. 1d, a connection end point 130 of a group-to-group communication stack is illustrated when a content processor examines input / output data in accordance with an embodiment of the present invention. Content filtration is a function of the content processor. The content processor determines where data must be placed in the application memory, the order of data and the time to inform the application that all requests are received. It operates in conjunction with the network interface device driver 177 and data is copied between the network interface 193 and the application memory 190 by the direct memory access controller 196.

  Examining new incoming request data, the content processor allocates memory 192 in the application space. The allocation size is individual for each application and is usually the size of the complete request data from the peer. Since the request data remains, no allocation is required even if a complete request memory is allocated. The output data 193 is assigned by the application itself. Further, there may be a copy of the segment of the request / response data 194, 195. With this scheme, application data is copied directly between the network interface input / output buffers and the application memory without going through the associated memory copy.

  Referring to FIG. 2, the first group 120 may be a set of servers including the nodes 100 (100a, b, c). The second group 160 may include a set of client nodes 150 (150x, y). The nodes 100 and 150 in each group 120 and 160 are connected to each other via the connection device 180. The connection device 180 may include a normal / multiplex broadcast device, for example, an Ethernet bus or a layer switch 2/3. Network 189 may be any conventional network, for example, a local aerial network (“LAN”) or Ethernet, to which two node groups are connected. If both peer groups are connected directly via connection device 180, network 189 is not required.

  In one embodiment, a communication system that includes one or more network interface ports 185a, b, c at server nodes 100a, b, c is illustrated in FIG. Communication links 187 a, b, c and 188 a, b, c connect device 180 to node 100. Input data arriving via the connection end point 130 is copied to 188a, b, c by the connection device 180 using the capability of multiplex broadcasting or normal broadcasting, its layer 2 or layer 3. Arrival data is conveyed to ports 185a, 185b, 185c. There is no guarantee of data transport through 180 or the associated hardware port or link. Since the data transferred by the first group 120 via 187a, 187b and 187c are independent of each other, they operate in parallel. Data transferred to the peer group via 187a, 187b, 187c is not necessarily visible to 120. With respect to incoming data on connection 130, the signal transmitted on logical bus 105 is replicated by device 180 to links 188a, 188b, 188c. Data transmitted to the logical bus 105 in FIG. 1b is not visible to the server nodes 100a, 100b, 100c.

Signal In an embodiment of the present invention, the signal may have group common connection ID information. In addition, the signal may also have source and target ID information. The target ID information may be one or more nodes or the entire group.

  In an embodiment of the present invention, the signal in the communication system may include response input data from a peer representing the group and an IACK signal that is an input response signal. Leave the byte data expected from the peer group in the IACK, if the responded sequence number, window update sequence number, latest and maximum time stamp and receiving active sendHead must send to the peer group TCP ACK A PUSH flag to indicate may be included. The REQSH signal includes one request, which can request the latest sendHead assignment targeting the active sendHead. The target address may be the entire group.

  The GRANTSH signal includes active sendHead state information, bus time, a node list to which the GRANTSH is responded, and a message with the most recently known IACK information. The target of this signal assumes an active sendHead after updating the state information. The IACKSEG signal includes input data response information transmitted on behalf of the segment. This may have the same or similar information as the IACK signal. The REQJOIN signal is sent to the application segment that makes a request to connect to the service beyond the connection. The LEAVE signal is sent requesting permission to leave the service of the application segment on the connection.

  The ACKLEAVE signal gives permission to the application to leave the service on the connection. The RESET signal is sent to make a request to reset the connection. The CLOSE signal is sent by the application segment to make a request to close the connection's output stream. The ACKCLOSE signal responds with receipt of the CLOSE request.

Connection establishment and termination Conventional TCP state diagrams are known. A book by Richard Stevens entitled "TCP / IP Volumes 1 and 2 illustrated" shows and describes a stream diagram illustrating this state diagram, the contents of which are incorporated herein by reference. It is. In addition, TCP / IP protocols and options are also discussed in RFC '793 and RFC 1323, the relevant portions of which are incorporated herein by reference.

  During connection establishment and termination, a node is chosen to act as a bus controller that coordinates and controls processing and communicates with peer groups on behalf of the group. By default, a static bus controller is selected, but the application program optionally selects a bus controller as needed. In order to distribute the load to the group member nodes by controlling the bus, the function of the bus controller can be assigned to the node like round robin, or alternatively, the bus controller can be an increment of the incoming sequence number or It may be selected dynamically based on the source IP address / port address combination. In a scheme where a segment with a minimum ID assumes the role of bus controller, bus controller responsibility is assigned in a round robin fashion between replicas when a replica of the segment is available.

  In general, there are four types of connection operations in TCP. Each type follows different settings for state changes. When a group initiates a connection with a peer group, the connection process is initiated by the peer group while referred to as active activation and referred to as passive initiation. Similarly, when termination of a connection is initiated by a group, it is referred to as an active termination, and when termination is initiated by a peer group, it is referred to as a passive termination.

Establishing Passive Connection For passive initiation, when a synchronization (“SYN”) request arrives from the peer group, the bus controller sends a REQJOIN signal requesting the application segment to connect to the connection service. When this is received, the bus controller then responds to the peer group with a SYN request along with an ACK (response) for SYN. When the peer group responds to a SYN request sent on behalf of the group, the group node on which the application segment is executed responds with IACKSEG. When all required group nodes have connected to the connection service with an IACKSEG signal, the connection is considered established and data transfer can begin.

Establishing an active connection In an active start, for connections initiated from a group, the bus controller sends a REQJOIN signal toward the group nodes. After that, this sends a YSN request on behalf of the group and starts the connection process with the peer group. When the group node group receives a YSN request from the peer group with an ACK for the bus controller SYN, the group node group transmits IACKSEG instructing reception of a valid ACK from the peer group. Upon receipt of an IACKSEG from the required node, the bus controller sends an ACK for the SYN request from the peer group, and at the same time, the connection is considered established.

End of Passive Connection For passive end, upon receipt of a FIN segment from a peer group, the node sends an IACKSEG signal indicating receipt of the FIN. If IACKSEG is received from all requested segments, the bus controller responds to the peer group with an ACK for the received FIN (completed). When the node completes the data transmission, the node transmits a LEAVE signal indicating a desire to leave the connection. After receiving the FIN, when the LEAVE request signal is received from all the group nodes, the bus controller transmits the FIN request to the peer group. The bus controller transmits an ACKLEAVE signal and upon receipt thereof, the signal node target enters the CLOSED state. When the transmitted FIN request ACK arrives, the bus controller enters the CLOSED state.

End of Active Connection At the end of active, the application segment sends a CLOSE signal when it wants to disconnect the connection at the same time as completing the transmission of data. When receiving a CLOSE request from all group nodes, the bus controller sends a FIN request to the peer group. Upon receiving a FIN request from the peer group, the node sends a LEAVE request. When the LEAVE signal and the transmitted FIN ACK are received from the group node, the bus controller enters the TIME_WAIT state.

Data Input on Connection Referring to FIG. 3a, when a data packet arrives at a node, it is checked (311) whether the packet is addressed to a group address. If so and if the packet is a TCP fragment, a fragment reassembly operation (314) is performed that yields a complete TCP segment when the last fragment of the TCP segment arrives. In many cases, this operation does not occur because TCP segments are not fragmented.

  If the TCP segment is not addressed to a group, then the standard TCP / IP stack is taken over with the TCP segment for further processing (312).

  Referring to FIG. 3b, when the group's receiveHead receives the data, it activates the filter (315) and at the same time verifies whether there is data destined for the application on the node (316) and data not related to the application. Discard. Data packets that occur after filtering are checked for the validity of the time stamp, the validity of the checksum, and the validity of other TCP / IP parameters (317). All invalid packets are discarded immediately. receiveHead updates its state to reflect all valid packets examined. By performing checksums and other detail packets, verification after filtering avoids the extra computational burden of discarded packets.

  It is verified whether all of the data preceding the received data is guaranteed (320) to have been delivered to the appropriate application segment and whether any data immediately following it is delivered to the application segment (318). ) Then it is. The TCP ACK segment is sent to the peer group if necessary for each specification. However, if there is preceding data for which a data response is suspended, the data is queued for a response (319).

  If a segment instance results in an error while receiving data, any remaining instances continue to accept and respond. This allows the application to continue service without interruption due to node errors.

Data Filtering Referring to Figure 3c, receiveHead keeps the state of the input data as if the request was ignored and passed to the application, and at the beginning of a new request to determine the target etc. Subsequent data is required. Since one packet may contain one or more requests or partial request data, it is verified (330) whether there is remaining data to be processed in the packet. If there is no data left, the filtering process is complete.

  If data to be filtered remains in the packet, the current state is verified (331). When the current state indicates that the request data is to be discarded, the request data of the packet is discarded up to the maximum value, and the presence or absence of any remaining data (330) is further verified (332). ). Similarly, if request data is received and must be transported to the application segment, the remainder of the packet's request data is scheduled for transport. All carried data must be checksum calculated, time stamped and packet header validated (333) only once, while invalid packets are immediately discarded (336).

  When the current state indicates the start of a new request, an individual application filter is activated (334) to determine the current state that is updated to reflect the data target and verification results. If the filtration code cannot determine the request target due to lack of sufficient data, it is combined with any next data from the reassembly queue that holds the randomly arrived data. Still, if there is not enough data, the remaining data goes into the reassembly queue and the check is repeated until enough data arrives. Instead, if sufficient data is found, step 330 is repeated to filter the data.

Data entry response When atomic data transport to multiple endpoints is required, a response for received data is sent only if all endpoints are actively receiving data. An IACK signal is transmitted on the bus indicating receipt of data in the TCP order. The active sendHead sends a TCP ACK segment to the peer group if appropriate for each TCP specification after verifying whether all required nodes have received individual data.

Data output on connection Multiple endpoints of a group can transfer data in TCP order. Therefore, consecutive sequence numbers need to be assigned to the segments of data to be transferred. In order to avoid the confusion of the request / response distinction from the end point, it is also necessary to maintain the consistency of the transferred data. For this purpose, each complete request / response data is treated as one record by the forwarding node.

  Referring to FIG. 4, if the application process writes data (385), a new transfer record is created (386). If one or more write request data must be sent without any other intermediary data sent to the peer, the MORETOCOM flag is set until the last write is reached. If the forwarding node is not already an active sendHead (387), a request for the active sendHead is sent on the REQSH signal unless a previous request is answered or suspended. When the active sendHead state arrives with the GRANTSH signal addressed to the node (389), the active sendHead is assumed after updating with the latest information from GRNTSH and it is checked whether the active sendHead (387) is repeated (388). ).

  After becoming active sendHead, the node having the data to be transmitted assigns a new transfer sequence number to the sequential order record and starts transfer (390). No more data is expected on the extension of the write operation to be transferred (391) and there are no more records waiting to be assigned with the transfer sequence number (392) or the maximum value of the transfer exceeds the limit ( 393) If there is an arbitrary (394) wait, the active sendHead is granted to the next requesting node.

  The node that grants the next sendHead is determined by selecting the node with the closest node ID in the clockwise direction from the list of REQSH requester, highest priority REQSH requester, round robin, or any application specific scheme Is done. However, if more transfer records are waiting to be assigned a sequence number, step 387 is repeated with the transmission of the remaining data.

Active sendHead Assignment Referring to FIG. 5a, a scheme for active sendHead assignment will now be described.
The node (100a) transmits a REQSH (105r) signal requesting the role of the active sendHead, and the active sendHead (100c) grants the role together with the GRANTSH (105t) signal with state information necessary for the requester. The REQSH signal is transmitted by the node 100a, and the node 100b ignores the REQSH that is not the active sendHead. The node 100c that is the active sendHead at the time of the request responds to the 100a request with a GRANTSH signal because the sendHead is free to be granted.

  Upon receipt of the GRANTSH signal, the requesting node 100a assumes the active sendHead. The GRANTSH signal includes a list of pending requesters that are only maintained by the group. When node 100b verifies GRANTSH signal 105t, it checks to see if its own request for active sendHead has been responded by verifying the pending requester list of signals. When the response is made, the re-transfer of the REQSH signal is stopped.

  When a node grants an active sendHead to the other, if it has outstanding data to be transferred, it adds itself to the list of requesters to avoid additional request signals. An alternative to sending signals such as REQSH to all nodes is to send them directly to a target such as an active sendHead node. The advantage of this method is effective bandwidth utilization, but lacks location transparency.

  Referring to FIG. 5b, if the REQSH signal arrives on the active sendHead node (551) and if the sendHead does not become granted (553), the requester ID is a list of requesters (554). Be put in. However, if sendHead is granted, the GRANTSH signal is sent to a requester (555). GRANTSH functions as a response signal for all outstanding REQSHs related to the list containing the outstanding requesters. In order to respond to receipt of REQSH without granting anything else, sendHead grants itself. When GRANTSH arrives at the target node, the requester's list is added to the requester's local list. The GRANT signal is sequenced by monotonically increasing the sequence ID number by 1 and assigning it to each instance of the sendHead grant excluding retransmissions.

Intra-group TCP timestamps and round trip time (RTT) calculations Most TCP implementations follow RFC 1323. This specifies a method for measuring the round trip time using a time stamp. The round trip time is usually measured by subtracting the time stamp of the data server responded from the real time host. In order to specify an invalid packet by the wraparound sequence number, a time stamp that monotonously increases is required as a specification.

  It is intriguing to meet the specifications for many nodes of various types of various hardware timers. The ideal solution is a fully time-synchronized node, which is difficult at best. In one embodiment, the specification requirement for increasing the time stamp monotonically is met by synchronizing the time on the node where the data is sent with the time of the node where the data was last sent. This synchronization ensures that the data is always transmitted with a time stamp value equal to or higher than the time stamp of the previous TCP data segment.

  An example implementation of the scheme is given here. The node maintains real time, ie, usually “local time”, which is executed with a hardware clock that increments its value at fixed intervals. An intra-group real-time clock, or “bus time”, must be executed for each TCF connection by the node. The “bus time” on any node is calculated as [bus time] = [local time] − [base time].

  The base time is initially selected by the bus controller and can be any numerical value calculated there. Whenever a node is granted with an active sendHead, the grantor's bus time is sent with the GRANTSH signal. The node granted with the active sendHead adjusts its bus time as announced below upon receipt of the GRANTSH signal.

  When the bus time is smaller than the granter bus time received by the active sendHead, [bus time] = [granter bus time (that is, the bus time from the GRANTSH signal)].

  The bus time for the node satisfies the requirement to increase the time stamp value monotonically even if the above formula cannot be fully synchronized because of the GRANTSH transfer delay. By selecting a higher granularity of bus time than that of the transmitted timestamp, errors due to competing timestamps during simultaneous parallel retransmissions by the nodes are reduced. As an example when the time stamp has a granularity of 10 milliseconds and the bus time has a granularity of 1 microsecond, the error factor is reduced from 1 to 1 / 10,000. For accurate round trip calculations, the base time at the time of transfer is entered into the transfer record by sendHead. A constant time value is added to the grantor bus time at the GRANTSH target node to minimize signal latency. Using the bus time as a time stamp, the round trip time of the packet is calculated as [round trip time] = [bus time] − [time stamp].

A group's TCP window update window is the amount of data that an endpoint can accept in memory. A conventional TCP with only two endpoints can be easily agreed on at both endpoints with respect to its numerical value for optimal performance. When multiple endpoints are involved, it is critical that each endpoint with a different memory size and unpredictable data target performs optimal use and performance.

  Described here is a scheme where a single virtual window size within a group area is used for effective window management within the group. sendHead is responsible for updating the peer group with window information representing the group. Group nodes are initially assigned in virtual window size. The node updates the window to the active sendHead by sending the input sequence number of the data read by the application that has just been transported. The active sendHead updates the peer group for the window obtained by subtracting the amount of outstanding data passed to the application from the virtual window size in the group area.

  Window updates are piggybacked with an IACK signal to reduce the number of window update signals. To further reduce the number of window update signals and TCP segments, the reserved window size is maintained as well as the virtual window. At any time, the data that reaches the sum of these windows can be outstanding to be answered by the group. If a node sends an IACK response receipt for data of size less than the pending window and all previous data has been read by the application, the application will have as much data as the IACK sequence. Used as if it were read. Since window updates are made by IACK, this avoids additional window update signals that would otherwise be required. This technique or option is set or reset.

  Referring to FIG. 6, the unanswered input sequence is indicated at 610 and 620 indicates the maximum data sequence number expected to be published to peer group 619. 619 indicates up to the maximum pending window sequence value at which one window update may be sent. Reference numerals 611, 612, 613, and 614 denote window sequences of data received by the nodes 100a, 100c, 100b, and 100c, respectively. 615 is the total amount data that may be transmitted by the peer group. 617 and 618 show the window updates sent by nodes 100a and 100c, together with the IACK they send for 611 and 612. The maximum announcement window is indicated by 621 and the maximum hold window is indicated by 622.

Protection against sequences wrapped in group TCP In a high speed network such as 10 Gigabit Ethernet, the current 32-bit sequence number of TCP wraps around for a short time. Due to the sequence number wrapping around, a delayed signal such as IACK may be mistakenly considered valid if the sequence number is used to respond to data input. We use a scheme where the 32-bit TCP sequence number is mapped to a 64-bit TCP value that takes into account the distorted 32-bit sequence number many times since the start of the connection. The 64-bit sequence values used within the group are mapped back to 32 bits when used with peers.

  To map the 32-bit sequence, we divided the 64-bit sequence into two 32-bit values that represent the TCP sequence in which the least significant 32 bits are actively used with the peer. The high order 32 bits count sequences that have been wrapped around a number of times since the connection was initiated. In order to map a 64-bit value to a 32-bit sequence number, the least significant 32 bits are used. In an alternative embodiment, the overhead may be the same, but the IACK is sequenced.

Multiple instances of application segment instances and replica segments further allow load distribution between nodes.
The load within a replica segment can be shared by carrying the request to a segment that uses a scheme such as round robin, load minimization, hashing, similarity base, etc. in connection with the filter.

  Segment replicas allow for error tolerance. In the unlikely event that a replica fails during input, if there are remaining replicas, the service continues without interruption. This is done by a replica that maintains a consistent view of the input. The segment controller allows for consistency between replicas with atomic input transport. A new segment controller may be selected after an error occurs.

  If the replica fails during data transfer, the remaining replicas can continue service without interrupting the connection. Each replica matches the output instance, and the sendHead state is shared until the transfer begins. Each replica frees the memory as well as the data responded by the peer group.

  Each application segment is free to choose the number of replicas to maintain. A node that is dynamically selected as a segment controller adjusts the replica IACK that forms the segment IACK. The selection of segment controllers can be based on round robin, load minimization, hashing, or even static nodes. Connection establishment, termination, window management, etc. all operate as described herein in connection with the corresponding scheme described above. In all cases where the segment replica matches the receipt of an input segment, the controller responds on behalf of the replica. If the load is balanced between the segment instances that are opposed to the copy, the controller's involvement may be eliminated.

  When the replica receives data. Send an IACK instructing receipt of input data. When the segment controller that monitors the IACK from the replica determines all replicas whose inputs are received in the transmission order, the IACK is transmitted on behalf of the segment and the client ACK is activated. This IACK acts as a response to a replica that passes data to the application socket or takes some atomically necessary action. The selection of the segment controller is static for the selected replica as well as those with round robin or minimum replica ID for each request.

Nodes Based on Group-to-Group Communication Referring to FIG. 7a, this is a partial view of a general purpose computer and a suitable element for implementing the elements of the present invention. Here, the group-to-group communication stack is executed by the processor of the system.

Group-to-Group Communication Offloading Main CPU Referring to FIG. 7b, a partition diagram of a computer is a suitable element for implementing the elements of the present invention while offloading the main processor from the processing of an element. . The group-to-group communication stack is offloaded to the adapter card in the processor itself.

Group-to-Group Communication on an Integrated Circuit Referring to FIG. 7c, this is a partition diagram of a computer and the present invention while the main processor is offloading from processing certain elements to the dedicated hardware / accelerator integrated chip of the present invention. It is an element suitable for implementation of the element. Otherwise most of the offload required by the main CPU is due to having the group-to-group communication stack run in whole or in part.

  Referring to FIG. 8, an alternative embodiment for the present invention is illustrated. In this embodiment, a hardware device replicates a single TCP connection endpoint to multiple endpoints. The node group is indicated by 802. Connection (801) has an input stream 826 and an output stream 825, respectively. The device (820) is here external to the nodes 800a, b, c. Each server node has joint end points 801a, b, c of the same connection 801. Device 820 replicates single connection 801 to three endpoints (801a, b, c) on node 800. Device 820 has four ports 816, 817, 818, 819 while port 819 is coupled to the peer service connection. This device adds an extra network hop as well as a single point of potential error.

  Referring to FIG. 9, an atomic data transport and response scheme is shown between a client group and a server group where two data segments 910 and 911 must be transported to two nodes 902 and 904. . Reference numeral 901 represents a client group, and reference numerals 902, 903, 904, 905, and 906 represent server group nodes. Reference numerals 910 and 911 denote TCP data segments transmitted by the client group 901. 910 and 911 are potentially available on all server group nodes, but are only transported to nodes 902 and 904 as determined by this instance's input filtering system that may be programmed. Absent. Reference code 912 represents a TCP ACK segment transmitted from the server group sendHead 903 to the client group. When the data segment 910 arrives, no TCP ACK segment is sent, but when the second data segment arrives, the ACK segment is sent to the client by the server group according to the TCP specification when an alternative packet must be answered. The The plex controller 902 transmits an IACK signal 914 indicating atomic transport only when receiving a 913 PIACK (plex IACK) signal indicating a response of the same data segment in the required plex / replica 904. Since 902 is a controller responsible for transmitting an IACK instructing the atomic transfer of the data, it does not transmit a PIAK. When 903 having sendHead receives the IACK signal 914, it sends a TCP ACK segment 912 to the client group. In addition to sending a client ACK segment upon arrival of an alternative TCP segment, the ACK segment may optionally be sent at the end of every complete request input. In addition, since the client ACK segment is transmitted when an exceptional condition such as segment arrival disorder or segment waiting time is detected, the client and server group are synchronized, and the TCP segment is missing. It will be re-transferred as soon as possible. Should the server node fail to receive the IACK that sent the PIACK, the PIACK and the active sendHead will respond with another IACK indicating the latest sequence of input data allowed to enter the node.

  Referring to FIG. 10, there is illustrated a logical view of an embodiment where input data is shared as in a single bus, but output data is switched. 1000 is an input data stream from the peer group. Reference numeral 1010 denotes a logical half bus in the case where only input data of an information medium that is multiplexed or shared such as Ethernet (registered trademark) is shared. 1020, 1021 and 1022 represent bus input endpoints for nodes 1030, 1031 and 1032 respectively. Reference numerals 1040, 1041, and 1042 are output end points supplied to the layer 2 or layer 3 IP that switches the device 1050. 1060 represents the collective output produced by created nodes 1030, 1031 and 1032 for input 1000. 1000 and 1060 form a single connection input and output, respectively.

Load Sharing and Load Balancing Next, referring to FIG. 11, a symmetric multicomputing system according to an embodiment of the present invention is illustrated. The server group (1112) includes a number of nodes 1100a, b, c, d, e, and f. In the input stream (1110) of the TCP connection (1109), there are a plurality of end points 1110a, b, c, d, e, and f across the group nodes. Similarly, the output stream (1111) of the same connection consists of end points 1111a, b, c, d, e, and f.

  The application consists of three segments (1113, 1114, 1115) running in two instances for each application segment 1113a, b, 1114a, b, 1115a, b across the entire group. By programming the communication system, segments are carried along with individual tasks based on criteria such as the operations they perform, the data they manage. By configuring data transport so that separate subsets of requests for services are transported to individual application instances, application segmentation is often performed without code changes to existing applications. Applications can be segmented in a number of ways, examples of which include segmentation based on connection information such as the type and type of request that a segment can handle, hashing algorithms based on data content, or sequence numbers. It is. It is also possible for the application to be divided into different segments by segment programming.

  Since the group nodes are paired as replicas 1100a, b, 1100c, d, 1100e, f, each pair causes two instances of application segments 1113, 1114, 1115 to be executed. If a segment, for example, 1100a fails, pair 1100b continues to service without interruption. If an error occurs in one instance, eg, 1100a, during transfer, the other instance 1100b sends the remaining response to the peer avoiding service interruption. Similarly, new application segment instances may be added to the group to increase the natural error tolerance of added instances that are available to continue service in the face of errors. This can be done, for example, by running an application segment instance and then creating a new process where it is added to the group so that requests are allocated to it.

  In some modes of operation, non-empty subsets of groups are carried with requests in a specific order, such as round robin, and requests are essentially distributed among the non-empty subsets to balance the load on the nodes. Priorities are weighted.

  In one mode of operation, one or more replicas are carried with one task, and after the task is completed, the results from the instance are sent over the connection without considering other replicas. In another mode of replica operation, one or more replicas may be carried with the same task. The associated replica then performs the operation in parallel and yields the result. The output filter installed during the output stream of the group-to-group communication system compares the results, and a single instance of the results is sent to the peer group, which makes the group appear as a single entity to the peer group . The selection of the output instance to be transferred to the peer group depends on the policy set for filters such as peer output, majority vote match, correct answer or normal operation output. The choice of policy is up to the application. If there is a segment instance transfer error, the replica takes over and continues without interruption.

  If the output comparison is caused by an output content filter where the various outputs produced by the node are directed, the subset of replicas are considered to be in error and the rest of the endpoints will continue to service without interruption of the connection. Excluded from services. In one embodiment with the exclusion of one endpoint, this exclusion is based on a scheme where the majority of endpoints are consistent with the result of excluding others. Otherwise, endpoint exclusion may occur if there is an error in the operation. The exclusion of endpoints may be from any application with a specific scheme programmable by the filter.

  In yet another mode of operation, replicas are carried in operations that cause state changes such as memory or storage change data. In this way, the replicas maintain a consistent state. For operations that do not affect consistency between replicas, such as read operations, the task is conveyed to only one instance of the replica. This makes it possible to balance the load between replicas.

Adding and saving nodes
A filter at the connection end point of a TCP group-to-group communication system allows fine-grained control of the transport of data to the application segment. By dynamically configuring the filter, a certain task is conveyed to a certain node, and external control of task request conveyance to the node becomes possible. As a result, the request stream to the application segment is controlled like a switch for fine task distribution among nodes.

  Groups can be added with nodes at any time. Newly added nodes may share loads from existing connections. For existing connections, the node joins the service and starts accepting tasks arriving there. When necessary, the load between nodes is balanced by task movement.

  For node evacuation, node load responsibility is transferred to another round robin or application specific scheme that is selected using a scheme such as minimum load. During the evacuation, the node is completely released while waiting for a smaller task to complete while not accepting a new task. When a task with a long processing time is involved, task movement such as system-level process movement is used. The data open file is clearly moved to another node along with the movement of the entire context of the application process such as the stack. A node communicates with another node in the group that creates a connection to the address of the virtual entity represented by the group. This provides all the features described above for communication between group nodes.

Automatic preparation Resources are added to the group automatically and dynamically from the pool to meet the changing needs of the system. Similarly, nodes are evacuated and prepared dynamically and automatically. The system monitors the quality of the service delivered to the client and maintains the specific quality of the service that adds and evacuates resources. The operations can be performed outside the system and are potentially transparent to the peer group. After reading this publication, those skilled in the art will recognize additional alternative structure and functional design schemes for group-to-group communications over a single connection in accordance with the disclosed principles of the present invention. Thus, while specific embodiments and applications of the present invention have been illustrated and described, the present invention is not limited to the precise structure and components disclosed herein and is apparent to those skilled in the art. Various modifications, changes or variations that may be made, without departing from the spirit and scope of the present invention, as set forth in the appended claims, details of the arrangement, operation and method and apparatus disclosed herein. It should be understood that this may be done in

The invention has other advantages and features that will become more readily apparent from the following detailed description of the invention and the appended claims when taken in conjunction with the accompanying drawings.
That is,
1 is a general view of a communication system configured in accordance with an embodiment of the present invention. 1 is a block diagram illustrating a communication system according to an embodiment of the present invention. FIG. FIG. 2 shows a block diagram of a configuration of higher level components for implementation of a communication system according to an embodiment of the present invention. FIG. 4 shows a one-part diagram of the execution of lower level components for optimal performance of a communication system according to an embodiment of the present invention. FIG. 2 shows a block diagram of a hardware configuration of higher level components for execution of a communication system according to an embodiment of the present invention. FIG. 4 shows a one-stream diagram for an input data processing path on one connection according to an embodiment of the present invention. The remainder of FIG. 3a shows a one-stream diagram for an input data processing path on one connection according to an embodiment of the present invention. FIG. 4 shows a stream diagram for filtering incoming data on a connection according to an embodiment of the present invention. FIG. 5 shows a stream diagram for transfer data over one connection while limiting the maximum transfer size per time for fairness within a node according to an embodiment of the present invention. FIG. 4 shows a block diagram of a request / grant scheme for active sendHead assignment according to an embodiment of the present invention. FIG. 6 shows a one-stream diagram for processing one request for sendHead according to an embodiment of the present invention. FIG. 3 shows a one-part diagram illustrating a virtual window scheme for peer group window publicity according to an embodiment of the present invention. FIG. 1 shows a block diagram of a computing system for a communication system according to an embodiment of the present invention. FIG. 1 shows a block diagram of a computing system for a communication system that is prepared for offloading of the main processor according to an embodiment of the present invention. 1 shows a block diagram of a computing system for a communication system in which a dedicated hardware / accelerator chip according to an embodiment of the present invention is prepared for offloading the main processor. FIG. Fig. 2 shows a generalized scheme which is an alternative to a communication system according to an embodiment of the present invention. Fig. 4 illustrates a data transport and response scheme between a client group and a server group according to an embodiment of the present invention. Fig. 4 illustrates a logical view of execution according to an embodiment of the invention. 1 is a generalized diagram of a symmetric multicomputing system with error tolerance, load distribution, load sharing and a single system image according to an embodiment of the present invention.

Claims (14)

A method of communication between nodes for atomic transport that includes determining a node to transfer data, transferring a single instance of data, and continuation of transfer control to send data.
a first single address group and a second single address group each including a plurality of end points including at least one node in each group, and b. a communication protocol having a single logical connection capable of communication between the groups. C. A plurality of endpoints that enable response of data to a node of the first single address group that achieves communication between the non-empty subsets of each group and that allow transfer of data to the second single address group; A device for communication between groups in an included network.
The apparatus according to claim 2, wherein data communication between the groups does not pass through an intermediary node of either group.
The apparatus of claim 2, wherein the endpoint includes a protocol stack.
3. The apparatus of claim 2, wherein the node includes one from a group consisting of one process or one device.
The apparatus of claim 2 wherein each group is addressable to another group by a single address.
The apparatus of claim 2, wherein the communication protocol includes a "transfer control protocol" (TCP).
a) Multiple endpoints with at least one node in each group, 1st single address group and 2nd single address group, b. Communication protocol with a single logical connection possible for communication between each group C. Includes multiple endpoints capable of responding to nodes in the first single address group of data and transferring data up to the second single address group to achieve communication between the non-empty subsets of each group For communication between groups in a network
9. The apparatus of claim 8, wherein data communication between each group does not pass through any group of intermediary nodes.
The apparatus of claim 8, wherein one endpoint includes one protocol stack.
9. The apparatus according to claim 8, wherein the protocol guarantees the transport of data in the order transmitted by the group.
Device according to claim 8, wherein the end point is at a node.
9. The apparatus according to claim 8, wherein the end point is outside the node.
9. Apparatus according to claim 8, wherein the membership of the two groups varies during the lifetime of the connection.

Images