WO2000022797A1 - System and method for determining global virtual time in optimistic parallel discrete event simulations - Google Patents

Abstract

A system (100) and method for determining Global Virtual Time (GVT) is provided. The system (100) comprises a plurality of nodes (101, 102) of parallel processors including at least one requesting node (102) for generating a GVT update request and a plurality of responding nodes (102) for generating respective responses to the GVT update request. At least one intermediate node (101) is coupled to the requesting node (102) and to the responding nodes (102) such that respective responses are passed through the intermediate node (101). The intermediate node (101) includes a memory (203) configured to store therein information obtained from the response packets, the information including the LVTs of response packets. According to the method of the invention at least one GVT update request is sent by a requesting node. A plurality of responding nodes generate respective response packets directed to the requesting node. Each of the respective response packets includes the LVT of a corresponding responding node. As response packets pass through one or more intermediate nodes, the LVT of each response packet is compared with an LVT stored in a memory of the intermediate node to determine which LVT is earlier in time. If the LVT of the response packet is earlier in time than an LVT stored in a memory of the intermediate node, the response packet is routed through the intermediate device unchanged. If the LVT of the response packet is not earlier in time than the LVT in a memory of the intermediate device, the response packet is either updated or dropped, depending on whether the response packet is the first to pass through the intermediate device. The requesting node compares the LVTs of all responses it receives to determine which is earliest in time.

Description

SYSTEM AND METHOD FOR DETERMINING GLOBAL VIRTUAL TIME IN OPTIMISTIC PARALLEL DISCRETE EVENT SIMULATIONS

CROSS REFERENCE TO RELATED APPLICATIONS

This application claims priority to and the benefit of the filing date of Provisional Application Serial No. 60/103,936, filed October 13, 1998 entitled "ACTIVE NETWORK MANAGEMENT", and of Provisional Application Serial No. 60/105,464 filed October 23, 1998 entitled "ACTIVE NETWORK MANAGEMENT" both of which are incorporated herein by reference in their entirety. The U.S. Government may have certain rights in this invention pursuant to contract number F30602-98-0230 awarded by the Department of the Air Force.

BACKGROUND OF THE INVENTION

This invention relates to a method and apparatus for asynchronous parallel discrete event simulation (PDES), and more particularly, it relates to a system and method for updating global virtual time in parallel discrete event simulations.

The field of parallel discrete event simulation has grown over the past fifteen years to accommodate the need for increasingly complex simulations including simulations of complex networks such as the Internet. Various PDES -based tools have been developed to simulate and predict network behavior under a variety of conditions so that optimum network designs and implementations can be realized. In the field of parallel discrete event processing, the synchronization of many processors working on the same task is an important consideration. If not done efficiently, the synchronization task can over-burden the communication network and greatly reduce the effectiveness of parallel processing or add excessive cost. One synchronization approach is Global Virtual Time (GVT). In this approach, each processing node can be allowed to free-run and can keep track of its own local time, referred to herein as local virtual time (LVT), and can synchronize with the other processors on-demand. The general method used to synchronize is to request the local time of all processing nodes and to determine the minimal of those times, which is defined as GVT. In other words, GVT indicates how far the entire network has progressed. Without GVT, an individual node would lack knowledge of how far the other nodes had progressed and would be swamped with increasing amounts of data for out of date processes.

It is important to handle synchronization techniques such as GVT efficiently in parallel systems because, if not synchronized properly, calculating GVT can add a large overhead to parallel operations and negate the benefit of the parallelism. Therefore a need exists for systems and methods for determining global virtual time while mimmizing network traffic associated with the determination.

BRIEF SUMMARY OF THE INVENTION

A system and method for determining Global Virtual Time (GVT) is provided. The system comprises a plurality of nodes of parallel processors including at least one requesting node for generating a GVT update request and a plurality responding nodes for generating responses to the GVT update request. At least one intermediate node is coupled to the requesting node and to the responding nodes such that responses are passed through at least one intermediate node. The intermediate node includes a memory configured to store therein information obtained from the responses, the information including the LVTs of the responding nodes. According to a method of the invention, at least one GVT update request is sent by at least one requesting node. A plurality of responding nodes generate respective response packets directed to the requesting node. Each of the respective response packets includes the LVT of the corresponding responding node. As response packets pass through one or more intermediate nodes, the LVT of each response packet is compared with a filter LVT stored in a memory of the intermediate node to determine which LVT is earlier in time. If the LVT of a response packet is earlier in time than the filter LVT stored in a memory of the intermediate node, the response packet is routed through the intermediate device unchanged. If the LVT of the response packet is not earlier in time than the filter LVT in a memory of the intermediate device, the response packet is either updated or dropped, depending on whether the response packet is the first to pass through the intermediate device. The requesting node compares the LVTs of all responses it receives to determine which is earliest in time.

BRIEF DESCRIPTION OF THE DRAWINGS

FIGURE 1 is a pictorial diagram showing a general configuration of a computer network.

FIGURE 2 is a block diagram of an intermediate node of the present invention.

FIGURE 3 depicts a packet structure for a GVT response packet according to one embodiment of the present invention.

FIGURE 4 is a flow chart depicting a method for updating global virtual time according to an embodiment of the invention. DETAILED DESCRIPTION OF THE INVENTION

FIGURE 1 is a pictorial diagram depicting a parallel processor system 100. Parallel processor system 100 comprises a plurality of nodes 101 and 102 coupled to each other via network 104. As used in the computer science arts, a node is a processor coupled to a network of processors. Nodes of network 100 are selected from, but not limited to, the group consisting of computers, customized processors, personal computers, main frame computers, minicomputers, lap top computers and the like. Parallel processor system 100 includes one or more intermediate nodes 101. Intermediate nodes 101 are selected from, not limited to, the group consisting of routers, switches, repeaters, hubs and the like.

In one embodiment of the invention parallel processing system 100 is configured as an event simulator, that is, the system is configured to carry out parallel discrete event simulations. In another embodiment of the invention parallel processor system 100 represents a communications network, for example, the Internet. In yet another embodiment of the invention parallel processor system 100 is configured to act simultaneously as an event simulator and as a communications network. Such a configuration is described in co-pending application docket number RD-26556, entitled "Self- Predicting Communications Network", to Stephen F. Bush, assigned to General Electric Company, Schenectady, NY, filed in the United States on October 8, 1999.

For purposes of this specification, the term "configured" means including suitable components comprising software components and hardware components, with the components being programmed, workably assembled, and integrated to perform the functions described herein." A network is defined to be two or more processors connected for the purpose of exchanging messages and sharing data and system resources. In one embodiment of the invention network 104 represents the Internet. The Internet is a wide area network (WAN) that connects a plurality (for example thousands) of networks in a plurality of countries, for example, the U.S., Canada, Europe, and Asia. Generally, the Internet provides global communications between computers on networks selected from, but not limited to, the group consisting of government, educational, and industrial networks. Alternative embodiments of the invention include those in which network 104 comprises a cluster of processors dedicated to event simulation, and those in which network 104 comprises a Local Area Network, private or dedicated network lines, and a Wide Area Network and combinations thereof.

As shown in FIG. 1 communication is accomplished by relaying messages between nodes. In one embodiment of the invention a "packet switching" architecture is employed to relay messages. The term "packet-switched" describes the type of network in which messages comprise relatively small units of data called packets, and are routed through a network based on the destination address contained within each packet. Breaking communication down into packets allows the same data path to be shared among many users in the network.

Packets typically comprise at least a field containing routing information and a field containing the data representing at least a portion of a message. Packets are routed between source and destination components in accordance with a predetermined communication protocol. Examples of common communications protocols include Transmission Control Protocol/ Internet Protocol (TCP/IP) and Open Systems Interconnection (OSI) protocol.

At least one node of network 100 is configured to execute logical processes. For purposes of this specification, logical processes are processes that represent and emulate in the present time, physical processes expected to execute at some future time. A "physical process" is an executing task defined by program code. Examples of physical processes include updating a field in a table of a database, incrementing a counter, and computing a beam table for a beam angle in a beam-formed communication system.

Logical processes typically maintain their own version of time. A logical process emulates a physical process running in the future. The length of time into the future that the logical process is currently running is known in the simulation arts as Local Virtual Time. A logical process comprises the physical process plus additional data structures and instructions for rnaintaining message order and correct operation as the system executes ahead of real time.

Because nodes of network 100 each operate according to their own notion of time (LVT), determining the latest virtual time, i.e., the furthest into the future, that each unit of the entire network has emulated, is important in managing the network. The latest virtual time that all units of the entire network have emulated is known as Global Virtual Time (GVT).

FIG. 2 illustrates an intermediate node 101 configured in accordance with the present invention. Intermediate node 101 is selected from the group of network devices and components consisting of: routers, switches, hubs and repeaters and the like. Regardless of the selected device type, intermediate node 101 comprises processor 202 and memory 203 interconnected via data bus 207. Memory 203 stores therein programs, data and messages for transmission and reception.

In one embodiment of the invention, node 101 of the present invention further includes "small state" memory 203. For purposes of this specification, the term "state" means a set of parameters characterizing the operational status of a unit at a given time, usually upon execution of a process. The tabulated parameter set is stored in memory in the form of a state table. The term "small state" refers to any subset of the parameter set comprising a state table.

The term "computation" in simulation applications, and for purposes of this specification, refers to a set of operations, called "event computations", which modify the state of a unit. A prior state is stored in memory before the state is modified by a subsequent event computation. If necessary, the prior state can be restored by copying back the saved value.

When a logical process rujtning on a node of system 100 has a need to determine the current GVT of system 100, the node sends a GVT update request packet to all processes running on the other nodes of network 100. For purposes of this specification, a node sending a GVT update request is referred to herein as a "requesting node". A node responding to a GVT update request is referred to herein as a "responding node". As defined earlier, GVT is the minimum of the LVTs of all logical processes on the network.

Each process receiving the GVT update request sends a response to the requesting logical process. Frequently, responses sent from a responding node to a requesting node do not take a direct route. Instead, responses are typically routed through one or more intermediate nodes 101 before arriving at the node of the requesting process. Thus, a given intermediate node 101 sometimes receives a plurality of responses to the same GVT update request, each response from a different responding node. In conventional systems all of these responses are routed through intermediate unit 101 and propagate through the network until they reach the node of the requesting process. Thus, conventional methods for handling these responses result in increased network traffic conditions.

According to one embodiment of the present invention, A GVT update response packet 500, as illustrated in FIG. 3, in conjunction with a filter comprising memory 203 of intermediate device 101 (illustrated in FIG.2) having stored therein a filter LVT, is employed to drop unnecessary GVT responses from system 100. Unnecessary responses are those response packets containing LVT values which cannot be a minimum for network 100. Response packet 500 is configured to pass itself through intermediate device 101 only under certain conditions. Accordingly, response packet 500 comprises header 501 which represents destination address information; request ID portion 502 which represents the specific GVT request to which a node is responding, and LVT portion 505 which represents the LVT of the responding process. In one embodiment of the invention response packet 500 further includes responder ID portion 503 identifying the responding process. Response packet 500 further comprises a code portion 510. According to one embodiment of the invention, code portion 510 implements the GVT update algorithm depicted in FIG. 4. In one embodiment of the invention code portion 510 is implemented in the JAVA programming language. The code is executed by processor 202 on each intermediate node 101 through which GVT response packet 500 flows. The GVT update algorithm depicted in FIG. 4 is designed to self propagate response packet 500 through an intermediate node 101 only when packet 500 contains an LVT value which could be a minimum for the network. All other response packets are dropped.

The algorithm shown in FIG. 4 begins at step 301 when a GVT update response packet 500 is routed through an intermediate node 101 on a path to reach the node of the requesting process. According to steps 303 and 305, memory 203 of node 101 is examined to determine if this response packet is the first response packet 500 received by intermediate node 101 in response to a particular GVT update request. In one embodiment of the invention the portion of memory examined in step 301 is small state memory 212. However, as those of ordinary skill in the art will appreciate, alternative memory configurations including a variety of cache memory devices may be employed to implement the technique of the present invention.

To determine if a response packet is the first response packet to pass through intermediate node 101 in response to a GVT update request, memory 203 is examined for "traces" of other packets responding to the same request. A "trace" is a record in memory of the response packets routed through intermediate node 101 . The record includes such information as identification of the destination of the packet and the particular GVT update request to which the packet is responding. In one embodiment of the invention, the record also includes the LVT value carried by the response packet. In another embodiment of the invention, response packets 500 are indexed and stored according to a key as they pass through intermediate device 101. In one embodiment of the invention a key comprises destination and request ID information. Subsequent messages are compared with stored messages by searching for matching keys (destination - request time ID).

If this response packet is the first response packet to arrive at intermediate node 101, the LVT portion 505 of the packet is compared with a filter LVT stored in memory 203 of intermediate node 101 to determine if the LVT of the response packet is earlier in time than the filter LVT, according to steps 320 and 322. A filter LVT is an LVT stored in memory 203 of intermediate node 101 to which LVTs of response packets are compared. For the first response packet, the filter value is derived from the LVT of a process ruru ng on intermediate node 101. The filter LVT is updated as subsequent response packets arrive at intermediate node 101.

If the LVT of the first response packet is earlier in time than the LVT of the process running on the intermediate node 101, traces of the response packet are stored in memory 203 and the response packet proceeds to its destination carrying its earlier-in-time LVT. The LVT value thereby stored in memory 203 becomes the filter value for intermediate node 101 to which subsequent response packet LVTs are compared. If the LVT of node 101 is earlier in time than that of the response packet, the response packet is changed by replacing its LVT with the filter LVT.

If examination of memory 203 reveals a trace of a prior response to a particular GVT update request, i.e., this response is not the first response passing through intermediate node 101, then the LVT of the response is compared to the filter LVT (the LVT derived from the trace of the prior response) according to step 307. If the LVT of this response is earlier in time than the LVT of the prior response, then at least the LVT portion of the trace of the prior response is overwritten with the LVT of the present response in accordance with step 311. Then, the present response proceeds to its destination according to step 315. However, according to step 313, if the LVT of the trace is earlier in time than the LVT of the present response then the present response is destroyed, since it cannot represent GVT (a global minimum). Thus message traffic on network 100 is reduced by filtering out response packets containing LVTs not likely to be minimums for network 100. Thus, the node initiating the GVT update request will receive only those responses most likely to be minimums for network 100. To determine GVT, the lowest LVT of the responses is determined and selected. While only certain preferred features of the invention have been illustrated and described, many modifications and changes will occur to those skilled in the art. It is, therefore, to be understood that the appended claims are intended to cover all such modifications and changes as fall within the true spirit of the invention.

Claims

WHAT IS CLAIMED IS:

1. In a network 100 comprising a plurality of nodes of parallel processors, a Global Virtual Time (GVT) filter system comprising:

at least one requesting node 102 for generating a GVT update request;

a plurality of responding nodes 102 for generating respective responses to said GVT update request;

at least one intermediate node 101 coupled to said requesting node 102 and to said responding nodes 102 such that said responses are passed through said intermediate node 101;

at least one intermediate node 101 including a memory 203 configured to store therein a filter Local Virtual Time (LVT) value.

2. The system of claim 1 wherein said filter LVT is updated based on traces obtained from corresponding responses.

3. The system of claim 1 wherein said each of said traces comprises the identity of said requesting node 102, the LVT of one of said responding nodes

102 and the GVT update request to which said one of said responding nodes is responding.

4. The system of claim 1 wherein said memory is small state memory 212.

5. The system of claim 1 wherein said responses to said GVT request comprise response packets 500 include respective code portions 510, said code portions 510 executing on a processor 202 of said intermediate node 101.

6. The system 100 of claim 5 wherein each of said respective code portions 510 implements a method for comparing the LVT of said corresponding active response packet with a filter LVT stored in a memory 203 of said intermediate node 101.

7. A method for determining the lowest Local Virtual Time (LVT) of nodes on a network comprising the steps of:

sending, from a requesting node, a global virtual time update request to a plurality of responding nodes;

generating from a plurality of responding nodes respective response packets directed to said requesting node, said response packets including the LVTs of corresponding responding nodes;

in at least one intermediate node, comparing the LVTs of at least one of said response packets arriving at said node with a filter LVT stored in a memory of said node to determine which is later in time;

if said response packet LVT is later in time, destroying said response packet;

if said response packet LVT is earlier in time, updating said filter LVT with said earlier in time LVT and passing said response packet through said intermediate node.

8. The method of claim 7 further comprising the step of determining GVT at said requesting node based upon the earliest in time LVT of said response packets received at said requesting node.

9. The method of claim 7 wherein said response packets include code portions.

10. The method of claim 9 wherein said active packets include respective code portions, said code portions comprising a program for carrying out the steps of the method of claim 9 on a processor of said intermediate node.

11. A method for determining the lowest local virtual time (LVT) of nodes on a network comprising the steps of:

sending, from a requesting node, a global virtual time (GVT) update request to a plurality of responding nodes;

generating from a plurality of responding nodes respective response packets directed to said requesting node, said response packets including the LVTs of corresponding responding nodes;

storing in a memory of said at least one intermediate node a filter LVT value;

receiving said response packets at least one intermediate node and examining a memory of said intermediate node for traces of prior response packets corresponding to said GVT update request;

if no trace is found in the previous step, comparing the LVT of said response packet to a filter LVT of said intermediate node to determine which is earlier in time; if said response packet LVT is earlier in time, updating said filter LVT with said earlier in time LVT and passing said response packet through said intermediate node, and if said filter LVT value is earlier in time replacing said response packet LVT with said filter LVT and passing said response packet through said intermediate node;

if a trace is found in the receiving step, comparing the LVT of said response packet to said filter LVT to determine which is earlier in time; if said LVT of said response packet is earlier in time, updating said filter LVT with said