Classifications

• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F9/00—Arrangements for program control, e.g. control units
  • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
  • G06F9/46—Multiprogramming arrangements
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F9/00—Arrangements for program control, e.g. control units
  • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
  • G06F9/46—Multiprogramming arrangements
  • G06F9/52—Program synchronisation; Mutual exclusion, e.g. by means of semaphores
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F15/00—Digital computers in general; Data processing equipment in general
  • G06F15/16—Combinations of two or more digital computers each having at least an arithmetic unit, a program unit and a register, e.g. for a simultaneous processing of several programs
  • G06F15/163—Interprocessor communication
  • G06F15/173—Interprocessor communication using an interconnection network, e.g. matrix, shuffle, pyramid, star, snowflake

Definitions

• This invention relates to a method and apparatus for managing contention among users for access to serialized resources in an information handling system. More particularly, it relates to a method and apparatus for managing such contention in a system cluster containing a plurality of such systems.
• Resource contention is a well-known phenomenon in information handling systems. It occurs when a user (e.g., a process or other unit of work) attempts to access a resource that is already held by another user, and the access requested by the second user is inconsistent with that of the first user. This will occur, for example, if either user is requesting exclusive access to the resource in question.
• Resource managers are software components that manage contention among competing requesters for a resource that they control by granting one or more such users access to the resource as a holder and placing any remaining users in a pool of waiters until the resource becomes available.
• Contention chains can form, or put another way, contention can cross resources. For example, job A waits on resource Rl but holds R2, while job B holds Rl but is waiting for R3, which in turn is held by job C. Contention can cross systems. In the example above, each job could be on a separate system. Contention can cross resource managers. For example, Rl could be a GRS enqueue and R2 could be a DB2TM latch .
• GRS global resource serialization
• IRLM STM Resource Lock Manager
• Cross-resource contention is typically solved within a single resource manager (e.g. GRS) by tracking the topology of each resource's holders and waiters and finding any points of intersection.
• Cross-system contention is typically solved by making the resource manager aware of the entire cluster's data (managing the cluster as one unit rather than as independent systems).
• Cross-resource manager contention is typically "solved" by having a reporting product query all of the interfaces and correlate the data as if it were a virtual resource manager. Because the problem is of order O(2 n ) in the number of resources in contention, it is also computationally complex.
• the base MVSTM component of z/OS has a simple efficiency solution (known popularly as "enqueue promotion"): automatically (and temporarily) boost the CPU and MPL access of any work holding a resource reportedly in contention, with no attention paid to the neediness of the work. This is equivalent to managing a holder as if there were "important" waiter(s) for a resource, regardless of the actual topology. To appreciate the operation of this, consider the following example. Suppose that:
• Job A holds resource Rl .
• Job B holds resource R2 and waits for Rl .
• this can be represented as a chain C ⁇ B ⁇ A, where the capital letters represent the jobs, and the symbol "->" (the “link” in the chain) indicates that the job on the left of the symbol is waiting for a resource held by the job on the right of the symbol.
• the above chain means that job C is waiting for a resource held by job B, which in turn is waiting for a resource held by job A.
• the conventional MVS implementation would help jobs A and B because they hold resources under contention, promoting each equally and for a limited time. Helping B would do no good, however, since B is in fact waiting for A. If B is itself multitasking, then the help may actually harm competing work without doing anything about the resource contention.
• One aspect of the invention which is the subject of the concurrently filed application identified above, comprises a method and apparatus for managing contention among users for access to resources in an information handling system in which each user has an assigned need and may be either a holder or a waiter for a resource it is seeking to access.
• a user is identified that is not a waiter at a head a chain of users in which each user having a next user in the chain is holding a resource for which the next user is waiting. That user at the head of the chain is managed as if its need were at least that of the neediest waiter in the chain, preferably by allocating system resources to the user as if its need were at least that of such neediest waiter.
• such a contention chain is identified by identifying a cluster of resources in which each resource in the cluster is either held by a user that is waiting for another resource in the cluster or being waited for by a user that is holding another resource in the cluster and determining the need of a neediest waiter for any resource in the cluster.
• a user is identified that is a holder of a resource in the cluster but is not waiting for any other resource, and that holder of the resource is managed as if its need were at least that of the neediest waiter for any resource in the cluster, again preferably by allocating system resources to the user as if its need were at least that of such neediest waiter.
• the step of identifying a cluster is preferably performed in response to receiving a notification of a change in the contention status of a resource.
• a resource is newly assigned to a cluster if it is now being held by a user that is waiting for another resource in the cluster or being waited for by a user that is holding another resource in the cluster.
• a resource is removed from a cluster if it is no longer being held by a user that is waiting for another resource in the cluster or being waited for by a user that is holding another resource in the cluster.
• This aspect of the invention thus contemplates integration of the "neediness" factor into the base system resource allocation mechanism so that a job at the head of a chain (e.g., job A above, with a neediness factor of 4) can be run as if it had the neediness factor of a needier job elsewhere on the chain (e.g. job C above, with a need of 1) until it releases the resource.
• a job at the head of a chain e.g., job A above, with a neediness factor of 4
• job C e.g. job C above, with a need of 1
• Job A with a "need" of 4, holds resource Rl . (In this specification, lower numbers signify a greater need, so they can be thought of as "priority for helping”.)
• Job B with a need of 5, holds resource R2 and waits for Rl .
• Job C with a need of 1 , waits for R2.
• this can be represented as a chain C(l) -> B(5) - A(4), where the capital letters represent the jobs, the numbers in parentheses represent the "need” of those jobs, and the symbol “ ⁇ " (the “link” in the chain) indicates that the job on the left of the symbol is waiting for a resource held by the job on the right of the symbol.
• the above chain means that job C, with a need of 1 , is waiting for a resource held by job B, with a need of 5, which in turn is waiting for a resource held by job A, with a need of 4.
• This first aspect of the invention may be practiced either on a single system or in a system cluster containing a plurality of such systems.
• the variant of this invention that identifies resource clusters is especially suited for use in a multisystem implementation, as it requires an exchange of only a subset of the local contention data, as described below.
• Another aspect of the invention which is the subject of the present application, contemplates a protocol for managing resource allocation across multiple systems while passing very little data around, of order O(n) in the number of multisystem resources in contention.
• This other aspect of the invention which incorporates aspects of the single-system invention described above, contemplates a method and apparatus for managing contention among users for access to resource in a system cluster containing a plurality of systems, each user having an assigned need and being capable of being either a holder or a waiter for a resource it is seeking to access.
• each such system operating as a local system, stores local cluster data indicating a grouping of the resources into local clusters on the basis of contention on the local system and indicating for each local cluster a need for one or more resources in the local cluster.
• Each system also receives remote cluster data from other systems in the system cluster, operating as remote systems, indicating for each such remote system a grouping of the resources into remote clusters on the basis of contention on the remote system and indicating for each remote cluster a need for one or more resources in the remote cluster.
• Each local system combines the local cluster data and the remote cluster data to generate composite cluster data indicating a grouping of the resources into composite clusters on the basis of contention across the systems and indicating for each composite cluster a need for one or more resources in that composite cluster.
• Each local system then uses this composite cluster data to manage holders on the local system of resources in the composite clusters.
• the local, remote and composite cluster data indicates the need of the neediest waiter for any resource in the cluster in question, and holders on the local system of resources in the composite clusters are managed by identifying such holders that are not waiting for any other resource and allocating system resources to such holders as if their need were at least that of a neediest waiter for any resource in the corresponding composite cluster.
• each local system assigns a pair of resources to a common local cluster if a user on the local system is holding one of the resources while waiting for the other of the resources, and updates the local cluster data in response to receiving a notification of a change in the contention status of a resource with regard to a user on the local system.
• Each local system also transmits its local cluster data, including any updates, to the remote systems, which, treating the transmitted cluster data as remote cluster data relative to the receiving systems, then update their composite cluster data accordingly.
• the transmitted local cluster data indicates a resource, a cluster to which the resource is assigned on the basis of contention on the local system, and a need on the local system for the resource.
• the protocol passes around only one set of information per resource , instead of the full list of holders and waiters from each system, so that no system has a complete view of contention across the cluster.
• the data itself consists only of: a cluster-unique resource name, the neediness value of the neediest waiter on the sending system, and a sending-system-unique token. If the latter token matches for two resources, then their management must be integrated (the tokens are assigned based on the sending system's local data only).
• the protocol also sends only data about resources in contention, even if some of the pieces of work in the topology hold other resources not in contention.
• the sending system cluster information can be encoded in various ways.
• the local system can, as in a preferred embodiment, send a cluster name based upon remote contention as well, together with an indication of whether a non-trivial cluster assignment (i.e., an assignment to a cluster containing more than one resource) is based upon local or remote information.
• a non-trivial cluster assignment i.e., an assignment to a cluster containing more than one resource
• the invention is preferably implemented as part of a computer operating system or as
• Such a software implementation contains logic in the form of a program of instructions that are executable by the hardware machine to perform the method steps of the invention.
• the program of instructions may be embodied on a program storage device comprising one or more volumes using semiconductor, magnetic, optical or other storage technology.
• Fig. 1 shows a computer system cluster incorporating the present invention.
• Figs. 2A-2C show various types of contention chains.
• Fig. 3 shows procedure for allocating resources to a user at the head of a chain of contention.
• Fig. 4 shows a typical contention scenario among transactions and resources on several systems.
• Fig. 5 shows the general procedure followed in response to a notification from a local resource manager.
• Fig. 6 shows the general procedure followed in response to receiving a broadcast of contention data from a remote system.
• Figs. 7A-7G show the multisystem contention state in various examples of operation.
• Figs. 8A-8H show the various data structures for storing contention data in one embodiment of the present invention.
• Fig. 9 shows how the contention scenario shown in Fig. 4 is captured by one of the systems of the cluster. DESCRIPTION OF THE PREFERRED EMBODIMENTS
• Fig. 1 shows a computer system cluster 100 incorporating the present invention.
• Cluster 100 contains individual systems 102 (Syl, Sy2, Sy3) coupled together by an interconnection 104 of any suitable type. Although an exemplary three systems are shown, the invention is not limited to any particular number of systems.
• Cluster 100 has one or more global or multisystem resources 106 that are contended for by requesters from the various systems.
• Each system 102 of the cluster may comprise either a separate physical machine or a separate logical partition of one or more physical machines.
• Each system contains an operating system (OS) 108 that performs the usual functions of providing system services and managing the use of system resources in addition to performing the functions of the present invention.
• OS operating system
• each system 102 comprises an instance of the IBM z/OS operating system running on an IBM zSeriesTM server or a logical partition of such server.
• Each system 102 contains one or more requesters 110 that contend among each other for access to multisystem resources 106 and, optionally, local resources 112 that are available only to requesters on the same system.
• a requester 110 may comprise any entity that contends for access to resources 106 or 112 and is treated as a single entity for the purpose of allocating system resources.
• the system resources that are allocated to requesters 110 should be distinguished from the resources 106 and 112 that are the subjects of contention among the requesters.
• System resources are allocated to requesters 110 in a manner that is usually transparent to the requesters themselves to improve some performance measure such as throughput or response time.
• the resources 106 and 110 are explicitly requested by the requesters as part of their execution. Where it is necessary to distinguish them, the latter class of resources will sometimes be referred to using a term such as "serialized resources" or the like.
• Each operating system 108 contains several components of interest to the present invention, including one or more resource managers 114 and a workload manager (WLM) 116.
• Each resource manager 114 manages contention among competing requesters 110 for a resource 106 or 112 that it controls by granting access by one or more such requesters to the resource as a holder and placing any remaining requesters in a pool of waiters until the resource becomes available.
• one such resource manager (used for multisystem resources 106) may be the Global Resource Serialization (GRS) component of the z/OS operating system, described in such references as the IBM publication z/OS MVS Planning: Global Resource Serialization, SA22-7600-02 (March 2002), incorporated herein by reference.
• GRS Global Resource Serialization
• SA22-7600-02 March 2002
• WLM Workload manager
• WLM Workload manager
• units of work which may be address spaces, enclaves, etc.
• a "need" value that is assigned to that unit of work (or the service class to which it belongs) and reflects in some sense the relative priority of that unit of work relative to other units of work being processed.
• the invention is not limited to any particular workload manager, one such workload manager is the workload management component of the IBM z/OS operating system, described in such references as the IBM publications z/OS MVS Planning: Workload Management, SA22-7602-04 (Oct. 2002), and z/OS MVS Programming: Workload Management Services, SA22-7619-03 (Oct. 2002), both of which are incorporated herein by reference.
• Such a workload management component works in conjunction with a system resources manager (SRM) component of the IBM z/OS operating system, as described in such references as the IBM publication z/OS MVS Initialization and Tuning Guide, SA22-7591-01 (March.2002), especially chapter 3 (pages 3-1 to 3-84), incorporated herein by reference. Since the particular manner in which these components interact is not a part of the present invention, both components are assumed to be referenced by the box 116 labeled "WLM" in Fig. 1.
• SRM system resources manager
• the need value should be one that has a similar meaning across the system cluster. In the embodiment shown it is a calculated dynamic value, based on the active
• WLM policy that integrates resource group limits and importance into a single quantity that can be safely compared across systems. While the ordering is arbitrary, in this description lowers numbers represent higher need or priority, so that a user with a need of 1 is "needier" than a user with a need of 5.
• Figs. 2A-2C show various contention chains that may develop among the resources 106 and 112 in the system cluster 100. These chains are known more formally as directed graphs, but the chain terminology will be used herein. Each link in these chains, depicted by an arrow, represents a relationship where a user (represented by a node at the tail of the arrow) is waiting for a resource held by another user (represented by a node at the head of the arrow).
• the "transitive closure" of such a relationship is the chain formed by including all such relationships involving any node of the chain so that if one follows the arrows, all nodes eventually point to a holder that is not waiting for any resources in contention and thus stands at the head of the chain. (Whether a chain can have more than one head is discussed below in the description of Fig. 2D.)
• Fig. 2A shows the contention scenario described in the background and summary portions above, in which a user C is waiting for a resource R2 held by a user B who is in turn waiting for a resource Rl held by a user A.
• user A who is a holder but not a waiter and therefore at the head of the chain, is allocated system resources as if its need were at least that of the neediest of the waiters B and C, since both of its waiters will benefit from having A finish with the resource Rl.
• User B is also a holder, but is not given this preferential allocation since it is waiting for a resource and therefore not running; thus, there would be no point at this time in allocating more resources to B (although there may be later when B acquires resource Rl as a holder).
• the contention scenario shown in Fig.2A is a straight chain, in which each user is holding and/or waiting for a resource heldby a single user. In general, however, contention chains can be branched, so that a single user may be holding a resource waited for by multiple users or waiting for resources held by multiple users. Some resources can also be requested for shared access, allowing multiple concurrent holders.
• Fig. 2B shows a contention scenario with branching of the first type, which differs from the scenario shown in Fig. 2A in that now an additional user D is waiting for a resource R3 held by user B.
• user A is allocated system resources as if its need were at least that of the neediest of the waiters B, C and D, since all of these waiters will benefit from having A finish with the resource Rl .
• Fig. 2C shows a contention scenario with branching of both types, which differs from the scenario shown in Fig. 2A in that now user C is waiting for an additional resource R3 controlled by a user D, who is waiting for a resource R4 controlled by user A.
• user A is allocated system resources as if its need were at least that of the neediest of the waiters B, C and D, since all of these waiters will benefit from having A finish with the resource Rl .
• FIG. 2D shows a contention scenario with branching of the second type, which differs from the chain shown in Fig. 2A in that now user C is also waiting for a resource R3 held by user D, who in turn is waiting for a resource R4 held by user E.
• this could be analyzed as two partially overlapping chains each having one head, one chain being C — > B — ⁇ > A and the other chain being C ⁇ D ⁇ E.
• user A is allocated system resources as if its need were at least that of the neediest of the waiters B and C
• user E is allocated system resources as if its need were at least that of the neediest of the waiters C and D.
• a user that is not a waiter at a head a chain of users in which each user having a next user in the chain is holding a resource for which the next user is waiting (step 302).
• these chains are analyzed as two separate resource clusters: a first containing resources R1-R2 and a second containing resources R3-R4.
• first cluster user A is allocated system resources as if its need were at least that of the neediest of the waiters (B and C) for any of the resources (Rl and R2) in that first cluster.
• second cluster user E is allocated system resources as if its need were at least that of the neediest of the waiters (C and D) for any of the resources (R3 and R4) in that second cluster.
• the contention chains are acyclic, meaning that one cannot form a closed path by following the links along the directions of their arrows. If there were such a closed path, there would be a resource deadlock, which could only be broken by terminating one or more of the users involved in the deadlock.
• Fig. 4 shows a typical contention scenario among transactions and resources on several systems.
• a transaction TxA (with a need of 1) on system Syl is waiting for a resource Ra held by transactions TxB (with a need of 2) and TxD (with a need of 4) on system Sy2.
• Transaction TxB on system Sy2 is in turn waiting for a resource Rb held by transaction TxC (with a need of 3) on system Sy3, as is transaction TxE (with a need of 5) on system Sy3.
• system Sy2 does not store or maintain a complete global picture of contention in the cluster, but rather a subset of such contention information as indicated in the following table.
• system Sy2 stores a complete set of contention data ("local system info") for its local transactions TxB and TxD that are contending for resources either as holders or as waiters. For each such resource for which a local transaction is in contention, Sy2 tracks the local holders and waiters, including their intrinsic "need" values.
• System Sy2 has also assigned resources Ra and Rb to a common cluster Cab, since at least one local transaction (TxB) is both a holder of one requested resource (Ra) and a waiter for another requested resource (TxB).
• the data shown in the above table or otherwise tracked by a local instance of WLM includes local cluster data, remote cluster data, and composite cluster data.
• Local cluster data indicates the grouping of the resources into local clusters on the basis of contention on the local system and, for each such local cluster, the need of the neediest waiter for any resource in the local cluster.
• remote cluster data indicates, for a particular remote system, the grouping of the resources into remote clusters on the basis of contention on the remote system and, for each such remote cluster, the need of the neediest waiter for any resource in the remote cluster.
• composite cluster data generated by combining the corresponding local and remote data, indicates the grouping of the resources into composite clusters on the basis of contention across the systems and, for each such composite cluster, the need of the neediest waiter for any resource in the composite cluster.
• the items under the caption "Remote waiter info” represent remote cluster data, since they are based only on contention on particular remote systems.
• the need of the neediest waiter is indicated in the adjacent "NQO" column.
• the grouping of resources into clusters on the basis of contention data from a particular remote system is not indicated in the above table, but is tracked by the local WLM instance so that it can be combined with the local cluster assignment information to obtain a composite cluster assignment. Combining of clusters is done in a straightforward manner.
• a first system assigns resources A and B to a common cluster (on the basis of its local contention data)
• a second system similarly assigns resources B and C to a common cluster
• a third system assigns resources C and D to a common cluster
• the resulting composite cluster contains resources A, B, C and D.
• the first column (“Resource Cluster”) represents composite cluster data, since its assignment of a resource to a cluster is based both on local cluster data and remote cluster data.
• the final column (“NQO”) likewise represents composite cluster data, since the need listed is that of the neediest waiter for the resource across all systems (as reported to the local system).
• System Sy2 could store the contention data in the tabular form shown above, but more typically would distribute such across data a number of data structures to optimize the ease of manipulation, as described further below.
• Fig. 5 shows the general procedure 500 followed by a local instance of WLM in response to a contention notification from a local resource manager. Although a particular sequence of steps is described, the sequence maybe varied as long as the necessary input data is available when each step is performed.
• the procedure 500 begins when the WLM instance receives a notification from a local resource manager of a change in the contention state of a resource as it relates to local users.
• a change may signify any of the following: 1.
• a local user has become a waiter for a resource held by another user.
• a local user is no longer a waiter for a resource. This may be either because it has acquired the resource as a holder or because it is no longer interested in the resource as either a holder or a waiter (possibly because it has terminated and therefore no longer exists, as described in an example below).
• a resource held by a local user is no longer in contention.
• the notification from the local resource manager would identify the resource as well as the local holders and waiters.
• WLM obtains the respective "needs" of these holders and waiters (their intrinsic needs, not their needs as altered in accordance with the present invention) from the SRM component not separately shown; the particular source of this data, though, forms no part of the present invention.
• the local instance of WLM first updates the local contention data for the resource in question (step 504).
• Such updating can include creating a new entry for a resource newly in contention on the local system, modifying an existing entry for a resource already in contention on the local system, or deleting an existing entry for a resource no longer in contention on the local system.
• This local contention data includes an identification of any local user holding or waiting for the resource, together with the "need" of such user.
• the local instance of WLM updates the resource's cluster assignment if necessary (step 506).
• a resource is assigned to a trivial cluster that contains only itself as a member.
• a resource is assigned to a non-trivial cluster containing at least one other resource if such assignment is dictated either by local contention data or by remote contention data.
• a resource is assigned to a cluster containing another resource on the basis of local contention data if that data indicates that the same local user is holding one of the resources while waiting for the other — that is, that the resource is either held by a user that is waiting for the other resource or being waited for by a user that is holding the other resource.
• a resource is assigned to a cluster containing another resource on the basis of remote contention data if that data indicates that at least one remote system has assigned the two resources to a common cluster on the basis of contention data that is local to that remote system.
• This cluster assignment step may thus involve: (1) leaving the cluster assignment for the resource unchanged; (2) newly assigning the resource to a non-trivial cluster if the changed local contention data and any existing remote contention data dictate such assignment; or (3) breaking up an existing cluster if the changed local contention data and any existing remote contention data no longer dictate such assignment. If the resource's cluster assignment is changed, the cluster information for the other resources affected by the change is similarly modified at this time.
• the local instance of WLM updates an imputed "need" value for the resource that is based only upon local contention data for the resource (step 508).
• This imputed need is the greatest of the needs of any local waiter for the resource, as indicated by the local contention data for the resource.
• the local instance of WLM updates its composite cluster data, which includes: (1) an imputed need value for the resource, based upon both local and remote contention data (the "NQO" column in the above table); (2) a grouping of the resources into a composite cluster, based upon local and remote contention data; and (3) an imputed "need" value for the resource cluster as a whole (step 510).
• the last named is simply the greatest of the needs of any of the resources making up the composite cluster, where here as well the need is based upon remote as well as local contention data for the resources making up the cluster.
• the local instance of WLM then broadcasts a summary of its updated local contention data to the other systems in the cluster (step 512).
• This data summary includes:
• the local system name 2. The resource name. If the resource is a multisystem resource, the resource name is the actual name of the resource as recognized across the cluster. If the resource is a local resource, the resource name is a generic local resource name serving as a "proxy" for the actual local resource name, as described in Example 2 below.
• the cluster name is given as a concatenation of the multisystem resources in the cluster, purely as a mnemonic device to facilitate reader comprehension. However, in the preferred embodiment, the "cluster name" is actually an opaque "cluster ID" that receiving systems can only test for equality with other cluster IDs originating on the same sending system.
• the "need" for the resource based solely on the sending system's "local system information” — i.e., the neediest local waiter for the resource. This may be considered as a vote of what this system thinks the need should be if only its data were being considered. If there is no local waiter for the resource, then a dummy value indicating that there is no local need is transmitted, as described in Example 1 below.
• An indication of whether any transaction on the sending system forces the resource to be included in the cluster i.e., whether the resource is being assigned to a non-trivial cluster based upon local contention data. This is a boolean value, but rather than yes/no it will be given values of local/remote in this description.
• Local means: (1) that the sending system has at least one transaction that is both a waiter for one resource and a holder of another resource; and (2) that the same transaction is either a waiter or a holder for this resource (thus the sending system requires the group of resources connected with the transaction(s) to be managed as a group).
• Remote means that nothing in the sending system's local data requires that the resource be part of a non-trivial cluster.
• Trivial clusters have exactly one resource, and always have a value of 'remote' to make the clustering code a bit easier. If there has been a cluster reassignment, WLM also broadcasts similar information for each other resource affected by the reassignment.
• the local WLM instance makes any necessary adjustments to the "need" values of local users (step 514). More particularly, WLM adjusts the "need" of any local holder of a resource that is not also a waiter for another resource (and thus is at the head of a contention chain) so that it at least matches the intrinsic need of the neediest waiter in the cluster containing the resource.
• the adjusted value is the imputed "need" value that is actually used to allocate system resources to the holder, not the intrinsic need value that is assigned to that user (and used to impute values to other users). Thus, if the reason for imputing a particular need value goes away, the need value imputed to a user reverts either to the intrinsic need value or to a lesser imputed need value.
• Fig. 6 shows the general procedure 600 followed by a local instance of WLM in response to receiving a broadcast of remote contention data from an WLM instance on a remote system (step 602).
• This broadcast includes, for each affected resource, the information listed in the description of step 512.
• the local instance of WLM first updates the remote contention data for the resource in question (step 604).
• updating can include creating a new entry for a resource newly in contention on the local system, modifying an existing entry for a resource already in contention on the local system, or deleting an existing entry for a resource no longer in contention on the local system.
• This remote contention data includes an identification of any remote system having a waiter for the resource, together with the need of the neediest such waiter on the remote system for the resource.
• the local instance of WLM updates its composite cluster data for the resource, as it did in step 510.
• the composite cluster updated includes: (1) an imputed need value for the resource, based upon both local and remote contention data; (2) a grouping of the resources into a composite cluster, based upon local and remote contention data; and (3) an imputed "need" value for the resource cluster as a whole, based upon local and remote contention data (step 606).
• the local WLM instance makes any necessary adjustments to the "need" values of local users by adjusting the "need” of any local holder of a resource that is not also a waiter for another resource (and thus is at the head of a contention chain) so that it at least matches the intrinsic need of the neediest waiter in the cluster containing the resource (step 608).
• This example is a cross-system transitive closure case: more than one resource is involved, and an unneedy user holding one resource is helped in order to get another (needy) user waiting on a different resource moving.
• the topology is multisystem, with holders and waiters for the same resource on different systems.
• Each holder and waiter is a transaction (Txn, e.g. TxA, TxB) and has an NQO (eNQueue Order) value. NQO values are such that smaller values are needier (more deserving of help).
• Txn e.g. TxA, TxB
• NQO eNQueue Order
• Each system is numbered (Syl, Sy2), and all of these systems are in the same "system cluster”.
• Each resource has a lowercase letter (Ra, Rb) and is multisystem in scope.
• Each resource cluster has one or more lowercase letters (Ca, Cab) showing the list of resources in the cluster. Requests to obtain resources are for exclusive control unless otherwise noted.
• TxB Since the neediest entity involved in the topology for Rb that Syl knows about (in fact, the only one at this point) is TxB, it uses TxB's NQO (4) as the NQO for Rb.
• Rb is a multisystem resource
• Syl broadcasts Rb's information to all other systems in the system cluster.
• the information sent for Rb includes the system name, the resource name, the cluster ED, the NQO for the resource based solely on the sending system's "local system information", and a boolean value (local/remote) that when set to "local” indicates that a transaction on the sending system forces the resource to be included in the cluster.
• the data sent is: Syl, Rb, Cb, 4, remote.
• Sy2 receives this information; concurrently, the resource manager instance running on Sy2 notifies Sy2 of contention on Rb.
• the order of operations is irrelevant, but they will be listed in the order previously described.
• the only "trick" in the code is that if the resource manager on Sy2 wins the race, when the remote data arrives the code must recognize that it already has an equivalent cluster built and add the remote information to its existing data.
• the local NQO on Sy2 for Rb is 4, not 5, which is TxC's NQO.
• the NQO(s) of resource holders never influence the resource's NQO; since the holder is running, WLM's policy adjustment code is already using the NQO implicitly.
• Sy2 now knows that somewhere else in the system cluster a transaction with an NQO of 4 is waiting; since 4 is defined as being needier than 5, the NQO for Rb must be no less needy than 4.
• Syl When Syl next reevaluates its topology, it knows based on local information that a single transaction (TxB) is involved with two different resources (Ra and Rb), and therefore the management of those resources must be integrated (in other words, Ra and Rb must be in the same resource cluster Cab).
• the NQO of the cluster is the neediest NQO of its member resources (1 in this case).
• the "signal" that Ra and Rb must be managed together is the presence of at least one transaction that is both holding one or more resources under contention and waiting on one or more other resources under contention.
• Syl After reevaluating its view of the topology, Syl (as before) broadcasts its view to other systems in the cluster.
• the dummy NQO value is simply one that is less needy than anything WLM could ever generate. Syl has no purely local NQO value since it has no local waiters, but it does need to send out the "virtual message" that Ra and Rb must be managed as a unit based on its local data.
• Sy2 integrates the data (including the fact that Ra and Rb must be managed as a unit, meaning that Ca and Cb must be merged), yielding the following.
• the resource manager instance on Syl finds out that Rb is available and gives it to the first waiter on its queue (Syl : TxB is resumed and acquires Rb). Since the resource manager's wait queue is now empty, it notifies WLM to indicate that Rb's contention has ended. Syl removes Rb from its resource cluster, since within each system any single resource can only belong to a single cluster (although two systems might have the same resource in different clusters due to timing windows).
• Sy2 sees the end of contention as well (Sy2: TxA is resumed and acquires Ra (no contention)).
• the resource manager instance on Sy2 notifies WLM to signal the end of Ra's contention.
• This example is another cross-system transitive closure case: more than one resource is involved, and an unneedy user holding one resource must be helped in order to get another (needy) user waiting on a different resource moving.
• the topology is again multisystem, with holders and waiters for the same resource on different systems, fri addition, and in contrast to Example 1, each system has contention involving the same transactions on purely local (non-multisystem) resources. This shows what happens when both multisystem and single system resources are involved in the same resource cluster.
• Rlocal is a proxy name for "some unknown set of resources which are local in scope to a remote system". The actual value is irrelevant, the only requirement being that all participants agree to the value and that it not be allowed to collide with any valid resource name.
• Sy2 After receiving this data and updating its topology, Sy2 believes this to be the state.
• Sy2 Similar processing occurs on Sy2 as just did on Syl, and then Sy2 broadcasts its data to Syl .
• Sy2 broadcasts the following:
• the name of the proxy for local resources on Sy2 is implicitly qualified by the cluster name since, as noted below, a proxy is defined for each resource cluster, not just for the system cluster as a whole. Also, only the broadcasts for Ra and Rlocal contain the boolean value "local", since only those two resources are assignable to a common cluster on the basis of local data.
• Sy2 sees the end of contention as well (Sy2: TxA is resumed and acquires Ra).
• the resource manager instance on Sy2 notifies WLM to signal the end of Ra's contention.
• This example involves breaking a resource cluster into smaller clusters without contention ending for any of the resources involved.
• the transaction linking Ra and Rb is cancelled, but since each resource has other waiters both resources are still in contention afterward. Notation is as in Example 1.
• Syl explicitly sends data to indicate that it no longer believes a given resource cluster is necessary. For example, send: Ra, Ca, 4, remote.
• Ra Ra
• Ca Ra
• remote When Sy2 replaces Syl's earlier data for Ra, it no longer sees any requirement to manage Ra and Rb together coming from Syl; if Sy2 has no other "votes" to continue the cluster, Sy2 can break the cluster locally.
• TTL time to live
• a resource cluster joined only by common holder(s) can be treated either as one resource cluster of "n" resources or as "n” clusters of one resource each. This result is surprising enough to be worth documenting.
• TxA ends up inheriting an NQO of 1 regardless of whether this scenario is treated as one or two resource clusters, we can choose either. Since managing two "trivial" (single resource) clusters is more efficient than a single composite cluster due to the tests for when the composite needs to be decomposed, this case is treated as two trivial resource clusters.
• Example 5 Simple Three- Way Scenario (3wayEasy)
• Example 1 This example is a simple three-system scenario. It is also a transitive closure case, but its asymmetric topology forces systems to track resources for which it has no local waiter/holder information coming from the resource manager. Notation is as in Example 1.
• the state data at this point is:
• This example is a three-system transitive closure case, where a large cluster is broken into smaller ones without any "end of contention" events to drive us.
• This example also shows a topology with multiple shared holders of a resource. Notation is as in Example 1.
• the state data at this point is:
• TxB a single transaction
• Figs. 8A-8H show one possible set of data structures for storing contention data in accordance with the present invention.
• a resource contention control table (RCCT) 802 is used to anchor various items of interest only (or mainly) to a single WLM subcomponent. It contains: 1. An anchor 804 for resource cluster elements (RCLUs) 806 (Fig.8B).
• RCCT resource contention control table
• An anchor 808 for resource elements (RSRCs) 810 (Fig. 8C).
• each resource cluster element (RCLU) 806 contains data related to a single resource cluster. It contains:
• a cluster NQO 818 (the minimum of all resources in the cluster).
• An anchor 820 for the resource elements (RSRCs) 810 (Fig. 8C) of the resources in the cluster.
• each resource element (RSRC) 810 describes a resource under contention. It contains:
• a resource NQO 824 (One might want to keep local/system cluster values separate for efficiency on the broadcast path; otherwise this is system cluster NQO.) 3.
• a pointer 826 to a cluster element (RCLU) 806 (Fig. 8B).
• An anchor 828 for resource contention queue elements (RCQEs) 830 (Fig. 8H) for local holders.
• RQEs resource contention queue elements
• An anchor 834 for system data anchors (SDAs) 836 (Fig. 8D) for remote data about this resource.
• each system data anchor (SDA) 836 serves as an anchor for remote system information for a single system. It contains:
• An anchor 840 for remote system data elements (RSDEs) 842 (Fig. 8E) from this system 3.
• a value 844 representing the remote system's highest known sending sequence number.
• the sending system includes a value (like a timestamp) that is the same for each 'batch' of topology data.
• Each receiving system compares the value in incoming messages against this value; if the message has a lower value (implying it is stale because the receiving system has already received later data from the same sender) then the message is ignored.
• each resource system data element (RSDE) 842 contains remote system information for a resource. It contains:
• RSRC resource element
• a sending timestamp 856 (the clock value on the remote system when sent), for debug only.
• a remote cluster ID 860 for this resource If the remote system had a transaction that was both a holder and a waiter, all resources involved will have the same cluster ID there and need to be in the same cluster here. If remote data from different systems disagrees about which resources belong to a cluster, the clusters are unioned locally.
• a time to live (TTL) duration 862 supplied by the remote system corresponding to how often the remote system plans to send data plus a bit extra. If the local time is greater than the receipt timestamp plus this value, the element is eligible for deletion.
• TTL time to live
• transaction table (TRXNT) 814 is used to anchor various items of interest only (or mainly) to a single WLM subcomponent. It contains:
• TRXNEs entries (up to the number 864) for transactions that are address spaces.
• An area 872 for entries (TRXNEs) (up to the number 866) for transactions that are enclaves.
• each entry (TRXNE) 874 in area 870 or 872 of transaction table (TRXNT) 814 contains information about a single transaction that is involved with at least one resource whose contention is being managed by WLM. It contains:
• a type 876 address space or enclave.
• An address space or enclave token 880 for this transaction ASIDs and enclave IDs are reusable; the tokens provide uniqueness within a single image even if the IDs are reused.
• An anchor 882 for a queue 884 of contention elements (RCQEs) 830 (Fig. 8H) for resources held by this transaction.
• each resource contention queue element (RCQE) 830 relates a transaction (holder or waiter) to a resource. It contains:
• Fig. 9 shows how the contention scenario shown in Fig. 4 and summarized for Sy2 in the table accompanying the Fig. 4 description is captured by the various data structures shown in

Landscapes

• Engineering & Computer Science (AREA)
• Theoretical Computer Science (AREA)
• Software Systems (AREA)
• Physics & Mathematics (AREA)
• General Engineering & Computer Science (AREA)
• General Physics & Mathematics (AREA)
• Computer Hardware Design (AREA)
• Mathematical Physics (AREA)
• Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
• Multi Processors (AREA)
• Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Priority Applications (2)

Applications Claiming Priority (2)

Publications (1)

Family

ID=32654967

Family Applications (1)

Country Status (6)

Families Citing this family (22)

Citations (1)

Family Cites Families (18)

• 2002
  • 2002-12-31 US US10/334,203 patent/US7228351B2/en not_active Expired - Fee Related
• 2003
  • 2003-11-14 WO PCT/EP2003/014865 patent/WO2004066148A1/en not_active Ceased
  • 2003-11-14 JP JP2004566814A patent/JP4370260B2/ja not_active Expired - Fee Related
  • 2003-11-14 KR KR1020057009822A patent/KR100834432B1/ko not_active Expired - Fee Related
  • 2003-11-14 AU AU2003300229A patent/AU2003300229A1/en not_active Abandoned
  • 2003-11-14 CN CNB2003801030057A patent/CN1331052C/zh not_active Expired - Fee Related

Patent Citations (1)

Non-Patent Citations (2)

Also Published As

Similar Documents

Legal Events