WO2020162800A1 - Prévention de conflit dans un ordonnancement de grappe dans un système de traitement de données - Google Patents

Prévention de conflit dans un ordonnancement de grappe dans un système de traitement de données Download PDF

Info

Publication number
WO2020162800A1
WO2020162800A1 PCT/SE2019/050094 SE2019050094W WO2020162800A1 WO 2020162800 A1 WO2020162800 A1 WO 2020162800A1 SE 2019050094 W SE2019050094 W SE 2019050094W WO 2020162800 A1 WO2020162800 A1 WO 2020162800A1
Authority
WO
WIPO (PCT)
Prior art keywords
resource
recited
ones
application
level
Prior art date
Application number
PCT/SE2019/050094
Other languages
English (en)
Inventor
Mina SEDAGHAT
Pontus SKÖLDSTRÖM
Original Assignee
Telefonaktiebolaget Lm Ericsson (Publ)
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Telefonaktiebolaget Lm Ericsson (Publ) filed Critical Telefonaktiebolaget Lm Ericsson (Publ)
Priority to PCT/SE2019/050094 priority Critical patent/WO2020162800A1/fr
Publication of WO2020162800A1 publication Critical patent/WO2020162800A1/fr

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/48—Program initiating; Program switching, e.g. by interrupt
    • G06F9/4806—Task transfer initiation or dispatching
    • G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
    • G06F9/4881—Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals

Definitions

  • the present disclosure is directed, in general, to the field of cloud computing and, more specifically, to a system and method to allocate resources in a data processing system.
  • a resource manager offers a coarse view of available resources in a cluster to a group of distributed application schedulers. This view may not be up-to-date as a plurality of application schedulers compete for resources and may claim the same resource concurrently. Overlapping claims for the same resources can produce scheduling conflicts.
  • a resource allocation process that addresses the aforementioned issues can enhance the efficiency with which resources are allocated in a data processing system including a data center, or the like.
  • an apparatus, and related method is configured to receive a request for resource views from application schedulers in response to a plurality of user requests to execute applications, categorize ones of the plurality of user requests into scheduling units, and provide the scheduling units to a resource manager to enable the resource manager to construct the resource views for the application schedulers.
  • a scheduler in a data processing system includes application schedulers, an arbiter and a resource manager, and a method of operating the same.
  • the application schedulers are configured to receive a plurality of user requests to execute applications.
  • the arbiter is configured to receive a request for resource views from the application schedulers in response to the plurality of user requests, and categorize ones of the plurality of user requests into scheduling units.
  • the resource manager is configured to receive the scheduling units, and construct the resource views for the application schedulers.
  • FIGURE 1 illustrates a block diagram of an embodiment of a two-level scheduler
  • FIGURE 2 illustrates a block diagram of an embodiment of a two-level scheduler employing an optimistic approach
  • FIGURES 3 to 5 illustrate block diagrams of embodiments of a two-level scheduler employing a pessimistic approach
  • FIGURES 6 and 7 illustrate block diagrams of embodiments of a two-level scheduler
  • FIGURE 8 illustrates a block diagram of an embodiment of a data processing system including a two-level scheduler demonstrating an operation thereof;
  • FIGURE 9 illustrates a block diagram of an embodiment of an arbiter
  • FIGURES 10 and 11 illustrate flow diagrams of embodiments of a method of operating a data processing system
  • FIGURE 12 illustrates a block diagram of an embodiment of a communication node in a data processing system.
  • resources allocable by a scheduler include, without limitation, servers, microprocessors and microprocessor cores and memory, bandwidth of communication systems, servers and portions thereof, transmitter power levels and associated receivers, and intervals of time over which such elements are enabled to operate.
  • a two-level scheduler is a scheduling platform for sharing and managing cluster resources between multiple application frameworks.
  • a two-level scheduler includes a resource manager (level one) and a set of application schedulers running in parallel (level two).
  • an application scheduler upon each application arrival requests a resource view of available resources from the resource manager.
  • the resource manager decides which resources to offer to each application, and the application scheduler decides which resources to accept and allocate, and finally, which tasks to execute on top of the allocated resources.
  • Each application scheduler is calibrated and designed to support the needs of a particular application framework. No single scheduling framework will be optimal for all types of applications, and therefore organizations should be enabled to run multiple application schedulers efficiently in the same cloud computing environment.
  • FIGURE 1 illustrated is a block diagram of an embodiment of a two-level scheduler 100.
  • the two-level scheduler 100 is formed with a resource manager (level one) 110 and a set of application schedulers (“AS”, collectively designated 120) running in parallel (level two).
  • AS application schedulers
  • the application scheduler 120 upon each application arrival, requests a resource view on available resources from the resource manager 110.
  • the resource manager 110 decides which resources
  • the application scheduler 120 decides which resources 130 to accept and allocate, and finally which tasks to execute on top of the allocated resources 130.
  • the application scheduler 120 is calibrated and designed to support the needs of a particular application framework.
  • a“pessimistic approach” conflicts are avoided by ensuring that a resource is only offered to one application scheduler at a time. (See, e.g ., the“Mesos article” referenced above.) This is achieved by either partitioning the full set of resources into independent resource views that are offered to different application schedulers or by offering all the available cluster resources to a single application scheduler at a time. Since only a single application scheduler is examining a particular set of resources (either the full set or a partition of it) at any point in time, it effectively holds a lock on those resources for the duration of a scheduling decision.
  • scheduling parallelism is reduced as resources are locked for each application scheduler, or else visibility of resources for each application scheduler is restricted.
  • the restrictions in parallelism through locking lead to lower scheduling throughput and therefore longer total scheduling time.
  • Offering a fraction of available resources leads to sub-optimal scheduling decisions for picky applications as not all available resources are considered when deciding what should be allocated. (See, e.g ., the“Omega article” referenced above.)
  • the pessimistic approach avoids conflicts, but reduces scheduling parallelism and offers only a fraction of locked resources.
  • an arbiter classifies application schedulers into different groups referred to as scheduling units.
  • the scheduling units correspond to a plurality of user requests that share resource views.
  • the arbiter uses this classification to decide how to parallelize the application schedulers or how to share the cluster view among the application schedulers.
  • the arbiter determines for what classes of user requests associated with application schedulers it is safe to concurrently operate on a full resource view, and for which classes of application schedulers the resource view should be either locked and offered sequentially and/or partitioned before being offered to either individual application schedulers or to groups of application schedulers.
  • the mechanism introduced herein retains a high level of parallelization offered by the optimistic approach (and thus provides higher scheduling throughput) by maintaining the idea of concurrently sharing a resource view. In addition, it reduces the risk of conflicts through a classification scheme that actively reduces or avoids conflicts.
  • the arbiter classifies and controls how to share a cluster resource view, in both time and space, and between a plurality of application schedulers.
  • the aim of the arbiter is to reduce the probability of conflicts, and ensure that the raised conflicts are not expensive, from a resource perspective or otherwise, to resolve. For instance, if conflicts are not resolved within an acceptable time and/or scheduling has to be repeated as a result thereof, extra overhead is employed to resolve the conflict.
  • the arbiter categorizes the application schedulers according to the following groups.
  • the arbiter groups the application schedulers to run in parallel with a shared resource view if either the risk of conflict between the application schedulers is low, or the cost of conflict resolution is cheap, for example, in the case of conflict, the conflicting applications can be co-located without notable performance impact. This would be the case when the cost of performance degradation for the application is cheaper, from a resource perspective or otherwise, than the cost of the decision to roll back and re-schedule.
  • the application schedulers responsible for scheduling short-lived applications can potentially run in parallel with the application schedulers of long-lived applications with no critical service level agreement (“SLA”).
  • SLA critical service level agreement
  • the application schedulers responsible for scheduling low-priority applications can be grouped with schedulers of high-priority applications to be run in parallel. In the case of conflict, the cost of aborting scheduling decisions for the conflicting low priority applications is negligible and would not impact high-priority operations.
  • the application schedulers that are responsible for scheduling applications with different resource demands or demand patterns can potentially schedule together. In this case, the applications do not compete over the same resource type, so it is less likely that they choose the same server. For example, an application scheduler scheduling a CPU- bound application (with no critical SLA) can run in parallel with an application scheduler scheduling an I/O-bound process working on a shared-resource view. In case of possible conflicts (e.g ., the server can be overcommitted), the applications can be collocated and without drastic performance degradation, since the performance of the different applications is weakly correlated due to using different resources.
  • the arbiter can determine if an application requires a full resource view for an improved scheduling decision or offering a sub-view would be enough for the scheduling without sacrificing the quality of the decision. For example, picky applications (an application with many constraints), often require access to the full state of the cluster as described in the“Omega article” referenced above, whereas applications schedulers with soft constraints can schedule their applications using only a fraction of available resources.
  • the application schedulers which schedule applications with specific requirements, (e.g., the applications require hardware features such as remote directory memory access (“RDMA”) or field programmable gate array (“FPGA”)), can work on an isolated sub view, independent from the rest of the cluster. Having a plurality of application schedulers, each running on an isolated sub-view of resources can increase parallelization and therefore higher scheduling throughput.
  • RDMA remote directory memory access
  • FPGA field programmable gate array
  • the arbiter may strictly isolate an application scheduler and run it with a locked resource view if there is a high risk of conflict, and the cost of conflict resolution is high.
  • An example includes if the application scheduler is scheduling a relatively large application compared to the offered view. In this case, it is highly probable that most of resources in the resource offer are accepted and allocated, and therefore the risk of conflict is high.
  • Another example is if the application has a strict short scheduling deadline and the possible conflict would lead to a violation of the scheduling deadline.
  • Another example includes if the application is large and the risk of repeated conflicts would increase the chance of starvation.
  • An application is often described with multiple features. For example, an application can be long-lived, while at the same time have critical SLAs and therefore high priority. Some of these features are usually correlated (e.g., applications with critical SLAs are often high-priority as well).
  • the arbiter weighs the features against each other, to devise a decision. In other words, the different properties of each feature are correlated to devise the decision.
  • the arbiter profiles the application schedulers, using a rough estimate on following example features of their associated applications, as illustrated in Table 1 below.
  • an arbiter uses profiles to group user requests into scheduling units.
  • Each scheduling unit is formed with a single or group of application schedulers that can run in parallel using a shared resource view. If a scheduling unit includes only one application scheduler, this application scheduler runs in isolation, and in sequence with other scheduling units.
  • the arbiter also decides and controls how to share the resource view between the application schedulers within a scheduling unit (z.e., if a view needs to be sub-divided).
  • the arbiter also iteratively collects, analyzes, and identifies conflicts. The analysis uses this information to update the application scheduler’s profile and makes better decisions over time.
  • the scheduling units correspond to a plurality of user requests that share resource views.
  • the arbiter categorizes the plurality of user requests into the scheduling units to reduce risk of conflict between the application schedulers.
  • FIGURE 2 illustrated is a block diagram of an embodiment of a two-level scheduler 200 employing an optimistic approach.
  • the two-level scheduler 200 is employing the optimistic approach with high parallelization, but with a chance of conflict.
  • a resource manager 210 operating in level one 220 provides a full resource view employing an optimistic approach with parallel scheduling by application schedulers (“AS”) running in level two 230.
  • AS application schedulers
  • all the application schedulers (designated AS 1, 2, 3, 4, 5) run in parallel and produce a conflict in resource allocation between application scheduler 4 and application scheduler 5, which is a likely result of employing an optimistic approach.
  • AS application schedulers
  • FIGURE 3 illustrated is a block diagram of an embodiment of a two-level scheduler 300 employing a pessimistic approach.
  • a resource manager 310 operating in level one 320 provides a full resource view employing a pessimistic approach with space partitioning.
  • the resource manager 310 provides sub views (designated SV 1, 2, 3, 4, 5) to corresponding application schedulers (designated AS 1, 2, 3, 4, 5) operating in level two 330.
  • This arrangement assumes that scheduling conflicts are high, and conflicting claims cannot be resolved in a safe and cheap manner.
  • each framework application scheduler is restricted to a sub-view of resources. This approach leads to sub-optimal decisions because of the limited view of resources, but no conflicts.
  • FIGURE 4 illustrated is a block diagram of an embodiment of a two-level scheduler 400 employing a pessimistic approach.
  • the applications are scheduled sequentially by a resource manager 410 operating level one 420, by the sequential arrangement of application schedulers (designated AS 1, 2, 3, 4, 5) operating in level two 430.
  • the resource manager 410 provides a full resource view and employs a pessimistic approach with sequentially scheduling.
  • FIGURE 5 illustrated is a block diagram of an embodiment of a two-level scheduler 500 employing a pessimistic approach.
  • a resource manager 510 operating in level one 520 offers a coarse, but full view of available resources in the cluster to a group of distributed application schedulers operating in level two 530 with time partitioning for the application schedulers (designated AS 1, 2, 3).
  • Applications are again scheduled temporally sequentially by the resource manager 510 such as indicated in FIGURE 5 by the temporally sequential arrangement of application schedulers 1, 2, 3.
  • this view may not necessarily always be up-to-date as multiple application schedulers are competing for resources and may claim the same resource concurrently which can result in scheduling conflicts. Sequential scheduling leads to lower scheduling throughput and delayed scheduling decisions, and a less than optimal arrangement that results from scheduling resources sequentially.
  • the two-level scheduler 600 includes an arbiter 610 that instructs a resource manager 620 operating in level one 630, which provides resource views for application schedulers (designated AS 1, 2, 3, 4, 5) operating in level two 640. In applications with a low risk of conflict, resources are scheduled in parallel by the arbiter 610. High risk application schedulers operating in level two 640 get a full resource view for improved ( e.g ., optimal) decisions.
  • the arbiter 610 selectively groups a portion of the application schedulers (i.e., AS 2, 3, 4), to share the resource view in a common interval of time. Additionally, the arbiter 610 also schedules applications temporally sequentially as indicated by the temporally sequential arrangement of application scheduler 1, then the group of applications schedulers 2, 3, 4, followed by the application scheduler 5.
  • the two-level scheduler 700 includes an arbiter 710 that instructs a resource manager 720 operating in level one 730, which provides resource views for application schedulers (designated AS 1, 2, 3, 4, 5) operating in level two 740. Again, in applications with a low risk of conflict, resources are scheduled in parallel by the arbiter 610. High risk application schedulers operating in level two 640 get a full resource view for improved (e.g., optimal) decisions.
  • the arbiter 610 selectively groups a portion of the application schedulers (i.e., AS 1, 5), to share the resource view in a common space with respect to a sub-view (“SV”) 750. Additionally, the arbiter 610 also schedules applications in parallel as indicated by the parallel arrangement of application schedulers 2, 3, 4 with respect to a sub-view 760.
  • the arbiters 610, 710 determine for what classes of application schedulers it is safe to concurrently operate on the full resource view, and for which classes of application schedulers the resource view should be either locked and offered sequentially and/or partitioned before being offered to either individual schedulers or groups of schedulers.
  • the arbiters 610, 710 classify and control how to share the cluster resource view, in both time and space, and between multiple application schedulers.
  • the arbiters 610, 710 use this classification to decide how to parallelize schedulers or how to share the cluster view among the application schedulers. The process retains parallelism and reduces the risk of conflict.
  • FIGURE 8 illustrated is a block diagram of an embodiment of a data processing system 800 including a two-level scheduler 810 operable on server(s) (generally designated 850) demonstrating an operation thereof.
  • the servers 850 may form at least a portion of a data center.
  • the two-level scheduler 810 includes a resource manager 820 at level one 825, a plurality of application schedulers (generally designated 830) at level two 835, and an arbiter 840 accessible by the resource manager 820 at level one 825 and the application schedulers 830 at level two 835.
  • the application schedulers 835 include, without limitation, a Dryad scheduler, a Hadoop scheduler, a service scheduler and a batch scheduler.
  • a plurality of users request resources (a user request) via a client (designated CT 1, 2, 3, 4 and generally designated 870) through a communication network 860 to execute applications from respective application schedulers 830.
  • the application schedulers 830 request the resources represented by a resource view from the arbiter 840.
  • the arbiter 840 categorizes ones of the user requests into scheduling units and provides the scheduling units to the resource manager 820.
  • the resource manager 820 offers the resource views based on recommendations of the arbiter 840.
  • the resource manager 820 shares the resource view among the application schedulers 830 grouped in a scheduling unit.
  • the arbiter 840 classifies application schedulers 830 into different groups based on scheduling units as set forth above.
  • the arbiter 910 receives user requests (designated SI, S2, S3, S4, S5, S6) asking for resource views from a queue 920.
  • the arbiter 860 receives the user requests from the application schedulers and retrieves generic profiles associated with the application schedulers from a profile database 930 that provides user parameters and constraints. Using the user requests and generic profiles, arbiter 860 groups the user requests and respective application schedulers into a plurality of scheduling units. Each scheduling unit specifies which application schedulers can run in parallel, and which schedulers should run in isolation or on an isolated sub-view.
  • the arbiter 860 produces an updated scheduling queue 940, where the user requests are grouped into the scheduling units.
  • the updated scheduling queue 940 is formed with scheduling user request SI assigned to one queue, scheduling user requests S2, S3, S4 assigned to a second queue, and scheduling user requests S5, S6 assigned to a third queue.
  • FIGURE 10 illustrated is a flow diagram of an embodiment of a method 1000 of operating a data processing system.
  • the method 1000 is operable in an apparatus (including, for instance, an arbiter 840 of the scheduler 810 of FIGURE 8) of the data processing system (e.g ., including a data center) and begins at a start step or module 1010.
  • the apparatus receives a request for resource views from application schedulers in response to a plurality of user requests to execute applications.
  • the apparatus categorizes ones of the plurality of user requests into scheduling units, at a step or module 1030 to, for instance, reduce the risk of conflict between the application schedulers.
  • Each of the scheduling units corresponds to the plurality of user requests that share ones of the resource views.
  • the apparatus provides the scheduling units to a resource manager to enable the resource manager to construct the resource views for the application schedulers.
  • the apparatus may instruct the resource manager to build a shared resource view wherein ones of the application schedulers schedule resources in parallel on a full cluster with a low risk of conflict.
  • the apparatus may instruct the resource manager to build an isolated resource view wherein there is a risk of conflict between ones of the plurality of user requests from
  • the apparatus may instruct the resource manager to build an isolated resource sub-view wherein ones of the application schedulers schedule resources that run on a partition of a cluster with a low risk of conflict. Also, the apparatus may instruct the resource manager to allocate resources in time and space to the application schedulers as indicated in the resource views and in accordance with providing the scheduling units.
  • the resource views may allocate resources including, without limitation, servers, processors, memory, bandwidth of communication systems, transmit power levels; and intervals of time to operate the resources.
  • the apparatus may operate within a two-level scheduler wherein the resource manager operates in a first level, the application schedulers operate in a second level, and the arbiter operates in a level accessible by the first level and the second level.
  • the method 1000 ends at a step or module 1050.
  • FIGURE 11 illustrated is a flow diagram of an embodiment of a method 1100 of operating a data processing system.
  • the method 1100 is operable in an apparatus (including, for instance, a scheduler 810 of FIGURE 8) of the data processing system (e.g ., including a data center) and begins at a start step or module 1110.
  • the apparatus receives a plurality of user requests to execute applications (for instance, at an application scheduler 830 of the scheduler 810 of FIGURE 8).
  • the apparatus categorizes ones of the plurality of user requests into scheduling units in response to the plurality of user requests (for example, by an arbiter 840 of the scheduler 810 of FIGURE 8) to, for instance, reduce the risk of conflict (e.g., between the application schedulers 830).
  • Each of the scheduling units corresponds to the plurality of user requests that share ones of the resource views.
  • the apparatus constructs resource views (e.g, for the application schedulers 830) in response to the scheduling units (for instance, by a resource manager 820 of the scheduler 810 of FIGURE 8).
  • the apparatus may build a shared resource view wherein resources are scheduled in parallel on a full cluster with a low risk of conflict.
  • the apparatus may build an isolated resource view wherein there is a risk of conflict between ones of the plurality of user requests from corresponding ones of application schedulers, the corresponding ones of the application schedulers running independently and sequentially on a full cluster.
  • the apparatus may build an isolated resource sub-view wherein ones of application schedulers schedule resources that run on a partition of a cluster with a low risk of conflict.
  • the apparatus allocates resources (e.g, in time and space) as indicated in the resource views in accordance with the scheduling units (for instance, by the resource manager 820 of the scheduler 810 of FIGURE 8).
  • the resource views may allocate resources including, without limitation, servers, processors, memory, bandwidth of communication systems, transmit power levels; and intervals of time to operate the resources.
  • the method 1100 may operate within a two-level scheduler wherein the functions associated with the resource manager operates in a first level, the functions associated with the application schedulers operate in a second level, and the functions associated with the arbiter operates in a level accessible by the first level and the second level.
  • the method 1100 ends at a step or module 1160.
  • the communication node 1200 may be a server configured to perform the functions of a scheduler in a data center.
  • the communication node 1200 includes a processor (or processing circuitry) 1210, a memory 1220 and a communication interface 1230.
  • the communication node 1200 may also include an antenna(s) 1240 depending on the type of device such as a server with wireless communication capability.
  • MTC machine type communication
  • M2M machine-to-machine
  • the functionality of the communication node 1200 may be provided by the processor 1210 executing instructions stored on a computer-readable medium, such as the memory 1220 shown in FIGURE 12.
  • Alternative embodiments of the communication node 1200 may include additional components (such as the interfaces, devices and circuits mentioned above) beyond those shown in FIGURE 12 that may be responsible for providing certain aspects of the device’s functionality, including any of the functionality to support the solution described herein.
  • the processor 1210 (or processors), which may be implemented with one or a plurality of processing devices, perform functions associated with its operation including, without limitation, performing the operations of an arbiter, decoding of individual bits forming a communication message, formatting of information and overall control of a respective communication node 1200.
  • Exemplary functions related to management of communication resources include, without limitation, hardware installation, traffic management, performance data analysis, configuration management, security, billing and the like.
  • the processor 1210 may be of any type suitable to the local application environment, and may include one or more of general-purpose computers, special purpose computers, microprocessors, digital signal processors (“DSPs”), field- programmable gate arrays (“FPGAs”), application-specific integrated circuits (“ASICs”), and processors based on a multi-core processor architecture, as non-limiting examples.
  • DSPs digital signal processors
  • FPGAs field- programmable gate arrays
  • ASICs application-specific integrated circuits
  • processors based on a multi-core processor architecture as non-limiting examples.
  • the processor 1210 may include, without limitation, application processing circuitry.
  • the application processing circuitry may be on separate chipsets.
  • part or all of the application processing circuitry may be combined into one chipset, and other application circuitry may be on a separate chipset.
  • part or all of the application processing circuitry may be on the same chipset, and other application processing circuitry may be on a separate chipset.
  • part or all of the application processing circuitry may be combined in the same chipset.
  • the processor 1210 may be configured to perform any operations described herein.
  • the operations as performed by the processor 1210 may include processing information obtained by the processor by, for example, converting the obtained in formation into other information, comparing the obtained information or converted information to information stored in the respective device, and/or performing one or more operations based on the obtained information or converted information, and, as a result of the processing, making a determination.
  • the memory 1220 may be one or more memories and of any type suitable to the local application environment, and may be implemented using any suitable volatile or nonvolatile data storage technology such as a semiconductor-based memory device, a magnetic memory device and system, an optical memory device and system, fixed memory and removable memory.
  • the programs stored in the memory 1220 may include program instructions or computer program code that, when executed by an associated processor, enable the respective communication node 1200 to perform its intended tasks.
  • the memory 1220 may form a data buffer for data transmitted to and from the same.
  • Exemplary embodiments of the system, subsystems, and modules as described herein may be implemented, at least in part, by computer software executable by the processor 1210, or by hardware, or by combinations thereof.
  • the communication interface 1230 modulates information onto a carrier waveform for transmission by the respective communication node 1200 to another communication node.
  • the respective communication interface 1230 also demodulates information received from another server for further processing.
  • the communication interface 1230 can support duplex operation for the respective server 1200, and supports communication with a core network.
  • the antenna 1240 may be any type of antenna capable of transmitting and receiving data and/or signals wirelessly. In some embodiments,
  • the antenna 1240 may include one or more omni-directional, sector or panel antennas operable to transmit/receive radio signals between, for example, 2 gigahertz (“GHz”) and 66 GHz.
  • GHz gigahertz
  • An omni-directional antenna may be used to
  • a sector antenna may be used to transmit/receive radio signals from devices within a particular area, and a panel antenna may be a line of sight antenna used to transmit/receive radio signals in a relatively straight line.
  • the antenna 1240 facilitates wireless communication for the communication node 1200
  • the communication node 1200 may also communicate via a wired communication path via the communication interface 1230 and, in such instances, the antenna 1240 may not be necessary.
  • the subsystems as introduced above with respect to the preceding FIGURES may be embodied in the communication node 1200 performed by, for instance, the processor 1210 in conjunction with the memory 1220.
  • a system and method has been introduced herein to allocate resources by a plurality of application schedulers in a data processing system.
  • the system and method can be performed in real-time, taking into account multiple criteria, to allocate resources within the data processing system.
  • an apparatus e.g ., an arbiter 840 embodied in a server 1200 in a data processing system (800) includes processing circuitry (1210) configured to receive a request for resource views from application schedulers (830) in response to a plurality of user requests (from user equipment 870) to execute applications.
  • the processing circuitry (1210) is also configured to categorize ones of the plurality of user requests into scheduling units to, for instance, reduce the risk of conflict between the application schedulers (830).
  • Each of the scheduling units corresponds to the plurality of user requests that share ones of the resource views.
  • the processing circuitry (1210) is also configured to provide the scheduling units to a resource manager (820) to enable the resource manager (820) to construct the resource views for the application schedulers (830).
  • the processing circuitry (1210) may instruct the resource manager (820) to build a shared resource view wherein ones of the application schedulers (830) schedule resources in parallel on a full cluster with a low risk of conflict.
  • the processing circuitry (1210) may instruct the resource manager (820) to build an isolated resource view wherein there is a risk of conflict between ones of the plurality of user requests from corresponding ones of the application schedulers (830), the corresponding ones of the application schedulers (830) running independently and sequentially on a full cluster.
  • the processing circuitry (1210) may instruct the resource manager (820) to build an isolated resource sub-view wherein ones of the application schedulers (830) schedule resources that run on a partition of a cluster with a low risk of conflict. Also, the processing circuitry (1210) may instruct the resource manager (820) to allocate resources in time and space to the application schedulers (830) as indicated in the resource views and in accordance with providing the scheduling units.
  • the resource views may allocate resources including, without limitation, servers, processors, memory, bandwidth of communication systems, transmit power levels; and intervals of time to operate the resources.
  • the apparatus (840, 1200) may operate within a two-level scheduler (e.g ., a scheduler 810 embodied in a server 1200) wherein the resource manager (820) operates in a first level (825), the application schedulers (830) operate in a second level (835), and the arbiter (840) operates in a level accessible by the first level (825) and the second level (835).
  • a scheduler in a data processing system (800) includes a resource manager (820), application schedulers (830) and an arbiter (840).
  • the application schedulers (830) are configured to receive a plurality of user requests (from user equipment 870) to execute applications.
  • the arbiter (840) is configured to receive a request for resource views from the application schedulers (830) in response to a plurality of user requests, and categorize ones of the plurality of user requests into scheduling units to, for instance, reduce the risk of conflict between the application schedulers (830).
  • Each of the scheduling units corresponds to the plurality of user requests that share ones of the resource views.
  • the resource manager (820) is configured to receive the scheduling units, and construct the resource views for the application schedulers (830). In accordance with the scheduling units, the resource manager (820) may build a shared resource view wherein ones of the application schedulers (830) schedule resources in parallel on a full cluster with a low risk of conflict. In another embodiment in accordance with the scheduling units, the resource manager (820) may build an isolated resource view wherein there is a risk of conflict between ones of the plurality of user requests from corresponding ones of the application schedulers (830), the corresponding ones of the application schedulers (830) running independently and sequentially on a full cluster.
  • the resource manager (820) may build an isolated resource sub-view wherein ones of the application schedulers (830) schedule resources that run on a partition of a cluster with a low risk of conflict. Also, the resource manager (820) may allocate resources in time and space to the application schedulers (830) as indicated in the resource views and in accordance with the scheduling units.
  • the resource views may allocate resources including, without limitation, servers, processors, memory, bandwidth of communication systems, transmit power levels; and intervals of time to operate the resources.
  • the resource manager (820) may operate in a first level (825), the application schedulers (830) may operate in a second level (835), and the arbiter (840) may operate in a level accessible by the first level (825) and the second level (835).
  • the exemplary embodiment provides both a method and corresponding apparatus consisting of various modules providing functionality for performing the steps of the method.
  • the modules may be implemented as hardware (embodied in one or more chips including an integrated circuit such as an application specific integrated circuit), or may be implemented as software or firmware for execution by a processor.
  • firmware or software the exemplary embodiment can be provided as a computer program product including a computer readable storage medium embodying computer program code (i.e., software or firmware) thereon for execution by the computer processor.
  • the computer readable storage medium may be non-transitory (e.g., magnetic disks; optical disks; read only memory; flash memory devices; phase-change memory) or transitory (e.g., electrical, optical, acoustical or other forms of propagated signals-such as carrier waves, infrared signals, digital signals, etc.).
  • the coupling of a processor and other components is typically through one or more busses or bridges (also termed bus controllers).
  • the storage device and signals carrying digital traffic respectively represent one or more non-transitory or transitory computer readable storage medium.
  • the storage device of a given electronic device typically stores code and/or data for execution on the set of one or more processors of that electronic device such as a controller.

Landscapes

  • Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Multi Processors (AREA)

Abstract

L'invention concerne un système et un procédé d'attribution de ressources dans un système de traitement de données (800). Dans un mode de réalisation, un appareil (840, 1200), et un procédé associé (1000), sont configurés pour recevoir une demande de vues de ressource provenant d'ordonnanceurs d'application (830) en réponse à une pluralité de demandes d'utilisateur pour exécuter des applications, catégoriser certaines demandes de la pluralité de demandes d'utilisateur dans des unités d'ordonnancement, et fournir les unités d'ordonnancement à un gestionnaire de ressources (820) pour permettre au gestionnaire de ressources (820) de construire les vues de ressources pour les ordonnanceurs d'application (830).
PCT/SE2019/050094 2019-02-05 2019-02-05 Prévention de conflit dans un ordonnancement de grappe dans un système de traitement de données WO2020162800A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/SE2019/050094 WO2020162800A1 (fr) 2019-02-05 2019-02-05 Prévention de conflit dans un ordonnancement de grappe dans un système de traitement de données

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/SE2019/050094 WO2020162800A1 (fr) 2019-02-05 2019-02-05 Prévention de conflit dans un ordonnancement de grappe dans un système de traitement de données

Publications (1)

Publication Number Publication Date
WO2020162800A1 true WO2020162800A1 (fr) 2020-08-13

Family

ID=65433708

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/SE2019/050094 WO2020162800A1 (fr) 2019-02-05 2019-02-05 Prévention de conflit dans un ordonnancement de grappe dans un système de traitement de données

Country Status (1)

Country Link
WO (1) WO2020162800A1 (fr)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2022047621A1 (fr) 2020-09-01 2022-03-10 Huawei Cloud Computing Technologies Co., Ltd. Systèmes et procédés de planification de distribution centralisée hybride sur des hôtes physiques partagés
CN116436806A (zh) * 2023-06-12 2023-07-14 亚信科技（中国）有限公司 算网资源编排方法、装置、电子设备及计算机存储介质

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9697045B2 (en) * 2015-03-24 2017-07-04 International Business Machines Corporation Selecting resource allocation policies and resolving resource conflicts
US20190014059A1 (en) * 2017-07-06 2019-01-10 Zhenhua Hu Systems and methods for allocating computing resources in distributed computing

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9697045B2 (en) * 2015-03-24 2017-07-04 International Business Machines Corporation Selecting resource allocation policies and resolving resource conflicts
US20190014059A1 (en) * 2017-07-06 2019-01-10 Zhenhua Hu Systems and methods for allocating computing resources in distributed computing

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
HINDMAN, BENJAMIN ET AL.: "A Common Substrate for Cluster Computing", HOTCLOUD, 2009
HINDMAN, BENJAMIN ET AL.: "Mesos: A Platform for Fine-Grained Resource Sharing in the Data Center", NSDI, vol. 11, 2011
SCHWARZKOPF, MALTE ET AL.: "Proceedings of the 8th ACM European Conference on Computer Systems", 2013, ACM, article "Omega: Flexible, Scalable Schedulers for Large Compute Clusters"

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2022047621A1 (fr) 2020-09-01 2022-03-10 Huawei Cloud Computing Technologies Co., Ltd. Systèmes et procédés de planification de distribution centralisée hybride sur des hôtes physiques partagés
EP4154111A4 (fr) * 2020-09-01 2023-04-12 Huawei Cloud Computing Technologies Co., Ltd. Systèmes et procédés de planification de distribution centralisée hybride sur des hôtes physiques partagés
CN116436806A (zh) * 2023-06-12 2023-07-14 亚信科技（中国）有限公司 算网资源编排方法、装置、电子设备及计算机存储介质
CN116436806B (zh) * 2023-06-12 2023-08-18 亚信科技（中国）有限公司 算网资源编排方法、装置、电子设备及计算机存储介质

Similar Documents

Publication Publication Date Title
US11855766B2 (en) Technologies for dynamically managing resources in disaggregated accelerators
Karanasos et al. Mercury: Hybrid centralized and distributed scheduling in large shared clusters
US9686141B2 (en) Systems and methods for resource sharing between two resource allocation systems
WO2018120993A1 (fr) Procédé et dispositif permettant d'attribuer une tâche de système distribué
KR20220011063A (ko) 서버 리소스 할당 방법, 장치, 전자 기기 및 저장 매체
US10686728B2 (en) Systems and methods for allocating computing resources in distributed computing
CN107688492B (zh) 资源的控制方法、装置和集群资源管理系统
EP4002115A1 (fr) Procédé d'attribution de ressources et procédé de déchargement de ressources
EP3114589B1 (fr) Système et procédé de traitement massivement parallèle d'une base de données
ES2808951T3 (es) Método, dispositivo y sistema de procesamiento de mensajes
US20230342191A1 (en) Task Scheduling Method and System
Wang et al. Application-aware offloading policy using SMDP in vehicular fog computing systems
WO2022111453A1 (fr) Procédé et appareil de traitement de tâches, procédé d'attribution de tâches, et dispositif et support électroniques
WO2020162800A1 (fr) Prévention de conflit dans un ordonnancement de grappe dans un système de traitement de données
CN112905342A (zh) 资源调度方法、装置、设备及计算机可读存储介质
CN110914805A (zh) 用于分层任务调度的计算系统
US10261832B2 (en) Sorting apparatus
CN116166395A (zh) 任务调度方法、装置、介质及电子设备
CN112506821A (zh) 一种系统总线接口请求仲裁方法及相关组件
CN111176812A (zh) 一种集群化定时任务调度系统
CN113157404B (zh) 任务处理方法和装置
Liao et al. Efficient and fair scheduler of multiple resources for MapReduce system
CN115904673B (zh) 云计算资源并发调度方法、装置、系统、设备及介质
Kaladevi et al. Processor co-allocation enabling advanced reservation of jobs in MultiCluster systems
US11973674B2 (en) Dynamic and deterministic acceleration of network scheduling for shared FPGAs

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19705402

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19705402

Country of ref document: EP

Kind code of ref document: A1