CN117009063A - Task execution method and device - Google Patents
Task execution method and device Download PDFInfo
- Publication number
- CN117009063A CN117009063A CN202210466320.0A CN202210466320A CN117009063A CN 117009063 A CN117009063 A CN 117009063A CN 202210466320 A CN202210466320 A CN 202210466320A CN 117009063 A CN117009063 A CN 117009063A
- Authority
- CN
- China
- Prior art keywords
- task
- exclusive
- target task
- target
- slots
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 82
- 230000008569 process Effects 0.000 claims abstract description 19
- 230000002159 abnormal effect Effects 0.000 claims abstract description 16
- 238000010586 diagram Methods 0.000 claims description 24
- 238000013507 mapping Methods 0.000 claims description 16
- 238000004590 computer program Methods 0.000 claims description 11
- 238000004364 calculation method Methods 0.000 claims description 8
- 238000004140 cleaning Methods 0.000 claims description 2
- 238000002955 isolation Methods 0.000 abstract description 8
- 230000005856 abnormality Effects 0.000 description 10
- 238000003860 storage Methods 0.000 description 10
- 238000004891 communication Methods 0.000 description 7
- 230000006870 function Effects 0.000 description 6
- 238000012545 processing Methods 0.000 description 5
- 230000003287 optical effect Effects 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 230000008901 benefit Effects 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 239000000835 fiber Substances 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 239000004065 semiconductor Substances 0.000 description 2
- 238000010276 construction Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5061—Partitioning or combining of resources
- G06F9/5072—Grid computing
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Mathematical Physics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Debugging And Monitoring (AREA)
Abstract
The invention discloses a task execution method and device, and relates to the technical field of computers. One embodiment of the method comprises the following steps: and calculating the number of task execution nodes required to be exclusive for the target task according to the maximum occupied number of the slots in the running process of the target task and the number of slots provided by each task execution node, if the number of task execution nodes required to be exclusive for the target task is not up to the number of task execution nodes required to be exclusive for the target task, searching the task execution nodes which are not exclusive for any task from the task execution nodes which are exclusive for the target task, marking the task execution nodes as being exclusive for the target task, and searching the slots in all the task execution nodes which are exclusive for the target task to execute the target task. According to the method and the device for achieving the task mixed deployment, the multi-task mixed deployment can achieve resource isolation at the level of the task manager, the task manager occupied by the task is shared independently, the task is prevented from being affected by each other when the task is abnormal, and a user can specify the number of resources required by the task when the task runs.
Description
Technical Field
The present invention relates to the field of computer technologies, and in particular, to a task execution method and apparatus.
Background
With the development of big data technology and the improvement of real-time requirements of services, more and more enterprises start to support internal services by using real-time computing. Flink is widely used in actual production practice as a streaming computing engine. Meanwhile, with the development of containerization technology, more and more applications are deployed through containerization. Flink clusters are also common in containerized deployments, often running on Kubernetes (K8 s) in actual production. However, in actual use, users often deploy multiple tasks simultaneously on the same cluster. In the existing scheme, a plurality of tasks are mixed and deployed, so that only slot level resource isolation is achieved, subtasks of a plurality of tasks on the same task manager (task execution node) often run at the same time, and when one task is abnormal and causes the task manager to be abnormal, all the tasks running on the task manager are abnormal, and the tasks are mutually influenced.
Disclosure of Invention
In view of this, the embodiments of the present invention provide a method and an apparatus for executing tasks, which can implement resource isolation at the level of a TaskManager during multi-task hybrid deployment, so that Job (task) exclusively shares the occupied TaskManager, avoiding the interaction when the task is abnormal, and a user may designate the number of resources required by the Job running with a slot number pre-estimated value as a reference.
To achieve the above object, according to one aspect of the embodiments of the present invention, there is provided a task execution method.
A method of task execution, comprising: calculating the number of task execution nodes required to be exclusive for a target task according to the maximum occupied number of slots when the target task runs and the number of slots provided by each task execution node, wherein the slots are used for executing the task; if the number of task execution nodes which are required to be exclusive by the target task does not reach the number of task execution nodes which are required to be exclusive by the target task, searching task execution nodes which are required to be exclusive by the target task from task execution nodes which are not required to be exclusive by any task according to the difference value between the number of task execution nodes which are required to be exclusive by the target task and the number of task execution nodes which are required to be exclusive by the target task, and marking the task execution nodes which are required to be exclusive by the target task; and searching slots in all task execution nodes exclusive to the target task to execute the target task, wherein all task execution nodes exclusive to the target task comprise the task execution node which is exclusive to the target task currently and the task execution node which can be exclusive.
Optionally, the target task is deployed in a link cluster; before calculating the number of task execution nodes required to be exclusive for the target task, the method comprises the following steps: calculating a maximum predicted value and a minimum predicted value of the slot position of the target task, wherein the maximum predicted value of the slot position is equal to the sum of the maximum parallelism in all slot position sharing groups in a stream execution area corresponding to the target task, and the stream execution area corresponding to the target task is a stream execution area to which a sub-task node in a data flow diagram of the target task belongs; the minimum predicted value of the slot is equal to the maximum value of the slot number required by each flow execution area corresponding to the target task; and determining the maximum occupied slot number of the target task in operation according to a preset rule according to the maximum predicted slot number value and the minimum predicted slot number value of the target task and the initial slot number designated by a user.
Optionally, the determining the maximum number of occupied slots when the target task runs according to a preset rule includes: if the initial number of slots designated by the user is smaller than the minimum preset value of slots, the minimum preset value of slots is used as the maximum number of occupied slots when the target task runs; if the initial number of slots designated by the user is larger than the maximum predicted value of the slots, taking the maximum predicted value of the slots as the maximum number of occupied slots when the target task runs; and if the initial number of slots designated by the user is greater than or equal to the minimum preset value of slots and less than or equal to the maximum preset value of slots, taking the initial number of slots designated by the user as the maximum number of occupied slots when the target task runs.
Optionally, the searching the task execution node which is not monopolized by any task and can be monopolized by the target task, and marking the task execution node which can be monopolized by the target task includes: the resource manager in the Flink cluster searches for a task execution node which is not monopolized by any task from task execution nodes registered to the resource manager, marks the task execution node as monopolized by the target task, and registers the task, the maximum occupied slot number of the task in running and the mapping relation between the task and the task execution node which is monopolized by the task currently in the resource manager.
Optionally, if the target task is executed successfully or an abnormality occurs in the executing process, releasing all task executing nodes exclusive to the target task.
Optionally, if an exception occurs in the cluster manager of the link cluster during the execution of the target task, after releasing all the task execution nodes that are exclusive to the target task, the method further includes: and after the resource manager and the task coordinator in the Flink cluster lose connection, cleaning the binding relation between the target task and all exclusive task execution nodes.
Optionally, if an abnormality occurs in the resource manager of the link cluster during the execution of the target task, the method further includes: and registering the task execution node exclusive to the target task with a new resource manager in the Flink cluster, and reconstructing a mapping relation between the target task and the exclusive task execution node according to a slot report sent by the task execution node exclusive to the target task by the new resource manager.
Optionally, after the task execution node exclusive to the target task successfully registers with a new resource manager in the link cluster, the method further includes: and the task execution node exclusive to the target task actively establishes connection with the new resource manager, and invokes a method for registering slot information of a slot manager implementation class to register the slot information of the task execution node exclusive to the target task.
Optionally, if an exception occurs in a task execution node exclusive to the target task in the target task execution process, the method further includes: and the resource manager in the Flink cluster releases the binding relation with the abnormal task execution node.
According to another aspect of the embodiment of the present invention, there is provided a task execution device.
A task execution device comprising: the task execution node number calculation module is used for calculating the number of task execution nodes required to be exclusive for a target task according to the maximum occupied number of slots in the running process of the target task and the number of slots provided by each task execution node, wherein the slots are used for executing the task; the task execution node marking module is used for searching task execution nodes which can be monopolized by the target task from task execution nodes which are not monopolized by any task according to the difference value between the task execution node number which needs to be monopolized by the target task and the task execution node number which needs to be monopolized by the target task if the task execution node number which is not monopolized by the target task currently does not reach the task execution node number which needs to be monopolized by the target task, and marking the task execution nodes which can be monopolized by the target task; and the task execution module is used for searching slots in all task execution nodes exclusive to the target task to execute the target task, wherein all task execution nodes exclusive to the target task comprise the task execution node which is exclusive to the target task currently and the task execution node which is exclusive to the target task.
Optionally, the target task is deployed in a link cluster; the device also comprises a slot position estimating module for: calculating a maximum predicted value and a minimum predicted value of the slot position of the target task, wherein the maximum predicted value of the slot position is equal to the sum of the maximum parallelism in all slot position sharing groups in a stream execution area corresponding to the target task, and the stream execution area corresponding to the target task is a stream execution area to which a sub-task node in a data flow diagram of the target task belongs; the minimum predicted value of the slot is equal to the maximum value of the slot number required by each flow execution area corresponding to the target task; and determining the maximum occupied slot number of the target task in operation according to a preset rule according to the maximum predicted slot number value and the minimum predicted slot number value of the target task and the initial slot number designated by a user.
Optionally, the slot estimation module includes a slot number calculation operator module for: if the initial number of slots designated by the user is smaller than the minimum preset value of slots, the minimum preset value of slots is used as the maximum number of occupied slots when the target task runs; if the initial number of slots designated by the user is larger than the maximum predicted value of the slots, taking the maximum predicted value of the slots as the maximum number of occupied slots when the target task runs; and if the initial number of slots designated by the user is greater than or equal to the minimum preset value of slots and less than or equal to the maximum preset value of slots, taking the initial number of slots designated by the user as the maximum number of occupied slots when the target task runs.
Optionally, the task execution node marking module is further configured to: and searching task execution nodes which are not monopolized by any task from task execution nodes registered to the resource manager through the resource manager in the Flink cluster, marking the task execution nodes as monopolized by the target task, wherein the resource manager is registered with the task, the maximum occupied slot number of the task in running and the mapping relation between the task and the task execution nodes which are monopolized currently.
Optionally, the system further comprises a task execution node releasing module, configured to release all task execution nodes exclusive to the target task if the target task is executed successfully or an abnormality occurs in the execution process.
Optionally, the method further includes a first exception handling module, configured to, if an exception occurs in a cluster manager of the Flink cluster during execution of the target task, clear a binding relationship between the target task and all the task execution nodes that are exclusive after the resource manager and the task coordinator in the Flink cluster lose connection after all the task execution nodes that are exclusive are released from the target task.
Optionally, the method further includes a second exception handling module, configured to register, if an exception occurs in a resource manager of the link cluster during the execution of the target task, with a new resource manager in the link cluster through a task execution node exclusive to the target task, where the new resource manager reconstructs a mapping relationship between the target task and the task execution node exclusive according to a slot report sent by the task execution node exclusive to the target task.
Optionally, the second exception handling module is further configured to: after the task execution node exclusive to the target task is successfully registered with a new resource manager in the Flink cluster, the task execution node exclusive to the target task actively establishes connection with the new resource manager, and invokes a method for registering slot information of a slot manager implementation class to register the slot information of the task execution node exclusive to the target task.
Optionally, the method further comprises a third exception handling module, configured to, if an exception occurs in a task execution node exclusive to the target task in the target task execution process, unbind, through a resource manager in the link cluster, a binding relationship with the task execution node having the exception.
According to yet another aspect of an embodiment of the present invention, an electronic device is provided.
An electronic device, comprising: one or more processors; and the memory is used for storing one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors are enabled to realize the task execution method provided by the embodiment of the invention.
According to yet another aspect of an embodiment of the present invention, a computer-readable medium is provided.
A computer readable medium having stored thereon a computer program which, when executed by a processor, implements a task execution method provided by an embodiment of the present invention.
One embodiment of the above invention has the following advantages or benefits: calculating the number of task execution nodes required to be exclusive for a target task according to the maximum occupied number of slots in the running process of the target task and the number of slots provided by each task execution node, wherein the slots are used for executing the task; if the number of task execution nodes which are required to be exclusive by the target task does not reach the number of task execution nodes which are required to be exclusive by the target task, searching task execution nodes which are required to be exclusive by the target task from task execution nodes which are not required to be exclusive by any task according to the difference value between the number of task execution nodes which are required to be exclusive by the target task and the number of task execution nodes which are required to be exclusive by the target task, and marking the task execution nodes which are required to be exclusive by the target task; and searching slots in all task execution nodes exclusive to the target task to execute the target task, wherein all task execution nodes exclusive to the target task comprise the task execution nodes exclusive to the target task currently and the task execution nodes exclusive to the target task. The method can realize the resource isolation at the level of the task manager during the multi-task mixed deployment, the task manager occupied by the task is exclusively shared, the task is prevented from being influenced mutually when the task is abnormal, and the user can specify the number of resources required by the task when the task is operated by taking the preset value of the number of slots as a reference by estimating the number of slots required by the task before the task is submitted to operate.
Further effects of the above-described non-conventional alternatives are described below in connection with the embodiments.
Drawings
The drawings are included to provide a better understanding of the invention and are not to be construed as unduly limiting the invention. Wherein:
FIG. 1 is a schematic diagram of the main steps of a task execution method according to one embodiment of the present invention;
FIG. 2 is a schematic diagram of a framework for task execution according to one embodiment of the invention;
FIG. 3 is a schematic diagram of the main modules of a task performing device according to one embodiment of the present invention;
FIG. 4 is an exemplary system architecture diagram in which embodiments of the present invention may be applied;
fig. 5 is a schematic diagram of a computer system suitable for use in implementing an embodiment of the invention.
Detailed Description
Exemplary embodiments of the present invention will now be described with reference to the accompanying drawings, in which various details of the embodiments of the present invention are included to facilitate understanding, and are to be considered merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the invention. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
Those skilled in the art will appreciate that embodiments of the invention may be implemented as a system, apparatus, device, method, or computer program product. Accordingly, the present disclosure may be embodied in the following forms, namely: complete hardware, complete software (including firmware, resident software, micro-code, etc.), or a combination of hardware and software.
FIG. 1 is a schematic diagram of the main steps of a task execution method according to one embodiment of the present invention. As shown in fig. 1, the task execution method according to an embodiment of the present invention mainly includes the following steps S101 to S103.
Step S101: calculating the number of task execution nodes required to be exclusive for a target task according to the maximum occupied number of slots in the running process of the target task and the number of slots provided by each task execution node, wherein the slots are used for executing the task;
the target task refers to a task to be currently executed.
Step S102: if the number of task execution nodes which are required to be exclusive by the target task does not reach the number of task execution nodes which are required to be exclusive by the target task, searching task execution nodes which are required to be exclusive by the target task from task execution nodes which are not required to be exclusive by any task according to the difference value between the number of task execution nodes which are required to be exclusive by the target task and the number of task execution nodes which are required to be exclusive by the target task, and marking the task execution nodes which are required to be exclusive by the target task;
Step S103: and searching slots in all task execution nodes exclusive to the target task to execute the target task, wherein all task execution nodes exclusive to the target task comprise the task execution nodes exclusive to the target task currently and the task execution nodes exclusive to the target task.
The target task of the embodiment of the invention is deployed in the Flink cluster.
The number of task execution nodes required to be exclusive for the target task is specifically a quotient obtained by dividing the maximum occupied slot number of the target task in running by the slot number provided by each task execution node.
Before calculating the number of task execution nodes required to be exclusive for the target task, the method comprises the following steps: calculating a maximum predicted value and a minimum predicted value of a slot position of a target task, wherein the maximum predicted value of the slot position is equal to the sum of the maximum parallelism in all slot position sharing groups in a stream execution area corresponding to the target task, and the stream execution area corresponding to the target task is a stream execution area to which a sub-task node in a data flow diagram of the target task belongs; the minimum predicted value of the slot is equal to the maximum value of the slot number required by each flow execution area corresponding to the target task; and determining the maximum occupied slot number of the target task in running according to a preset rule according to the maximum predicted slot number value and the minimum predicted slot number value of the target task and the initial slot number designated by the user.
The method for determining the maximum occupied slot number of the target task in operation according to the preset rule specifically comprises the following steps: if the initial number of slots designated by the user is smaller than the minimum preset value of slots, taking the minimum preset value of slots as the maximum number of occupied slots when the target task runs; if the initial number of slots designated by the user is larger than the maximum predicted value of the slots, taking the maximum predicted value of the slots as the maximum occupied number of slots when the target task operates; if the initial number of slots designated by the user is greater than or equal to the minimum predicted value of slots and less than or equal to the maximum predicted value of slots, the initial number of slots designated by the user is taken as the maximum number of occupied slots when the target task is running.
Searching task execution nodes which can be exclusive to a target task from task execution nodes which are not exclusive to any task, and marking the task execution nodes which can be exclusive to the target task, wherein the task execution nodes which can be exclusive to the target task specifically comprise: the resource manager in the Flink cluster searches for a task execution node which is not monopolized by any task from task execution nodes registered to the resource manager, marks the task execution node as monopolized by a target task, and registers the maximum occupied slot number of the task and the task running time in the resource manager and the mapping relation between the task and the task execution node which is monopolized currently.
And if the target task is successfully executed or an abnormality occurs in the execution process, releasing all task execution nodes which are exclusive to the target task.
In one embodiment, if an exception occurs in a cluster manager of the flank cluster during execution of the target task, after all task execution nodes monopolized by the target task are released, the resource manager in the flank cluster may clear a binding relationship between the target task and all task execution nodes monopolized after losing connection with the task coordinator.
In one embodiment, if an abnormality occurs in a resource manager of the Flink cluster during execution of the target task, a task execution node exclusive to the target task may register with a new resource manager in the Flink cluster, where the new resource manager rebuilds a mapping relationship between the target task and the task execution node exclusive according to a slot report sent by the task execution node exclusive to the target task.
After the task execution node exclusive to the target task is successfully registered with the new resource manager in the link cluster, the method may further include: and the task execution node exclusive to the target task actively establishes connection with a new resource manager, invokes a method for registering slot information of the slot manager implementation class, and registers the slot information of the task execution node exclusive to the target task.
In one embodiment, if an abnormality occurs in a task execution node exclusive to a target task in the target task execution process, a resource manager in the flank cluster may release a binding relationship with the task execution node having the abnormality.
The task execution flow of one embodiment of the invention comprises three parts of slot estimation, resource isolation and abnormal condition processing, and the task execution flow of the embodiment of the invention can realize the task manager level resource isolation of the whole cluster multi-task mixed deployment. FIG. 2 is a schematic diagram of a framework for task execution according to one embodiment of the invention. As shown in FIG. 2, jobMaster is a job coordinator, or task coordinator. The resource manager is responsible for slot allocation during task scheduling, and the resource manager establishes connection with a plurality of task coordinators. The task manager is a task execution node, and each task execution node can have a plurality of slots, namely slots, which are the minimum units for the management of the Flink resources. job is a task running on a flank cluster.
JobManager is a Flink cluster manager responsible for coordinating the operation of the entire cluster. Each application is controlled to be executed by a different JobManager, which receives the application to be executed first, the application including: a dataflow Graph (Job Graph), a logical dataflow Graph (ogical dataflow Graph), and a JAR package that packages all classes, libraries, and other resources. Jobmanager will transform Jobgraph into a data flow graph at a physical level, called an "execution graph" (Executiongraph), which contains all tasks that can be executed concurrently. JobManager requests the resources necessary to execute the task from the resource manager (resource manager), i.e., slots on the task execution node Taskmanager, and once enough resources are obtained, distributes the execution graph to the task execution node Taskmanager that actually runs them.
According to the embodiment of the invention, a task coordinator can be configured according to a slot sharing group (slot sharing group) and a position constraint (co-location constraint) of a SubTask node (jobVetex) in a data flow graph (jobGraph) of a task, and a flow execution area (pipeline region) to which the jobVertex belongs, so as to calculate the slot number slot maximum estimated value required by realizing optimal performance when the jobGraph runs, and the slot number slot minimum estimated value meeting simultaneous running of all the executions in the maximum region, wherein one area region comprises a plurality of executions, and in a life cycle of an execution graph, one executionVertex can be executed (restarted) multiple times, each time is called one executionVertex, a node included in the execution graph is called one JonVertex corresponding to one instance of the jobUgGraph, and one instance of the execution graph can be executed corresponding to one more than one instance of JobUgGraph, namely, and one instance of the execution graph can be executed in parallel. The maximum slot position predicted value slotMaxEstimate can be calculated specifically by the following method: and calculating the sum of the maximum parallelism in all slotsharinggroups in the region, namely, the maximum predicted value of the slot position. The minimum slot position estimated value slotMinEstimate can be calculated specifically by the following method: the number of slots required for all regions is calculated first, and then the maximum value of the number of slots is found.
After estimated to obtain slotmaxestite and slotmaxestite, the task coordinator Jobmaster is combined with the initial slot number designated by the user to obtain the final maximum occupied slot number when the job runs, specifically, the combination means that the maximum occupied slot number when the target task runs is determined according to a preset rule, and the specific steps include: if the initial slot number specified by the user is less than the slot minimum estimate, taking the minimum slot predicted value as the maximum occupied slot number when the target task runs; if the initial slot number specified by the user is greater than the slot MaxEstimate, taking the maximum predicted value of the slot as the maximum occupied slot number when the target task runs; if the initial slot number specified by the user is less than or equal to the initial slot number specified by the user, the initial slot number specified by the user is taken as the maximum occupied slot number when the target task runs.
When the resource manager establishes a connection with the jobmaster, the jobId id (task identifier) and the maximum number of slots occupied by the job runtime are registered to the slot manager (i.e., slot manager).
The groove position estimation is completed through the process. The following describes the isolation of resources. When the job coordinator Jobmaster starts to schedule the jobs, a slot request (slot request) is sent to a resource manager, and the resource manager registers the maximum occupation slot number of the jobs and the job running time and the mapping relation of the job executing node TaskManager which the jobs and the job are exclusive to currently. When meeting the slot request, the resource manager firstly checks the maximum occupation slot quantity of the job execution node TaskManager of the current slot request, calculates the quantity of the job execution node TaskManager required to be exclusive according to the value, checks whether the quantity of the job execution node TaskManager required to be exclusive by the current job is enough, namely judges whether the quantity of the job execution node TaskManager required to be exclusive by the current job is reached, if not, searches the task Manager which is not exclusive by any job from the job execution nodes TaskManager registered to the resource manager resource Manager, and marks the task manager as being exclusive by the current job. And finally, finding a proper slot from the exclusive TaskManager of the current job by applying the configured slot allocation policy, and assigning the slot to the slot request so as to execute the current job in the slot. When the job ends normally or an exception occurs, the TaskManager exclusive to the job needs to be released.
The abnormal condition handling of the embodiment of the invention mainly comprises the conditions of JobManager (cluster manager) abnormality, resource manager abnormality, task manager abnormality, abnormal disunion and the like, and is respectively described below.
When the cluster manager JobManager is abnormal, the task execution node TaskManager releases all slots, when a new cluster manager JobManager leader (one cluster manager for actually executing task scheduling) is selected, the task is re-scheduled after acquiring the task state as running from a zookeeper (one distributed service framework), the scheduling process is the same as the scheduling process when the task runs for the first time, and after the resource manager and the task coordinator JobManager lose connection, the binding relationship between job and the task execution node TaskManager needs to be cleaned, in particular, the binding relationship between the job and the task execution node TaskManager can be cleaned in a method closed JobManager provided by Flink.
When the resource manager is abnormal, the task is not restarted, and after the task execution node task manager detects that the heartbeat of the resource manager is overtime, the task execution node task manager can search the leader of the resource manager again (namely, the resource manager actually responsible for slot allocation during task scheduling). When a new resource manager selects a leader, the task execution node TaskManager will re-register with the new resource manager. The new resource manager has no mapping relation between the jobs and the TaskManager, and needs to reconstruct the mapping relation, and the new resource manager can reconstruct the mapping relation between the jobs and the TaskManager according to slot report (slot report, in which the number of slots that can be provided by the current TaskManager of the report is recorded) sent by the task execution node TaskManager.
When abnormal disunion occurs to the task execution node TaskManager, the resource manager resource Manager detects that the heartbeat of the task execution node TaskManager is overtime, a method provided by the Flink resource manager resource Manager is called to close the connection disconnection, wherein an unregisterTaskManager method of the slotmanagel is called, binding of the job and the disunion TaskManager is released, the slotmaager is a slot manager, the component provided by the Flink is called, and the unregisterTaskManager method is used for canceling the registration of the TaskManager. The task may restart, where the restarting task uses availableslot in slotpool, where slotpool represents all slots provided by the task manager of the task execution node, and availableslot represents the remaining available slots, and if slots are insufficient, a new slot may be requested from the resource manager, and then the method occupytask manager forjob provided by the Flink is called to occupy the new task manager.
After the newly registered task executing node task manager is successfully registered at the resource manager, the task executing node task manager actively establishes connection with the resource manager, invokes a method sendslot, finally invokes a method register task manager for realizing a similar slot manager image, registers slot information, and the sendslot method is used for reporting the slot number which can be provided by the current task manager. If at this time there is a slot request (e.g., pendingSlotRequest) of a particular blob matching the slot, the TaskManager is marked as exclusive to that blob.
Fig. 3 is a schematic diagram of main modules of a task performing device according to an embodiment of the present invention.
As shown in fig. 3, a task execution device 300 according to an embodiment of the present invention mainly includes: the task execution node number calculation module 301, the task execution node marking module 302 and the task execution module 303.
The task execution node number calculation module 301 is configured to calculate, according to the maximum number of occupied slots when the target task runs and the number of slots provided by each task execution node, the number of task execution nodes that the target task needs to monopolize, where the slots are used to execute the task;
the task execution node marking module 302 is configured to, if the number of task execution nodes that are currently monopolized by the target task does not reach the number of task execution nodes that are required to be monopolized by the target task, find a task execution node that is monopolized by the target task from task execution nodes that are not monopolized by any task according to a difference value between the number of task execution nodes that are required to be monopolized by the target task and the number of task execution nodes that are currently monopolized by the target task, and mark the task execution node that is monopolized by the target task;
and the task execution module 303 is configured to find slots in all task execution nodes exclusive to the target task to execute the target task, where all task execution nodes exclusive to the target task include a task execution node that the target task has been currently exclusive and an exclusive task execution node.
The target tasks may be deployed in a flank cluster.
The task execution device 300 further includes a slot estimation module, configured to: calculating a maximum predicted value and a minimum predicted value of the slot position of a target task, wherein the maximum predicted value of the slot position is equal to the sum of the maximum parallelism in all slot position sharing groups in a stream execution area corresponding to the target task, and the stream execution area corresponding to the target task is a stream execution area to which a sub-task node in a data flow diagram of the target task belongs; the minimum predicted value of the slot is equal to the maximum value of the slot number required by each flow execution area corresponding to the target task; and determining the maximum occupied slot number of the target task in running according to a preset rule according to the maximum predicted slot number value and the minimum predicted slot number value of the target task and the initial slot number designated by the user.
The slot position estimating module comprises a slot position number calculating operator module and is used for: if the initial number of slots designated by the user is smaller than the minimum preset value of slots, taking the minimum preset value of slots as the maximum number of occupied slots when the target task runs; if the initial number of slots designated by the user is larger than the maximum predicted value of the slots, taking the maximum predicted value of the slots as the maximum occupied number of slots when the target task operates; if the initial number of slots designated by the user is greater than or equal to the minimum predicted value of slots and less than or equal to the maximum predicted value of slots, the initial number of slots designated by the user is taken as the maximum number of occupied slots when the target task is running.
The task execution node marking module 302 may be specifically configured to: and searching task execution nodes which are not monopolized by any task from task execution nodes registered to the resource manager through the resource manager in the Flink cluster, marking the task execution nodes as monopolized by a target task, and registering the maximum occupied slot number of the task and the task running in the resource manager and the mapping relation between the task and the task execution nodes which are monopolized currently.
The task execution device 300 may further include a task execution node releasing module, configured to release all task execution nodes exclusive to the target task if the target task is executed successfully or an exception occurs in the execution process.
In an embodiment, the task execution device 300 may further include a first exception handling module, configured to, if an exception occurs in a cluster manager of the Flink cluster during execution of the target task, clear a binding relationship between the target task and all task execution nodes that are exclusive after all task execution nodes that are exclusive to the target task are released and after connection between a resource manager and a task coordinator in the Flink cluster is lost.
In an embodiment, the task execution device 300 may further include a second exception handling module, configured to register, through a task execution node exclusive to the target task, with a new resource manager in the flank cluster if an exception occurs in a resource manager of the flank cluster during execution of the target task, where the new resource manager rebuilds a mapping relationship between the target task and the task execution node exclusive according to a slot report sent by the task execution node exclusive to the target task.
The second exception handling module is further to: after the task execution node exclusive to the target task is successfully registered with a new resource manager in the Flink cluster, the task execution node exclusive to the target task actively establishes connection with the new resource manager, and invokes a method for registering slot information of a slot manager implementation class to register the slot information of the task execution node exclusive to the target task.
In an embodiment, the task execution device 300 may further include a third exception handling module, configured to, if an exception occurs in a task execution node that is monopolized by the target task in the target task execution process, unbind, through a resource manager in the Flink cluster, a binding relationship with the task execution node that is in the exception.
Any two or three of the first exception handling module, the second exception handling module and the third exception handling module may be implemented by a unified exception handling module when they are in the same embodiment.
In addition, the specific implementation of the task execution device in the embodiment of the present invention has been described in detail in the task execution method described above, so the description is not repeated here.
Fig. 4 illustrates an exemplary system architecture 400 in which a task execution method or task execution device of an embodiment of the present invention may be applied.
As shown in fig. 4, the system architecture 400 may include terminal devices 401, 402, 403, a network 404, and a server 405. The network 404 is used as a medium to provide communication links between the terminal devices 401, 402, 403 and the server 405. The network 404 may include various connection types, such as wired, wireless communication links, or fiber optic cables, among others.
A user may interact with the server 405 via the network 404 using the terminal devices 401, 402, 403 to receive or send messages or the like. Various communication client applications, such as shopping class applications, web browser applications, search class applications, instant messaging tools, mailbox clients, social platform software, etc. (by way of example only) may be installed on the terminal devices 401, 402, 403.
The terminal devices 401, 402, 403 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smartphones, tablets, laptop and desktop computers, and the like.
The server 405 may be a server providing various services, such as a background management server (by way of example only) providing support for shopping-type websites browsed by users using the terminal devices 401, 402, 403. The background management server may analyze and process the received data such as the product information query request, and feedback the processing result (e.g., the target push information, the product information—only an example) to the terminal device.
It should be noted that, the task execution method provided in the embodiment of the present application is generally executed by the server 405, and accordingly, the task execution device is generally disposed in the server 405.
It should be understood that the number of terminal devices, networks and servers in fig. 4 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.
Referring now to FIG. 5, there is illustrated a schematic diagram of a computer system 500 suitable for use in implementing a terminal device or server in accordance with an embodiment of the present application. The terminal device or server shown in fig. 5 is only an example, and should not impose any limitation on the functions and scope of use of the embodiments of the present application.
As shown in fig. 5, the computer system 500 includes a Central Processing Unit (CPU) 501, which can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) 502 or a program loaded from a storage section 508 into a Random Access Memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the system 500 are also stored. The CPU 501, ROM 502, and RAM 503 are connected to each other through a bus 504. An input/output (I/O) interface 505 is also connected to bus 504.
The following components are connected to the I/O interface 505: an input section 506 including a keyboard, a mouse, and the like; an output portion 507 including a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker, and the like; a storage portion 508 including a hard disk and the like; and a communication section 509 including a network interface card such as a LAN card, a modem, or the like. The communication section 509 performs communication processing via a network such as the internet. The drive 510 is also connected to the I/O interface 505 as needed. A removable medium 511 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 510 as needed so that a computer program read therefrom is mounted into the storage section 508 as needed.
In particular, according to the disclosed embodiments of the application, the processes described above with reference to the main step schematic diagrams may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method shown in the main step schematic. In such an embodiment, the computer program may be downloaded and installed from a network via the communication portion 509, and/or installed from the removable media 511. The above-described functions defined in the system of the present application are performed when the computer program is executed by a Central Processing Unit (CPU) 501.
The computer readable medium shown in the present application may be a computer readable signal medium or a computer readable storage medium, or any combination of the two. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present application, however, the computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, with the computer-readable program code embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.
The primary step diagrams and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the main step diagrams or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or main step diagrams, and combinations of blocks in the block diagrams or main step diagrams, can be implemented by special purpose hardware-based systems that perform the specified functions or operations, or combinations of special purpose hardware and computer instructions.
The modules involved in the embodiments of the present application may be implemented in software or in hardware. The described modules may also be provided in a processor, for example, as: a processor comprises a task execution node number calculation module, a task execution node marking module and a task execution module. The names of these modules do not limit the module itself in some cases, for example, the task execution node number calculation module may also be described as "a module for calculating the number of task execution nodes that need to be exclusive for a target task according to the maximum occupied number of slots when the target task runs and the number of slots provided by each task execution node".
As another aspect, the present invention also provides a computer-readable medium that may be contained in the apparatus described in the above embodiments; or may be present alone without being fitted into the device. The computer readable medium carries one or more programs which, when executed by a device, cause the device to include: calculating the number of task execution nodes required to be exclusive for a target task according to the maximum occupied number of slots when the target task runs and the number of slots provided by each task execution node, wherein the slots are used for executing the task; if the number of task execution nodes which are required to be exclusive by the target task does not reach the number of task execution nodes which are required to be exclusive by the target task, searching task execution nodes which are required to be exclusive by the target task from task execution nodes which are not required to be exclusive by any task according to the difference value between the number of task execution nodes which are required to be exclusive by the target task and the number of task execution nodes which are required to be exclusive by the target task, and marking the task execution nodes which are required to be exclusive by the target task; and searching slots in all task execution nodes exclusive to the target task to execute the target task, wherein all task execution nodes exclusive to the target task comprise the task execution node which is exclusive to the target task currently and the task execution node which can be exclusive.
According to the technical scheme provided by the embodiment of the invention, the multi-task mixed deployment can realize the resource isolation at the level of the task manager, the task manager occupied by the task is exclusively shared, the task is prevented from being influenced mutually when the task is abnormal, and the user can specify the number of resources required by the task when the task is operated by taking the preset value of the number of slots as a reference by estimating the number of slots required by the task before the task is submitted to operate.
The above embodiments do not limit the scope of the present invention. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives can occur depending upon design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of the present invention.
Claims (20)
1. A method of performing a task, comprising:
calculating the number of task execution nodes required to be exclusive for a target task according to the maximum occupied number of slots when the target task runs and the number of slots provided by each task execution node, wherein the slots are used for executing the task;
if the number of task execution nodes which are required to be exclusive by the target task does not reach the number of task execution nodes which are required to be exclusive by the target task, searching task execution nodes which are required to be exclusive by the target task from task execution nodes which are not required to be exclusive by any task according to the difference value between the number of task execution nodes which are required to be exclusive by the target task and the number of task execution nodes which are required to be exclusive by the target task, and marking the task execution nodes which are required to be exclusive by the target task;
And searching slots in all task execution nodes exclusive to the target task to execute the target task, wherein all task execution nodes exclusive to the target task comprise the task execution node which is exclusive to the target task currently and the task execution node which can be exclusive.
2. The method of claim 1, wherein the target task is deployed in a Flink cluster;
before calculating the number of task execution nodes required to be exclusive for the target task, the method comprises the following steps:
calculating a maximum predicted value and a minimum predicted value of the slot position of the target task, wherein the maximum predicted value of the slot position is equal to the sum of the maximum parallelism in all slot position sharing groups in a stream execution area corresponding to the target task, and the stream execution area corresponding to the target task is a stream execution area to which a sub-task node in a data flow diagram of the target task belongs; the minimum predicted value of the slot is equal to the maximum value of the slot number required by each flow execution area corresponding to the target task;
and determining the maximum occupied slot number of the target task in operation according to a preset rule according to the maximum predicted slot number value and the minimum predicted slot number value of the target task and the initial slot number designated by a user.
3. The method according to claim 2, wherein determining the maximum number of occupied slots in the target task according to a preset rule includes:
if the initial number of slots designated by the user is smaller than the minimum preset value of slots, the minimum preset value of slots is used as the maximum number of occupied slots when the target task runs;
if the initial number of slots designated by the user is larger than the maximum predicted value of the slots, taking the maximum predicted value of the slots as the maximum number of occupied slots when the target task runs;
and if the initial number of slots designated by the user is greater than or equal to the minimum preset value of slots and less than or equal to the maximum preset value of slots, taking the initial number of slots designated by the user as the maximum number of occupied slots when the target task runs.
4. The method of claim 2, wherein the searching for a task execution node that is exclusive to the target task from task execution nodes that are not exclusive to any task and marking the task execution node that is exclusive to the target task comprises:
the resource manager in the Flink cluster searches for a task execution node which is not monopolized by any task from task execution nodes registered to the resource manager, marks the task execution node as monopolized by the target task, and registers the task, the maximum occupied slot number of the task in running and the mapping relation between the task and the task execution node which is monopolized by the task currently in the resource manager.
5. The method of claim 2, wherein if the target task is executed successfully or an exception occurs during execution, releasing all task execution nodes that are exclusive to the target task.
6. The method according to claim 5, wherein if an exception occurs in a cluster manager of the flank cluster during the execution of the target task, after releasing all task execution nodes that are exclusive to the target task, further comprising:
and after the resource manager and the task coordinator in the Flink cluster lose connection, cleaning the binding relation between the target task and all exclusive task execution nodes.
7. The method of claim 5, wherein if an exception occurs in a resource manager of the flank cluster during execution of the target task, the method further comprises:
and registering the task execution node exclusive to the target task with a new resource manager in the Flink cluster, and reconstructing a mapping relation between the target task and the exclusive task execution node according to a slot report sent by the task execution node exclusive to the target task by the new resource manager.
8. The method of claim 7, wherein after the task execution node exclusive to the target task successfully registers with a new resource manager in the flank cluster, the method further comprises:
And the task execution node exclusive to the target task actively establishes connection with the new resource manager, and invokes a method for registering slot information of a slot manager implementation class to register the slot information of the task execution node exclusive to the target task.
9. The method of claim 5, wherein if an exception occurs in a task execution node that is exclusive to the target task during execution of the target task, the method further comprises:
and the resource manager in the Flink cluster releases the binding relation with the abnormal task execution node.
10. A task execution device, characterized by comprising:
the task execution node number calculation module is used for calculating the number of task execution nodes required to be exclusive for a target task according to the maximum occupied number of slots in the running process of the target task and the number of slots provided by each task execution node, wherein the slots are used for executing the task;
the task execution node marking module is used for searching task execution nodes which can be monopolized by the target task from task execution nodes which are not monopolized by any task according to the difference value between the task execution node number which needs to be monopolized by the target task and the task execution node number which needs to be monopolized by the target task if the task execution node number which is not monopolized by the target task currently does not reach the task execution node number which needs to be monopolized by the target task, and marking the task execution nodes which can be monopolized by the target task;
And the task execution module is used for searching slots in all task execution nodes exclusive to the target task to execute the target task, wherein all task execution nodes exclusive to the target task comprise the task execution node which is exclusive to the target task currently and the task execution node which is exclusive to the target task.
11. The apparatus of claim 10, wherein the target task is deployed in a Flink cluster;
the device also comprises a slot position estimating module for:
calculating a maximum predicted value and a minimum predicted value of the slot position of the target task, wherein the maximum predicted value of the slot position is equal to the sum of the maximum parallelism in all slot position sharing groups in a stream execution area corresponding to the target task, and the stream execution area corresponding to the target task is a stream execution area to which a sub-task node in a data flow diagram of the target task belongs; the minimum predicted value of the slot is equal to the maximum value of the slot number required by each flow execution area corresponding to the target task;
and determining the maximum occupied slot number of the target task in operation according to a preset rule according to the maximum predicted slot number value and the minimum predicted slot number value of the target task and the initial slot number designated by a user.
12. The apparatus of claim 11, wherein the slot estimation module comprises a slot number calculation sub-module to:
if the initial number of slots designated by the user is smaller than the minimum preset value of slots, the minimum preset value of slots is used as the maximum number of occupied slots when the target task runs;
if the initial number of slots designated by the user is larger than the maximum predicted value of the slots, taking the maximum predicted value of the slots as the maximum number of occupied slots when the target task runs;
and if the initial number of slots designated by the user is greater than or equal to the minimum preset value of slots and less than or equal to the maximum preset value of slots, taking the initial number of slots designated by the user as the maximum number of occupied slots when the target task runs.
13. The apparatus of claim 11, wherein the task execution node marking module is further configured to:
and searching task execution nodes which are not monopolized by any task from task execution nodes registered to the resource manager through the resource manager in the Flink cluster, marking the task execution nodes as monopolized by the target task, wherein the resource manager is registered with the task, the maximum occupied slot number of the task in running and the mapping relation between the task and the task execution nodes which are monopolized currently.
14. The apparatus of claim 11, further comprising a task execution node release module configured to release all task execution nodes that are exclusive to the target task if the target task is executed successfully or an exception occurs during execution.
15. The apparatus of claim 14, further comprising a first exception handling module configured to clear a binding relationship between the target task and all task execution nodes that are exclusive after the task execution nodes that are exclusive to the target task are released after the resource manager and the task coordinator in the flank cluster lose connection if an exception occurs in a cluster manager of the flank cluster during execution of the target task.
16. The apparatus of claim 14, further comprising a second exception handling module configured to register, through a task execution node exclusive to the target task, with a new resource manager in the flank cluster if an exception occurs in a resource manager of the flank cluster during execution of the target task, where the new resource manager rebuilds a mapping relationship between the target task and the task execution node exclusive according to a slot report sent by the task execution node exclusive to the target task.
17. The apparatus of claim 16, wherein the second exception handling module is further to: after the task execution node exclusive to the target task is successfully registered with a new resource manager in the Flink cluster, the task execution node exclusive to the target task actively establishes connection with the new resource manager, and invokes a method for registering slot information of a slot manager implementation class to register the slot information of the task execution node exclusive to the target task.
18. The apparatus of claim 14, further comprising a third exception handling module configured to, if an exception occurs in a task execution node that is exclusive to the target task during execution of the target task, unbind, by a resource manager in the Flink cluster, a binding relationship with the task execution node that is abnormal.
19. An electronic device, comprising:
one or more processors;
a memory for storing one or more programs,
the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the method of any of claims 1-9.
20. A computer readable medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the method according to any of claims 1-9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210466320.0A CN117009063A (en) | 2022-04-29 | 2022-04-29 | Task execution method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210466320.0A CN117009063A (en) | 2022-04-29 | 2022-04-29 | Task execution method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117009063A true CN117009063A (en) | 2023-11-07 |
Family
ID=88574917
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210466320.0A Pending CN117009063A (en) | 2022-04-29 | 2022-04-29 | Task execution method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117009063A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117278424A (en) * | 2023-11-17 | 2023-12-22 | 广州市玄武无线科技股份有限公司 | Method, device, equipment and storage medium for deploying Flink cluster |
-
2022
- 2022-04-29 CN CN202210466320.0A patent/CN117009063A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117278424A (en) * | 2023-11-17 | 2023-12-22 | 广州市玄武无线科技股份有限公司 | Method, device, equipment and storage medium for deploying Flink cluster |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108737270B (en) | Resource management method and device for server cluster | |
| US20220075643A1 (en) | Unified resource management for containers and virtual machines | |
| US10713080B1 (en) | Request-based virtual machine memory transitioning in an on-demand network code execution system | |
| US10216545B2 (en) | Method and system for modeling and analyzing computing resource requirements of software applications in a shared and distributed computing environment | |
| US10394477B2 (en) | Method and system for memory allocation in a disaggregated memory architecture | |
| US8914805B2 (en) | Rescheduling workload in a hybrid computing environment | |
| US8595722B2 (en) | Preprovisioning virtual machines based on request frequency and current network configuration | |
| JP6618614B2 (en) | Distributed stream-based database trigger | |
| US8819683B2 (en) | Scalable distributed compute based on business rules | |
| US20130061220A1 (en) | Method for on-demand inter-cloud load provisioning for transient bursts of computing needs | |
| CN109117252B (en) | Method and system for task processing based on container and container cluster management system | |
| US20120054770A1 (en) | High throughput computing in a hybrid computing environment | |
| US20140282582A1 (en) | Detecting deployment conflicts in heterogenous environments | |
| JP2015011716A (en) | Task execution by idle resources in grid computing system | |
| CN108021435B (en) | Cloud computing task flow scheduling method with fault tolerance capability based on deadline | |
| CN110166507B (en) | Multi-resource scheduling method and device | |
| CN109218356B (en) | Method and apparatus for managing stateful applications on a server | |
| US11245748B1 (en) | Proxied nodes in a container orchestration environment for scalable resource allocation | |
| US20150100967A1 (en) | Resolving deployment conflicts in heterogeneous environments | |
| US20130305245A1 (en) | Methods for managing work load bursts and devices thereof | |
| US11748168B2 (en) | Flexible batch job scheduling in virtualization environments | |
| US8812578B2 (en) | Establishing future start times for jobs to be executed in a multi-cluster environment | |
| CN117009063A (en) | Task execution method and device | |
| RU2696299C2 (en) | Control when initiating elementary tasks on server platform | |
| Hung et al. | Task scheduling for optimizing recovery time in cloud computing |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |