WO2021036936A1 - Method and apparatus for allocating resources and tasks in distributed system, and system - Google Patents

Abstract

A method and apparatus for allocating resources and tasks in a distributed system, and a system. The method comprises: receiving a job to be executed in a distributed system (S2100); according to the resource-related information of each task type in the job and the resource upper limit of each worker node in the distributed system, predicting a resource requirement needing to be allocated to each task executed by the worker node (S2200); allocating each task to an appropriate worker node according to the predicted resource requirement (S2300); and in the process of executing the allocated task by the worker node, performing dynamic adjustment according to a resource use situation (S2400).

Description

Method, device and system for allocating resources and tasks in a distributed system

This disclosure requires the priority of a Chinese patent application filed with the Chinese Patent Office on August 23, 2019, the application number is 201910783327.3, and the application title is "Resources and Task Allocation Methods, Devices and Systems in Distributed Systems", all of which The content is incorporated into this disclosure by reference.

Technical field

The present disclosure relates to the field of distributed technology, and more specifically, to a method for allocating resources and tasks in a distributed system, a device for allocating resources and tasks in a distributed system, and a device for allocating resources and tasks in a distributed system And distributed systems.

Background technique

In the existing scheduling system, such as Hadoop Yarn, tasks are run inside the container. Therefore, you need to apply for the container first. When applying, you need to manually judge and specify the size of the container resource. The resource size is fixed and cannot be changed. However, the amount of resource usage when a task is running is variable, not a fixed value. Therefore, in order for the task to run safely, larger resources are often applied, which will cause a certain degree of waste of resources.

In order to avoid waste of resources, reasonable resource limits need to be set, but this is often more difficult. Especially in some machine learning tasks, the amount of data involved will be relatively large, and the internal process of the task will be more complicated. Usually, multiple machines are required to run, so it is necessary to divide several tasks to run, and the resources of each task How much is the limit, it is very difficult. Although Google's Borg will reclaim the allocated but unused resources, and use these resources to run some tasks with low resource quality requirements to achieve maximum utilization of resources, once these resources are used, all resources exceed the resource limit. All tasks will be killed directly.

Summary of the invention

An object of the present disclosure is to provide a new technical solution for the allocation of resources and tasks in a distributed system.

According to a first aspect of the present disclosure, there is provided a method for allocating resources and tasks in a distributed system, which includes: receiving a job for execution in the distributed system; The resource upper limit of each working node in the distributed system, predicting the resource demand for each task executed by the working node; assigning each task to a suitable working node according to the predicted resource demand; and, executing on the working node In the process of assigning tasks, dynamic adjustments are made to resource usage.

In a possible implementation of the first aspect, the task type includes a parameter server task and/or a training learning task in machine learning; and, the resource-related information includes the scale of processing data and processing content of the corresponding task type. At least one of.

In a possible implementation of the first aspect, the resource demand includes each resource type required by the task and the corresponding resource demand value; wherein, the resource demand value includes the peak demand value and the general demand value. At least one item.

In a possible implementation of the first aspect, the step of predicting the resource requirements of each task performed by the worker node includes: predicting the resource requirements of each task performed by the worker node according to rules and/or a machine learning model. The allocated resource requirements, and the method further includes: collecting actual resource usage of the working node when the task is executed, so as to obtain the rule and/or the machine learning model.

In a possible implementation of the first aspect, the allocating each task to a suitable worker node according to the predicted resource demand includes: obtaining the current resource usage, current task running status, and total resources of each worker node Maximum limit; using the preset allocation algorithm, according to the predicted resource demand, combined with the current resource usage of each working node, the current task running situation and the maximum limit of total resources, filtering from multiple working nodes in the distributed system A working node suitable for executing each task is selected, and each task is assigned to the selected working node.

In a possible implementation of the first aspect, the step of dynamically adjusting resource usage during the execution of the assigned task by the working node includes: monitoring the resource usage of the task; When the use exceeds the predicted resource demand value, it is determined whether the total current use of the certain resource exceeds the maximum limit of the total resource of the certain resource; when the total current use of the certain resource exceeds the certain resource When the total resources of this resource are at the maximum limit, dynamic adjustment is made according to the compressibility of the certain resource.

In a possible implementation of the first aspect, the step of dynamically adjusting according to the compressibility of the certain resource includes: searching for a resource demand value exceeding a predicted resource demand value for the certain resource in the working node The task is selected as a candidate task, and the candidate task is selected according to the processing priority and/or start time; according to the compressibility of the certain resource, the selected candidate task is dynamically adjusted.

In a possible implementation manner of the first aspect, the dynamic adjustment for the selected candidate task according to the compressibility of the certain resource includes: when the certain resource is a compressible resource Next, limit the resource usage of the candidate task for the certain resource.

In a possible implementation of the first aspect, the dynamic adjustment for the selected candidate task according to the compressibility of the certain resource includes: when the certain resource is an incompressible resource Next, determine whether the candidate task supports expansion; in the case that the candidate task supports expansion, determine whether there are other working nodes that can perform the candidate task; in the case where the other working nodes exist, Extracting the uncompleted part of the task among the candidate tasks; sending the extracted part of the task to the other working nodes.

In a possible implementation of the first aspect, the method further includes: when the candidate task does not support expansion, determining whether the candidate task supports freezing; In this case, the memory data of the candidate task is written into the disk of the working node.

In a possible implementation of the first aspect, the method further includes: in the absence of the other working nodes, determining whether the candidate task supports freezing; in the case that the candidate task supports freezing Next, write the memory data of the candidate task into the disk of the working node.

In a possible implementation of the first aspect, after writing the memory data of the candidate task to the disk of the working node, the method further includes: determining whether the candidate task supports migration; In the case that the candidate task supports migration, it is determined whether there are other working nodes that can execute the candidate task; and the memory data is sent to the other working nodes.

In a possible implementation of the first aspect, the method further includes: in the case that the candidate task does not support migration, in response to a set trigger event, obtaining the current resource usage of the candidate task ; Based on the current resource usage of the candidate task, continue to execute the candidate task by the working node.

In a possible implementation manner of the first aspect, the trigger event includes at least one of any one of the assigned tasks in the working node has been completed, and the released resource in the working node.

In a possible implementation of the first aspect, the method further includes: directly killing the candidate task when the candidate task does not support freezing.

In a possible implementation of the first aspect, after directly killing the candidate task, the method further includes: collecting resource usage of the candidate task sent by the working node; based on the Resource usage, expanding the resource demand of the candidate task, so as to allocate the candidate task to a suitable work node again according to the expanded resource demand.

According to the second aspect of the present disclosure, there is also provided an apparatus for allocating resources and tasks in a distributed system, which includes:

The job receiving unit is configured to receive jobs for execution in the distributed system;

The resource demand prediction unit is configured to predict the resource demand to be allocated for each task executed by the working node based on the resource-related information of each task type in the job and the resource upper limit of each working node in the distributed system;

The task allocation unit is configured to allocate each task to a suitable work node according to the predicted resource demand;

The resource scheduling unit is configured to dynamically adjust the resource usage in the process of executing the assigned task by the working node.

According to the third aspect of the present disclosure, there is also provided a device for allocating resources and tasks in a distributed system, which includes:

The memory is configured to store executable instructions;

The processor is configured to run the resource and task allocation device in the distributed system according to the control of the executable instruction to execute the method for allocating resources and tasks in the distributed system as described in the first aspect of the present disclosure .

According to the third aspect of the present disclosure, there is also provided a computer-readable storage medium on which a computer program is stored, and the computer program, when executed by a processor, implements the distributed system as described in the first aspect of the present disclosure How to allocate resources and tasks.

According to the fourth aspect of the present disclosure, there is also provided a distributed system, which includes:

Multiple devices configured to provide resources;

The device for allocating resources and tasks in a distributed system as described in the second aspect of the present disclosure or the device for allocating resources and tasks in a distributed system as described in the third aspect of the present disclosure.

According to the method, device, equipment and system of the embodiments of the present disclosure, on the one hand, it is not for business personnel to artificially determine the resources required by each task in the distributed system, but based on the resource-related information and distributed information of each task type. The upper limit of the resource of each working node in the system, using the system to predict the resource demand that each task performed by the working node needs to be allocated, which can effectively improve the efficiency and accuracy of resource calculation; on the other hand, it can be based on the predicted Resource requirements, assign each task to a suitable working node, and dynamically adjust the resource usage during the execution of the assigned task by the working node, so as to achieve efficient task allocation and resource scheduling, and improve task execution efficiency And resource utilization.

Description of the drawings

The drawings incorporated in the specification and constituting a part of the specification illustrate the embodiments of the present disclosure, and together with the description thereof, serve to explain the principle of the present disclosure.

FIG. 1 is a block diagram showing an example of a hardware configuration of a distributed system that can implement an embodiment of the present disclosure.

2 is a schematic flowchart of a method for allocating resources and tasks in a distributed system according to an embodiment of the present disclosure;

3 is a schematic flowchart of a method for allocating resources and tasks in a distributed system according to another embodiment of the present disclosure;

4 is a schematic flowchart of a method for allocating resources and tasks in a distributed system according to a third embodiment of the present disclosure;

Fig. 5 is a functional block diagram of an apparatus for allocating resources and tasks in a distributed system according to an embodiment of the present disclosure;

Fig. 6 is a functional block diagram of a device for allocating resources and tasks in a distributed system according to an embodiment of the present disclosure;

Fig. 7 is a block diagram of a distributed system according to an embodiment of the present disclosure;

Fig. 8 is a schematic flowchart of a method for allocating resources and tasks according to an example of the present disclosure.

detailed description

Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that unless specifically stated otherwise, the relative arrangement of components and steps, numerical expressions and numerical values set forth in these embodiments do not limit the scope of the present disclosure.

The following description of at least one exemplary embodiment is actually only illustrative, and in no way serves as any limitation to the present disclosure and its application or use.

The technologies, methods, and equipment known to those of ordinary skill in the relevant fields may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be regarded as part of the specification.

In all examples shown and discussed herein, any specific value should be interpreted as merely exemplary, rather than as a limitation. Therefore, other examples of the exemplary embodiment may have different values.

It should be noted that similar reference numerals and letters indicate similar items in the following drawings, and therefore, once an item is defined in one drawing, it does not need to be further discussed in subsequent drawings.

Figure 1 shows a block diagram of the hardware configuration that can implement the distributed system of this embodiment.

As shown in Fig. 1, the distributed system of this embodiment includes multiple servers 1000. Fig. 1 shows four servers 1000, namely server 1000A, server 1000B, server 1000C, and server 1000D.

In this embodiment, the number of servers 1000 in the distributed system can be determined according to actual scenarios, and there is no limitation here.

In this embodiment, these servers 1000 form a distributed system, and each server 1000 can be used as a resource and task allocation device in the distributed system.

In this embodiment, it can be any server 1000 with an execution (Executor) node in the distributed system submitting a job job for execution in the distributed system, or it can be a client connected to the distributed system to Any server 1000 in the distributed system submits a job job to be executed in the distributed system, and the resource prediction (ResourceGuess) node in the server 1000 is based on the resource-related information of each task type in the job and the distributed system The resource upper limit of each working node, predicting the resource demand to be allocated for each task executed by the working node, and the scheduling (Scheduler) node in the server 1000 according to the predicted resource demand, assigning each task to a suitable The working node, and in turn, the server 1000 having a worker node in the distributed system dynamically adjusts the resource usage in the process of executing the assigned task.

The server 1000 provides service points for processing, database, and communication facilities. The server 1000 may be an integral server or a distributed server that spans multiple computers or computer data centers. The server can be of various types, such as, but not limited to, a web server, a news server, a mail server, a message server, an advertisement server, a file server, an application server, an interactive server, a database server, or a proxy server. In some embodiments, each server may include hardware, software, or an embedded logic component or a combination of two or more such components configured to perform suitable functions supported or implemented by the server. For example, the server may be a blade server, a cloud server, etc., or may be a server group composed of multiple servers, and may include one or more of the foregoing types of servers, and so on.

In an embodiment, the server 1000 may be as shown in FIG. 1 and includes a processor 1100, a memory 1200, an interface device 1300, a communication device 1400, a display device 1500, and an input device 1600.

In this embodiment, the server 1000 may also include a speaker, a microphone, etc., which are not limited herein.

The processor 1100 may be a dedicated server processor, or may be a desktop processor or a mobile processor that meets performance requirements, and is not limited herein. The memory 1200 includes, for example, ROM (Read Only Memory), RAM (Random Access Memory), nonvolatile memory such as a hard disk, and the like. The interface device 1300 includes, for example, various bus interfaces, such as a serial bus interface (including a USB interface), a parallel bus interface, and the like. The communication device 1400 can perform wired or wireless communication. The display device 1500 is, for example, a liquid crystal display, an LED display touch screen, and the like. The input device 1600 may include, for example, a touch screen, a keyboard, and the like.

Although multiple devices of the server 1000 are shown in FIG. 1, the present disclosure may only relate to some of the devices. For example, the server 1000 only relates to the storage 1200, the communication device 1400, and the processor 1100.

The network 2000 may be a wireless communication network or a wired communication network, and may be a local area network or a wide area network. In the distributed system shown in FIG. 1, multiple servers 1000 can communicate through a network 2000. In addition, the network 2000 on which the communication between the multiple servers 1000 is based may be the same or different.

The distributed system shown in FIG. 1 is only for explanatory purposes, and is by no means intended to limit the present disclosure, its application or use. In actual applications, it may also include other numbers of distributed systems, for example, it may also include two distributions. Type system, 3 distributed systems, 5 distributed systems, or even more, there are no restrictions here. Applied to the embodiments of the present disclosure, the memory 1200 of the server 1000 is configured to store instructions, and the instructions are configured to control the processor 1100 to operate to execute any one of the resources and tasks in the distributed system provided in the embodiments of the present disclosure. Method of distribution. Technicians can design instructions according to the solutions disclosed in this disclosure. How the instruction controls the processor to operate is well known in the art, so it will not be described in detail here.

Fig. 2 is a schematic flowchart of a method for allocating resources and tasks in a distributed system according to an embodiment.

Referring to FIG. 2, the method for allocating resources and tasks in a distributed system of this embodiment can be implemented by a device for allocating resources and tasks in a distributed system, or it can be implemented by resources and tasks in a distributed system. Implementation of the allocation device, the device for allocating resources and tasks in a distributed system or the device for allocating resources and tasks in a distributed system may be specifically distributed on the device that provides resources. The resource scheduling method of this embodiment may include the following steps S2100～S2400:

Step S2100: Receive a job for execution in the distributed system.

A job job is the basic unit for submitting tasks. A job job includes multiple tasks, and the multiple tasks are related to each other.

Task task is the smallest unit of task operation. Under normal circumstances, a process can be called a task task.

After receiving the job for execution in the distributed system through step S2100, it can be combined with subsequent steps to predict each job executed by the working node based on the resource-related information of each task type in the job and the resource upper limit of each working node in the distributed system. Tasks need to be allocated resource requirements, and according to the predicted resource requirements, each task is allocated to a suitable working node, so that the working node provides resources to the corresponding task according to the predicted resource demand, thereby improving resource utilization.

After receiving the job for execution in the distributed system, enter:

In step S2200, according to the resource-related information of each task type in the job and the resource upper limit of each working node in the distributed system, predict the resource demand to be allocated for each task executed by the working node.

Task types include parameter server tasks and/or training learning tasks in machine learning, where the parameter server tasks are tasks for parameter processing (for example, parameter updates), and the training learning tasks are for model training (for example, , Sample calculation) task.

The resource-related information includes at least one of the processing data scale and processing content of the corresponding task type.

In this embodiment, for any task, the required resources may often be multiple. Here, in order to distinguish different resources, the resource requirements include at least each type of resource required by the task and the corresponding resource demand value. Of course, the resource requirements may also include other information about the resources required by the task, which is not limited here.

The resource type may include, for example, CPU, memory usage, disk usage, disk input/output I/O (Input/Output), network I/O, graphics processing unit (GPU), and field programmable gate array (Field Programmable Gate Array). -Programmable Gate Array, FPGA).

The resource demand value includes at least one of a peak demand value and a general demand value. In this embodiment, on the one hand, the predicted resource demand value may be greater than the actual use value, resulting in a waste of resources; on the other hand, the predicted resource demand value may also be less than the actual use value, resulting in insufficient resources. In this case, according to the following step S2400, during the process of executing the assigned tasks on the working node, the resource usage can be dynamically adjusted to improve the resource utilization. This step S2200 will not be described in detail here.

In this embodiment, using the resource-related information of each task type in the job and the resource upper limit of each working node in the distributed system, it may divide a certain task in the submitted work into multiple tasks, and further predict Each task performed by the work node needs to be allocated the resource type and the resource demand value of the corresponding type, and then the task is allocated to the appropriate work node, so that the work node provides the corresponding resource to perform the task.

In this embodiment, predicting the resource requirements to be allocated for each task executed by the working node in step S2200 may further include:

According to rules and/or machine learning models, the resource requirements that need to be allocated for each task executed by the worker nodes are predicted.

The machine learning model can be a neural network model, such as but not limited to a BP (Back Propagation) neural network model, a convolutional neural network model, etc. Of course, the machine learning model can also be a logistic regression model. This is not a machine learning model. Specific limitations are made, as long as any machine learning model that can predict the resource requirements that need to be allocated for each task executed by the working node belongs to the content protected by the embodiments of the present disclosure.

In this embodiment, the data scale and task type actually involved in the task may be used as features, and each resource actually corresponding to the task and its resource usage value, etc., are used as markers to form a training sample and input to the machine that performs resource demand prediction. Among the learning models, the data scale may include, for example, at least one of the number of rows of data and the number of columns of data, and the machine learning model can be trained to predict the task corresponding to the task to be predicted based on the data scale and task type. Each resource and its resource demand value.

In this embodiment, it is also possible to obtain the actual resource usage of the task as a new training sample after the worker node finishes executing the assigned task, and modify the rules and/or machine learning model to make the resource demand forecast more and more accurate.

Through step S2200, according to the resource-related information of each task type in the job and the resource upper limit of each working node in the distributed system, the resource demand for each task to be executed by the working node is predicted, and according to the predicted resource demand, each Each task is allocated to a suitable working node, so that the working node provides resources to the corresponding task according to the predicted resource demand, thereby improving resource utilization.

After predicting the resource requirements to be allocated for each task executed by the working node according to the resource-related information of each task type in the job and the resource upper limit of each working node in the distributed system, step S2300 is entered.

In step S2300, each task is allocated to a suitable working node according to the predicted resource demand.

The tasks may include multiple tasks included in the work when the work is submitted, or multiple tasks obtained by dividing a certain task according to step S2200 after the work is submitted.

In this embodiment, the predicted resource demand may be sent to the working node to control the working node to provide resources to the task according to the predicted resource demand, or it may be the working node after receiving the predicted resource information, according to the prediction. Resource requirements, provide resources to tasks.

In this embodiment, the step S2300 assigning each task to a suitable working node according to the predicted resource demand may further include the following steps S2310 to S2320:

Step S2310: Obtain the current resource usage status, current task running status, and total resource maximum limit of each working node.

Step S2320, using the preset allocation algorithm, according to the predicted resource demand, combined with the current resource usage of each working node, the current task running situation and the maximum limit of total resources, to filter from multiple working nodes in the distributed system Work nodes suitable for performing each task are selected, and each task is assigned to the selected work nodes.

In the embodiments of the present disclosure, any allocation algorithm may be used for task allocation, so there is no limitation here.

In this step S2320, only one task can be allocated to one working node, or multiple tasks can be allocated to one working node. The multiple tasks can be executed at the same time, or it can be based on the predicted resource demand value. The execution order of multiple tasks in one working node is not limited to the execution order of multiple tasks in one working node.

According to step S2300, the resource demand for each task to be allocated to be executed by the working node is predicted, and each task is allocated to a suitable working node according to the predicted resource demand, so that the working node will forward the corresponding resource demand to the corresponding working node according to the predicted resource demand. Tasks provide resources, thereby improving resource utilization.

According to the predicted resource demand, after each task is allocated to a suitable working node, step S2400 is entered.

Step S2400, during the execution of the assigned task by the working node, dynamic adjustment is made according to the resource usage.

In this embodiment, the resource demand value predicted according to step S2200 may be greater than the actual use value, resulting in waste of resources; or the predicted resource demand value may also be less than the actual use value, resulting in insufficient resources. In this case, according to this step S2400, in the process of executing the assigned tasks by the working node, dynamic adjustments can be made to the resource usage to improve the resource usage.

In this embodiment, referring to FIG. 3, the step S2400 in the process of executing the assigned task by the working node, dynamically adjusting the resource usage may further include the following steps S2410 to S2440:

Step S2410: monitor the resource usage of the task, and determine whether a certain resource usage of the task exceeds the predicted resource demand value, if yes, execute step S2430, if not, execute step S2420.

According to this step S2410, the resource usage of the task can be monitored in real time after the working node starts to execute the assigned task.

Taking resources including CPU and GPU as an example, it can be used to monitor the CPU and GPU usage of tasks in the worker node in real time during the execution of the assigned tasks of the worker node, and determine whether the task's use of the CPU or GPU exceeds the forecast If the task’s use of the CPU or GPU does not exceed the task’s predicted CPU or GPU resource demand value, no processing will be performed according to step S2420, and the task will continue to be executed by the worker node. If the usage of the CPU or GPU exceeds the predicted resource demand value of the CPU or GPU, it is further determined according to step S2430 whether the total current usage of the CPU or GPU exceeds the maximum limit of the total resource of the CPU or GPU.

In step S2420, no processing is performed, and the task continues to be executed by the working node.

Continuing with the example of step S2410, if the task's use of the CPU or GPU does not exceed the task’s predicted resource demand value of the CPU or GPU, no processing is performed according to this step S2420, and the task continues to be executed by the worker node, and only the scheduling node is notified The actual CPU or GPU resource usage.

In step S2430, in the case that a certain resource usage of the task exceeds the predicted resource demand value, it is judged whether the current total usage of a certain resource exceeds the total resource maximum limit of a certain resource.

Continuing the example of step S2410 above, if the task's use of the CPU exceeds the predicted resource demand value of the CPU, then according to this step S2430, it is further determined whether the total current usage of the CPU exceeds the maximum limit of the total resources of the CPU, if it exceeds the total resources of the CPU Maximum limit, execute step S2440 to dynamically adjust according to the compressibility of the CPU. If the maximum limit of the total resources of the CPU is not exceeded, no processing will be performed according to step S2450, and the task will continue to be executed by the working node, and only the scheduling node will be notified of the actual CPU resource usage.

In step S2440, when the total current usage of a certain resource does not exceed the maximum total resource limit of a certain resource, no processing is performed, and the task continues to be executed by the working node.

Continuing the example of step S2430 above, if the maximum limit of the total resources of the CPU is not exceeded, no processing is performed according to this step S2440, the task is continued to be executed by the working node, and only the scheduling node is notified of the actual CPU resource usage.

Step S2450, in the case that the total current usage of a certain resource exceeds the maximum limit of the total resource of a certain resource, dynamic adjustment is made according to the compressibility of the certain resource.

According to the compressibility of resources, resources can be divided into compressible resources and incompressible resources. Among them, compressible resources include CPU, disk I/O, and network I/O; and incompressible resources include memory usage, disk usage, and GPU. And FPGA.

Continuing the example of step S2430 above, if the maximum limit of the total resources of the CPU is exceeded, this step S2450 is executed to dynamically adjust according to the compressibility of the CPU, thereby realizing efficient resource scheduling and improving resource utilization.

According to the method of the embodiment of the present disclosure, on the one hand, it is not for the user to artificially determine the resources required by each task in the distributed system, but according to the resource-related information of each task type and the resource upper limit of the working node in the distributed system. , Using the system to predict the resource requirements that need to be allocated for each task executed by the working node, which can effectively improve the efficiency and accuracy of resource calculation; on the other hand, it can allocate each task according to the predicted resource demand Give suitable working nodes and dynamically adjust resource usage during the execution of assigned tasks by the working nodes, thereby realizing efficient task allocation and resource scheduling, and improving task execution efficiency and resource utilization.

In an embodiment, the dynamic adjustment according to the compressibility of a certain resource in the above step S2450 may further include the following steps:

Step S2451: Search for tasks in the working node whose resources exceed the predicted resource demand value as candidate tasks, and select the candidate tasks according to the processing priority and/or start time.

In this step S2451, for example, when the total current usage of a certain resource exceeds the maximum limit of the total resource of a certain resource, first search for tasks in the working node that exceed the predicted resource demand value for a certain resource as candidates. Tasks, and then select the candidate task with a low processing priority as the selected candidate task according to the ascending order of the processing priority of the candidate task. It can also be the case where there are multiple candidate tasks with the lowest priority. Successively, the candidate task with the longest starting time is selected as the selected candidate task according to the ascending order of the starting time of the multiple candidate tasks with the lowest priority.

Continuing the example of step S2450 above, if the maximum limit of the total resources of the CPU is exceeded, the task that exceeds the predicted resource demand value for the CPU in the working node is first searched as a candidate task. Illustratively, the searched candidate task may be, for example, It is task 1, task 2, and task 3. Here, it can be obtained after sorting task 1, task 2, and task 3 in order of processing priority: task 3, task 2, task 1, and select the processing priority Task 3 with the lowest level is selected as the candidate task; in addition, for example, if the processing priority of task 3 and task 2 are the same, then task 3 and task 2 are successively sorted from smallest to largest start time To get: task 2, task 3, and select task 3 with the longest startup time as the selected candidate task.

Step S2452, according to the compressibility of a certain resource, dynamically adjust the selected candidate task.

In this step S2452, after the candidate task is selected according to the above step S2451, the selected candidate task can be dynamically adjusted according to the compressibility of a certain resource to improve resource utilization.

In an example of the present disclosure, the step S2452 dynamically adjusting for the selected candidate task according to the compressibility of a certain resource may further include:

In the case that a certain resource is a compressible resource, the resource usage of the candidate task for a certain resource is restricted.

Continuing the example of step S2451 above, since the CPU is a compressible resource, here, the resource usage of the selected candidate task, that is, task 3, for the CPU may be restricted.

In an example of the present disclosure, referring to FIG. 4, the step S2452 dynamically adjusts the selected candidate task according to the compressibility of a certain resource may further include:

In step S2452-1, when a certain resource is an incompressible resource, it is judged whether the candidate task supports capacity expansion, if so, step S2452-2 is executed, otherwise, step S2452-5 is executed.

Exemplarily, taking the resource as FPGA and the selected candidate task is still task 3 as an example, since FPGA is an incompressible resource, here, it can be judged whether task 3 supports expansion. If task 3 supports expansion, then According to the following step S2452-2, it is judged whether there are other working nodes that can execute task 3, otherwise, according to the following step S2452-5, it is judged whether task 3 supports freezing.

In step S2452-2, it is judged whether there are other working nodes that can perform the candidate task. If so, step S2452-3 is performed, otherwise, step S2452-5 is performed.

Continuing the example of step S2452-1 above, in the case that task 3 supports capacity expansion, further determine whether there are other working nodes that can perform task 3, and if there are other working nodes that can perform task 3, perform the following step S2452-3 extraction For some uncompleted tasks in task 3, on the contrary, step S2452-5 is executed to determine whether the candidate tasks support freezing.

In step S2452-3, when there are other working nodes, extract the uncompleted part of the tasks among the candidate tasks.

Continuing the example of step S2452-2 above, if there are other working nodes that can execute task 3, extract the uncompleted part of tasks in task 3, and continue to execute step S2452-4.

Step S2452-4, sending some of the extracted tasks to other working nodes.

Continuing the example of step S2452-3 above, after extracting the unfinished part of task 3, the extracted part of the task can be sent to other working nodes according to this step S2452-4, so that other working nodes can continue to execute the part task.

In step S2452-5, it is judged whether the candidate task supports freezing, if so, step S2452-6 is executed, otherwise, step S2452-12 is executed.

Continuing the example of step S2452-1 or step S2452-2 above, in the case that task 3 does not support capacity expansion, or there is no other working node that can perform alternative tasks, it is further judged whether task 3 supports freezing, if If task 3 supports freezing, execute step S2452-6 to freeze task 3, otherwise, execute step S2452-12.

In step S2452-6, when the candidate task supports freezing, the memory data of the candidate task is written into the disk of the working node.

Continuing the example of step S2452-5 above, in the case that task 3 supports freezing, task 3 is frozen, that is, the memory data of task 3 is written to the disk of the working node, which can be in writing the memory data of task 3 to the working node. After the node is in the disk, continue to perform step S2452-7 to determine whether task 3 supports migration.

In step S2452-7, it is judged whether the candidate task supports migration, if so, step S2452-8 is executed, otherwise, step S2452-10 is executed.

Continuing the example of step S2452-6, in the case that task 3 supports migration, according to the following step S2452-8, it is further judged whether there are other working nodes that can execute the candidate task, and vice versa, step S2452-10 waits for recovery.

In step S2452-8, if the candidate task supports migration, it is determined whether there are other working nodes that can execute the candidate task. If so, step S2452-9 is executed, otherwise, S2452-5 is executed.

Continuing the example of step S2452-7 above, in the case that task 3 supports migration, it is further determined whether there are other working nodes that can execute task 3, and if there are other working nodes that can execute task 3, step S2452-9 is executed to save the memory The data is sent to other working nodes, otherwise, step S2452-5 is executed to determine whether task 3 supports freezing.

In step S2452-9, the memory data is sent to other working nodes.

Continuing the example of step S2452-8 above, if there are other working nodes that can perform task 3, execute this step S2452-9 to send the memory data to other working nodes.

Step S2452-10, in response to the set trigger event, obtain the current resource usage of the candidate task.

The set trigger event includes at least one of any assigned task in the working node has been completed, and at least one of the released resources in the working node.

Continuing the example of step S2452-8 above, in the case that task 3 does not support migration, perform the waiting recovery of step S2452-10, for example, it can be that any task assigned in the working node has been completed, and there is a node in the working node. When at least one of the GPUs is released, the current GPU usage of task 3 is acquired.

Step S2452-11, based on the current resource usage of the candidate task, continue to execute the candidate task by the working node.

Continuing the example of step S2452-10 above, based on the current GPU usage of task 3, task 3 will continue to be executed by the worker node.

Step S2452-12, directly kill the candidate task.

Continuing the example of S2452-5 above, if task 3 does not support freezing, execute this step S2452-12 to directly kill task 3, and continue to execute step S2452-13.

Step S2452-13: Collect the resource usage status of the candidate tasks sent by the working node.

Continuing the example of S2452-12 above, after killing task 3, the GPU usage of task 3 sent by the worker node can be collected, and new resource requirements can be derived automatically.

In step S2452-14, based on the resource usage, the resource requirements of the candidate tasks are expanded, so as to allocate the candidate tasks to suitable working nodes again according to the expanded resource requirements.

Continuing the example of S2452-13 above, after collecting the GPU usage of task 3 sent by the worker node, the resource demand of task 3 can be expanded according to this step S2452-14, so that task 3 can be allocated again according to the expanded resource demand Give a suitable working node, and then perform task 3 again by the suitable working node.

According to this embodiment, when the total current usage of a certain resource exceeds the maximum limit of the total resource of a certain resource, it can dynamically adjust according to the compressibility of a certain resource, thereby improving task processing efficiency and resources. Utilization rate.

In this embodiment, an apparatus 5000 for allocating resources and tasks in a distributed system is also provided. As shown in FIG. 5, it includes a job receiving unit 5100, a resource demand prediction unit 5200, a task allocation unit 5300, and a resource scheduling unit. 5400.

The job receiving unit 5100 is configured to receive jobs for execution in a distributed system.

The resource demand prediction unit 5200 is configured to predict the resource demand to be allocated for each task executed by the working node based on the resource-related information of each task type in the job and the resource upper limit of each working node in the distributed system.

The task allocation unit 5300 is configured to allocate each task to a suitable working node according to the predicted resource demand.

The resource scheduling unit 5400 is configured to dynamically adjust the resource usage in the process of executing the assigned task by the working node.

In an embodiment, the task types include parameter server tasks and/or training learning tasks in machine learning; and,

The resource-related information includes at least one of the processing data scale and processing content of the corresponding task type.

In one embodiment, the resource requirement includes each resource type and corresponding resource requirement value required by the task;

Wherein, the resource demand value includes at least one of a peak demand value and a general demand value.

In one embodiment, the resource demand prediction unit 5200 is further configured to predict the resource demand to be allocated for each task performed by the worker node according to rules and/or machine learning models; and,

Collect the actual resource usage of the working node when performing tasks to obtain the rules and/or machine learning models.

In one embodiment, the task allocation unit 5300 is further configured to obtain the current resource usage status, current task running status, and total resource maximum limit of each working node;

Using the preset allocation algorithm, according to the predicted resource demand, combined with the current resource usage of each working node, the current task running situation and the maximum limit of total resources, the multiple working nodes of the distributed system are selected to be suitable for execution. The working node of each task is assigned, and each task is assigned to the selected working node.

In an embodiment, the resource scheduling unit 5400 is also configured to monitor the resource usage of the task;

In the case where a certain resource usage of the task exceeds the predicted resource demand value, judging whether the total current usage of the certain resource exceeds the maximum total resource limit of the certain resource;

In the case that the total current usage of the certain resource exceeds the maximum limit of the total resource of the certain resource, dynamic adjustment is made according to the compressibility of the certain resource.

In an embodiment, the resource scheduling unit 5400 is further configured to search for tasks in the working node that exceed the predicted resource demand value for the certain resource as candidate tasks, and start the task according to the processing priority and/or Time to choose alternative tasks;

According to the compressibility of the certain resource, the selected candidate task is dynamically adjusted.

In an embodiment, the resource scheduling unit 5400 is further configured to limit the resource usage of the candidate task for the certain resource when the certain resource is a compressible resource.

In an embodiment, the resource scheduling unit 5400 is further configured to determine whether the candidate task supports expansion when the certain resource is an incompressible resource;

In the case that the candidate task supports capacity expansion, determining whether there are other working nodes that can execute the candidate task;

In the case where the other working nodes exist, extract the uncompleted part of the task among the candidate tasks;

Send the extracted part of the task to the other working nodes.

In an embodiment, the resource scheduling unit 5400 is further configured to determine whether the candidate task supports freezing when the candidate task does not support capacity expansion;

In the case that the candidate task supports freezing, the memory data of the candidate task is written into the disk of the working node.

In an embodiment, the resource scheduling unit 5400 is further configured to determine whether the candidate task supports freezing when the other working node does not exist;

In the case that the candidate task supports freezing, the memory data of the candidate task is written into the disk of the working node.

In an embodiment, the resource scheduling unit 5400 is further configured to determine whether the candidate task supports migration;

In the case that the candidate task supports migration, determining whether there are other working nodes that can execute the candidate task;

Sending the memory data to the other working nodes.

In one embodiment, the resource scheduling unit 5400 is further configured to obtain the current resource usage status of the candidate task in response to a set trigger event when the candidate task does not support migration;

Based on the current resource usage of the candidate task, continue to execute the candidate task by the working node.

In an embodiment, the trigger event includes at least one of any one of the assigned tasks in the working node has been completed, and the resource that has been released in the working node.

In an embodiment, the resource scheduling unit 5400 is further configured to directly kill the candidate task when the candidate task does not support freezing.

In an embodiment, the resource scheduling unit 5400 is further configured to collect resource usage of the candidate task sent by the working node;

Based on the resource usage, the resource requirements of the candidate tasks are expanded, so as to allocate the candidate tasks to suitable working nodes again according to the expanded resource requirements.

In this embodiment, a device 6000 for allocating resources and tasks in a distributed system is also provided, as shown in FIG. 6, including:

The memory 6100 is configured to store executable instructions;

The processor 6200 is configured to execute the resource and task allocation device in the distributed system according to the control of the executable instruction to execute the resource and task allocation method in the distributed system as provided in this embodiment.

In this embodiment, the resource and task allocation device 6000 in a distributed system may be a server. For example, the resource and task allocation device 6000 in a distributed system may be the server 1000 as shown in FIG. 1.

The resource and task allocation equipment 6000 in a distributed system may also include other devices, for example, the server 1000 shown in FIG. 1 may also include an input device, a communication device, an interface device, and a display device.

In this embodiment, a computer-readable storage medium is also provided, on which a computer program is stored. When the computer program is executed by a processor, the method for allocating tasks and resources in a distributed system as in any embodiment of the present disclosure is implemented. .

In this embodiment, a distributed system 7000 is also provided, as shown in FIG. 7, including:

A plurality of devices 7100 configured to provide resources may be, for example, a device 7100A configured to provide resources and a device 7100B configured to provide resources. The plurality of devices 7100 configured to provide resources may be in a server cluster. Devices can also be devices that belong to different server clusters.

In this embodiment, the number of devices 7100 configured to provide resources can be determined according to actual scenarios, and there is no limitation here.

In this embodiment, the distributed system 7000 further includes a device 5000 for allocating resources and tasks in the distributed system or a device 6000 for allocating resources and tasks in the distributed system. The device 5000 for allocating resources and tasks in a distributed system or the device 6000 for allocating resources and tasks in a distributed system may be distributed on a device 7100 that provides resources.

The distributed system 7000 is not only applicable to machine learning scenarios, but also applicable to other non-machine learning scenarios that do not impose strict resource restrictions on tasks. The machine learning scenario is: due to the complexity of the system or some unknown reasons for the data, it is difficult to make accurate judgments on the specific resource usage of the task and give correct results. For example, feature processing tasks, offline machine learning training tasks, and online machine learning prediction tasks. The non-machine learning scenario is: some online services generally have some peak periods, such as a take-out system, there will be a large peak during lunch time, the system will require a lot of resources, but in the middle of the night, the usage will be very small , Using the system 7000, some resources will be reclaimed for use by other tasks during low peak periods.

The following will further illustrate the resource and task allocation method implemented by the distributed system 7000 provided in this embodiment.

In this example, the distributed system 7000 includes a plurality of devices 7100 configured to provide resources and a device 5000 for allocating resources and tasks in the distributed system. The device 7100 may have execution nodes, resource prediction nodes, scheduling nodes, and The server of the working node, the device 5000 for allocating resources and tasks in the distributed system can be distributed on multiple devices 7100, for example, the job receiving in the device 5000 for allocating resources and tasks in the distributed system can be realized through the execution node The function corresponding to the unit 5100 and the function corresponding to the resource prediction demand unit 5200 in the device 5000 for allocating resources and tasks in a distributed system through the resource prediction node, and the device 5000 for allocating resources and tasks in the distributed system through the scheduling node. The function corresponding to the task allocation unit 5300 in the medium, and the function corresponding to the resource scheduling unit 5400 in the resource and task allocation device 5000 in a distributed system is realized through the working node. As shown in FIG. 8, in this example, the resource and task allocation Distribution methods can include:

Step S8010, the execution node submits the job executed in the distributed system to the resource prediction node.

The job job can include task task1, task2, and task3.

In step S8020, the resource prediction node predicts the resource requirements to be allocated for each task executed by the working node based on the resource-related information of each task type in the job and the resource upper limit of each working node in the distributed system, and then calculates the predicted resource The demand returns to the execution node.

In step S8030, the execution node submits the tasks task1, task2, and task3 included in the job job, and the predicted resource requirements for each task executed by the worker node to be allocated to the scheduling node.

In step S8040, the scheduling node assigns task task1 to the work node 1 according to the predicted resource demand; and assigns the task task2 and the task task3 to the work node 2.

In this step S8040, a preset allocation algorithm can be used, according to the predicted resource demand, combined with the current resource usage of each working node, the current task running situation, and the maximum limit of total resources, from multiple distributed systems. The work nodes that are suitable for executing tasks task1, task2, and task3 are selected from the work nodes, and task task1 is assigned to the selected work node 1, and tasks task2 and task3 are assigned to the selected work node 2.

In step S8050, the worker node 1 receives task task1, starts task task1, and monitors the resource usage of task1 in real time; and, after worker node 2 receives task task2 and task task3, starts task2 and task3, and monitors the task in real time Resource usage of task2 and task3.

In step S8060, during the execution of task task1 by the worker node 1, the current usage of the resource and the execution status of task task1 are sent to the execution node; and, during the execution of task task1 and task2 by the worker node 2, the resource will be sent to the execution node. The current usage status and the execution status of task task2 and task3 are sent to the execution node.

In step S8070, after task task1 is executed, the end status and resource usage of working node 1 are fed back to working node 1; and, after task task2 and task3 are executed, the end status and resource usage of working node 2 are fed back to working node 2. .

In step S8080, the working node 1 reports the information fed back by the task task1 to the scheduling node; and the working node 2 reports the information fed back by the tasks task2 and task3 to the scheduling node.

Step S8090, the scheduling node summarizes the information and resource usage of all tasks in the job and submits it to the resource prediction node, so that the resource prediction node continuously collects the real resource usage to update the rules and/or machines used to perform resource prediction Learning model.

The present disclosure may be a system, method and/or computer program product. The computer program product may include a computer-readable storage medium loaded with computer-readable program instructions configured to cause a processor to implement various aspects of the present disclosure.

The computer-readable storage medium may be a tangible device that can hold and store instructions used by the instruction execution device. The computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples (non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM) Or flash memory), static random access memory (SRAM), portable compact disk read-only memory (CD-ROM), digital versatile disk (DVD), memory stick, floppy disk, mechanical encoding device, such as a printer with instructions stored thereon The protruding structure in the hole card or the groove, and any suitable combination of the above. The computer-readable storage medium used here is not interpreted as the instantaneous signal itself, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (for example, light pulses through fiber optic cables), or through wires Transmission of electrical signals.

The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to various computing/processing devices, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and/or a wireless network. The network may include copper transmission cables, optical fiber transmission, wireless transmission, routers, firewalls, switches, gateway computers, and/or edge servers. The network adapter card or network interface in each computing/processing device receives computer-readable program instructions from the network, and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing/processing device .

The computer program instructions configured to perform the operations of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or in one or more programming languages Source code or object code written in any combination of, the programming language includes object-oriented programming languages—such as Smalltalk, C++, etc., and conventional procedural programming languages—such as "C" language or similar programming languages. Computer-readable program instructions can be executed entirely on the user's computer, partly on the user's computer, executed as a stand-alone software package, partly on the user's computer and partly executed on a remote computer, or entirely on the remote computer or server carried out. In the case of a remote computer, the remote computer can be connected to the user's computer through any kind of network-including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (for example, using an Internet service provider to connect to the user's computer) connection). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), can be customized by using the status information of the computer-readable program instructions. The computer-readable program instructions are executed to realize various aspects of the present disclosure.

Various aspects of the present disclosure are described herein with reference to flowcharts and/or block diagrams of methods, apparatuses (systems) and computer program products according to embodiments of the present disclosure. It should be understood that each block of the flowcharts and/or block diagrams, and combinations of blocks in the flowcharts and/or block diagrams, can be implemented by computer-readable program instructions.

These computer-readable program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine that makes these instructions when executed by the processor of the computer or other programmable data processing device , A device that implements the functions/actions specified in one or more blocks in the flowcharts and/or block diagrams is produced. It is also possible to store these computer-readable program instructions in a computer-readable storage medium. These instructions make computers, programmable data processing apparatuses, and/or other devices work in a specific manner. Thus, the computer-readable medium storing the instructions includes An article of manufacture, which includes instructions for implementing various aspects of the functions/actions specified in one or more blocks in the flowcharts and/or block diagrams.

It is also possible to load computer-readable program instructions on a computer, other programmable data processing device, or other equipment, so that a series of operation steps are executed on the computer, other programmable data processing device, or other equipment to produce a computer-implemented process , So that the instructions executed on the computer, other programmable data processing apparatus, or other equipment realize the functions/actions specified in one or more blocks in the flowcharts and/or block diagrams.

The flowcharts and block diagrams in the accompanying drawings show the possible implementation architecture, functions, and operations of the system, method, and computer program product according to multiple embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, program segment, or part of an instruction, and the module, program segment, or part of an instruction includes one or more that are configured to implement prescribed logical functions. Executable instructions. In some alternative implementations, the functions marked in the block may also occur in a different order than the order marked in the drawings. For example, two consecutive blocks can actually be executed substantially in parallel, or they can sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and/or flowchart, and the combination of the blocks in the block diagram and/or flowchart, can be implemented by a dedicated hardware-based system that performs the specified functions or actions Or it can be realized by a combination of dedicated hardware and computer instructions. It is well known to those skilled in the art that realization by hardware, realization by software, and realization by a combination of software and hardware are all equivalent.

The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Without departing from the scope and spirit of the described embodiments, many modifications and changes are obvious to those of ordinary skill in the art. The choice of terms used herein is intended to best explain the principles, practical applications, or technical improvements in the market of the various embodiments, or to enable other ordinary skilled in the art to understand the various embodiments disclosed herein. The scope of the present disclosure is defined by the appended claims.

Industrial applicability

Through the embodiments of the present disclosure, during the process of executing the assigned tasks by the working nodes, it dynamically adjusts the resource usage, thereby realizing efficient task allocation and resource scheduling, and improving task execution efficiency and resource utilization. Therefore, the present disclosure has strong industrial applicability.

Claims (20)

1. A method for allocating resources and tasks in a distributed system. The method includes:

   Receive jobs for execution in a distributed system;

   According to the resource-related information of each task type in the job and the resource upper limit of each working node in the distributed system, predict the resource demand that needs to be allocated for each task executed by the working node;

   According to the predicted resource demand, assign each task to the appropriate work node; and

   During the execution of assigned tasks by the working nodes, dynamic adjustments are made to resource usage.
2. The method of claim 1, wherein:

   The task types include parameter server tasks and/or training learning tasks in machine learning; and,

   The resource-related information includes at least one of the processing data scale and processing content of the corresponding task type.
3. The method according to claim 1 or 2, wherein the resource requirement includes each resource type and corresponding resource requirement value required by the task;

   Wherein, the resource demand value includes at least one of a peak demand value and a general demand value.
4. The method according to any one of claims 1 to 3, wherein the step of predicting the resource requirements of each task performed by the working node comprises: predicting the execution by the working node according to rules and/or a machine learning model The resource requirements for each task to be allocated,

   In addition, the method further includes: collecting actual resource usage of the working nodes when performing tasks to obtain the rules and/or machine learning models.
5. The method according to any one of claims 1 to 4, wherein the allocating each task to a suitable working node according to the predicted resource demand comprises:

   Get the current resource usage, current task running status and maximum limit of total resources of each working node;

   Using the preset allocation algorithm, according to the predicted resource demand, combined with the current resource usage of each working node, the current task running situation and the maximum limit of total resources, the multiple working nodes of the distributed system are selected to be suitable for execution. The working node of each task is assigned, and each task is assigned to the selected working node.
6. The method according to any one of claims 1 to 5, wherein the step of dynamically adjusting resource usage in the process of executing the assigned task by the working node comprises:

   Monitor the resource usage of tasks;

   In the case where a certain resource usage of the task exceeds the predicted resource demand value, judging whether the total current usage of the certain resource exceeds the maximum total resource limit of the certain resource;

   In the case that the total current usage of the certain resource exceeds the maximum limit of the total resource of the certain resource, dynamic adjustment is made according to the compressibility of the certain resource.
7. The method according to any one of claims 1 to 6, wherein the step of dynamically adjusting according to the compressibility of the certain resource comprises:

   Searching for tasks in the working node whose resources exceed the predicted resource demand value as candidate tasks, and selecting the candidate tasks according to processing priority and/or start time;

   According to the compressibility of the certain resource, the selected candidate task is dynamically adjusted.
8. The method according to any one of claims 1 to 7, wherein the dynamic adjustment for the selected candidate task according to the compressibility of the certain resource comprises:

   In the case that the certain resource is a compressible resource, the resource usage amount of the candidate task for the certain resource is restricted.
9. The method according to any one of claims 1 to 8, wherein the dynamic adjustment for the selected candidate task according to the compressibility of the certain resource comprises:

   In the case that the certain resource is an incompressible resource, judging whether the candidate task supports expansion;

   In the case that the candidate task supports capacity expansion, determining whether there are other working nodes that can execute the candidate task;

   In the case where the other working nodes exist, extract the uncompleted part of the task among the candidate tasks;

   Send the extracted part of the task to the other working nodes.
10. The method according to any one of claims 1 to 9, wherein the method further comprises:

    In the case that the candidate task does not support capacity expansion, determine whether the candidate task supports freezing;

    In the case that the candidate task supports freezing, the memory data of the candidate task is written into the disk of the working node.
11. The method according to any one of claims 1 to 10, wherein the method further comprises:

    In the absence of the other working nodes, determine whether the candidate task supports freezing;

    In the case that the candidate task supports freezing, the memory data of the candidate task is written into the disk of the working node.
12. The method according to any one of claims 1 to 11, wherein after writing the memory data of the candidate task to the disk of the working node, the method further comprises:

    Determine whether the candidate task supports migration;

    In the case that the candidate task supports migration, determining whether there are other working nodes that can execute the candidate task;

    Sending the memory data to the other working nodes.
13. The method according to any one of claims 1 to 12, wherein the method further comprises:

    In the case that the candidate task does not support migration, in response to a set trigger event, obtain the current resource usage of the candidate task;

    Based on the current resource usage of the candidate task, continue to execute the candidate task by the working node.
14. The method according to any one of claims 1 to 13, wherein:

    The trigger event includes at least one of the allocated tasks being completed in the working node, and the released resource in the working node.
15. The method according to any one of claims 1 to 14, wherein the method further comprises:

    If the candidate task does not support freezing, directly kill the candidate task.
16. The method according to any one of claims 1 to 15, wherein, after the method directly kills the candidate task, the method further comprises:

    Collecting resource usage of the candidate task sent by the working node;

    Based on the resource usage, the resource requirements of the candidate tasks are expanded, so as to allocate the candidate tasks to suitable working nodes again according to the expanded resource requirements.
17. A device for allocating resources and tasks in a distributed system, including:

    The job receiving unit is configured to receive jobs for execution in the distributed system;

    The resource demand prediction unit is configured to predict the resource demand to be allocated for each task executed by the working node based on the resource-related information of each task type in the job and the resource upper limit of each working node in the distributed system;

    The task allocation unit is configured to allocate each task to a suitable work node according to the predicted resource demand;

    The resource scheduling unit is configured to dynamically adjust the resource usage in the process of executing the assigned task by the working node.
18. A device for allocating resources and tasks in a distributed system, including:

    The memory is configured to store executable instructions;

    A processor configured to execute the resource and task allocation device in the distributed system according to the control of the executable instruction to execute the resource and task in the distributed system according to any one of claims 1 to 16 Method of distribution.
19. A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method for allocating resources and tasks in a distributed system according to any one of claims 1 to 16 .
20. A distributed system, including:

    Multiple devices configured to provide resources;

    The device for allocating resources and tasks in a distributed system according to claim 17 or the device for allocating resources and tasks in a distributed system according to claim 18.

Images