CN113127160B

CN113127160B - Task scheduling method, system and equipment

Info

Publication number: CN113127160B
Application number: CN201911397074.2A
Authority: CN
Inventors: 车漾; 张凯; 徐晓舟; 李青坪
Original assignee: Alibaba Group Holding Ltd
Current assignee: Alibaba Group Holding Ltd
Priority date: 2019-12-30
Filing date: 2019-12-30
Publication date: 2024-05-28
Anticipated expiration: 2039-12-30
Also published as: CN113127160A

Abstract

The embodiment of the application provides a task scheduling method, a task scheduling system and task scheduling equipment. The method comprises the following steps: determining a heterogeneous resource combination from each computing node in the heterogeneous computing cluster according to the task to be scheduled; determining a target heterogeneous resource combination from the determined heterogeneous resource combinations according to the runtime information of each heterogeneous resource combination; and dispatching the task to be dispatched to a target computing node where the target heterogeneous resource combination is located, so that the target computing node is distributed to the target heterogeneous resource combination for processing. The scheme provided by the embodiment of the application can determine the proper target heterogeneous resource combination according to the actual running condition of each heterogeneous resource combination so as to improve the processing speed of the task to be scheduled.

Description

Task scheduling method, system and equipment

Technical Field

The present application relates to the field of computer technologies, and in particular, to a task scheduling method, system, and device.

Background

In the field of high performance computing and artificial intelligence, it is often desirable to accelerate processing of computing tasks based on heterogeneous computing clusters. Typically, the heterogeneous computing cluster includes a plurality of computing nodes, each of which is composed of at least one central processing unit CPU and a plurality of heterogeneous computing chips. The heterogeneous computing chip may be a GPU (Graphics Processing Unit, image processor), an FPGA (Field Programmable GATE ARRAY ).

Currently, a scheduling policy related to heterogeneous computing clusters is to schedule computing tasks to available computing nodes for processing. The scheduling method is simple, and is difficult to find a proper computing node for the computing task, so that the processing speed of the computing task is influenced.

Disclosure of Invention

In view of the foregoing, the present application has been made to provide a task scheduling method, system and apparatus that solve or at least partially solve the foregoing problems.

Thus, in one embodiment of the present application, a task scheduling method is provided. The method comprises the following steps:

determining a heterogeneous resource combination from each computing node in the heterogeneous computing cluster according to the task to be scheduled;

Determining a target heterogeneous resource combination from the determined heterogeneous resource combinations according to the runtime information of each heterogeneous resource combination;

and dispatching the task to be dispatched to a target computing node where the target heterogeneous resource combination is located, so that the target computing node is distributed to the target heterogeneous resource combination for processing.

In another embodiment of the present application, a task scheduling method is provided. The method comprises

Acquiring a task to be scheduled and a processing requirement thereof; wherein the processing requirements include: when the task to be scheduled is processed, the interconnection types required among heterogeneous resources are processed;

Combining the interconnection types, and determining a target heterogeneous resource combination from computing nodes in the heterogeneous computing cluster;

and dispatching the task to be dispatched to a target computing node where the target heterogeneous resource combination is located in the heterogeneous computing cluster, so that the target computing node distributes the task to be dispatched to the target heterogeneous resource combination for processing.

In another embodiment of the present application, a task scheduling system is provided. The system comprises: heterogeneous computing clusters; the heterogeneous computing cluster comprises: a scheduler and a plurality of computing nodes;

the scheduler is used for: determining a heterogeneous resource combination from each computing node in the heterogeneous computing cluster according to the task to be scheduled; determining a target heterogeneous resource combination from the determined heterogeneous resource combinations according to the runtime information of each heterogeneous resource combination; and dispatching the task to be dispatched to a target computing node where the target heterogeneous resource combination is located, so that the target computing node is distributed to the target heterogeneous resource combination for processing.

the scheduler is used for: acquiring a task to be scheduled and a processing requirement thereof; wherein the processing requirements include: when the task to be scheduled is processed, the interconnection types required among heterogeneous resources are processed; according to the interconnection type, determining a target heterogeneous resource combination from computing nodes in the heterogeneous computing cluster; and dispatching the task to be dispatched to a target computing node where the target heterogeneous resource combination is located in the heterogeneous computing cluster, so that the target computing node distributes the task to be dispatched to the target heterogeneous resource combination for processing.

In another embodiment of the present application, a task scheduling method is provided. The method comprises the following steps:

according to the task to be scheduled, determining a computing resource combination from each computing node in the computing cluster;

determining a target computing resource combination from the determined computing resource combinations according to the runtime information of each computing resource combination;

And dispatching the task to be dispatched to a target computing node where the target computing resource combination is located, so that the target computing node can be distributed to the target computing resource combination for processing.

Acquiring a task to be scheduled and a processing requirement thereof; wherein the processing requirements include: when the task to be scheduled is processed, calculating the interconnection type required among the resources;

combining the interconnection types, and determining a target computing resource combination from computing nodes in a computing cluster;

And dispatching the task to be dispatched to a target computing node where the target computing resource is located in the computing cluster, so that the target computing node distributes the task to be dispatched to the target computing resource combination for processing.

In another embodiment of the present application, an electronic device is provided. The electronic device includes: a memory and a processor, wherein,

The memory is used for storing programs;

The processor, coupled to the memory, is configured to execute the program stored in the memory for:

The memory is used for storing programs;

Determining a GPU combination of the graphics processor from all computing nodes in the heterogeneous computing cluster according to the image recognition task to be scheduled;

determining a target GPU combination from the determined GPU combinations according to the runtime information of each GPU combination;

And dispatching the image recognition task to a target computing node where the target GPU combination is located, so that the target computing node is distributed to the target GPU combination for acceleration processing.

Acquiring an image recognition task to be scheduled and a processing requirement thereof; wherein the processing requirements include: when the image recognition task is processed, the interconnection types required among the GPUs are processed;

Combining the interconnection types, and determining a target GPU combination from computing nodes in the heterogeneous computing clusters;

And dispatching the image recognition task to a target computing node where the target GPU combination is located in the heterogeneous computing cluster, so that the target computing node distributes the task to be dispatched to the target GPU combination for acceleration processing.

The memory is used for storing programs;

According to the technical scheme provided by the embodiment of the application, the runtime information of the heterogeneous resource combination is detected when the heterogeneous computing cluster runs, so that the real running condition of the heterogeneous resource combination can be truly reflected. According to the real running condition of each heterogeneous resource combination, the to-be-scheduled task can be matched to a proper target heterogeneous resource combination, and the processing speed of the to-be-scheduled task is improved.

According to the technical scheme provided by the embodiment of the application, the target heterogeneous resource combination finally used for processing the task to be scheduled is determined from the calculation nodes in the heterogeneous calculation cluster by combining the interconnection type required by the processing of the task to be scheduled. That is, the technical scheme provided by the embodiment of the application can combine the self characteristics of the task to be scheduled, select a proper heterogeneous resource combination to process the task to be scheduled, and effectively improve the processing speed of the task to be scheduled.

According to the technical scheme provided by the embodiment of the application, the runtime information of the computing resource combination is detected when the computing cluster runs, so that the real running condition of the computing resource combination can be truly reflected. According to the real running condition of each computing resource combination, the task to be scheduled can be matched to a proper target computing resource combination, and the processing speed of the task to be scheduled is improved.

According to the technical scheme provided by the embodiment of the application, the target computing resource combination finally used for processing the task to be scheduled is determined from the computing cluster by combining the interconnection type required by the processing of the task to be scheduled. That is, the technical scheme provided by the embodiment of the application can combine the self characteristics of the task to be scheduled, select a proper computing resource combination to process the task to be scheduled, and effectively improve the processing speed of the task to be scheduled.

Drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions of the prior art, the following description will briefly explain the drawings used in the embodiments or the description of the prior art, and it is obvious that the drawings in the following description are some embodiments of the present application, and other drawings can be obtained according to these drawings without inventive effort for a person skilled in the art.

FIG. 1a is a flowchart illustrating a task scheduling method according to an embodiment of the present application;

FIG. 1b is a schematic diagram of a computing node according to an embodiment of the present application;

FIG. 1c is a schematic illustration of internal communication of a virtual machine according to an embodiment of the present application;

FIG. 1d is a schematic diagram illustrating a task scheduling system according to an embodiment of the present application;

FIG. 1e is a flowchart illustrating a task scheduling method according to an embodiment of the present application;

FIG. 2 is a flowchart of a task scheduling method according to another embodiment of the present application;

FIG. 3 is a flowchart illustrating a task scheduling method according to another embodiment of the present application;

FIG. 4 is a flowchart of a task scheduling method according to another embodiment of the present application;

FIG. 5 is a block diagram illustrating a task scheduler according to an embodiment of the present application;

FIG. 6 is a block diagram illustrating a task scheduler according to another embodiment of the present application;

Fig. 7 is a block diagram of an electronic device according to an embodiment of the present application.

Detailed Description

The inventor finds through observation of a large number of artificial intelligence model training and high performance computing that: although the GPU has strong parallel processing capability, the process of data processing can be greatly accelerated, the calculation power required in real large-scale calculation cannot be satisfied by the single-machine single-card GPU, and thus single-machine multi-card and even multi-machine multi-card collaborative training is required. In the single-machine multi-card or multi-machine multi-card cooperative processing, the factors influencing the processing speed are very numerous, and mainly include the following aspects:

1. The communication bandwidths between different GPUs on the same host are different, and different choices bring about different application performance performances. Communication can occur between GPUs on the same host, while the I/O communication overhead is different between different GPUs due to the physical topology of the GPUs. Such as: GPU DirectP 2P (Peer-to-Peer) point-to-point communication can be achieved between two GPUs on the same PCIe (PCI Express) bus (e.g., GPU0_80, GPU1_81, GPU2_82, and GPU3_83 in FIG. 1b are on the same PCIe bus; and GPU4_84, GPU5_85, GPU6_86, and GPU7_87 are on the same PCIe bus); two GPUs located on different PCIe buses (e.g., gpuse0_80 and gpuse4_84 in fig. 1b are located on different PCIe buses) require QPI communication and cannot implement GPU Direct P2P (Peer-to-Peer) point-to-point communication. The interconnection bandwidth of point-to-point communication between two GPUs on the same PCIe bus through GPU Direct P2P (Peer-to-Peer) is obviously larger than that of two GPUs on different PCIe buses through QPI communication, and the difference of 4 times and even 10 times can be achieved.

2. Due to the virtualization of GPU P2P, the bandwidth overhead order of communication between virtual machine GPUs is not consistent with the behavior of physical machine GPUs. The control right of the GPU card can be completely authorized to the Virtual Machine (VM) by using a PCI Pass-through virtualization technology, so that a GPU driver (GPU driver) in the Virtual Machine can directly control the GPU without participation of a Hypervisor of the Virtual Machine, and the performance can be close to that of a physical Machine. However, during PCI express, PCIe communications are routed to IOMMU (i/o memory management unit ) by router RC, that is, during point-to-point P2P communications, the same is required to be routed to IOMMU, as shown in fig. 1c, the point-to-point communication overhead between GPU0 and GPU1 is forced to the same PCIe switch 70 (PCIE SWITCH) due to both up and down traffic (arrow in fig. 1c represents the trend of traffic), but rather is larger than the communication overhead from GPU0_80 to GPU2_82 in fig. 1b, and the larger the communication overhead is, the smaller the communication bandwidth is. That is to say: on a physical machine, the bandwidth of PIX (Connection TRAVERSING A SINGLE PCIE SWITCH) communication is larger than that of PXB (Connection traversing multiple PCIE SWITCHES (without TRAVERSING THE PCIE Host Bridge)) communication; on a virtual machine, on the contrary, the bandwidth of the PXB communication is larger than that of the PIX communication. And (3) injection: the communication between gpuse0_80 and gpuse1_81 in fig. 1b and 1c both belong to PIX; the communication between gpuse0_80 and gpuse2_82 in fig. 1b belongs to PXB.

3. The barrel effect can affect the efficiency of single machine multi-card training. When GPU cards based on different communication modes work together, the narrowest bandwidth among them affects the training speed.

4. Since the container engine will internally order the selected GPU card order, for example: four GPUs with id numbers of id3, id5, id0 and id2 are automatically sequenced to form a communication ring, namely a data transmission path: id0 to id2, id2 to id3, id3 to id5, id5 to id0. That is, the bandwidths that are really significant are id0 to id2, id2 to id3, id3 to id5, id5 to id 0; whereas the bandwidths of id0 to id3, id2 to id5 need not be practically considered.

5. Whether the application uses the peer-to-peer P2P communication mode or not inside the application can also affect the selection of GPU cards.

From the analysis of the above aspects, when no virtual machine exists in the cluster, the communication overhead or the communication bandwidth in the real scene is determined to be more reliable according to the physical topological structure relation of the GPU; once virtual machines exist in the cluster, communication overhead or communication bandwidth in a real scene cannot be accurately judged according to the physical topological structure relation of the GPU. Therefore, in order to improve the task processing speed, the inventor proposes to select the GPU card according to the interconnection bandwidth between the GPU cards obtained by actual test, so that the rationality of selection can be effectively improved, and the processing speed is improved; and meanwhile, the universality of the scheduling method is improved. And when the GPU card is selected, the characteristics of the application are combined, so that the rationality of selection can be further improved, and the processing speed is further improved.

In order to enable those skilled in the art to better understand the present application, the following description will clearly and completely describe the technical solution according to the embodiments of the present application according to the accompanying drawings. It will be apparent that the described embodiments are only some, but not all, embodiments of the application. Based on the embodiments of the present application, other embodiments that may be obtained by those skilled in the art without making any inventive effort are within the scope of the present application.

Furthermore, in some of the flows described in the specification, claims, and drawings above, a plurality of operations occurring in a particular order may be included, and the operations may be performed out of order or concurrently with respect to the order in which they occur. The sequence numbers of operations such as 101, 102, etc. are merely used to distinguish between the various operations, and the sequence numbers themselves do not represent any order of execution. In addition, the flows may include more or fewer operations, and the operations may be performed sequentially or in parallel. It should be noted that, the descriptions of "first" and "second" herein are used to distinguish different messages, devices, modules, etc., and do not represent a sequence, and are not limited to the "first" and the "second" being different types.

Fig. 1d is a schematic structural diagram of a scheduling system according to an embodiment of the present application. As shown in fig. 1d, the scheduling system includes: computing clusters; the computing cluster comprises: a scheduler 100 and a plurality of computing nodes 200;

the scheduler is used for: according to the task to be scheduled, determining a computing resource combination from each computing node in the computing cluster; determining a target computing resource combination from the determined computing resource combinations according to the runtime information of each computing resource combination; and dispatching the task to be dispatched to a target computing node where the target computing resource combination is located, so that the target computing node can be distributed to the target computing resource combination for processing.

Wherein each computing node 200 may include a plurality of computing resources 2001 thereon. Each computing node may be a physical machine or virtual machine (e.g., a virtual machine on a cloud computing platform). The scheduler may be a physical machine or a virtual machine.

The task scheduling process is described in detail below with reference to the method embodiment.

Fig. 1e shows a flow chart of a task scheduling method according to an embodiment of the present application. The method provided by the embodiment of the application is executed by the scheduler. As shown in fig. 1e, the method comprises:

101. And determining the computing resource combination from each computing node in the computing cluster according to the task to be scheduled.

102. And determining a target computing resource combination from the determined computing resource combinations according to the runtime information of each computing resource combination.

103. And dispatching the task to be dispatched to a target computing node where the target computing resource combination is located, so that the target computing node can be distributed to the target computing resource combination for processing.

In the above 101, the computing cluster may include a plurality of computing nodes, each of which includes a plurality of computing resources. Wherein a computing node may be a physical machine or a virtual machine (e.g., a virtual machine on a cloud computing platform). A computing resource may be understood as a computing unit that a computing node needs to invoke when processing a task to be scheduled, such as: CPU (central processing unit ), RAM (Random Access Memory, random access memory), GPU, FPGA, etc.

In an example, the runtime information of each computing resource combination may include one or more of data transmission performance index among computing resources, load condition of each computing resource, and abnormal operation or abnormal operation information.

In general, when a task to be scheduled is processed, multiple computing resources on the same computing node or different computing nodes need to be called at the same time, and data transmission needs to be performed between the multiple computing resources, so that the size of a data transmission performance index between the computing resources affects the length of data transmission delay, that is, the processing speed of the computing node for processing the task to be scheduled is affected. In addition, the load condition and whether the operation of the computing resources is abnormal also affect the processing speed of the computing node for processing the task to be scheduled.

The task to be scheduled is the task to be calculated. In the data identification scenario of the artificial intelligence algorithm model, the task to be scheduled may specifically be a data identification task or a subtask in the data identification task. The data recognition task may be a data recognition task in a model training process (referred to as a model training task at this time) or a data recognition task in a model application process. In an example, the data recognition task may be an image recognition task, a video recognition task, or the like. At present, in order to improve the efficiency of deployment and operation environment switching, the containerization technology is widely applied. When employing containerization techniques, the tasks to be scheduled may be packaged into containers or pods. The container contains application programs corresponding to tasks to be scheduled and environment dependencies thereof, and Pod is a packaging structure of one or more containers. In an example, a task to be scheduled may be packaged into a Pod, and a subsequent task may be to schedule the Pod corresponding to the task to be scheduled to a suitable computing node.

It should be added that taking an artificial intelligence algorithm model training scene as an example, when the calculation amount of a complete model training task is quite huge, the model training task can be split into a plurality of subtasks, each subtask is a task to be scheduled, each subtask is packaged into a Pod, and thus the model training task can be processed through the cooperation of the plurality of pods, and the speed is improved. Multiple Pod corresponding to one model training task can be respectively scheduled to multiple computing nodes for processing.

And determining the computing resource combination from each computing node in the computing cluster according to the task to be scheduled. In one example, computing resources in a computing resource combination are all located on the same computing node. In one example, the type and number of computing resources required may be determined based on the task to be scheduled; and determining the computing resource combination from each computing node according to the type and the quantity of the required computing resources. Before determining the computing resource combination from each computing node according to the type and the number of the required computing resources, determining available computing resources from a plurality of computing resources on each computing node according to the node operation log reported by each computing node; and determining the computing resource combination from available computing resources on each computing node according to the type and the quantity of the required computing resources. In one example, available computing resources may be understood as computing resources that are functioning properly and idle. Generally, whether the computing resource is abnormal in operation and whether the node is idle can be determined according to the information about whether the computing resource is abnormal in operation and the load condition, and the specific implementation manner can be referred to the prior art and will not be described herein. The information about whether the operation of the computing resource is abnormal or not and the load condition can be obtained from the operation log.

In another example, computing resources in one computing resource combination may also be located on different computing nodes, and it should be added that, when such computing resource combination processes tasks, computing resources located on different computing nodes may communicate via RDMA (Remote Direct Memory Access, remote direct data access) technology, so as to reduce data transmission delay.

In the above 102, in one example, the data transmission performance index between computing resources in each computing resource combination may be tested and recorded in advance. For example: after the computing cluster is started, testing the data transmission performance index between any two computing resources in each computing node, and recording to obtain recording information for subsequent inquiry. Thus, the data transmission performance index among the computing resources in each computing resource combination can be queried from the recorded information. The data transmission performance index may include: one or more of interconnect bandwidth, historical data transfer rate, historical data transfer latency.

Taking the example that the runtime information of each heterogeneous resource combination includes the data transmission performance index between the computing resources, the step 102 of determining the target computing resource combination from the determined computing resource combinations according to the data transmission performance index between the computing resources in each computing resource combination may specifically be implemented by the following steps:

a1, summing the data transmission performance indexes among the computing resources aiming at each computing resource combination to obtain an index sum corresponding to each computing resource combination.

A2, determining a target computing resource combination from the determined computing resource combinations according to the index sum corresponding to each computing resource combination.

In the above a1, each computing resource combination includes a first computing resource combination, and the data transmission performance indexes among the computing resources in the first computing resource combination are summed to obtain an index sum corresponding to the first computing resource combination. Wherein the first computing resource combination refers to any one of the determined computing resource combinations. Specifically, the data transmission performance index between any two computing resources in the plurality of computing resources in the first computing resource combination can be obtained respectively; and summing the data transmission performance indexes of any two computing resources in the plurality of computing resources in the acquired first computing resource combination to obtain an index sum corresponding to the first computing resource combination.

For example: the first computing resource combination comprises a computing resource A, a computing resource B and a computing resource C; respectively acquiring a data transmission performance index a between the computing resource A and the computing resource B, a data transmission performance index B between the computing resource A and the computing resource C and a data transmission performance index C between the computing resource B and the computing resource C; and summing the data transmission performance index a, the data transmission performance index b and the data transmission performance index c to obtain an index sum h corresponding to the first computing resource combination. Where h=a+b+c.

In practical application, in order to achieve the optimal processing efficiency, in the above a2, the computing resource combination with the optimal index sum may be determined as the target computing resource combination. For example: when the index is the interconnection bandwidth, the computing resource combination with the maximum index synthesis can be determined as the target computing resource combination.

The above scheme for calculating the index sum corresponding to the computing resource combination is suitable for the case that the number of computing resources in the computing resource combination is greater than or equal to 3. When the number of computing resources in the computing resource combinations is 2, the target computing resource combination is determined from the determined computing resource combinations directly according to the data transmission performance indexes between the two computing resources in each computing resource combination without computing the index sum corresponding to the computing resource combinations. In one example, a computing resource combination that is optimal for the data transmission performance index between two computing resources may also be determined as the target computing resource combination.

In 103, the task to be scheduled is scheduled to the target computing node, that is, the task to be scheduled is bound to the target computing node. When in practical application, the task to be scheduled is scheduled to the target computing node, and meanwhile, the identification information of the computing resources in the target computing resource combination is required to be sent to the target computing node, so that the target computing node distributes the task to be scheduled to the target computing resource combination according to the identification information. The identification information may specifically be an id number of the computing resource.

The inventors found that: in screening the computing resource combinations, the data transmission performance indicators among certain computing resources in the computing resource combinations are not actually considered. For example: when the computing resource combination processes the task to be scheduled, some computing resources do not have communication (namely, no data transmission exists), and the data transmission performance indexes among the computing resources do not need to be considered, so that the influence on the final screening result can be avoided. Therefore, the method provided by the embodiment of the application further comprises the following steps:

104. And determining a communication path required by each computing resource combination to process the task to be scheduled.

105. And aiming at each computing resource combination, acquiring a data transmission performance index between two computing resources needing to be communicated on a communication path required by the computing resource combination to process the task to be scheduled.

The data transmission performance indexes among the computing resources in each computing resource combination comprise: the method processes the data transmission performance index between two computing resources needing to be communicated on the communication path required by the task to be scheduled.

In the above 104, the communication path required for processing the task to be scheduled is typically a communication loop. In practice, once a plurality of computing resources in a computing resource combination are determined, the hardware layer automatically ranks the plurality of computing resources in the resource combination. The communication path required by the computing resource combination to process the task to be scheduled is a communication loop formed by connecting a plurality of computing resources in sequence end to end.

In 105, each computing resource combination includes a first computing resource combination, where the first computing resource combination refers to any one of the determined computing resource combinations. For the first computing resource combination, the data transmission performance index between all the two computing resources needing to be communicated on the communication path required by the first computing resource combination to process the task to be scheduled can be obtained.

For example: the first computing resource combination comprises computing resource A, computing resource B, computing resource C and computing resource D, and the corresponding communication paths are as follows: computing resource a transmits data to computing resource B, computing resource B transmits data to computing resource C, computing resource C transmits data to computing resource D, and computing resource D transmits data to computing resource a. Then, the data transmission performance index between the computing resources in the first computing resource combination includes: data transmission performance index between computing resource a and computing resource B, data transmission performance index between computing resource B and computing resource C, data transmission performance index between computing resource C and computing resource D, and data transmission performance index between computing resource D and computing resource a. The supplementary ones are: wherein the data transmission performance index between the computing resource A and the computing resource C is not considered

In an example, the "determining the target computing resource combination from the determined computing resource combinations according to the runtime information of each computing resource combination" in 102 above may be implemented by:

1021a, for each computing resource combination, summing the data transmission performance indexes between two computing resources needing to be communicated on a communication path required by the computing resource combination to process the task to be scheduled, so as to obtain an index sum corresponding to each computing resource combination.

1022A, determining a target computing resource combination from the determined computing resource combinations according to the index sum corresponding to each computing resource combination.

In the 1021a, specifically, for the first computing resource combination, the data transmission performance indexes between all the two computing resources that need to be communicated on the communication path required by the first computing resource combination to process the task to be scheduled are summed to obtain an index sum corresponding to the first computing resource combination.

Along the above example: on a communication path required by the first computing resource combination processing task to be scheduled, the data transmission performance indexes between two computing resources needing to be communicated are as follows: a data transmission performance index a between the computing resource A and the computing resource B, a data transmission performance index B between the computing resource B and the computing resource C, a data transmission performance index C between the computing resource C and the computing resource D, and a data transmission performance index D between the computing resource D and the computing resource A; the index sum corresponding to the first computing resource combination is h. Where h=a+b+c+d.

In order to obtain the optimal processing efficiency in practical application, in the above 1022a, the computing resource combination with the optimal index sum may be used as the target computing resource combination. For example: and when the index is the interconnection bandwidth, selecting the computing resource combination with the maximum index sum as a target computing resource combination.

The inventors further studied and found that: the computing resource combination with the optimal sum of the indices does not necessarily achieve the fastest processing speed, because in real task processing, the worst data transmission performance index in the computing resource combination may affect the processing speed. Thus, in another example, the "determining the target computing resource combination from the determined computing resource combinations according to the runtime information of each computing resource combination" in 102 above may be implemented by:

1021b, determining the worst data transmission performance index of the data transmission performance indexes between two computing resources needing to be communicated on a communication path required by each computing resource combination for processing the task to be scheduled.

1022B, determining a target computing resource combination from the determined computing resource combinations according to the worst data transmission performance index corresponding to each computing resource combination.

In 1021b, for the first computing resource combination, the worst data transmission performance index of the data transmission performance indexes between the two computing resources that need to communicate on the communication path required by the first computing resource combination to process the task to be scheduled is determined.

In order to achieve the optimal processing efficiency, in 1022b, the computing resource combination with the worst data transmission performance index being optimal may be determined as the target computing resource combination.

For example: the determined computing resource combinations have computing resource combinations z1 and z2; and if the worst data transmission performance index corresponding to z1 is 5G and the worst data transmission performance index corresponding to z2 is 2G, selecting z1 as the target computing resource combination.

In practical application, a task submitting interface can be provided for a user, and various screening strategies are displayed on the task submitting interface for the user to select, so that user experience can be provided, and different requirements of the user are met. The plurality of screening measurements may have a first screening policy, a second screening policy, and a third screening policy. The first screening strategy is specifically an index sum optimal rule, the second screening strategy is specifically a worst index optimal strategy, and the third screening strategy is specifically a static strategy. Thus, the above method may further comprise:

c. And acquiring a screening strategy configured by a user for the task to be scheduled.

The screening strategy configured by the user aiming at the task to be scheduled is the screening strategy selected by the user in the task submitting interface.

Accordingly, in 102, the "determining the target computing resource combination from the determined computing resource combinations according to the runtime information of each available computing resource combination" specifically includes: and when the screening strategy is the first screening strategy or the second screening strategy, determining a target computing resource combination from the determined computing resource combinations according to the screening strategy and the running information of each computing resource combination.

In practical application, when the screening policy is the first screening policy (i.e. the index sum optimization rule), determining a target computing resource combination from the determined computing resource combinations according to the screening policy and runtime information of each computing resource combination, wherein the method specifically comprises the following steps: steps 1021a and 1022a described above; when the screening policy is the second screening policy (i.e. the worst index optimal policy), determining a target computing resource combination from the determined computing resource combinations according to the screening policy and the runtime information of each computing resource combination, wherein the method specifically comprises the following steps: steps 1021b and 1022b described above.

Further, the method further comprises the following steps:

d. And when the screening strategy is a third screening strategy, acquiring at least one alternative computing resource combination.

C. And determining a target computing resource combination from the at least one alternative computing resource combination.

The third filtering strategy is a static strategy.

In an example, in the step b, obtaining at least one candidate computing resource combination may include: determining the amount of computing resources required for the scheduled task; at least one candidate computing resource combination previously provided for the computing resource quantity is acquired.

Wherein the number of computing resources in each of the at least one candidate computing resource combination is equal to the number of computing resources.

In another example, the obtaining at least one candidate computing resource combination in the step b may include: determining the processing requirement of the task to be scheduled, wherein the processing requirement comprises the following steps: when the task to be scheduled is processed, the number of the required computing resources and the interconnection type required among the computing resources are processed; at least one alternative combination of computing resources previously provided for the number of computing resources and the interconnection type is obtained.

The at least one alternative combination of computing resources may be manually equipped, for example: the user prepares at least one alternative computing resource combination with better performance aiming at the computing resource quantity and the interconnection type according to the software and hardware condition or the operation condition.

At present, most computing resources in a computing cluster can communicate through a data transmission channel of a non-point-to-point interconnection type, and a data transmission channel of a point-to-point interconnection type is also opened between part or all of the computing resources. For example: by adopting the GPU Direct technology, point-to-point p2p interconnection communication between two GPUs on the same PCIe bus can be realized; for another example: point-to-point p2p interconnect communication between two GPUs across NUMA (Non Uniform Memory Access Architecture, non-uniform memory access architecture) can be achieved through NVLink technology. In general, the interconnect bandwidth corresponding to a point-to-point interconnect type is much greater than the interconnect bandwidth corresponding to a non-point-to-point interconnect type.

In practical application, some applications corresponding to tasks to be scheduled require a communication mode of a point-to-point interconnection type, and some applications corresponding to tasks to be scheduled require a communication mode of a non-point-to-point interconnection type. And the inventors have found that the scheduling scheme of the computing resources is related to the nature of the application itself, if the application itself does not use the capability of peer-to-peer p2p communication, then the performance is instead reduced according to the peer-to-peer p2p scheduling policy. In order to be able to screen out the optimal computing resource combination for the task to be scheduled, the processing requirements of the task to be scheduled need to be combined for screening. Specifically, the method may further include:

106. and determining the processing requirement of the task to be scheduled.

Wherein the processing requirements include: and when the task to be scheduled is processed, calculating the interconnection type required among the resources.

In 105, the "for each computing resource combination, obtain the data transmission performance index between two computing resources that need to be communicated on the communication path required by the computing resource combination to process the task to be scheduled" specifically includes: and aiming at each computing resource combination, acquiring a data transmission performance index corresponding to the interconnection type between two computing resources needing to be communicated on a communication path required by the computing resource combination to process the task to be scheduled.

In this embodiment, the data transmission performance indexes between two computing resources are divided into data transmission performance indexes corresponding to different interconnection types. For example: the interconnection type comprises a first interconnection type and a second interconnection type, and the data transmission performance index between the two computing resources is divided into a data transmission performance index corresponding to the first interconnection type and a data transmission performance index corresponding to the second interconnection type. The first interconnection type is a point-to-point interconnection type; the second interconnect type is a non-point-to-point interconnect type.

In practical application, the following method can be adopted to test the data transmission performance indexes corresponding to different interconnection types between two computing resources. I.e. the above method, may further comprise:

107. After a computing cluster is started and a communication mode corresponding to a first interconnection type of the computing cluster is started, testing interconnection bandwidth between any two computing resources on each computing node in the computing cluster, and recording the interconnection bandwidth as a data transmission performance index corresponding to the first interconnection type between the any two computing resources.

108. After a computing cluster is started and a communication mode corresponding to a first interconnection type of the computing cluster is closed, testing interconnection bandwidth between any two computing resources on each computing node in the computing cluster, and recording the interconnection bandwidth as a data transmission performance index corresponding to a second interconnection type between the any two computing resources.

Generally, when a computing cluster is started, if a communication mode corresponding to a first interconnection type is started, a data transmission channel corresponding to the first interconnection type between any two computing resources on each computing node is started, and at this time, the tested interconnection bandwidth between any two computing resources is the data transmission performance index corresponding to the first interconnection type between the any two computing resources.

If the communication mode corresponding to the first interconnection type is not started, the data transmission channel corresponding to the first interconnection type between any two computing resources on each computing node in the computing cluster is closed, only the data transmission channel corresponding to the second interconnection type is left, and at this time, the tested interconnection bandwidth between any two computing resources is the data transmission performance index corresponding to the second interconnection type between the any two computing resources.

And recording the data transmission performance index to form recording information for subsequent inquiry.

In 106, the processing requirements may be input by the user or determined according to the following scheme. Specifically: the "determining the processing requirement of the task to be scheduled" in 106 above may be implemented by the following steps:

1061. And inquiring whether the application program corresponding to the task to be scheduled is the application program which has been operated historically or not according to the operation log of the computing cluster.

1062. And if the application program corresponding to the task to be scheduled is the application program which has been operated historically, determining the interconnection type required between the computing resources when the task to be scheduled is processed according to the operation log.

In 1061, the running log may be a system call stack. If call information related to the application program corresponding to the task to be scheduled exists in the system call stack, the application program can be judged to be the application program which has been operated historically. If the call information related to the application program corresponding to the task to be scheduled does not exist in the system call stack, the application program can be judged not to be the application program which has been operated historically.

Taking artificial intelligence model training as an example, training of an artificial intelligence model may be performed a plurality of times, for example: when new training data is generated, model training can be started once based on the new training data. The application program corresponding to the training task is generally unchanged, but the training data on which the training task is based changes.

In an example, in the above 1062, "determining the type of interconnection required between computing resources when processing the task to be scheduled according to the running log", the following steps may be specifically adopted to implement:

S11, determining whether the application program uses a data transmission channel corresponding to the first interconnection type in the historical operation according to the operation log.

And S12, if the application program uses a data transmission channel corresponding to a first interconnection type in the operation of history, determining that the interconnection type is the first interconnection type.

The operation log in S11 may specifically be a system call stack, where whether the application program uses the data transmission channel corresponding to the first interconnection type in the historically operation is recorded in the system call stack.

In consideration of practical application, even if the computing cluster opens a communication mode corresponding to the point-to-point interconnection type, communication corresponding to the point-to-point interconnection type still cannot be realized due to the limitation of physical structures among some computing resources. For example: if there is no Nvlink link between two GPUs that are not on the same PCIe bus, communication corresponding to the point-to-point interconnection type cannot be implemented between the two GPUs. Thus, under the condition that the computing cluster opens a communication mode corresponding to the point-to-point interconnection type and the application program itself runs and also requires the point-to-point interconnection type, if two computing resources incapable of realizing the communication corresponding to the point-to-point interconnection type exist in the computing resource combination of the application program, due to the compatibility technology, the two computing resources incapable of realizing the communication corresponding to the point-to-point interconnection type are communicated through the data transmission channel corresponding to the non-point-to-point interconnection type, the running log is likely to record that the application program uses not only the data transmission channel corresponding to the point-to-point interconnection type but also the data transmission channel corresponding to the non-point-to-point interconnection type. However, in the case that the application itself requires a non-point-to-point interconnection type, only the data transmission channel corresponding to the non-point-to-point interconnection type is recorded in the operation log, but the data transmission channel corresponding to the point-to-point interconnection type cannot be recorded. Therefore, in 1062, "determining, according to the running log, the type of interconnection required between computing resources when processing the task to be scheduled", further includes:

And S13, if the application program does not use the data transmission channel corresponding to the first interconnection type in the historical processing, determining that the interconnection type is the second interconnection type.

Further, the "determining the processing requirement of the task to be scheduled" in the above 106 further includes:

1063. If the application program corresponding to the task to be scheduled is not the application program which has been operated historically, the interconnection type is defaulted to be a first interconnection type.

Of course, in practical application, the default interconnection type may be the second interconnection type. The embodiment of the present application is not particularly limited thereto.

Fig. 1d is a schematic structural diagram of a scheduling system according to an embodiment of the present application. As shown in fig. 1d, the scheduling system includes: computing clusters; the computing cluster comprises: a scheduler and a plurality of computing nodes;

The scheduler is used for: acquiring a task to be scheduled and a processing requirement thereof; wherein the processing requirements include: when the task to be scheduled is processed, calculating the interconnection type required among the resources; combining the interconnection types, and determining a target computing resource combination from computing nodes in a computing cluster; and dispatching the task to be dispatched to a target computing node where the target computing resource is located in the computing cluster, so that the target computing node distributes the task to be dispatched to the target computing resource combination for processing.

Wherein each computing node may include a plurality of computing resources thereon. Each computing node may be a physical machine or virtual machine (e.g., a virtual machine on a cloud computing platform). The scheduler may be a physical machine or a virtual machine.

Fig. 2 is a schematic flow chart of a task scheduling method according to another embodiment of the present application. The method provided by the embodiment of the application is executed by the scheduler. As shown in fig. 2, the method includes:

201. And acquiring tasks to be scheduled and processing requirements thereof.

202. And combining the interconnection types, and determining a target computing resource combination from computing nodes in the computing cluster.

203. And dispatching the task to be dispatched to a target computing node where the target computing resource is located in the computing cluster, so that the target computing node distributes the task to be dispatched to the target computing resource combination for processing.

The specific implementation of "obtaining the processing requirement of the task to be scheduled" in the above-mentioned 201 may refer to the relevant content of "determining the processing requirement of the task to be scheduled" in the above-mentioned embodiments, which is not described herein.

In one example, the interconnect types include a first interconnect type and a second interconnect type. The first interconnect type may be specifically a point-to-point interconnect type, and the second interconnect type may be specifically a non-point-to-point interconnect type.

In one implementation, in 202, a target computing resource combination that satisfies the interconnection type may be determined from computing nodes in the computing cluster. The target computing resource combination meeting the interconnection type refers to a computing resource combination capable of realizing communication in a communication mode corresponding to the interconnection type. The computing resources in the target computing resource combination are located on the same computing node. In actual application, determining that a plurality of computing resource combinations meeting the interconnection type possibly exist on computing nodes in a computing cluster; if there are a plurality, then one may be randomly selected from a plurality of combinations of computing resources that satisfy the interconnection type as the target combination of computing resources.

For example: the task to be scheduled is required to run on two computing resources that are interconnected point-to-point. That is, when the task to be scheduled is processed, the interconnection type required between the computing resources is a point-to-point interconnection type, and then a plurality of computing resource combinations are selected from the computing nodes in the computing cluster, wherein each computing resource combination consists of two computing resources which are interconnected point-to-point; subsequently, one of the plurality of computing resource combinations is randomly selected as a target computing resource combination.

The specific implementation of the foregoing 203 may be referred to the corresponding content in each embodiment, which is not described herein.

According to the technical scheme provided by the embodiment of the application, the target computing resource combination finally used for processing the task to be scheduled is determined from the computing cluster by combining the interconnection type required by the task to be scheduled to run. That is, the technical scheme provided by the embodiment of the application can combine the self characteristics of the task to be scheduled, select a proper computing resource combination to process the task to be scheduled, and effectively improve the processing speed of the task to be scheduled.

In another implementation manner, in the foregoing 202, "determining the target computing resource combination from the computing nodes in the computing cluster in combination with the interconnection type" may be implemented by:

2021. and determining the computing resource combination from each computing node in the computing cluster according to the task to be scheduled.

2022. And determining a target computing resource combination from the determined computing resource combinations according to the runtime information of each computing resource combination.

The specific implementation of 2021 may be referred to the corresponding content in the above embodiments, which is not described herein.

In 2022, the runtime information is detected when the heterogeneous computing cluster runs in a communication mode corresponding to the interconnection type.

In an example, the runtime information of each computing resource combination may include one or more of a data transmission performance index corresponding to the interconnection type between computing resources, a load condition of each computing resource, and abnormal operation information.

In practical application, the following method can be adopted to test the data transmission performance indexes corresponding to different interconnection types among computing resources. I.e. the above method, may further comprise:

204. after a computing cluster is started and a communication mode corresponding to a first interconnection type of the computing cluster is started, testing interconnection bandwidth between any two computing resources on each computing node in the computing cluster, and recording the interconnection bandwidth as a data transmission performance index corresponding to the first interconnection type between the any two computing resources.

205. After a computing cluster is started and a communication mode corresponding to a first interconnection type of the computing cluster is closed, testing interconnection bandwidth between any two computing resources on each computing node in the computing cluster, and recording the interconnection bandwidth as a data transmission performance index corresponding to a second interconnection type between the any two computing resources.

The first interconnection type may be a point-to-point interconnection type; the second interconnect type may specifically be a non-point-to-point interconnect type.

For details of 204 and 205, reference should be made to the corresponding details in the above embodiments, and details are not repeated here.

The runtime information combined by each computing resource includes: the actual measurement interconnection bandwidth corresponding to the interconnection type among the computing resources is as follows: in 2022, "determining a target computing resource combination from the determined computing resource combinations according to the runtime information of each computing resource combination", the method specifically includes the following steps:

b1, summing the data transmission performance indexes corresponding to the interconnection types among the computing resources aiming at each computing resource combination to obtain an index sum corresponding to each computing resource combination.

B2, determining a target computing resource combination from the determined computing resource combinations according to the index sum corresponding to each computing resource combination.

In the above step b1, each computing resource combination includes a first computing resource combination, and the data transmission performance indexes corresponding to the interconnection types among the computing resources in the first computing resource combination are summed to obtain an index sum corresponding to the first computing resource combination. Wherein the first computing resource combination refers to any one of the determined computing resource combinations. Specifically, data transmission performance indexes corresponding to the interconnection types between any two computing resources in the plurality of computing resources in the first computing resource combination can be obtained respectively; and summing the data transmission performance indexes corresponding to the interconnection types between any two computing resources in the plurality of computing resources in the acquired first computing resource combination to obtain an index sum corresponding to the first computing resource combination.

For example: the interconnection type is a first interconnection type; the first computing resource combination comprises a computing resource A, a computing resource B and a computing resource C; respectively acquiring a data transmission performance index a corresponding to a first interconnection type between a computing resource A and a computing resource B, a data transmission performance index B corresponding to a first interconnection type between the computing resource A and a computing resource C, and a data transmission performance index C corresponding to the first interconnection type between the computing resource B and the computing resource C; and summing the data transmission performance index a, the data transmission performance index b and the data transmission performance index c to obtain an index sum h corresponding to the first computing resource combination. Where h=a+b+c.

In practical application, in order to achieve the optimal processing efficiency, in the above a2, the computing resource combination with the optimal index sum may be determined as the target computing resource combination.

The above scheme for calculating the index sum corresponding to the computing resource combination is suitable for the case that the number of computing resources in the computing resource combination is greater than or equal to 3. When the number of computing resources in the computing resource combinations is 2, the target computing resource combinations are determined from the determined computing resource combinations directly according to the data transmission performance indexes corresponding to the interconnection types between the two computing resources in each computing resource combination without computing the index sum corresponding to the computing resource combinations. In an example, a computing resource combination with the largest data transmission performance index corresponding to the interconnection type between two computing resources may also be determined as the target computing resource combination.

Further, the method may further include:

206. and determining a communication path required by each computing resource combination to process the task to be scheduled.

207. And aiming at each computing resource combination, acquiring a data transmission performance index corresponding to the interconnection type between two computing resources needing to be communicated on a communication path required by the computing resource combination to process the task to be scheduled.

Wherein, the runtime information of each computing resource combination comprises: the method processes the data transmission performance index corresponding to the interconnection type between two computing resources needing to be communicated on a communication path required by the task to be scheduled.

For each computing resource combination, the data transmission performance index corresponding to the interconnection type between all the two computing resources needing to be communicated on the communication path required by the computing resource combination for processing the task to be scheduled can be obtained.

The specific implementation of the steps 206 and 207 may be referred to the corresponding content in the above embodiments, and will not be repeated here.

In an example, the above 2022 "determining the target computing resource combination from the determined computing resource combinations according to the runtime information of each computing resource combination" may be implemented by the following steps:

And S21a, summing the data transmission performance indexes corresponding to the interconnection types between two computing resources to be communicated on a communication path required by each computing resource combination for processing the task to be scheduled, so as to obtain the index sum corresponding to each computing resource combination.

S22a, determining a target computing resource combination from the determined computing resource combinations according to the index sum corresponding to each computing resource combination.

The specific implementation of the steps S21a and S22a may be referred to the corresponding content in the above embodiments, and will not be described herein.

In another example, the above 2022 "determining the target computing resource combination from the determined computing resource combinations according to the runtime information of each computing resource combination" may specifically be implemented by the following steps:

s21b, determining the worst data transmission performance index in the data transmission performance indexes corresponding to the interconnection type between two computing resources needing to be communicated on a communication path required by each computing resource combination to process the task to be scheduled.

S22b, determining a target computing resource combination from the determined computing resource combinations according to the worst data transmission performance indexes corresponding to the computing resource combinations.

The specific implementation of the steps S21b and S22b may be referred to the corresponding content in each embodiment, and will not be described herein.

In practical application, a task submitting interface can be provided for a user, and various screening strategies are displayed on the task submitting interface for the user to select, so that user experience can be provided, and different requirements of the user are met. The plurality of screening measurements may have a first screening policy, a second screening policy, and a third screening policy. The first screening strategy is specifically a bandwidth sum maximum rule, the second screening strategy is specifically a minimum bandwidth maximum strategy, and the third screening strategy is specifically a static strategy. Thus, the above method may further comprise:

Accordingly, in 2022, "determining a target computing resource combination from the determined computing resource combinations according to the runtime information of each computing resource combination", specifically: and when the screening strategy is the first screening strategy or the second screening strategy, determining a target computing resource combination from the determined computing resource combinations according to the screening strategy and the runtime information of each computing resource combination.

In practical application, when the screening policy is the first screening policy (i.e. the index sum optimization rule), determining a target computing resource combination from the determined computing resource combinations according to the screening policy and runtime information of each computing resource combination, wherein the method specifically comprises the following steps: steps S21a and S22a described above; when the screening policy is the second screening policy (i.e. the worst index optimal policy), determining a target computing resource combination from the determined computing resource combinations according to the screening policy and the runtime information of each computing resource combination, wherein the method specifically comprises the following steps: steps S21b and S22b described above.

Further, in the foregoing 202, "determining the target computing resource combination from the computing nodes in the computing cluster by combining the interconnection types", the method specifically includes the following steps:

d. and when the screening strategy is a third screening strategy, acquiring at least one alternative computing resource combination which is prepared for the interconnection type in advance.

E. and determining a target computing resource combination from the at least one alternative computing resource combination.

The third filtering strategy is a static strategy.

In an example, the obtaining at least one candidate computing resource combination in the step d may include: determining the amount of computing resources required by the task to be scheduled; at least one alternative combination of computing resources previously provided for the number of computing resources and the interconnection type is obtained. Wherein the number of computing resources in each of the at least one candidate computing resource combination is equal to the number of computing resources.

The at least one alternative combination of computing resources may be manually equipped, for example: the user prepares at least one alternative computing resource combination with better performance aiming at the computing resource quantity and the interconnection type according to the software and hardware condition or the historical operation condition.

In a specific application scenario, the computing clusters may be heterogeneous computing clusters, and accordingly, the computing resources are heterogeneous resources. Heterogeneous computing (Heterogeneous Computing), which refers primarily to the manner of computing in systems composed of computing units using different types of instruction sets, different architectures. The heterogeneous computing cluster can comprise a plurality of computing nodes, each computing node comprises a plurality of heterogeneous resources, and the heterogeneous resources can comprise a main processing unit and a co-processing unit. In practice, the number of co-processing units on each compute node is typically multiple. The main processing unit may be a CPU; the co-processing unit may be a GPU or FPGA.

Considering that in heterogeneous computing clusters, the primary impact on task processing speed is the data transfer rate between co-processing units. In the following embodiments, the heterogeneous resource is referred to as a co-processing unit. The task scheduling scheme for heterogeneous computing clusters is described in detail in connection with fig. 1d, 3 and 4.

Fig. 1d is a schematic structural diagram of a scheduling system according to an embodiment of the present application. As shown in fig. 1d, the scheduling system includes: heterogeneous computing clusters; the heterogeneous computing cluster comprises: a scheduler and a plurality of computing nodes;

Fig. 3 shows a flow chart of a task scheduling method according to an embodiment of the present application. As shown in fig. 3, the method includes:

301. and determining the heterogeneous resource combination from each computing node in the heterogeneous computing cluster according to the task to be scheduled.

302. And determining the target heterogeneous resource combination from the determined heterogeneous resource combinations according to the runtime information of the heterogeneous resource combinations.

303. And dispatching the task to be dispatched to a target computing node where the target heterogeneous resource combination is located, so that the target computing node is distributed to the target heterogeneous resource combination for processing.

Specific implementations of the above 301, 302 and 303 may refer to corresponding matters in the above embodiments, and are not described herein.

Further, the method further comprises the following steps:

304. And determining communication paths required by each heterogeneous resource combination to process the task to be scheduled.

305. And aiming at each heterogeneous resource combination, acquiring a data transmission performance index between two heterogeneous resources needing to be communicated on a communication path required by the heterogeneous resource combination to process the task to be scheduled.

The data transmission performance index is obtained by testing when the heterogeneous computing cluster operates.

Wherein, the runtime information of each heterogeneous resource combination comprises: the method processes the data transmission performance index between two heterogeneous resources needing to be communicated on the communication path required by the task to be scheduled.

In an example, the "determining the target heterogeneous resource combination from the determined heterogeneous resource combinations according to the runtime information of each heterogeneous resource combination" in the above 302 may be implemented by:

3021a, summing up the data transmission performance indexes of the two heterogeneous resources to be communicated on the communication path required by the heterogeneous resource combinations to process the task to be scheduled, so as to obtain the index sum corresponding to each heterogeneous resource combination.

3022A, determining a target heterogeneous resource combination from the determined heterogeneous resource combinations according to the index sum corresponding to each heterogeneous resource combination.

In another example, the "determining the target heterogeneous resource combination from the determined heterogeneous resource combinations according to the runtime information of each heterogeneous resource combination" in the above 302 may be implemented specifically by the following steps:

3021b, determining, for each heterogeneous resource combination, a worst data transmission performance index of data transmission performance indexes between two heterogeneous resources to be communicated on a communication path required by the heterogeneous resource combination to process the task to be scheduled.

3022B, determining the target heterogeneous resource combination from the determined heterogeneous resource combinations according to the worst data transmission performance index corresponding to each heterogeneous resource combination.

Accordingly, in the above 302, "determining the target heterogeneous resource combination from the determined heterogeneous resource combinations according to the runtime information of each available heterogeneous resource combination", specifically: and when the screening strategy is a first screening strategy or a second screening strategy, determining a target heterogeneous resource combination from the determined heterogeneous resource combinations according to the screening strategy and the runtime information of each heterogeneous resource combination.

In practical application, when the screening policy is the first screening policy (i.e. the rule of maximum bandwidth sum), determining a target heterogeneous resource combination from the determined heterogeneous resource combinations according to the screening policy and runtime information of each heterogeneous resource combination, wherein the method specifically comprises the following steps: steps 3021a and 3022a described above; when the screening policy is the second screening policy (i.e. the worst bandwidth optimal policy), determining a target heterogeneous resource combination from the determined heterogeneous resource combinations according to the screening policy and runtime information of each heterogeneous resource combination, wherein the method specifically comprises the following steps: steps 3021b and 3022b described above.

Further, the method further comprises the following steps:

d. and when the screening strategy is a third screening strategy, acquiring at least one alternative heterogeneous resource combination.

E. and determining a target heterogeneous resource combination from the at least one alternative heterogeneous resource combination.

The third filtering strategy is a static strategy.

In an example, in the step d, obtaining at least one alternative heterogeneous resource combination may include: determining the number of heterogeneous resources required by the scheduling task; at least one alternative heterogeneous resource combination which is prepared in advance for the heterogeneous resource quantity is acquired.

Wherein the number of heterogeneous resources in each of the at least one alternative heterogeneous resource combination is equal to the number of heterogeneous resources.

In another example, the obtaining at least one alternative heterogeneous resource combination in the step d may include: determining the processing requirement of the task to be scheduled, wherein the processing requirement comprises the following steps: when the task to be scheduled is processed, the number of the heterogeneous resources required and the interconnection types required among the heterogeneous resources are processed; at least one alternative heterogeneous resource combination which is prepared in advance for the heterogeneous resource quantity and the interconnection type is obtained.

The at least one alternative heterogeneous resource combination may be manually provided, for example: the user prepares at least one alternative heterogeneous resource combination with better performance aiming at the heterogeneous resource quantity and the interconnection type according to the software and hardware condition or the historical operation condition.

Further, the method further comprises the following steps:

306. Determining the processing requirement of the task to be scheduled, wherein the processing requirement comprises the following steps: and when the task to be scheduled is processed, the interconnection types required among the heterogeneous resources are processed.

In 305, the "for each heterogeneous resource combination, obtain the data transmission performance index between two heterogeneous resources that need to be communicated on the communication path required by the heterogeneous resource combination to process the task to be scheduled" specifically includes: and aiming at each heterogeneous resource combination, acquiring a data transmission performance index corresponding to the interconnection type between two heterogeneous resources needing to be communicated on a communication path required by the heterogeneous resource combination to process the task to be scheduled.

Further, the method further comprises the following steps:

307. after a heterogeneous computing cluster is started and a communication mode corresponding to a first interconnection type of the heterogeneous computing cluster is started, testing interconnection bandwidth between any two heterogeneous resources on each computing node in the heterogeneous computing cluster, and recording the interconnection bandwidth as a data transmission performance index corresponding to the first interconnection type between the any two heterogeneous resources.

308. And after starting the heterogeneous computing cluster and closing a communication mode corresponding to a first interconnection type of the heterogeneous computing cluster, testing interconnection bandwidth between any two heterogeneous resources on each computing node in the heterogeneous computing cluster, and recording as a data transmission performance index corresponding to a second interconnection type between the any two heterogeneous resources.

Further, the first interconnection type is a point-to-point interconnection type; the second interconnect type is a non-point-to-point interconnect type.

Further, in the above-mentioned step 306, "determining the processing requirement of the task to be scheduled", the following steps may be specifically adopted to implement:

3061. and inquiring whether the application program corresponding to the task to be scheduled is the application program which has been operated historically or not according to the operation log of the heterogeneous computing cluster.

3062. If the application program corresponding to the task to be scheduled is the application program which has been operated historically, determining the interconnection type required among heterogeneous resources when the task to be scheduled is processed according to the operation log.

Further, in the above 3062, "determining, according to the running log, the type of interconnection required between heterogeneous resources when the task to be scheduled is processed", the method may specifically be implemented by the following steps:

S31, determining whether the application program uses a data transmission channel corresponding to the first interconnection type in the historical operation according to the operation log.

S32, if the application program uses a data transmission channel corresponding to a first interconnection type in the operation of history, determining that the interconnection type is the first interconnection type.

Further, in the above 3062, "determining, according to the running log, a type of interconnection required between heterogeneous resources when the task to be scheduled is processed", further includes:

and S33, if the application program does not use the data transmission channel corresponding to the first interconnection type in the historical processing, determining that the interconnection type is the second interconnection type.

Further, the "determining the processing requirement of the task to be scheduled" in the above-mentioned 306 further includes:

3063. If the application program corresponding to the task to be scheduled is not the application program which has been operated historically, the interconnection type is defaulted to be a first interconnection type.

Further, the heterogeneous computing cluster comprises a plurality of computing nodes; the plurality of computing nodes comprises a first computing node;

in the above 301, "determining a heterogeneous resource combination from a first computing node in a heterogeneous computing cluster according to a task to be scheduled", the method specifically may be implemented by the following steps:

3011. Determining the number of heterogeneous resources required by the task to be scheduled;

3012. And determining available heterogeneous resources on the first computing node from a plurality of heterogeneous resources on the first computing node.

3013. Determining at least one heterogeneous resource combination from available heterogeneous resources on the first computing node when the number of the available heterogeneous resources on the first computing node is greater than or equal to the number of the heterogeneous resources; the number of heterogeneous resources in the heterogeneous resource combination is equal to the number of heterogeneous resources.

In 3013, if the number of available heterogeneous resources on the first computing node is less than the number of heterogeneous resources, the first computing node is ignored.

The task scheduling scheme provided by the embodiment of the present application will be described by way of example with reference to fig. 1 a:

1. The GPU bandwidth tester 91 (GPU bandwidth analyzer) tests the interconnection bandwidths corresponding to the point-to-point P2P interconnection type and the non-point-to-point interconnection type between GPUs, and records and saves the GPU ids and the interconnection bandwidths between GPUs. For example, 0_1_p2p:25 represents that the data transmission performance index corresponding to the point-to-point p2p interconnection type between GPU0 and GPU1 is 25G; and 0_1_nop2p:9 represents that the data transmission performance index corresponding to the non-p 2p interconnection type between GPU0 and GPU1 is 9G.

2. The node manager 93 reports the GPU number of the present computing node 94 and the obtained interconnection bandwidth as node information to the scheduler 95.

3. And submitting the Pod corresponding to the task to be scheduled by the user, and designating the number of GPU cards required by the task to be processed, for example, the number of cards is 2.

4. The dispatcher 95 queries the system call stack 97 (App historical Repo) whether the application corresponding to the task to be dispatched is a historically submitted application, and if so, determines whether the application uses P2P. If P2P is used, selecting according to the rule of P2P, otherwise selecting according to the rule of non-P2P; if not submitted, the rule selection is performed according to P2P by default.

5. When the scheduler 95 finds the appropriate compute node 94, it delegates the binding method of the GPU topology scheduling extender 96 (GPU topo scheduler extender) to bind the compute node to the pod, where the extender needs to do two things: firstly, recording a selection result into information of an application, such as recording an allocation record of a target GPU combination in an annotation; and secondly, a system interface (for example, a Kubernetes system is taken as an example, and the system interface is called as a Kubernetes API) to bind the computing node and the pod.

6. GPU plugin 92 creates Pod: modifying the original deployment configuration: the command to run is modified from "python app1.Py" to gpu-analyzer 'python app1.Py'; adding an initialization container containing the gpu-analyzer, copying the gpu-analyzer to a shared directory with the application container, and then exiting; the actual work of the GPU-analyzer 'python app1.Py' is to detect the way app1.Py code uses P2P or non-P2P for the communication code between GPUs and report this information to App historical Repo. This information is used to determine, at a scheduled time in a subsequent process, whether the corresponding application uses a communication scheme corresponding to the P2P interconnect type.

In this embodiment, qualitative distinction is performed according to the bandwidth detection measurement value between GPUs during operation, rather than according to the physical topology relationship; screening a worst index optimal strategy taking a barrel theory as a core, and providing configurability; a feedback mechanism is provided to record whether the application uses a P2P based scheduling mode.

According to the scheme, the GPU affinity is scheduled on the ground in a real scene through the self-adaptive scheduling of detecting the bandwidth and various screening strategies and combining the application characteristics during running. In the case of using affinity scheduling, the processing speed can be improved by 10% or more than that without using affinity scheduling.

What needs to be explained here is: details of each step in the method provided in the embodiment of the present application may be referred to corresponding details in the above embodiment, which are not described herein. In addition, the method provided in the embodiment of the present application may further include other part or all of the steps in the above embodiments, and specific reference may be made to the corresponding content of the above embodiments, which is not repeated herein.

The scheduler is used for: acquiring a task to be scheduled and a processing requirement thereof; wherein the processing requirements include: when the task to be scheduled is processed, the interconnection types required among heterogeneous resources are processed; combining the interconnection types, and determining a target heterogeneous resource combination from computing nodes in the heterogeneous computing cluster; and dispatching the task to be dispatched to a target computing node where the target heterogeneous resource combination is located in the heterogeneous computing cluster, so that the target computing node distributes the task to be dispatched to the target heterogeneous resource combination for processing.

Wherein each computing node may include a plurality of heterogeneous resources thereon. Each computing node may be a physical machine or virtual machine (e.g., a virtual machine on a cloud computing platform). The scheduler may be a physical machine or a virtual machine.

Fig. 4 shows a flow chart of a task scheduling method according to an embodiment of the present application. As shown in fig. 4, the method includes:

401. And acquiring tasks to be scheduled and processing requirements thereof.

Wherein the processing requirements include: and when the task to be scheduled is processed, the interconnection types required among the heterogeneous resources are processed.

402. And combining the interconnection types, and determining the target heterogeneous resource combination from the computing nodes in the heterogeneous computing cluster.

403. And dispatching the task to be dispatched to a target computing node where the target heterogeneous resource combination is located in the heterogeneous computing cluster, so that the target computing node distributes the task to be dispatched to the target heterogeneous resource combination for processing.

The specific implementation of 401, 402 and 403 may be referred to the corresponding content in each embodiment, and will not be repeated here.

Further, in the foregoing 402, "determining the target heterogeneous resource combination from the computing nodes in the heterogeneous computing cluster by combining the interconnection type", the method specifically may be implemented by the following steps:

4021. And determining the heterogeneous resource combination from each computing node in the heterogeneous computing cluster according to the task to be scheduled.

4022. And determining the target heterogeneous resource combination from the determined heterogeneous resource combinations according to the runtime information of the heterogeneous resource combinations.

The runtime information is detected when the heterogeneous computing cluster runs in a communication mode corresponding to the interconnection type.

Further, the method may further include:

404. And determining communication paths required by each heterogeneous resource combination to process the task to be scheduled.

405. And aiming at each heterogeneous resource combination, acquiring a data transmission performance index corresponding to the interconnection type between two heterogeneous resources needing to be communicated on a communication path required by the heterogeneous resource combination to process the task to be scheduled.

Wherein, the runtime information of each heterogeneous resource combination comprises: the method processes the data transmission performance index corresponding to the interconnection type between two heterogeneous resources needing to be communicated on a communication path required by the task to be scheduled.

In specific implementation, the task to be scheduled is specifically an image recognition task to be scheduled; the heterogeneous resource is specifically a graphics processor GPU. In this way, the processing speed of the image recognition task can be accelerated by the GPU.

In an example, the task scheduling method specifically includes:

701. and determining the GPU combination of the graphics processor from all computing nodes in the heterogeneous computing cluster according to the image recognition task to be scheduled.

702. And determining a target GPU combination from the determined GPU combinations according to the runtime information of each GPU combination.

703. And dispatching the image recognition task to a target computing node where the target GPU combination is located, so that the target computing node is distributed to the target GPU combination for acceleration processing.

In this embodiment, specific implementations of 701, 702, and 703 may be referred to the corresponding matters in the foregoing embodiments, which are not repeated herein.

In another example, the task scheduling method specifically includes:

801. and acquiring an image identification task to be scheduled and processing requirements thereof.

Wherein the processing requirements include: and when the image recognition task is processed, the interconnection type required between the GPUs is realized.

802. And combining the interconnection types, and determining the target GPU combination from the calculation nodes in the heterogeneous calculation cluster.

803. And dispatching the image recognition task to a target computing node where the target GPU combination is located in the heterogeneous computing cluster, so that the target computing node distributes the task to be dispatched to the target GPU combination for acceleration processing.

In this embodiment, specific implementations of the foregoing 801, 802, and 803 may refer to corresponding matters in the foregoing embodiments, which are not repeated herein.

Fig. 5 shows a block diagram of a task scheduling device according to an embodiment of the present application. As shown in fig. 5, the apparatus includes:

The first determining module 501 is configured to determine, according to a task to be scheduled, a computing resource combination from each computing node in the computing cluster;

a second determining module 502, configured to determine, according to runtime information of each computing resource combination, a target computing resource combination from the determined computing resource combinations;

And the first scheduling module 503 is configured to schedule the task to be scheduled to a target computing node where the target computing resource combination is located, so that the target computing node allocates the task to the target computing resource combination for processing.

Further, the device further comprises:

A third determining module, configured to determine a communication path required by each computing resource combination to process the task to be scheduled;

the first acquisition module is used for acquiring data transmission performance indexes between two computing resources needing to be communicated on a communication path required by each computing resource combination for processing the task to be scheduled;

Wherein, the runtime information of each computing resource combination comprises: the method processes the data transmission performance index between two computing resources needing to be communicated on the communication path required by the task to be scheduled.

Further, the third determining module is further configured to: determining the processing requirement of the task to be scheduled, wherein the processing requirement comprises the following steps: when the task to be scheduled is processed, calculating the interconnection type required among the resources;

correspondingly, the first acquisition module is specifically configured to: and aiming at each computing resource combination, acquiring a data transmission performance index corresponding to the interconnection type between two computing resources needing to be communicated on a communication path required by the computing resource combination to process the task to be scheduled.

Further, the device further comprises:

And the first test module is used for testing the interconnection bandwidth between any two computing resources on each computing node in the computing cluster after starting the computing cluster and starting a communication mode corresponding to a first interconnection type of the computing cluster so as to record the data transmission performance index corresponding to the first interconnection type between the any two computing resources.

Further, the first test module is further configured to test an interconnection bandwidth between any two computing resources on each computing node in the computing cluster after the computing cluster is started and a communication mode corresponding to a first interconnection type of the computing cluster is closed, so as to record as a data transmission performance index corresponding to a second interconnection type between the any two computing resources.

Further, the device further comprises:

the fourth acquisition module is used for acquiring a screening strategy configured by a user for the task to be scheduled;

The second determining module 502 is specifically configured to: and when the screening strategy is the first screening strategy or the second screening strategy, determining a target computing resource combination from the determined computing resource combinations according to the screening strategy and the runtime information of each computing resource combination.

Further, the device further comprises:

A fifth determining module, configured to obtain at least one candidate computing resource combination when the screening policy is the third screening policy; and determining a target computing resource combination from the at least one alternative computing resource combination.

What needs to be explained here is: the task scheduling device provided in the foregoing embodiments may implement the technical solutions described in the foregoing method embodiments, and the specific implementation principles and beneficial effects of the foregoing modules or units may be referred to the corresponding content in the foregoing method embodiments, which are not described herein again.

Fig. 6 shows a block diagram of a task scheduling device according to an embodiment of the present application. As shown in fig. 6, the apparatus includes:

The second obtaining module 601 is configured to obtain a task to be scheduled and a processing requirement thereof; wherein the processing requirements include: when the task to be scheduled is processed, calculating the interconnection type required among the resources;

A fourth determining module 602, configured to determine, in combination with the interconnection type, a target computing resource combination from computing nodes in the computing cluster;

And the second scheduling module 603 is configured to schedule the task to be scheduled to a target computing node where the target computing resource is located in the computing cluster, so that the target computing node distributes the task to be scheduled to the target computing resource combination for processing.

Further, the fourth determining module 602 is specifically configured to determine, according to the task to be scheduled, a computing resource combination from each computing node in the computing cluster; and determining a target computing resource combination from the determined computing resource combinations according to the runtime information of each computing resource combination.

Further, the device further comprises:

Further, the apparatus may further include:

And a fifth determining module, configured to determine a communication path required by each computing resource combination to process the task to be scheduled.

And the third acquisition module is used for acquiring data transmission performance indexes corresponding to the interconnection types between two computing resources needing to be communicated on a communication path required by each computing resource combination for processing the task to be scheduled.

Further, the device further comprises:

and the fourth acquisition module is used for acquiring a screening strategy configured by a user for the task to be scheduled.

The fourth determining module 602 is specifically configured to: when the screening strategy is the first screening strategy or the second screening strategy, determining a target computing resource combination from the determined computing resource combinations according to the screening strategy and the data transmission performance index corresponding to the interconnection type among computing resources in each computing resource combination.

Further, the fourth determining module 602 is specifically configured to: when the screening strategy is a third screening strategy, acquiring at least one alternative computing resource combination which is prepared for the interconnection type in advance; and determining a target computing resource combination from the at least one alternative computing resource combination.

the first determining module 501 is configured to determine, according to a task to be scheduled, a heterogeneous resource combination from each computing node in the heterogeneous computing cluster;

A second determining module 502, configured to determine, according to runtime information of each heterogeneous resource combination, a target heterogeneous resource combination from the determined heterogeneous resource combinations;

And the first scheduling module 503 is configured to schedule the task to be scheduled to a target computing node where the target heterogeneous resource combination is located, so that the target computing node allocates the task to the target heterogeneous resource combination for processing.

The second obtaining module 601 is configured to obtain a task to be scheduled and a processing requirement thereof.

And a fourth determining module 602, configured to determine, in combination with the interconnection type, a target heterogeneous resource combination from computing nodes in the heterogeneous computing cluster.

And the second scheduling module 603 is configured to schedule the task to be scheduled to a target computing node where the target heterogeneous resource combination is located in the heterogeneous computing cluster, so that the target computing node distributes the task to be scheduled to the target heterogeneous resource combination for processing.

Fig. 7 is a schematic structural diagram of an electronic device according to an embodiment of the present application. As shown in fig. 7, the electronic device includes a memory 1101 and a processor 1102. The memory 1101 may be configured to store various other data to support operations on the electronic device. Examples of such data include instructions for any application or method operating on an electronic device. The memory 1101 may be implemented by any type of volatile or non-volatile memory device or combination thereof, such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disk.

The memory 1101 is configured to store a program;

the processor 1102 is coupled to the memory 1101, and is configured to execute the program stored in the memory 1101, so as to implement the task scheduling method provided in the above method embodiments.

Further, as shown in fig. 7, the electronic device further includes: communication component 1103, display 1104, power component 1105, audio component 1106, and other components. Only some of the components are schematically shown in fig. 7, which does not mean that the electronic device only comprises the components shown in fig. 7.

Accordingly, the embodiments of the present application also provide a computer readable storage medium storing a computer program, where the computer program when executed by a computer can implement the steps or functions of the task scheduling method provided in the foregoing method embodiments.

The apparatus embodiments described above are merely illustrative, wherein the elements illustrated as separate elements may or may not be physically separate, and the elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art will understand and implement the present invention without undue burden.

From the above description of embodiments, it will be apparent to those skilled in the art that the embodiments may be implemented by means of software plus necessary general hardware platforms, or of course by means of hardware. Based on this understanding, the foregoing technical solution may be embodied essentially or in a part contributing to the prior art in the form of a software product, which may be stored in a computer readable storage medium, such as ROM/RAM, a magnetic disk, an optical disk, etc., including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method described in the respective embodiments or some parts of the embodiments.

Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present application, and are not limiting; although the application has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for task scheduling, comprising:

Scheduling the task to be scheduled to a target computing node where the target heterogeneous resource combination is located, so that the target computing node is allocated to the target heterogeneous resource combination for processing;

Determining communication paths required by each heterogeneous resource combination to process the task to be scheduled;

Aiming at each heterogeneous resource combination, acquiring a data transmission performance index between two heterogeneous resources needing to be communicated on a communication path required by the heterogeneous resource combination for processing the task to be scheduled; the data transmission performance index is obtained by testing when the heterogeneous computing cluster operates;

Wherein, the runtime information of each heterogeneous resource combination comprises: the method processes the data transmission performance index between two heterogeneous resources needing to be communicated on a communication path required by the task to be scheduled;

Determining the processing requirement of the task to be scheduled, wherein the processing requirement comprises the following steps: when the task to be scheduled is processed, the interconnection types required among heterogeneous resources are processed;

For each heterogeneous resource combination, acquiring a data transmission performance index between two heterogeneous resources needing to be communicated on a communication path required by the heterogeneous resource combination to process the task to be scheduled, wherein the data transmission performance index comprises the following steps:

And aiming at each heterogeneous resource combination, acquiring a data transmission performance index corresponding to the interconnection type between two heterogeneous resources needing to be communicated on a communication path required by the heterogeneous resource combination to process the task to be scheduled.

2. The method of claim 1, wherein determining a target heterogeneous resource combination from the determined heterogeneous resource combinations based on runtime information for each heterogeneous resource combination, comprises:

summing the data transmission performance indexes between two heterogeneous resources needing to be communicated on a communication path required by each heterogeneous resource combination to process the task to be scheduled, so as to obtain index sum corresponding to each heterogeneous resource combination;

And determining the target heterogeneous resource combination from the determined heterogeneous resource combinations according to the index sum corresponding to each heterogeneous resource combination.

3. The method of claim 1, wherein determining a target heterogeneous resource combination from the determined heterogeneous resource combinations based on runtime information for each heterogeneous resource combination, comprises:

Determining the worst data transmission performance index in the data transmission performance indexes between two heterogeneous resources needing to be communicated on a communication path required by each heterogeneous resource combination for processing the task to be scheduled;

and determining a target heterogeneous resource combination from the determined heterogeneous resource combinations according to the worst data transmission performance indexes corresponding to the heterogeneous resource combinations.

4. The method as recited in claim 1, further comprising:

After a heterogeneous computing cluster is started and a communication mode corresponding to a first interconnection type of the heterogeneous computing cluster is started, testing interconnection bandwidth between any two heterogeneous resources on each computing node in the heterogeneous computing cluster, and recording the interconnection bandwidth as a data transmission performance index corresponding to the first interconnection type between the any two heterogeneous resources.

5. The method as recited in claim 4, further comprising:

And after starting the heterogeneous computing cluster and closing a communication mode corresponding to a first interconnection type of the heterogeneous computing cluster, testing interconnection bandwidth between any two heterogeneous resources on each computing node in the heterogeneous computing cluster, and recording as a data transmission performance index corresponding to a second interconnection type between the any two heterogeneous resources.

6. The method of claim 5, wherein the first interconnect type is a point-to-point interconnect type; the second interconnect type is a non-point-to-point interconnect type.

7. The method of claim 1, wherein determining the processing requirements of the task to be scheduled comprises:

Inquiring whether the application program corresponding to the task to be scheduled is a historically operated application program or not according to the running log of the heterogeneous computing cluster;

If the application program corresponding to the task to be scheduled is the application program which has been operated historically, determining the interconnection type required among heterogeneous resources when the task to be scheduled is processed according to the operation log.

8. The method of claim 7, wherein determining the type of interconnection required between heterogeneous resources to process the task to be scheduled based on the travel log comprises:

determining whether the application program uses a data transmission channel corresponding to the first interconnection type in the historically running process according to the running log;

and if the application program uses the data transmission channel corresponding to the first interconnection type in the operation of history, determining that the interconnection type is the first interconnection type.

9. The method of claim 8, wherein determining the type of interconnection required between heterogeneous resources when processing the task to be scheduled based on the travel log, further comprises:

And if the application program does not use the data transmission channel corresponding to the first interconnection type in the historical processing, determining that the interconnection type is the second interconnection type.

10. The method of claim 7, wherein determining the processing requirements of the task to be scheduled further comprises:

if the application program corresponding to the task to be scheduled is not the application program which has been operated historically, the interconnection type is defaulted to be a first interconnection type.

11. A method according to any one of claims 1 to 3, wherein the heterogeneous computing cluster comprises a plurality of computing nodes therein; the plurality of computing nodes comprises a first computing node;

According to the task to be scheduled, determining a heterogeneous resource combination from a first computing node in a heterogeneous computing cluster, including:

determining the number of heterogeneous resources required by the task to be scheduled;

Determining available heterogeneous resources on the first computing node from a plurality of heterogeneous resources on the first computing node;

Determining at least one heterogeneous resource combination from available heterogeneous resources on the first computing node when the number of the available heterogeneous resources on the first computing node is greater than or equal to the number of the heterogeneous resources; the number of heterogeneous resources in the heterogeneous resource combination is equal to the number of heterogeneous resources.

12. A method according to any one of claims 1 to 3, further comprising:

acquiring a screening strategy configured by a user for the task to be scheduled;

According to the runtime information of each heterogeneous resource combination, determining a target heterogeneous resource combination from the determined heterogeneous resource combinations, including:

And when the screening strategy is a first screening strategy or a second screening strategy, determining a target heterogeneous resource combination from the determined heterogeneous resource combinations according to the screening strategy and the runtime information of each heterogeneous resource combination.

13. The method as recited in claim 12, further comprising:

when the screening strategy is a third screening strategy, at least one alternative heterogeneous resource combination is obtained;

And determining a target heterogeneous resource combination from the at least one alternative heterogeneous resource combination.

14. The method of claim 13, wherein obtaining at least one alternative heterogeneous resource combination comprises:

determining the processing requirement of the task to be scheduled, wherein the processing requirement comprises the following steps: when the task to be scheduled is processed, the number of the heterogeneous resources required and the interconnection types required among the heterogeneous resources are processed;

At least one alternative heterogeneous resource combination which is prepared in advance for the heterogeneous resource quantity and the interconnection type is obtained.

15. A method for task scheduling, comprising:

scheduling the task to be scheduled to a target computing node where the target heterogeneous resource combination is located in the heterogeneous computing cluster, so that the target computing node distributes the task to be scheduled to the target heterogeneous resource combination for processing; combining the interconnection types, determining a target heterogeneous resource combination from computing nodes in the heterogeneous computing cluster, wherein the method comprises the following steps:

the runtime information is detected when the heterogeneous computing cluster runs in a communication mode corresponding to the interconnection type;

aiming at each heterogeneous resource combination, acquiring a data transmission performance index corresponding to the interconnection type between two heterogeneous resources needing to be communicated on a communication path required by the heterogeneous resource combination for processing the task to be scheduled;

16. A scheduling system, comprising: heterogeneous computing clusters; the heterogeneous computing cluster comprises: a scheduler and a plurality of computing nodes;

The scheduler is used for: determining a heterogeneous resource combination from each computing node in the heterogeneous computing cluster according to the task to be scheduled; determining a target heterogeneous resource combination from the determined heterogeneous resource combinations according to the runtime information of each heterogeneous resource combination; scheduling the task to be scheduled to a target computing node where the target heterogeneous resource combination is located, so that the target computing node is allocated to the target heterogeneous resource combination for processing; determining communication paths required by each heterogeneous resource combination to process the task to be scheduled; aiming at each heterogeneous resource combination, acquiring a data transmission performance index between two heterogeneous resources needing to be communicated on a communication path required by the heterogeneous resource combination for processing the task to be scheduled; the data transmission performance index is obtained by testing when the heterogeneous computing cluster operates; wherein, the runtime information of each heterogeneous resource combination comprises: the method processes the data transmission performance index between two heterogeneous resources needing to be communicated on a communication path required by the task to be scheduled; determining the processing requirement of the task to be scheduled, wherein the processing requirement comprises the following steps: when the task to be scheduled is processed, the interconnection types required among heterogeneous resources are processed;

The method for acquiring the data transmission performance index between two heterogeneous resources to be communicated on a communication path required by each heterogeneous resource combination for processing the task to be scheduled comprises the following steps: and aiming at each heterogeneous resource combination, acquiring a data transmission performance index corresponding to the interconnection type between two heterogeneous resources needing to be communicated on a communication path required by the heterogeneous resource combination to process the task to be scheduled.

17. A scheduling system, comprising: heterogeneous computing clusters; the heterogeneous computing cluster comprises: a scheduler and a plurality of computing nodes;

The scheduler is used for: acquiring a task to be scheduled and a processing requirement thereof; wherein the processing requirements include: when the task to be scheduled is processed, the interconnection types required among heterogeneous resources are processed; determining a target heterogeneous resource combination from computing nodes in the heterogeneous computing cluster according to the interconnection type; scheduling the task to be scheduled to a target computing node where the target heterogeneous resource combination is located in the heterogeneous computing cluster, so that the target computing node distributes the task to be scheduled to the target heterogeneous resource combination for processing; combining the interconnection types, determining a target heterogeneous resource combination from computing nodes in the heterogeneous computing cluster, wherein the method comprises the following steps: determining a heterogeneous resource combination from each computing node in the heterogeneous computing cluster according to the task to be scheduled; determining a target heterogeneous resource combination from the determined heterogeneous resource combinations according to the runtime information of each heterogeneous resource combination; the runtime information is detected when the heterogeneous computing cluster runs in a communication mode corresponding to the interconnection type; determining communication paths required by each heterogeneous resource combination to process the task to be scheduled; aiming at each heterogeneous resource combination, acquiring a data transmission performance index corresponding to the interconnection type between two heterogeneous resources needing to be communicated on a communication path required by the heterogeneous resource combination for processing the task to be scheduled; wherein, the runtime information of each heterogeneous resource combination comprises: the method processes the data transmission performance index corresponding to the interconnection type between two heterogeneous resources needing to be communicated on a communication path required by the task to be scheduled.

18. A method for task scheduling, comprising:

Scheduling the task to be scheduled to a target computing node where the target computing resource combination is located, so that the target computing node distributes the task to the target computing resource combination for processing;

Determining a communication path required by each computing resource combination to process the task to be scheduled;

Aiming at each computing resource combination, acquiring a data transmission performance index between two computing resources needing to be communicated on a communication path required by the computing resource combination for processing the task to be scheduled; the data transmission performance index is obtained by testing when the computing cluster runs;

wherein, the runtime information of each computing resource combination comprises: the method processes the data transmission performance index between two computing resources needing to be communicated on a communication path required by the task to be scheduled;

Determining the processing requirement of the task to be scheduled, wherein the processing requirement comprises the following steps: when the task to be scheduled is processed, calculating the interconnection type required among the resources;

For each computing resource combination, acquiring a data transmission performance index between two computing resources needing to be communicated on a communication path required by the computing resource combination to process the task to be scheduled, wherein the data transmission performance index comprises the following steps:

And aiming at each computing resource combination, acquiring a data transmission performance index corresponding to the interconnection type between two computing resources needing to be communicated on a communication path required by the computing resource combination to process the task to be scheduled.

19. A method for task scheduling, comprising:

Scheduling the task to be scheduled to a target computing node where the target computing resource is located in the computing cluster, so that the target computing node distributes the task to be scheduled to the target computing resource combination for processing;

combining the interconnection types, determining a target computing resource combination from computing nodes in a computing cluster, wherein the method comprises the following steps:

the runtime information is detected when the computing cluster runs in a communication mode corresponding to the interconnection type;

Aiming at each computing resource combination, acquiring a data transmission performance index corresponding to the interconnection type between two computing resources needing to be communicated on a communication path required by the computing resource combination for processing the task to be scheduled;

20. The method of claim 19, wherein obtaining the processing requirements of the task to be scheduled comprises:

Inquiring whether the application program corresponding to the task to be scheduled is a historically operated application program or not according to the operation log of the computing cluster;

And if the application program corresponding to the task to be scheduled is the application program which has been operated historically, determining the interconnection type required between the computing resources when the task to be scheduled is processed according to the operation log.

21. An electronic device, comprising: a memory and a processor, wherein,

The memory is used for storing programs;

22. An electronic device, comprising: a memory and a processor, wherein,

The memory is used for storing programs;

Scheduling the task to be scheduled to a target computing node where the target heterogeneous resource combination is located in the heterogeneous computing cluster, so that the target computing node distributes the task to be scheduled to the target heterogeneous resource combination for processing;

Combining the interconnection types, determining a target heterogeneous resource combination from computing nodes in the heterogeneous computing cluster, wherein the method comprises the following steps:

23. An electronic device, comprising: a memory and a processor, wherein,

The memory is used for storing programs;

24. An electronic device, comprising: a memory and a processor, wherein,

The memory is used for storing programs;

25. A method for task scheduling, comprising:

dispatching the image recognition task to a target computing node where the target GPU combination is located, so that the target computing node is distributed to the target GPU combination for acceleration processing;

determining a communication path required by each GPU combination to process the image recognition task;

Aiming at each GPU combination, acquiring a data transmission performance index between two GPUs needing to be communicated on a communication path required by the GPU combination for processing the image recognition task; the data transmission performance index is obtained by testing when the heterogeneous computing cluster operates;

the runtime information of each GPU combination comprises: on a communication path required by the image recognition task, the method processes data transmission performance indexes between two GPUs needing to be communicated;

Determining a processing requirement of the image recognition task, wherein the processing requirement comprises the following steps: when the image recognition task is processed, the interconnection types required among the GPUs are processed;

for each GPU combination, acquiring a data transmission performance index between two GPUs needing to communicate on a communication path required by the GPU combination for processing the image recognition task, wherein the data transmission performance index comprises the following steps:

and aiming at each GPU combination, acquiring a data transmission performance index corresponding to the interconnection type between two GPUs needing to be communicated on a communication path required by the GPU combination for processing the image recognition task.

26. A method for task scheduling, comprising:

dispatching the image recognition task to a target computing node where the target GPU combination is located in the heterogeneous computing cluster, so that the target computing node distributes the task to be dispatched to the target GPU combination for acceleration processing;

combining the interconnection types, determining a target GPU combination from computing nodes in the heterogeneous computing clusters, wherein the method comprises the following steps:

determining GPU combinations from all computing nodes in the heterogeneous computing clusters according to the image recognition task;

Aiming at each GPU combination, acquiring a data transmission performance index corresponding to the interconnection type between two GPUs needing to be communicated on a communication path required by the GPU combination for processing the image recognition task;

The runtime information of each GPU combination comprises: on a communication path required by the image recognition task, the method processes data transmission performance indexes corresponding to the interconnection types between two GPUs needing to be communicated.

27. An electronic device, comprising: a memory and a processor, wherein,

The memory is used for storing programs;

28. An electronic device, comprising: a memory and a processor, wherein,

The memory is used for storing programs;