CN114996018A - Resource scheduling method, node, system, device and medium for heterogeneous computing - Google Patents
Resource scheduling method, node, system, device and medium for heterogeneous computing Download PDFInfo
- Publication number
- CN114996018A CN114996018A CN202210676039.XA CN202210676039A CN114996018A CN 114996018 A CN114996018 A CN 114996018A CN 202210676039 A CN202210676039 A CN 202210676039A CN 114996018 A CN114996018 A CN 114996018A
- Authority
- CN
- China
- Prior art keywords
- computing
- resource
- computing power
- module
- power
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/48—Program initiating; Program switching, e.g. by interrupt
- G06F9/4806—Task transfer initiation or dispatching
- G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
- G06F9/4881—Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5061—Partitioning or combining of resources
- G06F9/5077—Logical partitioning of resources; Management or configuration of virtualized resources
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Abstract
The invention discloses a resource scheduling method, a node, a system, equipment and a medium for heterogeneous computing, wherein the system comprises the following steps: the application interface abstract module is used for providing an application module resource application interactive interface so as to obtain a job task submitted by a user in a computing network; the computing resource scheduling module is used for scheduling and distributing the job tasks to virtual computing resources; the computing power routing module is used for dispatching the computing tasks to the physical computing nodes of the computing power resource dispatching module from the computing power nodes; the resource dynamic sensing module is used for sensing the resources of the whole computing power resource pool in real time and distributing the operation tasks in the virtual computing power resources based on the real-time sensing of the whole computing power resource pool; and the computing power node resource module is used for mapping the virtual computing power resource to the corresponding computing power node for execution according to the operation task. The invention carries out unified management and scheduling on scattered and heterogeneous computing resources, and realizes the efficient utilization of computing resources.
Description
Technical Field
The present invention relates to the field of communications network computing, and in particular, to a resource scheduling method, node, system, device, and medium for heterogeneous computing.
Background
With the rapid development of internet technology, the data scale grows exponentially. Continuous progress of big data technology enables data resources to become important production elements, valuable information is extracted by analyzing and processing mass data in many industries, and powerful calculation support is needed for analyzing and processing mass data.
At present, in a plurality of data processing scenes, computing resources are relatively dispersed, and processing is usually performed only by relying on computing power of a data center or a terminal, so that the ever-increasing demand of computing power on the ever-increasing data scale is difficult to meet. Therefore, the computing power distributed at the cloud, the edge and the end needs to be managed uniformly to meet the increasing computing power requirement. With the development of edge computing and distributed computing technologies in recent years, the management of computing resources is evolving from early centralized management to coordinated scheduling between cloud computing and edge computing.
In the method adopted at present, the first is a heterogeneous computing system based on a cross bus, the heterogeneous computing system is divided into a heterogeneous computing core, an off-chip storage part, a bus system, nodes and the like, each node corresponds to a heterogeneous computing center, the nodes are respectively connected in a cross mode from the longitudinal direction and the transverse direction to form the cross bus, and the advantages of the chip such as high performance and expandability are fully exerted. The second is to connect each node through a special high-speed network, process data and computing tasks submitted by users through management nodes, balance the computing equipment distributed to the computing nodes for processing, and fully utilize the high floating point operation capability of the GPU by using a CUDA or OpenGL heterogeneous programming model.
The heterogeneous computing system based on the cross bus connects different nodes through the cross bus, which means that the computing nodes which can be connected with the heterogeneous computing system are relatively centralized, and the heterogeneous computing resources which are distributed more dispersedly cannot be effectively managed. When the computing resources are expanded, the bus needs to be redesigned, and the flexibility is poor. By using a heterogeneous computing framework of a CUDA (compute unified device architecture) or OpenGL (open graphics library) heterogeneous programming model, data I/O (input/output) operation needs to be processed by a special distributed file system, a management node manages data and computing tasks submitted by a user, and once the management node fails, the system cannot continue to operate. The heterogeneous programming model based on CUDA or OpenGL is only suitable for heterogeneous computation of GPU equipment, and utilization of heterogeneous resources is limited.
Disclosure of Invention
In view of the above-mentioned shortcomings, the present invention aims to provide a resource scheduling method, node, system, device and medium for heterogeneous computing, in which the scheduling system of the present invention constructs a resource scheduling system for heterogeneous computing to uniformly manage and schedule distributed and heterogeneous computing resources, thereby realizing efficient utilization of computing resources.
In order to achieve the purpose, the invention adopts the following technical scheme to realize the purpose:
a resource scheduling method facing heterogeneous computing comprises the following steps:
acquiring a job task submitted by a user in a computing network;
and allocating the job task to the virtual computing power resource, wherein the resource dynamic sensing module is used for distributing the job task in the virtual computing power resource based on the real-time sensing of the whole computing power resource pool, and mapping the job task from the virtual computing power resource to the computing power node, so that the job task is executed in the computing power node allocated with the computing power task.
As a further improvement of the invention, the resource dynamic sensing module is used for distributing the operation tasks based on the real-time sensing of the whole computing resource pool, and the current available quantity, health condition and consumption condition of the computing resources are considered, and the real-time sensing is carried out according to the computing resource node network topological structure.
As a further improvement of the invention, the task is mapped to the computing power node from the virtual computing power resource, and the virtual computing power unit is managed and scheduled according to the current use condition of different virtual computing power units based on the mapping relation between the constructed virtual computing power unit and the computing power physical equipment.
A resource scheduling node for heterogeneous computing, comprising:
the acquiring unit is used for acquiring a job task submitted by a user in a computing network;
and the allocation unit is used for allocating the job task to the virtual computing resource pool, and the resource dynamic sensing module is used for distributing the job task based on the real-time sensing of the whole computing resource pool and mapping the job task to the computing nodes from the virtual computing resource, so that the job task is executed in the computing nodes allocated with the computing tasks.
A heterogeneous computing-oriented resource scheduling system, comprising:
the application interface abstract module is used for providing an application module resource application interactive interface so as to obtain a job task submitted by a user in a computing network;
the computing resource scheduling module is used for scheduling and distributing the job tasks to virtual computing resources;
the computing power routing module is used for dispatching the computing tasks to the physical computing nodes of the computing power resource dispatching module from the computing power nodes;
the resource dynamic sensing module is used for sensing the resources of the whole computing power resource pool in real time, distributing the job tasks in the virtual computing power resources based on the real-time sensing of the whole computing power resource pool, and mapping the virtual computing power resources to computing power nodes;
and the computing power node resource module comprises a plurality of computing power nodes and is used for mapping the virtual computing power resources to the corresponding computing power nodes for execution according to the job tasks.
As a further improvement of the present invention, in the computing power node resource module, different computing power nodes include different computing resources, each computing power node includes a service component for virtualizing the same type of computing power of the computing power node into virtual computing power, and the different computing power nodes are connected through the internet/local area network; all the computing power nodes form a computing power resource pool.
As a further improvement of the invention, in the resource dynamic sensing module, the real-time sensing of the resources of the whole computing power resource pool is to manage the available quantity of heterogeneous computing power, the current consumption condition of the computing power resources and the health state of the resources to obtain a real-time sensing result.
As a further improvement of the invention, the calculation force routing module comprises a calculation force resource node network topological structure unit, a calculation force routing control unit and a calculation force routing forwarding unit; the calculation force routing control unit controls the calculation force resource node network topological structure unit to generate a calculation force topology according to the demand of the job task on the calculation force resource and the calculation force node information, and further generates a calculation force perception new routing table; the calculation routing forwarding is used for interacting with the calculation resource scheduling module and comprises calculation routing identification and calculation routing addressing.
As a further improvement of the present invention, in the computing resource scheduling module, the job task scheduling assignment includes job task selection, computing resource assignment, log management, and redundant backup is used for control nodes and data.
As a further improvement of the invention, the application interface abstract module provides an application module resource application interaction interface comprising a data transmission interface, a resource scheduling abstract interface and a job task life cycle management interface.
As a further improvement of the invention, the system also comprises an application module, wherein the application module is arranged at the upper layer of the application interface abstraction module, and the application module is provided with a uniform interface which is connected with different applications to run on the whole resource pool.
As a further improvement of the invention, the computational power node has logical operational capability, neural network operational capability and parallel computational capability.
An electronic device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, the processor implementing the steps of the heterogeneous computing oriented resource scheduling method when executing the computer program.
A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the steps of the heterogeneous computing oriented resource scheduling method.
Compared with the prior art, the invention has the following beneficial effects:
the scheduling system of the invention manages and schedules scattered and heterogeneous computing resources in a unified way by constructing the resource scheduling system facing heterogeneous computing, thereby realizing the high-efficiency utilization of computing resources. The resource scheduling system comprises a calculation node resource module, a resource dynamic sensing module, a calculation routing module, a calculation resource scheduling module, an application interface abstract module and an application module from bottom to top. Each computational node of the computational node resource module comprises a service component which is responsible for discovering and managing the computational resources of the computational node. And the resource dynamic sensing module senses the resources of the whole computing power resource pool in real time. And the computation routing module is responsible for dispatching the computation tasks to the physical computation nodes from the abstract computation unit. The computing resource scheduling module is responsible for job task scheduling allocation, computing resource allocation, log management and the like, also comprises a tenant management function, and realizes configuration management of computing resource tenants. The application interface abstract module provides an application module resource application interactive interface, and application of computing resources is facilitated.
Drawings
FIG. 1 is a flowchart of a resource scheduling method for heterogeneous computing according to the present invention;
fig. 2 is a schematic diagram of a resource scheduling node for heterogeneous computing according to an embodiment of the present invention;
FIG. 3 is a diagram of a resource scheduling system for heterogeneous computing according to the present invention;
FIG. 4 is a schematic diagram of a mapping between a virtual computing power unit and a computing power physical device according to the present invention (taking GPU virtualization as an example);
FIG. 5 is a schematic diagram of a computational resource scheduling mechanism according to the present invention;
fig. 6 is a schematic diagram of an electronic device according to the present invention.
Detailed Description
In order to make the technical solutions of the present invention better understood, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It should be noted that the terms "first," "second," and the like in the description and claims of the present invention and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the invention described herein are capable of operation in sequences other than those illustrated or described herein. Moreover, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
The invention aims to provide a resource scheduling system for layered heterogeneous computing, which can fully exert the computing potential of heterogeneous computing resources by decoupling application and physical hardware and pooling and virtualizing the computing resources.
As shown in fig. 1, the resource scheduling method for heterogeneous computing of the present invention is applied to a computing resource scheduling module, and includes:
acquiring a job task submitted by a user in a computing network;
and allocating the job task to the virtual computing power resource, wherein the resource dynamic sensing module is used for distributing the job task in the virtual computing power resource based on the real-time sensing of the whole computing power resource pool, and mapping the job task from the virtual computing power resource to the computing power node, so that the job task is executed in the computing power node allocated with the computing power task.
The computing power resource scheduling module acquires and distributes the job tasks submitted by the users in the computing power network, and the resource dynamic sensing module senses and distributes the job tasks in the virtual computing power resources in real time based on the whole computing power resource pool, so that the distribution efficiency is improved, the job tasks are mapped to the computing power nodes from the virtual computing power resources, the high efficiency and flexibility of the use of the computing resources can be realized, and the computing power potential of the computing resources is fully exerted.
As an optional embodiment of the present invention, the resource dynamic sensing module performs real-time sensing on the job task based on the whole computational resource pool, and performs real-time sensing according to the computational resource node network topology in consideration of the currently available amount, health condition, and consumption condition of computational resources.
The real-time sensing of the resource pool based on the whole calculation capacity is beneficial to the efficient utilization of resources, and the subsequent distribution efficiency is improved.
As an optional embodiment of the present invention, the mapping of the job task from the virtual power calculation resource to the power calculation node is based on a mapping relationship between the constructed virtual power calculation unit and the power calculation physical device, and the virtual power calculation unit is managed and scheduled according to current use conditions of different virtual power calculation units.
As shown in fig. 2, the present invention further provides a resource scheduling node for heterogeneous computing, including:
the acquiring unit is used for acquiring a job task submitted by a user in a computing network;
and the allocation unit is used for allocating the job task to the virtual computing resource pool, and the resource dynamic sensing module is used for distributing the job task based on the real-time sensing of the whole computing resource pool and mapping the job task to the computing nodes from the virtual computing resource, so that the job task is executed in the computing nodes allocated with the computing tasks.
The resource scheduling node for heterogeneous computing is applied to a computing resource scheduling module, so that job task scheduling distribution is realized, and job tasks are distributed to virtual computing resources.
The third purpose of the invention is to provide a resource scheduling system facing heterogeneous computing, comprising:
the application interface abstract module is used for providing an application module resource application interactive interface so as to obtain a job task submitted by a user in a computing network;
the computing resource scheduling module is used for scheduling and distributing the job tasks to virtual computing resources;
the computing power routing module is used for dispatching the computing tasks to the physical computing nodes of the computing power resource dispatching module from the computing power nodes;
the resource dynamic sensing module is used for sensing the resources of the whole computing power resource pool in real time, distributing the job tasks in the virtual computing power resources based on the real-time sensing of the whole computing power resource pool, and mapping the virtual computing power resources to computing power nodes;
and the computing power node resource module comprises a plurality of computing power nodes and is used for mapping the virtual computing power resources to the corresponding computing power nodes for execution according to the job tasks.
The scheduling system of the invention manages and schedules scattered and heterogeneous computing resources in a unified way by constructing the resource scheduling system facing heterogeneous computing, thereby realizing the high-efficiency utilization of computing resources.
The invention is described in further detail below with reference to the figures and with reference to examples. The invention is not limited to the examples given.
As shown in fig. 3, a hierarchical resource scheduling system for heterogeneous computation is constructed, which includes, from bottom to top, a computation node resource module, a resource dynamic sensing module, a computation routing module, a computation resource scheduling module, an application interface abstraction module, and an application module. Each computational node of the computational node resource module comprises a service component which is responsible for discovering and managing the computational resources of the computational node.
And the resource dynamic sensing module senses the resources of the whole computing power resource pool in real time. And the computation routing module is responsible for dispatching the computation tasks to the physical computation nodes from the abstract computation unit. The computing resource scheduling module is responsible for job task scheduling allocation, computing resource allocation, log management and the like, also comprises a tenant management function, and realizes configuration management of computing resource tenants. The application interface abstract module provides an application module resource application interactive interface, and application of computing resources is facilitated.
By decoupling the application from the physical hardware, pooling and virtualizing computing resources, the computing potential of heterogeneous computing resources is fully exploited.
The heterogeneous computational power networking of the invention adopts a layered structure. The concrete structure is as follows:
the bottom layer is an algorithm node resource module, different algorithm nodes comprise computing resources such as a CPU (X86, ARM and the like), a GPU, an NPU, a TPU and the like, each algorithm node comprises a service component, and the service component is responsible for discovering and managing the algorithm resources of the algorithm node and virtualizes the same type of algorithm of the algorithm node into virtual algorithm. The different force nodes are connected via the internet/local area network. All the computing power nodes form a computing power resource pool.
The upper layer of the computing power node resource module is a resource dynamic sensing module and a computing power routing module, and the resource dynamic sensing module senses the resources of the whole computing power resource pool in real time, particularly manages the available quantity of heterogeneous computing power, the current consumption condition of computing power resources, the health state of the resources and the like.
The calculation force routing module: the layer is based on abstracted computational resources, comprehensively considers network conditions and computing resource conditions, and schedules computing tasks to physical computing nodes. The layer comprises functions of computing resource node network topology, computing route control, computing route forwarding and the like. And the calculation power routing control combines the demand of the job task on the calculation power resource with the calculation power node information to generate a calculation power topology, and further generates a new routing table for calculation power perception. The calculation routing forwarding comprises functions of calculation routing identification, calculation routing addressing and the like.
The upper layers of the resource dynamic sensing module and the computing power routing module are computing power resource scheduling modules, the upper layer is responsible for job task scheduling distribution, including job task selection (determining dependence conditions, job task priority, resource quota and the like), computing resource distribution (resource scheduling, resource preemption, elastic expansion and the like), log management and the like, and the upper layer also comprises a tenant management function, so that configuration management of computing resource tenants is realized. In order to improve the reliability of the framework, redundant backup is adopted for the control nodes and the data, and once the node equipment which is currently responsible for resource scheduling fails, the standby nodes take over the data.
The computing resource scheduling module is provided with an application interface abstract module at the upper layer and provides an application module resource application interaction interface, a data transmission interface, a resource scheduling abstract interface, a job task life cycle management interface and the like.
The upper layer of the application interface abstract module is an application module, and the framework provides a uniform interface, so that different applications can transparently and perceptually run on the whole resource pool.
The framework decouples the application from the physical hardware, the application calls a logical computing unit through an application interface abstraction module, and the computational resource scheduling module matches the logical computing unit to a specific physical computing device according to the actual requirements of the application. For users, the high efficiency and flexibility of the use of computing resources can be realized through the pooling and virtualization of the computing resources, and the computing potential of the computing resources is fully exerted.
In the aspect of computing power measurement, different applications have different requirements on computing power. For example, the real-time reasoning task has high requirement on network delay and relatively low requirement on computing power. In the AI training type service, the training data scale is usually large, the neural network hierarchy is complex, and the requirements on the computing capability and the storage capability are high. The computing power is considered to be classified into logical computing power, neural network computing power, parallel computing power and the like. Specifically, the computing hardware generally used for the logical operation capability is represented by a CPU, and the computing capability measurement by the TOPS is considered. Neural network computational power: the method mainly aims at AI intensive computing tasks, and is used for accelerating computing of machine learning, deep learning and the like, and computing power is measured according to floating point computing power (TFLOPS/s) and video memory size. Parallel computing power: is generally used for processing a large amount of data with uniform types, belongs to general computing power and is measured by floating point computing power (TFLOPS/s).
As an alternative of the present invention, fig. 4 is a mapping relationship between a virtual computing power unit and a computing power physical device, taking GPU as an example. And constructing a mapping relation between the virtual force computing unit and the force computing physical equipment, and managing and scheduling the virtual force computing unit in a force computing resource scheduling module according to the current use condition of different virtual force computing units. The distribution of the tasks is completed by the resource dynamic sensing module, and the distribution of the specific tasks is carried out according to the computational resource node network topological structure by considering the current available quantity, the health condition and the consumption condition of the computational resources.
As an alternative to the present invention, fig. 5 is a computer resource scheduler diagram. After a user submits a job task in a computing network, a computing resource scheduler is responsible for scheduling computing resources and distributing the job task to virtual computing resources, a computing resource manager is responsible for mapping the job task from the virtual computing resources to computing nodes, and final execution of the final job task is carried out on the computing nodes distributed with the computing tasks.
By constructing a resource scheduling system for heterogeneous computing, distributed scattered and heterogeneous computing resources are uniformly managed and scheduled, and efficient utilization of computing resources is realized.
As shown in fig. 6, the present invention provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the steps of the resource scheduling method for heterogeneous computing when executing the computer program.
The resource scheduling method facing heterogeneous computing comprises the following steps:
acquiring a job task submitted by a user in a computing network;
and allocating the operation tasks to the virtual computing power resources, wherein the operation tasks are used for distributing the operation tasks in the virtual computing power resources through a resource dynamic sensing module based on the real-time sensing of the whole computing power resource pool, and the operation tasks are mapped to the computing power nodes from the virtual computing power resources, so that the operation tasks are executed in the computing power nodes to which the computing power tasks are allocated.
The present invention also provides a computer-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, the computer program implements the steps of the resource scheduling method for heterogeneous computing.
The resource scheduling method facing heterogeneous computing comprises the following steps:
acquiring a job task submitted by a user in a computing network;
and allocating the job task to the virtual computing power resource, wherein the resource dynamic sensing module is used for distributing the job task in the virtual computing power resource based on the real-time sensing of the whole computing power resource pool, and mapping the job task from the virtual computing power resource to the computing power node, so that the job task is executed in the computing power node allocated with the computing power task.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
Finally, it should be noted that: although the present invention has been described in detail with reference to the above embodiments, it should be understood by those skilled in the art that: modifications and equivalents may be made to the embodiments of the invention without departing from the spirit and scope of the invention, which is to be covered by the claims.
Claims (14)
1. A resource scheduling method facing heterogeneous computing is characterized by comprising the following steps:
acquiring a job task submitted by a user in a computing network;
and allocating the job task to the virtual computing power resource, wherein the resource dynamic sensing module is used for distributing the job task in the virtual computing power resource based on the real-time sensing of the whole computing power resource pool, and mapping the job task from the virtual computing power resource to the computing power node, so that the job task is executed in the computing power node allocated with the computing power task.
2. The method of claim 1, wherein the resource scheduling for heterogeneous computing,
the resource dynamic sensing module is used for distributing the operation tasks based on the real-time sensing of the whole computing resource pool, considering the current available quantity, the health condition and the consumption condition of the computing resources and carrying out real-time sensing according to the computing resource node network topological structure.
3. The method of claim 1, wherein the resource scheduling for heterogeneous computing,
the operation task is mapped to the computing power node from the virtual computing power resource, and the virtual computing power unit is managed and scheduled according to the current use condition of different virtual computing power units based on the constructed mapping relation between the virtual computing power unit and the computing power physical equipment.
4. A resource scheduling node for heterogeneous computing, comprising:
the acquiring unit is used for acquiring a job task submitted by a user in a computing network;
and the allocation unit is used for allocating the job task to the virtual computing resource pool, and the resource dynamic sensing module is used for distributing the job task based on the real-time sensing of the whole computing resource pool and mapping the job task to the computing nodes from the virtual computing resource, so that the job task is executed in the computing nodes allocated with the computing tasks.
5. A resource scheduling system for heterogeneous computing, comprising:
the application interface abstract module is used for providing an application module resource application interactive interface so as to obtain a job task submitted by a user in a computing network;
the computing resource scheduling module is used for scheduling and allocating the job tasks to virtual computing resources;
the computing power routing module is used for dispatching the computing tasks to the physical computing nodes of the computing power resource dispatching module from the computing power nodes;
the resource dynamic sensing module is used for sensing the resources of the whole computing power resource pool in real time, distributing the job tasks in the virtual computing power resources based on the real-time sensing of the whole computing power resource pool, and mapping the virtual computing power resources to computing power nodes;
and the computing power node resource module comprises a plurality of computing power nodes and is used for mapping the virtual computing power resources to the corresponding computing power nodes for execution according to the job tasks.
6. The heterogeneous computing-oriented resource scheduling system of claim 5,
in the computing power node resource module, different computing power nodes comprise different computing resources, each computing power node comprises a service component for virtualizing the same computing power of the computing power node into virtual computing power, and the different computing power nodes are connected through the Internet/local area network; all the computing power nodes form a computing power resource pool.
7. The heterogeneous computing-oriented resource scheduling system of claim 5,
in the resource dynamic sensing module, the real-time sensing of the resources of the whole computing power resource pool is to manage the available quantity of heterogeneous computing power, the current consumption condition of the computing power resources and the health state of the resources to obtain a real-time sensing result.
8. The heterogeneous computing-oriented resource scheduling system of claim 5,
the calculation force routing module comprises a calculation force resource node network topological structure unit, a calculation force routing control unit and a calculation force routing forwarding unit; the calculation force routing control unit controls the calculation force resource node network topological structure unit to generate a calculation force topology according to the demand of the job task on the calculation force resource and the calculation force node information, and further generates a calculation force perception new routing table; the calculation force route forwarding is used for interacting with the calculation force resource scheduling module and comprises calculation force route identification and calculation force route addressing.
9. The heterogeneous computing-oriented resource scheduling system of claim 5,
in the computing resource scheduling module, the job task scheduling distribution comprises job task selection, computing resource distribution and log management, and redundant backup is adopted for control nodes and data.
10. The resource scheduling system for heterogeneous computing according to claim 5, wherein the application interface abstraction module provides application module resource application interaction interfaces including a data transmission interface, a resource scheduling abstraction interface, and a job task lifecycle management interface.
11. The resource scheduling system for heterogeneous computing according to claim 5, further comprising an application module, the application module being disposed at an upper layer of the application interface abstraction module, the application module having a uniform interface for connecting different applications to run on the whole resource pool.
12. The resource scheduling system for heterogeneous computing according to claim 5, wherein the computation power node has logic computation power, neural network computation power and parallel computation power.
13. An electronic device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, the processor implementing the steps of the heterogeneous computing oriented resource scheduling method of any one of claims 1 to 3 when executing the computer program.
14. A computer readable storage medium, storing a computer program which, when executed by a processor, performs the steps of the heterogeneous computing oriented resource scheduling method of any one of claims 1 to 3.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210676039.XA CN114996018A (en) | 2022-06-15 | 2022-06-15 | Resource scheduling method, node, system, device and medium for heterogeneous computing |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210676039.XA CN114996018A (en) | 2022-06-15 | 2022-06-15 | Resource scheduling method, node, system, device and medium for heterogeneous computing |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114996018A true CN114996018A (en) | 2022-09-02 |
Family
ID=83035326
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210676039.XA Pending CN114996018A (en) | 2022-06-15 | 2022-06-15 | Resource scheduling method, node, system, device and medium for heterogeneous computing |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114996018A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115412482A (en) * | 2022-11-01 | 2022-11-29 | 北京邮电大学 | Calculation force routing method and device, electronic equipment and storage medium |
| CN115550972A (en) * | 2022-11-30 | 2022-12-30 | 成都中星世通电子科技有限公司 | Method and system for automatic decomposition and resource allocation of electromagnetic sensing task |
| CN116610270A (en) * | 2023-07-21 | 2023-08-18 | 湖南马栏山视频先进技术研究院有限公司 | Video processing calculation and separation method and video calculation and separation system |
| CN116627619A (en) * | 2023-07-24 | 2023-08-22 | 山东华科信息技术有限公司 | Cloud edge end cooperative information interaction method and system for multi-service heterogeneous resource scheduling |
| CN117421108A (en) * | 2023-12-15 | 2024-01-19 | 企商在线(北京)数据技术股份有限公司 | Heterogeneous computing power platform design method, heterogeneous computing power platform and resource scheduling method |
-
2022
- 2022-06-15 CN CN202210676039.XA patent/CN114996018A/en active Pending
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115412482A (en) * | 2022-11-01 | 2022-11-29 | 北京邮电大学 | Calculation force routing method and device, electronic equipment and storage medium |
| CN115550972A (en) * | 2022-11-30 | 2022-12-30 | 成都中星世通电子科技有限公司 | Method and system for automatic decomposition and resource allocation of electromagnetic sensing task |
| CN116610270A (en) * | 2023-07-21 | 2023-08-18 | 湖南马栏山视频先进技术研究院有限公司 | Video processing calculation and separation method and video calculation and separation system |
| CN116610270B (en) * | 2023-07-21 | 2023-10-03 | 湖南马栏山视频先进技术研究院有限公司 | Video processing calculation and separation method and video calculation and separation system |
| CN116627619A (en) * | 2023-07-24 | 2023-08-22 | 山东华科信息技术有限公司 | Cloud edge end cooperative information interaction method and system for multi-service heterogeneous resource scheduling |
| CN116627619B (en) * | 2023-07-24 | 2023-10-10 | 山东华科信息技术有限公司 | Cloud edge end cooperative information interaction method and system for multi-service heterogeneous resource scheduling |
| CN117421108A (en) * | 2023-12-15 | 2024-01-19 | 企商在线(北京)数据技术股份有限公司 | Heterogeneous computing power platform design method, heterogeneous computing power platform and resource scheduling method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Kaur et al. | Container-as-a-service at the edge: Trade-off between energy efficiency and service availability at fog nano data centers | |
| CN114996018A (en) | Resource scheduling method, node, system, device and medium for heterogeneous computing | |
| Singh et al. | Task scheduling in cloud computing | |
| CN112465129B (en) | On-chip heterogeneous artificial intelligent processor | |
| CN107122243B (en) | The method of Heterogeneous Cluster Environment and calculating CFD tasks for CFD simulation calculations | |
| CN103092698B (en) | Cloud computing application automatic deployment system and method | |
| Tao et al. | Dynamic resource allocation algorithm for container-based service computing | |
| CN104123182B (en) | Based on the MapReduce task of client/server across data center scheduling system and method | |
| CN103677990B (en) | Dispatching method, device and the virtual machine of virtual machine real-time task | |
| CN102541640A (en) | Cluster GPU (graphic processing unit) resource scheduling system and method | |
| CN106020934A (en) | Optimized deploying method based on virtual cluster online migration | |
| CN109144710A (en) | Resource regulating method, device and computer readable storage medium | |
| CN100495984C (en) | Grid task dispatching method based on time and expense restriction | |
| CN105069702B (en) | A kind of power grid integrated information processing method | |
| Pandey et al. | A heuristic method towards deadline-aware energy-efficient mapreduce scheduling problem in Hadoop YARN | |
| Han et al. | Energy efficient VM scheduling for big data processing in cloud computing environments | |
| Lan et al. | Task partitioning and orchestration on heterogeneous edge platforms: The case of vision applications | |
| Saravanan et al. | Advance Map Reduce Task Scheduling algorithm using mobile cloud multimedia services architecture | |
| CN114138488A (en) | Cloud-native implementation method and system based on elastic high-performance computing | |
| CN114490049A (en) | Method and system for automatically allocating resources in containerized edge computing | |
| CN116402318B (en) | Multi-stage computing power resource distribution method and device for power distribution network and network architecture | |
| Narayani et al. | Fairness-based heuristic workflow scheduling and placement in cloud computing | |
| Malathy et al. | Performance improvement in cloud computing using resource clustering | |
| Kherbache et al. | Scheduling live-migrations for fast, adaptable and energy-efficient relocation operations | |
| Li et al. | Real-time workflows oriented hybrid scheduling approach with balancing host weighted square frequencies in clouds |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |