CN111552558A

CN111552558A - Scheduling method and device of heterogeneous cloud resources

Info

Publication number: CN111552558A
Application number: CN202010266227.6A
Authority: CN
Inventors: 王琳; 胡建; 涂振印; 周志勇; 赵思宇
Original assignee: Diankeyun Beijing Technology Co ltd; CETC Big Data Research Institute Co Ltd
Current assignee: Diankeyun Beijing Technology Co ltd; CETC Big Data Research Institute Co Ltd
Priority date: 2020-04-07
Filing date: 2020-04-07
Publication date: 2020-08-18

Abstract

The invention provides a scheduling method and device of heterogeneous cloud resources, wherein the method comprises the following steps: acquiring cloud resource service operation; identifying the resource type required by the cloud resource service operation, and determining the total operation resource amount required by the cloud resource service operation; searching a resource domain corresponding to the resource type combination according to the resource type required by the cloud resource service operation; each resource domain corresponds to at least one cloud platform with the same resource type combination; matching a resource domain with sufficient resources from all searched resource domains corresponding to the combination of the resource types required by the cloud resource service operation according to the total operation resource amount; matching a cloud platform with sufficient resources from the cloud platforms corresponding to the matched resource domains according to the total operation resource amount; sending cloud resource service operation to perform operation processing by using the matched cloud platform; and outputting the received job processing result. By the scheme, the job task allocation efficiency and the processing efficiency in the multi-cloud heterogeneous environment can be improved.

Description

Scheduling method and device of heterogeneous cloud resources

Technical Field

The invention relates to the technical field of cloud resource management, in particular to a scheduling method and device of heterogeneous cloud resources.

Background

As more and more enterprises put their respective data centers into the cloud computing system, the cloud computing system is required to provide high-performance, high-reliability, and high-efficiency computing guarantees for various services of each enterprise user. The cloud computing system has the advantages of flexible resource expansion, large-scale storage, safety strategy management and the like, can enable enterprise users to apply for virtual resources according to actual business requirements, greatly reduces the operation cost of enterprises, and can provide mature and reliable safety guarantee for various sensitive data of the enterprises. In a multi-cloud heterogeneous resource environment, the full, efficient and reasonable utilization of cloud platform resources of various different infrastructures is a very key technical index. Therefore, how to reasonably call the increasingly large-scale cloud service tasks and how to effectively distribute the cloud service tasks to the heterogeneous cloud platform environment becomes an urgent problem to be solved.

Disclosure of Invention

In view of this, the present invention provides a scheduling method and apparatus for heterogeneous cloud resources, so as to improve job task allocation efficiency and processing efficiency in a multi-cloud heterogeneous environment.

In order to achieve the purpose, the invention is realized by adopting the following scheme:

according to an aspect of the embodiments of the present invention, there is provided a method for scheduling heterogeneous cloud resources, including:

acquiring cloud resource service operation;

identifying the resource type required by the cloud resource service operation, and determining the total operation resource amount required by the cloud resource service operation;

searching a resource domain corresponding to the resource type combination according to the resource type required by the cloud resource service operation; each resource domain corresponds to at least one cloud platform with the same resource type combination as the resource domain;

matching a resource domain with sufficient resources from all searched resource domains corresponding to the combination of the resource types required by the cloud resource service operation according to the total operation resource amount;

matching a cloud platform with sufficient resources from the cloud platforms corresponding to the matched resource domains according to the total operation resource amount;

sending the cloud resource service operation to perform operation processing by using the matched cloud platform;

and outputting the received job processing result of the cloud resource service job.

In some embodiments, obtaining a cloud resource service job comprises: receiving the submitted cloud resource service operation, and adding the received cloud resource service operation to a task queue; allocating a job number for the received cloud resource service job, and returning the allocated job number to display the allocated job number; outputting the received job processing result of the cloud resource service job, including: and outputting the received job processing result of the cloud resource service job and the corresponding job number for displaying.

In some embodiments, determining the total amount of job resources needed for the cloud resource service job comprises: analyzing the resource amount of each resource type required by the cloud resource service operation, and allocating resource weight to each resource type required by the cloud resource service operation; and calculating the total job resource quantity required by the cloud resource service job according to the resource quantity of each resource type and the resource weight of each resource type.

In some embodiments, the types of resources required for the cloud resource service job include one or more of computing resources, communication resources, and data resources; the resource type combination of the resource domain includes one or more of a computing resource, a communication resource, and a data resource.

In some embodiments, matching a resource domain with sufficient resources from all found resource domains corresponding to the combination of the resource types required by the cloud resource service job according to the total job resource amount includes: and matching a resource domain with total available resources larger than the total job resource amount from all the resource domains corresponding to the combination of the resource types required by the cloud resource service job.

In some embodiments, matching a cloud platform with sufficient resources from cloud platforms corresponding to the matched resource domains according to the total job resource amount includes: and matching the cloud platform with available resources larger than the total operation resource amount from all the cloud platforms corresponding to the matched resource domains.

In some embodiments, the method for scheduling heterogeneous cloud resources further includes: and if the resource domain corresponding to the resource type combination cannot be searched according to the resource type required by the cloud resource service operation, searching the resource domain which is matched best again from the resource domain which is not corresponding to the resource type combination required by the cloud resource service operation according to the resource type required by the cloud resource service operation and the total operation resource amount.

In some embodiments, the method for scheduling heterogeneous cloud resources further includes: and if a cloud platform with sufficient resources cannot be matched from the cloud platforms corresponding to the matched resource domains according to the total operation resource amount, re-matching a resource domain with sufficient resources from the resource domains corresponding to the searched combination of the rest resources required by the cloud resource service operation according to the total operation resource amount, so as to match a cloud platform with sufficient resources from all cloud platforms corresponding to the re-matched resource domains according to the total operation resource amount.

In some embodiments, before matching a cloud platform with sufficient resources from the cloud platforms corresponding to the matched resource domains according to the total job resource amount, the method further includes: and adding the cloud resource service operation into the resource queue of the matched resource domain according to the priority of the cloud resource service operation. Matching a cloud platform with sufficient resources from the cloud platforms corresponding to the matched resource domains according to the total job resource quantity, wherein the matching comprises the following steps: and matching cloud platforms with sufficient resources from the cloud platforms corresponding to the matched resource domains according to the total operation resource quantity and the sequence of the cloud resource service operation in the resource queues of the matched resource domains.

In some embodiments, sending the cloud resource service job for job processing using the matched cloud platform includes: and under the condition that the cloud resource service operation needs exclusive resources, sending the cloud resource service operation and a corresponding exclusive resource identifier so as to utilize the matched cloud platform to perform operation processing on the cloud resource service operation and enable the cloud resource service operation not to receive other cloud resource service operations before the operation processing on the cloud resource service operation is completed.

In some embodiments, sending the cloud resource service job for job processing using the matched cloud platform includes: and under the condition that the cloud resource service operation does not need to monopolize resources, the cloud resource service operation and the corresponding non-exclusive resource identification are sent, so that the matched cloud platform is utilized to perform operation processing on the cloud resource service operation, and other cloud resource service operations are allowed to be received to perform synchronous operation processing.

In some embodiments, after sending the cloud resource service job for job processing by using the matched cloud platform, the method further includes: and receiving the updated resource load condition returned by the matched cloud platform so as to update the total available resources of the matched resource domain and the available resources of the matched cloud platform.

According to another aspect of the embodiments of the present invention, there is provided a scheduling apparatus for heterogeneous cloud resources, including:

the cloud service scheduling engine module is used for acquiring cloud resource service jobs;

the resource type analysis module is used for identifying the resource type required by the cloud resource service operation and determining the total operation resource amount required by the cloud resource service operation; searching a resource domain corresponding to the resource type combination according to the resource type required by the cloud resource service operation; matching a resource domain with sufficient resources from all searched resource domains corresponding to the combination of the resource types required by the cloud resource service operation according to the total operation resource amount; each resource domain corresponds to at least one cloud platform with the same resource type combination as the resource domain;

the local scheduler is used for matching a cloud platform with sufficient resources from the cloud platforms corresponding to the matched resource domains according to the total job resource amount;

the cloud platform module is used for sending the cloud resource service operation so as to utilize the matched cloud platform to perform operation processing; and outputting the received job processing result of the cloud resource service job.

In some embodiments, the apparatus further comprises: a global scheduler, configured to search a resource domain corresponding to a resource type combination of the cloud resource service job according to a resource type required by the cloud resource service job, searching a resource domain which is matched with the best resource domain again from a resource domain which does not correspond to the combination of the resource types required by the cloud resource service operation according to the resource types required by the cloud resource service operation and the total operation resource amount, and/or if a cloud platform with sufficient resources cannot be matched from the cloud platforms corresponding to the matched resource domains according to the total job resource amount, then a resource domain with sufficient resources is matched out from the searched resource domains corresponding to the rest resource types required by the cloud resource service operation again according to the total operation resource amount, and matching a cloud platform with sufficient resources from all the cloud platforms corresponding to the re-matched resource domains according to the total operation resource amount.

According to another aspect of embodiments of the present invention, there is provided a computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor performing the steps of the method according to any of the above embodiments.

According to another aspect of embodiments of the present invention, there is provided a computer-readable storage medium having stored thereon a computer program for executing, by a processor, the steps of the method according to any of the above embodiments.

According to the scheduling method of the heterogeneous cloud resources, the scheduling device of the heterogeneous cloud resources, the computer equipment and the computer readable storage medium, the resource domains are classified and divided, the resource domains with resource types matched as much as possible are searched according to the resource types required by the cloud resource service operation, and then the cloud platform with sufficient resources is searched in the resource domains to process the operation, so that the cloud resources are classified and hierarchically scheduled, the cloud platform can be reasonably selected to process tasks, and the task allocation efficiency and the processing efficiency of the operation in the multi-cloud heterogeneous environment are improved.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts. In the drawings:

fig. 1 is a flowchart illustrating a scheduling method of heterogeneous cloud resources according to an embodiment of the present invention;

FIG. 2 is a schematic diagram of a network architecture according to an embodiment of the present invention;

fig. 3 is a schematic structural diagram of a scheduling apparatus of heterogeneous cloud resources according to an embodiment of the present invention;

fig. 4 is a schematic structural diagram of a scheduling apparatus of heterogeneous cloud resources according to another embodiment of the present invention;

fig. 5 is a schematic diagram of a scheduling model corresponding to a scheduling apparatus for heterogeneous cloud resources according to an embodiment of the present invention;

fig. 6 is a schematic diagram of a hierarchical scheduling flow of a scheduling method of heterogeneous cloud resources according to an embodiment of the present invention.

Detailed Description

In order to make the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the embodiments of the present invention are further described in detail below with reference to the accompanying drawings. The exemplary embodiments and descriptions of the present invention are provided to explain the present invention, but not to limit the present invention.

In the existing heterogeneous cloud resource scheduling method, a scheduling strategy is mainly that, no matter the amount of task resources and the complexity of computing, a task is firstly allocated to one cloud platform for processing, and when the computing resources of the cloud platform are insufficient, the task is rescheduled to other cloud platforms for execution. The method mainly comprises the steps that single switching is carried out among different cloud platforms, so that task scheduling processing in the heterogeneous cloud platform is disordered, and the task allocation efficiency and the task processing efficiency are reduced.

In order to solve the above problem, an embodiment of the present invention provides a scheduling method for heterogeneous cloud resources, so as to improve allocation efficiency and processing efficiency of tasks through hierarchical scheduling.

Fig. 1 is a flowchart illustrating a scheduling method of heterogeneous cloud resources according to an embodiment of the present invention. Fig. 2 is a schematic diagram of a network architecture according to an embodiment of the present invention. Referring to fig. 1 and 2, a plurality of heterogeneous cloud platforms may be divided into resource domains, and each resource domain may correspond to one or more cloud platforms. The cloud management platform can receive the tasks submitted by the user equipment and process the submitted tasks by scheduling the cloud platform resources of the resource domain. Referring back to fig. 1, the scheduling method of heterogeneous cloud resources according to some embodiments may include the following steps S110 to S170.

Specific embodiments of steps S110 to S170 will be described in detail below.

Step S110: and acquiring cloud resource service operation.

The cloud resource service operation may also be referred to as a cloud resource task request, and may correspond to one task, which may relate to a computing task, a storage task, a communication task, and the like.

This step S110, namely, acquiring a cloud resource service job, may specifically include the steps of: and S111, receiving the submitted cloud resource service jobs, and adding the received cloud resource service jobs to a task queue. A user may enter and submit a cloud resource service job via a user device (e.g., a computer). The task queue can be a first-in first-out queue, and cloud resource service jobs can be read from the task queue in sequence for subsequent scheduling processing.

In addition, the step S110 may further include the steps of: and S112, distributing the job number for the received cloud resource service job, and returning the distributed job number to display the distributed job number. The assigned job number may be used to uniquely identify a cloud resource service job. The allocated job number can be returned to an interface of the user equipment for displaying, so that the user can inquire the running condition of the cloud resource service job corresponding to the job number. In addition, the job number that is sub-matched can also be used when returning the job processing result.

Step S120: and identifying the resource type required by the cloud resource service operation, and determining the total operation resource amount required by the cloud resource service operation.

Wherein, the resource type required by the cloud resource service job can be determined according to the task content or the label of the cloud resource service job. The types of resources required for each cloud resource service job may be divided into several fixed types, such as computing resources, communication resources, and data resources. Therefore, the type of resources required for a cloud resource service job may include one or more of computing resources, communication resources, and data resources.

Since different resource types occupy different cloud resources when processing a job, for example, the amount of resources occupied by a computational resource is relatively large, the total amount of job resources required by a cloud resource service job is not necessarily equal to the sum of the initially obtained amounts of the resources of the respective types. For this reason, the total amount of job resources obtained can be made more accurate by setting different weights for different resource types.

In this step S120, the total job resource amount required by the cloud resource service job is determined, which may specifically include the steps of: s121, analyzing the resource amount of each resource type required by the cloud resource service operation, and allocating a resource weight to each resource type required by the cloud resource service operation; and S122, calculating the total job resource amount required by the cloud resource service job according to the resource amount of each resource type and the resource weight of each resource type.

In step S121, for example, the resource amount of the computing resource, the resource amount of the communication resource, the resource amount of the data resource, and the like required for the cloud resource service job may be analyzed. The weights corresponding to various resource types may be predetermined empirically, for example, the weights corresponding to the requested service type and the calculated amount of resources may be larger. In step S122, the total amount of job resources can be obtained by multiplying each resource amount by its respective weight and then summing up.

Step S130: searching a resource domain corresponding to the resource type combination according to the resource type required by the cloud resource service operation; each resource domain corresponds to at least one cloud platform with the same resource type combination.

In step S130, one or more cloud platforms may be divided into one resource domain in advance. Each cloud platform in one resource domain can have the same resource type combination, and then the resource domain can also have the resource type combination, and the resource domains with different resource type combinations can be obtained by classifying and dividing the cloud platforms (which can be heterogeneous cloud platforms, and can be private clouds or public clouds), so that the resource type combinations of the same type can be conveniently found to process cloud resource service jobs, and the job processing efficiency can be conveniently improved. The resource type combination of the resource domain may include one or more of a computing resource, a communication resource, and a data resource. The resource type combination of the resource domains can be correspondingly provided with a label, so that the resource domains with the same type combination can be found conveniently. For example, in the case where the resource types are divided into the amount of computation resources, the amount of communication resources, and the amount of data resources, the resource domain can be constructed by combining into 7 resource models, for example, 7 broad categories divided into computation intensive, communication intensive, data intensive, computation intensive-communication intensive, computation intensive and data intensive, communication intensive and data intensive, and general. Similarly, the resource types required by various cloud resource service jobs can be divided into several resource type combinations similar to various resource domains, so that the resource types can be better corresponded. In step S130, the resource domains corresponding to the combination of the resource types required by the cloud resource service job may be the same as the combination of the resource types.

In other embodiments, a resource domain may not be found whose resource type combination is exactly the same as the combination of resource types required by the cloud resource service job, i.e., a resource domain with a completely matching resource type may not be found, and at this time, a resource domain that matches as much as possible may be found.

For example, the method shown in fig. 1 may further include the steps of: and if the resource domain corresponding to the resource type combination cannot be searched according to the resource type required by the cloud resource service operation, searching the resource domain which is matched best again from the resource domain which is not corresponding to the resource type combination required by the cloud resource service operation according to the resource type required by the cloud resource service operation and the total operation resource amount.

In which, a resource domain with resource types matching as much as possible (e.g. the most matched resource types) can be found first, and then a resource domain with sufficient resources can be found. In addition, a resource domain which is best matched is searched from a resource domain which does not correspond to the combination of the resource types required by the cloud resource service operation again according to the resource types required by the cloud resource service operation and the total operation resource amount, and instead, the resource domain which is best matched is searched from the resource domain which does not correspond to the combination of the resource types required by the cloud resource service operation again according to the resource types required by the cloud resource service operation and the resource type allocation resource weight, in this case, the total operation resource amount required by the cloud resource service operation can be obtained by calculation according to the resource type allocation resource weight and the resource amount of various required resource types obtained by current analysis.

Step S140: and matching out a resource domain with sufficient resources from all the resource domains corresponding to the combination of the resource types required by the cloud resource service operation according to the total operation resource amount.

A resource domain may have the same resource category (or combination) as its cloud platform, and respective type labels may be placed on each cloud platform to indicate the type of resource it is skilled in handling. And according to the type label of the cloud platform, corresponding resource domains can be correspondingly determined. According to the type label of the cloud platform, a resource domain (or a corresponding cloud platform set) matched with the type of the cloud resource job task can be found. The load condition, the resource utilization rate, the computing capacity and the like of the cloud platform can be returned at regular time, so that the load condition of the resource domain can be obtained at regular time. And determining a proper resource domain according to the load condition of the resource domain and the total job resource amount of the job.

There may be a plurality of resource domains, and there may also be a plurality of resource domains of the same resource type combination. For a cloud resource service job having a certain resource type combination, multiple resource domains of the corresponding resource type combination can be found. A resource domain may also correspond to a certain resource size, for example, the resource size of each resource type may be synthesized. Further, a resource domain with sufficient resources can be selected from these resource domains. For example, the resource load condition of a resource domain may be obtained according to the resource load condition of each cloud platform, and if the resource load condition of the resource domain plus the resource load amount corresponding to the total resource amount of the combination of the resource types required by the cloud resource service job does not exceed the set safe operation threshold, the resource of the resource domain may be considered to be sufficient. Alternatively, a resource domain with sufficient resources may be found according to whether the available resources of the resource domain exceed the total resource amount required by the cloud resource service job.

For example, in the step S140, matching a resource domain with sufficient resources from all resource domains corresponding to the found combination of resource types required by the cloud resource service job according to the total job resource amount may specifically include: and S141, matching a resource domain with total available resources larger than the total job resource amount from all the resource domains corresponding to the combination of the resource types required by the cloud resource service job. Wherein, the total available resources of the resource domain can be calculated according to other corresponding parameters (such as resource load condition).

In some embodiments, the method of fig. 1 may further comprise the steps of: and if the resource domain with sufficient resources cannot be matched again from the resource domains corresponding to the searched remaining resource types required by the cloud resource service operation according to the total operation resource amount, searching the resource domain which is not matched with the resource domain corresponding to the resource type combination required by the cloud resource service operation again according to the resource type required by the cloud resource service operation and the total operation resource amount.

Step S150: and matching a cloud platform with sufficient resources from the cloud platforms corresponding to the matched resource domains according to the total operation resource amount.

In the step S150, that is, a cloud platform with sufficient resources is matched from the cloud platforms corresponding to the matched resource domains according to the total job resource amount, specifically, the method may include the steps of: and matching the cloud platform with available resources larger than the total operation resource amount from all the cloud platforms corresponding to the matched resource domains.

The classified resource domains are found in the step S130, the resource domains with sufficient resources are found in the step S140, and then the cloud platforms with sufficient resources are found from the resource domains in the step S150, so that the resource allocation efficiency can be improved by classifying and sequentially scheduling the cloud resources.

In step S140, after the resource domain is found, the cloud resource service job may be placed in the list of the resource domain to be allocated to the cloud platform resource. Further, the jobs may be placed in the list according to the priority of the cloud resource service jobs, for example, if the priority is higher, the jobs may be directly placed at the top of the list. Before the step S150, the method shown in fig. 1 may further include the steps of: and adding the cloud resource service operation into the resource queue of the matched resource domain according to the priority of the cloud resource service operation. For example, if the priority is higher, a cloud resource service job may be added to the front-most of the resource queue. In the step S150, that is, a cloud platform with sufficient resources is matched from the cloud platforms corresponding to the matched resource domains according to the total job resource amount, specifically, the method may include the steps of: and matching cloud platforms with sufficient resources from the cloud platforms corresponding to the matched resource domains according to the total operation resource quantity and the sequence of the cloud resource service operation in the resource queues of the matched resource domains. In this way, when scheduling resources, the priority of the cloud resource service job is taken into consideration, so that the job processing efficiency can be higher.

In other embodiments, none of the cloud platforms in the found resource domain may be able to provide sufficient cloud platforms, in which case scheduling may be done globally.

For example, the method shown in FIG. 1 may further include the steps of: and if a cloud platform with sufficient resources cannot be matched from the cloud platforms corresponding to the matched resource domains according to the total operation resource amount, re-matching a resource domain with sufficient resources from the resource domains corresponding to the searched combination of the rest resources required by the cloud resource service operation according to the total operation resource amount, so as to match a cloud platform with sufficient resources from all cloud platforms corresponding to the re-matched resource domains according to the total operation resource amount. If the resource domain with the matched resource type cannot be found, the resource domain which is matched as much as possible can be searched from the resource domain which is not matched so as to schedule the cloud resource.

In still other embodiments, the method shown in fig. 1 may further include the steps of: if a resource domain with sufficient resources cannot be matched from all resource domains corresponding to the searched combination of the resource types required by the cloud resource service operation according to the total operation resource amount, or if a cloud platform with sufficient resources cannot be matched from the cloud platforms corresponding to the matched resource domains according to the total operation resource amount, searching the best matched resource domain from all resource domains again based on the resource weight distributed to each resource type required by the cloud resource service operation and the total operation resource amount; and matching a cloud platform with sufficient resources from the cloud platform corresponding to the found resource domain with the best matching according to the total operation resource amount.

Step S160: and sending the cloud resource service operation to utilize the matched cloud platform to perform operation processing.

In step S160, a cloud resource service job may be sent to a node of the cloud platform, and the node may be driven to operate to perform job processing on the cloud resource service job. Meanwhile, the job number of the cloud resource service job can be sent to the cloud platform together, so that the processing result returned by the cloud platform corresponds to the job number.

In some embodiments, the cloud platform resource may be locked if the cloud resource service job requires exclusive resources. For example, the step S160 of sending the cloud resource service job to perform job processing by using the matched cloud platform may specifically include the steps of: and under the condition that the cloud resource service operation needs exclusive resources, sending the cloud resource service operation and a corresponding exclusive resource identifier so as to utilize the matched cloud platform to perform operation processing on the cloud resource service operation and enable the cloud resource service operation not to receive other cloud resource service operations before the operation processing on the cloud resource service operation is completed.

In other embodiments, if the cloud resource service job has no special requirement, if no exclusive resource is needed, the cloud platform may be enabled to process multiple cloud resource service jobs within a set resource threshold. For example, the step S160 of sending the cloud resource service job to perform job processing by using the matched cloud platform may specifically include the steps of: and under the condition that the cloud resource service operation does not need to monopolize resources, the cloud resource service operation and the corresponding non-exclusive resource identification are sent, so that the matched cloud platform is utilized to perform operation processing on the cloud resource service operation, and other cloud resource service operations are allowed to be received to perform synchronous operation processing.

Step S170: and outputting the received job processing result of the cloud resource service job.

This step S170, namely, outputting the received job processing result of the cloud resource service job, may specifically include the steps of: and outputting the received job processing result of the cloud resource service job and the corresponding job number for displaying. The job processing result corresponding to a certain job number can be returned to the user equipment, so that the processing result corresponding to the job number can be displayed for the user to check.

In addition, after the cloud platform processes the cloud resource service operation, resources can be released, redundancy is clear, and the latest resource load condition can be fed back, so that the load conditions of the cloud platform and the corresponding resource domain recorded in the cloud management platform are updated. For example, after sending the cloud resource service job to perform job processing by using the matched cloud platform, the method shown in fig. 1 may further include the steps of: and receiving the updated resource load condition returned by the matched cloud platform so as to update the total available resources of the matched resource domain and the available resources of the matched cloud platform. In addition, if the cloud platform reports an error when processing the cloud resource service operation or the processing time exceeds the set duration, the processing state can be returned for the user to check.

Based on the same inventive concept as the scheduling method of heterogeneous cloud resources shown in fig. 1, an embodiment of the present invention further provides a scheduling apparatus of heterogeneous cloud resources, as described in the following embodiments. The principle of solving the problems of the scheduling device of the heterogeneous cloud resources is similar to that of the scheduling method of the heterogeneous cloud resources, so the implementation of the scheduling device of the heterogeneous cloud resources can refer to the implementation of the scheduling method of the heterogeneous cloud resources, and repeated parts are not described again.

Fig. 3 is a schematic structural diagram of a scheduling apparatus of heterogeneous cloud resources according to an embodiment of the present invention. As shown in fig. 3, the scheduling apparatus for heterogeneous cloud resources of the embodiments may include a cloud service scheduling engine module 210, a resource type analysis module 220, a local scheduler 230, and a cloud platform module 240.

And the cloud service scheduling engine module 210 is configured to obtain a cloud resource service job.

A resource type analysis module 220, configured to identify a resource type required by the cloud resource service job, and determine a total job resource amount required by the cloud resource service job; searching a resource domain corresponding to the resource type combination according to the resource type required by the cloud resource service operation; matching a resource domain with sufficient resources from all searched resource domains corresponding to the combination of the resource types required by the cloud resource service operation according to the total operation resource amount; each resource domain corresponds to at least one cloud platform with the same resource type combination.

And the local scheduler 230 is configured to match a cloud platform with sufficient resources from the cloud platforms corresponding to the matched resource domains according to the total job resource amount.

The cloud platform module 240 is configured to send the cloud resource service job to perform job processing by using the matched cloud platform; and outputting the received job processing result of the cloud resource service job.

Fig. 4 is a schematic structural diagram of a scheduling apparatus of heterogeneous cloud resources according to another embodiment of the present invention. As shown in fig. 4, in a further embodiment, the scheduling apparatus of heterogeneous cloud resources shown in fig. 3 may further include: a global scheduler 250.

A global scheduler 250, configured to, if it is not possible to search a resource domain corresponding to the resource type combination according to the resource type required by the cloud resource service job, searching a resource domain which is matched with the best resource domain again from a resource domain which does not correspond to the combination of the resource types required by the cloud resource service operation according to the resource types required by the cloud resource service operation and the total operation resource amount, and/or if a cloud platform with sufficient resources cannot be matched from the cloud platforms corresponding to the matched resource domains according to the total job resource amount, then a resource domain with sufficient resources is matched out from the searched resource domains corresponding to the rest resource types required by the cloud resource service operation again according to the total operation resource amount, and matching a cloud platform with sufficient resources from all the cloud platforms corresponding to the re-matched resource domains according to the total operation resource amount.

In some embodiments, the cloud service scheduling engine module 210 is specifically configured to receive a submitted cloud resource service job, and add the received cloud resource service job to the task queue; and allocating a job number for the received cloud resource service job, and returning the allocated job number to display the allocated job number.

The cloud platform module 240 may be specifically configured to output the received job processing result of the cloud resource service job and the corresponding job number for display.

In some embodiments, the resource type analysis module 220 is specifically configured to analyze the resource amount of each resource type required by the cloud resource service job, and allocate a resource weight to each resource type required by the cloud resource service job; and calculating the total job resource quantity required by the cloud resource service job according to the resource quantity of each resource type and the resource weight of each resource type.

In some embodiments, the resource type analysis module 220 is specifically configured to match one resource domain, where the total available resource is greater than the total job resource amount, from all resource domains corresponding to the found combination of resource types required by the cloud resource service job.

In some embodiments, the local scheduler 230 may be specifically configured to match a cloud platform whose available resources are greater than the total amount of the job resources from all cloud platforms corresponding to the matched resource domains.

In some embodiments, the resource type analysis module 220 may be further configured to, if the resource domain corresponding to the combination of the resource types of the cloud resource service job cannot be found according to the resource type required by the cloud resource service job, find a resource domain that matches best again from the resource domains that do not correspond to the combination of the resource types required by the cloud resource service job according to the resource type required by the cloud resource service job and the total job resource amount.

In some embodiments, the local scheduler 230 may be further configured to, if a cloud platform with sufficient resources cannot be matched from cloud platforms corresponding to the matched resource domains according to the total job resource amount, match a resource domain with sufficient resources from resource domains corresponding to the combination of the remaining searched resource types required by the cloud resource service job again according to the total job resource amount, so as to match a cloud platform with sufficient resources from all cloud platforms corresponding to the resource domains matched again according to the total job resource amount.

In some embodiments, the resource type analysis module 220 may be further configured to add the cloud resource service job to a resource queue of the matched resource domain according to the priority of the cloud resource service job. The local scheduler 230 may be specifically configured to match, according to the total job resource amount and according to the sequence of the cloud resource service job in the resource queue of the matched resource domain, a cloud platform with sufficient resources from the cloud platforms corresponding to the matched resource domain.

In some embodiments, the cloud platform module 240 may be specifically configured to, when the cloud resource service job requires an exclusive resource, send the cloud resource service job and a corresponding exclusive resource identifier, so as to perform job processing on the cloud resource service job by using the matched cloud platform and enable the cloud resource service job not to receive other cloud resource service jobs before the job processing on the cloud resource service job is completed.

In some embodiments, the cloud platform module 240 may be specifically configured to send the cloud resource service job and the corresponding non-exclusive resource identifier when the cloud resource service job does not require exclusive resource, so as to perform job processing on the cloud resource service job by using the matched cloud platform and allow receiving of other cloud resource service jobs for synchronous job processing.

In some embodiments, the cloud platform module 240 may further be configured to receive an updated resource load condition returned by the matched cloud platform, so as to update the total available resources of the matched resource domain and the available resources of the matched cloud platform.

In addition, an embodiment of the present invention further provides a computer device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, where the processor implements the steps of the method according to any one of the above embodiments when executing the computer program.

Embodiments of the present invention further provide a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the steps of the method according to any of the above embodiments.

In order that those skilled in the art will better understand the present invention, embodiments of the present invention will be described below with reference to specific examples.

In a specific embodiment, an optimization scheme for task scheduling in a heterogeneous cloud environment is provided, after different users apply for cloud resource services, service jobs are classified and processed, and an optimal resource platform is selected for cloud service processing in a cloudy heterogeneous environment according to a reasonable job type.

Resource division is carried out on heterogeneous cloud platform resources managed by the hybrid cloud according to respective processing resource capacity, the classification mode is mainly considered from 3 aspects, and the resource domains are respectively calculated, communicated and data resource quantities and further combined into 7 resource models to construct resource domains. And cloud platforms of the same resource category are placed in the same resource domain.

After receiving the cloud resource operation, the cloud management platform classifies the operation tasks according to 7 resource models. And after identifying the resource type, the resource analysis module delivers the job to the resource domain of the corresponding type for job processing. And for the resource domain with insufficient resources, the global scheduler performs rescheduling.

The local scheduler maintains the resource load of the cloud platform under the current resource domain. The global scheduler can maintain the total load of resources of the whole resource domain, and reasonably schedule the jobs in the whole heterogeneous cloud environment. The resource load use conditions of all the resource domains are uniformly provided by the resource scheduling engine.

Fig. 5 is a schematic diagram of a scheduling model corresponding to a scheduling apparatus for heterogeneous cloud resources according to an embodiment of the present invention. Referring to fig. 5, the scheduling apparatus of heterogeneous cloud resources may include a cloud service scheduling engine module, a resource type analysis module, a local scheduler, a global scheduler, and a cloud platform module. Each resource domain has a local scheduler and corresponds to one or more cloud platforms, and the cloud platforms can be private clouds or public clouds.

The cloud Service scheduling Engine module (Engine-Service) is an interface for interaction between a user and the device, and is responsible for receiving cloud resource Service jobs submitted by the user, putting the cloud resource Service jobs into a task queue, allocating a job number to the submitted cloud resource Service jobs, and returning the job number to the user. The job number can uniquely identify each cloud resource service job for use by a submitter to query the operation of the cloud resource service job. The Engine-Service may deliver the cloud resource Service job submitted by the user to the resource type analysis module, so as to perform classification processing on the submitted cloud resource Service job according to, for example, a calculation amount, a traffic amount, and a data amount, and calculate a resource load of a task of the cloud resource Service job. The Engine-Service is also responsible for receiving the running result of the task of the cloud resource Service operation and returning the running result to the user.

The Resource type Analysis module (Resource-Analysis) may be configured to calculate the required Resource weight according to the Resource type of the cloud Resource task request job (cloud Resource service job). For the resource type of the request service type, a larger weight can be allocated, and for other types of resources, a smaller weight can be allocated, so that the total job resource amount load required by the job is calculated. The resource load calculation for a task (cloud resource service job) can be represented by the following parameters: the total calculation load weight can be represented by a parameter w, the required calculation amount is represented by U, the required communication amount is represented by T, the required data amount is represented by S, the calculation amount, the communication amount and the data amount are respectively represented by x1, x2 and x3, and the weights of the calculation amount, the communication amount and the data amount occupied in one job task (cloud resource service job) are compared with each other, and then the final job total load (total job resource amount load) is calculated by the formula w-x 1-U + x 2-T + x 3. In a resource request service (cloud resource service operation), the x1 weight can be increased correspondingly for the resource request of the calculation amount type. And according to the total load of the resource amount obtained by calculation, finding the best matched resource domain in the resource domain list, and sending the resource domain (cloud resource service operation) to the local scheduler.

The Local-Scheduler (Local-Scheduler) is responsible for resource allocation within its resource domain. And the Local-Scheduler receives the job resource amount transmitted by the resource type analysis module, sends a query request to the Engine-Service, acquires the load condition of the cloud platform in the current domain, and schedules the job to run. The Local-Scheduler also listens to a resource load transmission request sent by the Scheduler, and sends the resource load condition in the resource domain to the Scheduler. And inquiring a cloud platform load list in the current resource domain according to the calculated task resource quantity (total operation resource quantity load), and distributing tasks (cloud resource service operation) to the cloud platform with sufficient resources for operation processing according to the cloud platform resource load condition.

For various cloud resource service jobs, the cloud scheduling jobs (cloud resource service jobs) can be classified into 7 categories, which are computation intensive, communication intensive, data intensive, computation intensive-communication intensive, computation intensive and data intensive, communication intensive and data intensive, and general, according to the resource demand type of the jobs. Resources in multiple clouds also form 7 different resource domains according to the size of computing capacity and communication capacity, and all cloud platforms in the same resource domain can have strong capacity for processing the resources. The job scheduling principle may be to try to match the job type of the submitted job with the resource domain type. The scheduling of the scheduler may adopt a mechanism combining periodic monitoring and event triggering. After the operation of the job is finished, when the job discovery mechanism discovers that a new resource is added and a new job is enqueued, the resource allocation process of the allocator can be triggered.

A global Scheduler (Scheduler) is used to schedule resource domain resources on a global scale. When the resource of the job request (cloud resource Service job) submitted by the user exceeds the maximum available condition of the current resource domain, the Engine-Service delivers the task to be scheduled and executed by the global scheduler. And the Scheduler selects a proper resource domain to run the job in the global scope according to the job resource weight information transmitted by each Local-Scheduler. And the Scheduler maintains the resource use load condition in each resource domain and reasonably selects the resource domain to schedule the job processing according to the job task priority.

The Cloud platform module (Cloud-Partform) represents the Cloud platform resources for each running job. The nodes of the cloud platform receive the jobs transmitted from the scheduler (global scheduler or local scheduler) and drive the jobs to run. And if the current cloud resource job task needs to monopolize the resource, after receiving the current service task, locking the resource and not receiving other job requests transmitted by the scheduler. If no special resource requirement exists, the cloud platform can simultaneously receive a plurality of cloud resource task job requests and synchronously process the requests within the range of the resource threshold value. After the cloud resource operation is completed, the cloud platform informs the scheduler of the completed operation number, transmits the operation result to the Engine-Service, and simultaneously updates the cloud platform load condition in the resource domain to the Engine-Service. The cloud platform also needs to regularly clear redundancy, make mistakes and perform overtime operation tasks to release resources, and send the current load condition of the cloud platform to the Engine service.

And completing resource scheduling in a hierarchical mode through the five modules. In the hierarchical scheduling process, in an environment with sufficient resources, a plurality of job tasks can be synchronously and concurrently scheduled to be executed on the same cloud platform, and the cloud platform resources are fully utilized. The load condition of the resource domain can be timely fed back to the Engine-Service for unified management in an active and passive triggering mode, and the cloud resource task can be conveniently and reasonably called. The specific scheduling process is shown in fig. 6, a user submits a cloud Resource Service request to an Engine-Service, the Engine-Service serves as the brain of a scheduling Engine, assigns a job number to a task job, returns task execution information to an interface module, and facilitates the user to check the job execution state and submit the job to a Resource-Analysis module for further processing. In the Resource-Analysis module, a system maintains a task queue and reasonably schedules the next job. And (3) periodically pulling the load condition of the bottom Resource domain by Resource-Analysis, analyzing the type of the task submitted by the Engine-Service, and putting the type into a corresponding classified Resource queue. And when the Resource load in the Resource domain is less than the set threshold value of safe operation, the Resource-Analysis module schedules task jobs in the Resource queue to execute tasks in the corresponding Resource domain. And Local-Scheduler performs second job scheduling on the jobs in the resource domain according to the task priority according to the running load condition of the resources of the underlying cloud platform. When the scheduling job fails, a scheduling job number is returned, job failure information is sent to the Engine-Service, the Engine-Service reschedules the failed job task to the global Schedule, the Schedule re-evaluates the current resource use condition according to the resource load information transmitted to the Engine-Service by each Local-Schedule, and selects a proper cloud platform to operate the job again in the global scope.

The embodiment realizes hierarchical scheduling based on the multi-cloud heterogeneous resource domain. Firstly, according to the operation type of the cloud resource task, the cloud resource request operation is classified according to the resource request amount. During design, the heterogeneous cloud platforms are classified according to the resource types, and each cloud platform is marked with a respective type label. Secondly, when a resource job request comes, the scheduling mode is selected as much as possible to be distributed to the cloud platforms of the corresponding types according to the characteristics of the cloud resource job. The cloud platform is reasonably selected in the heterogeneous cloud environment for task processing, bottom layer virtualization resources are fully utilized, timeliness, high efficiency and reliability of resource request task processing are improved, and the problem of task scheduling processing chaos in the heterogeneous cloud platform is effectively solved.

By adopting the hierarchical scheduling mode of the multi-cloud heterogeneous resources in the embodiment, the resource categories are hierarchically divided in the heterogeneous cloud environment, and the job is scheduled and processed in a classified manner, so that the problem of task scheduling confusion in the heterogeneous cloud environment is effectively solved. According to the embodiment, the current resource utilization rate and the computing capacity are fed back at regular time according to the load conditions of the cloud platform and the resource domain, the allocation mechanism is dynamically adjusted, the operation is reasonably scheduled into the resource domain to be executed, the bottom layer virtualized resources are fully and efficiently utilized, and the task allocation efficiency and the processing efficiency in the multi-cloud heterogeneous environment are greatly improved.

In summary, according to the scheduling method of heterogeneous cloud resources, the scheduling device of heterogeneous cloud resources, the computer device, and the computer readable storage medium of the embodiments of the present invention, resource domains are classified and divided, and a resource domain with resource types matching as much as possible is searched according to resource types required by cloud resource service jobs, so as to search a cloud platform with sufficient resources in the resource domain to process jobs, and thus, cloud resources are scheduled hierarchically and classified, and a cloud platform can be reasonably selected to perform task processing, so that job task allocation efficiency and processing efficiency in a multi-cloud heterogeneous environment are improved.

In the description herein, reference to the description of the terms "one embodiment," "a particular embodiment," "some embodiments," "for example," "an example," "a particular example," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, the schematic representations of the terms used above do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. The sequence of steps involved in the various embodiments is provided to schematically illustrate the practice of the invention, and the sequence of steps is not limited and can be suitably adjusted as desired.

As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

The above-mentioned embodiments are intended to illustrate the objects, technical solutions and advantages of the present invention in further detail, and it should be understood that the above-mentioned embodiments are only exemplary embodiments of the present invention, and are not intended to limit the scope of the present invention, and any modifications, equivalent substitutions, improvements and the like made within the spirit and principle of the present invention should be included in the scope of the present invention.

Claims

1. A scheduling method of heterogeneous cloud resources is characterized by comprising the following steps:

acquiring cloud resource service operation;

2. The method for scheduling heterogeneous cloud resources of claim 1,

obtaining cloud resource service jobs, comprising:

receiving the submitted cloud resource service operation, and adding the received cloud resource service operation to a task queue; allocating a job number for the received cloud resource service job, and returning the allocated job number to display the allocated job number;

outputting the received job processing result of the cloud resource service job, including:

and outputting the received job processing result of the cloud resource service job and the corresponding job number for displaying.

3. The method for scheduling heterogeneous cloud resources according to claim 1, wherein determining the total job resource amount required by the cloud resource service job comprises:

analyzing the resource amount of each resource type required by the cloud resource service operation, and allocating resource weight to each resource type required by the cloud resource service operation;

and calculating the total job resource quantity required by the cloud resource service job according to the resource quantity of each resource type and the resource weight of each resource type.

4. The method for scheduling heterogeneous cloud resources according to claim 1, wherein the resource types required by the cloud resource service job include one or more of computing resources, communication resources, and data resources; the resource type combination of the resource domain includes one or more of a computing resource, a communication resource, and a data resource.

5. The method for scheduling heterogeneous cloud resources according to claim 1, wherein matching out a resource domain with sufficient resources from all found resource domains corresponding to the combination of resource types required by the cloud resource service job according to the total job resource amount comprises:

and matching a resource domain with total available resources larger than the total job resource amount from all the resource domains corresponding to the combination of the resource types required by the cloud resource service job.

6. The method for scheduling heterogeneous cloud resources according to claim 1, wherein the step of matching a cloud platform with sufficient resources from cloud platforms corresponding to the matched resource domains according to the total job resource amount comprises:

and matching the cloud platform with available resources larger than the total operation resource amount from all the cloud platforms corresponding to the matched resource domains.

7. The method for scheduling heterogeneous cloud resources of claim 1, further comprising:

and if the resource domain corresponding to the resource type combination cannot be searched according to the resource type required by the cloud resource service operation, searching the resource domain which is matched best again from the resource domain which is not corresponding to the resource type combination required by the cloud resource service operation according to the resource type required by the cloud resource service operation and the total operation resource amount.

8. The method for scheduling heterogeneous cloud resources of claim 1, further comprising:

and if a cloud platform with sufficient resources cannot be matched from the cloud platforms corresponding to the matched resource domains according to the total operation resource amount, re-matching a resource domain with sufficient resources from the resource domains corresponding to the searched combination of the rest resources required by the cloud resource service operation according to the total operation resource amount, so as to match a cloud platform with sufficient resources from all cloud platforms corresponding to the re-matched resource domains according to the total operation resource amount.

9. The method for scheduling heterogeneous cloud resources of claim 1,

before matching a cloud platform with sufficient resources from the cloud platforms corresponding to the matched resource domains according to the total job resource amount, the method further comprises:

adding the cloud resource service operation to a resource queue of the matched resource domain according to the priority of the cloud resource service operation;

matching a cloud platform with sufficient resources from the cloud platforms corresponding to the matched resource domains according to the total job resource quantity, wherein the matching comprises the following steps:

and matching cloud platforms with sufficient resources from the cloud platforms corresponding to the matched resource domains according to the total operation resource quantity and the sequence of the cloud resource service operation in the resource queues of the matched resource domains.

10. The method for scheduling heterogeneous cloud resources according to claim 1, wherein sending the cloud resource service job to perform job processing by using the matched cloud platform comprises:

and under the condition that the cloud resource service operation needs exclusive resources, sending the cloud resource service operation and a corresponding exclusive resource identifier so as to utilize the matched cloud platform to perform operation processing on the cloud resource service operation and enable the cloud resource service operation not to receive other cloud resource service operations before the operation processing on the cloud resource service operation is completed.

11. The method for scheduling heterogeneous cloud resources according to claim 1, wherein sending the cloud resource service job to perform job processing by using the matched cloud platform comprises:

and under the condition that the cloud resource service operation does not need to monopolize resources, the cloud resource service operation and the corresponding non-exclusive resource identification are sent, so that the matched cloud platform is utilized to perform operation processing on the cloud resource service operation, and other cloud resource service operations are allowed to be received to perform synchronous operation processing.

12. The method for scheduling heterogeneous cloud resources according to claim 1, wherein after sending the cloud resource service job to perform job processing by using the matched cloud platform, the method further comprises:

and receiving the updated resource load condition returned by the matched cloud platform so as to update the total available resources of the matched resource domain and the available resources of the matched cloud platform.

13. A scheduling apparatus for heterogeneous cloud resources, comprising:

14. The apparatus for scheduling heterogeneous cloud resources of claim 13, further comprising:

a global scheduler, configured to search a resource domain corresponding to a resource type combination of the cloud resource service job according to a resource type required by the cloud resource service job, searching a resource domain which is matched with the best resource domain again from a resource domain which does not correspond to the combination of the resource types required by the cloud resource service operation according to the resource types required by the cloud resource service operation and the total operation resource amount, and/or if a cloud platform with sufficient resources cannot be matched from the cloud platforms corresponding to the matched resource domains according to the total job resource amount, then a resource domain with sufficient resources is matched out from the searched resource domains corresponding to the rest resource types required by the cloud resource service operation again according to the total operation resource amount, and matching a cloud platform with sufficient resources from all the cloud platforms corresponding to the re-matched resource domains according to the total operation resource amount.

15. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the steps of the method according to any of claims 1 to 12 are implemented when the program is executed by the processor.

16. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 12.