CN114579268A - Resource scheduling method and device - Google Patents

Resource scheduling method and device Download PDF

Info

Publication number
CN114579268A
CN114579268A CN202210114086.5A CN202210114086A CN114579268A CN 114579268 A CN114579268 A CN 114579268A CN 202210114086 A CN202210114086 A CN 202210114086A CN 114579268 A CN114579268 A CN 114579268A
Authority
CN
China
Prior art keywords
task
resource
cluster
processing
resource scheduling
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210114086.5A
Other languages
Chinese (zh)
Inventor
张晓聪
张文军
陈文韬
王盼
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Alibaba China Co Ltd
Original Assignee
Alibaba China Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Alibaba China Co Ltd filed Critical Alibaba China Co Ltd
Priority to CN202210114086.5A priority Critical patent/CN114579268A/en
Publication of CN114579268A publication Critical patent/CN114579268A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/48—Program initiating; Program switching, e.g. by interrupt
    • G06F9/4806—Task transfer initiation or dispatching
    • G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
    • G06F9/4881—Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/465—Distributed object oriented systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
    • G06F9/5038—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering the execution order of a plurality of tasks, e.g. taking priority or time dependency constraints into consideration
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2209/00—Indexing scheme relating to G06F9/00
    • G06F2209/50—Indexing scheme relating to G06F9/50
    • G06F2209/5011—Pool
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

An embodiment of the present specification provides a resource scheduling method and an apparatus, where the resource scheduling method includes: responding to a resource scheduling processing request, predicting the amount of tasks to be processed of each task cluster in a target time interval in at least two task clusters, generating a first prediction result, determining task processing resources corresponding to each task cluster, predicting the resource consumption result of the task processing resources in each task cluster based on the resource portrait of the task processing resources and the first prediction result, and generating a second prediction result. And determining a resource scheduling strategy between the at least two task clusters according to the second prediction result, and performing resource scheduling processing on the task processing resources based on the resource scheduling strategy.

Description

Resource scheduling method and device
Technical Field
The embodiment of the specification relates to the technical field of computers, in particular to a resource scheduling method.
Background
The dispatching of the cluster resources in a single machine room is to allocate the proper cluster resources to execute the tasks to be executed, and when no idle cluster resources exist in the machine room, the tasks to be executed are placed in a queue waiting state until the idle and proper cluster resources exist. In addition, because the cluster resources configured in each machine room are relatively fixed, and the task volumes to be executed in different time periods of different machine rooms are dynamically changed, in this case, the fixed cluster resources cannot flexibly cope with the dynamically changed task volumes. If the task amount of the machine room in a certain time period is large, the fixed resource configuration cannot meet the task processing requirement, so that the task processing of the machine room consumes a long time, and the task processing efficiency is reduced.
At present, in a big data system, cluster resources distributed in multiple machine rooms can be used to execute tasks to be executed, but due to the limitation of communication functions or permissions, the resources and task states in other machine rooms cannot be known among the machine rooms, and flexible allocation of the cluster resources among the multiple machine rooms is difficult to achieve, so an effective method is urgently needed to solve the problems.
Disclosure of Invention
In view of this, the embodiments of the present disclosure provide a resource scheduling method. One or more embodiments of the present specification also relate to a resource scheduling apparatus, a computing device, a computer-readable storage medium, and a computer program, so as to solve the technical deficiencies of the prior art.
According to a first aspect of embodiments herein, there is provided a resource scheduling method, including:
responding to the resource scheduling processing request, predicting the amount of tasks to be processed of each task cluster in at least two task clusters in a target time interval, and generating a first prediction result;
determining a task processing resource corresponding to each task cluster, predicting a resource consumption result of the task processing resource in each task cluster based on the resource portrait of the task processing resource and the first prediction result, and generating a second prediction result;
determining a resource scheduling strategy between the at least two task clusters according to the second prediction result;
and performing resource scheduling processing on the task processing resources based on the resource scheduling strategy.
Optionally, the performing resource scheduling processing on the task processing resource based on the resource scheduling policy includes:
and creating a scheduling instance between the at least two task clusters based on the resource scheduling strategy, and performing resource scheduling processing on the task processing resources by executing the scheduling instance.
Optionally, the resource scheduling method further includes:
determining a historical processing task of each task cluster, and determining a reference processing flow of the historical processing task;
determining a task processing resource corresponding to each task cluster, and determining a historical processing flow corresponding to the historical processing task processed by the task processing resource;
and determining a resource representation of the task processing resource based on the reference processing flow and the historical processing flow.
Optionally, the determining a resource scheduling policy between the at least two task clusters according to the second prediction result includes:
dividing the at least two task clusters according to the position information of the at least two task clusters to generate corresponding resource pools;
and determining a resource scheduling strategy among task clusters in a target resource pool according to a second prediction result corresponding to each task cluster in the target resource pool, wherein the target resource pool is one of the resource pools.
Optionally, the determining a resource scheduling policy between the at least two task clusters according to the second prediction result includes:
determining a first task cluster with resource scheduling requirements and a second task cluster with idle task processing resources according to the second prediction result;
determining a target resource image of the idle task processing resource, and determining the matching degree between the idle task processing resource and a task to be processed in the resource scheduling requirement according to the target resource image;
and determining a resource scheduling strategy between the first task cluster and the second task cluster according to the matching degree and the resource scheduling requirement.
Optionally, the performing resource scheduling processing on the task processing resource based on the resource scheduling policy includes:
creating a scheduling instance between the first task cluster and the second task cluster based on the resource scheduling policy;
submitting a resource scheduling application to the second task cluster by executing the scheduling instance;
and under the condition that the second task cluster is determined to respond to the resource scheduling application and open the resource scheduling authority to the first task cluster, scheduling the idle task processing resources of the second task cluster to the first task cluster.
Optionally, the predicting the resource consumption result of the task processing resource in each task cluster based on the resource representation of the task processing resource and the first prediction result includes:
dividing the tasks to be processed corresponding to the first prediction result into at least two classes according to task classes, inputting the tasks to be processed of the target task class into a first prediction model, and generating a third prediction result of the task processing duration corresponding to the target task class, wherein the target task class is one of the at least two classes;
and inputting the third prediction result and the resource portrait of the task processing resource into a second prediction model so that the second prediction model predicts the resource consumption result of the task processing resource in each task cluster based on the resource portrait and the third prediction result.
Optionally, the resource scheduling method further includes:
and acquiring target task processing time corresponding to the task to be processed of the target task category in a target time interval, and updating the first prediction model based on the target task processing time.
Optionally, the determining, according to the second prediction result, that the first task cluster with the resource scheduling requirement exists and the second task cluster with the idle task processing resource exists includes:
determining a target consumption result of the task processing resources in each task cluster in the target time interval according to historical consumption information of the task processing resources in each task cluster;
and determining a first task cluster with resource scheduling requirements and a second task cluster with idle task processing resources according to the second prediction result and the target consumption result.
According to a second aspect of embodiments herein, there is provided a resource scheduling apparatus, including:
the first prediction module is configured to respond to the resource scheduling processing request, predict the amount of tasks to be processed in a target time interval of each task cluster in at least two task clusters, and generate a first prediction result;
the second prediction module is configured to determine a task processing resource corresponding to each task cluster, predict a resource consumption result of the task processing resource in each task cluster based on the resource representation of the task processing resource and the first prediction result, and generate a second prediction result;
a determining module configured to determine a resource scheduling policy between the at least two task clusters according to the second prediction result;
and the scheduling module is configured to perform resource scheduling processing on the task processing resources based on the resource scheduling policy.
According to a third aspect of embodiments herein, there is provided a computing device comprising:
a memory and a processor;
the memory is configured to store computer-executable instructions, and the processor is configured to execute the computer-executable instructions to implement any of the steps of the resource scheduling method.
According to a fourth aspect of embodiments herein, there is provided a computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the steps of any one of the resource scheduling methods.
According to a fifth aspect of embodiments herein, there is provided a computer program, wherein the computer program, when executed in a computer, causes the computer to perform the steps of the above-mentioned resource scheduling method.
In one embodiment of the present description, in response to a resource scheduling processing request, a task amount to be processed of each task cluster in at least two task clusters within a target time interval is predicted, a first prediction result is generated, a task processing resource corresponding to each task cluster is determined, a resource consumption result of the task processing resource in each task cluster is predicted based on a resource representation of the task processing resource and the first prediction result, a second prediction result is generated, a resource scheduling policy between the at least two task clusters is determined according to the second prediction result, and resource scheduling processing is performed on the task processing resource based on the resource scheduling policy.
The embodiment of the specification predicts the task amount to be processed of each task cluster in the target time interval and predicts the resource consumption result of each task cluster in the target time interval so as to determine the resource scheduling strategy among different task clusters according to the prediction results of the task clusters and realize the resource scheduling among different task clusters, thereby being beneficial to improving the resource utilization rate of the task processing resources in each task cluster and avoiding the problems that when the task amount of a task cluster is large in a certain time period, the fixed resource configuration cannot meet the task processing requirement, the task processing time consumption of the task cluster is long, the task processing efficiency is reduced, and being beneficial to improving the safe operation capability of each task cluster, and optimizing the resource allocation.
Drawings
Fig. 1 is a flowchart of a resource scheduling method according to an embodiment of the present disclosure;
fig. 2a is a schematic diagram of a resource scheduling process provided in an embodiment of the present specification;
FIG. 2b is a diagram illustrating another resource scheduling process provided by an embodiment of the present specification;
fig. 3 is a flowchart illustrating a processing procedure of a resource scheduling method according to an embodiment of the present disclosure;
fig. 4 is a schematic structural diagram of a resource scheduling apparatus according to an embodiment of the present specification;
fig. 5 is a block diagram of a computing device according to an embodiment of the present disclosure.
Detailed Description
In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present description. This description may be embodied in many different forms and should not be construed as limited to the embodiments set forth herein, as those skilled in the art will be able to make and use the present disclosure without departing from the spirit and scope of the present disclosure.
The terminology used in the description of the one or more embodiments is for the purpose of describing the particular embodiments only and is not intended to be limiting of the description of the one or more embodiments. As used in this specification and the appended claims, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used in one or more embodiments of the present specification refers to and encompasses any and all possible combinations of one or more of the associated listed items.
It will be understood that, although the terms first, second, etc. may be used herein in one or more embodiments to describe various information, these information should not be limited by these terms. These terms are only used to distinguish one type of information from another. For example, a first can also be referred to as a second and, similarly, a second can also be referred to as a first without departing from the scope of one or more embodiments of the present description. The word "if," as used herein, may be interpreted as "at … …" or "when … …" or "in response to a determination," depending on the context.
First, the noun terms referred to in one or more embodiments of the present specification are explained.
Work orders: and (5) operation and maintenance task vouchers. One work order corresponds to one operation and maintenance task in the data center, and the operation and maintenance personnel of the data center take over and process the operation and maintenance task.
Work order prediction system: and predicting the operation and maintenance workload of each machine room in a future period of time.
Resource portrayal: the method can be used for representing the time consumed by task processing resources (such as robots in a machine room) for processing different tasks to be processed, or can be used for representing the task processing efficiency of the task processing resources for processing different tasks to be processed and the like.
Resource portrait digitizing system: and based on the operation data of the task processing resources, the resource operation and maintenance portrait is accurately realized through a resource quantitative evaluation model.
Resource scheduling: and selecting a certain part of task processing resources of the data center through (artificial or system) decision, and transferring the task processing resources from one computer room to other computer rooms.
A resource scheduling engine: and outputting a cross-machine-room resource scheduling decision system based on the input of the work order prediction system and the resource portrait digitization system.
Under the condition that a plurality of master station machine rooms need to carry out data operation and maintenance, task processing resources used for carrying out operation and maintenance operations in the machine rooms need to complete tens of thousands of operation and maintenance tasks every day. Due to the fact that the scales of different machine rooms are different, the workload required to be completed every day is different, and the number of the reference task processing resources allocated to the machine rooms is different.
In addition, the operation and maintenance tasks of the data center are dynamically changed every day, and for the same machine room, the operation and maintenance work orders per day may fluctuate violently, the work orders in a certain period of time are more, and the work orders in another period of time are less. In contrast, the number of task processing resources allocated to each room is constant. Therefore, the contradiction between the daily dynamically changing workload and the constant resource allocation between the computer rooms becomes a problem to be solved urgently.
Based on this, to solve the problems of dynamic change of operation and maintenance tasks and uneven busy/idle of different task processing resources among different machine rooms of the data center, embodiments of the present specification provide a resource scheduling method, which predicts the amount of tasks to be processed of each task cluster in a target time interval, predicts the resource consumption result of each task cluster in the target time interval, and determines a resource scheduling policy among different task clusters according to the prediction results of the two, thereby implementing resource scheduling among different task clusters, and not only being beneficial to improving the resource utilization rate of task processing resources in each task cluster, but also being beneficial to avoiding that when the amount of tasks of a task cluster in a certain time period is large due to the fact that fixed cluster resources configured in a single task cluster (machine room) cannot flexibly cope with the amount of tasks that dynamically change, the fixed resource configuration cannot meet the task processing requirements, the task processing of the task cluster is long in time consumption and the task processing efficiency is reduced, so that the capability of safe operation of each task cluster is improved, and the resource allocation is optimized.
In this specification, a resource scheduling method is provided, and the specification also relates to a resource scheduling apparatus, a computing device, a computer-readable storage medium, and a computer program, which are described in detail in the following embodiments one by one.
Fig. 1 shows a flowchart of a resource scheduling method according to an embodiment of the present specification, which specifically includes the following steps.
Step 102, responding to the resource scheduling processing request, predicting the amount of tasks to be processed of each task cluster in at least two task clusters in a target time interval, and generating a first prediction result.
The resource scheduling method provided by the embodiment of the specification is applied to a resource scheduling platform.
Specifically, the task clusters are objects with tasks to be processed, and each task cluster may have task processing resources for processing the tasks to be processed. In practical application, the task cluster may be a machine room, and the task processing resource may be a task processing object belonging to different machine rooms, such as a robot.
In the embodiment of the present specification, to ensure that task processing resources in each task cluster can be reasonably utilized to improve the resource utilization rate, the task amount of the task to be processed in each task cluster in the target time interval in at least two task clusters may be predicted in advance, so as to obtain a first prediction result. The amount of the to-be-processed tasks may be the number of the to-be-processed tasks, or may be a time length required to process the to-be-processed tasks, or the like.
After the amount of the tasks to be processed of each task cluster in the target time interval is obtained through prediction, the task processing resources required to be consumed for processing the tasks to be processed in each task cluster can be further predicted, and the resource scheduling strategies among different task clusters are determined according to the prediction results of the task processing resources and the task processing resources.
During specific implementation, the task volumes corresponding to different source channels of the tasks to be processed can be predicted, so that the task volumes to be processed of all the task clusters can be predicted in a full-scale mode. The source channel of the to-be-processed task of each task cluster includes, but is not limited to, manual creation by a user, automatic creation of an upstream project processing flow, automatic creation of a resource scheduling engine, and the like.
In practical application, a manual user creating way can be applied by a scheduling manager, and operation and maintenance tasks of each task cluster in a target time interval are generally sensed based on offline information; an automatic upstream project processing flow establishing way senses operation and maintenance tasks of each task cluster based on the online processing flows of large-batch operations such as scrapping, moving and the like; and the resource scheduling engine approach outputs the operation and maintenance tasks of each task cluster based on the input of the work order prediction system and the resource portrait digitization system.
And 104, determining the task processing resources corresponding to each task cluster, predicting the resource consumption result of the task processing resources in each task cluster based on the resource representation of the task processing resources and the first prediction result, and generating a second prediction result.
Specifically, since each task cluster may have task processing resources for processing the task to be processed, after the amount of the task to be processed in the target time interval of each task cluster is obtained, the task processing resources required to be consumed for processing the amount of the task to be processed in each task cluster can be predicted, that is, the resource consumption result of the task processing resources in each task cluster is predicted.
The resource attributes of the task processing resources in different task clusters are different, or the resource attributes of different task processing resources in the same task cluster are different, so that when the resource consumption result of each task cluster is predicted, the resource portrayal of the different task processing resources in each task cluster can be combined for prediction, and the resource portrayal of the task processing resources can represent the consumed time of the task processing resources for processing different tasks to be processed, or can represent the task processing efficiency of the task processing resources for processing different tasks to be processed, and the like.
Wherein the resource representation of the different task processing resources may be determined by:
determining a historical processing task of each task cluster, and determining a reference processing flow of the historical processing task;
determining a task processing resource corresponding to each task cluster, and determining a historical processing flow corresponding to the historical processing task processed by the task processing resource;
and determining a resource representation of the task processing resource based on the reference processing flow and the historical processing flow.
Specifically, when each task cluster has a task to be processed and allocates the task to be processed to different task processing resources for task processing, the task to be processed may be allocated in a work order manner. Therefore, before task allocation, the resource images of different task processing resources in each task cluster can be determined, and corresponding tasks can be allocated to each task processing resource by combining the resource images, so that the task processing efficiency is ensured.
The resource sketch can represent the time length required by the task processing resources to process different tasks to be processed, or can represent the task processing efficiency of the task processing resources to process different tasks to be processed, so that when the resource sketch of each task processing resource is determined, the historical processing task of each task cluster can be determined, the reference processing flow of the historical processing task is determined, then the historical processing flow corresponding to the historical processing task processed by the task processing resources is determined, the resource sketch of the task processing resources is determined according to the two processing flows, and the task processing time consumption or the task processing efficiency of the task processing resources can be determined according to the difference between the two processing flows.
In specific implementation, the resource consumption result of the task processing resource in each task cluster is predicted based on the resource representation of the task processing resource and the first prediction result, and the method can be specifically implemented by the following steps:
dividing the tasks to be processed corresponding to the first prediction result into at least two classes according to task classes, inputting the tasks to be processed of the target task class into a first prediction model, and generating a third prediction result of the task processing duration corresponding to the target task class, wherein the target task class is one of the at least two classes;
and inputting the third prediction result and the resource portrait of the task processing resource into a second prediction model so that the second prediction model predicts the resource consumption result of the task processing resource in each task cluster based on the resource portrait and the third prediction result.
Specifically, the first prediction result is used for representing the prediction result of the task amount of the task to be processed in the target time interval of each task cluster, and different task clusters may have different task source channels, different task source channels may have different task types of the task to be processed, and different task types of the task to be processed may also have different processing flows.
Therefore, after the first prediction result is obtained, the to-be-processed tasks corresponding to the first prediction result may be classified, or the first prediction result may directly include the task amounts corresponding to the to-be-processed tasks of the respective task types, for example, in the first prediction result, the task amount of the to-be-processed task of the task type a is 100, the task amount of the to-be-processed task of the task type B is 200, and the task amount of the to-be-processed task of the task type C is 300.
Furthermore, when the tasks to be processed are allocated to different task processing resources for task processing, the tasks to be processed can be allocated in a work order mode. The first prediction model is used for predicting the task processing time of different work orders, so that when the first prediction model is built, the first prediction model can be built based on each task type and the predicted time consumption of the tasks to be processed of each task type, wherein the predicted time consumption of the tasks to be processed of each task type can be determined according to the execution flow of the tasks to be processed of each task type, the first prediction model specifies a reasonable time consumption interval required by actual execution of the tasks to be processed of different task types, and after training is completed, the first prediction model can be used for predicting the predicted time consumption of the tasks to be processed of different task types.
Therefore, after the first prediction model is built, the tasks to be processed of each task category in the first prediction result can be input into the first prediction model, so as to generate a third prediction result of the task processing duration corresponding to each task category.
Furthermore, the resource representation may be used to represent a processing time required by each task processing resource to process the task to be processed, or may be used to represent a processing efficiency of each task processing resource to process the task to be processed, so after obtaining the resource representation of each task processing resource in each task cluster and obtaining a third prediction result of the task processing time of each task to be processed in each task cluster, the consumption result of the task processing resource in each task cluster may be predicted based on the resource representation and the third prediction result, that is, according to the resource representation of the task processing resource of the task cluster, how many task processing resources need to be consumed in the process of processing the task to be processed in the task cluster is determined.
In practical application, the second prediction model may be used to predict the resource consumption result of the task processing resource in each task cluster, and specifically, the third prediction result corresponding to each task cluster and the resource representation of the task processing resource of each task cluster may be input into the second prediction model, so that the second prediction model predicts the resource consumption result of the task processing resource in each task cluster based on the input content.
Specifically, a second prediction model may be constructed in advance according to historical resource consumption data of each task cluster, that is, task information (time length required to be consumed for task execution) of a historical task of each task cluster and historical consumption information of task processing resources of each task cluster are used as training samples, the second prediction model to be trained is trained to obtain the second prediction model, then, in an application process, a third prediction result corresponding to each task cluster and a resource representation of the task processing resources of each task cluster may be input into the second prediction model, so that resource consumption results of the task processing resources in each task cluster in a target time interval are predicted by using the second prediction model.
Further, the first prediction model may be optimized based on the actual task processing duration of each task cluster in the target time interval, specifically, the target task processing duration corresponding to the to-be-processed task of the target task category in the target time interval is obtained, and the first prediction model is updated based on the target task processing duration.
Specifically, after the target time interval is ended, the actual task processing duration of the to-be-processed task in each task cluster in the target time interval can be obtained, the actual task processing duration can be determined according to the standard execution flow of the to-be-processed task, then the third prediction result of the task processing duration of the to-be-processed task in each task cluster and the loss value between the actual task processing durations can be calculated, and the model parameter of the first prediction model is adjusted according to the loss value.
By determining the task processing time corresponding to each task category to-be-processed task in the task cluster and determining the resource consumption result of the task processing resource in each task cluster based on the task processing time and the resource portrait of the task processing resource in the task cluster, the accuracy of the determination result of the resource consumption result is improved, and therefore the reasonability and the high efficiency of resource scheduling among the task clusters are ensured, the resource utilization rate of the task processing resource in each task cluster is improved, and resource allocation is optimized.
And 106, determining a resource scheduling strategy between the at least two task clusters according to the second prediction result.
Specifically, the second prediction result may be used to characterize resource consumption results of task processing resources corresponding to the tasks to be processed in different task clusters, so that after the second prediction result of each task cluster is determined, the resource scheduling policy among the task clusters may be determined based on the second prediction result. The resource scheduling policy may represent a resource scheduling manner among different task clusters, for example, the task processing resource Z1 in the task cluster 2 is scheduled to the task cluster 1 for task processing, and the task processing resource Z2 in the task cluster 2 is scheduled to the task cluster 3 for task processing.
In practical application, a resource scheduling engine in a resource scheduling platform serves as a recommendation decision module of the whole resource scheduling system, and a predicted resource consumption result of each task cluster in a target time interval is calculated based on predicted data of the amount of tasks to be processed of each task cluster and resource portrayal of task processing resources of each task cluster in a resource portrayal digital system. And finally outputting the scheduling or scheduled relation of the task processing resources among the task clusters by combining the existing resource configuration of the task clusters on the basis of the resource consumption result prediction.
In a specific implementation, the resource scheduling policy between the at least two task clusters is determined according to the second prediction result, which can be specifically implemented in the following manner:
dividing the at least two task clusters according to the position information of the at least two task clusters to generate corresponding resource pools;
and determining a resource scheduling strategy among task clusters in a target resource pool according to a second prediction result corresponding to each task cluster in the target resource pool, wherein the target resource pool is one of the resource pools.
Specifically, under the condition that a plurality of task clusters exist, if the distance between two task clusters is long, task processing resources are scheduled between the two task clusters, and therefore, the problems that time consumed for resource scheduling is long, task processing of the task clusters is not timely, and resource utilization rate is low may exist.
Therefore, in the embodiments of the present description, according to the location information of each task cluster, at least two task clusters are divided into one or at least two resource pools, where, when a resource pool includes at least two task clusters, a distance between any two task clusters in the resource pool is less than or equal to a preset distance threshold, and resource scheduling can be performed between at least two task clusters in the resource pool. After the resource pools are divided, only resource scheduling between any two task clusters in the resource pools can be performed, namely, a resource scheduling strategy between each task cluster in the target resource pool is determined according to a second prediction result corresponding to each task cluster in the target resource pool, wherein the target resource pool can be one or any one of at least two resource pools containing two or more task clusters, so that the efficiency of a task processing resource scheduling process is ensured, the timeliness of task processing is further ensured, and the utilization rate of task processing resources is improved.
For example, in the case that the task cluster is a machine room, a plurality of machine rooms in the same geographic area may be divided into one regional site scheduling pool. And resource scheduling can be performed among all the machine rooms in the same regional station scheduling pool. Commonly, a plurality of machine rooms in the same city can form a regional site dispatching pool.
And each computer room is available for scheduled task processing resources to enter the scheduling pool. The addition or removal of the task processing resources in the scheduling pool can be operated by a manager, and automatic decision-making can be completed by the resource representation digitizing system according to the resource representation of the task processing resources. Usually, if the average operation and maintenance workload of each task processing resource in the computer room is low, more task processing resources in the task cluster are added to the resource pool, which means that the task processing resources can be scheduled and used by other computer rooms.
In addition, the resource scheduling policy between the at least two task clusters is determined according to the second prediction result, which can also be implemented by:
determining a first task cluster with resource scheduling requirements and a second task cluster with idle task processing resources according to the second prediction result;
determining a target resource image of the idle task processing resource, and determining the matching degree between the idle task processing resource and a task to be processed in the resource scheduling requirement according to the target resource image;
and determining a resource scheduling strategy between the first task cluster and the second task cluster according to the matching degree and the resource scheduling requirement.
Further, determining, according to the second prediction result, a first task cluster having a resource scheduling requirement and a second task cluster having an idle task processing resource, includes:
determining a target consumption result of the task processing resources in each task cluster in the target time interval according to the historical consumption information of the task processing resources in each task cluster;
and determining a first task cluster with resource scheduling requirements and a second task cluster with idle task processing resources according to the second prediction result and the target consumption result.
Specifically, when determining the resource scheduling policy among different task clusters, a first task cluster having a resource scheduling requirement may be determined first, a second task cluster having idle resources may be determined, and a matching degree between the idle resources in the second task cluster and the to-be-processed tasks in the first task cluster may be determined, so that a better resource scheduling policy among the task clusters may be selected according to the matching degree.
In practical application, when a first task cluster with resource scheduling requirements and a second task cluster with idle task processing resources are determined, a target consumption result of the task processing resources in each task cluster in a target time interval can be determined according to historical consumption information of the task processing resources in each task cluster, and the first task cluster with the resource scheduling requirements and the second task cluster with the idle task processing resources are determined according to a second prediction result and the target consumption result.
The historical consumption information is that which task processing resources are actually used for processing the tasks to be processed in the task cluster exist in the task processing resources in each task cluster in the historical time interval; and a target consumption result, that is, in the target time interval, which task processing resources can be used for processing the task to be processed in each task cluster exist in the task processing resources in each task cluster.
Due to the task processing resources configured in the task cluster, in the actual use process, a component may be down due to a failure, and thus the component cannot be used for processing the task to be processed. Therefore, in practical application, according to the historical use condition (historical consumption information) of the task processing resources in each task cluster, the task processing resources (target consumption result) available in each task cluster in the target time interval can be determined, that is, the task processing resources which are possibly failed in each task cluster in the target time interval are predicted in advance, so that the resource scheduling strategy among each task cluster can be predicted more accurately according to the prediction result.
In addition, the second prediction result can represent the resource consumption result of the task processing resources in each task cluster, and the second prediction result does not consider the fault condition of the task processing resources, so that in order to ensure the accuracy of the resource scheduling strategy, after the second prediction result is obtained, the first task cluster with the resource scheduling requirement and the second task cluster with the idle task processing resources can be determined according to the second prediction result and the target consumption result corresponding to each task cluster.
In this case, if it is determined that the free resources in the second task cluster are the task resource Z1, the task processing resource Z2, and the task processing resource Z3, but the task processing resource Z3 is down due to a failure, the task resource Z1 and the task processing resource Z2 can be scheduled to another task cluster to perform task processing.
Specifically, after a first task cluster with a resource scheduling requirement and a second task cluster with an idle task processing resource are determined, the matching degree between the idle task processing resource and a task to be processed in the resource scheduling requirement of the first task cluster can be determined according to a target resource representation of the idle task processing resource in the second task cluster, and a resource scheduling strategy between the first task cluster and the second task cluster is determined according to the matching degree and the resource scheduling requirement.
The target resource portrait of the idle task processing resource can be used for representing the time length required by the idle task processing resource to process the task to be processed, or representing the efficiency of the idle task processing resource to process the task to be processed, so when the matching degree is determined, the task processing time length of the task to be processed in the first task cluster, that is, the time length required by processing the task to be processed can be determined, and then the target resource portrait of the idle task processing resource is compared with the task processing time length of the task to be processed, specifically, the time length L1 required by the idle task processing resource to process the task to be processed can be compared with the task processing time length L2 of the task to be processed, if the target resource portrait of the idle task processing resource and the task to be processed satisfy that L1 is greater than or equal to L2, the closer L1 and L2 are, the greater the matching degree of the idle task to the task to be processed is obtained.
After the matching degree is determined, a resource scheduling policy between the first task cluster and the second task cluster can be determined according to the matching degree and the resource scheduling requirement, for example, if the task cluster 1 has a resource scheduling requirement (the task to be processed in the task cluster 1 is allocated to each task processing resource in the task cluster 1, and under the condition that the task amount of the task to be processed received by each task processing resource reaches the upper limit, the task to be processed r1 which is not allocated still exists in the task cluster 1), while an idle task processing resource Z1 and an idle task processing resource Z2 exist in task cluster 2, and when it is determined that the matching degree of the idle task processing resource Z1 and the task r1 is greater than the matching degree of the idle task processing resource Z2 and the task r1, it may be determined that the resource scheduling policy between the task cluster 1 and the task cluster 2 is: the idle task processing resource Z1 is scheduled from task cluster 2 to task cluster 1.
The embodiment of the specification determines the resource scheduling policy between any two task clusters by determining the matching degree, which is beneficial to ensuring the accuracy of the determination result of the resource scheduling policy, thereby being beneficial to ensuring the rationality and the high efficiency of resource scheduling between the task clusters, further being beneficial to improving the resource utilization rate of task processing resources in each task cluster, and optimizing the resource configuration.
And 108, performing resource scheduling processing on the task processing resources based on the resource scheduling strategy.
Specifically, after determining the resource scheduling policy among different task clusters, the resource scheduling processing may be performed on the task processing resources in the task clusters based on the resource scheduling policy.
In specific implementation, the resource scheduling processing is performed on the task processing resource based on the resource scheduling policy, which can be specifically implemented in the following manner:
and creating a scheduling instance between the at least two task clusters based on the resource scheduling strategy, and performing resource scheduling processing on the task processing resources by executing the scheduling instance.
In addition, after determining a first task cluster with a resource scheduling requirement and a second task cluster with idle task processing resources, and determining a resource scheduling policy between the first task cluster and the second task cluster, performing resource scheduling processing on the task processing resources based on the resource scheduling policy, which can be specifically implemented in the following manner:
creating a scheduling instance between the first task cluster and the second task cluster based on the resource scheduling policy;
submitting a resource scheduling application to the second task cluster by executing the scheduling instance;
and under the condition that the second task cluster is determined to respond to the resource scheduling application and open the resource scheduling authority to the first task cluster, scheduling the idle task processing resources of the second task cluster to the first task cluster.
Specifically, in the scheduling execution module of the resource scheduling platform, one scheduling instance represents execution of a resource scheduling process. The primary scheduling process comprises the following core steps: submitting a scheduling application, examining and approving the scheduling application, handling scheduling authority, executing scheduling operation, recovering the scheduling authority and the like. Therefore, after the resource scheduling policy between any two task clusters is determined, a scheduling instance between the two task clusters (a first task cluster and a second task cluster) can be created based on the resource scheduling policy, a resource scheduling application is submitted to the second task cluster capable of providing idle resources by executing the scheduling instance, and the idle task processing resources of the second task cluster are scheduled to the first task cluster with the scheduling requirement under the condition that the second task cluster is determined to have the resource scheduling authority.
In the embodiment of the specification, on one hand, complex operations of field operation and maintenance personnel are avoided through an online and automatic scheduling execution module, and on the other hand, historical record query and tracing are realized.
A schematic diagram of a resource scheduling process provided in an embodiment of the present specification is shown in fig. 2 a. The resource scheduling platform predicts the amount of tasks to be processed of each task cluster based on three task source channels of manual user creation, automatic upstream project processing flow creation and automatic resource scheduling engine creation respectively, predicts the scheduling requirements of different task clusters according to the prediction results and resource representations of task processing resources in each task cluster, and then puts the predicted resource scheduling requirements into a scheduling requirement pool. For each scheduling requirement in the scheduling requirement pool, a scheduling instance can be respectively created for each scheduling requirement, and then resource scheduling is carried out among task clusters by executing the scheduling instances.
Another schematic diagram of a resource scheduling process provided in this embodiment is shown in fig. 2 b. The task amount to be processed of the task cluster in the target time interval can be predicted through the task amount prediction model to generate a first prediction result, then the first prediction result is input into the task processing duration prediction model (corresponding to the first prediction model of fig. 2 b), the time consumption prediction is performed on the task to be processed of the task cluster through the task processing duration prediction model to generate a second prediction result, and then the resource portrait of the task processing resources in the task cluster is predicted through the resource portrait prediction model (corresponding to the second prediction model of fig. 2 b) to obtain a third prediction result; and then inputting the second prediction result and the third prediction result into a resource consumption result prediction model to obtain a resource consumption result corresponding to the task cluster, namely a fourth prediction result.
In addition, in the embodiment of the present specification, a target consumption result of the task processing resources in the task cluster may be predicted through a target consumption result prediction model to obtain a fifth prediction result, then, a resource scheduling policy between the task clusters is predicted according to the fourth prediction result and the fifth prediction result, and based on the prediction result of the resource scheduling policy, resource scheduling between the task clusters is performed in the resource pool.
In practical application, the task quantity prediction model, the task processing duration prediction model, the resource portrait prediction model, the resource consumption result prediction model and the target consumption result prediction model can be fed back in the forward direction through actual data, model parameters of the models are adjusted, and the models are updated, so that the accuracy of output results of the models is guaranteed. For the resource scheduling process, the resource scheduling process can be properly adjusted through forward feedback of actual data so as to ensure the reasonability and the high efficiency of resource scheduling.
In one embodiment of the present description, in response to a resource scheduling processing request, a task amount to be processed of each task cluster in at least two task clusters within a target time interval is predicted, a first prediction result is generated, a task processing resource corresponding to each task cluster is determined, a resource consumption result of the task processing resource in each task cluster is predicted based on a resource representation of the task processing resource and the first prediction result, a second prediction result is generated, a resource scheduling policy between the at least two task clusters is determined according to the second prediction result, and resource scheduling processing is performed on the task processing resource based on the resource scheduling policy.
The embodiment of the specification predicts the task amount to be processed of each task cluster in the target time interval and predicts the resource consumption result of each task cluster in the target time interval so as to determine the resource scheduling strategy among different task clusters according to the prediction results of the task clusters and realize the resource scheduling among different task clusters, thereby being beneficial to improving the resource utilization rate of the task processing resources in each task cluster and avoiding the problems that when the task amount of a task cluster is large in a certain time period, the fixed resource configuration cannot meet the task processing requirement, the task processing time consumption of the task cluster is long, the task processing efficiency is reduced, and being beneficial to improving the safe operation capability of each task cluster, and optimizing the resource allocation.
The resource scheduling method provided in this specification is further described below with reference to fig. 3, taking an application of the resource scheduling method in an airport room as an example. Fig. 3 shows a flowchart of a processing procedure of a resource scheduling method according to an embodiment of the present specification, which specifically includes the following steps.
Step 302, in response to the resource scheduling processing request, predicting the amount of tasks to be processed in each machine room within a target time interval in at least two machine rooms, and generating a first prediction result.
And step 304, determining the historical processing task of each machine room, and determining the reference processing flow of the historical processing task.
Step 306, determining the task processing resources corresponding to each machine room, and determining the historical processing flow corresponding to the historical processing task processed by the task processing resources.
Step 308, determining a resource representation of the task processing resource based on the reference processing flow and the historical processing flow.
And 310, predicting the resource consumption result of the task processing resources in each machine room based on the resource portrait of the task processing resources and the first prediction result to generate a second prediction result.
Step 312, according to the location information of the at least two machine rooms, the at least two machine rooms are divided to generate corresponding resource pools.
And step 314, determining a first machine room with resource scheduling requirements and a second machine room with idle task processing resources in the target resource pool according to a second prediction result corresponding to each machine room in the target resource pool, wherein the target resource pool is one of the resource pools.
And step 316, determining a target resource image of the idle task processing resource, and determining the matching degree between the idle task processing resource and the task to be processed in the resource scheduling requirement according to the target resource image.
And step 318, determining resource scheduling strategies of the first machine room and the second machine room according to the matching degree and the resource scheduling requirements.
Step 320, creating a scheduling instance of the first machine room and the second machine room based on the resource scheduling policy.
Step 322, submitting resource scheduling application to the second machine room by executing the scheduling instance.
Step 324, under the condition that the second machine room is determined to respond to the resource scheduling application and open the resource scheduling authority to the first machine room, scheduling the idle task processing resource of the second machine room to the first machine room.
The embodiment of the specification predicts the task amount to be processed of each machine room in the target time interval, predicts the resource consumption result of each machine room in the target time interval, and determines the resource scheduling strategy between different machine rooms according to the prediction results of the two, so that resource scheduling of different machine rooms is realized, which is not only beneficial to improving the resource utilization rate of task processing resources in each machine room, but also beneficial to avoiding the problems that when the task amount of the machine room is more due to the fact that the fixed task processing resources configured in a single machine room cannot flexibly cope with the dynamically-changed task amount, the fixed resource configuration cannot meet the task processing requirement, so that the task processing of the machine room consumes longer time, the task processing efficiency is reduced, thereby being beneficial to improving the capability of safe operation of each machine room and optimizing the resource configuration.
Corresponding to the above method embodiment, this specification further provides a resource scheduling apparatus embodiment, and fig. 4 shows a schematic structural diagram of a resource scheduling apparatus provided in an embodiment of this specification. As shown in fig. 4, the apparatus includes:
a first prediction module 402, configured to predict, in response to the resource scheduling processing request, a to-be-processed task amount of each task cluster in the at least two task clusters within the target time interval, and generate a first prediction result;
a second prediction module 404, configured to determine a task processing resource corresponding to each task cluster, predict a resource consumption result of the task processing resource in each task cluster based on the resource representation of the task processing resource and the first prediction result, and generate a second prediction result;
a determining module 406 configured to determine a resource scheduling policy between the at least two task clusters according to the second prediction result;
a scheduling module 408 configured to perform resource scheduling processing on the task processing resource based on the resource scheduling policy.
Optionally, the scheduling module 408 is further configured to:
and creating a scheduling instance between the at least two task clusters based on the resource scheduling strategy, and performing resource scheduling processing on the task processing resources by executing the scheduling instance.
Optionally, the resource scheduling apparatus further includes a processing module configured to:
determining a historical processing task of each task cluster, and determining a reference processing flow of the historical processing task;
determining a task processing resource corresponding to each task cluster, and determining a historical processing flow corresponding to the historical processing task processed by the task processing resource;
and determining a resource representation of the task processing resource based on the reference processing flow and the historical processing flow.
Optionally, the determining module 406 is further configured to:
dividing the at least two task clusters according to the position information of the at least two task clusters to generate corresponding resource pools;
and determining a resource scheduling strategy among task clusters in a target resource pool according to a second prediction result corresponding to each task cluster in the target resource pool, wherein the target resource pool is one of the resource pools.
Optionally, the determining module 406 is further configured to:
determining a first task cluster with resource scheduling requirements and a second task cluster with idle task processing resources according to the second prediction result;
determining a target resource image of the idle task processing resource, and determining the matching degree between the idle task processing resource and a task to be processed in the resource scheduling requirement according to the target resource image;
and determining a resource scheduling strategy between the first task cluster and the second task cluster according to the matching degree and the resource scheduling requirement.
Optionally, the determining module 406 is further configured to:
creating a scheduling instance between the first task cluster and the second task cluster based on the resource scheduling policy;
submitting a resource scheduling application to the second task cluster by executing the scheduling instance;
and under the condition that the second task cluster is determined to respond to the resource scheduling application and open the resource scheduling authority to the first task cluster, scheduling the idle task processing resources of the second task cluster to the first task cluster.
Optionally, the first prediction module 402 is further configured to:
dividing the tasks to be processed corresponding to the first prediction result into at least two classes according to task classes, inputting the tasks to be processed of a target task class into a first prediction model, and generating a third prediction result of the task processing duration corresponding to the target task class, wherein the target task class is one of the at least two classes;
and inputting the third prediction result and the resource portrait of the task processing resource into a second prediction model so that the second prediction model predicts the resource consumption result of the task processing resource in each task cluster based on the resource portrait and the third prediction result.
Optionally, the resource scheduling apparatus further includes an update module configured to:
and acquiring target task processing time corresponding to the task to be processed of the target task category in a target time interval, and updating the first prediction model based on the target task processing time.
Optionally, the determining module 406 is further configured to:
determining a target consumption result of the task processing resources in each task cluster in the target time interval according to the historical consumption information of the task processing resources in each task cluster;
and determining a first task cluster with resource scheduling requirements and a second task cluster with idle task processing resources according to the second prediction result and the target consumption result.
The foregoing is a schematic scheme of a resource scheduling apparatus in this embodiment. It should be noted that the technical solution of the resource scheduling apparatus and the technical solution of the resource scheduling method belong to the same concept, and details that are not described in detail in the technical solution of the resource scheduling apparatus can be referred to the description of the technical solution of the resource scheduling method.
FIG. 5 illustrates a block diagram of a computing device 500 provided in accordance with one embodiment of the present description. The components of the computing device 500 include, but are not limited to, a memory 510 and a processor 520. Processor 520 is coupled to memory 510 via bus 530, and database 550 is used to store data.
Computing device 500 also includes access device 540, access device 540 enabling computing device 500 to communicate via one or more networks 560. Examples of such networks include the Public Switched Telephone Network (PSTN), a Local Area Network (LAN), a Wide Area Network (WAN), a Personal Area Network (PAN), or a combination of communication networks such as the internet. The access device 540 may include one or more of any type of network interface, e.g., a Network Interface Card (NIC), wired or wireless, such as an IEEE802.11 Wireless Local Area Network (WLAN) wireless interface, a worldwide interoperability for microwave access (Wi-MAX) interface, an ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a bluetooth interface, a Near Field Communication (NFC) interface, and so forth.
In one embodiment of the present description, the above-described components of computing device 500, as well as other components not shown in FIG. 5, may also be connected to each other, such as by a bus. It should be understood that the block diagram of the computing device architecture shown in FIG. 5 is for purposes of example only and is not limiting as to the scope of the present description. Those skilled in the art may add or replace other components as desired.
Computing device 500 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., tablet, personal digital assistant, laptop, notebook, netbook, etc.), mobile phone (e.g., smartphone), wearable computing device (e.g., smartwatch, smartglasses, etc.), or other type of mobile device, or a stationary computing device such as a desktop computer or PC. Computing device 500 may also be a mobile or stationary server.
Wherein the processor 520 is configured to execute computer-executable instructions that, when executed by the processor, implement the steps of the resource scheduling method described above.
The above is an illustrative scheme of a computing device of the present embodiment. It should be noted that the technical solution of the computing device and the technical solution of the resource scheduling method belong to the same concept, and details that are not described in detail in the technical solution of the computing device can be referred to the description of the technical solution of the resource scheduling method.
An embodiment of the present specification further provides a computer-readable storage medium, which stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, the steps of the resource scheduling method are implemented.
The above is an illustrative scheme of a computer-readable storage medium of the present embodiment. It should be noted that the technical solution of the storage medium belongs to the same concept as the technical solution of the above resource scheduling method, and details that are not described in detail in the technical solution of the storage medium can be referred to the description of the technical solution of the above resource scheduling method.
An embodiment of the present specification further provides a computer program, wherein when the computer program is executed in a computer, the computer is caused to execute the steps of the resource scheduling method.
The above is an illustrative scheme of a computer program of the present embodiment. It should be noted that the technical solution of the computer program and the technical solution of the resource scheduling method belong to the same concept, and details that are not described in detail in the technical solution of the computer program can be referred to the description of the technical solution of the resource scheduling method.
The foregoing description has been directed to specific embodiments of this disclosure. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims may be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing may also be possible or may be advantageous.
The computer instructions comprise computer program code which may be in the form of source code, object code, an executable file or some intermediate form, or the like. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, usb disk, removable hard disk, magnetic disk, optical disk, computer Memory, Read-Only Memory (ROM), Random Access Memory (RAM), electrical carrier wave signals, telecommunications signals, software distribution medium, etc. It should be noted that the computer readable medium may contain content that is subject to appropriate increase or decrease as required by legislation and patent practice in jurisdictions, for example, in some jurisdictions, computer readable media does not include electrical carrier signals and telecommunications signals as is required by legislation and patent practice.
It should be noted that, for the sake of simplicity, the foregoing method embodiments are described as a series of acts, but those skilled in the art should understand that the present embodiment is not limited by the described acts, because some steps may be performed in other sequences or simultaneously according to the present embodiment. Further, those skilled in the art should also appreciate that the embodiments described in this specification are preferred embodiments and that acts and modules referred to are not necessarily required for an embodiment of the specification.
In the above embodiments, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.
The preferred embodiments of the present specification disclosed above are intended only to aid in the description of the specification. Alternative embodiments are not exhaustive and do not limit the invention to the precise embodiments described. Obviously, many modifications and variations are possible in light of the above teaching. The embodiments were chosen and described in order to best explain the principles of the embodiments and the practical application, to thereby enable others skilled in the art to best understand and utilize the embodiments. The specification is limited only by the claims and their full scope and equivalents.

Claims (12)

1. A resource scheduling method comprises the following steps:
responding to the resource scheduling processing request, predicting the amount of tasks to be processed of each task cluster in at least two task clusters in a target time interval, and generating a first prediction result;
determining a task processing resource corresponding to each task cluster, predicting a resource consumption result of the task processing resource in each task cluster based on the resource portrait of the task processing resource and the first prediction result, and generating a second prediction result;
determining a resource scheduling strategy between the at least two task clusters according to the second prediction result;
and performing resource scheduling processing on the task processing resources based on the resource scheduling strategy.
2. The resource scheduling method of claim 1, wherein the performing resource scheduling processing on the task processing resource based on the resource scheduling policy comprises:
and creating a scheduling instance between the at least two task clusters based on the resource scheduling strategy, and performing resource scheduling processing on the task processing resources by executing the scheduling instance.
3. The resource scheduling method of claim 1, further comprising:
determining a historical processing task of each task cluster, and determining a reference processing flow of the historical processing task;
determining a task processing resource corresponding to each task cluster, and determining a historical processing flow corresponding to the historical processing task processed by the task processing resource;
and determining a resource representation of the task processing resource based on the reference processing flow and the historical processing flow.
4. The resource scheduling method according to claim 1, wherein the determining the resource scheduling policy between the at least two task clusters according to the second prediction result comprises:
dividing the at least two task clusters according to the position information of the at least two task clusters to generate corresponding resource pools;
and determining a resource scheduling strategy among task clusters in a target resource pool according to a second prediction result corresponding to each task cluster in the target resource pool, wherein the target resource pool is one of the resource pools.
5. The resource scheduling method according to claim 1, wherein the determining the resource scheduling policy between the at least two task clusters according to the second prediction result comprises:
determining a first task cluster with resource scheduling requirements and a second task cluster with idle task processing resources according to the second prediction result;
determining a target resource image of the idle task processing resource, and determining the matching degree between the idle task processing resource and a task to be processed in the resource scheduling requirement according to the target resource image;
and determining a resource scheduling strategy between the first task cluster and the second task cluster according to the matching degree and the resource scheduling requirement.
6. The resource scheduling method of claim 5, wherein the performing resource scheduling processing on the task processing resource based on the resource scheduling policy comprises:
creating a scheduling instance between the first task cluster and the second task cluster based on the resource scheduling policy;
submitting a resource scheduling application to the second task cluster by executing the scheduling instance;
and under the condition that the second task cluster is determined to respond to the resource scheduling application and open the resource scheduling authority to the first task cluster, scheduling the idle task processing resources of the second task cluster to the first task cluster.
7. The method of claim 1, wherein predicting the resource consumption of the task processing resources in each task cluster based on the resource representation of the task processing resources and the first prediction result comprises:
dividing the tasks to be processed corresponding to the first prediction result into at least two classes according to task classes, inputting the tasks to be processed of the target task class into a first prediction model, and generating a third prediction result of the task processing duration corresponding to the target task class, wherein the target task class is one of the at least two classes;
and inputting the third prediction result and the resource portrait of the task processing resource into a second prediction model so that the second prediction model predicts the resource consumption result of the task processing resource in each task cluster based on the resource portrait and the third prediction result.
8. The resource scheduling method of claim 7, further comprising:
and acquiring target task processing time corresponding to the task to be processed of the target task category in a target time interval, and updating the first prediction model based on the target task processing time.
9. The resource scheduling method according to claim 5, wherein the determining, according to the second prediction result, that there is a first task cluster with resource scheduling requirement and that there is a second task cluster with idle task processing resource, includes:
determining a target consumption result of the task processing resources in each task cluster in the target time interval according to historical consumption information of the task processing resources in each task cluster;
and determining a first task cluster with resource scheduling requirements and a second task cluster with idle task processing resources according to the second prediction result and the target consumption result.
10. A resource scheduling apparatus, comprising:
the first prediction module is configured to respond to the resource scheduling processing request, predict the amount of tasks to be processed in a target time interval of each task cluster in at least two task clusters, and generate a first prediction result;
the second prediction module is configured to determine a task processing resource corresponding to each task cluster, predict a resource consumption result of the task processing resource in each task cluster based on the resource representation of the task processing resource and the first prediction result, and generate a second prediction result;
a determining module configured to determine a resource scheduling policy between the at least two task clusters according to the second prediction result;
and the scheduling module is configured to perform resource scheduling processing on the task processing resources based on the resource scheduling policy.
11. A computing device, comprising:
a memory and a processor;
the memory is configured to store computer-executable instructions, and the processor is configured to execute the computer-executable instructions, which when executed by the processor, implement the steps of the resource scheduling method of any one of claims 1 to 9.
12. A computer-readable storage medium storing computer-executable instructions which, when executed by a processor, perform the steps of the resource scheduling method of any one of claims 1 to 9.
CN202210114086.5A 2022-01-30 2022-01-30 Resource scheduling method and device Pending CN114579268A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210114086.5A CN114579268A (en) 2022-01-30 2022-01-30 Resource scheduling method and device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210114086.5A CN114579268A (en) 2022-01-30 2022-01-30 Resource scheduling method and device

Publications (1)

Publication Number Publication Date
CN114579268A true CN114579268A (en) 2022-06-03

Family

ID=81772256

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210114086.5A Pending CN114579268A (en) 2022-01-30 2022-01-30 Resource scheduling method and device

Country Status (1)

Country Link
CN (1) CN114579268A (en)

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108564247A (en) * 2018-03-02 2018-09-21 阿里巴巴集团控股有限公司 Data adjustment method, device and server
CN110287003A (en) * 2019-06-28 2019-09-27 北京九章云极科技有限公司 The management method and management system of resource
WO2019223283A1 (en) * 2018-05-24 2019-11-28 国网上海市电力公司 Combinatorial optimization scheduling method for predicting task execution time
CN111198767A (en) * 2020-01-07 2020-05-26 平安科技（深圳）有限公司 Big data resource processing method and device, terminal and storage medium
CN112596898A (en) * 2020-12-16 2021-04-02 北京三快在线科技有限公司 Task executor scheduling method and device
CN113934535A (en) * 2021-10-11 2022-01-14 广东科诺勘测工程有限公司 Mass point cloud data processing method, device, server and system

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108564247A (en) * 2018-03-02 2018-09-21 阿里巴巴集团控股有限公司 Data adjustment method, device and server
WO2019223283A1 (en) * 2018-05-24 2019-11-28 国网上海市电力公司 Combinatorial optimization scheduling method for predicting task execution time
CN110287003A (en) * 2019-06-28 2019-09-27 北京九章云极科技有限公司 The management method and management system of resource
CN111198767A (en) * 2020-01-07 2020-05-26 平安科技（深圳）有限公司 Big data resource processing method and device, terminal and storage medium
CN112596898A (en) * 2020-12-16 2021-04-02 北京三快在线科技有限公司 Task executor scheduling method and device
CN113934535A (en) * 2021-10-11 2022-01-14 广东科诺勘测工程有限公司 Mass point cloud data processing method, device, server and system

Similar Documents

Publication Publication Date Title
Tuli et al. COSCO: Container orchestration using co-simulation and gradient based optimization for fog computing environments
Chen et al. Computation offloading and task scheduling for DNN-based applications in cloud-edge computing
US11436050B2 (en) Method, apparatus and computer program product for resource scheduling
CN111274036B (en) Scheduling method of deep learning task based on speed prediction
CN109165808B (en) Power communication network on-site operation and maintenance work order distribution method
CN112433852B (en) Internet of things edge calculation control method, device, equipment and storage medium
Wang et al. Joint server assignment and resource management for edge-based MAR system
CN113467944B (en) Resource deployment device and method for complex software system
CN113946431B (en) Resource scheduling method, system, medium and computing device
CN111949795A (en) Work order automatic classification method and device
CN109377063A (en) A kind of method for scheduling task, device, equipment and storage medium
CN113240226A (en) Resource distribution method and device between satellite ground stations
CN111369011A (en) Method and device for applying machine learning model, computer equipment and storage medium
CN116127899A (en) Chip design system, method, electronic device, and storage medium
CN115599524A (en) Data lake system based on cooperative scheduling processing of streaming data and batch data
Tan et al. Graph neural network-based cell switching for energy optimization in ultra-dense heterogeneous networks
EP4300305A1 (en) Methods and systems for energy-efficient scheduling of periodic tasks on a group of processing devices
CN114579268A (en) Resource scheduling method and device
WO2020062047A1 (en) Scheduling rule updating method, device, system, storage medium and terminal
CN115952054A (en) Simulation task resource management method, device, equipment and medium
CN114035919A (en) Task scheduling system and method based on power distribution network layered distribution characteristics
CN114862098A (en) Resource allocation method and device
CN113238846A (en) Task scheduling method and device
CN113821313A (en) Task scheduling method and device and electronic equipment
CN112884391A (en) Receiving and dispatching piece planning method and device, electronic equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination