CN111258737B

CN111258737B - Resource scheduling method and device and filter scheduler

Info

Publication number: CN111258737B
Application number: CN202010057844.5A
Authority: CN
Inventors: 朱旭琪
Original assignee: Huawei Technologies Co Ltd
Current assignee: Huawei Technologies Co Ltd
Priority date: 2016-10-19
Filing date: 2016-10-19
Publication date: 2023-12-08
Anticipated expiration: 2036-10-19
Also published as: CN107968802B; CN107968802A; WO2018072687A1; CN111258737A

Abstract

The invention relates to a resource scheduling method, a device and a filter type scheduler, wherein the method comprises the following steps: the method comprises the steps that a first filter type scheduler obtains a scheduling request message carrying service demands, converts the scheduling request message into a scheduling task message, informs each filter type scheduler in a scheduling system in an asynchronous communication mode, and indicates the filter type scheduler to determine a scheduling result in a virtual group associated with the filter type scheduler according to the service demands, and sends the scheduling result to the first filter type scheduler; acquiring a scheduling result determined by a filter scheduler in a scheduling system according to the scheduling task message; when the preset condition is met, determining a final scheduling result in the acquired scheduling results according to a preset algorithm, wherein the final scheduling result comprises one or more servers meeting the service requirements. Therefore, the processing capacity and the dispatching efficiency of the whole dispatching system are improved.

Description

Resource scheduling method and device and filter scheduler

Technical Field

The present invention relates to the field of communications, and in particular, to a method, an apparatus, and a filter scheduler for resource scheduling in the field of virtualization management.

Background

Along with the wide application of virtualization technology, how to better realize unified management and scheduling of resources by using a cloud computing management platform is getting more attention, one of the important technologies of the management platform is resource scheduling, namely, distributing service requests to appropriate resources, including computing, storage, network and other resources, so as to improve the utilization rate of the resources.

In a cloud computing management platform, the most commonly used scheduler at present is a filtering scheduler, namely when a scheduling request is received, firstly screening out all server sets meeting service requirements according to the service requirements carried in the scheduling request, and then sequencing all servers in the server sets according to a predefined weight computing method, wherein the scheduler selects a server with the highest weight to process the service requirements. However, with the increasing service scenario in a large-scale data center, a single filter scheduler is responsible for scheduling all resources in a virtualized system, so that service demands cannot be met, a serious performance bottleneck is caused, and more data centers adopt a plurality of filter schedulers to implement scheduling requests in parallel so as to share the scheduling tasks of the whole large-scale data center.

In the prior art, a plurality of filter schedulers are adopted in a scheduling system to form a scheduler cluster, when a plurality of scheduling requests to be processed exist in a message queue, the filter schedulers in the scheduler cluster can respectively acquire different scheduling requests and determine scheduling results according to the different scheduling requests, but for the same scheduling request, a filter scheduler firstly screens out a server set meeting the scheduling requests from all servers in the scheduling system, then calculates weights of all servers in the server set according to a predefined weight calculation method, sorts the servers according to the weights, finally selects one or more servers as scheduling results of the scheduling requests, and in a large-scale data center, the same scheduling method needs the same filter scheduler to screen and sort in all servers, so that the processing time of the same scheduling task is long and the processing efficiency is low. On the other hand, in the prior art, the available resource of each server is recorded and saved by the database, but the server only notifies the database to update the available resource of the server when the service requirement in the scheduling request is completed. That is, the available resource information of the server recorded in the database is non-real-time information, and the server determined by the filter scheduler according to the information may not actually meet the service requirement, so that the judgment of the filter scheduler is affected, and the probability of scheduling failure is increased. Therefore, the method for parallel scheduling of multiple filter schedulers in the prior art has the problems of long scheduling request processing time, low efficiency and service processing failure caused by misjudgment, and affects the resource scheduling capability and resource allocation efficiency of a scheduling system.

Disclosure of Invention

The embodiment of the invention provides a resource scheduling method, a resource scheduling device and a filtering scheduler, which can solve the problems of service processing failure caused by long scheduling request processing time, low efficiency and misjudgment of a scheduling system in the prior art, and improve the resource scheduling capability and resource allocation efficiency of the scheduler system.

In order to achieve the above purpose, the invention adopts the following technical scheme:

in a first aspect, a method for scheduling resources is provided, where the method is applied to a scheduling system, the scheduling system includes at least two filter schedulers and at least two servers, each filter scheduler is associated with a virtual group, each virtual group includes all servers that can be scheduled by the filter scheduler associated with the virtual group, and one server belongs to only one virtual group; firstly, a first filter type scheduler obtains a scheduling request message, wherein the scheduling request message carries service requirements, and the first filter type scheduler is any filter type scheduler in a scheduling system; then, the first filter type dispatcher converts the dispatching request message into dispatching task messages, and notifies each filter type dispatcher in a dispatching system in an asynchronous communication mode, wherein the dispatching task messages carry service requirements and role information of the first filter type dispatcher, the dispatching task messages are used for indicating the filter type dispatcher to determine dispatching results in virtual groups associated with the filter type dispatcher according to the service requirements, and the dispatching results are sent to the first filter type dispatcher, and the dispatching results comprise one or more servers meeting the service requirements; next, the first filter type scheduler obtains a scheduling result determined by the filter type scheduler in the scheduling system according to the scheduling task information; when the preset condition is met, the first filter type scheduler determines a final scheduling result in the obtained scheduling results according to a preset algorithm, wherein the final scheduling result comprises one or more servers meeting service requirements.

It is worth noting that one or more servers are included in each virtual group.

Optionally, the scheduling task message may further include identification information and IP address information of the first filter scheduler.

Optionally, during the process of processing the scheduling task message, the first filter scheduler acquiring the scheduling request message may record the role information of each filter scheduler in the scheduling task message by using a data table in a designated file or database, for example, the roles of the filter schedulers may include a Leader and a Worker, where the filter scheduler of the Worker role is responsible for determining, in the set of at least one server that can be scheduled by the filter scheduler, that one or more servers meeting the service requirement in the associated virtual group are scheduling results after screening and sorting; the filter type dispatcher of the Leader role is a filter type dispatcher for acquiring dispatching request information, and is responsible for converting the dispatching request information into dispatching task information, notifying all filter type dispatchers in a dispatching system that the dispatching task information to be processed exists, summarizing dispatching results determined by the filter type dispatchers of all the workbench roles, and selecting one or more services meeting service requirements from all the summarized dispatching results as a final dispatching result according to a preset algorithm.

It should be noted that, the role information is temporary information, that is, only takes effect in the processing procedure of the scheduling request message. When a plurality of scheduling request messages exist in the system at the same time, after the scheduling task messages are acquired by different filter schedulers, each scheduling request message is responsible for determining a final scheduling result by the filter scheduler of one Leader role, and each type of role can only process one message at a time, namely the filter scheduler of the Leader role can only process one scheduling request message at a time, and the filter scheduler of the Worker role can only process one scheduling task message at a time. And after the processing of the scheduling request message is finished, the filter scheduler releases the role information corresponding to the current scheduling request message processing process.

In this embodiment, each filter scheduler in the scheduler system determines a scheduling result in a server set capable of scheduling according to the same scheduling request, and sends the scheduling result to the filter scheduler for obtaining the scheduling request message, when a preset condition is met, the filter scheduler determines one or more servers as a final scheduling result from all summarized scheduling results, and compared with the prior art that only one filter scheduler determines a scheduling result for the same scheduling request message, the method has the advantages that the problems of long scheduling task processing time and low efficiency in the prior art are solved by parallel processing of multiple filter schedulers, and resource scheduling capability and resource allocation efficiency of the scheduling system are improved.

In one possible implementation, the notifying each filter scheduler in the scheduling system by an asynchronous communication manner includes storing the scheduling task message in a message queue by a first filter scheduler, so that each filter scheduler listens to the message queue and acquires the scheduling task message.

It should be noted that, when the scheduling request message in the message queue is taken away by any filter scheduler, in order to avoid the problem of resource waste caused by repeated processing, the message queue software will delete the scheduling request message in the message queue, and other filter schedulers cannot acquire the scheduling request message again. The scheduling task always exists in the processing process of the scheduling request until the first filter processes the scheduling task, and then the message queue software is notified to delete the scheduling task message, so that all filter schedulers in the storage cluster can acquire the scheduling task in an asynchronous mode. After the filtered schedulers in the storage cluster acquire the scheduled task, a server which is most suitable for executing the service requirement in the scheduled task in the virtual group corresponding to each filtered scheduler can be determined.

Alternatively, the scheduling task message may be stored in a database, and the first filter scheduler may acquire the scheduling request message from the database by monitoring the database, that is, when there is a scheduling request message to be processed in the database.

In another possible implementation, the notifying each filter scheduler in the scheduling system by asynchronous communication includes the first filter scheduler storing the scheduling task message in a database such that each filter scheduler listens to the database and obtains the scheduling task message.

In another possible implementation manner, the determining, by the filtered scheduler, a scheduling result in a virtual group associated with the filtered scheduler according to the service requirement includes: the filter scheduler obtains the scheduling task message; the filter type dispatcher determines a server meeting the service requirement in a virtual group associated with the filter type dispatcher according to the service requirement; and the filter type dispatcher sorts the determined servers according to a predefined weight algorithm, and sequentially selects one or more servers as the dispatching result according to the sequence.

In another possible implementation manner, the preset condition includes:

the first filter scheduler obtains that the number of the scheduling results meets a first threshold; or (b)

The first filtered scheduler obtains that the number of scheduling results meets a first threshold and the time of receipt meets a second threshold.

It should be understood by those skilled in the art that each filter scheduler can only process one scheduling task message at a time, when there are multiple scheduling task messages to be processed in the scheduling system at the same time, for the same filter scheduler, if the current scheduling task request message is not processed for a long time, the filter scheduler cannot acquire and process a new scheduling task message, in order to avoid the influence of the processing process of the previous scheduling task on the current scheduling task, the first filter scheduler may set a first threshold when determining the final scheduling result, and when the number of summarized scheduling results meets the first threshold, one or more servers are selected as the final scheduling result from the acquired scheduling results.

In another possible implementation manner, the method for determining the final scheduling result in the scheduling results by the first filter scheduler according to a preset algorithm includes any one of the following ways:

The first filtering scheduler sorts all servers in the scheduling result according to the weight determined by the predefined weight algorithm, and sequentially selects one or more servers as the final scheduling result; or (b)

The first filter type scheduler sequentially selects one or more servers from the scheduling results according to the receiving time sequence as the final scheduling result; or (b)

The first filter scheduler randomly selects one or more servers from the scheduling results as the final scheduling results;

the predefined weight algorithm may define different weights according to at least one of a CPU remaining amount, a memory remaining amount, a disk remaining amount, a network traffic, a CPU type, a disk type, a network card type, and a number of times that the disk performs reading/writing per second of the server, and calculate to obtain a weight of each server.

It should be noted that, after determining the final scheduling result, the first filter scheduler may notify the message queue software to delete the processed scheduling task message. Meanwhile, the first filter type scheduler also sends the optimal scheduling result to a task allocation node, the task allocation node sends the service requirement in the scheduling task to a server corresponding to the final scheduling result, and the server executes the specific implementation process of the service requirement.

In summary, for a scheduling request message, each filter scheduler in the scheduling system performs work division cooperation, determines a scheduling result in each associated virtual group, and one filter scheduler gathers the scheduling results determined by the filter scheduler according to the scheduling task message, and determines one or more servers as a final scheduling result in the scheduling results. On the other hand, each filter scheduler uses the resource pre-occupying information to record the resource information required by the determined scheduling result, so that the scheduling failure problem caused by the fact that the resource occupying condition of the record server in the database cannot be updated in real time in the prior art is solved, and the capacity of the scheduler cluster for managing new resources in large-scale data is improved.

In a second aspect, there is provided an apparatus for resource scheduling, the apparatus comprising means for performing the method for resource scheduling in the first aspect or any one of the possible implementations of the first aspect.

In a third aspect, a method for scheduling resources is provided, where the method is applied to a scheduling system, where the scheduling system includes at least two filter schedulers and at least two servers, where each filter scheduler is associated with a virtual group, each virtual group includes all servers that can be scheduled by the filter scheduler associated with the virtual group, and one server belongs to only one virtual group; firstly, a first filter type scheduler obtains a scheduling request message, wherein the scheduling request message carries service requirements, and the first filter type scheduler is any filter type scheduler in a scheduling system; then, the first filter type scheduler converts the scheduling request message into a scheduling task message, and sends the scheduling task message to other servers except the first filter type scheduler in a scheduling system, wherein the scheduling task message carries service requirements and role information of the first filter type scheduler, the scheduling task message is used for indicating the other filter type schedulers except the first filter type scheduler in the scheduling system to determine scheduling results in virtual groups associated with the other filter type schedulers according to the service requirements, and sends the scheduling results to the first filter type scheduler; next, the first filter type scheduler obtains a scheduling result determined by the filter type scheduler in the scheduling system according to the scheduling task information; when the preset condition is met, the first filter type scheduler determines a final scheduling result in the obtained scheduling results according to a preset algorithm, wherein the final scheduling result comprises one or more servers meeting service requirements.

Through the description, the first filtering type and other filtering type schedulers in the scheduling system directly communicate, and for the same scheduling request message, each filtering type scheduler in the scheduling system performs work division cooperation, and the filtering type scheduler for acquiring the scheduling request message determines a final scheduling result, so that the processing time is reduced, and the processing efficiency of the scheduling request message is improved.

In one possible implementation, the preset condition includes:

In another possible implementation manner, the method for determining the final scheduling result in the obtained scheduling results by the first filter scheduler according to a preset algorithm includes any one of the following ways:

the predefined weight algorithm may define different weights according to at least one of a CPU remaining amount, a memory remaining amount, a disk remaining amount, a network flow, a CPU type, a disk type, a network card type, and a number of times that the disk performs reading/writing per second of the server, and calculate to obtain a weight of the server.

Through the description of the above, for one scheduling request message, each filter type scheduler in the scheduling system performs work division cooperation, determines a scheduling result in each associated virtual group, and one filter type scheduler gathers all scheduling results and determines a final scheduling result. On the other hand, each filter scheduler uses the resource pre-occupying information to record the resource information required by the determined scheduling result, so that the scheduling failure problem caused by the fact that the resource occupying condition of the record server in the database cannot be updated in real time in the prior art is solved, and the capacity of the scheduler cluster for managing new resources in large-scale data is improved.

In a fourth aspect, a filtered scheduler is provided, where the filtered scheduler includes a processor, a memory, a communication interface, and a bus, where the processor, the memory, and the communication interface are connected by the bus and complete communication with each other, where the memory is used to store computer-executable instructions, and where the processor, when the filtered scheduler is running, executes the computer-executable instructions in the memory to perform the method described in the first aspect and any one of the possible implementations of the first aspect by using hardware resources in the filtered scheduler.

In a fifth aspect, a computer readable medium is provided for storing a computer program comprising instructions for performing the method of the first aspect and any one of the possible implementations of the first aspect.

In summary, for a scheduling request, each filter scheduler in the scheduler cluster performs work division cooperation, determines a scheduling result in each corresponding virtual group, and determines one or more servers in the obtained scheduling result as a final scheduling result by the filter scheduler obtaining the scheduling request, so as to avoid the problem of resource competition caused by parallel work of multiple filter schedulers in the prior art. On the other hand, each filter scheduler uses the resource pre-occupying information to record the resource information required by the determined scheduling result, so that the scheduling failure problem caused by the fact that the server resource occupying condition cannot be updated in real time in the database in the prior art is solved, the information of the scheduler cluster resource allocation is improved, and the capacity of the scheduler cluster for managing new resources in large-scale data is improved.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are needed in the embodiments of the present invention will be briefly described below, and it is obvious that the drawings described below are only some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort to a person of ordinary skill in the art.

FIG. 1 is a schematic diagram of a scheduling system according to an embodiment of the present invention;

fig. 2 is a flow chart of a method for scheduling resources according to an embodiment of the present invention;

FIG. 3 is a flowchart illustrating another method for scheduling resources according to an embodiment of the present invention;

FIG. 4 is a flowchart illustrating another method for scheduling resources according to an embodiment of the present invention;

FIG. 5 is a flowchart illustrating another method for scheduling resources according to an embodiment of the present invention;

fig. 6 is a schematic diagram of an apparatus for scheduling resources according to an embodiment of the present invention;

fig. 7 is a schematic diagram of a filter scheduler according to an embodiment of the present invention.

Detailed Description

The following description of the embodiments of the present invention will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are some, but not all embodiments of the invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the present invention without making any inventive effort, shall fall within the scope of the present invention.

Fig. 1 is a schematic diagram of a scheduling system provided by the present invention, where the scheduling system includes a management node, a task allocation node, a filtered scheduler, a server, and a database, where two or more filtered schedulers form a scheduler cluster.

The management node may create a virtual group for each filter scheduler in the scheduling system in advance, where the virtual group includes all servers that the filter scheduler can schedule, i.e. the virtual group includes one or more servers, and a server belongs to only one virtual group.

Illustratively, as shown in the scheduling system of fig. 1, there are 3 filter schedulers in the scheduling system, and the management node creates 1 virtual group for each filter scheduler in advance, including:

virtual group 1 is a virtual group associated with filter scheduler 1, and includes server 1 and server 2.

The virtual group 2 is a virtual group associated with the filter scheduler 2, and the virtual group includes a server 3 and a server 4.

The virtual group 3 is a virtual group associated with the filter scheduler 3, and the virtual group includes a server 5 and a server 6.

The correspondence of the filtered scheduler, virtual group and server may be recorded using a database as shown in fig. 1. The database is also used for recording the used resource information and the available resource information of each server, such as the used condition and the available condition of resources such as a CPU, a memory, a disk, a network card and the like of the server. When the filtered scheduler processes a scheduling request, the filtered scheduler may query its associated virtual group and available resource information for each server in the virtual group via a database.

Alternatively, the filtered scheduler may record the correspondence between the filtered scheduler, the virtual group, and the server using a specified file.

As a possible embodiment, when creating the virtual group, the management node may divide the virtual group into different virtual groups according to attributes such as Identification (ID), operating system type, hard disk type, and the like of the server.

It should be noted that the number of servers in each virtual group may be the same or different. In the following detailed description of the present invention, three filter schedulers are included in the scheduler cluster, and two servers are included in a virtual group corresponding to each filter scheduler.

Alternatively, the management node may adjust the number of servers in each virtual group according to a predefined adjustment policy when there is a failed server in the virtual group or there is an added server in the scheduling system. For example, if there is a failed server in virtual group 1, the management node may adjust the number of servers in each virtual group according to the priority order of the filter scheduler; if a filter scheduler is newly added in the scheduling system, the management node may add the server to a virtual group associated with the filter scheduler according to the processing capability of each filter scheduler.

Alternatively, the virtual groups may also be managed by the task allocation node, i.e., the task allocation node creates a virtual group for each filter scheduler and dynamically adjusts the number of servers in the virtual group.

It should be noted that, in fig. 1, the management node and the task allocation node may be disposed in the same server, or may be disposed in different servers separately, which is not limited by the present invention. In the following description of the present invention, the management node and the filter scheduler shown in fig. 1 are disposed in the same server for example.

Alternatively, the virtual group may be managed by a third party management device or apparatus outside the scheduling system that is able to learn all the filtered scheduler and server information.

In the scheduling system, the task allocation node is further configured to receive a scheduling request Message, store the scheduling request Message carrying a service requirement in a Message Queue (MQ), monitor whether a scheduling request Message to be processed exists in the Message Queue by using a monitoring manner, acquire the scheduling request Message when the scheduling request Message to be processed exists is monitored, process and determine a final scheduling result, and then send the final scheduling result to the task allocation node, where the task allocation node sends the service requirement to one or more servers corresponding to the final scheduling result for execution.

The message queues are containers for storing messages during the transmission of messages, as in the scheduling system shown in fig. 1, the server 100 is used for deploying message queue software, the message queue software can store and delete messages transferred between the task allocation node and the filter schedulers and messages transferred between all the filter schedulers by using the memory of the server 100, wherein the server 100 comprises at least one message queue, each message queue can store different types of messages according to an initial configuration, for example, two message queues exist in the storage system, the message queue 1 is used for storing messages transferred between the task allocation node and the scheduling cluster, such as scheduling request messages and final scheduling results, and the message queue 2 is used for storing messages transferred between the filter schedulers, such as scheduling task messages and scheduling results determined by each filter scheduler. Alternatively, the message queue software may also store messages in the form of message queue clusters, where each message queue cluster includes at least two message queues, where different message queue clusters are used to store different types of messages, for example, two message queue clusters exist in the storage system, where message queue cluster 1 is used to store messages transferred between the task allocation node and the scheduler cluster, for example, one message queue in message queue cluster 1 is used to store a scheduling request message, and another message queue is used to store a final scheduling result; the message queue cluster 2 is used for storing messages transferred between the filter schedulers, for example, one message queue in the message queue cluster 2 is used for storing scheduling task messages, and the other message queue is used for storing scheduling results determined by each filter scheduler.

It should be noted that, in the implementation process, the message queue may be deployed by using a single server, or may be deployed by using multiple servers at the same time, or may share server resources with other types of software. In the following description of the invention, a scenario in which a message queue is deployed by a separate server as shown in fig. 1 is described in further detail.

Alternatively, the communication manner between the task allocation node and the filter-type schedulers in the scheduling system and between all the filter-type schedulers may be implemented by using other asynchronous communication manners such as a database, for example, a Redis database.

Next, a method for scheduling resources according to an embodiment of the present invention is further described with reference to fig. 2, where, as shown in the figure, the method includes:

s201, the first filter type dispatcher obtains a dispatching request message.

The scheduling request message carries service requirements, and the first filter scheduler is any filter scheduler in a scheduling system. The first filter scheduler may acquire the scheduling request message by listening to the message queue, i.e. when there is a pending scheduling request message in the message queue, the scheduling request message is acquired from the message queue.

Optionally, the task allocation node may also send the scheduling request message directly to the first filtered scheduler.

S202, the first filter type dispatcher converts the dispatching request message into a dispatching task message and notifies each filter type dispatcher in the dispatching system in an asynchronous communication mode.

Specifically, the task scheduling message carries the service requirement and role information of the first filter scheduler, and the task scheduling message is used for indicating each filter scheduler in the scheduling system to determine a scheduling result in the associated virtual group according to the service requirement, and summarizing the scheduling result to the first filter scheduler, wherein the scheduling result comprises one or more servers meeting the service requirement.

The first filter type dispatcher informs each filter type dispatcher of the scheduled task information to be processed in a dispatching system, the scheduled task information can be stored in a message queue, and each filter type dispatcher in the dispatching system acquires the scheduled task information when the scheduled task information to be processed exists in the message queue.

Optionally, the first filtered scheduler informs each filtered scheduler that there is a scheduled task message to be processed in the scheduling system, and may also store the scheduled task message in a database used as asynchronous communication, such as a Redis database, and all filtered schedulers acquire the scheduled task message by periodically listening to the database.

Optionally, the first filtered scheduler informs each filtered scheduler of the scheduled task message to be processed in the scheduling system, and may also obtain IP addresses of all filtered schedulers in the scheduling system from the database, and send the scheduled task message to all filtered schedulers except the first filtered scheduler according to the IP addresses.

S203, the first filter type scheduler obtains a scheduling result determined by the filter type scheduler in the scheduling system according to the scheduling task information.

Specifically, after determining a scheduling result, a filter scheduler in a scheduling system determines a message queue for receiving the scheduling result by a first filter scheduler according to role information in a scheduling task message, stores the scheduling result in the message queue, monitors the message queue, and acquires the scheduling result when a newly added scheduling result exists in the message queue.

And S204, when the preset condition is met, the first filter type scheduler determines a final scheduling result in the obtained scheduling results according to a preset algorithm.

Specifically, the meeting of the preset condition includes that the number of the scheduling results obtained by the first filter scheduler meets a first threshold; or the first filtered scheduler obtains that the number of scheduling results meets a first threshold and the time of receipt meets a second threshold. The final scheduling result determined by the first filtered scheduler includes one or more servers that meet the traffic demand.

In this embodiment, each filter scheduler in the scheduler system determines a scheduling result in a set of servers capable of scheduling according to the same scheduling request, and sends the scheduling result to the filter scheduler that obtains the scheduling request message, when a preset condition is met, the filter scheduler determines one or more servers as a final scheduling result from all the obtained scheduling results, and compared with the scheduling result determined by only one filter scheduler in the prior art, the parallel processing of multiple filter schedulers solves the problems of long scheduling task processing time and low efficiency in the prior art, and improves the resource scheduling capability and resource allocation efficiency of the scheduling system.

In the following, taking a communication mode between a task allocation node and a filter scheduler as an example of a message queue, a resource scheduling method provided by an embodiment of the present invention is described in detail with reference to fig. 3, and as shown in the figure, the method includes:

s301, the first filter type dispatcher acquires a dispatching request message from a message queue.

After receiving a scheduling request message sent by an application layer, a task allocation node stores the scheduling request message in a message queue, and a first filter type scheduler acquires the scheduling request message, wherein the first filter type scheduler is any filter type scheduler which acquires the scheduling request message in a scheduling system. The service requirement carried in the scheduling request message comprises service requirement information such as information for creating the virtual machine, migration information of the virtual machine, scheduling of virtual machine resources and the like.

Table 1 is an example of a message format of a scheduling request message:

TABLE 1

Specifically, each filter type scheduler in the scheduling system identifies whether a scheduling request message to be processed exists in a message queue in a monitoring mode, when the scheduling request message to be processed exists in the message queue, the idle filter type schedulers can acquire the scheduling request from the message queue, and at the moment, the first filter type scheduler which acquires the scheduling request message is responsible for processing the scheduling request.

It should be noted that, when the scheduling request message in the message queue is taken away by any filter scheduler, in order to avoid the problem of resource waste caused by repeated processing, the message queue software will delete the scheduling request message in the message queue, and other filter schedulers cannot acquire the scheduling request message again.

S302, the first filter type dispatcher converts the dispatching request message into a dispatching task message and stores the dispatching task message in a message queue.

Specifically, the first filtered scheduler adds role information of the first filtered scheduler in the scheduling request message. The scheduling task message is used for indicating that the filter type scheduler in the scheduling cluster determines a scheduling result and sending the determined scheduling result to the first filter type scheduler. The scheduling task message also carries a service requirement, that is, the service requirement carried in the scheduling request message in step S301.

Table 2 is an example of a message format in which the first filtered scheduler converts the scheduling request message in table 1 into a scheduling task message:

TABLE 2

As can be seen from the above tables 1 and 2, in the process of converting the scheduling request message into the scheduling task message, the identification and role information of the first filtered scheduler and the IP address of the first filtered scheduler are added to the message format. The message formats included in the scheduling request message and the scheduling request task message in tables 1 and 2 are only one example, and in the specific implementation process, the message formats of other contents may also be included.

For example, in the scheduling system shown in fig. 1, if the filtered scheduler 1 obtains the scheduling request message, the filtered scheduler 1 records role information in the processing process of the scheduling request message, the filtered scheduler 1 is a Leader of the current scheduling request message, the filtered scheduler 2 and the filtered scheduler 3 are both in a Worker role, optionally, the filtered scheduler 1 may also have a Worker role, and the filtered scheduler 1, the filtered scheduler 2 and the filtered scheduler 3 are respectively responsible for determining a scheduling result in an associated virtual group thereof and sending the determined scheduling result to the filtered scheduler 1, and the filtered scheduler 1 is responsible for obtaining the scheduling result determined by the filtered scheduler according to the scheduling task message and determining one or more servers as a final scheduling result.

Further, the first filtered scheduler stores the scheduling task message in a message queue so that the filtered scheduler in the scheduling system can monitor the message queue, acquire the scheduling task message from the message queue and process the scheduling task message.

Specifically, the scheduling task message may be stored in the message queue for storing the scheduling request message in step S301, and in the process of monitoring the message queue, all filter schedulers in the scheduling system distinguish different scheduling request messages and scheduling task messages by identifying the scheduling request message identifier and the scheduling task message identifier; alternatively, a separate message queue may be used to store the scheduling request message, and all filter schedulers in the scheduling system identify whether there is a scheduling task message to be processed by listening to the specified message queue.

It should be noted that, unlike the manner that the scheduling request message is deleted after being acquired by the first filter scheduler in step S301, the scheduling task in step S302 is stored in the message queue, and the scheduling task is always present in the processing process of the scheduling request until the first filter processes the scheduling task, and then notifies the message queue software to delete the scheduling task message, so that all filter schedulers in the storage cluster can acquire the scheduling task in an asynchronous manner. After the filtered schedulers in the storage cluster acquire the scheduled task, a server which is most suitable for executing the service requirement in the scheduled task in the virtual group corresponding to each filtered scheduler can be determined.

Optionally, after converting the scheduling request message into the scheduling task message, the first filter scheduler may also store the scheduling task message in a database for asynchronous communication, where each filter scheduler identifies and acquires the scheduling task message to be processed by periodically monitoring the database.

Optionally, after the first filtered scheduler converts the scheduling request into the scheduling task message, the IP addresses of all the filtered schedulers in the scheduling system may also be obtained from the database, and the scheduling task message may be directly sent to each filtered scheduler according to the IP address.

S303, the first filter type dispatcher acquires the dispatching task information from the information queue, and stores the determined dispatching result in the information queue.

When the message queue has the task message to be processed, the filter scheduler in each idle state in the scheduling system can acquire and process the scheduling request message in a monitoring mode. When the first filter type dispatcher monitors that the to-be-processed dispatching task information exists and the first filter type dispatcher is in an idle state, the first filter type dispatcher acquires the dispatching task information and selects one or more servers meeting service requirements from the associated virtual group as dispatching results.

It should be noted that each filter scheduler can only process one scheduling task message at a time, that is, each filter scheduler can acquire and process a new scheduling task message in the message queue only after the current scheduling task message is processed.

Specifically, the method for determining the scheduling result by each filter scheduler in the scheduling system is the same, fig. 4 is a flowchart of a method for determining the scheduling result by the filter schedulers, and as shown in the drawing, the specific method includes:

s303a, the filter type scheduler determines a server meeting the service requirement in the virtual group associated with the filter type scheduler according to the service requirement.

The server meeting the service requirement is determined by the filter scheduler according to the service requirement and the remaining available resource conditions of the servers in the virtual group associated with the filter scheduler, namely, one or more servers meeting the service requirement are determined in the virtual group associated with the filter scheduler.

And S303b, the filter type dispatcher calculates and sorts the determined servers according to a predefined weight algorithm, and sequentially selects one or more servers according to the order as the dispatching result.

Specifically, each filter scheduler calculates the weight of the servers meeting the service requirements according to a predefined weight algorithm, sorts the servers according to the weight, and sequentially selects one or more servers from high to low or from low to high in sequence as determined scheduling results. The predefined weight algorithm can define different weights for at least one of the attributes of CPU resource, memory resource, disk residual quantity, number of times of reading and writing per second of disk, network bandwidth and the like of the server according to the customization requirement of specific service application, and calculate the weight of the server.

Optionally, the scheduling result further includes weight information of the server.

For example, if the servers that can be scheduled by the filter scheduler 1 include the server 1 and the server 2, the service requirements carried in the scheduling request message are virtual machines for creating 1 and 2 CPUs, 8G memory, 32G disk and 1 network card, and it is determined through screening that the server 1 and the server 2 both meet the specification requirements for creating the virtual machines, the server meeting the service requirements includes two servers; assuming that the service application has a customized requirement on the residual quantity of the disk and the memory of the server, calculating according to the available memory weight of the server as 0.6 and the available disk weight as 0.4 in a predefined weight algorithm. Specifically, if the available memory resource of the server 1 is 10G, the available disk is 100G; the available memory resource of the server 2 is 20G, the available disk is 100G, the weight of the server 1 is 46 (0.6x10+0.4x100=46), the weight of the server 2 is 50 (0.5x20+0.4x100=50), the weight ranks of the servers meeting the service requirements are sequentially from high to low, the server 2 and the server 1, and the filtered scheduler 1 determines that the server 1 is a scheduling result of the scheduling request message in a server set which can be scheduled by the server 1, and optionally, the scheduling result further comprises weight information 50 of the server 2.

Table 3 is an example of a message format of a scheduling result in the above embodiment:

TABLE 3 Table 3

Through the descriptions of steps S303a to S303b, each filter scheduler in the scheduling system selects at least one server meeting the service requirement as a scheduling result according to a first-screening and then-ordering manner among the servers that can be scheduled by the filter scheduler.

Further, after determining the scheduling result, the filter scheduler in the scheduling system sends the determined scheduling result to the designated filter scheduler according to the role information in the scheduled task, for example, to the first filter scheduler in step S301. The manner in which the filtered scheduler sends the scheduling result to the first filtered scheduling may be any one of the following manners:

mode one: the filter scheduler stores the determined scheduling result in a message queue according to the role information in the scheduling task message.

The message queue storing the scheduling result determined by the filter scheduler may be the same as the message queue storing the scheduling request message in step S301, and at this time, the first filter scheduler distinguishes among the scheduling request message, the scheduling task message, and the scheduling result by specifying a message format, where the specified message format determining the scheduling result may be "scheduling result" in table 3. And the scheduling result can also be directly sent to a message queue of the first filter scheduler for receiving the scheduling result according to the role information in the scheduling task message.

And secondly, according to the IP address of the first filter type scheduler in the scheduling task message, the determined scheduling result is sent to the first filter type scheduler.

It should be understood that, for the first filter scheduler, if the first filter scheduler has both a Leader role and a Worker role in the task, when the first filter scheduler is used as the Leader role of the task message, the first filter scheduler may be in a non-idle state, and the task message may not be the first task message to be processed in the message queue, for simplifying the implementation process, the first filter scheduler may directly store the converted task message in the message queue, and when the first filter scheduler is in an idle state, the task message is re-acquired and processed, and the scheduling result is stored in the message queue for storing the scheduling result or in the local memory. Optionally, when the first filter scheduler has a Leader role and a Worker role of one scheduling task message, the scheduling task message may also be stored in the local memory, and when the first filter scheduler is in an idle state, different scheduling task messages are distinguished through the scheduling task message identifier, and the locally stored scheduling task message is preferentially processed.

S304, other filter schedulers in the scheduling system acquire scheduling task information from the information queue, and store determined scheduling results in the information queue.

Specifically, the other filter schedulers are filter schedulers in any idle state except the first filter scheduler in the scheduler cluster, when a to-be-processed scheduling task message exists in the message queue, the other filter schedulers also acquire the scheduling task message, and at least one server meeting the service requirement is selected from the associated virtual group as a scheduling result according to the service requirement carried in the scheduling task message. The process of determining the scheduling result by the other filter schedulers is the same as the method described in step S303, and will not be described here.

S305, the first filtered scheduler obtains a scheduling result determined by each filtered scheduler in the scheduler system.

Specifically, the manner in which the first filtered scheduler obtains the scheduling result may be that the first filtered scheduler obtains the scheduling result from the message queue or that the first filtered scheduler receives the scheduling result sent by the filtered scheduler, as in step S303.

And S306, when the preset condition is met, the first filter type scheduler determines a final scheduling result in the obtained scheduling results according to a preset algorithm, wherein one or more servers meeting the service requirements in the final scheduling result.

The preset condition may be any of the following modes:

mode one: the first filtered scheduler sums the number of scheduling results to meet a first threshold.

Mode two: the first filtered scheduler receives the number of scheduling results satisfying the first threshold and the reception time satisfying the second threshold.

Optionally, the first filtered scheduler may limit the reception time in addition to limiting the number of summarized scheduling results when determining the final scheduling result, and select one final scheduling result from the summarized scheduling results when both meet the requirement.

Optionally, in the implementation process, the first filtered scheduler may further select a final scheduling result after receiving the scheduling results sent by all the filtered schedulers in the scheduling system, that is, one or more servers meeting the service requirement are selected from all the servers in the scheduling system as the final scheduling result, where the final scheduling result is the optimal one or more servers meeting the service requirement.

Further, the processing mode of determining the final scheduling result in the scheduling results by the first filter scheduler according to a preset algorithm comprises any one of the following modes:

mode one: and the first filtering scheduler sorts all servers in the obtained scheduling results according to the weight determined by the predefined weight algorithm, and sequentially selects one or more servers meeting the service requirements as the final scheduling result.

For example, table 4 is an example of the scheduling results summarized by the first filtered scheduler, and assuming that the service requirement is 1 virtual machine created, the first filtered scheduler ranks all servers from high to low as server 3, server 5, and server 1 according to the weight determined by the predefined weight algorithm, and the final scheduling result is server 3, that is, server 3 is the best server satisfying the service requirement.

TABLE 4 Table 4

Scheduling results	Server information	Weight information
			Scheduling result 1	Server 1	46
Scheduling result 2	Server 3	60
			Scheduling result 3	The server 5	55

The predefined weight algorithm may be the same as the method for determining the weight in step S303b, or redefining the weight of the server resource according to the specific application requirement, and calculating the weight of the server, where the server resource includes a CPU type, a disk type, a network card type, an available CPU resource, an available memory resource, a disk remaining amount, a network bandwidth, a number of times of reading and writing per second of the disk, a CPU remaining amount, a memory remaining amount, a disk remaining amount, and the like, and the final scheduling result is the optimal server or servers that meet the service requirement.

Alternatively, the predefined weight algorithm may also determine the weights of the server resources according to the service application without paying attention to the resource information, and calculate the weight information of each server, where the first filter scheduler may sequentially select, as the final scheduling result, one or more servers that meet the service requirement from low to high in order.

Mode two: the first filtered scheduler sequentially selects one or more servers from the received scheduling results determined by each filtered scheduler according to the receiving time sequence as a final scheduling result.

Mode three: the first filter scheduler randomly selects one or more servers from the scheduling results as final scheduling results.

Besides the first mode, in order to improve the processing efficiency and reduce the processing time of the dispatching system, the final dispatching result can be determined according to the second mode and the third mode, namely, the first filtering dispatcher determines the final dispatching result according to the receiving time sequence or in a random mode, and the final dispatching result comprises at least one server meeting the service requirement.

Through the descriptions of step S301 to step S306, when there is a scheduling request message to be processed in the scheduling system, a filter scheduler obtains the scheduling request message, converts the scheduling request message into a scheduling task message, stores the scheduling request message in a message queue, each filter scheduler in the scheduling system processes the scheduling task message, and gathers the determined scheduling result to the filter scheduler, and the filter scheduler determines, according to a preset algorithm, an optimal server or servers meeting the service requirements from the gathered scheduling results as a final scheduling result. Compared with the prior art, for the same scheduling request message, each filter type scheduler in the scheduling system performs work division cooperation and parallel processing, so that the problems of long scheduling processing time and low efficiency in the prior art are solved, and the scheduling capability and resource scheduling efficiency of the whole scheduling system are improved.

In another possible embodiment of the present invention, after determining the scheduling result in the virtual group to which each filter scheduler corresponds, the method may also solve the problem that the available resources of the server recorded in the data in the prior art cannot be updated in real time, resulting in misjudgment of the filter scheduler to cause scheduling failure, as shown in fig. 5, where the method includes:

s501, the filter scheduler records the resource pre-occupying information after determining the scheduling result according to the scheduling task information.

Specifically, the resource pre-emption information is used for recording the scheduling result determined by the filter scheduler. For example, if the service requirement carried in the task scheduling message 1 is to create a virtual machine, the specification of the virtual machine includes 2 CPUs, 10G memories, 2 20G disks, and 1 network card, and if the filter scheduler determines that the server 1 in the associated virtual group is used as a scheduling result, the filter scheduler will record the task scheduling message and the scheduling result as resource preemption information.

Table 5 is an example of resource preemption information:

alternatively, the filtered scheduler may record the resource preemption information using a specified file, or may record the resource preemption information using a database.

It should be noted that, the resource pre-allocation information recorded by the first filter scheduler is an information identifier recorded by the filter scheduler, and the physical resource is not really divided in the server corresponding to the first scheduling result.

Optionally, when the resource pre-occupation information is recorded and the scheduling result is not finally determined, if the new scheduling task information is executed again, the filter scheduler may acquire the available resource information of the server in the database for the server involved in the resource pre-occupation information in the processing process, subtract the resource pre-occupation information recorded by the filter scheduler, determine the remaining resource information of the server, and then determine whether the server can execute the service requirement in the new scheduling task information according to the remaining resource condition of the server.

S502, when the first filter type scheduler determines a final scheduling result, the first filter type scheduler informs the filter type scheduler of the final scheduling result so that the filter type scheduler can clear the recorded resource pre-occupation information.

After the first filtered scheduler determines the final scheduling result, all the filtered schedulers in the scheduling system are informed of the final scheduling result, so that the filtered schedulers in the scheduling system can clear the recorded resource pre-occupation information according to the final scheduling result. The notification mode may store the final scheduling result in the message queue, or may obtain the IP addresses of all schedulers except the first filter scheduler in the database, and send the final scheduling result according to the IP address information.

Further, the process of clearing the resource pre-emption information includes the following two cases:

case one: if the server in the final dispatching result does not belong to the virtual group associated with the filter dispatcher, the filter dispatcher directly clears the recorded resource pre-occupation information.

And a second case: if the server in the final scheduling result belongs to the virtual group associated with the filter scheduler, the filter scheduler clears the recorded resource pre-occupation information after the server in the final scheduling result finishes business processing.

It should be noted that, after the server corresponding to the final scheduling result completes the service requirement, the server will notify the database in the scheduling system to update the resource occupation condition, and notify the server that the corresponding service requirement processing of the filter scheduler is successful, and the filter scheduler clears the recorded resource occupation information accordingly.

In summary, for one scheduling request message, each filter scheduler in the scheduling system performs work division cooperation, determines a scheduling result in each associated virtual group, and one filter scheduler gathers all scheduling results and determines a final scheduling result. On the other hand, each filter scheduler uses the resource pre-occupying information to record the resource information required by the determined scheduling result, so that the scheduling failure problem caused by the fact that the resource occupying condition of the record server in the database cannot be updated in real time in the prior art is solved, and the capacity of the scheduler cluster for managing new resources in large-scale data is improved.

It should be noted that, for simplicity of description, the above method embodiments are all described as a series of combinations of actions, but it should be understood by those skilled in the art that the present invention is not limited by the order of actions described, and further, that the embodiments described in the specification belong to preferred embodiments, and that the actions are not necessarily required by the present invention.

Other reasonable combinations of steps that can be conceived by those skilled in the art from the foregoing description are also within the scope of the invention. Furthermore, those skilled in the art will be familiar with the preferred embodiments, and the description of the preferred embodiments does not necessarily require the invention.

The method and the scheduling system for scheduling resources provided according to the embodiments of the present invention are described in detail above with reference to fig. 1 to 5, and the filter scheduler provided according to the embodiments of the present invention will be described below with reference to fig. 6 to 7.

Fig. 6 is a device 600 for scheduling resources, where the device 600 includes an obtaining unit 601, a processing unit 602, and a determining unit 603;

The acquiring unit is used for acquiring a scheduling request message, wherein the scheduling request message carries service requirements;

the processing unit is configured to convert the scheduling request message into a scheduling task message, and notify each filtering scheduler in the scheduling system in an asynchronous communication manner, where the scheduling task message carries the service requirement and role information of the device, and the scheduling task message is configured to instruct the filtering scheduler to determine a scheduling result in a virtual group associated with the filtering scheduler according to the service requirement, and send the scheduling result to the device, where the scheduling result includes one or more servers that meet the service requirement;

the acquisition unit is further used for acquiring a scheduling result determined by the filter scheduler in the scheduling system according to the scheduling task message;

and the determining unit is used for determining a final scheduling result in the obtained scheduling results according to a preset algorithm when the preset condition is met, wherein the final scheduling result comprises one or more servers meeting the service requirement.

It should be appreciated that the apparatus 600 of embodiments of the present invention may be implemented by an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), a programmable logic device (Programmable Logic Device, PLD), which may be a complex program logic device (Complex Programmable Logic Device, CPLD), a Field programmable gate array (Field-Programmable Gate Array, FPGA), a general purpose array logic (Generic Array Logic, GAL), or any combination thereof. When the processing method for resource scheduling shown in fig. 2 is implemented by software, the apparatus 600 and its respective modules may also be software modules.

Through the description of the device 600, each filter scheduler in the scheduler system determines a scheduling result in a server set capable of scheduling according to the same scheduling request, and sends the scheduling result to the filter scheduler for obtaining the scheduling request message, when a preset condition is met, the filter scheduler selects a final scheduling result from all summarized scheduling results, and compared with the scheduling result determined by only one filter scheduler in the prior art, the parallel processing of multiple filter schedulers solves the problems of long scheduling task processing time and low efficiency in the prior art, and improves the resource scheduling capability and resource allocation efficiency of the scheduling system.

Optionally, the processing unit notifies each filter scheduler in the scheduling system through an asynchronous communication mode, wherein the method comprises the steps of storing the scheduling task message in the message queue, so that each filter scheduler monitors the message queue and acquires the scheduling task message.

Specifically, each filter type scheduler is informed of the existence of the scheduling task information to be processed in the scheduling system, the scheduling task information can be stored in a message queue, and each filter type scheduler in the scheduling system acquires the scheduling task information when monitoring that the scheduling task information to be processed exists in the message queue.

Optionally, the acquiring unit is further configured to acquire the scheduling task message;

the processing unit is further configured to determine, according to the service requirement, a server that meets the service requirement in a virtual group associated with the filter scheduler; and sequencing the determined servers according to a predefined weight algorithm, and sequentially selecting one or more servers according to the sequence as the scheduling result.

Optionally, the preset conditions include: the first filter scheduler obtains that the number of the scheduling results meets a first threshold; or (b)

Optionally, the determining unit determines, according to a preset algorithm, a final scheduling result from the obtained scheduling results, including any one of the following manners:

Sequencing all servers in the scheduling result according to the weight determined by the predefined weight algorithm, and sequentially selecting one or more servers as the final scheduling result; or (b)

Sequentially selecting one or more servers from the scheduling results according to the receiving time sequence as the final scheduling result; or (b)

Randomly selecting one or more servers from the scheduling results as the final scheduling results;

Optionally, the apparatus further comprises a recording unit; the recording unit is used for recording resource pre-occupation information after the scheduling result is determined, wherein the resource pre-occupation information is used for identifying resource information which is determined by the filter scheduler and needs to be occupied by the scheduling result in a corresponding server;

the processing unit is further configured to notify the filtered scheduler of the final scheduling result after the first filtered scheduler determines the final scheduling result.

The apparatus 600 according to the embodiment of the present invention may correspond to performing the method described in the embodiment of the present invention, and the above and other operations and/or functions of each unit in the apparatus 600 are respectively for implementing the corresponding flows of each method in fig. 1 to 5, which are not described herein for brevity.

Fig. 7 is a diagram of a filter scheduler 700 according to the present invention, as shown in fig. 7, the filter scheduler 700 includes a processor 701, a memory 702, a communication interface 703, and a bus 704, where the processor 701, the memory 702, and the communication interface 703 are connected by the bus 704 and complete communication with each other, the memory 702 is used for storing computer execution instructions, and when the filter scheduler 700 operates, the processor 701 executes the computer execution instructions in the memory 702 to perform the following operations by using hardware resources in the filter scheduler 700:

Acquiring a scheduling request message, wherein the scheduling request message carries service requirements;

converting the scheduling request message into a scheduling task message, and notifying each filter-type scheduler in the scheduling system in an asynchronous communication mode, wherein the scheduling task message carries the service requirement and role information of the filter-type scheduler 700, the scheduling task message is used for indicating the filter-type scheduler to determine a scheduling result in a virtual group associated with the filter-type scheduler according to the service requirement, and the scheduling result is sent to the filter-type scheduler 700, and the scheduling result comprises one or more servers meeting the service requirement;

acquiring a scheduling result determined by a filter scheduler in the scheduling system according to the scheduling task message;

and when the preset condition is met, determining a final scheduling result in the obtained scheduling results according to a preset algorithm, wherein the final scheduling result comprises one or more servers meeting the service requirement.

It should be appreciated that in embodiments of the present invention, the processor 701 may be a CPU, and the processor 701 may also be other general purpose processors, digital Signal Processors (DSPs), application Specific Integrated Circuits (ASICs), off-the-shelf programmable gate arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.

The memory 702 may include read only memory and random access memory and provides instructions and data to the processor 701. A portion of the memory 702 may also include non-volatile random access memory. For example, the memory 702 may also store information of device type.

The system bus 704 may include a power bus, a control bus, a status signal bus, and the like in addition to a data bus. But for clarity of illustration, the various buses are labeled as system bus 704 in the drawing.

Through the description of the above, for a scheduling request, each filter type scheduler in the scheduler cluster performs work division cooperation, determines scheduling results in each corresponding virtual group, and the first filter type scheduler for acquiring the scheduling request gathers all results and finally determines the optimal scheduling result, so that the problem of resource competition caused by parallel work of a plurality of filter type schedulers in the prior art is avoided. On the other hand, each filter scheduler uses the resource pre-occupying information to record the resource information required by the determined scheduling result, so that the scheduling failure problem caused by the fact that the server resource occupying condition cannot be updated in real time in the database in the prior art is solved, the information of the scheduler cluster resource allocation is improved, and the capacity of the scheduler cluster for managing new resources in large-scale data is improved.

It should be understood that the filtered scheduler 700 according to an embodiment of the present invention may correspond to the apparatus 600 for resource scheduling in an embodiment of the present invention and may correspond to the respective bodies performing the method shown in fig. 2 according to an embodiment of the present invention, and that the above and other operations and/or functions of the respective modules in the filtered scheduler 700 are respectively for implementing the respective flows of the respective methods in fig. 1 to 5, and are not repeated herein for brevity.

In summary, for a scheduling request, each filter scheduler in the scheduler cluster performs work division cooperation, determines a scheduling result in each corresponding virtual group, and the first filter scheduler that obtains the scheduling request gathers all results and finally determines an optimal scheduling result, so as to avoid the problem of resource competition caused by parallel work of multiple filter schedulers in the prior art. On the other hand, each filter scheduler uses the resource pre-occupying information to record the resource information required by the determined scheduling result, so that the scheduling failure problem caused by the fact that the server resource occupying condition cannot be updated in real time in the database in the prior art is solved, the information of the scheduler cluster resource allocation is improved, and the capacity of the scheduler cluster for managing new resources in large-scale data is improved.

Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.

It will be clear to those skilled in the art that, for convenience and brevity of description, specific working procedures of the above-described systems, apparatuses and units may refer to corresponding procedures in the foregoing method embodiments, and are not repeated herein.

In the several embodiments provided by the present application, it should be understood that the disclosed systems, devices, and methods may be implemented in other manners. For example, the apparatus embodiments described above are merely illustrative, e.g., the division of the units is merely a logical function division, and there may be additional divisions when actually implemented, e.g., multiple units or components may be combined or integrated into another system, or some features may be omitted or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, which may be in electrical, mechanical or other form.

The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in the embodiments of the present invention may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit.

The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution, in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.

The foregoing is merely illustrative of the present invention, and the present invention is not limited thereto, and any person skilled in the art will readily recognize that variations or substitutions are within the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. A method for scheduling resources, wherein the method is applied to a scheduling system, the scheduling system comprises at least two filter schedulers and at least two virtual groups, wherein each filter scheduler is associated with at least one virtual group, and each virtual group comprises a server which can be scheduled by the filter scheduler associated with the virtual group; the method comprises the following steps:

a first filter type scheduler obtains a scheduling request message, wherein the scheduling request message carries service requirements, and the first filter type scheduler is any filter type scheduler in the scheduling system;

the first filter scheduler converts the scheduling request message into a scheduling task message;

the first filter scheduler informs other filter schedulers in the scheduling system of the scheduling task message;

The first filtered scheduler obtains scheduling results determined by other filtered schedulers in the scheduling system according to the scheduling task information, and determines a final scheduling result in the scheduling results, wherein the final scheduling result comprises one or more servers meeting the service requirements.

2. The method of claim 1, wherein one server in the dispatch system belongs to one virtual group.

3. The method according to claim 1 or 2, wherein the first filtered scheduler informs other filtered schedulers in the scheduling system of the scheduled task message, comprising:

the first filtered scheduler stores the scheduling task message in a message queue, so that other filtered schedulers in the scheduling system monitor the message queue and acquire the scheduling task message.

4. The method according to claim 1 or 2, wherein said determining a final scheduling result among said scheduling results comprises:

and when the preset condition is met, the first filter type scheduler determines a final scheduling result in the scheduling results according to a preset algorithm.

5. The method of claim 4, wherein the meeting a preset condition comprises:

6. The method of claim 4, wherein the first filtered scheduler determining a final scheduling result among the scheduling results according to a preset algorithm, comprising:

and the first filtering scheduler determines the weight of the servers in the scheduling results according to a predefined weight algorithm, sorts all the servers in the scheduling results according to the weight, and selects one or more servers as the final scheduling result according to the sorting.

7. The method of claim 6, wherein the first filtered scheduler determining weights for servers in the scheduling results according to a predefined weight algorithm, comprising:

the first filter type dispatcher determines the weight of the server in the dispatching result according to at least one of the CPU residual quantity, the memory residual quantity, the disk residual quantity, the network flow, the CPU type, the disk type, the network card type and the number of times of reading and writing of the disk per second.

8. The method of any one of claims 1, 2, 5-7, wherein the method further comprises: the first filter type scheduler informs other filter type schedulers in the scheduling system of the final scheduling result so that the other filter type schedulers in the scheduling system can clear resource pre-occupation information, wherein the resource pre-occupation information is recorded by the filter type schedulers after determining the scheduling result, and the resource pre-occupation information is used for indicating resource information which needs to be occupied by a server in the scheduling result determined by the filter type schedulers.

9. An apparatus for scheduling resources, wherein the apparatus is applied to a first filtered scheduler, the first filtered scheduler belongs to a scheduling system, the scheduling system comprises at least two filtered schedulers and at least two virtual groups, each filtered scheduler is associated with at least one virtual group, each virtual group comprises a server which can be scheduled by the filtered scheduler associated with the virtual group, and the apparatus comprises an acquisition unit, a processing unit and a determination unit;

The processing unit is used for converting the scheduling request message into a scheduling task message and notifying other filtering schedulers in a scheduling system of the scheduling task message;

the obtaining unit is further configured to obtain a scheduling result determined by other filter schedulers in the scheduling system according to the scheduling task message;

the determining unit is configured to determine a final scheduling result from the scheduling results, where the final scheduling result includes one or more servers that meet the service requirement.

10. The apparatus of claim 9, wherein the processing unit configured to notify a filtered scheduler in a scheduling system of the scheduled task message comprises:

the processing unit is used for storing the scheduling task information in an information queue so that other filter schedulers in the scheduling system monitor the information queue and acquire the scheduling task information.

11. The apparatus of claim 9, wherein the determining unit configured to determine a final scheduling result from the scheduling results comprises:

and the determining unit is used for determining a final scheduling result in the scheduling results according to a preset algorithm when the preset condition is met.

12. The apparatus of claim 9, wherein the satisfaction of the preset condition comprises:

acquiring the quantity of the scheduling results to meet a first threshold; or (b)

And acquiring that the number of the scheduling results meets a first threshold and the receiving time meets a second threshold.

13. The apparatus of claim 11, wherein the determining a final scheduling result among the scheduling results according to a preset algorithm comprises:

and determining weights of servers in the scheduling results according to a predefined weight algorithm, sorting all servers in the scheduling results according to the weights, and selecting one or more servers as the final scheduling results according to the sorting.

14. The apparatus of claim 13, wherein said determining weights for servers in the scheduling results according to a predefined weight algorithm comprises:

and determining the weight of the server in the scheduling result according to at least one of the CPU residual quantity, the memory residual quantity, the disk residual quantity, the network flow, the CPU type, the disk type, the network card type and the number of times of reading and writing of the disk per second.

15. The device according to any one of claims 9, 13, 14, wherein,

The processing unit is further configured to notify other filter schedulers in the scheduling system of the final scheduling result, so that the other filter schedulers in the scheduling system clear resource pre-emption information, where the resource pre-emption information is recorded by the filter schedulers after determining the scheduling result, and the resource pre-emption information is used to indicate resource information that a server in the scheduling result determined by the filter schedulers needs to be occupied.

16. A filtered scheduler, characterized in that it comprises a processor, a memory, in which computer-executable instructions are stored, which processor executing the computer-executable instructions in the memory performs the method according to any of the preceding claims 1 to 8.