WO2018072687A1 - Procédé et appareil de programmation de ressources, et programmateur filtré - Google Patents

Procédé et appareil de programmation de ressources, et programmateur filtré Download PDF

Info

Publication number
WO2018072687A1
WO2018072687A1 PCT/CN2017/106553 CN2017106553W WO2018072687A1 WO 2018072687 A1 WO2018072687 A1 WO 2018072687A1 CN 2017106553 W CN2017106553 W CN 2017106553W WO 2018072687 A1 WO2018072687 A1 WO 2018072687A1
Authority
WO
WIPO (PCT)
Prior art keywords
scheduling
scheduler
filter
scheduling result
message
Prior art date
Application number
PCT/CN2017/106553
Other languages
English (en)
Chinese (zh)
Inventor
朱旭琪
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Publication of WO2018072687A1 publication Critical patent/WO2018072687A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/48—Program initiating; Program switching, e.g. by interrupt
    • G06F9/4806—Task transfer initiation or dispatching
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533—Hypervisors; Virtual machine monitors
    • G06F9/45558—Hypervisor-specific management and integration aspects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/48—Program initiating; Program switching, e.g. by interrupt
    • G06F9/4806—Task transfer initiation or dispatching
    • G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
    • G06F9/485—Task life-cycle, e.g. stopping, restarting, resuming execution
    • G06F9/4856—Task life-cycle, e.g. stopping, restarting, resuming execution resumption being on a different machine, e.g. task migration, virtual machine migration
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/54—Interprogram communication
    • G06F9/546—Message passing systems or structures, e.g. queues
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533—Hypervisors; Virtual machine monitors
    • G06F9/45558—Hypervisor-specific management and integration aspects
    • G06F2009/4557—Distribution of virtual machine instances; Migration and load balancing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2209/00—Indexing scheme relating to G06F9/00
    • G06F2209/54—Indexing scheme relating to G06F9/54
    • G06F2209/548—Queue

Definitions

  • the present invention relates to the field of communications, and in particular, to a resource scheduling method, apparatus, and filter scheduler in the field of virtualization management.
  • resource scheduling which is the business. Requests are allocated to appropriate resources, including computing, storage, networking, etc., to improve resource utilization.
  • the most commonly used scheduler is a filtering scheduler, that is, when receiving a scheduling request, first filtering out all the server sets satisfying the business requirements according to the business requirements carried in the scheduling request, and then All servers in the server set are sorted according to a predefined weight calculation method, and the scheduler selects the server with the highest weight to process the service demand.
  • the single filter scheduler is responsible for the scheduling of all resources in the virtualized system, which cannot meet the business requirements, resulting in serious performance bottlenecks.
  • More and more data centers adopt Multiple filter schedulers implement scheduling requests in parallel to share the scheduling tasks of the entire large-scale data center.
  • a plurality of filter schedulers are used in the scheduling system to form a scheduler cluster.
  • the filter scheduler in the scheduler cluster separately obtains different scheduling requests, and The scheduling result is determined according to different scheduling requests, but for the same scheduling request, a filtered scheduler first filters out the server set that satisfies the scheduling request in all servers in the scheduling system, and then according to the predefined rights.
  • the value calculation method calculates the weights of all the servers in the server set, and then sorts the servers according to the weights, and finally selects one or more servers as the scheduling result of the scheduling request.
  • the scheduling method is The same filter scheduler is required to filter and sort all servers, resulting in long processing time and low efficiency of the same scheduled task.
  • the available resources of each server in the prior art are recorded and saved by the database, but the server only informs the database to update the available resources of the server when the business requirements in the scheduling request are completed. That is to say, the available resource information of the server recorded in the database is non-real-time information, and the server determined by the filter scheduler according to the information may not meet the service requirement, thereby affecting the judgment of the filter scheduler and increasing the scheduling failure. Probability. Therefore, the method for parallel scheduling of multiple filter schedulers in the prior art has the problems that the scheduling request processing time is long, the efficiency is low, and the misjudgment causes the service processing to fail, which affects the resource scheduling capability and resource allocation efficiency of the scheduling system.
  • the embodiment of the invention provides a method, a device and a filter scheduler for resource scheduling, which can solve the problem that the scheduling system processes the scheduling request for a long time, the efficiency is low, and the misjudgment causes the service processing to fail, and the scheduler system is improved. Resource scheduling capabilities and resource allocation efficiency.
  • a method for resource scheduling is provided, where the method is applied to a scheduling system, where the scheduling system includes at least two filter schedulers and at least two servers, wherein each filter scheduler is associated with one virtual group a group, each virtual group includes all servers that can be scheduled by the filter scheduler associated with the virtual group, and one server belongs to only one virtual group; first, the first filter scheduler obtains a scheduling request message, The scheduling request message carries a service requirement, and the first filtering scheduler is any filtering scheduler in the scheduling system; then, the first filtering scheduler converts the scheduling request message into a scheduling task message, and asynchronously The communication mode notifies each of the filter schedulers in the scheduling system, wherein the scheduled task message carries the service requirement and the role information of the first filter scheduler, where the schedule task message is used to indicate that the filter scheduler is in accordance with the service requirement.
  • the first filtering scheduler obtains a scheduling result determined by the filtering scheduler in the scheduling system according to the scheduling task message; when the preset condition is met The first filter scheduler determines a final scheduling result in the obtained scheduling result according to a preset algorithm, where the final scheduling result includes one or more servers that meet the service requirement.
  • each virtual group includes one or more servers.
  • the scheduling task message may further include identifier information and IP address information of the first filter scheduler.
  • the first filtering scheduler that obtains the scheduling request message may record the role information of each filtering scheduler in the current scheduling task message by using the data table in the specified file or the database.
  • the role of the filter scheduler may include both Leader and Worker, wherein the filter scheduler of the Worker role is responsible for filtering the associated virtual in the set of at least one server that it can schedule.
  • the group determines one or more servers that meet the service requirements as the scheduling result;
  • the filter scheduler of the Leader role is a filter scheduler that obtains the scheduling request message, and is responsible for converting the scheduling request message into a scheduling task message, and notifying the scheduling system. All the filter schedulers have scheduling task messages to be processed, summarize the scheduling results determined by the filter scheduler of all the worker roles, and select one or more services satisfying the service requirements among all the summarized scheduling results according to the preset algorithm. For the final scheduling results.
  • the role information is temporary information, that is, it is only valid during the processing of the current scheduling request message.
  • each scheduling request message is determined by a leader role of the filtering scheduler to determine the final scheduling result, and each type of role Only one message can be processed at a time, that is, the filter role of the Leader role can only process one scheduling request message at a time, and the filter scheduler of the Worker role can only process one scheduling task message at a time.
  • the filtering scheduler releases the role information corresponding to the current scheduling request message processing.
  • each of the filter schedulers in the scheduler system determines the scheduling result in the respective set of servers that can be scheduled according to the same scheduling request, and sends the scheduling result to the filtering schedule for acquiring the scheduling request message.
  • the filter scheduler determines one or more servers as the final scheduling result among all the summarized scheduling results, and has only one filtering schedule for the same scheduling request message in the prior art.
  • the parallel processing of multiple filtering schedulers solves the problem of long processing time and low efficiency of scheduling tasks in the prior art, and improves the resource scheduling capability and resource allocation efficiency of the scheduling system.
  • the notifying, by the asynchronous communication manner, each filtering scheduler in the scheduling system includes: the first filtering scheduler storing the scheduling task message in a message queue, so that Every A filter scheduler listens to the message queue and acquires the scheduled task message.
  • the message queue software deletes the scheduling request message in the message queue to avoid waste of resources caused by repeated processing.
  • the scheduler cannot obtain the scheduling request message again.
  • the scheduling task persists during the processing of the scheduling request until the first filter processes the scheduling task, and then notifies the message queue software to delete the scheduling task message, so that all the filtering schedulers in the storage cluster can be asynchronous. The way to get the scheduled task.
  • the filtering scheduler in the storage cluster can determine the server in the virtual group corresponding to each filtering scheduler that is most suitable for performing the business requirements in the scheduling task.
  • the scheduling task message may also be stored in the database, and the manner in which the first filtering scheduler obtains the scheduling request message may also be performed by listening to the database, that is, when there is a scheduling request message to be processed in the database, the slave database Obtain the scheduling request message.
  • the notifying, by the asynchronous communication manner, each filtering scheduler in the scheduling system includes: the first filtering scheduler storing the scheduling task message in a database, so that Each of the filter schedulers listens to the database and acquires the scheduled task message.
  • the filtering scheduler determines, according to the service requirement, a scheduling result in a virtual group associated with the filtered scheduler, including: the filtering scheduler obtaining the Scheduling a task message; the filter scheduler determines, in the virtual group associated with the filter scheduler, a server that satisfies the service requirement according to the service requirement; and the filter scheduler performs a pair of predefined weight algorithm
  • the determined servers are sorted, and one or more are sequentially selected in order as the scheduling result.
  • the preset conditions include:
  • the first filter scheduler obtains that the number of the scheduling results meets a first threshold and the receiving time satisfies a second threshold.
  • each filter scheduler can only process one scheduled task message at a time.
  • the first filtering scheduler is When the final scheduling result is determined, the first threshold may be set. When the number of the summarized scheduling results meets the first threshold, one or more servers are selected as the final scheduling result in the obtained scheduling result.
  • the method for determining, by the first filtering scheduler, the final scheduling result in the scheduling result according to a preset algorithm includes any one of the following manners:
  • the first filter scheduler sorts all servers in the scheduling result according to the weight determined by the predefined weight algorithm, and sequentially selects one or more servers as the final scheduling result; or
  • the first filter scheduler sequentially selects one or more servers from the scheduling result in the order of receiving time as the final scheduling result;
  • the first filter scheduler randomly selects one or more servers from the scheduling result as the final scheduling result
  • the predefined weight algorithm may be based on at least a CPU, a remaining amount of memory, a remaining amount of the disk, a network traffic, a CPU type, a disk type, a network card type, and a number of times the disk reads and writes per second.
  • the first filtering scheduler after determining the final scheduling result, notifies the message queue software to delete the processed scheduling task message. At the same time, the first filter scheduler also sends the optimal scheduling result to the task assignment node, and the task assignment node sends the service requirement in the scheduling task to the server corresponding to the final scheduling result, and the server performs the business requirement.
  • the specific implementation process
  • each filtering scheduler in the scheduling system divides and cooperates, and respectively determines scheduling results in respective associated virtual groups, and a filtering scheduler summarizes the filtering scheduler according to the scheduling task message. Determining the scheduling result, and determining one or more servers as the final scheduling result in the foregoing scheduling result, compared with the prior art, the same scheduling request message is processed by multiple filtering schedulers in parallel, reducing scheduling Processing time improves the processing efficiency of the scheduling system.
  • each filter scheduler uses resource preemption information to record the resource information required for the determined scheduling result, thereby solving the scheduling failure problem caused by the fact that the record server resource occupancy cannot be updated in real time in the database in the prior art, thereby Improved the ability of the scheduler cluster to manage new resources in large data.
  • an apparatus for resource scheduling comprising various modules for performing the resource scheduling method of the first aspect or any of the possible implementations of the first aspect.
  • a method for resource scheduling is provided, where the method is applied to a scheduling system, where the scheduling system includes at least two filter schedulers and at least two servers, wherein each filter scheduler is associated with one virtual group a group, each virtual group includes all servers that can be scheduled by the filter scheduler associated with the virtual group, and one server belongs to only one virtual group; first, the first filter scheduler obtains a scheduling request message, The scheduling request message carries a service requirement, and the first filtering scheduler is any filtering scheduler in the scheduling system; then, the first filtering scheduler converts the scheduling request message into a scheduling task message, and the scheduling is performed.
  • the task message is sent to the server in the scheduling system except the first filter scheduler, where the scheduling task message carries the service requirement and the role information of the first filter scheduler, and the schedule task message is used to indicate the scheduling system.
  • Other filter schedulers other than the first filter scheduler are associated with other filter schedulers based on business requirements. Determining a scheduling result in the quasi-group, and transmitting the scheduling result to the first filtering scheduler; next, the first filtering scheduler acquires a scheduling result determined by the filtering scheduler in the scheduling system according to the scheduling task message; When the preset condition is met, the first filter scheduler determines the final scheduling result in the acquired scheduling result according to the preset algorithm, and the final scheduling result includes one or more servers that meet the service requirement.
  • the first filtering type and other filtering schedulers in the scheduling system directly communicate.
  • each filtering scheduler in the scheduling system divides and cooperates, and the filtering of the scheduling request message is obtained.
  • the scheduler determines the final scheduling result, reduces the processing time, and improves the processing efficiency of the scheduling request message.
  • the preset conditions include:
  • the first filter scheduler obtains that the number of the scheduling results meets a first threshold and the receiving time satisfies a second threshold.
  • the method for determining, by the first filtering scheduler, the final scheduling result in the acquired scheduling result according to a preset algorithm includes any one of the following manners:
  • the weight of the first filter scheduler determined according to the predefined weight algorithm is used in the scheduling result There are servers to sort, and one or more servers are selected in turn for the final scheduling result; or
  • the first filter scheduler sequentially selects one or more servers from the scheduling result in the order of receiving time as the final scheduling result;
  • the first filter scheduler randomly selects one or more servers from the scheduling result as the final scheduling result
  • the predefined weight algorithm may be based on at least one of a CPU remaining amount of the server, a remaining memory amount, a remaining amount of the disk, a network traffic, a CPU type, a disk type, a network card type, and a number of times the disk reads and writes per second. Define different weights and calculate the weight of the obtained server.
  • each filtering scheduler in the scheduling system divides and cooperates, determines scheduling results in respective associated virtual groups, and summarizes all scheduling results by a filtering scheduler and determines final scheduling.
  • multiple filtering schedulers are processed in parallel, which reduces the scheduling processing time and improves the processing efficiency of the scheduling system.
  • each filter scheduler uses resource preemption information to record the resource information required for the determined scheduling result, thereby solving the scheduling failure problem caused by the fact that the record server resource occupancy cannot be updated in real time in the database in the prior art, thereby Improved the ability of the scheduler cluster to manage new resources in large data.
  • a filter scheduler in a fourth aspect, includes a processor, a memory, a communication interface, and a bus, and the processor, the memory, and the communication interface are connected by a bus and complete communication with each other.
  • the memory is configured to store computer execution instructions, and when the filter scheduler is running, the processor executes a computer execution instruction in the memory to perform the first aspect and the first with hardware resources in the filter scheduler Any of the aspects described in the possible implementations.
  • a fifth aspect a computer readable medium for storing a computer program, the computer program comprising instructions for performing the method of the first aspect and the possible implementation of any of the first aspects.
  • each of the filter schedulers in the scheduler cluster cooperates and determines the scheduling result in each corresponding virtual group, and is obtained by the filter scheduler that obtains the scheduling request.
  • the scheduling result determines that one or more servers are final scheduling results, which avoids resource competition problems caused by multiple filtering schedulers working in parallel in the prior art.
  • each filter scheduler uses the resource camping information to record the resource information required for the determined scheduling result, and solves the problem of scheduling failure caused by the inability of the record server resource occupancy in the database in the prior art to be updated in real time.
  • the dispatcher cluster resource allocation message improves the ability of the scheduler cluster to manage new resources in large data.
  • FIG. 1 is a schematic diagram of a scheduling system according to an embodiment of the present invention
  • FIG. 2 is a schematic flowchart of a method for resource scheduling according to an embodiment of the present invention
  • FIG. 3 is a schematic flowchart diagram of another method for resource scheduling according to an embodiment of the present disclosure.
  • FIG. 4 is a schematic flowchart diagram of another method for resource scheduling according to an embodiment of the present disclosure.
  • FIG. 5 is a schematic flowchart diagram of another method for resource scheduling according to an embodiment of the present disclosure.
  • FIG. 6 is a schematic diagram of an apparatus for resource scheduling according to an embodiment of the present invention.
  • FIG. 7 is a schematic diagram of a filter scheduler according to an embodiment of the present invention.
  • the scheduling system includes a management node, a task allocation node, a filter scheduler, a server, and a database, wherein two or more filters are used.
  • the scheduler forms a scheduler cluster.
  • the management node may pre-create a virtual group for each filter scheduler in the scheduling system, and the virtual group includes all servers that the filter scheduler can schedule, that is, the virtual group includes one or more servers, and A server belongs to only one virtual group.
  • the management node creates one virtual group for each filter scheduler in advance, including:
  • the virtual group 1 is a virtual group associated with the filter scheduler 1, and the virtual group includes the server 1 and the server 2.
  • the virtual group 2 is a virtual group associated with the filter scheduler 2, which includes the server 3 and the server 4.
  • the virtual group 3 is a virtual group associated with the filter scheduler 3, which includes the server 5 and the server 6.
  • the correspondence between the filter scheduler, the virtual group, and the server can utilize database records as shown in FIG.
  • the database is also used to record the used resource information and available resource information of each server, for example, the usage and availability of the server's CPU, memory, disk, network card and other resources.
  • the filtered scheduler can query its associated virtual group and the available resource information of each server in the virtual group through the database.
  • the filter scheduler may also record the correspondence between the filter scheduler, the virtual group, and the server by using a specified file.
  • the management node may divide the server into different virtual groups according to attributes such as an identification (ID), an operating system type, and a hard disk type.
  • each virtual group can be the same or different.
  • three types of filter schedulers are included in the scheduler cluster, and two virtual servers in the virtual group corresponding to each filter scheduler are exemplified in detail.
  • the management node may adjust the number of servers in each virtual group according to a predefined adjustment policy. For example, if there is a faulty server in the virtual group 1, the management node can adjust each virtual according to the priority order of the filter scheduler. The number of servers in the quasi-group; if a new filter scheduler is added to the scheduling system, the management node can add the server to a virtual group associated with the filter scheduler according to the processing capability of each filter scheduler. in.
  • the virtual group may also be managed by the task assignment node, that is, the task assignment node creates a virtual group for each filter scheduler and dynamically adjusts the number of servers in the virtual group.
  • management node and the task allocation node in FIG. 1 may be deployed in the same server, or may be deployed in different servers separately, which is not limited by the present invention.
  • management node and the filter scheduler shown in FIG. 1 are deployed in the same server as an example for detailed description.
  • the virtual group may also be managed by a third party management device or device outside the scheduling system that is capable of learning all of the filtered scheduler and server information.
  • the task allocation node is further configured to receive the scheduling request message, and store the scheduling request message carrying the service requirement in a message queue (Message Queue, MQ), where each filtering scheduler in the scheduling system is monitored.
  • MQ message Queue
  • the scheduling request message is obtained, and the final scheduling result is determined, and then the final scheduling result is sent to the message.
  • the task assignment node is sent by the task assignment node to the one or more servers corresponding to the final scheduling result.
  • the message queue is a container for saving messages during the transmission of the message.
  • the server 100 is used to deploy the message queue software, and the message queue software can use the memory of the server 100 to save and delete the task allocation node and the filtering method.
  • Message queue 1 is used to save messages transmitted between the task assignment node and the scheduling cluster, such as scheduling request messages and final scheduling results.
  • Message queue 2 is used to save messages transmitted between the filter schedulers, such as The task message is scheduled and the scheduling result determined by each filter scheduler.
  • the message queue software may also save the message in the form of a message queue cluster.
  • Each message queue cluster includes at least two message queues, and different message queue clusters are used to store different types of messages. For example, there are two in the storage system.
  • message queue cluster message queue cluster 1 is used to save messages transmitted between the task assignment node and the scheduler cluster.
  • one message queue in message queue cluster 1 is used to save the scheduling request message, and another message queue is used to save the final scheduling result.
  • Message queue cluster 2 is used to save messages transmitted between the filter schedulers.
  • one message queue in message queue cluster 2 is used to save scheduling task messages, and another message queue is used to save the schedule determined by each filter scheduler. result.
  • the message queue can be deployed by a separate server, or multiple servers can be deployed simultaneously, and server resources can be shared with other types of software.
  • a scenario in which a message queue is deployed by a separate server as shown in FIG. 1 is taken as an example for further elaboration.
  • the communication mode between the task allocation node and the filtering scheduler in the scheduling system and all the filtering schedulers may also be implemented by using other asynchronous communication methods such as a database, such as a Red is database.
  • a resource scheduling method provided by an embodiment of the present invention is further introduced in conjunction with FIG. 2, as shown in the figure, the method includes:
  • the first filter scheduler acquires a scheduling request message.
  • the scheduling request message carries the service requirement, and the first filtering scheduler is any filtering type in the scheduling system. scheduler.
  • the manner in which the first filter scheduler obtains the scheduling request message may be a manner of listening to the message queue, that is, when there is a scheduling request message to be processed in the message queue, the scheduling request message is obtained from the message team.
  • the scheduling task message may also be stored in the database, and the manner in which the first filtering scheduler obtains the scheduling request message may also be performed by listening to the database, that is, when there is a scheduling request message to be processed in the database, the slave database Obtain the scheduling request message.
  • the task assignment node may also send the scheduling request message directly to the first filtered scheduler.
  • the first filtering scheduler converts the scheduling request message into a scheduling task message, and notifies each filtering scheduler in the scheduling system by asynchronous communication.
  • the scheduling task message carries the service requirement and the role information of the first filtering scheduler, and the task scheduling message is used to indicate that each filtering scheduler in the scheduling system determines the scheduling result in the associated virtual group according to the service requirement.
  • the scheduling result is summarized into a first filtered scheduler, and the scheduling result includes one or more servers that meet the business requirements.
  • the first filtering scheduler notifies each filtering scheduler that there is a scheduling task message to be processed in the scheduling system, and the scheduling task message may be stored in the message queue, and each filtering scheduler in the scheduling system is listening to the message. When there is a scheduled task message to be processed in the queue, the scheduling task message is obtained.
  • the first filtering scheduler notifies each filtering scheduler that there is a scheduling task message to be processed in the scheduling system, and may also store the scheduling task message in a database used for asynchronous communication, such as a Red is database. All filter schedulers obtain the scheduled task message by periodically listening to the database.
  • the first filtering scheduler notifies each filtering scheduler that there is a scheduling task message to be processed in the scheduling system, and may also obtain an IP address of all filtering schedulers in the scheduling system from the database, according to the IP address.
  • the scheduled task message is sent to all filter schedulers except the first filter scheduler.
  • the first filter scheduler acquires a scheduling result determined by the filter scheduler in the scheduling system according to the scheduling task message.
  • the filtering scheduler in the scheduling system determines, according to the role information in the scheduling task message, a message queue used by the first filtering scheduler to receive the scheduling result, and stores the scheduling result in the message queue.
  • the first filter scheduler listens to the message queue, and obtains the scheduling result when there is a new scheduling result in the message queue.
  • the first filter scheduler determines a final scheduling result in the acquired scheduling result according to a preset algorithm.
  • satisfying the preset condition includes that the quantity of the first filter scheduler acquiring the scheduling result satisfies the first threshold; or the number of the first filter scheduler acquiring the scheduling result satisfies the first threshold and the receiving time meets the second threshold.
  • the final scheduling result determined by the first filter scheduler includes one or more servers that meet the business requirements.
  • each of the filter schedulers in the scheduler system determines the scheduling result in the respective set of servers that can be scheduled according to the same scheduling request, and sends the scheduling result to the filtering schedule for acquiring the scheduling request message.
  • the filter scheduler determines one or more servers as the final scheduling result among all the acquired scheduling results, and only one filter scheduler determines the scheduling result in the prior art.
  • the parallel processing of multiple filter schedulers solves the problem of long processing time and low efficiency of scheduling tasks in the prior art, and improves resource scheduling capability and resource allocation efficiency of the scheduling system.
  • the following is a description of the resource scheduling method provided by the embodiment of the present invention.
  • the method includes:
  • the first filtered scheduler obtains a scheduling request message in a message queue.
  • the task allocation node After receiving the scheduling request message sent by the application layer, the task allocation node stores the scheduling request message in the message queue, and the first filtering scheduler obtains the scheduling request message, where the first filtering scheduler is a scheduling Any filter scheduler that obtains the scheduling request message in the system.
  • the service requirement carried in the scheduling request message, the service requirement includes information about creating a virtual machine, virtual machine migration information, and virtual machine resource scheduling.
  • Table 1 shows an example of a message format for a scheduling request message:
  • each filtering scheduler in the scheduling system identifies, by means of the interception manner, whether there is a scheduling request message to be processed in the message queue.
  • the idle scheduling filter scheduler will The scheduling request is obtained from the message queue.
  • the first filtering scheduler that obtains the scheduling request message is responsible for processing the scheduling request.
  • the message queue software deletes the scheduling request message in the message queue to avoid waste of resources caused by repeated processing. The scheduler cannot obtain the scheduling request message again.
  • the first filter scheduler converts the scheduling request message into a scheduling task message, and stores the scheduling task message in a message queue.
  • the first filtered scheduler adds the role information of the first filtered scheduler in the scheduling request message.
  • the scheduling task message is used to indicate that the filtering scheduler in the scheduling cluster determines the scheduling result, and sends the determined scheduling result to the first filtering scheduler.
  • the scheduling task message also carries the service requirement, that is, the service requirement carried in the scheduling request message in step S301.
  • the scheduling task message may further include identifier information and IP address information of the first filter scheduler.
  • Table 2 is an example of a message format in which the first filter scheduler converts the scheduling request message in Table 1 into a scheduling task message:
  • the scheduling request message is converted into a scheduling task message in the message grid.
  • the identifier and role information of the first filter scheduler and the IP address of the first filter scheduler are added.
  • the message format included in the scheduling request message and the scheduling request task message in Table 1 and Table 2 is only an example. In a specific implementation process, a message format of other content may also be included.
  • the first filtering scheduler that obtains the scheduling request message may record the role information of each filtering scheduler in the current scheduling task message by using the data table in the specified file or the database.
  • the role of the filter scheduler may include both Leader and Worker, wherein the filter scheduler of the Worker role is responsible for filtering the associated virtual in the set of at least one server that it can schedule.
  • the group determines one or more servers that meet the service requirements as the scheduling result;
  • the filter scheduler of the Leader role is a filter scheduler that obtains the scheduling request message, and is responsible for converting the scheduling request message into a scheduling task message, and notifying the scheduling system. All the filter schedulers have scheduling task messages to be processed, summarize the scheduling results determined by the filter scheduler of all the worker roles, and select one or more services satisfying the service requirements among all the summarized scheduling results according to the preset algorithm. For the final scheduling results.
  • the filtering scheduler 1 acquires the scheduling request message, the filtering scheduler 1 records the role information in the processing of the current scheduling request message, and the filtering scheduler 1 is this.
  • the leader of the scheduling request message, the filter scheduler 2 and the filter scheduler 3 are all Worker roles.
  • the filter scheduler 1 may also have a Worker role, then the filter scheduler 1, the filter scheduler 2, and The filter scheduler 3 is responsible for determining the scheduling result in its associated virtual group, and transmitting the determined scheduling result to the filter scheduler 1, and the filter scheduler 1 is responsible for obtaining the filter scheduler according to the scheduled task message. Determine the scheduling results and determine one or more servers as the final scheduling results.
  • the role information is temporary information, that is, it is only valid during the processing of the current scheduling request message.
  • each scheduling request message is determined by a leader role of the filtering scheduler to determine the final scheduling result, and each type of role Only one message can be processed at a time, that is, the filter role of the Leader role can only process one scheduling request message at a time, and the filter scheduler of the Worker role can only process one scheduling task message at a time.
  • the filtering scheduler releases the role information corresponding to the current scheduling request message processing.
  • the first filtering scheduler stores the scheduling task message in the message queue, so that the filtering scheduler in the scheduling system can listen to the message queue, obtain the scheduling task message from the message queue, and process the message.
  • the scheduling task message may be saved in the message queue for storing the scheduling request message in step S301, and all the filtering schedulers in the scheduling system identify the scheduling request message and the scheduling task message during the process of listening to the message queue.
  • the identifier distinguishes between different scheduling request messages and scheduling task messages; optionally, the scheduling request message may also be saved by using a separate message queue, and all filtering schedulers in the scheduling system identify whether there are scheduled scheduling task messages by listening to the specified message queue. .
  • the scheduling task in a manner that the scheduling request message is deleted after being acquired by the first filtering scheduler in step S301, the scheduling task is stored in the message queue in step S302, and the scheduling task is in the processing process of the current scheduling request. It persists until the first filter processes the scheduled task, and then notifies the message queue software to delete the scheduled task message, so that all the filtered schedulers in the storage cluster can acquire the scheduled task in an asynchronous manner. After the filtering scheduler in the storage cluster obtains the scheduling task, it can determine that each filtering scheduler is correct. The server in the virtual group that is best suited to perform the business needs in the scheduled task.
  • the first filtering scheduler may also save the scheduling task message in a database for asynchronous communication, and each filtering scheduler periodically listens to the database manner. Identify and obtain the scheduled task message to be processed.
  • the first filtering scheduler may also obtain the IP addresses of all the filtering schedulers in the scheduling system from the database, and directly report to each filtering according to the IP address.
  • the dispatcher sends the scheduled task message.
  • the first filter scheduler obtains the scheduling task message in a message queue, and saves the determined scheduling result in a message queue.
  • the filtering scheduler of each idle state in the scheduling system acquires the scheduling request message and processes it by means of interception.
  • the first filter scheduler detects that there is a scheduled task message to be processed, and the first filter scheduler is in an idle state, the first filter scheduler obtains the schedule task message and is associated with the virtual group. Select one or more servers in the group that meet the business requirements as the scheduling result.
  • each filter scheduler can only process one scheduled task message at a time, that is, each filter scheduler will acquire and process a new schedule in the message queue only after the current scheduled task message is processed. Task message.
  • FIG. 4 is a flowchart of a method for determining a scheduling result by the filtering scheduler. As shown in the figure, the specific method includes:
  • the filter scheduler determines, according to the service requirement, a server that meets the service requirement in the virtual group associated with the filter scheduler.
  • the server that meets the service requirement is determined by the filter scheduler according to the service requirement and the remaining available resources of the server in the virtual group associated with the filter scheduler, that is, the virtual group associated with the filter scheduler. Identify one or more servers that meet the business needs.
  • the filter scheduler calculates the sorted servers according to the predefined weight algorithm calculation, and sequentially selects one or more servers as the scheduling result in sequence.
  • each filter scheduler calculates a weight of a server that satisfies a business requirement according to a predefined weight algorithm, and sorts the server for the weight, and sequentially selects one or more of a sequence from high to low or low to high.
  • the servers serve as the determined scheduling results.
  • the predefined weight algorithm may define at least one of a server resource, a memory resource, a disk remaining amount, a disk read/write times per second, and a network bandwidth according to a customized requirement of a specific service application. The weights are calculated and the weight of the server is calculated.
  • the scheduling result further includes weight information of the server.
  • the service requirement carried in the scheduling request message is to create a virtual machine with 2 CPUs, 8 GB memory, 32 G disks, and 1 network card.
  • the server that satisfies the business requirement includes two servers; if the business application has customization requirements for the remaining amount of the disk and memory of the server, the predefined right In the value algorithm, the available memory weight of the server is 0.6, and the available disk weight is 0.4.
  • the available memory resource of the server 1 is 10G
  • the available disk is 100G
  • the available memory resource of the server 2 is 20G
  • the available disk is 100G
  • the weights of the servers satisfying the service requirement are the server 2 and the server 1 in descending order
  • the filter scheduler 1 determines that the server 1 is The scheduling result of the scheduling request message in the server set that it can schedule.
  • the scheduling result further includes the weight information 50 of the server 2.
  • Table 3 is an example of a message format of a scheduling result in the above embodiment:
  • each filter scheduler in the scheduling system selects at least one server that satisfies the service requirement as the scheduling result in the manner in which the server can be scheduled according to the first screening and reordering.
  • the filtering scheduler sends the determined scheduling result to the specified filtering scheduler according to the role information in the scheduling task, for example, to the first filtering scheduler in step S301.
  • the manner in which the filtering scheduler sends the scheduling result to the first filtering schedule may be any one of the following modes:
  • the filter scheduler stores the determined scheduling result in the message queue according to the role information in the scheduling task message.
  • the message queue for storing the scheduling result determined by the filter scheduler may be the same as the message queue for saving the scheduling request message in step S301.
  • the first filter scheduler distinguishes the scheduling request message and the scheduling task by specifying the message format.
  • the message and the scheduling result, wherein the specified message format for determining the scheduling result may be the "scheduling result" in Table 3.
  • the scheduling result may also be directly sent to the message queue of the first filtering scheduler for receiving the scheduling result according to the role information in the scheduling task message.
  • Manner 2 Send the determined scheduling result to the first filtering scheduler according to the IP address of the first filtering scheduler in the scheduling task message.
  • the first filter scheduler if there is a Leader role and a Worker role in the current task, when the first filter scheduler is used as the Leader role of the scheduled task message, the first filter type The scheduler may be in a non-idle state, and the scheduled task message may not be the first scheduled task message to be processed in the message queue. To simplify the implementation process, the first filter will save the converted schedule task message directly. In the message queue, when the first filter scheduler is in an idle state, the scheduled task message is re-acquired for processing, and the scheduling result is stored in a message queue for saving the scheduling result or stored in the local storage.
  • the scheduled task message may also be stored in the local storage, when the first filter scheduler is idle.
  • the scheduling task message identifier distinguishes different scheduling task messages, and the locally saved scheduling task message is preferentially processed.
  • the other filter scheduler in the scheduling system acquires a scheduling task message in the message queue, and stores the determined scheduling result in the message queue.
  • the other filter scheduler is a filter scheduler of any idle state except the first filter scheduler in the scheduler cluster, and other filter schedules exist when there are scheduled task tasks to be processed in the message queue.
  • the device also obtains the scheduling task message, and selects at least one server that meets the service requirement in the associated virtual group as the scheduling result according to the service requirement carried in the scheduling task message.
  • the process of determining the scheduling result by the other filtering scheduler is the same as that described in step S303, and details are not described herein again.
  • the first filter scheduler obtains a scheduling result determined by each filter scheduler in the scheduler system.
  • the manner in which the first filtering scheduler obtains the scheduling result is the manner in which the filtering scheduler sends the scheduling result in step S303, and the first filtering scheduler may obtain the scheduling result from the message queue, or may receive the filtering scheduling.
  • the scheduling result sent by the device is the manner in which the filtering scheduler sends the scheduling result in step S303, and the first filtering scheduler may obtain the scheduling result from the message queue, or may receive the filtering scheduling. The scheduling result sent by the device.
  • the first filter scheduler determines, in the acquired scheduling result, a final scheduling result according to the preset algorithm, where one or more servers satisfying the service requirement are included in the final scheduling result.
  • the preset condition may be any one of the following methods:
  • Manner 1 The first filter scheduler summarizes the number of scheduling results to meet the first threshold.
  • Manner 2 The first filter scheduler receives the number of scheduling results that meet the first threshold and the receiving time meets the second threshold.
  • each filter scheduler can only process one scheduled task message at a time.
  • the first filtering scheduler is When the final scheduling result is determined, the first threshold may be set. When the number of the summarized scheduling results meets the first threshold, one or more servers are selected as the final scheduling result in the obtained scheduling result.
  • the first filtering scheduler may limit the receiving time in addition to the number of the scheduled scheduling results, when both meet the requirements, that is, in the summarized scheduling result. Choose a final scheduling result.
  • the first filter scheduler may further select a final scheduling result after receiving the scheduling result sent by all the filtering schedulers in the scheduling system, that is, selecting one of all servers in the scheduling system or A plurality of servers satisfying the business requirements are used as the final scheduling result, and the final scheduling result is one or more servers that satisfy the optimal business requirements.
  • processing manner that the first filtering scheduler determines the final scheduling result in the scheduling result according to the preset algorithm includes any one of the following manners:
  • the first filter scheduler sorts all the servers in the obtained scheduling result according to the weight determined by the predefined weight algorithm, and sequentially selects one or more servers that meet the service requirements as the final scheduling result.
  • Table 4 is an example of the scheduling result summarized by the first filter scheduler. It is assumed that the service requirement is to create one virtual machine, and the first filter scheduler determines the weight according to the predefined weight algorithm to all servers. Enter The row sorting from high to low is server 3, server 5, and server 1. The final scheduling result is server 3, that is, server 3 is the best server that satisfies the business requirements.
  • Scheduling result server information Weight information Scheduling result 1 Server 1 46 Scheduling result 2 Server 3 60 Scheduling result 3 Server 5 55
  • the predefined weight algorithm may be the same as the method for determining the weight in step S303b, or the weight of the server resource may be redefined according to the specific application requirement, and the server weight is calculated.
  • the server resource includes a CPU type, a disk type, a network card type, and Available CPU resources, available memory resources, disk remaining amount, network bandwidth, disk read/write times per second, CPU remaining amount, memory remaining amount, disk remaining amount, etc., the final scheduling result is the optimal one to meet the business requirements. Or multiple servers.
  • the predefined weight algorithm may also determine the weight of the server resource according to the service application not paying attention to the resource information, and calculate the weight information of each server.
  • the first filter scheduler may sequentially select the sequence from low.
  • One or more servers that meet high business needs are the final scheduling results.
  • the first filter scheduler selects one or more servers as the final scheduling result in the received scheduling result determined by each filter scheduler in sequence.
  • Manner 3 The first filter scheduler randomly selects one or more servers from the scheduling result as a final scheduling result.
  • the second scheduling method may be determined according to the second filtering mode, that is, the first filtering scheduler determines the final scheduling result according to the receiving time sequence or a random manner, and the final scheduling result is obtained. It includes at least one server that meets business needs.
  • the first filtering scheduler after determining the final scheduling result, notifies the message queue software to delete the processed scheduling task message. At the same time, the first filter scheduler also sends the optimal scheduling result to the task assignment node, and the task assignment node sends the service requirement in the scheduling task to the server corresponding to the final scheduling result, and the server performs the business requirement.
  • the specific implementation process
  • step S301 to step S306 when there is a scheduling request message to be processed in the scheduling system, the scheduling request message is obtained by a filtering scheduler, the scheduling request message is converted into a scheduling task message, and the scheduling is performed.
  • the request message is stored in the message queue, and each filter scheduler in the scheduler system processes the scheduled task message, and summarizes the determined scheduling result to the filter scheduler, and the filter scheduler follows the preset algorithm.
  • the optimal one or more servers that meet the business requirements are determined in the aggregated scheduling result as the final scheduling result.
  • each filtering scheduler in the scheduling system divides and cooperates and performs parallel processing, which solves the problem that the scheduling processing time is long and the efficiency is low in the prior art, and the whole scheduling system is improved. Scheduling capability and resource scheduling efficiency.
  • each filter scheduler may also record the records recorded in the prior art by recording resource camp-on information after determining the scheduling result in the corresponding virtual group.
  • the available resources of the server cannot be updated in real time, which causes the filter scheduler to misjudge the problem of scheduling failure.
  • the specific method is as shown in FIG. 5, and the method includes:
  • the filter scheduler records resource camping information after determining the scheduling result according to the scheduling task message.
  • the resource camping information is used to record the scheduling result determined by the filter scheduler.
  • the specifications of the virtual machine include 2 CPUs, 10 Gbytes of memory, 2 20 G disks, and 1 network card, assuming that the filter scheduler determines the virtual group associated with it. If the server 1 in the group is used as the scheduling result, the filtering scheduler records the above scheduling task message and the scheduling result as resource camping information.
  • Table 5 shows an example of resource preemption information
  • the filter scheduler may record resource preemption information by using a specified file, or may use a database to record resource camp-on information.
  • the resource pre-emption information recorded by the first filter scheduler is an information identifier recorded by the filter scheduler, and the physical resources are not actually divided in the server corresponding to the first scheduling result.
  • the filtering scheduler processes the filtering method for the server involved in the resource camping information during the processing.
  • the scheduler may first obtain the available resource information of the server in the database, subtract the resource preemption information recorded by the filter scheduler, determine the remaining resource information of the server, and determine the server according to the remaining resources of the server. Whether it is possible to execute the business requirements in the new scheduled task message.
  • the first filter scheduler determines the final scheduling result
  • the first filter scheduler notifies the filter scheduler of the final scheduling result, so that the filter scheduler clears the recorded resource camp-on information.
  • the final scheduling result is notified to all the filter schedulers in the scheduling system, so that the filtered scheduler in the scheduling system can clear the recorded resource camp-on information according to the final scheduling result.
  • the notification mode may store the final scheduling result in the message queue, or obtain the IP addresses of all the schedulers except the first filter scheduler in the database, and send the final scheduling result according to the IP address information.
  • process of clearing resource camp-on information includes the following two situations:
  • Case 2 If the server in the final scheduling result belongs to the virtual group associated with the filter scheduler, the filter scheduler clears the recorded resource camp-on information after the server in the final scheduling result completes the service processing. .
  • the server notifies the database in the scheduling system to update its resource occupancy, and notifies the server that the corresponding filter scheduler service requirement is successfully processed.
  • the scheduler accordingly clears the recorded resource preemption information.
  • each filtering scheduler in the scheduling system divides and cooperates, determines scheduling results in respective associated virtual groups, and summarizes all scheduling results by a filtering scheduler. The final scheduling result is determined.
  • the same scheduling request message is processed by multiple filtering schedulers in parallel, which reduces the scheduling processing time and improves the processing efficiency of the scheduling system.
  • each filter scheduler uses resource preemption information to record the resource information required for the determined scheduling result, thereby solving the scheduling failure problem caused by the fact that the record server resource occupancy cannot be updated in real time in the database in the prior art, thereby Improved the ability of the scheduler cluster to manage new resources in large data.
  • a method and a scheduling system for resource scheduling according to an embodiment of the present invention are described in detail above with reference to FIG. 1 to FIG. 5 .
  • a filter scheduler according to an embodiment of the present invention will be described below with reference to FIG. 6 to FIG. 7 . .
  • Figure 6 is a device 600 for resource scheduling according to an embodiment of the present invention.
  • the device 600 includes an obtaining unit 601, a processing unit 602, and a determining unit 603.
  • the acquiring unit is configured to acquire a scheduling request message, where the scheduling request message carries a service requirement;
  • the processing unit is configured to convert the scheduling request message into a scheduling task message, and notify each filtering scheduler in the scheduling system by using an asynchronous communication manner, where the scheduling task message carries the service a demand and a role information of the device, the scheduling task message is used to indicate that the filter scheduler determines a scheduling result in a virtual group associated with the filter scheduler according to the service requirement, and the scheduling result is Sent to the device, the scheduling result includes one or more servers that meet the service requirements;
  • the obtaining unit is further configured to acquire a scheduling result that is determined by the filtering scheduler in the scheduling system according to the scheduling task message;
  • the determining unit is configured to determine, in the preset scheduling result, a final scheduling result according to a preset algorithm, where the final scheduling result includes one or more servers that meet the service requirement. .
  • the apparatus 600 of the embodiment of the present invention may be implemented by an Application Specific Integrated Circuit (ASIC) or a Programmable Logic Device (PLD), and the PLD may be a complex program logic device ( Complex Programmable Logic Device (CPLD), Field-Programmable Gate Array (FPGA), Generic Array Logic (GAL), or any combination thereof.
  • ASIC Application Specific Integrated Circuit
  • PLD Programmable Logic Device
  • CPLD Complex Programmable Logic Device
  • FPGA Field-Programmable Gate Array
  • GAL Generic Array Logic
  • each of the filter schedulers in the scheduler system determines the scheduling result in the respective set of servers that can be scheduled according to the same scheduling request, and sends the scheduling result to the filtering method of acquiring the scheduling request message.
  • the scheduler selects a final scheduling result among all the scheduled scheduling results when the preset condition is met, and the plurality of filtering is compared with the prior art that only one filtering scheduler determines the scheduling result.
  • Parallel processing of the scheduler solves the problem of long processing time and low efficiency of scheduling tasks in the prior art. The problem is to improve the resource scheduling capability and resource allocation efficiency of the scheduling system.
  • the processing unit notifying, by asynchronous communication, each of the filtering schedulers in the scheduling system includes: storing the scheduling task message in the message queue, so that each of the filtering schedulers Listening to the message queue and obtaining the scheduling task message.
  • each filtering scheduler is notified that there is a scheduling task message to be processed, and the scheduling task message may be stored in a message queue, where each filtering scheduler exists in the listening message queue.
  • the scheduled task message is obtained when the task message is scheduled to be processed.
  • the message queue software deletes the scheduling request message in the message queue to avoid waste of resources caused by repeated processing.
  • the scheduler cannot obtain the scheduling request message again.
  • the scheduling task persists during the processing of the scheduling request until the first filter processes the scheduling task, and then notifies the message queue software to delete the scheduling task message, so that all the filtering schedulers in the storage cluster can be asynchronous. The way to get the scheduled task.
  • the filtering scheduler in the storage cluster can determine the server in the virtual group corresponding to each filtering scheduler that is most suitable for performing the business requirements in the scheduling task.
  • the first filtering scheduler notifies each filtering scheduler that there is a scheduling task message to be processed in the scheduling system, and may also store the scheduling task message in a database used for asynchronous communication, such as a Red is database. All filter schedulers obtain the scheduled task message by periodically listening to the database.
  • the first filtering scheduler notifies each filtering scheduler that there is a scheduling task message to be processed in the scheduling system, and may also obtain an IP address of all filtering schedulers in the scheduling system from the database, according to the IP address.
  • the scheduled task message is sent to all filter schedulers except the first filter scheduler.
  • the acquiring unit is further configured to obtain the scheduling task message
  • the processing unit is further configured to: determine, according to the service requirement, a server that meets the service requirement in a virtual group associated with the filter scheduler; sort the determined server according to a predefined weight algorithm, and One or more servers are sequentially selected in order as the scheduling result.
  • the preset condition includes: the first filter scheduler acquires the number of the scheduling results to meet a first threshold; or
  • the first filter scheduler obtains that the number of the scheduling results meets a first threshold and the receiving time satisfies a second threshold.
  • the determining unit determines, according to the preset algorithm, the final scheduling result in the acquired scheduling result, including any one of the following manners:
  • the predefined weight algorithm may be based on at least one of a CPU remaining amount of the server, a remaining memory amount, a remaining amount of the disk, a network traffic, a CPU type, a disk type, a network card type, and a number of times the disk reads and writes per second. Define different weights and calculate the weight of each server.
  • the device further includes a recording unit, and the recording unit is configured to: after determining the scheduling result, Recording resource pre-occupation information, where the resource pre-occupation information is used to identify resource information that needs to be occupied by the filtering result determined by the filtering scheduler in a corresponding server;
  • the processing unit is further configured to notify the filtered scheduler of the final scheduling result after the first filtered scheduler determines the final scheduling result.
  • each filtering scheduler in the scheduling system divides and cooperates, determines scheduling results in respective associated virtual groups, and summarizes all scheduling results by a filtering scheduler and determines final scheduling results.
  • multiple filtering schedulers are processed in parallel, which reduces the scheduling processing time and improves the processing efficiency of the scheduling system.
  • each filter scheduler uses resource preemption information to record the resource information required for the determined scheduling result, thereby solving the scheduling failure problem caused by the fact that the record server resource occupancy cannot be updated in real time in the database in the prior art, thereby Improved the ability of the scheduler cluster to manage new resources in large data.
  • Apparatus 600 in accordance with an embodiment of the present invention may correspond to performing the methods described in the embodiments of the present invention, and the above and other operations and/or functions of the various units in apparatus 600 are respectively implemented to implement the various methods of FIGS. 1 through 5. The corresponding process, for the sake of brevity, will not be described here.
  • FIG. 7 is a filter scheduler 700 provided by the present invention.
  • the filter scheduler 700 includes a processor 701, a memory 702, a communication interface 703, and a bus 704.
  • the processor 701 The memory 702 and the communication interface 703 are connected and communicate with each other via a bus 704 for storing computer execution instructions.
  • the processor 701 executes the memory 702.
  • the computer executes instructions to utilize the hardware resources in the filtered scheduler 700 to perform the following operations:
  • the scheduling task message carries the service requirement and the filtered scheduling
  • the role information of the device 700 the scheduling task message is used to instruct the filter scheduler to determine a scheduling result in a virtual group associated with the filter scheduler according to the service requirement, and send the scheduling result to the
  • the scheduling result includes one or more servers that meet the service requirements;
  • the final scheduling result is determined in the acquired scheduling result according to a preset algorithm, where the final scheduling result includes one or more servers that meet the service requirement.
  • the processor 701 may be a CPU, and the processor 701 may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), and off-the-shelf programmable Gate array (FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware components, etc.
  • DSPs digital signal processors
  • ASICs application specific integrated circuits
  • FPGA off-the-shelf programmable Gate array
  • the general purpose processor may be a microprocessor or the processor or any conventional processor or the like.
  • the memory 702 can include read only memory and random access memory and provides instructions and data to the processor 701. A portion of the memory 702 can also include a non-volatile random access memory. For example, the memory 702 can also store information of the device type.
  • the system bus 704 can include a power bus, a control bus, a status signal bus, and the like in addition to the data bus. However, for clarity of description, various buses are labeled as system bus 704 in the figure.
  • each filter scheduler uses the resource camping information to record the resource information required for the determined scheduling result, and solves the problem of scheduling failure caused by the inability of the record server resource occupancy in the database in the prior art to be updated in real time.
  • the dispatcher cluster resource allocation message improves the ability of the scheduler cluster to manage new resources in large data.
  • the filter scheduler 700 may correspond to the resource scheduling apparatus 600 in the embodiment of the present invention, and may correspond to performing corresponding ones in the method shown in FIG. 2 according to an embodiment of the present invention.
  • the above-mentioned and other operations and/or functions of the respective modules in the filter scheduler 700 are respectively implemented in order to implement the respective processes of the respective methods in FIG. 1 to FIG. 5, and are not described herein again for brevity.
  • each of the filter schedulers in the scheduler cluster cooperates and determines the scheduling result in each corresponding virtual group, and is summarized by the first filtered scheduler that obtains the scheduling request. All the results and finally determine the optimal scheduling result, avoiding the problem of resource competition caused by the parallel operation of multiple filter schedulers in the prior art.
  • each filter scheduler uses the resource camping information to record the resource information required for the determined scheduling result, and solves the problem of scheduling failure caused by the inability of the record server resource occupancy in the database in the prior art to be updated in real time.
  • the dispatcher cluster resource allocation message improves the ability of the scheduler cluster to manage new resources in large data.
  • the disclosed systems, devices, and methods may be implemented in other manners.
  • the device embodiments described above are merely illustrative.
  • the division of the unit is only a logical function division.
  • there may be another division manner for example, multiple units or components may be combined or Can be integrated into another system, or some features can be ignored or not executed.
  • the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interface, device or unit, and may be in an electrical, mechanical or other form.
  • the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of the embodiment.
  • each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
  • the functions may be stored in a computer readable storage medium if implemented in the form of a software functional unit and sold or used as a standalone product. Based on such understanding, the part of the technical solution of the present invention that contributes in essence or to the prior art or the part of the technical solution can be embodied in the form of a software product.
  • the computer software product is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method of various embodiments of the present invention.
  • the foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, and the like. .

Landscapes

  • Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computer And Data Communications (AREA)

Abstract

La présente invention concerne un procédé et un appareil de programmation de ressources, et un programmateur filtré. Le procédé comprend les étapes suivantes : un premier programmateur filtré acquiert un message de demande de programmation contenant des exigences de service, convertit le message de demande de programmation en un message de tâche de programmation, et informe chaque programmateur filtré dans un système de programmation au moyen d'une communication asynchrone, le message de tâche de programmation étant utilisé pour ordonner au programmateur filtré de déterminer, d'après les exigences de service, un résultat de programmation dans un groupe virtuel associé au programmateur filtré, et pour envoyer le résultat de programmation au premier programmateur filtré; acquérir un résultat de programmation déterminé, d'après le message de tâche de programmation, par le programmateur filtré dans le système de programmation; et lorsqu'une condition prédéfinie est satisfaite, déterminer, selon un algorithme prédéfini, un résultat de programmation final dans le résultat de programmation acquis, le résultat de programmation final comprenant un ou plusieurs serveurs satisfaisant les exigences de service. Le procédé améliore la capacité de traitement d'une programmation et l'efficacité de programmation globale du système.
PCT/CN2017/106553 2016-10-19 2017-10-17 Procédé et appareil de programmation de ressources, et programmateur filtré WO2018072687A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201610914532.5A CN107968802B (zh) 2016-10-19 2016-10-19 一种资源调度的方法、装置和过滤式调度器
CN201610914532.5 2016-10-19

Publications (1)

Publication Number Publication Date
WO2018072687A1 true WO2018072687A1 (fr) 2018-04-26

Family

ID=61997130

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2017/106553 WO2018072687A1 (fr) 2016-10-19 2017-10-17 Procédé et appareil de programmation de ressources, et programmateur filtré

Country Status (2)

Country Link
CN (2) CN111258737B (fr)
WO (1) WO2018072687A1 (fr)

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109343941A (zh) * 2018-08-14 2019-02-15 阿里巴巴集团控股有限公司 任务处理方法、装置、电子设备及计算机可读存储介质
CN111182047A (zh) * 2019-12-24 2020-05-19 中国银联股份有限公司 用于在跨网络的大数据平台之间转移文件的方法和系统
CN111831420A (zh) * 2020-07-20 2020-10-27 北京百度网讯科技有限公司 用于任务调度的方法、装置、电子设备及计算机可读存储介质
CN111949386A (zh) * 2020-07-09 2020-11-17 北京齐尔布莱特科技有限公司 一种任务调度方法、系统、计算装置及可读存储介质
CN113032126A (zh) * 2021-04-07 2021-06-25 北京理工大学 一种高并发云工作流调度引擎跨集群通信系统及方法
CN113760513A (zh) * 2021-09-16 2021-12-07 康键信息技术（深圳）有限公司 一种分布式任务调度方法、装置、设备和介质
CN114461356A (zh) * 2021-12-22 2022-05-10 天翼云科技有限公司 调度器进程数量的控制方法、IaaS云平台调度系统

Families Citing this family (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110971920B (zh) * 2018-09-30 2021-11-26 武汉斗鱼网络科技有限公司 一种消息的降级方法及相关装置
CN110109976B (zh) * 2019-05-15 2021-09-10 成都四方伟业软件股份有限公司 数据处理方法、装置、系统及存储介质
CN110267060B (zh) * 2019-07-24 2021-03-26 北京视界云天科技有限公司 视频文件存储注入方法、装置、计算机设备和存储介质
CN110912967A (zh) * 2019-10-31 2020-03-24 北京浪潮数据技术有限公司 一种服务节点调度方法、装置、设备及存储介质
CN111198749A (zh) * 2020-01-06 2020-05-26 紫光云技术有限公司 一种提高创建虚拟机物理节点调度效率的方法
CN111240822B (zh) * 2020-01-15 2023-11-17 华为技术有限公司 任务调度方法、装置、系统及存储介质
CN113382424B (zh) * 2020-03-10 2023-07-18 中国移动通信集团辽宁有限公司 一种lte基站硬件资源调度处理方法及装置
CN113112139A (zh) * 2021-04-07 2021-07-13 上海联蔚盘云科技有限公司 云平台账单的处理方法和设备
CN114429313B (zh) * 2022-04-01 2022-07-05 苏州万店掌网络科技有限公司 一种权重排期方法、装置、设备及存储介质

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102662636A (zh) * 2012-03-27 2012-09-12 北京华如科技股份有限公司 一种基于两层虚拟机的并行作业调度的方法
US9141433B2 (en) * 2009-12-18 2015-09-22 International Business Machines Corporation Automated cloud workload management in a map-reduce environment
CN105159762A (zh) * 2015-08-03 2015-12-16 冷明 基于贪心策略的启发式云计算任务调度方法
CN105893158A (zh) * 2016-06-08 2016-08-24 北京工业大学 私有云条件下的大数据混合调度模型

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8205208B2 (en) * 2007-07-24 2012-06-19 Internaitonal Business Machines Corporation Scheduling grid jobs using dynamic grid scheduling policy
CN101957780B (zh) * 2010-08-17 2013-03-20 中国电子科技集团公司第二十八研究所 一种基于资源状态信息的网格任务调度处理器及方法
US8984519B2 (en) * 2010-11-17 2015-03-17 Nec Laboratories America, Inc. Scheduler and resource manager for coprocessor-based heterogeneous clusters
US9229774B1 (en) * 2012-07-13 2016-01-05 Google Inc. Systems and methods for performing scheduling for a cluster
CN104503832B (zh) * 2014-12-22 2017-07-28 西安电子科技大学 一种公平和效率均衡的虚拟机调度系统及方法

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9141433B2 (en) * 2009-12-18 2015-09-22 International Business Machines Corporation Automated cloud workload management in a map-reduce environment
CN102662636A (zh) * 2012-03-27 2012-09-12 北京华如科技股份有限公司 一种基于两层虚拟机的并行作业调度的方法
CN105159762A (zh) * 2015-08-03 2015-12-16 冷明 基于贪心策略的启发式云计算任务调度方法
CN105893158A (zh) * 2016-06-08 2016-08-24 北京工业大学 私有云条件下的大数据混合调度模型

Cited By (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109343941A (zh) * 2018-08-14 2019-02-15 阿里巴巴集团控股有限公司 任务处理方法、装置、电子设备及计算机可读存储介质
CN109343941B (zh) * 2018-08-14 2023-02-21 创新先进技术有限公司 任务处理方法、装置、电子设备及计算机可读存储介质
CN111182047A (zh) * 2019-12-24 2020-05-19 中国银联股份有限公司 用于在跨网络的大数据平台之间转移文件的方法和系统
CN111182047B (zh) * 2019-12-24 2023-04-07 中国银联股份有限公司 用于在跨网络的大数据平台之间转移文件的方法和系统
CN111949386A (zh) * 2020-07-09 2020-11-17 北京齐尔布莱特科技有限公司 一种任务调度方法、系统、计算装置及可读存储介质
CN111831420A (zh) * 2020-07-20 2020-10-27 北京百度网讯科技有限公司 用于任务调度的方法、装置、电子设备及计算机可读存储介质
CN111831420B (zh) * 2020-07-20 2023-08-08 北京百度网讯科技有限公司 用于任务调度的方法、相关装置及计算机程序产品
CN113032126A (zh) * 2021-04-07 2021-06-25 北京理工大学 一种高并发云工作流调度引擎跨集群通信系统及方法
CN113032126B (zh) * 2021-04-07 2022-09-20 北京理工大学 一种高并发云工作流调度引擎跨集群通信系统及方法
CN113760513A (zh) * 2021-09-16 2021-12-07 康键信息技术（深圳）有限公司 一种分布式任务调度方法、装置、设备和介质
CN114461356A (zh) * 2021-12-22 2022-05-10 天翼云科技有限公司 调度器进程数量的控制方法、IaaS云平台调度系统

Also Published As

Publication number Publication date
CN107968802B (zh) 2020-02-14
CN111258737B (zh) 2023-12-08
CN111258737A (zh) 2020-06-09
CN107968802A (zh) 2018-04-27

Similar Documents

Publication Publication Date Title
WO2018072687A1 (fr) Procédé et appareil de programmation de ressources, et programmateur filtré
CN107087019B (zh) 一种基于端云协同计算架构的任务调度方法及装置
CN107241281B (zh) 一种数据处理方法及其装置
EP3072260B1 (fr) Procédés, systèmes, et supports lisibles par ordinateur pour un concentrateur d'informations de virtualisation de fonctions de réseau
CN111522636B (zh) 应用容器的调整方法、调整系统、计算机可读介质及终端设备
US10200295B1 (en) Client selection in a distributed strict queue
US9577961B2 (en) Input/output management in a distributed strict queue
WO2019001092A1 (fr) Moteur d'équilibrage de charge, client, système informatique distribué, et procédé d'équilibrage de charge
CN108776934A (zh) 分布式数据计算方法、装置、计算机设备及可读存储介质
CN109697122A (zh) 任务处理方法、设备及计算机存储介质
US9591101B2 (en) Message batching in a distributed strict queue
WO2012037760A1 (fr) Procédé, serveur et système d'amélioration d'efficacité de traitement d'alarme
US20150381514A1 (en) Multi-tiered processing using a distributed strict queue
US9575820B2 (en) Client control in a distributed strict queue
CN103414657A (zh) 一种跨数据中心的资源调度方法、超级调度中心和系统
US9577878B2 (en) Geographic awareness in a distributed strict queue
US10986036B1 (en) Method and apparatus for orchestrating resources in multi-access edge computing (MEC) network
CN113794755A (zh) 基于微服务架构的共享服务推送方法及系统
US9990240B2 (en) Event handling in a cloud data center
CN113342463B (zh) 计算机程序模块的容量调整方法、装置、设备和介质
US12047253B2 (en) System and method to provide priority based quality of service for telemetry data
US20240291825A1 (en) Data management operations for data from edge locations
WO2024087663A1 (fr) Procédé et appareil de planification de tâche, et puce
CN115766878A (zh) 面向算力枢纽的数据传输方法、系统、介质及电子设备
CN116301567A (zh) 一种数据处理系统、方法及设备

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17862850

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 17862850

Country of ref document: EP

Kind code of ref document: A1