CN110764890A - Computing resource scheduling method, system and related equipment - Google Patents
Computing resource scheduling method, system and related equipment Download PDFInfo
- Publication number
- CN110764890A CN110764890A CN201911001129.3A CN201911001129A CN110764890A CN 110764890 A CN110764890 A CN 110764890A CN 201911001129 A CN201911001129 A CN 201911001129A CN 110764890 A CN110764890 A CN 110764890A
- Authority
- CN
- China
- Prior art keywords
- computing
- task
- tasks
- computing tasks
- task sequence
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/48—Program initiating; Program switching, e.g. by interrupt
- G06F9/4806—Task transfer initiation or dispatching
- G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
- G06F9/4881—Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
- G06F9/5038—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering the execution order of a plurality of tasks, e.g. taking priority or time dependency constraints into consideration
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Debugging And Monitoring (AREA)
Abstract
The embodiment of the invention provides a method, a system and related equipment for scheduling computing resources, which are used for improving the fairness of computing resource scheduling and improving user experience. The method provided by the embodiment of the invention comprises the following steps: receiving computing tasks submitted by at least two clients; respectively carrying out calculation task classification processing on the calculation tasks of each client, wherein the calculation task classification processing comprises the following steps: selecting a preset number N of computing tasks from M computing tasks initiated by the same client to be added into a first task sequence, and adding the rest (M-N) computing tasks into a second task sequence, wherein N is less than or equal to M; and calling the computing resources in the resource pool to preferentially execute the computing tasks in the first task sequence, and after the computing tasks in the first task sequence are completed, executing the computing tasks in the second task sequence.
Description
Technical Field
The present invention relates to the field of data processing technologies, and in particular, to a method, a system, and a related device for scheduling computing resources.
Background
In some specific application fields, computers are often required to process computing tasks in batches. The task processing strategy in the existing scheme is to distribute tasks to execute according to the order of submitting tasks and by adopting a first-in first-out principle.
In the existing scheme, because each calculation task can be executed within a certain time, if a user submits a large number of tasks at one time in a peak period, the user can independently occupy the resources of the whole system for a long time, so that the tasks submitted by the following users cannot process the resources of the tasks later, namely the system resources are unevenly distributed, the direct feeling given to the user is that the system efficiency is low, the tasks submitted by the user cannot be executed later, and the user experience is not good.
Disclosure of Invention
The embodiment of the invention provides a method, a system and related equipment for scheduling computing resources, which are used for improving the fairness of computing resource scheduling and improving user experience.
A first aspect of an embodiment of the present invention provides a method for scheduling computing resources, where the method may include:
receiving computing tasks submitted by at least two clients;
respectively carrying out calculation task classification processing on the calculation tasks of each client, wherein the calculation task classification processing comprises the following steps: selecting a preset number N of computing tasks from M computing tasks initiated by the same client to be added into a first task sequence, and adding the rest (M-N) computing tasks into a second task sequence, wherein N is less than or equal to M;
and calling the computing resources in the resource pool to preferentially execute the computing tasks in the first task sequence, and after the computing tasks in the first task sequence are completed, executing the computing tasks in the second task sequence.
Optionally, as a possible implementation manner, the method for scheduling computing resources in the embodiment of the present invention further includes:
dividing computing resources in the resource pool into a plurality of minimum execution units;
periodically counting the number D of unexecuted computing tasks and the number X of minimum execution units in the resource pool;
if D is more than or equal to [ X (R +1) + R ] in continuous A statistical cycles, adding Z minimum execution units in the resource pool, wherein Z is [ (D-X)/R-X ], and R is a positive integer;
and if Min is less than or equal to D and less than X in the continuous A statistical periods, reducing S minimum execution units in the resource pool, wherein S is (X-D) and S is less than or equal to (X-Min), and Min is a positive integer.
Optionally, as a possible implementation manner, in the embodiment of the present invention, the executing the computing task in the second task sequence includes:
and randomly extracting a computing task from the second task sequence for execution.
Optionally, as a possible implementation manner, the method for scheduling computing resources in the embodiment of the present invention further includes:
configuring a maximum execution time for each type of computing task;
and if the execution of the target computing task by the first execution unit exceeds the corresponding maximum execution time, calling a second execution unit to replace the first execution unit to execute the target computing task again.
Optionally, as a possible implementation manner, the method for scheduling computing resources in the embodiment of the present invention further includes:
when the execution of the computing task fails, judging whether the computing data corresponding to the computing task is correct or not, if not, not re-executing the computing task, otherwise, re-executing.
A second aspect of an embodiment of the present invention provides a computing resource scheduling system, which may include:
the receiving unit is used for receiving the computing tasks submitted by at least two clients;
the classification unit is used for performing calculation task classification processing on the calculation tasks of each client respectively, and the calculation task classification processing comprises the following steps: selecting a preset number N of computing tasks from M computing tasks initiated by the same client to be added into a first task sequence, and adding the rest (M-N) computing tasks into a second task sequence, wherein N is less than or equal to M;
and the scheduling unit is used for calling the computing resources in the resource pool to preferentially execute the computing tasks in the first task sequence, and executing the computing tasks in the second task sequence after the computing tasks in the first task sequence are completed.
Optionally, as a possible implementation manner, the computing resource scheduling system in the embodiment of the present invention further includes:
a dividing unit, configured to divide the computing resources in the resource pool into a plurality of minimum execution units;
the counting unit is used for periodically counting the number D of the unexecuted computing tasks and the number X of the minimum execution units in the resource pool;
an adding unit, wherein if D is more than or equal to [ X (R +1) + R ] in continuous A statistical cycles, Z minimum execution units are added in the resource pool, wherein Z is [ (D-X)/R-X ], and R is a positive integer;
and a reduction unit, if Min is less than or equal to D and less than X in A continuous statistical periods, reducing S minimum execution units in the resource pool, wherein S is (X-D) and S is less than or equal to (X-Min), and Min is a positive integer.
Optionally, as a possible implementation manner, in the computing resource scheduling system in the embodiment of the present invention, the scheduling unit includes:
and the scheduling module is used for calling the computing resources in the resource pool to preferentially execute the computing tasks in the first task sequence, executing the computing tasks in the second task sequence after the computing tasks in the first task sequence are completed, and randomly extracting the computing tasks from the second task sequence to execute.
A third aspect of an embodiment of the present invention provides a computer apparatus, where the computer apparatus includes a processor, and the processor is configured to implement the steps of the computing resource scheduling method according to any one of the first aspect and the first aspect when executing a computer program stored in a memory.
A fourth aspect of embodiments of the present invention provides a computer-readable storage medium having a computer program stored thereon, characterized in that: the computer program when executed by a processor performs the steps of the method for scheduling computing resources according to any one of the first aspect and the first aspect.
According to the technical scheme, the embodiment of the invention has the following advantages:
in the embodiment of the present invention, after receiving the computing tasks submitted by a plurality of clients, the computing resource scheduling system may perform computing task classification processing on the computing task of each client, where the computing task classification processing includes: selecting a preset number N of computing tasks from M computing tasks initiated by the same client to be added into a first task sequence, and adding the rest (M-N) computing tasks into a second task sequence, wherein N is less than or equal to M; and calling the computing resources in the resource pool to preferentially execute the computing tasks in the first task sequence, and after the computing tasks in the first task sequence are completed, executing the computing tasks in the second task sequence. Regardless of the time sequence of tasks submitted by the users, a part of the computing tasks in the computing task set submitted by each user through the client enter the first sequence to be preferentially executed, so that the condition that the same user independently occupies the resources of the whole system in a peak period is avoided, the fairness of computing resource scheduling is improved, and the user experience is improved.
Drawings
FIG. 1 is a diagram illustrating an embodiment of a computing resource scheduling method according to an embodiment of the present invention;
FIG. 2 is a diagram of a computing resource scheduling system according to an embodiment of the present invention;
fig. 3 is a schematic diagram of an embodiment of a computer device according to an embodiment of the present invention.
Detailed Description
The embodiment of the invention provides a method, a system and related equipment for scheduling computing resources, which are used for improving the fairness of computing resource scheduling and improving user experience.
In order to make the technical solutions of the present invention better understood, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The terms "first," "second," "third," "fourth," and the like in the description and in the claims, as well as in the drawings, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It will be appreciated that the data so used may be interchanged under appropriate circumstances such that the embodiments described herein may be practiced otherwise than as specifically illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
For easy understanding, the following describes a specific process in an embodiment of the present invention, with reference to fig. 1
101. Receiving computing tasks submitted by at least two clients;
the user can install a client on the terminal or directly use the browser to access the WEB client, and the client has the main functions of providing an operation interaction interface for the user, and is used for the functions of inputting calculation data by the user, initiating a calculation task, displaying the execution result of the calculation task and the like.
The user can initiate a computing task through the client, the computing resource scheduling system can receive the computing tasks submitted by at least two clients, and the same client can submit a plurality of users or a plurality of computing tasks of one user.
102. Respectively carrying out calculation task classification processing on the calculation tasks of each client, wherein the calculation task classification processing comprises the following steps: selecting a preset number N of computing tasks from M computing tasks initiated by the same client to be added into a first task sequence, and adding the rest (M-N) computing tasks into a second task sequence, wherein N is less than or equal to M;
specifically, in order to avoid uneven resource allocation caused by too many users submitting computing tasks in a peak period, the computing resource scheduling system may perform computing task classification processing on the computing tasks of each user, select a preset number N of computing tasks from M computing tasks initiated by the same user to add to the first task sequence, and add the remaining (M-N) computing tasks to the second task sequence, thereby dividing all the computing tasks into two sequences with different priorities. The preset number N of computing tasks are selected from M computing tasks initiated by the same user, and may be randomly selected, or the computing task submitted first may be selected according to a time sequence, and the specific details are not limited herein. Optionally, after the computation tasks in the first task sequence are completed, the execution sequence of the computation tasks in the second task sequence may be executed according to a time sequence, or may be randomly selected to be executed, and preferably, the execution sequence of the computation tasks in the second task sequence is randomly executed, so as to further improve fairness of computing resource allocation.
103. And calling the computing resources in the resource pool to preferentially execute the computing tasks in the first task sequence, and after the computing tasks in the first task sequence are completed, executing the computing tasks in the second task sequence.
In the embodiment of the present invention, after receiving the computing tasks submitted by a plurality of clients, the computing resource scheduling system may perform computing task classification processing on the computing task of each client, where the computing task classification processing includes: selecting a preset number N of computing tasks from M computing tasks initiated by the same client to be added into a first task sequence, and adding the rest (M-N) computing tasks into a second task sequence, wherein N is less than or equal to M; and calling the computing resources in the resource pool to preferentially execute the computing tasks in the first task sequence, and after the computing tasks in the first task sequence are completed, executing the computing tasks in the second task sequence. Regardless of the time sequence of tasks submitted by the users, a part of the computing tasks in the computing task set submitted by each user through the client enter the first sequence to be preferentially executed, so that the condition that the same user independently occupies the resources of the whole system in a peak period is avoided, the fairness of computing resource scheduling is improved, and the user experience is improved.
On the basis of the embodiment shown in fig. 1, in order to reduce the delay of the peak time of the number of computing tasks and achieve the balance between hardware resources and user requirements, in the embodiment of the present invention, the number of minimum execution units may be dynamically adjusted according to a preset algorithm to reduce the delay of the peak time of the number of computing tasks. Optionally, as a possible implementation manner, the method for scheduling computing resources in the embodiment of the present invention may further include:
periodically counting the number D of unexecuted computing tasks and the current number X of the minimum execution units in the resource pool; if D is more than or equal to X (R +1) + R ] in continuous A statistical cycles, adding Z minimum execution units in the cloud service resource pool, wherein Z is [ (D-X)/R-X ], Z is a positive integer not less than 1, namely [ (D-X)/R-X ] is more than or equal to 1, and R is a positive integer; if Min is less than or equal to D and less than X in continuous A statistical periods, S minimum execution units are reduced in the cloud service resource pool, wherein A is a preset positive integer, S is (X-D), S is less than or equal to (X-Min), and Min is the minimum execution unit number set by a user. Optionally, the maximum number Max of minimum execution units may also be set such that (Z + X) < Max.
Specifically, when the number D of the computing tasks is far greater than the number of the minimum execution units for a long time, it is necessary to increase the number of the minimum execution units, but in order to avoid a situation that the performance price is low because the increased minimum execution units are idle after executing fewer computing tasks, the increased minimum execution units need to averagely execute R computing tasks, and specific R may be reasonably set according to the requirements of users, which is not limited herein. The minimum execution unit may be a software process automation RPA script or a crawler service program, and is not limited herein. The computing resource scheduling system may periodically count the number D of unexecuted computing tasks and the number X of existing minimum execution units in the resource pool, and when D ≧ [ X (R +1) + R ] in a consecutive a counting periods, Z minimum execution units are added in the cloud service resource pool, where Z ═ [ (D-X)/R-X ], and a is a positive integer, and may be set according to a user's requirement, for example, a ═ 2.
On the basis of any of the above embodiments, in order to avoid the delay in the execution of the computation task caused by the failure of a single minimum execution unit, a timeout processing mechanism may be provided in the embodiments of the present invention. Optionally, as a possible implementation manner, the method for scheduling computing resources in the embodiment of the present invention may further include:
configuring a maximum execution time for each type of computing task; and if the execution of the target computing task by the first minimum execution unit exceeds the corresponding maximum execution time, calling the second minimum execution unit to replace the first minimum execution unit to execute the target computing task again. When the target computing task execution time-out is identified, other minimum execution units can be automatically called to re-execute the target computing task, so that long time delay is avoided.
On the basis of any of the above embodiments, in order to avoid the delay in the execution of the computation task caused by the failure of a single minimum execution unit, an error retry processing mechanism may be provided in the embodiment of the present invention. Optionally, as a possible implementation manner, the method for scheduling computing resources in the embodiment of the present invention may further include:
and when the execution of the calculation task fails, judging whether the calculation data (data input by a user) corresponding to the target calculation task is correct or not, if not, not re-executing the target calculation task, otherwise, re-executing the target calculation task. In practical application, two types of situations exist in the failure of executing the calculation task, one type is that tax return related information data input by a user is wrong, and the situation needs the user to input correct data again to execute the target calculation task again; another type of situation is a system failure, requiring no user intervention, and requiring only a timed re-execution of the target computing task until successful execution.
Referring to fig. 2, an embodiment of the present invention further provides a computing resource scheduling system, which includes:
a receiving unit 201, configured to receive computing tasks submitted by at least two clients;
a classifying unit 202, configured to perform computation task classification processing on the computation task of each client, where the computation task classification processing includes: selecting a preset number N of computing tasks from M computing tasks initiated by the same client to be added into a first task sequence, and adding the rest (M-N) computing tasks into a second task sequence, wherein N is less than or equal to M;
the scheduling unit 203 is configured to call a computing resource in the resource pool to preferentially execute a computing task in the first task sequence, and after the computing task in the first task sequence is completed, execute a computing task in the second task sequence.
Optionally, as a possible implementation manner, the computing resource scheduling system in the embodiment of the present invention further includes:
the dividing unit is used for dividing the computing resources in the resource pool into a plurality of minimum execution units;
the counting unit is used for periodically counting the number D of the unexecuted computing tasks and the number X of the minimum execution units in the resource pool;
an adding unit, wherein if D ≧ [ X (R +1) + R ] in A continuous statistical cycles, Z minimum execution units are added in the resource pool, wherein Z ═ D-X/R-X ], A, R is a preset positive integer;
and a reduction unit, if Min is less than or equal to D and less than X in A continuous statistical periods, reducing S minimum execution units in the resource pool, wherein S is (X-D) and S is less than or equal to (X-Min), and Min is a positive integer.
Optionally, as a possible implementation manner, in the computing resource scheduling system in the embodiment of the present invention, the scheduling unit includes:
and the scheduling module is used for calling the computing resources in the resource pool to preferentially execute the computing tasks in the first task sequence, executing the computing tasks in the second task sequence after the computing tasks in the first task sequence are completed, and randomly extracting the computing tasks from the second task sequence to execute.
It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
Fig. 3 is a schematic diagram illustrating only a portion related to an embodiment of the present application for convenience of description, and please refer to a method portion of the embodiment of the present application for details not disclosed. Referring to fig. 3, the computer apparatus 3 includes: memory 310, processor 320, wired or wireless network module 330, and computer programs stored in the memory and executable on the processor. The processor, when executing the computer program, implements the steps in the various embodiments of the computing resource scheduling method described above, such as steps 101 to 103 shown in fig. 1. Alternatively, the processor, when executing the computer program, implements the functions of each module or unit in the above-described device embodiments.
In some of the embodiments of the application, the processor, when executing the computer program stored in the memory, implements the steps of:
receiving computing tasks submitted by at least two clients;
respectively carrying out calculation task classification processing on the calculation tasks of each client, wherein the calculation task classification processing comprises the following steps: selecting a preset number N of computing tasks from M computing tasks initiated by the same client to be added into a first task sequence, and adding the rest (M-N) computing tasks into a second task sequence, wherein N is less than or equal to M;
and calling the computing resources in the resource pool to preferentially execute the computing tasks in the first task sequence, and after the computing tasks in the first task sequence are completed, executing the computing tasks in the second task sequence.
Optionally, in some embodiments of the present application, the processor may be further configured to implement the following steps:
dividing computing resources in a resource pool into a plurality of minimum execution units;
periodically counting the number D of unexecuted computing tasks and the number X of minimum execution units in the resource pool;
if D is more than or equal to [ X (R +1) + R ] in continuous A statistical cycles, adding Z minimum execution units in the resource pool, wherein Z is [ (D-X)/R-X ], A, R is a positive integer;
and if Min is less than or equal to D and less than X in the continuous A statistical periods, reducing S minimum execution units in the resource pool, wherein S is (X-D) and S is less than or equal to (X-Min), and Min is a positive integer.
Optionally, in some embodiments of the present application, the processor may be further configured to implement the following steps:
and randomly extracting the computing task from the second task sequence for execution.
Optionally, in some embodiments of the present application, the processor may be further configured to implement the following steps:
configuring a maximum execution time for each type of computing task;
and if the execution of the target computing task by the first execution unit exceeds the corresponding maximum execution time, calling the second execution unit to replace the first execution unit to execute the target computing task again.
Optionally, in some embodiments of the present application, the processor may be further configured to implement the following steps:
and when the execution of the computing task fails, judging whether the computing data corresponding to the computing task is correct or not, if not, not re-executing the computing task, otherwise, re-executing.
It will be appreciated by those skilled in the art that the arrangement shown in fig. 3 does not constitute a limitation of the computer arrangement, and that the computer arrangement 3 may comprise more or less components than those shown, or some components may be combined, or a different arrangement of components, e.g. the computer arrangement may also comprise input-output devices, buses etc.
The Processor may be a general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic, discrete hardware components, or the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The memory may be used to store computer programs and/or modules, and the processor may implement various functions of the computer device by executing or executing the computer programs and/or modules stored in the memory, as well as by invoking data stored in the memory. The memory may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required by at least one function (such as a sound playing function, an image playing function, etc.), and the like; the storage data area may store data (such as audio data, a phonebook, etc.) created according to the use of the cellular phone, and the like. In addition, the memory may include high speed random access memory, and may also include non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), at least one magnetic disk storage device, a Flash memory device, or other volatile solid state storage device.
The present application further provides a computer-readable storage medium having a computer program stored thereon, which when executed by a processor, can implement the steps of:
receiving computing tasks submitted by at least two clients;
respectively carrying out calculation task classification processing on the calculation tasks of each client, wherein the calculation task classification processing comprises the following steps: selecting a preset number N of computing tasks from M computing tasks initiated by the same client to be added into a first task sequence, and adding the rest (M-N) computing tasks into a second task sequence, wherein N is less than or equal to M;
and calling the computing resources in the resource pool to preferentially execute the computing tasks in the first task sequence, and after the computing tasks in the first task sequence are completed, executing the computing tasks in the second task sequence.
Optionally, in some embodiments of the present application, the processor may be further configured to implement the following steps:
dividing computing resources in a resource pool into a plurality of minimum execution units;
periodically counting the number D of unexecuted computing tasks and the number X of minimum execution units in the resource pool;
if D is more than or equal to [ X (R +1) + R ] in continuous A statistical cycles, adding Z minimum execution units in the resource pool, wherein Z is [ (D-X)/R-X ], A, R is a positive integer;
and if Min is less than or equal to D and less than X in the continuous A statistical periods, reducing S minimum execution units in the resource pool, wherein S is (X-D) and S is less than or equal to (X-Min), and Min is a positive integer.
Optionally, in some embodiments of the present application, the processor may be further configured to implement the following steps:
and randomly extracting the computing task from the second task sequence for execution.
Optionally, in some embodiments of the present application, the processor may be further configured to implement the following steps:
configuring a maximum execution time for each type of computing task;
and if the execution of the target computing task by the first execution unit exceeds the corresponding maximum execution time, calling the second execution unit to replace the first execution unit to execute the target computing task again.
Optionally, in some embodiments of the present application, the processor may be further configured to implement the following steps:
and when the execution of the computing task fails, judging whether the computing data corresponding to the computing task is correct or not, if not, not re-executing the computing task, otherwise, re-executing.
In the several embodiments provided in the present application, it should be understood that the disclosed system, apparatus and method may be implemented in other manners. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is only one logical division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
The above-mentioned embodiments are only used for illustrating the technical solutions of the present invention, and not for limiting the same; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (10)
1. A method for scheduling computing resources, comprising:
receiving computing tasks submitted by at least two clients;
respectively carrying out calculation task classification processing on the calculation tasks of each client, wherein the calculation task classification processing comprises the following steps: selecting a preset number N of computing tasks from M computing tasks initiated by the same client to be added into a first task sequence, and adding the rest (M-N) computing tasks into a second task sequence, wherein N is less than or equal to M;
and calling the computing resources in the resource pool to preferentially execute the computing tasks in the first task sequence, and after the computing tasks in the first task sequence are completed, executing the computing tasks in the second task sequence.
2. The method of claim 1, further comprising:
dividing computing resources in the resource pool into a plurality of minimum execution units;
periodically counting the number D of unexecuted computing tasks, the number X of minimum execution units in a resource pool and the number K of minimum execution units in an idle state;
if D is more than or equal to [ X (R +1) + R ] in continuous A statistical cycles, adding Z minimum execution units in the resource pool, wherein Z is [ (D-X)/R-X ], A, R is a preset positive integer;
and when D is more than or equal to (K + Min-X) and less than K in continuous B statistical periods, eliminating S virtual machines in the resource pool, wherein S is more than or equal to (K-D) and S is less than or equal to (X-Min), and B, Min is a preset positive integer.
3. The method of claim 1 or 2, wherein the performing the computing task in the second sequence of tasks comprises:
and randomly extracting a computing task from the second task sequence for execution.
4. The method of claim 3, further comprising:
configuring a maximum execution time for each type of computing task;
and if the execution of the target computing task by the first execution unit exceeds the corresponding maximum execution time, calling a second execution unit to replace the first execution unit to execute the target computing task again.
5. The method of claim 3, further comprising:
when the execution of the computing task fails, judging whether the computing data corresponding to the computing task is correct or not, if not, not re-executing the computing task, otherwise, re-executing.
6. A computing resource scheduling system, comprising:
the receiving unit is used for receiving the computing tasks submitted by at least two clients;
the classification unit is used for performing calculation task classification processing on the calculation tasks of each client respectively, and the calculation task classification processing comprises the following steps: selecting a preset number N of computing tasks from M computing tasks initiated by the same client to be added into a first task sequence, and adding the rest (M-N) computing tasks into a second task sequence, wherein N is less than or equal to M;
and the scheduling unit is used for calling the computing resources in the resource pool to preferentially execute the computing tasks in the first task sequence, and executing the computing tasks in the second task sequence after the computing tasks in the first task sequence are completed.
7. The system of claim 6, further comprising:
a dividing unit, configured to divide the computing resources in the resource pool into a plurality of minimum execution units;
the counting unit is used for periodically counting the number D of the unexecuted computing tasks and the number X of the minimum execution units in the resource pool;
an adding unit, wherein if D is more than or equal to [ X (R +1) + R ] in continuous A statistical cycles, Z minimum execution units are added in the resource pool, wherein Z is [ (D-X)/R-X ], and R is a positive integer;
and a reduction unit, if Min is less than or equal to D and less than X in A continuous statistical periods, reducing S minimum execution units in the resource pool, wherein S is (X-D) and S is less than or equal to (X-Min), and Min is a positive integer.
8. The system according to claim 6 or 7, wherein the scheduling unit comprises:
and the scheduling module is used for calling the computing resources in the resource pool to preferentially execute the computing tasks in the first task sequence, executing the computing tasks in the second task sequence after the computing tasks in the first task sequence are completed, and randomly extracting the computing tasks from the second task sequence to execute.
9. A computer arrangement, characterized in that the computer arrangement comprises a processor for implementing the steps of the method according to any one of claims 1 to 5 when executing a computer program stored in a memory.
10. A computer-readable storage medium having stored thereon a computer program, characterized in that: the computer program when executed by a processor implementing the steps of the method according to any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911001129.3A CN110764890A (en) | 2019-10-21 | 2019-10-21 | Computing resource scheduling method, system and related equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911001129.3A CN110764890A (en) | 2019-10-21 | 2019-10-21 | Computing resource scheduling method, system and related equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110764890A true CN110764890A (en) | 2020-02-07 |
Family
ID=69331553
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911001129.3A Pending CN110764890A (en) | 2019-10-21 | 2019-10-21 | Computing resource scheduling method, system and related equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110764890A (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20130283097A1 (en) * | 2012-04-23 | 2013-10-24 | Yahoo! Inc. | Dynamic network task distribution |
| CN103577267A (en) * | 2012-08-03 | 2014-02-12 | 上海博泰悦臻电子设备制造有限公司 | Resource distribution method and resource distribution device of vehicle-mounted device |
| CN105246143A (en) * | 2015-10-23 | 2016-01-13 | 江苏省电力公司南京供电公司 | Transmitting power setting and resource block scheduling method for electric wireless private network |
| CN105573850A (en) * | 2015-11-09 | 2016-05-11 | 广州多益网络科技有限公司 | Multi-process interactive method and system and server |
| CN107025139A (en) * | 2017-03-21 | 2017-08-08 | 北京天云融创软件技术有限公司 | A kind of high-performance calculation Scheduling Framework based on cloud computing |
| CN109471727A (en) * | 2018-10-29 | 2019-03-15 | 北京金山云网络技术有限公司 | A kind of task processing method, apparatus and system |
-
2019
- 2019-10-21 CN CN201911001129.3A patent/CN110764890A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20130283097A1 (en) * | 2012-04-23 | 2013-10-24 | Yahoo! Inc. | Dynamic network task distribution |
| CN103577267A (en) * | 2012-08-03 | 2014-02-12 | 上海博泰悦臻电子设备制造有限公司 | Resource distribution method and resource distribution device of vehicle-mounted device |
| CN105246143A (en) * | 2015-10-23 | 2016-01-13 | 江苏省电力公司南京供电公司 | Transmitting power setting and resource block scheduling method for electric wireless private network |
| CN105573850A (en) * | 2015-11-09 | 2016-05-11 | 广州多益网络科技有限公司 | Multi-process interactive method and system and server |
| CN107025139A (en) * | 2017-03-21 | 2017-08-08 | 北京天云融创软件技术有限公司 | A kind of high-performance calculation Scheduling Framework based on cloud computing |
| CN109471727A (en) * | 2018-10-29 | 2019-03-15 | 北京金山云网络技术有限公司 | A kind of task processing method, apparatus and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110764872A (en) | Automatic tax declaring method and system based on cloud service architecture and related equipment | |
| CN110750341B (en) | Task scheduling method, device, system, terminal equipment and storage medium | |
| CN109669775B (en) | Distributed task scheduling method, system and storage medium | |
| CN102546946B (en) | Method and device for processing task on mobile terminal | |
| CN112003797B (en) | Method, system, terminal and storage medium for improving performance of virtualized DPDK network | |
| CN109684080B (en) | Centralized task scheduling method, system and storage medium | |
| CN109343972B (en) | Task processing method and terminal equipment | |
| CN110708369B (en) | File deployment method and device for equipment nodes, scheduling server and storage medium | |
| CN108829519A (en) | Method for scheduling task, cloud platform and computer readable storage medium based on cloud platform | |
| CN108304272B (en) | Data IO request processing method and device | |
| CN106339802A (en) | Task allocation method, task allocation device and electronic equipment | |
| CN109947568A (en) | Timed task processing method, system, computer equipment and storage medium | |
| CN115033352A (en) | Task scheduling method, device and equipment for multi-core processor and storage medium | |
| CN112395075A (en) | Resource processing method and device and resource scheduling system | |
| CN111538572A (en) | Task processing method, device, scheduling server and medium | |
| CN103299298A (en) | Service processing method and system | |
| CN108897850B (en) | Data processing method and device | |
| CN104572276A (en) | Cloud-computation-based concurrent data processing method | |
| CN110764873B (en) | Virtual machine resource management method, system and related equipment | |
| CN117149665A (en) | Continuous integration method, control device, continuous integration system, and storage medium | |
| CN110764890A (en) | Computing resource scheduling method, system and related equipment | |
| CN111831408A (en) | Asynchronous task processing method and device, electronic equipment and medium | |
| CN109558254B (en) | Asynchronous callback method, system, device and computer readable storage medium | |
| CN111259045A (en) | Data processing method, device, server and medium | |
| CN110851166A (en) | User-unaware application program updating method and device and computer equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200207 |