CN106095550A - A kind of calculating method for scheduling task and device - Google Patents
A kind of calculating method for scheduling task and device Download PDFInfo
- Publication number
- CN106095550A CN106095550A CN201610398415.8A CN201610398415A CN106095550A CN 106095550 A CN106095550 A CN 106095550A CN 201610398415 A CN201610398415 A CN 201610398415A CN 106095550 A CN106095550 A CN 106095550A
- Authority
- CN
- China
- Prior art keywords
- operational module
- task
- business
- module
- offline business
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/48—Program initiating; Program switching, e.g. by interrupt
- G06F9/4806—Task transfer initiation or dispatching
- G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
- G06F9/4881—Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/30—Arrangements for executing machine instructions, e.g. instruction decode
- G06F9/30003—Arrangements for executing specific machine instructions
- G06F9/30007—Arrangements for executing specific machine instructions to perform operations on data operands
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention discloses a kind of calculating method for scheduling task and device, described method includes: the calculating task of each business is scheduled to operational module in each storage device of being deployed in cloud storage system respectively, so that operational module is by multiplexing institute calculating resource on a storage device, complete the process to calculating task.Apply scheme of the present invention, it is possible to reduce resource and the consumption etc. of cost.
Description
[technical field]
The present invention relates to Computer Applied Technology, calculate method for scheduling task and device particularly to one.
[background technology]
Store the files such as substantial amounts of user video, picture on cloud storage system, along with the development evolution of product, need
There is provided substantial amounts of computing capability that the file of user is processed, as video code conversion, file decompression, pictorial information extract
Deng.
In prior art, generally use and build the mode of physical computing cluster to provide above-mentioned computing capability, turn with video
As a example by Ma, need to develop the framework that a set of video code conversion is special, it is anticipated that video code conversion scale, Quantity arrives hundreds of
Thousand of physical equipment clusters is used for computing, and this needs to expend huge resource and cost.
[summary of the invention]
The invention provides a kind of calculating method for scheduling task and device, it is possible to reduce resource and the consumption of cost.
Concrete technical scheme is as follows:
A kind of calculating method for scheduling task, including:
The calculating task of each business is scheduled to respectively the Working mould in each storage device of being deployed in cloud storage system
Block, in order to described operational module, by multiplexing institute calculating resource on a storage device, completes the process to calculating task.
According to one preferred embodiment of the present invention, described business includes: offline business and at line service；
Described calculating task includes: off-line task and online task.
According to one preferred embodiment of the present invention, when calculating task is off-line task, task scheduling will be calculated to described work
Include as module:
Receive described operational module and pulling the dispatch request sent before off-line task every time, draw needed for determining this
The offline business taken, carries the offline business determined mark and returns to described operational module in dispatch command, in order to institute
State operational module and pull the off-line task that described offline business mark is corresponding.
According to one preferred embodiment of the present invention, described determine that the offline business pulled needed for this includes:
Arranging dispatching cycle, each dispatching cycle is divided into N number of continuous print time period, the value of N is equal to offline business
Number, the most corresponding offline business of each time period；
When receiving the dispatch request from described operational module, determine the time period being presently in, and by residing
Offline business corresponding to time period is defined as the offline business pulled needed for this.
According to one preferred embodiment of the present invention, the method farther includes:
For needing the independent offline business calculated, from each operational module, select part operational module, when receiving
When the operational module selected sends the dispatch request come, it would be desirable to the independent offline business mark calculated is carried in dispatch command
Return to selected operational module, in order to selected operational module pulls needs the independent offline business calculated mark correspondence
Off-line task.
According to one preferred embodiment of the present invention, the method farther includes:
When the resource using status of place storage device exceedes off-line alarm threshold, described operational module no longer obtains appoints
The off-line task of He Xin, until the resource using status of place storage device is less than described off-line alarm threshold.
According to one preferred embodiment of the present invention, when calculating task is online task, task scheduling will be calculated to described work
Include as module:
When receiving front-end module and sending the online task scheduling request come, select an operational module, by selected
Operational module mark carry in dispatch command, be sent to described front-end module, in order to described front-end module is by described online
Business is sent to selected operational module.
According to one preferred embodiment of the present invention, select an operational module described in include:
An operational module is selected from the resource pool safeguarded；
Described resource pool by the resource using status of place storage device not less than the operational module group of online alarm threshold
Become.
According to one preferred embodiment of the present invention, the method farther includes:
The resource quota of each business is notified to each operational module respectively, in order to described operational module is processing each calculating
During task, however, it is determined that the occupation condition of this calculating task exceedes the resource quota of correspondence, then this calculating task of forced termination.
A kind of calculating task scheduling apparatus, including:
First scheduling unit, respectively deposits for be scheduled to the calculating task of each business respectively to be deployed in cloud storage system
Operational module on storage equipment, in order to described operational module by multiplexing calculating resource on a storage device, complete meter
The process of calculation task.
According to one preferred embodiment of the present invention, described business includes: offline business and at line service；
Described calculating task includes: off-line task and online task.
According to one preferred embodiment of the present invention, when calculating task is off-line task, described first scheduling unit receives institute
State operational module and pulling the dispatch request sent before off-line task every time, determine the offline business pulled needed for this,
The offline business determined mark is carried in dispatch command, returns to described operational module, in order to described operational module pulls
The off-line task that described offline business mark is corresponding.
According to one preferred embodiment of the present invention, described first scheduling unit arranges dispatching cycle, is drawn each dispatching cycle
Being divided into N number of continuous print time period, the value of N is equal to offline business number, and the most corresponding offline business of each time period, when connecing
When receiving the dispatch request from described operational module, determine the time period being presently in, and by corresponding for the residing time period
Offline business is defined as the offline business pulled needed for this.
According to one preferred embodiment of the present invention, described first scheduling unit is further used for,
For needing the independent offline business calculated, from each operational module, select part operational module, when receiving
When the operational module selected sends the dispatch request come, it would be desirable to the independent offline business mark calculated is carried in dispatch command
Return to selected operational module, in order to selected operational module pulls needs the independent offline business calculated mark correspondence
Off-line task.
According to one preferred embodiment of the present invention, when calculating task is online task, described first scheduling unit is receiving
Send the online task scheduling request come to front-end module after, select an operational module, by selected operational module mark
Carry in dispatch command, be sent to described front-end module, in order to described online task is sent to selected by described front-end module
Operational module.
According to one preferred embodiment of the present invention, described first scheduling unit selects a job from the resource pool safeguarded
Module；
Described resource pool by the resource using status of place storage device not less than the operational module group of online alarm threshold
Become.
According to one preferred embodiment of the present invention, described device farther includes:
Second scheduling unit, for notifying to each operational module respectively by the resource quota of each business, in order to described work
Module is when processing each calculating task, however, it is determined that the occupation condition of this calculating task exceedes the resource quota of correspondence, then
This calculating task of forced termination.
It can be seen that use scheme of the present invention, map out the work module on a storage device by described above, will be each
The calculating task of business is scheduled to each operational module respectively, the meter that each operational module can be left unused on a storage device by multiplexing
Calculate resource, complete the process to the task of calculating, thus in the way of near-zero cost, got substantial amounts of computing capability, compare
In prior art, it is not necessary to build physical computing cluster the most specially, and then significantly reduce the consumption to resource and cost, and carry
The high resource utilization of storage device, and, use scheme of the present invention, off-line can be realized and mix portion with online multitask
Scheduling, both provides great convenience to the exploitation of various calculating tasks, deployment, O&M.
[accompanying drawing explanation]
Fig. 1 is the flow chart that the present invention calculates method for scheduling task embodiment.
Fig. 2 is off-line task scheduling mode schematic diagram of the present invention.
Fig. 3 is the present invention online task scheduling mode schematic diagram.
Fig. 4 is the composition structural representation that the present invention calculates task scheduling apparatus embodiment.
[detailed description of the invention]
Cloud storage system, in order to store user file, has the storage device cluster of substantial amounts, owing to being mainly used in depositing
Storage file, therefore to be that disk takies comparison high for the common feature of these storage devices, but calculates resource such as CPU
(CPU, Central Processing Unit) etc. exist a large amount of idle, thus cause the huge waste calculating resource.
For the problems referred to above, propose a kind of general-purpose computing system, i.e. mammoth system, wherein comprise the steps that operational module
(mammoth worker), scheduler module (mammoth observer) and front-end module (mommoth front) etc., by respectively
Cooperating between module, completes the various process such as task access, task scheduling and task computation.
Wherein, front-end module is mainly responsible for task and is accessed, and scheduler module is mainly responsible for task scheduling, and operational module is mainly born
Responsibility business processes (task computation).
Operational module and scheduler module all can be deployed in the storage device in cloud storage system, and number is multiple,
For each storage device, can dispose in operational module or scheduler module above, the number of scheduler module is led to
Often far fewer than the number of operational module, each scheduler module can distinguish corresponding multiple operational module, i.e. for being the multiple of correspondence
Operational module provides dispatch service.
Operational module may utilize calculating resource idle in storage device and processes calculating task, thus without again
Build physical computing cluster specially, and then significantly reduce the consumption to resource and cost, and improve the money of storage device
Source utilization rate.
In order to make technical scheme clearer, clear, develop simultaneously embodiment referring to the drawings, to institute of the present invention
The scheme of stating is described in further detail.
Embodiment one
Fig. 1 is the flow chart that the present invention calculates method for scheduling task embodiment, as it is shown in figure 1, include implementations below:
In 11, the calculating task of each business is scheduled to each storage being deployed in cloud storage system by scheduler module respectively
Operational module on equipment；
In 12, operational module, by multiplexing institute calculating resource on a storage device, completes the process to calculating task.
Above-mentioned business comprises the steps that offline business and at line service, correspondingly, calculating task comprise the steps that off-line task and
Line task.
According to the difference of calculating task type, scheduler module can use different scheduling strategies, be introduced individually below.
One) off-line task
Fig. 2 is off-line task scheduling mode schematic diagram of the present invention, as in figure 2 it is shown, scheduler module receives operational module often
Secondary pull the dispatch request sent before off-line task, determine the offline business pulled needed for this, the off-line that will determine
Service identification carries and returns to operational module in dispatch command, in order to operational module pulls the off-line that offline business mark is corresponding
Task.
Operational module, pulling off-line task before every time, all can send dispatch request to scheduler module, i.e. can inquire tune
Degree this task of needing to pull which offline business of module, scheduler module receives after the inquiry request of operational module,
First determine the offline business pulled needed for this, backward operational module return dispatch command, wherein carry and determine
Offline business mark, notify that operational module goes to pull the task of which offline business in this way.
Operational module receives after the dispatch command of scheduler module, can pull in dispatch command from message-oriented middleware
The off-line task of the offline business mark correspondence carried, the off-line task in message-oriented middleware can be to send out after front-end module receives
Deliver in message-oriented middleware.
Front-end module is responsible for task and is accessed, and by interacting with user, obtains all types of tasks that user issues.Front end
The off-line task received can be sent in message-oriented middleware by module.
Scheduler module receives operational module after pulling the dispatch request sent before off-line task every time, the most really
The offline business determining to pull needed for this can be decided according to the actual requirements, such as: arranges dispatching cycle, is drawn each dispatching cycle
Being divided into N number of continuous print time period, the value of N is equal to offline business number, and the most corresponding offline business of each time period, when connecing
When receiving the dispatch request from operational module, determine the time period being presently in, and by the offline business corresponding this time period
It is defined as the offline business pulled needed for this.
For pending each offline business, the resource distribution ratio of each offline business can be determined according to amount of calculation size etc.
Example, resource allocation proportion is the biggest, and the duration of corresponding time period is the longest, such as, it is assumed that have two offline business, be respectively from
Line service A and offline business B, the resource allocation proportion of offline business A and offline business B is 3:7, then offline business A is corresponding
The ratio of duration of the duration of time period time period corresponding with offline business B be also 3:7, for each operational module,
In each dispatching cycle, within the time period that offline business A is corresponding, this operational module only can pull the off-line of offline business A and appoint
Business, within the time period that offline business B is corresponding, this operational module only can pull the off-line task of offline business B.
The resource allocation proportion of each offline business can be determined by user and be saved in scheduler module afterwards, and can according to from
Described resource allocation proportion is dynamically adjusted by the amount of calculation change etc. of the change of line service number and offline business.
By the way, it is achieved that multi-service mixes the calculating scheduling in portion, and achieves the resource use point between each business
Join.
Distinguishingly, some offline business is too high due to resource consumption, does not reaches the condition calculated with other business portion of mixing, right
Need the independent offline business calculated in this, the present embodiment proposes a kind of dynamic isolation mechanism.
Described dynamic isolation mechanism includes: the offline business independently calculated for needs, selects portion from each operational module
Divide operational module, when receiving selected operational module and sending the dispatch request come, it would be desirable to the independent off-line industry calculated
Business mark is carried and is returned to selected operational module in dispatch command, in order to selected operational module pulls and needs independence
The off-line task of the offline business mark correspondence calculated.
It is to say, for selected operational module, when it sends dispatch request, scheduler module is the most only notified that every time
It goes to pull the off-line task needing the independent offline business calculated.
How scheduler module selects part operational module can be decided according to the actual requirements, and such as can select part work at random
Module, depending on the particular number of selected operational module is also dependent on being actually needed.
It addition, each operational module all can monitor the resource using status of place storage device in real time, when resource uses shape
When state exceedes off-line alarm threshold, can no longer obtain any new off-line task, until resource using status is guarded against less than off-line
Threshold value.
Described resource using status comprises the steps that CPU usage, network io, disk io, number of processes etc., uses for CPU
Each state parameter such as rate, can be respectively provided with an alarm threshold, once has a state parameter to exceed the alarm threshold of correspondence,
Then it is believed that resource using status exceedes off-line alarm threshold.
Described resource using status specifically includes the concrete value of which content and each alarm threshold can be according to actual need
Depending on Yaoing.
2) online task
Fig. 3 is the present invention online task scheduling mode schematic diagram, as it is shown on figure 3, scheduler module receiving front-end module sends
The online task scheduling request come, selects an operational module, selected operational module mark is carried in dispatch command
It is sent to front-end module, in order to online task is sent to selected operational module by front-end module.
After front-end module often receives an online task, dispatch request can be sent to scheduler module, inquire scheduler module
Which operational module be responsible for this online task of process is, scheduler module receives after the dispatch request of front-end module, can
From numerous operational modules, select one process this online task, and the operational module selected mark is carried refer in scheduling
Order returns to front-end module, and then by front-end module, this online task is sent to selected operational module.
Scheduler module can safeguard a resource pool, this resource pool by the resource using status of place storage device not less than
The operational module composition of line alarm threshold, is receiving after the dispatch request of front-end module, and scheduler module can be from being positioned at money
Each operational module in pond, source selects an operational module to process this online task, such as can select a Working mould at random
Block.
Each operational module constantly can report the resource using status of place storage device to scheduler module, if a certain work
The resource using status of module is the best, i.e. exceedes online alarm threshold, can be removed from resource pool by this operational module, all
Line task will not be scheduled to this operational module again.
By the way, it is possible to provide operational module idle as far as possible is given and used at line service, to ensure at line service
Real-time etc., and the storage service of storage device effectively can be protected, i.e. prevent from calculating service to former in storage device
Some storage services impact.
Depending on the concrete value of described online alarm threshold is also dependent on being actually needed, but the value of online alarm threshold is led to
Often less than aforementioned off-line alarm threshold, so, for operational module, receives new online task by first stopping, then stops
Pull new off-line task.
It addition, either offline business is still at line service, scheduler module can in advance by the resource quota of each business respectively
Notice is to each operational module, in order to each operational module is when processing each calculating task, however, it is determined that the resource of this calculating task accounts for
Exceed the resource quota of correspondence by situation, then can this calculating task of forced termination.
The resource quota of each business is consulted to determine the most in advance, once finds a certain during actual calculating
The occupation condition of calculating task exceedes the resource quota consulting in advance to determine, then in order to other business just prevent
Often process and impact, can this calculating task of forced termination, to eliminate abnormal process.
Either off-line task or online task, operational module all can be by multiplexing institute calculating money on a storage device
Source, completes the process to task, how to carry out being processed as prior art.
Embodiment two
Fig. 4 is the composition structural representation that the present invention calculates task scheduling apparatus embodiment, this calculating task scheduling apparatus
I.e. corresponding to the scheduler module in aforementioned mammoth system, as shown in Figure 4, including: the first scheduling unit 41, also can be further
Including: the second scheduling unit 42.
First scheduling unit 41, each for be scheduled to the calculating task of each business respectively to be deployed in cloud storage system
Operational module in storage device, in order to operational module by multiplexing calculating resource on a storage device, complete calculating
The process of task.
Above-mentioned business comprises the steps that offline business and at line service, correspondingly, calculating task comprise the steps that off-line task and
Line task.
According to the difference of calculating task type, the first scheduling unit 41 can use different scheduling strategies.
When calculating task is off-line task, the first scheduling unit 41 receive operational module pull every time off-line task it
The dispatch request that front transmission comes, determines the offline business pulled needed for this, the offline business determined mark is carried and is adjusting
Degree instruction returns to operational module, in order to operational module pulls the off-line task that offline business mark is corresponding.
Operational module, pulling off-line task before every time, all can send dispatch request to the first scheduling unit 41, i.e. can
Inquiring this task of needing to pull which offline business of the first scheduling unit 41, the first scheduling unit 41 receives from work
Module inquiry request after, it is first determined go out the offline business pulled needed for this, backward operational module return dispatch command,
Wherein carry the offline business mark determined, notify that operational module goes to pull which offline business in this way
Task.
Operational module receives after the dispatch command of the first scheduling unit 41, can pull scheduling from message-oriented middleware
The off-line task of the offline business mark correspondence carried in instruction, the off-line task in message-oriented middleware can be that front-end module receives
It is sent in message-oriented middleware after to.
Front-end module is responsible for task and is accessed, and by interacting with user, obtains all types of tasks that user issues.Front end
The off-line task received can be sent in message-oriented middleware by module.
First scheduling unit 41 receives operational module after pulling the dispatch request sent before off-line task every time,
How determine that the offline business pulled needed for this can be decided according to the actual requirements, such as: dispatching cycle is set, by each scheduling
Cycle is divided into N number of continuous print time period, and the value of N is equal to offline business number, the most corresponding off-line industry of each time period
Business, when receiving the dispatch request from operational module, determines the time period being presently in, and by corresponding for this time period from
Line service is defined as the offline business pulled needed for this.
For pending each offline business, the resource distribution ratio of each offline business can be determined according to amount of calculation size etc.
Example, resource allocation proportion is the biggest, and the duration of corresponding time period is the longest.The resource allocation proportion of each offline business can be true by user
Reserve and be saved in afterwards in the first scheduling unit 41, and can be according to the change of offline business number and the amount of calculation change etc. of offline business
Described resource allocation proportion is dynamically adjusted.
By the way, it is achieved that multi-service mixes the calculating scheduling in portion, and achieves the resource use point between each business
Join.
Distinguishingly, some offline business is too high due to resource consumption, does not reaches the condition calculated with other business portion of mixing, right
Need the independent offline business calculated in this, the present embodiment proposes a kind of dynamic isolation mechanism.
For needing the independent offline business calculated, the first scheduling unit 41 selects part Working mould from each operational module
Block, when receiving selected operational module and sending the dispatch request come, it would be desirable to the independent offline business mark calculated is taken
Band returns to selected operational module in dispatch command, in order to selected operational module pull need independent calculate from
The off-line task that line service mark is corresponding.
It is to say, for selected operational module, when it sends dispatch request, the first scheduling unit 41 is all only every time
It is notified that it goes to pull the off-line task needing the independent offline business calculated.
How first scheduling unit 41 selects part operational module can be decided according to the actual requirements, and such as can select portion at random
Divide operational module, depending on the particular number of selected operational module is also dependent on being actually needed.
When calculating task is online task, the first scheduling unit 41 is receiving the online task that front-end module transmission comes
After dispatch request, select an operational module, selected operational module mark is carried in dispatch command, is sent to front end
Module, in order to online task is sent to selected operational module by front-end module.
After front-end module often receives an online task, dispatch request can be sent to the first scheduling unit 41, inquiry the
Which operational module what one scheduling unit 41 was responsible for this online task of process is, the first scheduling unit 41 receives from front end mould
After the dispatch request of block, one can be selected from numerous operational modules and process this online task, and the Working mould that will select
Block identification carries and returns to front-end module in dispatch command, and then is sent to selected by front-end module by this online task
Operational module.
First scheduling unit 41 can safeguard a resource pool, and this resource pool is by the resource using status of place storage device not
Exceed the operational module composition of online alarm threshold, receiving after the dispatch request of front-end module, the first scheduling unit
41 can select an operational module to process this online task from each operational module being positioned at resource pool, such as can select at random
Go out an operational module.
By the way, it is possible to provide operational module idle as far as possible is given and used at line service, to ensure at line service
Real-time etc., and the storage service of storage device effectively can be protected, i.e. prevent from calculating service to former in storage device
Some storage services impact.
It addition, either offline business is still at line service, the resource of each business can be joined by the second scheduling unit 42 in advance
Volume notifies to each operational module respectively, in order to each operational module is when processing each calculating task, however, it is determined that this calculating task
Occupation condition exceedes the resource quota of correspondence, then can this calculating task of forced termination.
The resource quota of each business is consulted to determine the most in advance, once finds a certain during actual calculating
The occupation condition of calculating task exceedes the resource quota consulting in advance to determine, then in order to other business just prevent
Often process and impact, can this calculating task of forced termination, to eliminate abnormal process.
In a word, using scheme of the present invention, map out the work module on a storage device, the calculating task of each business is divided
It is not scheduled to each operational module, the calculating resource that each operational module can be left unused on a storage device by multiplexing, complete meter
The process of calculation task, thus in the way of near-zero cost, got substantial amounts of computing capability, compared to prior art, it is not necessary to
Build physical computing cluster the most specially, and then significantly reduce the consumption to resource and cost, and improve storage device
Resource utilization, and, use scheme of the present invention, off-line can be realized and mix ministry dispatching, to various meters with online multitask
The exploitation of calculation task, deployment, O&M both provide great convenience.
In several embodiments provided by the present invention, it should be understood that disclosed apparatus and method, can be passed through it
Its mode realizes.Such as, device embodiment described above is only schematically, such as, and the division of described unit, only
Being only a kind of logic function to divide, actual can have other dividing mode when realizing.
The described unit illustrated as separating component can be or may not be physically separate, shows as unit
The parts shown can be or may not be physical location, i.e. may be located at a place, or can also be distributed to multiple
On NE.Some or all of unit therein can be selected according to the actual needs to realize the mesh of the present embodiment scheme
's.
It addition, each functional unit in each embodiment of the present invention can be integrated in a processing unit, it is also possible to
It is that unit is individually physically present, it is also possible to two or more unit are integrated in a unit.Above-mentioned integrated list
Unit both can realize to use the form of hardware, it would however also be possible to employ hardware adds the form of SFU software functional unit and realizes.
The above-mentioned integrated unit realized with the form of SFU software functional unit, can be stored in an embodied on computer readable and deposit
In storage media.Above-mentioned SFU software functional unit is stored in a storage medium, including some instructions with so that a computer
Equipment (can be personal computer, server, or the network equipment etc.) or processor (processor) perform the present invention each
The part steps of method described in embodiment.And aforesaid storage medium includes: USB flash disk, portable hard drive, read only memory (ROM,
Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disc or CD etc. each
Plant the medium that can store program code.
The foregoing is only presently preferred embodiments of the present invention, not in order to limit the present invention, all essences in the present invention
Within god and principle, any modification, equivalent substitution and improvement etc. done, within should be included in the scope of protection of the invention.
Claims (17)
1. one kind calculates method for scheduling task, it is characterised in that including:
The calculating task of each business is scheduled to respectively the operational module in each storage device of being deployed in cloud storage system, with
Toilet states operational module by multiplexing institute calculating resource on a storage device, completes the process to calculating task.
Method the most according to claim 1, it is characterised in that
Described business includes: offline business and at line service；
Described calculating task includes: off-line task and online task.
Method the most according to claim 2, it is characterised in that
When calculating task is off-line task, include calculating task scheduling to described operational module:
Receive described operational module and pulling the dispatch request sent before off-line task every time, pull needed for determining this
Offline business, carries the offline business determined mark and returns to described operational module in dispatch command, in order to described work
The off-line task that described offline business mark is corresponding is pulled as module.
Method the most according to claim 3, it is characterised in that
Described determine that the offline business pulled needed for this includes:
Arranging dispatching cycle, each dispatching cycle is divided into N number of continuous print time period, the value of N is equal to offline business number, often
The most corresponding offline business of individual time period；
When receiving the dispatch request from described operational module, determine the time period being presently in, and by the residing time
The offline business of section correspondence is defined as the offline business pulled needed for this.
Method the most according to claim 3, it is characterised in that
The method farther includes:
For needing the independent offline business calculated, from each operational module, select part operational module, selected when receiving
Operational module when sending the dispatch request come, it would be desirable to the independent offline business mark calculated is carried and is returned in dispatch command
To selected operational module, in order to selected operational module pull need the independent offline business calculated mark corresponding from
Line task.
Method the most according to claim 2, it is characterised in that
The method farther includes:
When the resource using status of place storage device exceedes off-line alarm threshold, described operational module no longer obtain any newly
Off-line task, until the resource using status of place storage device is less than described off-line alarm threshold.
Method the most according to claim 2, it is characterised in that
When calculating task is online task, include calculating task scheduling to described operational module:
When receiving front-end module and sending the online task scheduling request come, select an operational module, by selected work
Carry as module id in dispatch command, be sent to described front-end module, in order to described online task is sent out by described front-end module
Give selected operational module.
Method the most according to claim 7, it is characterised in that
Described select an operational module and include:
An operational module is selected from the resource pool safeguarded；
Described resource pool is made up of not less than the operational module of online alarm threshold the resource using status of place storage device.
9. according to the method according to any one of claim 1～8, it is characterised in that
The method farther includes:
The resource quota of each business is notified to each operational module respectively, in order to described operational module is processing each calculating task
Time, however, it is determined that the occupation condition of this calculating task exceedes the resource quota of correspondence, then this calculating task of forced termination.
10. one kind calculates task scheduling apparatus, it is characterised in that including:
First scheduling unit, sets for each storage being scheduled to the calculating task of each business respectively be deployed in cloud storage system
A standby upper operational module, in order to described operational module, by multiplexing institute calculating resource on a storage device, completes calculating times
The process of business.
11. devices according to claim 10, it is characterised in that
Described business includes: offline business and at line service；
Described calculating task includes: off-line task and online task.
12. devices according to claim 11, it is characterised in that
When calculating task is off-line task, described first scheduling unit receives described operational module and is pulling off-line task every time
The dispatch request before sent, determines the offline business pulled needed for this, the offline business determined mark is carried
Dispatch command returns to described operational module, in order to the off-line that described operational module pulls described offline business mark corresponding is appointed
Business.
13. devices according to claim 12, it is characterised in that
Described first scheduling unit arranges dispatching cycle, and each dispatching cycle is divided into N number of continuous print time period, the value of N
Equal to offline business number, the most corresponding offline business of each time period, when receiving the scheduling from described operational module
During request, determine the time period being presently in, and pull needed for the offline business corresponding residing time period is defined as this
Offline business.
14. devices according to claim 12, it is characterised in that
Described first scheduling unit is further used for,
For needing the independent offline business calculated, from each operational module, select part operational module, selected when receiving
Operational module when sending the dispatch request come, it would be desirable to the independent offline business mark calculated is carried and is returned in dispatch command
To selected operational module, in order to selected operational module pull need the independent offline business calculated mark corresponding from
Line task.
15. devices according to claim 11, it is characterised in that
When calculating task is online task, described first scheduling unit is receiving the online task tune that front-end module transmission comes
After degree request, select an operational module, selected operational module mark is carried be sent in dispatch command described before
End module, in order to described online task is sent to selected operational module by described front-end module.
16. devices according to claim 15, it is characterised in that
Described first scheduling unit selects an operational module from the resource pool safeguarded；
Described resource pool is made up of not less than the operational module of online alarm threshold the resource using status of place storage device.
17. according to the device according to any one of claim 10～16, it is characterised in that
Described device farther includes:
Second scheduling unit, for notifying to each operational module respectively by the resource quota of each business, in order to described operational module
When processing each calculating task, however, it is determined that the occupation condition of this calculating task exceedes the resource quota of correspondence, then force
Terminate this calculating task.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610398415.8A CN106095550A (en) | 2016-06-07 | 2016-06-07 | A kind of calculating method for scheduling task and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610398415.8A CN106095550A (en) | 2016-06-07 | 2016-06-07 | A kind of calculating method for scheduling task and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN106095550A true CN106095550A (en) | 2016-11-09 |
Family
ID=57227383
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610398415.8A Pending CN106095550A (en) | 2016-06-07 | 2016-06-07 | A kind of calculating method for scheduling task and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106095550A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2020001320A1 (en) * | 2018-06-27 | 2020-01-02 | 阿里巴巴集团控股有限公司 | Resource allocation method, device, and apparatus |
| CN111435943A (en) * | 2019-01-14 | 2020-07-21 | 阿里巴巴集团控股有限公司 | Data processing method, device, system and storage medium |
| WO2021088207A1 (en) * | 2019-11-04 | 2021-05-14 | 深圳先进技术研究院 | Mixed deployment-based job scheduling method and apparatus for cloud computing cluster, server and storage device |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101072133A (en) * | 2007-05-23 | 2007-11-14 | 华中科技大学 | High-performance computing system based on peer-to-peer network |
| CN101441580A (en) * | 2008-12-09 | 2009-05-27 | 华北电网有限公司 | Distributed paralleling calculation platform system and calculation task allocating method thereof |
| US20120185848A1 (en) * | 2011-01-17 | 2012-07-19 | International Business Machines Corporation | Task prioritization management in a virtualized environment |
| CN104102543A (en) * | 2014-06-27 | 2014-10-15 | 北京奇艺世纪科技有限公司 | Load regulation method and load regulation device in cloud computing environment |
| CN105260244A (en) * | 2015-10-30 | 2016-01-20 | 北京奇艺世纪科技有限公司 | Task scheduling method and device for distributed system |
-
2016
- 2016-06-07 CN CN201610398415.8A patent/CN106095550A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101072133A (en) * | 2007-05-23 | 2007-11-14 | 华中科技大学 | High-performance computing system based on peer-to-peer network |
| CN101441580A (en) * | 2008-12-09 | 2009-05-27 | 华北电网有限公司 | Distributed paralleling calculation platform system and calculation task allocating method thereof |
| US20120185848A1 (en) * | 2011-01-17 | 2012-07-19 | International Business Machines Corporation | Task prioritization management in a virtualized environment |
| CN104102543A (en) * | 2014-06-27 | 2014-10-15 | 北京奇艺世纪科技有限公司 | Load regulation method and load regulation device in cloud computing environment |
| CN105260244A (en) * | 2015-10-30 | 2016-01-20 | 北京奇艺世纪科技有限公司 | Task scheduling method and device for distributed system |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2020001320A1 (en) * | 2018-06-27 | 2020-01-02 | 阿里巴巴集团控股有限公司 | Resource allocation method, device, and apparatus |
| CN110647394A (en) * | 2018-06-27 | 2020-01-03 | 阿里巴巴集团控股有限公司 | Resource allocation method, device and equipment |
| CN110647394B (en) * | 2018-06-27 | 2022-03-11 | 阿里巴巴集团控股有限公司 | Resource allocation method, device and equipment |
| CN111435943A (en) * | 2019-01-14 | 2020-07-21 | 阿里巴巴集团控股有限公司 | Data processing method, device, system and storage medium |
| CN111435943B (en) * | 2019-01-14 | 2022-07-19 | 阿里巴巴集团控股有限公司 | Data processing method, device, system and storage medium |
| WO2021088207A1 (en) * | 2019-11-04 | 2021-05-14 | 深圳先进技术研究院 | Mixed deployment-based job scheduling method and apparatus for cloud computing cluster, server and storage device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103309946B (en) | Multimedia file processing method, Apparatus and system | |
| CN107515786B (en) | Resource allocation method, master device, slave device and distributed computing system | |
| CN105487930B (en) | A kind of optimizing and scheduling task method based on Hadoop | |
| CN104866374B (en) | Discrete event parallel artificial and method for synchronizing time based on multitask | |
| EP3129880B1 (en) | Method and device for augmenting and releasing capacity of computing resources in real-time stream computing system | |
| CN101652750B (en) | Data processing device, distributed processing system and data processing method | |
| CN102508717B (en) | Memory scheduling method and memory scheduling device for multi-core processor | |
| CN102193832B (en) | Cloud computing resource scheduling method and applied system | |
| CN107025139A (en) | A kind of high-performance calculation Scheduling Framework based on cloud computing | |
| CN108600300B (en) | Log data processing method and device | |
| CN105376100B (en) | A kind of distributed warning rule evaluation method suitable for cloud platform monitoring resource | |
| CN106095550A (en) | A kind of calculating method for scheduling task and device | |
| CN103731372A (en) | Resource supply method for service supplier under hybrid cloud environment | |
| CN102902587A (en) | Distribution type task scheduling method, distribution type task scheduling system and distribution type task scheduling device | |
| CN101694633A (en) | Equipment, method and system for dispatching of computer operation | |
| CN109840877B (en) | Graphics processor and resource scheduling method and device thereof | |
| CN110659110B (en) | Block chain based distributed computing method and system | |
| CN101408853A (en) | Apparatus and method for scheduling virtual machine | |
| CN102904961A (en) | Method and system for scheduling cloud computing resources | |
| CN107577534A (en) | A kind of resource regulating method and device | |
| CN110716800B (en) | Task scheduling method and device, storage medium and electronic equipment | |
| CN102662761A (en) | Method and device for scheduling memory pool in multi-core central processing unit system | |
| CN111506430A (en) | Method and device for data processing under multitasking and electronic equipment | |
| CN109032769A (en) | A kind of continuous integrating CI task processing method and device based on container | |
| CN108121599A (en) | A kind of method for managing resource, apparatus and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20161109 |
|
| RJ01 | Rejection of invention patent application after publication |