CN105786611A - Method and device for task scheduling of distributed cluster - Google Patents
Method and device for task scheduling of distributed cluster Download PDFInfo
- Publication number
- CN105786611A CN105786611A CN201410784419.0A CN201410784419A CN105786611A CN 105786611 A CN105786611 A CN 105786611A CN 201410784419 A CN201410784419 A CN 201410784419A CN 105786611 A CN105786611 A CN 105786611A
- Authority
- CN
- China
- Prior art keywords
- task
- node
- scheduling
- module
- distributed type
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Landscapes
- Multi Processors (AREA)
Abstract
The invention provides a method and device for task scheduling of a distributed cluster. The method for the task scheduling comprises the steps that a task registration request carrying a service scheduling template is received, and correctness of the service scheduling template is verified; according to the service scheduling temperature which passes verification, a service module corresponding to the service scheduling template is established; and according to the service module, scheduling management is carried out on a task corresponding to the service module. The method and device provided by the embodiment of the invention are characterized in that through analysis of the task registration request carrying the service scheduling template, the service module corresponding to the service scheduling template is established, so that a task scheduling system can complete the task scheduling of multiple types of services; and a uniform REST interface is provided, and the task scheduling management is carried out on the distributed cluster by the REST interface, so that increasingly expanded demands for the task scheduling can be satisfied.
Description
Technical field
The present invention relates to communication technical field, particularly to method for scheduling task and the device of a kind of distributed type assemblies.
Background technology
Distributed type assemblies is formed together by multiple stage computer so that certain organizational structure is loosely connected, from the outside, feels that it is just as a separate payment, also has the feature such as extensibility and high availability simultaneously.Distributed type assemblies node is generally all located at network than in LAN faster, and each node can run the operating system of oneself.The low-cost technologies brought along with the development of computer technology, high-speed communicating network, high-performance computer software etc., the application of distributed type assemblies also constantly emerges in large numbers.The development of cloud computing in recent years and big data also promotes the development of distributed type assemblies.
Distributed type assemblies to widely use the task scheduling also making distributed type assemblies more and more important.The scheduling of distributed type assemblies and application task thereof is often required for various configuration file, it is also desirable to perform dependent tasks schedule control command on each server node.If carrying out manual task scheduling for each node, that will necessarily do and many repeat lengthy and tedious work.
Hence for the task scheduling of distributed type assemblies, industry it is also proposed some schemes.The task scheduling approach of several distributed type assemblies is listed below:
Scheme one: a kind of distributed timing method for scheduling task, comprises the steps: a) task distributor system start-up, the registration of the performer in the submission task of monitoring task submitter's system and performer system；B) described performer system start-up, from task distributor system registry described in trend；C) described task submitter's system submits task to described task distributor's system；D), after described task distributor's system receives task, performer corresponding to the registered and described task in described performer system that notifies is to take described task away；E) the described performer corresponding with described task is taken described task away and performs described task.The distributed timing task scheduling system utilizing the present invention has the management feature such as simple, easy to operate, it is possible to settlement procedure data are more, and the problem such as managerial confusion.The task of the single business that the program is applicable to cluster is simply dispatched, suitable for application under the scene that those group services are continually changing.
Scheme two: the method for scheduling task of a kind of Distributed Computer System and system, wherein said method comprises the steps: 1. Distributed Computer System cluster interior joint to be arranged Centroid and working node；2. input file is divided into multiple data segment by Centroid；3. Centroid is working node distribution data segment and task, and Centroid therefrom extracts initial key-value pair and processes key-value pair in the middle of generation, stores to public territory；4. Centroid judges whether the process of all data segments completes, if then performing 5., then to perform 3. if not；5. centre key-value pair it is divided into multiple district and sorts；Middle key-value pair is processed by the working node 6. finally completing data segment process, and final result stores affiliated district；7. return.The method for scheduling task of Distributed Computer System of the present invention and system, the Centroid being responsible for distribution task is merely responsible for distribution task, finally collects computing result and is completed by working node, thus alleviating the system bottleneck problem in the abbreviation stage.The program completes the Portable Batch System of distributed type assemblies, it is only limited to the Portable Batch System of single business equally, better task scheduling Interface Expanding new business cannot be provided better so that it is the task of new business can also be able to be uniformly scheduled to external system.
Scheme three: a kind of system and method for processing multistage distributed task dispatching in data processing field, described system is made up of some the main frames being responsible for concrete Business Processing, one or more product process of aggregation is responsible for by every main frame, by storage area network share and access disk between all of main frame, and carry out data between hosts synchronization by shared disk, message synchronization is carried out by distributed lock between All Activity main frame, functional module is responsible for the request of ownership the machine product set and is processed, distributed subtransaction processes the main functional module at every main frame and completes.The present invention is compared with the existing technology, have an advantage in that, by the distributed treatment effeciency improving multistage distributed complex affairs, reduce the cost of rollback during this type of issued transaction failure, be particularly well-suited in securities data treatment industry to have the multistage, markov property, sub stage distributivity complex transaction high efficient and reliable process.This dispatching technique is suitable for the task scheduling of specific area, but cannot accomplish with multiple external systems equally well mutual.
Concrete, the method for scheduling task of existing Based on Distributed cluster mainly has following several ways:
1. manual task scheduling: this dispatching method is flexible, scheduling is simple directly, and supervisor's motility is strong.Its shortcoming is also a lot, and the scale cluster being suitable for is less, and class of business is also few, less can do too many change, is also required to user simultaneously and goes scheduling the more time；
2. automanual task scheduling: the general utility functions of task scheduling can be completed, but personalized scheduling is accomplished by artificial participation, it is impossible to meet specific task scheduling demand；
3. automatization's task scheduling: task scheduling work is substantially such as through its management, there is certain personalized task scheduling demand, but increase in the scale in the face of cluster, task scheduling needs in ever-increasing situation, this automatization task scheduling is difficult to meet this kind of demand, and its exploitation O&M cost is also surging therewith simultaneously.
Along with scale and the business of distributed type assemblies constantly expand, existing cluster task scheduling needs have following several feature:
1. individual demand is also on the increase, and task scheduling is also constantly needed to change, sometimes even also can the change of business structure and need to change scheduling model；
2. along with the increase of business can produce Portable Batch System new in a large number, it is therefore desirable to provide a management and running mode more easily to rapidly adapt to the development of business.Such as: in hadoop cluster, scheduling hbase task in this cluster or real-time Computational frame Storm are needed when application needs and changes；
3. the regulatory requirement of task scheduling is of a great variety now, changes speed also quickly, and the management and running in distributed type assemblies are more complicated also with the increase of application, and even some management and running service can rely on other management and running service；
4. each operation system is it should be understood that the health status of business, it is also desirable to understand schedule information or the change of distributed type assemblies, it is desirable to can complete information gathering and task management by some general management and running interfaces.
But present existing distributed type assemblies task scheduling system is provided to solve the Mission Scheduling of specific transactions, the modules coupling of system is high, and operation expanding is difficult.
Summary of the invention
It is an object of the invention to provide the method for scheduling task of a kind of distributed type assemblies and device, corresponding task is carried out modularity task scheduling so that a task scheduling system can complete the task scheduling of each business.
In order to achieve the above object, the embodiment of the present invention provides the method for scheduling task of a kind of distributed type assemblies, including:
Receive the registration request carrying traffic scheduling template, and verify the correctness of described traffic scheduling template；
The business module corresponding with described traffic scheduling template is built according to the described traffic scheduling template being verified；
It is scheduling management according to the task that described business module pair is corresponding with described business module.
Wherein, the registration request of traffic scheduling template is carried in described reception, particularly as follows:
Shift REST interface by sign state and carry the registration request of traffic scheduling template.
Wherein, described method also includes:
Receive and perform the Node registry information that node sends；
Node registry is carried out according to described Node registry information.
Wherein, described method also includes:
Receive and perform the operation result information that node sends；
Generate the corresponding operation queue performing node according to described operation result information, and described operation queue is fed back to described execution node so that described execution node can carry out associative operation according to described operation queue.
Wherein, described method also includes:
Preset task information is sent so that described execution node can according to described preset task information and executing corresponding operating to performing node.
Wherein, described method also includes:
Receive the application programming interface API Calls request that external business sends；
Corresponding implement plan is generated according to the request of described API Calls；
Generate corresponding operating list according to described implement plan, and carry out corresponding operating.
Wherein, described reception or transmission particularly as follows:
The Node registry information that node sends is performed by REST interface；Or
The operation result information that node sends is performed by REST interface；Or
By the API Calls request that REST interface external business sends；Or
Preset task information is sent to performing node by REST interface.
The embodiment of the present invention also provides for the task scheduling apparatus of a kind of distributed type assemblies, including:
First receiver module, for receiving the registration request carrying traffic scheduling template, and verifies the correctness of described traffic scheduling template；
Build module, for building the business module corresponding with described traffic scheduling template according to the described traffic scheduling template being verified；
Scheduler module, for being scheduling management according to the task that described business module pair is corresponding with described business module.
Wherein, described first receiver module includes:
Receive submodule, carry the registration request of traffic scheduling template for shifting REST interface by sign state.
Wherein, described device also includes:
Second receiver module, performs, for receiving, the Node registry information that node sends；
Registering modules, for carrying out Node registry according to described Node registry information.
Wherein, described device also includes:
3rd receiver module, performs, for receiving, the operation result information that node sends；
Generate feedback module, for generating the corresponding operation queue performing node according to described operation result information, and described operation queue is fed back to described execution node so that described execution node can carry out associative operation according to described operation queue.
Wherein, described device also includes:
Sending module, for performing node transmission preset task information so that described execution node can according to described preset task information and executing corresponding operating.
Wherein, described device also includes:
4th receiver module, for receiving the application programming interface API Calls request that external business sends；
Plan generation module, for generating corresponding implement plan according to the request of described API Calls；
List Generating Module, for generating corresponding operating list according to described implement plan, and carries out corresponding operating.
Wherein, it is characterised in that described device also includes:
First receives submodule, for performing, by REST interface, the Node registry information that node sends；Or
Second receives submodule, for performing, by REST interface, the operation result information that node sends；Or
3rd receives submodule, for the API Calls request sent by REST interface external business；Or
Send submodule, for sending preset task information by REST interface to performing node.
The technique scheme of the present invention at least has the advantages that
In the method for scheduling task of the distributed type assemblies of the embodiment of the present invention and device, by resolving the registration request carrying traffic scheduling template, set up the business module corresponding with traffic scheduling template so that a task scheduling system can complete the task scheduling of multiple business；Unified REST interface is provided simultaneously, by described REST interface, distributed type assemblies is carried out Portable Batch System, meet the demand that task scheduling demand constantly increases.
Accompanying drawing explanation
Fig. 1 represents the basic step schematic diagram of the method for scheduling task of the distributed type assemblies of the embodiment of the present invention；
Fig. 2 represents the structural representation of the distributed task dispatching system of the embodiment of the present invention；
Fig. 3 represents the composition structural representation of the task scheduling apparatus of the distributed type assemblies of the embodiment of the present invention；
Fig. 4 represents the composition structural representation of the Centroid of the distributed task dispatching system of the embodiment of the present invention；
Fig. 5 represents the composition structural representation performing node of the distributed task dispatching system of the embodiment of the present invention；
Fig. 6 represents that the Centroid of the distributed task dispatching system of the embodiment of the present invention carries out the flow chart of task scheduling.
Detailed description of the invention
For making the technical problem to be solved in the present invention, technical scheme and advantage clearly, it is described in detail below in conjunction with the accompanying drawings and the specific embodiments.
The present invention is directed in prior art in the face of the scale of distributed type assemblies increases, automatization's task scheduling is difficult to meet the ever-increasing problem of mission requirements, method for scheduling task and the device of a kind of distributed type assemblies are provided, by resolving the registration request carrying traffic scheduling template, set up the business module corresponding with traffic scheduling template so that a task scheduling system can complete the task scheduling of multiple business；Unified REST interface is provided simultaneously, by described REST interface, distributed type assemblies is carried out Portable Batch System, meet the demand that task scheduling demand constantly increases.
As it is shown in figure 1, the embodiment of the present invention also provides for the method for scheduling task of a kind of distributed type assemblies, including:
Step 11, receives the registration request carrying traffic scheduling template, and verifies the correctness of described traffic scheduling template；
Step 12, builds the business module corresponding with described traffic scheduling template according to the described traffic scheduling template being verified；
Step 13, is scheduling management according to the task that described business module pair is corresponding with described business module.
The method for scheduling task of the distributed type assemblies of the above embodiment of the present invention is applied in a task scheduling system, concrete, the task of each business is required for the mode (i.e. the mode of service registry) by task registration by the task registration of this business to this task scheduling system.It is also preferred that the left task scheduling system is typical host-guest architecture, as in figure 2 it is shown, be mainly made up of two parts: Centroid and execution node.Centroid is center control nodes, is arranged on certain machine, and concentrated controling management whole cluster state；Perform node to need to be arranged in each node of distributed type assemblies, monitor the service status information of this node, perform the service operations relevant to this node；Centroid also provides corresponding management operation application programming interfaces API to outside simultaneously, for the task scheduling to cluster service.
Concrete, after Centroid in this task scheduling system receives registration request, traffic scheduling template in registration request can be carried out task management, and verify correctness and the legitimacy (ensureing that the dependent tasks schedule parameter that user provides is correct) of its traffic scheduling template, in this task scheduling system, the traffic scheduling template for being verified builds corresponding business module so that Centroid can be scheduling management according to the task that business module pair is corresponding with this business module.
Such as, when one task scheduling system needs the task of newly increasing a WORD document function (this task scheduling system original state can not resolve WORD document), Centroid receives the registration request of the traffic scheduling template carrying WORD document, after verifying its correctness and legitimacy, create the business module of a WORD document in this task scheduling system and preserve to data base, then WORD document can be scheduling managing according to the business module of above-mentioned WORD document by described Centroid.The method is by being divided into modules by traffic scheduling according to business, different tasks has only to realize different modules, make to have only to a task scheduling system and can complete the task scheduling of multiple business, for Added Business, have only to realize the relevant interface of task scheduling system and scheduler module thereof, it is registered in this task scheduling system, it is possible to complete the task scheduling of its business.
Concrete, in the above embodiment of the present invention, step 11 particularly as follows:
Step 111, shifts REST interface by sign state and carries the registration request of traffic scheduling template.
It should be noted that a kind of software architecture style that REST is RoyFielding to be put forward in doctor's thesis of 2000, full name is the transfer of sign state.This software architecture thought is existing after doctor RoyFielding puts forward, and just receives academia and Internet circles pay close attention to widely.WEB application framework based on REST style is more more succinct than simple object access protocol and XML remote method invocation (RMI) XML-RPC, therefore also creates many webpage WEB application frameworks achieving REST framework style.
In WEB application network, REST treats whole network by the angle of resource, and the content belonged in network is used as a kind of resource, and every kind of resource can identify URI (Uniform Resource Identifier) with one and uniquely represent.Therefore client obtains the sign of resource by URI, it is thus achieved that these signs cause these application programs to change its state.Along with the sign constantly obtaining resource, client application constantly changes its state, i.e. so-called sign state transfer.
And the REST that doctor RoyFielding proposes is it is thus desirable to meet following several rule:
1. resource is specified by URI；2. the operation of pair resource includes acquisition, creates, revises, deletes, the method for these operations just GET, POST, PUT, DELETE that correspondence http protocol provides.
Owing to different tasks has the management and running rule of oneself, it is also required to provide a user with corresponding interface by they operations, therefore the embodiment of the present invention provides the interface that some are unified, i.e. REST interface, controls the task schedule information of management distributed type assemblies for applications.
Owing to Centroid manages the status information of each node and task so that it is have the feature of the centralized scheduling of distributed type assemblies.Emphasis describes node and reports the handling process of (step 21-step 24), node execution (step 25) and API Calls (step 26-step 28) below.
Further, in the above embodiment of the present invention, described method also includes:
Step 21, receives and performs the Node registry information that node sends；Concrete, perform, by REST interface, the Node registry information that node sends
Step 22, carries out Node registry according to described Node registry information.
Further, in the above embodiment of the present invention, described method also includes:
Step 23, receives and performs the operation result information that node sends；Concrete, perform, by REST interface, the operation result information that node sends
Step 24, generates the corresponding operation queue performing node according to described operation result information, and described operation queue feeds back to described execution node so that described execution node can carry out associative operation according to described operation queue.
Further, in the above embodiment of the present invention, described method also includes:
Step 25, sends preset task information to performing node so that described execution node can according to described preset task information and executing corresponding operating.Concrete, send preset task information by REST interface to performing node
Further, in the above embodiment of the present invention, described method also includes:
Step 26, receives the application programming interface API Calls request that external business sends；Concrete, by the API Calls request that REST interface external business sends
Step 27, generates corresponding implement plan according to the request of described API Calls；
Step 28, generates corresponding operating list according to described implement plan, and carries out corresponding operating.
The above embodiment of the present invention proposes a kind of distributed type assemblies modularity method for scheduling task in the applied environment of distributed type assemblies, this method for scheduling task can be divided into various module by business task scheduling, different tasks has only to realize different modules, it is registered to the task scheduling system of this distributed type assemblies, it is possible to corresponding task is carried out modularity task scheduling simultaneously；Simultaneously also can provide unified REST service to applications, as long as it calls corresponding REST interface, it is possible to distributed type assemblies is carried out Portable Batch System.So in the task scheduling for those distributed systems when being continually changing, just have only to realize corresponding dispatching management module according to certain rule and be registered in modularity dispatching management information system, it is possible to be readily accomplished the management and running function of corresponding task.
In order to better realize above-mentioned purpose, as it is shown on figure 3, the embodiment of the present invention also provides for the task scheduling apparatus of a kind of distributed type assemblies, including:
First receiver module 31, for receiving the registration request carrying traffic scheduling template, and verifies the correctness of described traffic scheduling template；
Build module 32, for building the business module corresponding with described traffic scheduling template according to the described traffic scheduling template being verified；
Scheduler module 33, for being scheduling management according to the task that described business module pair is corresponding with described business module.
It is also preferred that the left in the above embodiment of the present invention, described first receiver module 31 includes:
Receive submodule, carry the registration request of traffic scheduling template for shifting REST interface by sign state.
It is also preferred that the left in the above embodiment of the present invention, described device also includes:
Second receiver module, performs, for receiving, the Node registry information that node sends；
Registering modules, for carrying out Node registry according to described Node registry information.
It is also preferred that the left in the above embodiment of the present invention, described device also includes:
3rd receiver module, performs, for receiving, the operation result information that node sends；
Generate feedback module, for generating the corresponding operation queue performing node according to described operation result information, and described operation queue is fed back to described execution node so that described execution node can carry out associative operation according to described operation queue.
It is also preferred that the left in the above embodiment of the present invention, described device also includes:
Sending module, for performing node transmission preset task information so that described execution node can according to described preset task information and executing corresponding operating.
It is also preferred that the left in the above embodiment of the present invention, described device also includes:
4th receiver module, for receiving the application programming interface API Calls request that external business sends；
Plan generation module, for generating corresponding implement plan according to the request of described API Calls；
List Generating Module, for generating corresponding operating list according to described implement plan, and carries out corresponding operating.
It is also preferred that the left in the above embodiment of the present invention, described device also includes:
First receives submodule, for performing, by REST interface, the Node registry information that node sends；Or
Second receives submodule, for performing, by REST interface, the operation result information that node sends；Or
3rd receives submodule, for the API Calls request sent by REST interface external business；Or
Send submodule, for sending preset task information by REST interface to performing node.
It should be noted that, above-mentioned modules or submodule be only the function of the dispatching device that the clearer description embodiment of the present invention provides and according to the virtual module arranged, below in conjunction with Centroid and perform each concrete module method for scheduling task that the embodiment of the present invention is provided of node and be described further:
As shown in Figure 4, Centroid is mainly made up of several modules following, and these modules are divided into four layers from top to bottom:
Ground floor is metadata management module (data base), divides from data type, is broadly divided into two classes: nodal information and mission bit stream.Divide by state, can be largely classified into three classes: virtual condition, it is desirable to state, mode of operation.
The second layer mainly includes Node registry module, task registration module (i.e. service register module), state management module, implement plan generation module, operation management module etc..Node registry module is for the registration management of node, and new node joins in this task scheduling system；Task registration management module is for the registration management of task, it is necessary first to by the checking of system manager；State management module includes the maintenance and management of node and task status information；Implement plan management module changes into corresponding implement plan for request instruction；Operation management module generates a series of operation corresponding implement plan.
Third layer is divided into execution node tasks processing module and API task processing module.REST asks forwarding module checking REST can mainly generate two class instructions after asking legitimacy: the first kind is carried out node tasks instruction, and it can be transmitted to execution node tasks processing module and process；Equations of The Second Kind is API request instruction, and it can be transmitted to API request processing module and process.
4th layer mainly has REST to ask forwarding module, and primary responsibility performs the REST request of node and external management service.REST request management module can check these REST legitimacy asked, and otherwise can refuse this request.If legal REST request, then corresponding processing instruction can be changed into and be transmitted to execution node tasks processing module or the API request processing module of next layer.
As it is shown in figure 5, the framework performing node is mainly made up of three modules and two queues.Three modules are the communication module, operation executing module and the task acquisition module that communicate with Centroid.Two queue respectively operation queues and message queue.Communication module mainly communicates with Centroid, and there is operation queue returning, from Centroid, the operational order come, described operational order is sent to operation executing module (or operation executing module obtains operational order from described operation queue) and performs with this by operation queue.Task acquisition module is primarily used to the mission bit stream gathered on this node, additionally also can with the basic status information of some nodes, and mission bit stream and status information are existed in result queue, again by result queue by these mission bit streams and status information feedback to Centroid so that Centroid can connect the state of each node in time.
In this method for scheduling task, performing node can regularly to Centroid report information, and Centroid also can return to execution node sequence of operations.These operations are broadly divided into three major types:
1. registration operation: this generic operation mainly notifies that performing node sends registration request to Centroid.Being for general on new node and join the registration carrying out new node when coming in cluster, this registration request also can be verified by Centroid, if legal, will pass through to register.
2. state operation: this generic operation mainly tells that performing node needs to send the status information of this some task of node to Centroid.
3. perform operation: require that performing node performs some corresponding task, mainly includes the loading of node tasks module, the execution of task configuration, change of task status etc..
Handling process task registration, API Calls and node being reported three aspects below in conjunction with the module of concrete Centroid and execution node is described specifically:
One, task registration flow process
In the method for scheduling task of this distributed type assemblies, the task of each business is required for the mode (i.e. the mode of service registry) by task registration, the task registration of this business to this task scheduling system.Operation system can adopt the REST mode asked to send registration request to Centroid, after task registration module receives request, the scheduling template that business provides can be carried out task management, and verify the correctness of its task module and legitimacy (ensureing that the dependent tasks schedule parameter that user provides is correct), the relevant information of this task registration can be stored in data base simultaneously.When performing node and reporting nodal information to Centroid, what can obtain corresponding task registration performs operation.
Two, node reports flow process
Perform node and periodically can send the mission bit stream of interdependent node and operation thereof to Centroid.When Centroid receives this category information, this category information can be transmitted to node tasks processing module, if Node registry information, then carry out Node registry entering Node registry module.Also can produce the event relevant to node passes to state management module simultaneously.State management module can carry out the adjustment of node state according to event property, the virtual condition of node can be write in data base simultaneously.Node tasks processing module also can pass to operation management module operating result.Operation management module maintains the execution process of each task operating, also operation queue can be returned to node tasks processing module simultaneously.Node tasks processing module, will return to execution node the operation queue of this node, performs node and carries out associative operation according to action type.
Three, API Calls flow process
REST asks forwarding module to receive the API Calls request of external business or application equally, and it can be given to API request processing module such request.API request processing module can first be stored in these mission bit streams in data base, and also can call implement plan generation module is transformed into relevant operating list task status change directive simultaneously, then these operating lists is sent to operation management module.
The embodiment of the present invention is a kind of method of task United Dispatching being adapted in distributed type assemblies each business.In large-scale distributed type assemblies, have only to a task scheduling system and can complete the task scheduling of each business, for Added Business, it is only necessary to realize relevant interface and its scheduling template of task scheduling system, it is registered in this task scheduling system, it is possible to complete the task scheduling of its business.
It should be noted that the embodiment of the present invention provide distributed and dispatching device be the device applying above-mentioned dispatching method, then all embodiments of above-mentioned dispatching method are all suitable in this device, and all can reach same or analogous beneficial effect.
The above is the preferred embodiment of the present invention; it should be pointed out that, for those skilled in the art, under the premise without departing from principle of the present invention; can also making some improvements and modifications, these improvements and modifications also should be regarded as protection scope of the present invention.
Claims (14)
1. the method for scheduling task of a distributed type assemblies, it is characterised in that including:
Receive the registration request carrying traffic scheduling template, and verify the correctness of described traffic scheduling template；
The business module corresponding with described traffic scheduling template is built according to the described traffic scheduling template being verified；
It is scheduling management according to the task that described business module pair is corresponding with described business module.
2. the method for scheduling task of distributed type assemblies according to claim 1, it is characterised in that the registration request of traffic scheduling template is carried in described reception, particularly as follows:
Shift REST interface by sign state and carry the registration request of traffic scheduling template.
3. the method for scheduling task of distributed type assemblies according to claim 1, it is characterised in that described method also includes:
Receive and perform the Node registry information that node sends；
Node registry is carried out according to described Node registry information.
4. the method for scheduling task of distributed type assemblies according to claim 1, it is characterised in that described method also includes:
Receive and perform the operation result information that node sends；
Generate the corresponding operation queue performing node according to described operation result information, and described operation queue is fed back to described execution node so that described execution node can carry out associative operation according to described operation queue.
5. the method for scheduling task of distributed type assemblies according to claim 1, it is characterised in that described method also includes:
Preset task information is sent so that described execution node can according to described preset task information and executing corresponding operating to performing node.
6. the method for scheduling task of distributed type assemblies according to claim 1, it is characterised in that described method also includes:
Receive the application programming interface API Calls request that external business sends；
Corresponding implement plan is generated according to the request of described API Calls；
Generate corresponding operating list according to described implement plan, and carry out corresponding operating.
7. the method for scheduling task of the distributed type assemblies according to any one of claim 3 to 6, it is characterised in that described reception or transmission particularly as follows:
The Node registry information that node sends is performed by REST interface；Or
The operation result information that node sends is performed by REST interface；Or
By the API Calls request that REST interface external business sends；Or
Preset task information is sent to performing node by REST interface.
8. the task scheduling apparatus of a distributed type assemblies, it is characterised in that including:
First receiver module, for receiving the registration request carrying traffic scheduling template, and verifies the correctness of described traffic scheduling template；
Build module, for building the business module corresponding with described traffic scheduling template according to the described traffic scheduling template being verified；
Scheduler module, for being scheduling management according to the task that described business module pair is corresponding with described business module.
9. the task scheduling apparatus of distributed type assemblies according to claim 8, it is characterised in that described first receiver module includes:
Receive submodule, carry the registration request of traffic scheduling template for shifting REST interface by sign state.
10. the task scheduling apparatus of distributed type assemblies according to claim 8, it is characterised in that described device also includes:
Second receiver module, performs, for receiving, the Node registry information that node sends；
Registering modules, for carrying out Node registry according to described Node registry information.
11. the task scheduling apparatus of distributed type assemblies according to claim 8, it is characterised in that described device also includes:
3rd receiver module, performs, for receiving, the operation result information that node sends；
Generate feedback module, for generating the corresponding operation queue performing node according to described operation result information, and described operation queue is fed back to described execution node so that described execution node can carry out associative operation according to described operation queue.
12. the task scheduling apparatus of distributed type assemblies according to claim 8, it is characterised in that described device also includes:
Sending module, for performing node transmission preset task information so that described execution node can according to described preset task information and executing corresponding operating.
13. the task scheduling apparatus of distributed type assemblies according to claim 8, it is characterised in that described device also includes:
4th receiver module, for receiving the application programming interface API Calls request that external business sends；
Plan generation module, for generating corresponding implement plan according to the request of described API Calls；
List Generating Module, for generating corresponding operating list according to described implement plan, and carries out corresponding operating.
14. the task scheduling apparatus of the distributed type assemblies according to any one of claim 10 to 13, it is characterised in that described device also includes:
First receives submodule, for performing, by REST interface, the Node registry information that node sends；Or
Second receives submodule, for performing, by REST interface, the operation result information that node sends；Or
3rd receives submodule, for the API Calls request sent by REST interface external business；Or
Send submodule, for sending preset task information by REST interface to performing node.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410784419.0A CN105786611A (en) | 2014-12-16 | 2014-12-16 | Method and device for task scheduling of distributed cluster |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410784419.0A CN105786611A (en) | 2014-12-16 | 2014-12-16 | Method and device for task scheduling of distributed cluster |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN105786611A true CN105786611A (en) | 2016-07-20 |
Family
ID=56374741
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410784419.0A Pending CN105786611A (en) | 2014-12-16 | 2014-12-16 | Method and device for task scheduling of distributed cluster |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105786611A (en) |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106095562A (en) * | 2016-08-23 | 2016-11-09 | 北京云纵信息技术有限公司 | The method and apparatus of Portable Batch System |
| CN106878378A (en) * | 2016-12-26 | 2017-06-20 | 贵州银行股份有限公司 | Scatterplot processing method in network service management |
| CN106980543A (en) * | 2017-04-05 | 2017-07-25 | 福建智恒软件科技有限公司 | The distributed task dispatching method and device triggered based on event |
| CN108762949A (en) * | 2018-05-21 | 2018-11-06 | 招银云创（深圳）信息技术有限公司 | Centralized dispatching method, system, computer equipment and storage medium |
| CN108874536A (en) * | 2018-05-24 | 2018-11-23 | 北京五八信息技术有限公司 | A kind of dispatching method of cloud service, device, cloud service actuator and storage medium |
| CN109445929A (en) * | 2018-11-16 | 2019-03-08 | 杭州数澜科技有限公司 | A kind of method and system of scheduler task |
| CN109787792A (en) * | 2017-11-10 | 2019-05-21 | 阿里巴巴集团控股有限公司 | A kind of system managing distributed service cluster |
| CN110012008A (en) * | 2019-04-02 | 2019-07-12 | 山东国子软件股份有限公司 | Heterogeneous data source distributed coordination coherence method and system based on http protocol |
| CN110661824A (en) * | 2018-06-28 | 2020-01-07 | 阿里巴巴集团控股有限公司 | Flow regulation and control method and device for servers in distributed cluster |
| CN117093352A (en) * | 2023-10-13 | 2023-11-21 | 之江实验室 | Template-based computing cluster job scheduling system, method and device |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101729514A (en) * | 2008-10-23 | 2010-06-09 | 华为技术有限公司 | Method, device and system for implementing service call |
| CN102509197A (en) * | 2011-12-01 | 2012-06-20 | 中国移动通信集团广东有限公司 | Workflow management system and workflow management method |
| CN102685173A (en) * | 2011-04-14 | 2012-09-19 | 天脉聚源(北京)传媒科技有限公司 | Asynchronous task distribution system and scheduling distribution computing unit |
| CN103428287A (en) * | 2013-08-09 | 2013-12-04 | 华为技术有限公司 | Method and device for establishing connection |
-
2014
- 2014-12-16 CN CN201410784419.0A patent/CN105786611A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101729514A (en) * | 2008-10-23 | 2010-06-09 | 华为技术有限公司 | Method, device and system for implementing service call |
| CN102685173A (en) * | 2011-04-14 | 2012-09-19 | 天脉聚源(北京)传媒科技有限公司 | Asynchronous task distribution system and scheduling distribution computing unit |
| CN102509197A (en) * | 2011-12-01 | 2012-06-20 | 中国移动通信集团广东有限公司 | Workflow management system and workflow management method |
| CN103428287A (en) * | 2013-08-09 | 2013-12-04 | 华为技术有限公司 | Method and device for establishing connection |
Cited By (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106095562A (en) * | 2016-08-23 | 2016-11-09 | 北京云纵信息技术有限公司 | The method and apparatus of Portable Batch System |
| CN106878378A (en) * | 2016-12-26 | 2017-06-20 | 贵州银行股份有限公司 | Scatterplot processing method in network service management |
| CN106878378B (en) * | 2016-12-26 | 2020-02-21 | 贵州银行股份有限公司 | Scatter processing method in network communication management |
| CN106980543B (en) * | 2017-04-05 | 2020-02-07 | 福建智恒软件科技有限公司 | Distributed task scheduling method and device based on event triggering |
| CN106980543A (en) * | 2017-04-05 | 2017-07-25 | 福建智恒软件科技有限公司 | The distributed task dispatching method and device triggered based on event |
| CN109787792A (en) * | 2017-11-10 | 2019-05-21 | 阿里巴巴集团控股有限公司 | A kind of system managing distributed service cluster |
| CN108762949A (en) * | 2018-05-21 | 2018-11-06 | 招银云创（深圳）信息技术有限公司 | Centralized dispatching method, system, computer equipment and storage medium |
| CN108762949B (en) * | 2018-05-21 | 2021-07-09 | 招银云创信息技术有限公司 | Centralized scheduling method, system, computer equipment and storage medium |
| CN108874536A (en) * | 2018-05-24 | 2018-11-23 | 北京五八信息技术有限公司 | A kind of dispatching method of cloud service, device, cloud service actuator and storage medium |
| CN110661824A (en) * | 2018-06-28 | 2020-01-07 | 阿里巴巴集团控股有限公司 | Flow regulation and control method and device for servers in distributed cluster |
| CN110661824B (en) * | 2018-06-28 | 2022-04-12 | 阿里巴巴集团控股有限公司 | Flow control method of server in distributed cluster and storage medium |
| CN109445929A (en) * | 2018-11-16 | 2019-03-08 | 杭州数澜科技有限公司 | A kind of method and system of scheduler task |
| CN110012008A (en) * | 2019-04-02 | 2019-07-12 | 山东国子软件股份有限公司 | Heterogeneous data source distributed coordination coherence method and system based on http protocol |
| CN110012008B (en) * | 2019-04-02 | 2022-03-22 | 山东国子软件股份有限公司 | Heterogeneous data source distributed coordination consistency method and system based on HTTP (hyper text transport protocol) |
| CN117093352A (en) * | 2023-10-13 | 2023-11-21 | 之江实验室 | Template-based computing cluster job scheduling system, method and device |
| CN117093352B (en) * | 2023-10-13 | 2024-01-09 | 之江实验室 | Template-based computing cluster job scheduling system, method and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105786611A (en) | Method and device for task scheduling of distributed cluster | |
| CN103414761B (en) | Mobile terminal cloud resource scheduling method based on Hadoop framework | |
| Zhang et al. | Flexible management of resource service composition in cloud manufacturing | |
| CN105843182A (en) | Power dispatching accident handling scheme preparing system and power dispatching accident handling scheme preparing method based on OMS | |
| CN109491790A (en) | Industrial Internet of Things edge calculations resource allocation methods and system based on container | |
| US8307366B2 (en) | Post-processing phase in a distributed processing system using assignment information | |
| CN110287016A (en) | A kind of distribution flow chart Heterogeneous Computing dispatching method | |
| CN101815013B (en) | Method for monitoring operation of satellite application system based on Ajax and Web service technology | |
| CN105872068A (en) | Cloud platform and automatic operation check method based on same | |
| CN104579792A (en) | Architecture and method for achieving centralized management of various types of virtual resources based on multiple adaptive modes | |
| CN107040416A (en) | A kind of virtual data center visual management method based on Cairngorm frameworks | |
| CN106503091A (en) | A kind of implementation method of changeable data structure automatic synchronization coupling | |
| CN111581635B (en) | Data processing method and system | |
| CN105760498A (en) | Achievement method of XML data and relational database matching | |
| CN105007311A (en) | System and method for resource management based on cloud platform and cloud computing | |
| CN105338045A (en) | Cloud computing resource processing device, method and cloud computing system | |
| Lin et al. | Key technologies and solutions of remote distributed virtual laboratory for E-learning and E-education | |
| CN109144562B (en) | A kind of smart code publication alarm method based on zookeeper | |
| CN108416657A (en) | A kind of order generation method and equipment based on counseling services | |
| JP2010128597A (en) | Information processor and method of operating the same | |
| CN106354507A (en) | Enterprise-level application management system and method for operating same | |
| CN112351106B (en) | Service grid platform containing event grid and communication method thereof | |
| CN103561113B (en) | The generation method and device of Web Service interface | |
| CN108600357A (en) | A kind of foster management system of oil and gas equipment dimension and working method based on SOA | |
| CN102541563A (en) | Method and system for generating monitoring interfaces |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20160720 |