CN104572286A - Task scheduling method based on distributed memory clusters - Google Patents
Task scheduling method based on distributed memory clusters Download PDFInfo
- Publication number
- CN104572286A CN104572286A CN201510047784.8A CN201510047784A CN104572286A CN 104572286 A CN104572286 A CN 104572286A CN 201510047784 A CN201510047784 A CN 201510047784A CN 104572286 A CN104572286 A CN 104572286A
- Authority
- CN
- China
- Prior art keywords
- task
- distributed memory
- scheduling
- engine
- monitoring module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Landscapes
- Debugging And Monitoring (AREA)
Abstract
The invention relates to the technical field of task scheduling, in particular to a task scheduling method based on distributed memory clusters. The method includes: adding a created task into a task queue to wait for loading; adding one task in the task queue to a distributed memory each time; knowing utilization conditions and utilization rates of task machines by the aid of a monitoring module, and knowing execution conditions of each task by the aid of the monitoring module; allocating the task loaded each time into the task machine lower in utilization rate. By adoption of the task scheduling method based on the distributed memory clusters, dynamically transverse expansion and shrinkage can be realized without affecting normality in system operation, and timeliness in control of task execution conditions and modification of task related parameters is realized; tasks can be allocated reasonably to prevent each machine from overloaded task execution; a complete system monitoring interface is provided to enable people to timely know operation conditions of a whole system; the system is high in scheduling reliability and stability.
Description
Technical field
The present invention relates to task scheduling technique field, particularly a kind of method for scheduling task based on distributed memory cluster.
Background technology
In recent years, along with the high speed development of internet arena, task engine (server) scale that internet relies on is also more and more huger.And along with the continuous expansion of application, multiple application of user often perform in different task engines.Therefore how these service relations across machine application are managed, safeguard and dispatch, just become a problem demanding prompt solution.
For a large amount of and also the task of complexity, the configuration of vertical-lift computer can not meet the service requirement of task.Horizontal extension is the best approach of dealing with problems.Following problem is there is in existing Dispatching System:
1. can not, according to machine combination property reasonable distribution task, be all competition or equivalent allocating task.
2. can not accomplish real-time control task.After a lot of Frame Design is all task matching to machine, in its operational process, can not revise to it look into etc. operation.
3. can not automatic monitoring whole system, the running status of monitor task and the task bearing capacity of whole system.
Chinese invention patent CN 102387208A discloses a kind of distributed task dispatching method, comprises the following steps: task list distributor sends task list to multiple task engine; According to the task of the correspondence in task list and dependence table, multiple task engine determines whether self is initial task machine respectively; Initial task machine performs the task corresponding to initial task machine according to task list; After the tasks carrying of correspondence completes, initial task machine performs corresponding task according to the subsequent tasks machine of dependence table notice initial task machine.
Summary of the invention
The technical issues that need to address of the present invention provide a kind of can reasonable distribution task and the high method for scheduling task of system call reliability.
For solving above-mentioned technical matters, a kind of method for scheduling task based on distributed memory cluster of the present invention, comprises the following steps:
The task of establishment is joined in task queue, waits to be loaded; A task in each loading tasks queue is in distributed memory; Understood service condition and the utilization rate of task engine by monitoring module, understand the implementation status of every task additionally by monitoring module; By the task matching that loads at every turn in the less task engine of utilization rate.
Further, by execution status of task and tasks carrying parameter in operational module amendment distributed memory.
Further, when monitoring module finds to occur task suspension or deletion action in certain tasks carrying process, this task is made in time and is operated the response conformed to, and operational module revises this execution status of task and tasks carrying parameter.
Further, the maximum tasks carrying number of described each task engine can manual configuration being saved in distributed memory.
Further, the service condition of described task engine comprises for judging the heartbeat whether machine normally runs.
Further, after monitoring module detects that certain task engine heartbeat stops, the task that this task engine is assigned to moves to other task engines and continues to run.
Further, time in a task to distributed memory in each loading tasks queue, judge whether this task loaded, if repeat to load, then ignore this task; If this task does not load, then start to dispatch this task.
Adopt after said method, the present invention can be extending transversely and reduce, the not normal operation of influential system, the executing state of timely control task and amendment task correlation parameter dynamically.Reasonable distribution task, avoids each machine to overload and executes the task.Complete system monitoring interface is provided, understands the operation conditions of whole system in time.This system call reliability is high, and system stability is strong.
Accompanying drawing explanation
Below in conjunction with the drawings and specific embodiments to being originally described in further detail.
Fig. 1 is the process flow diagram of a kind of method for scheduling task based on distributed memory cluster of the present invention.
Fig. 2 is the structured flowchart of task scheduling system of the present invention.
Embodiment
As shown in Figure 2, system of the present invention comprises some task engines, distributed memory framework, monitoring module and operational module.The present invention utilizes distributed memory framework for intermediary, controls the resource of each machine.Reasonably be assigned to each machine to the task queue entering into distributed memory to go to perform, and monitored by distributed memory and control each task.Wherein, every platform task engine connects distributed memory frame system respectively, following parameter is saved in distributed memory.
A) performance index (task maximum actual figure) that machine is comprehensive, manual configuration.
B) heartbeat: be used for judging whether machine normally runs, after this machine stops heartbeat, the task of this machine moves on to other machines and continues to run.
C) task status, can be used for and application contacts.Such as: when task status is " deletion ", then this task out of service and delete this task in distributed system.
D) task basic parameter, can be used for and application contacts.Such as: job start time etc.
In addition, distributed memory framework is the core of whole scheduler task, and communication, clusters of machines between each machine are communicated with application.Monitoring and operation task has been come by corresponding monitoring module and operational module.
As shown in Figure 1, a kind of method for scheduling task based on distributed memory cluster of the present invention, is realized by following steps.
Step S101: join in task queue by the task of establishment, waits to be loaded.
Step S102 a: task in each loading tasks queue, in distributed memory, carrys out scheduler task by distributed memory.During a task in each loading tasks queue, in order to prevent the situation repeating to load, first judge whether this task loaded; If repeat to load, then ignore this task; If this task does not load, then start to dispatch this task.
Step S103: service condition and the utilization rate of being understood task engine by monitoring module, understands the implementation status of every task additionally by monitoring module.The implementation status of every task and the service condition of task engine are all stored in distributed memory, and monitoring module is by the service condition of distributed memory monitor task machine and the implementation status of every task.Understand the service condition of task engine and utilization rate be for the ease of distributed memory by the task matching that newly loads in the less task engine of utilization rate.Wherein whether the service condition of task engine also comprises task engine and can normally run, and adopts this state of heartbeat to represent the normal operation of task engine in present embodiment.After monitoring module detects that certain task engine heartbeat stops, first distributed memory can recover this task, then task is reentered in task queue, and this task is moved to other task engines continuation operation.In addition, monitoring module also understands the implementation status of every task by distributed memory; When by operational module amendment execution status of task and tasks carrying correlation parameter; comprise the operation such as task suspension or deletion; after this lower operational order is sent to distributed memory by operational module, distributed memory by task engine this task out of service or will delete this task.
Step S104: distributed memory obtains service condition and the utilization rate of each task engine by monitoring module, by the task matching that newly loads in the less task engine of utilization rate, reasonable distribution task, avoids each machine to overload and executes the task.
Although the foregoing describe the specific embodiment of the present invention; but those skilled in the art are to be understood that; these only illustrate; various changes or modifications can be made to present embodiment; and not deviating from principle and the essence of invention, protection scope of the present invention is only defined by the appended claims.
Claims (7)
1. based on a method for scheduling task for distributed memory cluster, it is characterized in that, comprise the following steps:
The task of establishment is joined in task queue, waits to be loaded;
A task in each loading tasks queue is in distributed memory;
Understood service condition and the utilization rate of task engine by monitoring module, understand the implementation status of every task additionally by monitoring module;
By the task matching that loads at every turn in the less task engine of utilization rate.
2. according to a kind of method for scheduling task based on distributed memory cluster according to claim 1, it is characterized in that: by execution status of task and tasks carrying parameter in operational module amendment distributed memory.
3. according to a kind of method for scheduling task based on distributed memory cluster according to claim 2; it is characterized in that: when monitoring module finds to occur task suspension or deletion action in certain tasks carrying process; this task is made in time and is operated the response conformed to, and operational module revises this execution status of task and tasks carrying parameter.
4. according to a kind of method for scheduling task based on distributed memory cluster according to claim 1, it is characterized in that: the maximum tasks carrying number of described each task engine can manual configuration being saved in distributed memory.
5. according to a kind of method for scheduling task based on distributed memory cluster according to claim 1, it is characterized in that: the service condition of described task engine comprises for judging the heartbeat whether machine normally runs.
6. according to a kind of method for scheduling task based on distributed memory cluster according to claim 5, it is characterized in that: after monitoring module detects that certain task engine heartbeat stops, the task that this task engine is assigned to moves to other task engines and continues to run.
7. according to a kind of method for scheduling task based on distributed memory cluster according to claim 1, it is characterized in that: time in a task to distributed memory in each loading tasks queue, judge whether this task loaded, if repeat to load, then ignore this task; If this task does not load, then start to dispatch this task.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510047784.8A CN104572286A (en) | 2015-01-30 | 2015-01-30 | Task scheduling method based on distributed memory clusters |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510047784.8A CN104572286A (en) | 2015-01-30 | 2015-01-30 | Task scheduling method based on distributed memory clusters |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN104572286A true CN104572286A (en) | 2015-04-29 |
Family
ID=53088433
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510047784.8A Pending CN104572286A (en) | 2015-01-30 | 2015-01-30 | Task scheduling method based on distributed memory clusters |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104572286A (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107483601A (en) * | 2017-08-28 | 2017-12-15 | 郑州云海信息技术有限公司 | A kind of implementation method and execution system of distributed timing task |
| CN107728643A (en) * | 2017-11-10 | 2018-02-23 | 西安电子科技大学 | A kind of unmanned aerial vehicle group distributed task dispatching method under dynamic environment |
| CN108182108A (en) * | 2017-12-19 | 2018-06-19 | 山东浪潮商用系统有限公司 | A kind of timed task cluster and its execution method |
| CN108446176A (en) * | 2018-02-07 | 2018-08-24 | 平安普惠企业管理有限公司 | A kind of method for allocating tasks, computer readable storage medium and terminal device |
| CN108733470A (en) * | 2017-04-25 | 2018-11-02 | 深圳市优网科技有限公司 | A kind of distributed task dispatching system and method |
| CN110888925A (en) * | 2019-10-11 | 2020-03-17 | 广州大气候农业科技有限公司 | Data loading and distributing method and device and storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20120297393A1 (en) * | 2009-12-30 | 2012-11-22 | Zte Corporation | Data Collecting Method, Data Collecting Apparatus and Network Management Device |
| CN103092698A (en) * | 2012-12-24 | 2013-05-08 | 中国科学院深圳先进技术研究院 | System and method of cloud computing application automatic deployment |
| CN103246592A (en) * | 2013-05-13 | 2013-08-14 | 北京搜狐新媒体信息技术有限公司 | Monitoring acquisition system and method |
| CN103324539A (en) * | 2013-06-24 | 2013-09-25 | 浪潮电子信息产业股份有限公司 | Job scheduling management system and method |
-
2015
- 2015-01-30 CN CN201510047784.8A patent/CN104572286A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20120297393A1 (en) * | 2009-12-30 | 2012-11-22 | Zte Corporation | Data Collecting Method, Data Collecting Apparatus and Network Management Device |
| CN103092698A (en) * | 2012-12-24 | 2013-05-08 | 中国科学院深圳先进技术研究院 | System and method of cloud computing application automatic deployment |
| CN103246592A (en) * | 2013-05-13 | 2013-08-14 | 北京搜狐新媒体信息技术有限公司 | Monitoring acquisition system and method |
| CN103324539A (en) * | 2013-06-24 | 2013-09-25 | 浪潮电子信息产业股份有限公司 | Job scheduling management system and method |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108733470A (en) * | 2017-04-25 | 2018-11-02 | 深圳市优网科技有限公司 | A kind of distributed task dispatching system and method |
| CN107483601A (en) * | 2017-08-28 | 2017-12-15 | 郑州云海信息技术有限公司 | A kind of implementation method and execution system of distributed timing task |
| CN107728643A (en) * | 2017-11-10 | 2018-02-23 | 西安电子科技大学 | A kind of unmanned aerial vehicle group distributed task dispatching method under dynamic environment |
| CN107728643B (en) * | 2017-11-10 | 2019-10-25 | 西安电子科技大学 | A kind of unmanned aerial vehicle group distributed task dispatching method under dynamic environment |
| CN108182108A (en) * | 2017-12-19 | 2018-06-19 | 山东浪潮商用系统有限公司 | A kind of timed task cluster and its execution method |
| CN108446176A (en) * | 2018-02-07 | 2018-08-24 | 平安普惠企业管理有限公司 | A kind of method for allocating tasks, computer readable storage medium and terminal device |
| CN108446176B (en) * | 2018-02-07 | 2020-07-03 | 平安普惠企业管理有限公司 | Task allocation method, computer readable storage medium and terminal device |
| CN110888925A (en) * | 2019-10-11 | 2020-03-17 | 广州大气候农业科技有限公司 | Data loading and distributing method and device and storage medium |
| CN110888925B (en) * | 2019-10-11 | 2022-06-17 | 广州大气候农业科技有限公司 | Data loading and distributing method and device and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104572286A (en) | Task scheduling method based on distributed memory clusters | |
| CN107018175B (en) | Scheduling method and device of mobile cloud computing platform | |
| CN111694633A (en) | Cluster node load balancing method and device and computer storage medium | |
| CN104468407A (en) | Method and device for performing service platform resource elastic allocation | |
| CN111209110B (en) | Task scheduling management method, system and storage medium for realizing load balancing | |
| CN104954411A (en) | Method for sharing network resource by distributed system, terminal thereof and system thereof | |
| CN103279331A (en) | Multi-task concurrent processing method and device for Android system | |
| CN111258746B (en) | Resource allocation method and service equipment | |
| CN104572279B (en) | A kind of virtual machine dynamic dispatching method of supporting node binding | |
| CN102117225A (en) | Industrial automatic multi-point cluster system and task management method thereof | |
| CN103354990A (en) | System and method for processing virtual machine in cloud platform | |
| CN103019849B (en) | Virtual machine management method under cloud computing environment | |
| CN104410511A (en) | Server management method and system | |
| CN104219290B (en) | A kind of multimode cloud application elasticity collocation method | |
| CN103677959A (en) | Virtual machine cluster migration method and system based on multicast | |
| CN113835834A (en) | K8S container cluster-based computing node capacity expansion method and system | |
| CN105183563A (en) | CPU resource dynamic self-configuration method facing mission critical computer | |
| CN111541646A (en) | Method for enhancing security service access capability of cipher machine | |
| CN110401939A (en) | A kind of low-power consumption bluetooth controller link layer device | |
| CN112860391B (en) | Dynamic cluster rendering resource management system and method | |
| CN115712572A (en) | Task testing method and device, storage medium and electronic device | |
| CN103973811A (en) | High-availability cluster management method capable of conducting dynamic migration | |
| CN104283943A (en) | Communication optimizing method for cluster server | |
| CN113822485A (en) | Power distribution network scheduling task optimization method and system | |
| CN114579298A (en) | Resource management method, resource manager, and computer-readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20150429 |
|
| RJ01 | Rejection of invention patent application after publication |