CN108287764A - Distributed task dispatching method and its system, storage medium, electronic equipment - Google Patents
Distributed task dispatching method and its system, storage medium, electronic equipment Download PDFInfo
- Publication number
- CN108287764A CN108287764A CN201810094656.2A CN201810094656A CN108287764A CN 108287764 A CN108287764 A CN 108287764A CN 201810094656 A CN201810094656 A CN 201810094656A CN 108287764 A CN108287764 A CN 108287764A
- Authority
- CN
- China
- Prior art keywords
- task
- administrative center
- producer
- consumer
- center
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5083—Techniques for rebalancing the load in a distributed system
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5083—Techniques for rebalancing the load in a distributed system
- G06F9/5088—Techniques for rebalancing the load in a distributed system involving task migration
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Transfer Between Computers (AREA)
Abstract
Present invention is disclosed distributed task dispatching method and its system, storage medium, electronic equipments.The distributed task dispatching system includes an at least task producer, at least a task consumer, registration center and administrative center, and the distributed task dispatching method includes the following steps：The task producer and task consumer register to registration center respectively；The log-on message of all task producers and task consumer are sent to administrative center by registration center, and administrative center caches the information of the task producer and task consumer；Task list is generated by a task producer, and task list is pushed into administrative center；Administrative center receives and lands task list；Task allocation rule is arranged in administrative center, and distributes each task according to task allocation rule in real time；Task consumer receives the task of administrative center's distribution；After task consumer's processing task, the result of feedback task processing is to administrative center.
Description
Technical field
The present invention relates to a kind of distributed task dispatching method and its system, storage medium, electronic equipments.
Background technology
Currently, more and more places need to touch using timing in the task scheduling system of many Large-Scale Interconnected nets enterprise
The function of hair task, for example：Periodic refreshing caches, timing migrating data etc..Since the workload of task processing before is little,
The time of task processing is short, it is therefore possible to use distributed lock or the mode of Active-Standby elections realize task tune
Only have individual server to handle task in degree system.But it is more and more with task, data are increasing, existing task
The task treating capacity of scheduling system also increasingly increases, and the time of task processing is also increasingly longer, cannot be satisfied the need of data processing
It asks.
Invention content
For the defects in the prior art, the object of the present invention is to provide a kind of distributed task dispatching method and its it is
System, storage medium, electronic equipment can reduce the handling duration of task, improve the efficiency and success rate of task, and can also
Real time monitoring task, the scheduling of management role.The utilization rate of server in raising system, ensures the load balancing of server.
A kind of distributed task dispatching method, the distributed task dispatching method are provided according to an aspect of the present invention
For the task scheduling of distributed task dispatching system, the distributed task dispatching system include an at least task producer,
At least a task consumer, registration center and administrative center, the distributed task dispatching method include the following steps：S10：
The task producer and the task consumer register to the registration center respectively；S20：The registration center is by institute
There are the log-on message of the task producer and the task consumer to be sent to the administrative center, administrative center's caching
The information of the task producer and the task consumer；S30：Task list is generated by the task producer, and will
The task list pushes to the administrative center；S40：The administrative center receives and lands the task list；S50：Institute
Administrative center's setting task allocation rule is stated, and distributes each task according to the task allocation rule in real time；S60：Described
Business consumer receives the task of administrative center's distribution；S70：After the task consumer processing task, feedback task processing
Result to the administrative center.
Preferably, just switched over main-standby mode between multiple task producers, synchronization only there are one
The task producer is working.
Preferably, the step S30 further includes following steps：S301：It is pending that the task producer generates at least one
Task；S302：The waiting task is split as multiple subtasks, multiple subtasks form a task list；S303：
The task list is pushed into the administrative center；S304：The message of cancel register is sent out to the registration center.
Preferably, the registration center receives the cancel register that the task producer or the task consumer send out
After message, the message of the cancel register is forwarded to the administrative center, the administrative center receives disappearing for the cancel register
After being waited for one second after breath, another described task producer is selected to receive task list or select another task consumer
Distribution task.
Preferably, the administrative center includes Rest interfaces, and the administrative center passes through described in the Rest interfaces
Task list that the task producer sends out, receive the task producer feedback task processing as a result, and sending out startup
Or the instruction stopped.
Preferably, in the step S70, task consumer feeds back the result that task is handled by ACK control characters and sends out
It send to the registration center, and changes the current task state information of the administrative center by the registration center.
Preferably, the registration center is Zookeeper.
According to another aspect of the present invention, a kind of distributed task dispatching system, the distributed task scheduling tune are also provided
Degree system includes an at least task producer, at least a task consumer, registration center and administrative center；Wherein, described
The business producer pushes to the task list in the management for forming task list after generating task and fractionation task
The heart；The task consumer is used to receive the task of administrative center's distribution, and after processing task, feedback task processing
As a result to the administrative center；The registration center is used for the registration of all the task producers and the task consumer
Information is sent to the administrative center, is registered to the task producer and the task consumer；The administrative center
For caching the information of the task producer and the task consumer, reception and landing the task list, setting task
Allocation rule, in real time according to the task allocation rule distribute each task, start or stop the dispatching cycle more corrected students' papers, supervise
The progress of control task.
According to a further aspect of the invention, a kind of computer readable storage medium is also provided, computer is stored thereon with
The step of program, which realizes above-mentioned distributed task dispatching method when being executed by processor.
According to a further aspect of the invention, a kind of electronic equipment is also provided, including：Processor；And memory, it is used for
Store the executable instruction of the processor；Wherein, the processor is configured to execute via the executable instruction is executed
The step of above-mentioned distributed task dispatching method.
Compared with the prior art, distributed task dispatching method and system provided in an embodiment of the present invention, storage medium,
It is noted respectively to registration center by the task producer and task consumer in distributed task dispatching method in electronic equipment
Volume generates task list by a task producer, and task list is pushed to administrative center；Administrative center, which receives and lands, to appoint
It is engaged in that task allocation rule is arranged after list, and distributes each task according to task allocation rule in real time；Task consumer's reception pipe
Reason center distribution task and processing after, feedback task processing result to administrative center mode.In turn, at least have as follows
Advantageous effect：
1, the handling duration of task is reduced；
2, by the fractionation to task, the processing energy of the Servers-all of distributed task dispatching system is preferably utilized
Power ensures the load balancing of server, improves the efficiency and success rate of task；
3, can be by administrative center, the preferably scheduling of monitoring and management role；
4, it can be retried using mistake, the modes such as failure transfer improve the coefficient of stabilization of task processing.
Description of the drawings
Upon reading the detailed description of non-limiting embodiments with reference to the following drawings, other feature of the invention,
Objects and advantages will become more apparent upon：
Fig. 1 is the flow chart of the distributed task dispatching method of one embodiment of the present of invention；
Fig. 2 generates task row for a task producer in the distributed task dispatching method of one embodiment of the present of invention
The flow chart of table and each step of push；
Fig. 3 is the module composition schematic diagram of the distributed task dispatching system of one embodiment of the present of invention；
Fig. 4 is that the present invention discloses a kind of computer readable storage medium schematic diagram in exemplary embodiment；And
Fig. 5 is that the present invention discloses a kind of electronic equipment schematic diagram in exemplary embodiment.
Specific implementation mode
Example embodiment is described more fully with reference to the drawings.However, example embodiment can be with a variety of shapes
Formula is implemented, and is not understood as limited to embodiment set forth herein；On the contrary, thesing embodiments are provided so that the present invention will
Fully and completely, and by the design of example embodiment comprehensively it is communicated to those skilled in the art.It is identical attached in figure
Icon note indicates same or similar structure, thus will omit repetition thereof.
Described feature, structure or characteristic can be incorporated in one or more embodiments in any suitable manner
In.In the following description, many details are provided to fully understand embodiments of the present invention to provide.However,
One of ordinary skill in the art would recognize that without one or more in specific detail, or using other methods, constituent element, material
Material etc., can also put into practice technical scheme of the present invention.In some cases, be not shown in detail or describe known features, material or
Person operates to avoid the fuzzy present invention.
In addition, attached drawing is only the schematic illustrations of the disclosure, it is not necessarily drawn to scale.Identical attached drawing mark in figure
Note indicates same or similar part, thus will omit repetition thereof.Some block diagrams shown in attached drawing are work(
Energy entity, not necessarily must be corresponding with physically or logically independent entity.Software form may be used to realize these work(
Energy entity, or these functional entitys are realized in one or more hardware modules or integrated circuit, or at heterogeneous networks and/or place
These functional entitys are realized in reason device device and/or microcontroller device.
Purport design according to the present invention, a kind of distributed task dispatching method of the invention are used for distributed task scheduling tune
The task scheduling of degree system, the distributed task dispatching system include an at least task producer, an at least task consumer,
Registration center and administrative center, the distributed task dispatching method include the following steps：The task producer and described
Task consumer registers to the registration center respectively；The registration center is by all task producers and described
The log-on message of business consumer is sent to the administrative center, and the administrative center caches the task producer and the task
The information of consumer；Task list is generated by the task producer, and the task list is pushed in the management
The heart；The administrative center receives and lands the task list；Task allocation rule, and real-time basis is arranged in the administrative center
The task allocation rule distributes each task；The task consumer receives the task of administrative center's distribution；Described
After consumer's processing task of being engaged in, the result of feedback task processing is to the administrative center.
The technology contents of the present invention are described further with reference to the accompanying drawings and examples.
Fig. 1 is referred to, it illustrates the flow charts of the distributed task dispatching method of one embodiment of the present of invention.Specifically
For, distributed task dispatching method of the invention is mainly used for the task scheduling of distributed task dispatching system, wherein described
Distributed task dispatching system includes an at least task producer, at least a task consumer, registration center and administrative center.
Specifically, as shown in Figure 1, in an embodiment of the present invention, which includes the following steps：
Step S10：The task producer and the task consumer register to the registration center respectively.At this
In inventive embodiments, the task producer as the submitter of task and fractionation person, reasonably split task (fractionation task refer to by
Each task is split as multiple subtasks), to the load balancing of more advantageous task consumer cluster.Task consumer's conduct
Handle the subtask of operation.The registration center is Zookeeper or other databases.Wherein, above-mentioned task can be
Periodic refreshing is data cached, timing migrating data etc..Specifically, the task producer start when (submit task and
When fractionation task) it needs to be registered to registration center.Similarly, task consumer start when (i.e. receive and
When processing task) it is also required to be registered to registration center.
Step S20：The log-on message of all task producers and the task consumer are sent out by the registration center
It send to the administrative center, the administrative center caches the information of the task producer and the task consumer.
Step S30：Task list is generated by the task producer, and the task list is pushed into the management
Center.In embodiments of the present invention, it is just switched over main-standby mode between multiple task producers, is ensured together with this
One moment only had the task producer and is working.Each task producer is by registration center's connection management center, often
When secondary connection management center, the task producer registers to registration center, and sends out and take to registration center when offline
Disappear the message of registration.Specifically, Fig. 2 is referred to, it illustrates the distributed task dispatching sides of one embodiment of the present of invention
A task producer generates task list and the flow chart of each step of push in method.As shown in Fig. 2, the step S30
Further include following steps：
Step S301：The task producer generates an at least waiting task.Wherein, the task producer can use
Quartz expression formulas generate task (triggering operation).
Step S302：The waiting task is split as multiple subtasks, multiple subtasks form task row
Table.Specifically, it during the waiting task is split as multiple subtasks (generating task fragment), needs to close
Reason is split, if the too small network I/O that can influence cluster in subtask.
Step S303：The task list is pushed into the administrative center.Wherein, the task producer forms subtask
Task list administrative center is submitted to by Rest modes.Specifically, administrative center includes Rest interfaces, and administrative center is logical
Cross the task list that the Rest interface task producers send out.
Step S304：The message of cancel register is sent out to the registration center.
Step S40：The administrative center receives and lands the task list.Specifically, administrative center receives and falls
Ground task list preserves the subtask in the task list, in turn, assign the task to task consumer, also, further
Receive the task of other tasks producer transmission.In a preferred embodiment of the invention, registration center receives the task production
After the message for the cancel register that person sends out, the message of the cancel register is forwarded to administrative center, administrative center receives this and takes
After being waited for one second after the message of registration that disappears, selects another task producer to receive task list, network fluctuation etc. is prevented with this
Problem causes the task producer false offline.
Step S50：Task allocation rule is arranged in the administrative center, and is distributed respectively according to the task allocation rule in real time
Item task.Specifically, the administrative center is mainly used for the task of reception task producer transmission, setting task distribution rule
Then, each task, startup and the dispatching cycle for stopping task, changing task are dynamically distributed according to task allocation rule in real time,
The progress etc. of monitor task can also cache all task producers and task consumer information, and perceive task production
Person and the upper down status of task consumer.Further, administrative center can also provide back-stage management interface, manage and check and appoint
The state and process of business.
More specifically, in this step, which can make all of all distributed task dispatching systems
One kind in the modes such as line server all executes, only a server executes or specified portions server executes.Its
In, task allocation rule is can be realized in a manner of simple character expression or regular expression etc..Task distribution rule
Principle then is mainly distributed according to the processing capacity of Servers-all, if the processing of a certain server is faster, is distributed
Task is more.
Step S60：The task consumer receives the task of administrative center's distribution.Specifically, it is produced with task
Person similarly, also by registration center's connection management center, and when each connection management center, consume task consumer by task
Person registers to registration center, and sends out the message of cancel register to registration center when offline.In turn, registration center
After the message for the cancel register that reception task consumer sends out, the message of the cancel register is forwarded to administrative center.Preferably,
Administrative center receives wait for one second after the message of the cancel register after, selection selection another task consumer distribute task,
The problems such as preventing network fluctuation with this, causes task consumer false offline.
Step S70：After the task consumer processing task, the result of feedback task processing is to the administrative center.Tool
For body, in embodiments of the present invention, the result that task consumer can feed back task processing by ACK control characters is sent to
Registration center, and pass through the current task state information (task node) of modification administrative center of registration center.Further, such as
Upper described, in a preferred embodiment of the invention, administrative center includes Rest interfaces.Administrative center not only may be used by Rest interfaces
To realize the effect of task list that the above-mentioned reception task producer sends out, at the also acting as reception task producer feedback of the task
The result of reason, sends out the effects that instruction of startup or stopping at query task progress, and it will not be described here.Task consumer can
It to be submitted to administrative center by asynchronous Rest interfaces with Rate feedback systems, and is sent using delay, wherein for partly disappearing
Breath, which is lost, to be ignored, and need not be reissued.
In above-mentioned embodiment shown in FIG. 1, in distributed task dispatching method of the invention by the task producer and
Task consumer registers to registration center respectively, generates task list by a task producer, and task list is pushed
To administrative center；Administrative center receives and task allocation rule is arranged after landing task list, and is distributed advise according to task in real time
Then distribute each task；After task consumer receives the task of administrative center's distribution and processing, the result of feedback task processing is extremely
The mode of administrative center.In turn, it at least has the advantages that：
1, the handling duration of task is reduced；
2, by the fractionation to task, the processing energy of the Servers-all of distributed task dispatching system is preferably utilized
Power ensures the load balancing of server, improves the efficiency and success rate of task；
3, can be by administrative center, the preferably scheduling of monitoring and management role；
4, it can be retried using mistake, the modes such as failure transfer improve the coefficient of stabilization of task processing.
Further, the present invention also provides a kind of distributed task dispatching systems, for realizing above-mentioned distributed task scheduling
Dispatching method.Specifically, Fig. 3 is referred to, it illustrates the moulds of the distributed task dispatching system of one embodiment of invention
Block constitutes schematic diagram.The distributed task dispatching system includes at least a task producer 2, at least a task consumer 3, note
Volume center 4 and administrative center 1.
Specifically, the task producer 2 forms task list after task, fractionation task for generating, and by task list
Push to administrative center 1.
Task consumer 3 is used to receive the task of the distribution of administrative center 1, and after processing task, feedback task processing
As a result to administrative center 1.
Registration center 4 be used for by the log-on message of all task producers 2 and task consumer 3 be sent to administrative center 1,
The task producer 2 and task consumer 3 are registered.
Administrative center 1 is used to cache the information of the task producer 2 and task consumer 3, reception and lands task list, sets
Set task allocation rule, in real time according to task allocation rule distribute each task, start or stop the dispatching cycle more corrected students' papers,
The progress of monitor task.
In an exemplary embodiment of the disclosure, a kind of computer readable storage medium is additionally provided, meter is stored thereon with
Distributed task scheduling tune described in any one above-mentioned embodiment may be implemented in calculation machine program, the program when being executed by such as processor
Each step that control system and telephone traffic system execute in degree method.In some possible embodiments, of the invention each
Aspect is also implemented as a kind of form of program product comprising program code, when described program product on the terminal device
When operation, said program code is for making the terminal device execute the above-mentioned distributed task dispatching shown in FIG. 1 of this specification
Described in method part according to various illustrative embodiments of the invention the step of.
Refering to what is shown in Fig. 4, describing the program product for realizing the above method according to the embodiment of the present invention
300, portable compact disc read only memory (CD-ROM) may be used and include program code, and can in terminal device,
Such as it is run on PC.However, the program product of the present invention is without being limited thereto, in this document, readable storage medium storing program for executing can be with
To be any include or the tangible medium of storage program, the program can be commanded execution system, device either device use or
It is in connection.
The arbitrary combination of one or more readable mediums may be used in described program product.Readable medium can be readable letter
Number medium or readable storage medium storing program for executing.Readable storage medium storing program for executing for example can be but be not limited to electricity, magnetic, optical, electromagnetic, infrared ray or
System, device or the device of semiconductor, or the arbitrary above combination.The more specific example of readable storage medium storing program for executing is (non exhaustive
List) include：It is electrical connection, portable disc, hard disk, random access memory (RAM) with one or more conducting wires, read-only
Memory (ROM), erasable programmable read only memory (EPROM or flash memory), optical fiber, portable compact disc read only memory
(CD-ROM), light storage device, magnetic memory device or above-mentioned any appropriate combination.
The computer readable storage medium may include the data letter propagated in a base band or as a carrier wave part
Number, wherein carrying readable program code.Diversified forms, including but not limited to electromagnetism may be used in the data-signal of this propagation
Signal, optical signal or above-mentioned any appropriate combination.Readable storage medium storing program for executing can also be any other than readable storage medium storing program for executing
Readable medium, which can send, propagate either transmission for being used by instruction execution system, device or device or
Person's program in connection.The program code for including on readable storage medium storing program for executing can transmit with any suitable medium, packet
Include but be not limited to wireless, wired, optical cable, RF etc. or above-mentioned any appropriate combination.
It can be write with any combination of one or more programming languages for executing the program that operates of the present invention
Code, described program design language include object oriented program language-Java, C++ etc., further include conventional
Procedural programming language-such as " C " language or similar programming language.Program code can be fully in user
It executes on computing device, partly execute on a user device, being executed as an independent software package, partly in user's calculating
Upper side point is executed or is executed in remote computing device or server completely on a remote computing.It is being related to far
In the situation of journey computing device, remote computing device can pass through the network of any kind, including LAN (LAN) or wide area network
(WAN), it is connected to user calculating equipment, or, it may be connected to external computing device (such as utilize ISP
To be connected by internet).
In an exemplary embodiment of the disclosure, a kind of electronic equipment is also provided, which may include processor,
And the memory of the executable instruction for storing the processor.Wherein, the processor is configured to via described in execution
Executable instruction executes control system and traffic system in distributed task dispatching method described in any one above-mentioned embodiment
Each step that system executes.
Person of ordinary skill in the field it is understood that various aspects of the invention can be implemented as system, method or
Program product.Therefore, various aspects of the invention can be embodied in the following forms, i.e.,：It is complete hardware embodiment, complete
The embodiment combined in terms of full Software Implementation (including firmware, microcode etc.) or hardware and software, can unite here
Referred to as circuit, " module " or " system ".
The electronic equipment 600 of this embodiment according to the present invention is described referring to Fig. 5.The electronics that Fig. 5 is shown
Equipment 600 is only an example, should not bring any restrictions to the function and use scope of the embodiment of the present invention.
As shown in figure 5, electronic equipment 600 is showed in the form of universal computing device.The component of electronic equipment 600 can wrap
It includes but is not limited to：At least one processing unit 610, at least one storage unit 620, (including the storage of connection different system component
Unit 620 and processing unit 610) bus 630, display unit 640 etc..
Wherein, the storage unit has program stored therein code, and said program code can be held by the processing unit 610
Row so that the processing unit 610 execute described in this specification above-mentioned distributed task dispatching method part according to this hair
The step of bright various illustrative embodiments.For example, the processing unit 610 can execute as shown in Figure 1 by control system
Each step that system and telephone traffic system are completed.
The storage unit 620 may include the readable medium of volatile memory cell form, such as random access memory
Unit (RAM) 6201 and/or cache memory unit 6202 can further include read-only memory unit (ROM) 6203.
The storage unit 620 can also include program/practicality work with one group of (at least one) program module 6205
Tool 6204, such program module 6205 include but not limited to：Operating system, one or more application program, other programs
Module and program data may include the realization of network environment in each or certain combination in these examples.
Bus 630 can be to indicate one or more in a few class bus structures, including storage unit bus or storage
Cell controller, peripheral bus, graphics acceleration port, processing unit use the arbitrary bus structures in a variety of bus structures
Local bus.
Electronic equipment 600 can also be with one or more external equipments 700 (such as keyboard, sensing equipment, bluetooth equipment
Deng) communication, can also be enabled a user to one or more equipment interact with the electronic equipment 600 communicate, and/or with make
Any equipment that the electronic equipment 600 can be communicated with one or more of the other computing device (such as router, modulation /demodulation
Device etc.) communication.This communication can be carried out by input/output (I/O) interface 650.Also, electronic equipment 600 can be with
By network adapter 660 and one or more network (such as LAN (LAN), wide area network (WAN) and/or public network,
Such as internet) communication.Network adapter 660 can be communicated by bus 630 with other modules of electronic equipment 600.It should
Understand, although not shown in the drawings, other hardware and/or software module can be used in conjunction with electronic equipment 600, including but it is unlimited
In：Microcode, device driver, redundant processing unit, external disk drive array, RAID system, tape drive and number
According to backup storage system etc..
Through the above description of the embodiments, those skilled in the art is it can be readily appreciated that example described herein is implemented
Mode can also be realized by software realization in such a way that software is in conjunction with necessary hardware.Therefore, according to the disclosure
The technical solution of embodiment can be expressed in the form of software products, the software product can be stored in one it is non-volatile
Property storage medium (can be CD-ROM, USB flash disk, mobile hard disk etc.) in or network on, including some instructions are so that a calculating
Equipment (can be personal computer, server or network equipment etc.) executes the above-mentioned distribution according to disclosure embodiment
Formula method for scheduling task.
In conclusion distributed task dispatching method provided in an embodiment of the present invention and system, storage medium, electronics are set
It is registered respectively to registration center by the task producer and task consumer in standby, task row is generated by a task producer
Table, and task list is pushed into administrative center；Administrative center receives and task allocation rule is arranged after landing task list, and
In real time each task is distributed according to task allocation rule；After task consumer receives the task of administrative center's distribution and processing, instead
Feedback task processing result to administrative center mode.In turn, it at least has the advantages that：
1, the handling duration of task is reduced；
2, by the fractionation to task, the processing energy of the Servers-all of distributed task dispatching system is preferably utilized
Power ensures the load balancing of server, improves the efficiency and success rate of task；
3, can be by administrative center, the preferably scheduling of monitoring and management role；
4, it can be retried using mistake, the modes such as failure transfer improve the coefficient of stabilization of task processing.
Although the present invention is disclosed as above with alternative embodiment, it is not limited to the present invention.Belonging to the present invention
Those skilled in the art, without departing from the spirit and scope of the present invention, when various change and modification can be made.Therefore,
Protection scope of the present invention is subject to the range defined depending on claims.
Claims (10)
1. a kind of distributed task dispatching method, which is characterized in that the distributed task dispatching method is used for distributed task scheduling
The task scheduling of scheduling system, the distributed task dispatching system include an at least task producer, the consumption of an at least task
Person, registration center and administrative center, the distributed task dispatching method include the following steps：
S10：The task producer and the task consumer register to the registration center respectively；
S20：The log-on message of all task producers and the task consumer are sent to described by the registration center
Administrative center, the administrative center cache the information of the task producer and the task consumer；
S30：Task list is generated by the task producer, and the task list is pushed into the administrative center；
S40：The administrative center receives and lands the task list；
S50：Task allocation rule is arranged in the administrative center, and distributes each task according to the task allocation rule in real time；
S60：The task consumer receives the task of administrative center's distribution；
S70：After the task consumer processing task, the result of feedback task processing is to the administrative center.
2. distributed task dispatching method as described in claim 1, which is characterized in that between multiple task producers
It is switched over the standby mode of master-, synchronization is only working there are one the task producer.
3. distributed task dispatching method as described in claim 1, which is characterized in that the step S30 further includes walking as follows
Suddenly：
S301：The task producer generates an at least waiting task；
S302：The waiting task is split as multiple subtasks, multiple subtasks form a task list；
S303：The task list is pushed into the administrative center；
S304：The message of cancel register is sent out to the registration center.
4. distributed task dispatching method as claimed in claim 3, which is characterized in that the registration center receives the task
After the message for the cancel register that the producer or the task consumer send out, the message of the cancel register is forwarded to the management
Center, the administrative center receives wait for one second after the message of the cancel register after, select another task producer
It receives task list or another described task consumer is selected to distribute task.
5. distributed task dispatching method as described in claim 1, which is characterized in that the administrative center includes that Rest connects
Mouthful, the administrative center passes through the task list that the task producer sends out described in the Rest interfaces, the reception task
The task processing of producer's feedback as a result, and sending out the instruction of startup or stopping.
6. distributed task dispatching method as described in claim 1, which is characterized in that in the step S70, task consumption
The result that person feeds back task processing by ACK control characters is sent to the registration center, and is repaiied by the registration center
Change the current task state information of the administrative center.
7. such as distributed task dispatching method according to any one of claims 1 to 6, which is characterized in that the registration center
For Zookeeper.
8. a kind of distributed task dispatching system, which is characterized in that the distributed task dispatching system includes an at least task
The producer, at least a task consumer, registration center and administrative center；
Wherein, the task producer forms task list after task and fractionation task for generating, and by the task list
Push to the administrative center；
The task consumer is used to receive the task of administrative center's distribution, and after processing task, feedback task processing
Result to the administrative center；
The registration center is used to the log-on message of all task producers and the task consumer being sent to described
Administrative center registers the task producer and the task consumer；
The administrative center is used to cache described of information, reception and the landing of the task producer and the task consumer
Business list, according to the task allocation rule distributes each task in real time, starts or stops and more transform at setting task allocation rule
The dispatching cycle of industry, the progress of monitor task.
9. a kind of computer readable storage medium, is stored thereon with computer program, which is characterized in that the program is held by processor
The step of distributed task dispatching method described in any one of claim 1 to 7 is realized when row.
10. a kind of electronic equipment, which is characterized in that including：
Processor；And
Memory, the executable instruction for storing the processor；
Wherein, the processor is configured to come described in any one of perform claim requirement 1 to 7 via the execution executable instruction
Distributed task dispatching method the step of.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810094656.2A CN108287764A (en) | 2018-01-31 | 2018-01-31 | Distributed task dispatching method and its system, storage medium, electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810094656.2A CN108287764A (en) | 2018-01-31 | 2018-01-31 | Distributed task dispatching method and its system, storage medium, electronic equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108287764A true CN108287764A (en) | 2018-07-17 |
Family
ID=62836316
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810094656.2A Pending CN108287764A (en) | 2018-01-31 | 2018-01-31 | Distributed task dispatching method and its system, storage medium, electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108287764A (en) |
Cited By (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109508232A (en) * | 2018-11-26 | 2019-03-22 | 北京大米科技有限公司 | A kind of method, apparatus and electronic equipment of Portable Batch System |
| CN109729086A (en) * | 2018-12-28 | 2019-05-07 | 北京奇安信科技有限公司 | Policy management method, system, equipment and medium |
| CN110062038A (en) * | 2019-04-09 | 2019-07-26 | 网宿科技股份有限公司 | A kind of data transmission scheduling method and system |
| CN110275762A (en) * | 2019-04-30 | 2019-09-24 | 北京媒球信息科技有限公司 | Calculation method, server, mobile terminal and device |
| CN110968485A (en) * | 2019-11-28 | 2020-04-07 | 广州零世纪信息科技有限公司 | Operation and maintenance alarm processing method, device and system and storage medium |
| CN111026529A (en) * | 2019-11-26 | 2020-04-17 | 支付宝(杭州)信息技术有限公司 | Task stopping method and device for distributed task processing system |
| CN111147571A (en) * | 2019-12-24 | 2020-05-12 | 京东数字科技控股有限公司 | Distributed task scheduling method, device, system, storage medium and electronic equipment |
| CN111240760A (en) * | 2020-01-08 | 2020-06-05 | 江苏满运软件科技有限公司 | Application publishing method, system, storage medium and equipment based on registry |
| CN111382178A (en) * | 2020-03-10 | 2020-07-07 | 澳克多普有限公司 | Network task allocation method, system, equipment and storage medium |
| CN111694645A (en) * | 2020-05-26 | 2020-09-22 | 平安普惠企业管理有限公司 | Task processing method and related device in distributed task scheduling system |
| CN111752970A (en) * | 2020-06-26 | 2020-10-09 | 武汉众邦银行股份有限公司 | Distributed query service response method based on cache and storage medium |
| CN111767122A (en) * | 2019-05-17 | 2020-10-13 | 北京京东尚科信息技术有限公司 | Distributed task scheduling management method and device |
| CN111917814A (en) * | 2019-05-10 | 2020-11-10 | 北京百度网讯科技有限公司 | Data publishing method, data subscribing method, data publishing device, data subscribing system and readable storage medium |
| CN114020408A (en) * | 2021-10-28 | 2022-02-08 | 北京沃东天骏信息技术有限公司 | Task fragment configuration method and device, equipment and storage medium |
| CN114979100A (en) * | 2022-04-15 | 2022-08-30 | 深信服科技股份有限公司 | Cloud resource checking method and related device |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20100211954A1 (en) * | 2009-02-17 | 2010-08-19 | International Business Machines Corporation | Practical contention-free distributed weighted fair-share scheduler |
| CN103581225A (en) * | 2012-07-25 | 2014-02-12 | 中国银联股份有限公司 | Distributed system node processing task method |
| CN104461752A (en) * | 2014-11-21 | 2015-03-25 | 浙江宇视科技有限公司 | Two-level fault-tolerant multimedia distributed task processing method |
| CN104461747A (en) * | 2014-12-31 | 2015-03-25 | 合一网络技术(北京)有限公司 | Distributed type task scheduling system |
| CN106844018A (en) * | 2015-12-07 | 2017-06-13 | 阿里巴巴集团控股有限公司 | A kind of task processing method, apparatus and system |
| CN106909451A (en) * | 2017-02-28 | 2017-06-30 | 郑州云海信息技术有限公司 | A kind of distributed task dispatching system and method |
-
2018
- 2018-01-31 CN CN201810094656.2A patent/CN108287764A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20100211954A1 (en) * | 2009-02-17 | 2010-08-19 | International Business Machines Corporation | Practical contention-free distributed weighted fair-share scheduler |
| CN103581225A (en) * | 2012-07-25 | 2014-02-12 | 中国银联股份有限公司 | Distributed system node processing task method |
| CN104461752A (en) * | 2014-11-21 | 2015-03-25 | 浙江宇视科技有限公司 | Two-level fault-tolerant multimedia distributed task processing method |
| CN104461747A (en) * | 2014-12-31 | 2015-03-25 | 合一网络技术(北京)有限公司 | Distributed type task scheduling system |
| CN106844018A (en) * | 2015-12-07 | 2017-06-13 | 阿里巴巴集团控股有限公司 | A kind of task processing method, apparatus and system |
| CN106909451A (en) * | 2017-02-28 | 2017-06-30 | 郑州云海信息技术有限公司 | A kind of distributed task dispatching system and method |
Cited By (20)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109508232A (en) * | 2018-11-26 | 2019-03-22 | 北京大米科技有限公司 | A kind of method, apparatus and electronic equipment of Portable Batch System |
| CN109729086A (en) * | 2018-12-28 | 2019-05-07 | 北京奇安信科技有限公司 | Policy management method, system, equipment and medium |
| CN109729086B (en) * | 2018-12-28 | 2021-02-23 | 奇安信科技集团股份有限公司 | Policy management method, system, device, and medium |
| CN110062038A (en) * | 2019-04-09 | 2019-07-26 | 网宿科技股份有限公司 | A kind of data transmission scheduling method and system |
| CN110275762A (en) * | 2019-04-30 | 2019-09-24 | 北京媒球信息科技有限公司 | Calculation method, server, mobile terminal and device |
| CN111917814A (en) * | 2019-05-10 | 2020-11-10 | 北京百度网讯科技有限公司 | Data publishing method, data subscribing method, data publishing device, data subscribing system and readable storage medium |
| CN111767122A (en) * | 2019-05-17 | 2020-10-13 | 北京京东尚科信息技术有限公司 | Distributed task scheduling management method and device |
| CN111026529A (en) * | 2019-11-26 | 2020-04-17 | 支付宝(杭州)信息技术有限公司 | Task stopping method and device for distributed task processing system |
| CN110968485A (en) * | 2019-11-28 | 2020-04-07 | 广州零世纪信息科技有限公司 | Operation and maintenance alarm processing method, device and system and storage medium |
| CN111147571A (en) * | 2019-12-24 | 2020-05-12 | 京东数字科技控股有限公司 | Distributed task scheduling method, device, system, storage medium and electronic equipment |
| CN111147571B (en) * | 2019-12-24 | 2023-04-18 | 京东科技控股股份有限公司 | Distributed task scheduling method, device, system, storage medium and electronic equipment |
| CN111240760A (en) * | 2020-01-08 | 2020-06-05 | 江苏满运软件科技有限公司 | Application publishing method, system, storage medium and equipment based on registry |
| CN111240760B (en) * | 2020-01-08 | 2022-09-16 | 江苏满运软件科技有限公司 | Application publishing method, system, storage medium and equipment based on registry |
| CN111382178A (en) * | 2020-03-10 | 2020-07-07 | 澳克多普有限公司 | Network task allocation method, system, equipment and storage medium |
| CN111694645A (en) * | 2020-05-26 | 2020-09-22 | 平安普惠企业管理有限公司 | Task processing method and related device in distributed task scheduling system |
| CN111752970A (en) * | 2020-06-26 | 2020-10-09 | 武汉众邦银行股份有限公司 | Distributed query service response method based on cache and storage medium |
| CN111752970B (en) * | 2020-06-26 | 2024-01-30 | 武汉众邦银行股份有限公司 | Distributed query service response method based on cache and storage medium |
| CN114020408A (en) * | 2021-10-28 | 2022-02-08 | 北京沃东天骏信息技术有限公司 | Task fragment configuration method and device, equipment and storage medium |
| CN114979100A (en) * | 2022-04-15 | 2022-08-30 | 深信服科技股份有限公司 | Cloud resource checking method and related device |
| CN114979100B (en) * | 2022-04-15 | 2024-02-23 | 深信服科技股份有限公司 | Cloud resource inspection method and related device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108287764A (en) | Distributed task dispatching method and its system, storage medium, electronic equipment | |
| US7185046B2 (en) | Submitting jobs in a distributed computing environment | |
| EP4083802A1 (en) | Resource scheduling method and apparatus, device, storage medium, and program product | |
| US8695005B2 (en) | Model for hosting and invoking applications on virtual machines in a distributed computing environment | |
| CN111406250A (en) | Provisioning using prefetched data in a serverless computing environment | |
| CN109783472A (en) | Moving method, device, computer equipment and the storage medium of table data | |
| EP2739097B1 (en) | A mobile telecommunication device in a cellular-digital wireless telecommunication system and a method for operating thereof | |
| KR20130142961A (en) | Automatic application updates | |
| CN104506620A (en) | Extensible automatic computing service platform and construction method for same | |
| US11169846B2 (en) | System and method for managing tasks and task workload items between address spaces and logical partitions | |
| CN109634915A (en) | File dispositions method, Cloud Server, system and storage medium | |
| WO2014039311A1 (en) | Managing and revoking power allocated through bus interfaces | |
| CN102147746A (en) | Dynamic thread pool management | |
| KR20140110486A (en) | System for Resource Management in Mobile Cloud computing and Method thereof | |
| CN110458468A (en) | A kind of task processing method, device, electronic equipment and storage medium | |
| US20160019089A1 (en) | Method and system for scheduling computing | |
| CN108139931A (en) | It synchronizes to accelerate task subgraph by remapping | |
| CN107729559A (en) | Method, system, equipment and the storage medium of data base read-write asynchronous access | |
| US20150127182A1 (en) | Power management method, power management server, and office machine for managing electric power | |
| CN106534281A (en) | Data request responding method, apparatus and system | |
| US20200004560A1 (en) | Adaptive user-interface assembling and rendering | |
| CN109818810A (en) | A kind of access server connection optimization method, access server and communication system | |
| EP3486774B1 (en) | Method and device for dynamically assigning task and providing resources and system thereof | |
| CN108605017A (en) | Inquiry plan and operation perception communication buffer management | |
| CN107615711A (en) | System for event propagation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20180717 |
|
| RJ01 | Rejection of invention patent application after publication |