CN106681824A

CN106681824A - Distributed system scheduling method and device and electronic equipment

Info

Publication number: CN106681824A
Application number: CN201510761934.1A
Authority: CN
Inventors: 张广
Original assignee: Alibaba Group Holding Ltd
Current assignee: Cainiao Smart Logistics Holding Ltd
Priority date: 2015-11-10
Filing date: 2015-11-10
Publication date: 2017-05-17

Abstract

The invention discloses distributed system scheduling method and device and electronic equipment. The distributed system scheduling method includes that a management center of a distributed system acquires quantity and sequence number M of resource providing units of the distributed system; the management center of the distributed system acquires quantity and number of resource consumption units; the quantity of the resource consumption units is divided by the quantity of resource providing units to obtain the resource consumption unit average bearing quantity N born by the resource providing units; N or (N-1) or (N+1) resource consumption units are extracted from the resource consumption units according to the sequence of the number of the resource consumption units and are taken as the resource consumption units distributed to the resource providing units on the basis of the resource providing unit sequence number M. According to the technical scheme, problems of low task handling efficiency and deadlock possibility caused during task distribution carried out by means of distributed locks of the distributed system, and low speed of convergence in task distribution and long distribution time caused by circular preemption are solved.

Description

For the dispatching method of distributed system, device and electronic equipment

Technical field

The application is related to field of computer technology, and in particular to two kinds of dispatching methods for being used for distributed system；Phase Should be in said method, the application is related to two kinds of dispatching devices and two kinds of electronics for distributed system simultaneously Equipment.

Background technology

In today that Internet technology constantly develops, increasing data processing relies on higher computing capability, Diversified work cannot be processed in order to solve single treatment node, and higher and higher meter cannot be met The requirement of calculation ability, often processes the problems referred to above using distributed system at present.

In a distributed system, in order to allow collaborative work between each system, it is proposed that the concept of distributed lock, point Cloth lock is a kind of mode for controlling to be accessed between distributed system shared resource, gets the machine of distributed lock In thread, task can be continued executing with, other machines wait lock release.

As can be seen here, because distributed lock has the process for robbing lock-release lock, and the same time can only have one Individual machine gets lock, gets the machine of lock and could run, so machine is less efficient in process task, And machine is during using distributed lock, the machine if machine for getting lock is delayed suddenly, and the lock is also not It is released, then can causes deadlock.In addition, when seizing, if machine once seize it is unsuccessful, also Circulation is needed to seize, so task distribution convergence is relatively slow, the distribution time is long.

The content of the invention

The application provides two kinds of dispatching methods for being used for distributed system, to solve of the prior art above-mentioned ask Topic.The application is related to two kinds of dispatching devices and two kinds of electronic equipments for distributed system simultaneously.

This application provides a kind of dispatching method for distributed system, the scheduling for distributed system Method includes：

The resource of distributed system provides unit and obtains the distributed system by the administrative center of distributed system In resource Board Lot is provided, and the resource provides the money that unit sheet is provided in unit in all resources Source provides unit sequence M；

Resource consumption Board Lot is obtained by the administrative center of distributed system, the resource consumption unit is suitable Sequence is numbered；

Board Lot is provided divided by the resource in the distributed system by the resource consumption Board Lot, is obtained The resource consumption unit that each resource provides unit and should undertake averagely undertakes quantity N, and N is the numerical value after rounding；

The resource provides the serial number M and average mark that unit is provided in unit according to oneself in all resources The principle matched somebody with somebody, according to the order of resource consumption unit number, from resource consumption unit extract quantity be N or (N-1) or (N+1) resource consumption unit, as distributing to the resource consumption unit of oneself.

Optionally, it is machine in a distributed system that the resource provides unit, and the resource consumption unit is Process the sets of threads of task items to be allocated.

Optionally, the resource provides the sets of threads that unit is the task items for processing to be allocated, the resource consumption Unit is task items to be allocated.

Optionally, if the mode that rounds that the resource consumption unit averagely undertakes quantity N is:Remove the resource Consume Board Lot and provide the decimal place that Board Lot obtains value divided by the resource in the distributed system；

Now, the order according to resource consumption unit number, it is N that quantity is extracted from resource consumption unit The either resource consumption unit of (N-1) or (N+1), be extract from resource consumption unit quantity be N or (N+1) resource consumption unit, including：

By resource consumption unit undertake quantity N be multiplied by resource provide Board Lot round resource consumption unit Quantity；

Resource consumption Board Lot is deducted and rounds resource consumption Board Lot acquisition surplus resources consumption units Amount X；

For the numerical value that the numerical value that resource provides unit sequence M consumes Board Lot X less than or equal to surplus resources Resource provide unit, according to resource provide unit sequence M order, from resource consumption unit according to The order of resource consumption unit number extracts resource consumption unit of the quantity for N+1；

The numerical value for providing unit sequence M for resource consumes the resource of the numerical value of Board Lot X more than surplus resources Unit is provided, according to resource the order of unit sequence M is provided, disappear according to resource from resource consumption unit The order of consumption unit number extracts resource consumption unit of the quantity for N.

Optionally, if the mode that rounds that the resource consumption unit undertakes quantity N is:The resource consumption unit Quantity provides Board Lot value divided by the resource in the distributed system, if there is decimal place, Take the smallest positive integral more than the acquisition value；

Now, the order according to resource consumption unit number, it is N that quantity is extracted from resource consumption unit The either resource consumption unit of (N-1) or (N+1), be extract from resource consumption unit quantity be N or (N-1) resource consumption unit, including：

Resource consumption Board Lot will be rounded and deduct resource consumption Board Lot acquisition spare resources consumption units Amount Y；

For the numerical value that the numerical value that resource provides unit sequence M consumes Board Lot Y less than or equal to spare resources Resource provide unit, according to resource provide unit sequence M order, from resource consumption unit according to The order of resource consumption unit number extracts resource consumption unit of the quantity for (N-1)；

The numerical value for providing unit sequence M for resource consumes the resource of the numerical value of Board Lot Y more than spare resources Unit is provided, according to resource the order of unit sequence M is provided, disappear according to resource from resource consumption unit The order of consumption unit number extracts resource consumption unit of the quantity for N.

Accordingly, present invention also provides a kind of dispatching device for distributed system, described for distributed The dispatching device of system includes：

Resource provides unit acquiring unit, and the resource for distributed system provides unit by distributed system Administrative center obtains the resource in the distributed system and provides Board Lot, and the resource provides unit itself The resource provided in unit in all resources provides unit sequence M；

Numbered cell, for obtaining resource consumption Board Lot, the money by the administrative center of distributed system Source consumes unit and is sequentially numbered；

Undertake amount calculation unit, for by the resource consumption Board Lot divided by the distributed system Resource provides Board Lot, and the resource consumption unit that obtaining each resource offer unit should undertake averagely undertakes Quantity N, N is the numerical value after rounding；

Resource consumption unit allocation unit, provides unit and provides single in all resources according to oneself for the resource The principle of serial number M and mean allocation in position, according to the order of resource consumption unit number, disappears from resource The resource consumption unit that quantity is N either (N-1) or (N+1) is extracted in consumption unit, as distributing to oneself Resource consumption unit.

Optionally, if described undertake amount calculation unit, remove specifically for removing the resource consumption Board Lot The decimal place that Board Lot obtains value is provided with the resource in the distributed system, then the resource consumption Unit allocation unit, including：Round computation subunit, remainder computation subunit, first resource and consume single Bit allocation subelement and Secondary resource consume unit distribution subelement；

It is described to round computation subunit, for extracting the money that quantity is N or (N+1) from resource consumption unit Source consume unit, by resource consumption unit undertake quantity N be multiplied by resource provide Board Lot round resource Consume Board Lot；

The remainder computation subunit, for resource consumption Board Lot to be deducted resource consumption units is rounded Amount obtains surplus resources and consumes Board Lot X；

The first resource consumes unit distribution subelement, for providing the numerical value of unit sequence M for resource Unit is provided less than or equal to the resource of the numerical value that surplus resources consume Board Lot X, according to resource unit is provided The order of serial number M, extracting quantity according to the order of resource consumption unit number from resource consumption unit is The resource consumption unit of N+1；

The Secondary resource consumes unit distribution subelement, and the numerical value for providing unit sequence M for resource is big The resource that the numerical value of Board Lot X is consumed in surplus resources provides unit, and according to resource unit sequence M is provided Order, from resource consumption unit according to resource consumption unit number order extract quantity for N resource Consume unit.

Optionally, if described undertake amount calculation unit, if specifically for there is decimal place, taking and being obtained more than this The smallest positive integral that must be worth, then the resource consumption unit allocation unit, including：Round computation subunit, many Remainder computation subunit, first resource consume unit distribution subelement and Secondary resource consumes unit distribution Unit；

It is described to round computation subunit, for extracting the money that quantity is N or (N-1) from resource consumption unit Source consume unit, by resource consumption unit undertake quantity N be multiplied by resource provide Board Lot round resource Consume Board Lot；

The redundant digit computation subunit, for rounding resource consumption Board Lot resource consumption units is deducted Amount obtains spare resources and consumes Board Lot Y；

The first resource consumes unit distribution subelement, for providing the numerical value of unit sequence M for resource Unit is provided less than or equal to the resource of the numerical value that spare resources consume Board Lot Y, according to resource unit is provided The order of serial number M, extracting quantity according to the order of resource consumption unit number from resource consumption unit is (N-1) resource consumption unit；

The Secondary resource consumes unit distribution subelement, and the numerical value for providing unit sequence M for resource is big The resource that the numerical value of Board Lot Y is consumed in spare resources provides unit, and according to resource unit sequence M is provided Order, from resource consumption unit according to resource consumption unit number order extract quantity for N resource Consume unit.

Additionally, present invention also provides a kind of electronic equipment, the electronic equipment includes：

Display；

Processor；

Memory, for the scheduler program of distributed storage system, described program is being held by processor reading During row, following operation is performed:The resource of distributed system is provided unit and is obtained by the administrative center of distributed system Take the resource in the distributed system and Board Lot is provided, and the resource provides unit sheet in all resources The resource provided in unit provides unit sequence M；Resource consumption is obtained by the administrative center of distributed system Board Lot, the resource consumption unit is sequentially numbered；By the resource consumption Board Lot divided by described point Resource in cloth system provides Board Lot, obtains each resource and provides the resource consumption that unit should undertake Unit averagely undertakes quantity N, and N is the numerical value after rounding；The resource provides unit according to oneself in all moneys Source provides the principle of the serial number M in unit and mean allocation, according to the order of resource consumption unit number, The resource consumption unit that quantity is N either (N-1) or (N+1) is extracted from resource consumption unit, as point The resource consumption unit of dispensing oneself.

Additionally, present invention also provides another kind of dispatching method for distributed system, described for distributed The dispatching method of system includes：

Read the control information stored in zookeeper systems；The control information is included in zookeeper The server registered in system, and the specific strategy for performing, the specific strategy are needed in distributed system The different concrete executive mode of description and each requirement of concrete executive mode to sets of threads quantity；

According to it is described in zookeeper systems register number of servers, and it is described need perform it is concrete Strategy, recalculating each server needs the sets of threads quantity of distribution；

Unit is provided as resource using the server registered in zookeeper systems, needs to divide by described The sets of threads matched somebody with somebody as resource consumption unit, and usage right require described in 1-5 any one for point The dispatching method of cloth system, is that server configures corresponding sets of threads, including distributes new sets of threads, with And the sets of threads that stopping has been started up.

Optionally, the control information also includes：The sets of threads registered in zookeeper systems, and treat The task items of process；

After the step of the configuration corresponding sets of threads for server, including：

According to the sets of threads quantity registered in zookeeper systems, and the pending task , calculating each sets of threads needs the task items quantity of distribution；

Unit is provided as resource using the sets of threads quantity registered in zookeeper systems, by the need Task items to be processed are used as resource consumption unit, and usage right requires the use described in 1-5 any one It is that each sets of threads configures corresponding task items, including to the allocated in the dispatching method of distributed system Task items redistribute, new task items distributed for process group and the task items for stopping are eliminated the need for.

Optionally, the control information for storing in the zookeeper systems is stored in server local, and with The control information of server local storage performs the foundation of distributed task dispatching as the server；If There is abnormal, then directly to store with server local control information execution distributed task scheduling in zookeeper systems Scheduling.

Optionally, before the control information for storing in the reading zookeeper systems, including：

Each server have reload field Reload；

There are each server fixed time intervals just to read guarding for the value for reloading field Reload Thread；

If the value for reloading field Reload that the daemon thread reads is true, into institute State the step of reading the control information stored in zookeeper systems；And reload field Reload by described Value be set to false.

Optionally, the zookeeper systems stays are monitored；Arrange described if there is following arbitrarily item Field Reload is reloaded for true：Zookeeper system reboots, zookeeper system configurations change, The control information change of zookeeper systems storage, server initiation.

Optionally, each sets of threads of the distributed system be respectively arranged with to should sets of threads loading judge Field Needload；

Described each sets of threads just reads the loading and judges field Needload with fixed time intervals respectively Value sets of threads finger daemon；

If the value for reloading field Needload that the sets of threads finger daemon reads is true, Then enter it is described according to it is described in zookeeper systems register sets of threads quantity, and it is described need place The task items of reason, the step of calculating each sets of threads and need the task items quantity of distribution；And sentence the loading The value of disconnected field Needload is set to false.

Optionally, it is described configure corresponding task items for each sets of threads the step of after, including：

Read the control information version number of zookeeper；

Obtain the control information version number of server；

Judge whether the version number of server is identical with the version number of zookeeper；

If it is not, then reading field Reload of reloading for being stored in server local, and add again by described in The value for carrying field is revised as true.

Optionally, each server is provided with server running mark field isRunning；The server operation Attribute field is true in the case where server normally runs；When the server needs to log off, Server running mark field isRunning is then set and is set to false, and stop server initiation thread, The company of halt system daemon thread, all sets of threads, log-on server and zookeeper of stopping server Connect.

Accordingly, it is described for being distributed present invention also provides another kind of dispatching device for distributed system The dispatching device of formula system includes：

Control information reading unit, for reading zookeeper systems in store control information；The control Information is included in the server registered in zookeeper systems, and the tool for performing is needed in distributed system Body strategy, the specific strategy describes different concrete executive mode and each concrete executive mode to thread The requirement of group quantity；

Recalculate unit, for according to it is described in zookeeper systems register number of servers, and The specific strategy for needing to perform, recalculating each server needs the sets of threads quantity of distribution；

Sets of threads allocation unit, for the server registered in zookeeper systems to be carried as resource For unit, using the sets of threads for needing distribution as resource consumption unit, and usage right requires that 1-5 appoints The dispatching method for distributed system described in meaning one, is that server configures corresponding sets of threads, including The new sets of threads of distribution, and stop the sets of threads having been started up.

Optionally, the control information reading unit, the control information of reading also includes：In zookeeper The sets of threads registered in system, and pending task items；

The dispatching device for distributed system, also includes：

Task items computing unit, for it is described configure corresponding sets of threads for server after, according to it is described The sets of threads quantity registered in zookeeper systems, and the pending task items, calculate each thread Group needs the task items quantity of distribution；

Task items allocation unit, for using it is described in zookeeper systems register sets of threads quantity as money Source provides unit, and using need task items to be processed as resource consumption unit, and usage right requires 1- The dispatching method for distributed system described in 5 any one, is that each sets of threads configures corresponding task , including redistributing to the allocated task items, be that process group distributes new task items and eliminates and needs The task items to be stopped.

Optionally, the dispatching device for distributed system, also includes：

Control information memory cell, for the control information stored in the zookeeper systems to be stored in into clothes Business device is local, and performs distributed task dispatching as the server using the control information of server local storage Foundation；If exception occur in zookeeper systems, the control information for directly being stored with server local is held Row distributed task dispatching.

Additionally, present invention also provides a kind of electronic equipment, including：

Display；

Processor；

Memory, for the scheduler program of distributed storage system, described program is being held by processor reading During row, following operation is performed:Read the control information stored in zookeeper systems；The control information bag The server registered in zookeeper systems is included, and the specific strategy for performing is needed in distributed system, The specific strategy describes different concrete executive mode and each concrete executive mode to sets of threads quantity Require；According to it is described in zookeeper systems register number of servers, and it is described need perform Specific strategy, recalculating each server needs the sets of threads quantity of distribution；By described in zookeeper The server registered in system provides unit as resource, needs the sets of threads of distribution to disappear as resource using described Consumption unit, and usage right requires the dispatching method for distributed system described in 1-5 any one, Corresponding sets of threads is configured for server, including distributes new sets of threads, and stop the thread having been started up Group.

Compared with prior art, the application has advantages below：

A kind of dispatching method for distributed system, device and a kind of electronic equipment that the application is provided.It is logical The resource offer unit for crossing distributed system is obtained in the distributed system by the administrative center of distributed system Resource Board Lot is provided, and the resource provides the resource that unit sheet is provided in unit in all resources Unit sequence M is provided；Resource consumption Board Lot, the money are obtained by the administrative center of distributed system Source consumes unit and is sequentially numbered；By the resource consumption Board Lot divided by the money in the distributed system Source provides Board Lot, and the resource consumption unit that obtaining each resource offer unit should undertake averagely undertakes number Amount N, N is the numerical value after rounding；The resource provides unit and is provided in unit in all resources according to oneself The principle of serial number M and mean allocation, according to the order of resource consumption unit number, from resource consumption unit It is middle to extract the resource consumption unit that quantity is N either (N-1) or (N+1), as distributing to the resource of oneself Consume unit.The technical scheme is solved in distributed system distributes the place that task is produced by distributed lock It is less efficient during reason task, deadlock and circulation can be caused to seize cause task distribution convergence when distributing more slowly Between long problem.

The application also provides a kind of multiprocessor scheduling method specifically for zookeeper system environments, this The method of kind has implemented above-mentioned scheduling mode in zookeeper systems.

Description of the drawings

In order to be illustrated more clearly that the embodiment of the present application or technical scheme of the prior art, below will be to reality Apply the accompanying drawing to be used needed for example or description of the prior art to be briefly described, it should be apparent that, below Accompanying drawing in description is only some embodiments described in the application, for those of ordinary skill in the art For, can be with according to these other accompanying drawings of accompanying drawings acquisition.

Fig. 1 shows the dispatching method for distributed task scheduling according to the offer of the first embodiment of the application Flow chart；

Fig. 2 shows that the quantity of extracting from resource consumption unit provided according to the first embodiment of the application is The flow chart of N either resource consumption units of (N-1) or (N+1)；

Fig. 3 shows that the another kind provided according to the first embodiment of the application is extracted from resource consumption unit Quantity is the flow chart of N either resource consumption units of (N-1) or (N+1)；

Fig. 4 shows the dispatching device for distributed task scheduling according to the offer of the second embodiment of the application Schematic diagram；

Fig. 5 shows the schematic diagram of the electronic equipment provided according to the 3rd embodiment of the application；

Fig. 6 shows the dispatching method for distributed system according to the offer of the fourth embodiment of the application Flow chart；

Fig. 7 is shown in the dispatching method for distributed system provided according to the fourth embodiment of the application For the flow chart that each sets of threads configures corresponding task items；

Fig. 8 shows that the another kind provided according to the 5th embodiment of the application is used for the scheduling of distributed task scheduling The schematic diagram of device；

Fig. 9 shows the schematic diagram of the electronic equipment provided according to the sixth embodiment of the application.

Specific embodiment

In order to be more clearly understood that above-mentioned purpose, the feature and advantage of the application, below in conjunction with the accompanying drawings and tool Body embodiment is further described in detail to the application.It should be noted that in situation about not conflicting Under, the feature in embodiments herein and embodiment can be mutually combined.

Elaborate many details in order to fully understand the application in the following description.But, the application Can to be much different from alternate manner described here implementing, those skilled in the art can without prejudice to Similar popularization is done in the case of the application intension, therefore, the application by it is following public be embodied as limit System.

Before the concrete steps of the present embodiment are described in detail, the distributed scheduling that first the technical program is related to It is briefly described.

Distributed scheduling is referred to：When the task amount for needing scheduling is very big, if only with a machine processing task, Then the processing speed of task can be relatively slower, therefore often processed with multiple stage machine from distributed system and appoint Business, needs for substantial amounts of task to be divided into multiple tasks block when processing, by coordinating how to perform faster These task blocks, make that the task block that the machine in distributed system is born is more balanced, rapid, disaster tolerance is strong Scheduling scheme, be called distributed scheduling, realize distributed scheduling need use distributed scheduling component.

In a distributed system, at present often using zookeeper to the multiple servers in system in it is all enter Journey configuration information is managed concentratedly.Zookeeper as the term suggests zooman, in computer technology neck Zookeeper is a reliable coordination system for large-scale distributed system in domain, there is provided function include： Configuring maintenance, name Service, distributed synchronization, group service etc..The existing information based on zookeeper Allocation plan is：The strong dependence that each process will be set up and zookeeper between in each server, often Individual process by calling the configuration information of the interface monitoring own process of zookeeper, when some process Configuration information need modification when, operating personnel only need to change zookeeper on corresponding configuration information, one Denier finds that zookeeper configuration informations change, and corresponding server processes upgrade in time own process Configuration information.

The first embodiment of the application provides a kind of dispatching method for distributed system.The dispatching method pair There is universality in the resource allocation of distributed system, be not limited to using the distributed system of zookeeper systems System.The dispatching method embodiment for distributed system is as follows：

Fig. 1 is refer to, the tune for distributed system provided according to the first embodiment of the application is provided The flow chart of degree method.First embodiment is described in detail below in conjunction with Fig. 1.

Step S101, the resource of distributed system provides unit should by administrative center's acquisition of distributed system Resource in distributed system provides Board Lot, and the resource provides unit sheet and provides in all resources Resource in unit provides unit sequence M.

Firstly the need of explanation, there are various types of resources to provide unit in distributed system, in the present embodiment, Machine in a distributed manner in system (main frame or server are can be generally referred to as in computer system, it is contemplated that The possibility that other distributed systems are present, summarizes used here as the word of machine one) as the distributed system Resource provide unit example.

According to this step, each resource in the distributed system provide unit, i.e. each machine by with point After the administrative center of cloth system is attached, the quantity of the registered machine in the distributed system is obtained, And serial number M of the machine sheet in all machines.The machine is run in the distributed system Physical machine, the serial number of the machine is order when registering in the distributed system according to each machine The numbering produced to distinguish each machine.

Physical machine herein, can show as in computer systems Java Virtual Machine (being abbreviated as JVM) Java Virtual Machine) abbreviation, JVM is a kind of specification for computing device, using JVM can realize with The independence of platform, if general language will run on different platforms, at least needs to be compiled into difference Object code；After using JVM, need not recompilate when running in different platform, only need to generate The object code (bytecode) run on JVM, it is possible to run without modification in kinds of platform.Java When bytecode is performed, machine instruction bytecode being construed on platform specific is performed virtual machine.

For example：The a certain specific machine for running in a distributed system is by the administrative center with distributed system After being attached, get registered 10 machines in the distributed system, and the machine sheet in point It is the 3rd registration in cloth system, then gets serial number of the machine in all machines for 3.

Step S103, by the administrative center of distributed system resource consumption Board Lot is obtained, and the resource disappears Consumption unit is sequentially numbered.

In the present embodiment, the resource consumption unit is in machine in a distributed system, to treat for processing The sets of threads of the task items of distribution, in this step, each machine in the distributed system by with point After the administrative center of cloth system is attached, obtaining from the administrative center from be used for when this is dispatched The quantity of the sets of threads of reason task items to be allocated, the sets of threads is sequentially numbered.

For example：The a certain specific machine for running in a distributed system is by the administrative center with distributed system After being attached, the machine gets from the administrative center and have 30 when this is dispatched and treat for process The sets of threads of the task items of distribution, the sets of threads has the numbering from 1-30, and each sets of threads is according to suitable Ordered pair answers a specific numbering.

Step S105, provides single by the resource consumption Board Lot divided by the resource in the distributed system Bit quantity, the resource consumption unit that obtaining each resource offer unit should undertake averagely undertakes quantity N, and N is Numerical value after rounding.

In the present embodiment, the distributed system for having obtained in this step receiving step S101 and step S103 In registered machine quantity and this when dispatching for processing the number of the sets of threads of task items to be allocated Amount, and by the quantity of the sets of threads divided by the machine quantity, obtaining in each machine averagely to hold Quantity N of the sets of threads of load, quantity N of the sets of threads that should averagely undertake in described each machine is exactly at this During secondary scheduling, quantity N for processing the sets of threads of task items to be allocated distributed in each machine, N For the numerical value after rounding.

For example：The distribution that a certain specific machine for running in a distributed system is obtained in step S101 The quantity of the registered machine in formula system is 10, is got in step s 103 when this is dispatched There are 30 sets of threads for processing task items to be allocated, then by the quantity 30 of the sets of threads divided by institute The quantity 10 of machine is stated, the quantity 3 of the sets of threads that should averagely undertake in each machine is obtained, i.e.,：At this Distribute 3 sets of threads during secondary scheduling in this machine for processing task items to be allocated.

It should be noted that the resource consumption Board Lot is being carried divided by the resource in the distributed system During for Board Lot, because the resource consumption Board Lot possibly cannot divide exactly the resource units is provided Amount, so the resource consumption unit that each resource provides unit should be undertaken averagely undertakes quantity N, N is to round Numerical value afterwards.The resource consumption unit that each resource offer unit should undertake averagely is held in step S105 The mode that rounds of load quantity N includes following two modes：

First kind of way is：Remove the resource consumption Board Lot to carry divided by the resource in the distributed system The decimal place of value is obtained for Board Lot；

The second way is：The resource consumption Board Lot provides single divided by the resource in the distributed system Bit quantity is worth, if there is decimal place, takes the smallest positive integral more than the acquisition value.

Which kind of certainly, also it is not excluded for using rounding up or other modes are collected evidence, no matter the side of rounding taken Formula, final to round effect inevitable identical with the one kind in above two mode.

For example：Under the first technique, a certain specific machine for running in a distributed system is in step S101 The quantity of the registered machine in the distributed system of middle acquisition is 10, is obtained in step s 103 To there is 33 sets of threads for processing task items to be allocated when this is dispatched, then by the sets of threads Quantity 33 obtains the quantity of the sets of threads that should averagely undertake in each machine divided by the quantity 10 of the machine 3；Under a second technique, a certain specific machine for running in a distributed system is obtained in step S101 The quantity of the registered machine in the distributed system for taking is 10, is got in step s 103 There are 33 sets of threads for processing task items to be allocated during this scheduling, then by the quantity of the sets of threads 33 divided by the machine quantity 10, obtain the quantity 4 of the sets of threads that should averagely undertake in each machine.

Step S107, the resource provides unit and provides the serial number M in unit in all resources according to oneself And the principle of mean allocation, according to the order of resource consumption unit number, from resource consumption unit number is extracted Measure as the resource consumption unit of N either (N-1) or (N+1), as distributing to the resource consumption unit of oneself.

In the present embodiment, the serial number M according to the machine sheet for having obtained in all machines, from described For in the sets of threads for processing task items to be allocated according to the sets of threads numbering order, extract quantity Sets of threads for N either (N-1) or (N+1) distributes to the machine.

For example：Serial number of a certain specific machine for running in a distributed system in all machines is 3, According to Principle of Average Allocation, the machine that serial number is 1 and 2 is waited from for processing task items to be allocated After extraction sets of threads is finished in sets of threads, from the sets of threads for processing task items to be allocated 3 are extracted Individual sets of threads, and the numbering of the sets of threads extracted is 7-9.

There is provided single divided by the resource in the distributed system by the resource consumption Board Lot in step S105 During bit quantity, Board Lot is provided because the resource consumption Board Lot possibly cannot divide exactly the resource, So the resource consumption unit that each resource provides unit should be undertaken averagely undertakes quantity N, after N is rounded Numerical value, so the technical scheme of the present embodiment provides a kind of preferred embodiment, under preferred embodiment, if The resource consumption unit averagely undertakes rounding mode and rounding mode for the first for quantity N, that is, remove described Resource consumption Board Lot provides the decimal that Board Lot obtains value divided by the resource in the distributed system Position, then extract the resource consumption list that quantity is N either (N-1) or (N+1) in the unit from resource consumption Position, is that the resource consumption unit that quantity is N or (N+1) is extracted from resource consumption unit, is specifically included Step S107-1 to S107-4,2 is described further below in conjunction with the accompanying drawings.

Fig. 2 is refer to, to be it illustrates and extract number from resource consumption unit according to what embodiments herein was provided Measure as the flow chart of N either resource consumption units of (N-1) or (N+1).

Step S107-1, by resource consumption unit undertake quantity N be multiplied by resource provide Board Lot round Resource consumption Board Lot.

In the present embodiment, it is described by resource consumption unit undertake quantity N be multiplied by resource provide Board Lot obtain Rounding resource consumption Board Lot is, will distribute in each machine for processing task items to be allocated Quantity N of sets of threads is multiplied by the quantity of registered machine in a distributed system, after rounding in machine The sets of threads for processing task items to be allocated distributed in device rounds sets of threads quantity.

For example：The quantity of the sets of threads that should averagely undertake in each machine is 3, in the distributed system Registered machine quantity be 10, then by the quantity of the sets of threads that should averagely undertake in each machine 3 quantity 10 for being multiplied by the registered machine in the distributed system, the distribution in the machine after rounding The sets of threads for processing task items to be allocated round sets of threads quantity 30.

Step S107-2, resource consumption Board Lot is deducted and rounds the remaining money of resource consumption Board Lot acquisition Source consumes Board Lot X.

In the present embodiment, described deduct resource consumption Board Lot rounds resource consumption Board Lot and obtains surplus Remaining resource consumption Board Lot X is will to obtain from the administrative center and be treated for process when this is dispatched The quantity of the sets of threads of the task items of distribution rounds sets of threads quantity described in deducting, and obtains remaining sets of threads quantity X。

For example：Getting from the administrative center has 33 when this is dispatched for processing to be allocated appointing The sets of threads of business item, the quantity 33 for processing the sets of threads of task items to be allocated is deducted and rounds line Journey group quantity 30, obtains remaining sets of threads quantity 3.

Step S107-3, for the numerical value that resource provides unit sequence M consumes single less than or equal to surplus resources The resource of the numerical value of bit quantity X provides unit, the order of unit sequence M is provided according to resource, from resource Consume in unit and extract resource consumption unit of the quantity for N+1 according to the order of resource consumption unit number.

In the present embodiment, it is described for the numerical value that resource provides unit sequence M disappears less than or equal to surplus resources The resource of the numerical value of consumption Board Lot X provides unit, and according to resource the order of unit sequence M is provided, from Resource consumption unit of the quantity for N+1 is extracted in resource consumption unit according to the order of resource consumption unit number Be, to the numerical value of serial number less than or equal to the numerical value of remaining sets of threads quantity X in the distributed system The machine of registration, according to the order of serial number M of the machine sheet in all machines, from described for from The order of the numbering in the sets of threads of task items to be allocated according to the sets of threads is managed, quantity is extracted for (N+1) Sets of threads distribute to the machine.

For example：It is 3 to have obtained remaining sets of threads quantity, the number of the sets of threads that should averagely undertake in each machine Measure as 3, then in the distributed system registered machine bag of the serial number less than or equal to remaining sets of threads quantity Include：Serial number is 1 machine, the machine that serial number is 2 and machine that serial number is 3, then from the use Extract in the sets of threads for processing task items to be allocated 4 sets of threads distribute to machine that serial number is 1 and The numbering of the sets of threads of extraction is 1-4；4 are extracted from the sets of threads for processing task items to be allocated Individual sets of threads distributes to the machine that serial number is 2 and the numbering of the sets of threads extracted is 5-8；From described for from 4 sets of threads are extracted in the sets of threads of reason task items to be allocated distribute to the machine and extraction that serial number is 3 The numbering of sets of threads be 9-12.

Step S107-4, the numerical value for providing unit sequence M for resource consumes Board Lot X more than surplus resources Numerical value resource provide unit, according to resource provide unit sequence M order, from resource consumption unit According to resource consumption unit number order extract quantity for N resource consumption unit.

In the present embodiment, the numerical value for providing unit sequence M for resource consumes unit more than surplus resources The resource of the numerical value of quantity X provides unit, and according to resource the order of unit sequence M is provided, and disappears from resource Extract quantity according to the order of resource consumption unit number in consumption unit is for the resource consumption unit of N, to suitable The numerical value of sequence number is more than the registered machine in the distributed system of the numerical value of remaining sets of threads quantity X, According to the order of serial number M of the machine sheet in all machines, from described for processing to be allocated appointing Business item sets of threads according to the sets of threads numbering order, extract quantity be N sets of threads distribute to The machine.

For example：It is 3 to have obtained remaining sets of threads quantity, the number of the sets of threads that should averagely undertake in each machine Measure as 3, then serial number includes more than the registered machine in the distributed system of remaining sets of threads quantity： Serial number is the machine of 4-10, if the serial number of a certain specific machine for running in a distributed system is 4, 3 sets of threads and the sets of threads extracted then are extracted from the sets of threads for processing task items to be allocated Numbering be 13-15, serial number for 5-10 machine then in order number from the line that the numbering of sets of threads is 16 Journey group starts to extract 3 sets of threads successively according to the order of the numbering of sets of threads distributes to machine, until will Sets of threads is assigned.

Above the one kind to providing in the present embodiment rounds mode and is illustrated, and also provides in the technical program Another kind rounds the preferred embodiment of mode, under preferred embodiment, if the resource consumption unit undertakes The mode that rounds of quantity N is the second way, i.e., described resource consumption Board Lot is divided by the distributed system Resource in system provides Board Lot value, if there is decimal place, takes the minimum more than the acquisition value Integer, then extract the resource consumption that quantity is N either (N-1) or (N+1) in the unit from resource consumption Unit, is that the resource consumption unit that quantity is N or (N-1) is extracted from resource consumption unit, concrete bag Step S107-5 to S107-8 is included, 3 is described further below in conjunction with the accompanying drawings.

Refer to Fig. 3, it illustrates according to embodiments herein provide another kind from resource consumption unit Extract the flow chart that quantity is N either resource consumption units of (N-1) or (N+1).

Step S107-5, by resource consumption unit undertake quantity N be multiplied by resource provide Board Lot round Resource consumption Board Lot.

For example：The quantity of the sets of threads that should averagely undertake in each machine is 4, in the distributed system Registered machine quantity be 10, then by the quantity of the sets of threads that should averagely undertake in each machine 4 quantity 10 for being multiplied by the registered machine in the distributed system, the distribution in the machine after rounding The sets of threads for processing task items to be allocated round sets of threads quantity 40.

Step S107-6, will round resource consumption Board Lot and deducts the unnecessary money of resource consumption Board Lot acquisition Source consumes Board Lot Y.

In the present embodiment, it is described will round resource consumption Board Lot deduct resource consumption Board Lot obtain it is many Remaining resource consumption Board Lot Y is that the sets of threads quantity that rounds is deducted into the acquisition from the administrative center When this is dispatched for processing the quantity of the sets of threads of task items to be allocated, unnecessary sets of threads quantity is obtained Y。

For example：Getting from the administrative center has 33 when this is dispatched for processing to be allocated appointing The sets of threads of business item, by it is described round sets of threads quantity 40 deduct it is described for processing task items to be allocated The quantity 33 of sets of threads, obtains unnecessary sets of threads quantity 7.

Step S107-7, for the numerical value that resource provides unit sequence M consumes single less than or equal to spare resources The resource of the numerical value of bit quantity Y provides unit, the order of unit sequence M is provided according to resource, from resource Consume in unit and extract resource consumption unit of the quantity for (N-1) according to the order of resource consumption unit number.

In the present embodiment, it is described for the numerical value that resource provides unit sequence M disappears less than or equal to spare resources The resource of the numerical value of consumption Board Lot Y provides unit, and according to resource the order of unit sequence M is provided, from Resource consumption list of the quantity for (N-1) is extracted in resource consumption unit according to the order of resource consumption unit number Position is, to the numerical value of serial number noting in the distributed system less than the numerical value of unnecessary sets of threads quantity Y The machine of volume, according to the order of serial number M of the machine sheet in all machines, from described for processing In the sets of threads of task items to be allocated according to the sets of threads numbering order, extract quantity for (N-1) Sets of threads distribute to the machine.

For example：It is 7 to have obtained unnecessary sets of threads quantity, the number of the sets of threads that should averagely undertake in each machine Measure as 4, then in the distributed system registered machine bag of the serial number less than or equal to unnecessary sets of threads quantity Include：Serial number is the machine of 1-7, then start according to sets of threads from the sets of threads that the numbering of sets of threads is 1 The order of numbering extracts successively 3 sets of threads and distributes to machine, until in the machine that serial number is 7 by According to complete 3 sets of threads of order-assigned of the numbering of sets of threads.

Step S107-8, the numerical value for providing unit sequence M for resource consumes Board Lot Y more than spare resources Numerical value resource provide unit, according to resource provide unit sequence M order, from resource consumption unit According to resource consumption unit number order extract quantity for N resource consumption unit.

In the present embodiment, the numerical value for providing unit sequence M for resource consumes unit more than spare resources The resource of the numerical value of quantity Y provides unit, and according to resource the order of unit sequence M is provided, and disappears from resource Extract quantity according to the order of resource consumption unit number in consumption unit is for the resource consumption unit of N, to suitable The numerical value of sequence number is more than the registered machine in the distributed system of the numerical value of unnecessary sets of threads quantity Y, According to the order of serial number M of the machine sheet in all machines, from described for processing to be allocated appointing Business item sets of threads according to the sets of threads numbering order, extract quantity be N sets of threads distribute to The machine.

For example：It is 7 to have obtained unnecessary sets of threads quantity, the number of the sets of threads that should averagely undertake in each machine Measure as 4, then serial number includes more than the registered machine in the distributed system of unnecessary sets of threads quantity： Serial number is 8 machine, the machine that serial number is 9 and machine that serial number is 10, then from described for from 4 sets of threads are extracted in the sets of threads of reason task items to be allocated distribute to the machine and extraction that serial number is 8 The numbering of sets of threads be 22-25；4 are extracted from the sets of threads for processing task items to be allocated Sets of threads distributes to the machine that serial number is 9 and the numbering of the sets of threads extracted is 26-29；From described for from 4 sets of threads are extracted in the sets of threads of reason task items to be allocated distribute to the machine and extraction that serial number is 10 The numbering of sets of threads be 30-33.

It is machine in a distributed system to provide the resource unit above, and the resource consumption unit is place During the sets of threads of reason task items to be allocated, the dispatching method for distributed system has been carried out specifically It is bright, it should be noted that it can also be to process to be allocated appointing that resource described in the technical program provides unit The sets of threads of business item, the resource consumption unit can also be task items to be allocated.

Specifically, the sets of threads that unit is the task items for processing to be allocated is provided in the resource, the resource disappears When consumption unit is task items to be allocated, the dispatching method for distributed system, including：

By the administrative center of distributed system obtain in the distributed system for processing task to be allocated The quantity of the sets of threads of item, and serial number M of the sets of threads sheet in all sets of threads；

The quantity of task items to be allocated, the task to be allocated are obtained by the administrative center of distributed system Item is sequentially numbered；

By the quantity of the task items to be allocated divided by the distributed system in it is to be allocated for processing The quantity of the sets of threads of task items, the task items to be allocated that obtaining each sets of threads should undertake averagely undertake Quantity N, N is the numerical value after rounding；

The sets of threads for processing task items to be allocated is according to oneself serial number in all sets of threads The principle of M and mean allocation, according to the order that task items to be allocated are numbered, from task items to be allocated The task items to be allocated that quantity is N either (N-1) or (N+1) are extracted, as distributing to the task of oneself .

Because the main distinction is simply the particular content that the resource provides unit and the resource consumption unit Difference, and the processing procedure such as follow-up calculating and distribution be it is essentially identical, due in preceding embodiment Jing compares detailed description to this, and here is omitted.

In the above-described embodiment, there is provided a kind of dispatching method for distributed system, with it is above-mentioned for point The dispatching method of cloth system is corresponding, and the application second embodiment provides a kind of for distributed system Dispatching device.Because the embodiment of device is substantially similar to the embodiment of method, so describing simpler Single, related part is illustrated referring to the part of embodiment of the method.Device embodiment described below is only Schematically.The dispatching device embodiment for distributed system is as follows：

Fig. 4 is refer to, the dispatching device for distributed system of the application second embodiment offer is provided Schematic diagram.

The dispatching device for distributed system, including：Resource provides unit acquiring unit 401, numbering Unit 403, undertake amount calculation unit 405 and resource consumption unit allocation unit 407；

The resource provides unit acquiring unit 401, and the resource for distributed system provides unit by distribution The administrative center of formula system obtains the resource in the distributed system and provides Board Lot, and the resource is provided The resource that unit sheet is provided in unit in all resources provides unit sequence M；

The numbered cell 403, for obtaining resource consumption Board Lot by the administrative center of distributed system, The resource consumption unit is sequentially numbered；

It is described to undertake amount calculation unit 405, for by the resource consumption Board Lot divided by described distributed Resource in system provides Board Lot, obtains each resource and provides the resource consumption unit that unit should undertake Quantity N is averagely undertaken, N is the numerical value after rounding；

The resource consumption unit allocation unit 407, unit is provided according to oneself in all moneys for the resource Source provides the principle of the serial number M in unit and mean allocation, according to the order of resource consumption unit number, The resource consumption unit that quantity is N either (N-1) or (N+1) is extracted from resource consumption unit, as point The resource consumption unit of dispensing oneself.

Optionally, if described undertake amount calculation unit 405, specifically for removing the resource consumption units Measure and provide the decimal place that Board Lot obtains value divided by the resource in the distributed system, then the resource Unit allocation unit 407 is consumed, including：Round computation subunit, remainder computation subunit, first resource Consume unit distribution subelement and Secondary resource consumes unit distribution subelement；

Optionally, if described undertake amount calculation unit 405, if specifically for there is decimal place, taking and being more than The smallest positive integral of the acquisition value, then the resource consumption unit allocation unit 407, including：Round calculating single Unit, redundant digit computation subunit, first resource consume unit distribution subelement and Secondary resource consumes unit Distribution subelement；

In the above-described embodiment, there is provided a kind of dispatching method and one kind for distributed system is used to divide The dispatching device of cloth system, following the application 3rd embodiment provides a kind of electronic equipment；The electronics Apparatus embodiments are as follows：

Fig. 5 is refer to, the schematic diagram of the electronic equipment provided according to embodiments herein is provided.

The electronic equipment, including：Display 501；Processor 503；Memory 505；

The memory 505, for the scheduler program of distributed storage system, described program is by the process Device is read when performing, and performs following operation:The resource of distributed system provides pipe of the unit by distributed system Reason center obtains resource in the distributed system and provides Board Lot, and the resource provide unit sheet in The resource that all resources are provided in unit provides unit sequence M；Obtained by the administrative center of distributed system Resource consumption Board Lot, the resource consumption unit is sequentially numbered；The resource consumption Board Lot is removed Board Lot is provided with the resource in the distributed system, each resource is obtained and is provided what unit should undertake Resource consumption unit averagely undertakes quantity N, and N is the numerical value after rounding；The resource provides unit according to oneself The principle of the serial number M in unit and mean allocation is provided in all resources, is compiled according to resource consumption unit Number order, the resource consumption list that quantity is N either (N-1) or (N+1) is extracted from resource consumption unit Position, as distributing to the resource consumption unit of oneself.

In above-mentioned first to 3rd embodiment, there is provided a kind of dispatching method for distributed system, The dispatching device and a kind of electronic equipment for distributed system is planted, additionally, present invention also provides another Plant the dispatching method specifically for the distributed system using zookeeper.This is used for the tune of distributed system Degree method is completed by component is dispatched, and dispatches component according to the control information meter stored in zookeeper systems Calculating each server needs the sets of threads quantity of distribution and calculates the task item number of each sets of threads needs distribution Amount, and be each sets of threads distribution task items for each server-assignment sets of threads.

It should be noted that in the present embodiment, component is dispatched using the distributed concurrent asynchronous scheduling group increased income The task that part TBSchedule 4, TBSchedule 4 can make batch tasks or constantly produce and change, Executed in parallel in the different sets of threads of multiple machines is dynamically assigned to, all of task can not weighed It is multiple, the quick process do not omitted.The operation level of the scheduling component, is divided into following 3 layer：By multiple Physical server (JVM) composition factory layers, in this layer each operation server be referred to as factory；The server layers of the sets of threads composition run in each server, in this layer each operation Sets of threads be referred to as server, allocation strategy of the sets of threads on factory is referred to as strategy, Strategy can dynamically be increased or be stopped, and in the present embodiment the strategy is called specific strategy； And the taskI tem layers of the task items composition processed in each sets of threads, each task items quilt in this layer Referred to as taskItem.It is above-mentioned for using the dispatching method of the distributed system of zookeeper the following 4th It is explained in embodiment.

Fig. 6 is refer to, the tune for distributed system provided according to the fourth embodiment of the application is provided The flow chart of degree method.

The dispatching method for distributed system, including：

Step S601, reads the control information stored in zookeeper systems；The control information is included in The server registered in zookeeper systems, and the specific strategy for performing is needed in distributed system, it is described Specific strategy describes different concrete executive mode and each concrete executive mode is wanted to sets of threads quantity Ask.

In the present embodiment, the control information for storing in the reading zookeeper systems, can adopt as follows Mode is realized：Scheduling component is set up after being connected with zookeeper, and scheduling component reads zookeeper systems In be stored in control information under directory node.The control information includes：Note in zookeeper systems The server of volume, and the specific strategy for performing is needed in distributed system, it is interior that the specific strategy is included The concrete executive mode that appearance includes expressed by the specific strategy, for example repeat, certain interval of time execution, According to different concrete executive modes such as priority execution, and each concrete executive mode is to sets of threads quantity Requirement.

It should be noted that be the storage information in the form of directory node in zookeeper systems, it is mutually similar The information Store of type is under same directory node.It should be understood that described register in zookeeper systems Server information, be stored in zookeeper /factory directory nodes under machine list In factorylist, in the information of the server registered in reading zookeeper systems, scheduling component is visited Ask be stored in zookeeper /factory directory nodes under machine list factorylist obtain The server of registration；The specific strategy for performing is needed in the distributed system, in being stored in zookeeper / strategy directory nodes in, read distributed system in need perform specific strategy when, scheduling group Part access be stored in zookeeper /strategy directory nodes obtain distributed system in need perform Specific strategy.

For example：In zookeeper /factory directory nodes under machine list factorylist in There are server A, server B and server C, server exists according to each server in factorylist Order when registering in the distributed system generates the serial number for characterizing registration order as each server； In zookeeper /strategy directory nodes in distributed system in need perform specific strategy have 4 Bar, every specific strategy is required to sets of threads quantity.

In the present embodiment, in the control information for storing in step S601 reads zookeeper systems, institute Stating the scheduling control information that reads of component also includes：The sets of threads registered in zookeeper systems, with And pending task items.It should be understood that the sets of threads registered in zookeeper systems, is to deposit Storage in zookeeper /server directory nodes in, read zookeeper systems in register thread Group when, dispatch component accesses be stored in zookeeper /server directory nodes under thread Groups List obtain It is taken at the sets of threads registered in zookeeper systems；The pending task items, are stored in zookeeper In /taskItem directory nodes in, when the pending task items are read, scheduling component accesses storage In zookeeper /taskItem directory nodes under pending task items, obtain it is described pending The quantity and numbering of task items.

Due to when distributed scheduling is performed every time, the number of the server of process task being performed in a distributed system The quantity of task items is not fixed when amount is fixed or dispatches every time, when distributed scheduling is performed every time, is needed The quantity of the specific strategy to be performed can change, so the technical scheme of the present embodiment is there is provided a kind of excellent Select embodiment, under preferred embodiment, in zookeeper /strategy directory nodes in store it is each Individual specific strategy includes：Policy Status.The Policy Status include opening and closed mode.It is described Opening is referred to：The strategy can be performed in this scheduling；On the other side, the closed mode is referred to： The strategy is rejected, the strategy is not performed in this scheduling.

It should be noted that the state of strategy can be modified by external control stand or recalculate clothes Business device need distribution sets of threads be server configure corresponding sets of threads when, change to should server tool The state of body strategy.If the state of strategy is closed mode, rejects the strategy and refer to：When this is dispatched The strategy is not performed, it is server-assignment sets of threads to be not based on the strategy, rather than is deleted from zookeeper Except the strategy, that is to say, that state is the strategy of closed mode, can need to divide by recalculating server The sets of threads matched somebody with somebody is the state that server configures corresponding sets of threads or external control stand modification strategy.

In order that after new server is registered in the distributed system, can in the distributed system process task Or the annexation of scheduling component and zookeeper, when changing, scheduling component can be right according to changing Sets of threads is redistributed and loaded to sets of threads in each server, and the technical scheme of the present embodiment is provided A kind of preferred embodiment, under preferred embodiment, each server registered in zookeeper systems With reloading field Reload, and each server have fixed time intervals just read it is described again plus Carry the daemon thread of the value of field Reload.

In the present embodiment, the daemon thread every five seconds for example carries out single pass, reads Boolean Class in the server The value of the Reload fields of type, if the value of the Reload fields for scanning is true, just triggers again Loading sets of threads, in the readings zookeeper systems the step of control information of storage, and by institute State and reload the value of field Reload and be set to false；If the value of the Reload fields for scanning is False, then do not carry out any operation, treats 5 seconds to be scanned next time afterwards.

In the present embodiment, the control information for storing in execution step S601 reads zookeeper systems Before step, scheduling component is also needed to be set up between zookeeper and is connected, described to dispatch component and zookeeper Between set up connection, can realize in the following way：Start initialization thread group, with zookeeper it Between set up connection, when in zookeeper connections, in the reading zookeeper systems storage The step of control information, simultaneously triggers the initialization for dispatching component.

In the present embodiment, in the startup initialization thread group, the step being connected is set up between zookeeper After rapid, in the scheduling component the lasting monitoring to zookeeper systems is started；Realization is persistently monitored Mode, for example can be that the item monitored as needed sets up corresponding monitor, for example：Connection status Monitor, child node change monitor and configuration variation monitor.How below explanation adopts in this way Realization is persistently monitored.

The connection status monitor is used to monitor the connection status between scheduling component and zookeeper, works as tune When connection status between degree component and zookeeper changes, for example：When disconnecting or reconnecting, The value that field Reload is reloaded described in each server is set to into true；The child node becomes Change monitor be used for monitor in zookeeper /factory directory nodes under machine list Factorylist, machine row that can be under/factory directory nodes when server is registered in a distributed system The directory node of the server, same reason, server and zookeeper are created in table factorylist Between can delete when disconnecting and created in the machine list factorylist under/factory directory nodes The server directory node, child node change monitor when listening to directory node and changing, The value that field Reload is reloaded described in each server is set to into true；The configuration variation Monitor be used to monitoring in zookeeper /strategy directory nodes in need what is performed in distributed system Specific strategy, thread of the configuration variation monitor required for specific strategy or specific strategy is listened to When changing, the value that field Reload is reloaded described in each server is set to into true.

The monitor refers to a kind of event monitor program, and when an event is produced, event source will send Notification message simultaneously provides an event object as parameter, and the message is received by event monitor program, and is carried out Relevant treatment, event source is that the watcher mechanism based on curator and zookeeper is realized.

In the specific implementation, when listen to change in zookeeper systems (event source) when, can send One event (event), this event can by above-mentioned connection status monitor, child node change monitor or A kind of monitor program in configuration variation monitor is received, and carries out respective handling.This monitor journey The sequence moment is ceaselessly run, once zookeeper systems send event, will timely receive, to ensure Server logic and zookeeper systems compliants.

Step S603, according to the number of servers registered in zookeeper systems, and need The specific strategy to be performed, recalculating each server needs the sets of threads quantity of distribution.

In the present embodiment, it is described according to the number of servers registered in zookeeper systems, and The specific strategy for needing to perform, recalculating each server needs the sets of threads quantity of distribution, can be with Realize in the following way：Concrete executive mode according to expressed by the specific strategy, and each is concrete Requirement of the executive mode to sets of threads quantity, calculating needs the sets of threads quantity distributed in each server.

For example：It is to the sets of threads quantity of requirement in the server A in the specific strategy of one description server A 3。

It should be noted that the method provided according to first embodiment carries out configuration can also be considered as a kind of strategy, Referred to as basic scheme.

Step S605, unit is provided using the server registered in zookeeper systems as resource, Using the sets of threads for needing distribution as resource consumption unit, and usage right requires 1-5 any one The described dispatching method for distributed system, is that server configures corresponding sets of threads, including distribution is newly Sets of threads, and stop the sets of threads that has been started up.

Specifically, when using described in first embodiment for the dispatching method of distributed system, it is described according to The number of servers registered in zookeeper systems, and the specific strategy for needing to perform, Recalculating each server needs the sets of threads quantity of distribution, including：

Scheduling component obtain in zookeeper systems register number of servers and in zookeeper systems The serial number of the server of middle registration；

The specific strategy for performing is needed to obtain the sets of threads number that distribution is needed when this is dispatched by described；Institute State sets of threads to be sequentially numbered；

By the sets of threads number for needing distribution divided by the server registered in zookeeper systems Quantity, obtains the sets of threads number N that each server should be allocated, and N is the numerical value after rounding；

The server registered in zookeeper systems is according to the service registered in zookeeper systems The serial number M and the principle of mean allocation of device, according to the order that the sets of threads is numbered, from the thread The sets of threads that quantity is N either (N-1) or (N+1) is extracted in group, as distributing in zookeeper The sets of threads of each server registered in system.

During in order that task items or sets of threads changing in the distributed system, scheduling component can be according to change Change carries out redistributing and loading tasks item again to each task items, and the technical scheme of the present embodiment is provided A kind of preferred embodiment, under preferred embodiment, each sets of threads of the distributed system is respectively arranged with To should the loading of sets of threads judge field Needload, and there is respectively described each sets of threads interval to fix Time just reads the sets of threads finger daemon of the value for loading and judging field Needload.

In the present embodiment, the finger daemon every five seconds for example carries out single pass, reads Boolean Class in the server The value of the Needload fields of type, if the value of the Needload fields for scanning is true, enters The sets of threads quantity according to the registration in zookeeper systems, and to be processed of the need Business item, the step of calculating each sets of threads and need the task items quantity of distribution；And the loading is judged into field The value of Needload is set to false；If the value of the Needload fields for scanning is false, no Any operation is carried out, treats 5 seconds to be scanned next time afterwards.

It should be noted that when distributing task items between sets of threads, being allocated on the server with sets of threads Strategy be similar to, difference is, finger daemon here is carried out for the Needload in each sets of threads Scanning, therefore not to repeat here.

In the present embodiment, after the step of the configuration corresponding sets of threads for server, step is specifically included Rapid S606 to S607,7 is described further below in conjunction with the accompanying drawings.

Fig. 7 is refer to, to be it illustrates and configure corresponding for each sets of threads according to what embodiments herein was provided The flow chart of task items.

Step S606, according to the sets of threads quantity registered in zookeeper systems, and described treats The task items of process, calculating each sets of threads needs the task items quantity of distribution；

Step S607, provides single using the sets of threads quantity registered in zookeeper systems as resource Position, using need task items to be processed as resource consumption unit, and usage right requires that 1-5 is any The dispatching method for distributed system described in one, is that each sets of threads configures corresponding task items, bag Include and the allocated task items are redistributed, new task items is distributed for process group and is eliminated the need for stopping Task items.

Specifically, when using described in first embodiment for the dispatching method of distributed system, it is described according to The sets of threads quantity registered in zookeeper systems, and the pending task items, calculate Each sets of threads needs the task items quantity of distribution, including：

Scheduling component obtain in zookeeper systems register sets of threads quantity and in zookeeper systems The serial number of the sets of threads of middle registration；

The number of the pending task items；The pending task items are sequentially numbered；

By the number of the pending task items divided by the sets of threads registered in zookeeper systems Quantity, obtains the task items number N that each sets of threads should be allocated, and N is the numerical value after rounding；

The sets of threads registered in zookeeper systems is according to the thread registered in zookeeper systems The serial number M of group and the principle of mean allocation, according to the order that the pending task items are numbered, from The task items that quantity is N either (N-1) or (N+1) are extracted in the pending task items, as Distribute to the task items of each sets of threads registered in zookeeper systems.

In order to ensure to be disconnected between each server registered in zookeeper systems and zookeeper After can normally continue executing with scheduler task, in the technical program provide a kind of preferred embodiment, Under preferred embodiment, the control information stored in the zookeeper systems is stored in server local, And the foundation of distributed task dispatching is performed as the server using the control information of server local storage；If There is abnormal, then directly to store with server local control information execution distributed task scheduling in zookeeper systems Scheduling.

If it should be understood that run into and zookeeper disconnections, or zookeeper delays the abnormal conditions such as machine, Current scheduler task is continued executing with by being buffered in the control information of server local, scheduling can not be stopped Component after being attached with zookeeper, then re-executes the reading of step S601 so as to continue executing with The step of taking the control information stored in zookeeper systems.

In the present embodiment, due to a certain registered server and zookeeper disconnections in a distributed system During connection, it is possible to have new server registration in the distributed system, so carrying in the technical program A kind of preferred embodiment is supplied, under preferred embodiment, corresponding task has been configured for each sets of threads described After the step of item, specifically include：

Read the control information version number of zookeeper；

Obtain the control information version number of server；

In the present embodiment, can be in zookeeper when server is registered in distributed system The directory node of the server is created in machine list factorylist under/factory directory nodes, together The reason of sample, can delete under/factory directory nodes when disconnecting between server and zookeeper Machine list factorylist in the directory node of the server that creates, in every time/factory catalogues Node can update the version number of the control information of zookeeper when changing, zookeeper updates control letter To with zookeeper the server being connected can be kept to send the control after updating after the version number of breath version number Information version number, the version number of each server sync control information for making to keep being connected with zookeeper, Zookeeper is set to be consistent with the version of each server.For example：Make the control information of zookeeper Version number numerical value+1.

In order that the backed off after random distributed system that each server can be completed in scheduling, in the technical program There is provided a kind of preferred embodiment, under preferred embodiment, each server is provided with server running mark Field isRunning；The server running mark field is true in the case where server normally runs； When the server needs to log off, then server running mark field isRunning is set and is set to False, and all threads for stopping server initiation thread, halt system daemon thread, stopping server The connection of group, log-on server and zookeeper.

In the above-described embodiment, there is provided another kind is used for for the dispatching method of distributed system with above-mentioned The dispatching method of distributed system is corresponding, and the application the 5th implements to provide one kind for distributed system Dispatching device.Because the embodiment of device is substantially similar to the embodiment of method, so describing simpler Single, related part is illustrated referring to the part of embodiment of the method.Device embodiment described below is only Schematically.The dispatching device embodiment for distributed system is as follows：

Fig. 8 is refer to, the scheduling for distributed system provided according to the embodiment of the application the 5th is provided The schematic diagram of device.

The dispatching device for distributed system, including：Control information reading unit 801, recalculate Unit 803 and sets of threads allocation unit 805；

The control information reading unit 801, for reading zookeeper systems in store control information； The control information is included in the server registered in zookeeper systems, and needs in distributed system The specific strategy of execution, the specific strategy describes different concrete executive mode and each concrete side of execution Requirement of the formula to sets of threads quantity；

It is described to recalculate unit 803, for according to the server count registered in zookeeper systems Amount, and the specific strategy for needing to perform, recalculating each server needs the sets of threads number of distribution Amount；

The sets of threads allocation unit 805, for the server registered in zookeeper systems to be made Unit is provided for resource, using the sets of threads for needing distribution as resource consumption unit, and usage right will The dispatching method for distributed system described in 1-5 any one is sought, is that server configures corresponding line Journey group, including distribute new sets of threads, and stop the sets of threads having been started up.

Optionally, the control information reading unit 801, the control information of reading also includes： The sets of threads registered in zookeeper systems, and pending task items；

The dispatching device for distributed system, also includes：

Optionally, the dispatching device for distributed system, including：Control information memory cell；

The control information memory cell, for the control information storage that will be stored in the zookeeper systems In server local, and distributed task scheduling is performed as the server using the control information of server local storage The foundation of scheduling；If there is abnormal, directly to store with server local control letter in zookeeper systems Breath performs distributed task dispatching.

Optionally, the dispatching device for distributed system, including：Control information reads trigger element；

The control information reads trigger element, for the control stored in the reading zookeeper systems Before information, if guarding line with what fixed time intervals just read the value for reloading field Reload The value for reloading field Reload that journey reads is true, then trigger the control information and read list Unit；And the value for reloading field Reload is set to into false.

Optionally, the dispatching device for distributed system, including：Monitoring unit；

The monitoring unit, for monitoring the zookeeper systems stays；If there is following arbitrarily item Described in then arranging field Reload is reloaded for true：Zookeeper system reboots, zookeeper systems System configuration variation, the control information change of zookeeper systems storage, server initiation.

Optionally, the dispatching device for distributed system, including：Task items calculate trigger element；

The task items calculate trigger element, if for the sets of threads finger daemon read described in again plus The value for carrying field Needload is true, then enter the triggering task items computing unit；And by the loading The value for judging field Needload is set to false.

Optionally, the dispatching device for distributed system, including：Version number's reading unit, version number Acquiring unit, version number's judging unit and reload field modification unit；

Version number's reading unit, for it is described configure corresponding task items for each sets of threads after, read Take the control information version number of zookeeper；

Version number's acquiring unit, for obtaining the control information version number of server；

Version number's judging unit, for judging the whether version number with zookeeper of the version number of server It is identical；

It is described to reload field modification unit, for receiving the judged result of version number's judging unit, if It is no, then read and be stored in field Reload of reloading of server local, and reload field by described Value be revised as true.

Optionally, the dispatching device for distributed system, including：Server stop element；

The server stop element, for when server needs to log off, then arranging server operation mark Will field isRunning is set to false, and stop server initiation thread, halt system daemon thread, Stop the connection of all sets of threads, log-on server and zookeeper of server.

In the above-described embodiment, there is provided a kind of dispatching method for distributed system, a kind of for being distributed The dispatching device of formula system, a kind of electronic equipment, another kind of dispatching method for distributed system and another A kind of dispatching device for distributed system, additionally, present invention also provides another kind of electronic equipment；Institute State electronic equipment embodiment as follows：

Fig. 9 is refer to, the flow process of another kind of electronic equipment provided according to the application sixth embodiment is provided Figure.

The electronic equipment, including：Display 901；Processor 903；Memory 905；

The memory 905, for the scheduler program of distributed storage system, described program is by the process Device is read when performing, and performs following operation:Read the control information stored in zookeeper systems；The control Information processed is included in the server registered in zookeeper systems, and needs what is performed in distributed system Specific strategy, the specific strategy describes different concrete executive mode and each concrete executive mode to line The requirement of journey group quantity；It is and described according to the number of servers registered in zookeeper systems The specific strategy for performing is needed, recalculating each server needs the sets of threads quantity of distribution；By it is described The server registered in zookeeper systems provides unit as resource, needs the sets of threads of distribution to make by described For resource consumption unit, and usage right requires the tune for distributed system described in 1-5 any one Degree method, is that server configures corresponding sets of threads, including distributes new sets of threads, and stops having opened Dynamic sets of threads.

In a typical configuration, computing device includes one or more processors (CPU), input/output Interface, network interface and internal memory.

Internal memory potentially includes the volatile memory in computer-readable medium, random access memory (RAM) And/or the form, such as read-only storage (ROM) or flash memory (flash RAM) such as Nonvolatile memory.Internal memory is The example of computer-readable medium.

1st, computer-readable medium includes that permanent and non-permanent, removable and non-removable media can be by Any method or technique is realizing information Store.Information can be computer-readable instruction, data structure, journey The module of sequence or other data.The example of the storage medium of computer includes, but are not limited to phase transition internal memory (PRAM), static RAM (SRAM), dynamic random access memory (DRAM), other classes The random access memory (RAM) of type, read-only storage (ROM), Electrically Erasable Read Only Memory (EEPROM), fast flash memory bank or other memory techniques, read-only optical disc read-only storage (CD-ROM), numeral Multifunctional optical disk (DVD) or other optical storages, magnetic cassette tape, the storage of tape magnetic rigid disk or other magnetic Storage device or any other non-transmission medium, can be used to store the information that can be accessed by a computing device.Press Define according to herein, computer-readable medium does not include non-temporary computer readable media (transitory Media), such as the data-signal and carrier wave of modulation.

2nd, it will be understood by those skilled in the art that embodiments herein can be provided as method, system or computer Program product.Therefore, the application can adopt complete hardware embodiment, complete software embodiment or combine software With the form of the embodiment of hardware aspect.And, the application can be adopted and wherein include meter at one or more Calculation machine usable program code computer-usable storage medium (including but not limited to magnetic disc store, CD-ROM, Optical memory etc.) on implement computer program form.

Although the application is disclosed as above with preferred embodiment, it is not for limiting the application, Ren Heben Art personnel can make possible variation and modification in without departing from spirit and scope, Therefore the scope that the protection domain of the application should be defined by the application claim is defined.

Claims

1. a kind of dispatching method for distributed system, it is characterised in that include：

2. the dispatching method for distributed system according to claim 1, it is characterised in that described It is machine in a distributed system that resource provides unit, and the resource consumption unit is to process to be allocated appointing The sets of threads of business item.

3. the dispatching method for distributed system according to claim 1, it is characterised in that described Resource provides the sets of threads that unit is the task items for processing to be allocated, and the resource consumption unit is to be allocated Task items.

4. the dispatching method for distributed system according to claim 1, it is characterised in that if institute State resource consumption unit and averagely undertake the mode that rounds of quantity N and be:Remove the resource consumption Board Lot divided by Resource in the distributed system provides the decimal place that Board Lot obtains value；

5. the dispatching method for distributed system according to claim 1, it is characterised in that if institute State resource consumption unit and undertake the mode that rounds of quantity N and be:The resource consumption Board Lot is divided by the distribution Resource in formula system provides Board Lot value, if there is decimal place, takes more than the acquisition value Smallest positive integral；

6. a kind of dispatching device for distributed system, it is characterised in that include：

7. the dispatching device for distributed system according to claim 6, it is characterised in that if institute State and undertake amount calculation unit, specifically for removing the resource consumption Board Lot divided by the distributed system Resource in system provides the decimal place that Board Lot obtains value, then the resource consumption unit allocation unit, Including：Round computation subunit, remainder computation subunit, first resource consume unit distribution subelement with And Secondary resource consumes unit distribution subelement；

8. the dispatching device for distributed system according to claim 6, it is characterised in that if institute State and undertake amount calculation unit, if specifically for there is decimal place, taking the smallest positive integral more than the acquisition value, The then resource consumption unit allocation unit, including：Round computation subunit, redundant digit computation subunit, First resource consumes unit distribution subelement and Secondary resource consumes unit distribution subelement；

9. a kind of electronic equipment, it is characterised in that the electronic equipment includes：

Display；

Processor；

10. a kind of dispatching method for distributed system, it is characterised in that include：

11. dispatching methods for distributed system according to claim 10, it is characterised in that institute Stating control information also includes：The sets of threads registered in zookeeper systems, and pending task items；

12. dispatching methods for distributed system according to claim 11, it is characterised in that institute State the control information stored in zookeeper systems and be stored in server local, and deposited with server local The control information of storage performs the foundation of distributed task dispatching as the server；If zookeeper systems go out Now abnormal, then the control information for directly being stored with server local performs distributed task dispatching.

13. dispatching methods for distributed system according to claim 10 or 11, its feature It is, before the control information stored in the reading zookeeper systems, including：

Each server have reload field Reload；

14. dispatching methods for distributed system according to claim 13, it is characterised in that bag Include：

The zookeeper systems stays are monitored；Described reloading is set if there is following arbitrarily item Field Reload is true：Zookeeper system reboots, the change of zookeeper system configurations, zookeeper The control information change of system storage, server initiation.

15. dispatching methods for distributed system according to claim 10 or 11, its feature Be, each sets of threads of the distributed system be respectively arranged with to should the loading of sets of threads judge field Needload；

16. dispatching methods for distributed system according to claim 12, it is characterised in that It is described the step of configure corresponding task items for each sets of threads after, including：

Read the control information version number of zookeeper；

Obtain the control information version number of server；

17. dispatching methods for distributed system according to claim 14, it is characterised in that each Individual server is provided with server running mark field isRunning；The server running mark field is in service Device is true in the case of normally running；When the server needs to log off, then server fortune is set Line flag field isRunning is set to false, and stops server initiation thread, halt system and guard The connection of thread, all sets of threads, log-on server and zookeeper of stopping server.

18. a kind of dispatching devices for distributed system, it is characterised in that include：

19. dispatching devices for distributed system according to claim 18, it is characterised in that institute Control information reading unit is stated, the control information of reading also includes：Register in zookeeper systems Sets of threads, and pending task items；

The dispatching device for distributed system, also includes：

20. dispatching devices for distributed system according to claim 19, it is characterised in that bag Include：Control information memory cell, for the control information stored in the zookeeper systems to be stored in Server local, and distributed task scheduling tune is performed as the server using the control information of server local storage The foundation of degree；If there is abnormal, directly to store with server local control information in zookeeper systems Perform distributed task dispatching.

21. a kind of electronic equipment, it is characterised in that the electronic equipment includes：

Display；

Processor；