CN103188345B - Distributed dynamic load management system and method - Google Patents
Distributed dynamic load management system and method Download PDFInfo
- Publication number
- CN103188345B CN103188345B CN201310065185.XA CN201310065185A CN103188345B CN 103188345 B CN103188345 B CN 103188345B CN 201310065185 A CN201310065185 A CN 201310065185A CN 103188345 B CN103188345 B CN 103188345B
- Authority
- CN
- China
- Prior art keywords
- load
- processing node
- node
- processing
- event
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Abstract
A kind of distributed dynamic load management system and method, system is made up of the distributed treatment cluster that has dynamic load management function and have a complete reciprocity framework of P2P, this Processing Cluster does not have central controlled Centroid, only have multiple processing nodes, each processing node can independently provide the dynamic load managerial ability of distributed arbitration program, and authority is identical with dynamic load management function separately, but it is identical or different to process event capability; Each processing node is the control core of this dynamic load management system, and is all provided with event handling, action listener, case distribution, four modules of dynamic load management. System of the present invention can realize distributed dynamic load management under distributed network environment, effectively solves data processing request focus and changes the system load that brings and shift and when fluctuation and node dynamically change, ensure system load balancing; While also supporting that under P2P environment, the each processing node of distributed cluster system all transships, can effectively ensure the robustness of system.
Description
Technical field
The present invention relates to a kind of distributed dynamic load management system and method, belong to the technical field of computer.
Background technology
Along with the rapid growth of network traffic, visit capacity and data traffic, end-to-end P2P (Peer-to-Peer)Load management technology under peer to peer technology environment has become urgent problem. At present, P2P framework ringLoad-balancing technique in border becomes the focus of research already, and existing a lot of technical schemes can be optimized P2P systemThe load-balancing performance of system.
As everyone knows, P2P system is divided into structuring and the large class of destructuring two, based on distributed hash table(DHT) structured P 2 P system has solved the resource orientation problem of destructuring system. But, at knotIn structure P2P system, data processing request and node by uniformity hash function by Stochastic Equilibrium shine uponTo a logic box; The problem of load balancing day bringing due to some characteristics of data processing request taskBenefit is outstanding. Research shows, in real time mass data processing request information, most request tasks are depositedIn hot issue, roughly obey Zipfp and distribute: in certain time period, 60%～80% data processing pleaseAsk by uniformity hash function and can be mapped on certain or certain several nodes, cause some in P2P systemNode load is very high, and other node load is very light; And, different time sections data processing request taskFocus has modificability, thereby causes the overall laod unbalance of whole system, the treatment effeciency of system andStability reduces.
In order effectively to solve data processing request hot issue in structured P 2 P system, existing program causes at presentIn solving the communication overhead how reducing in load transfer, (Hash ring value and processing node contrast relationship change powerCommunication cost) problem. But because data processing request focus has modifiability, they do not examineConsider data processing request focus and change the fluctuation of the system load transfer bringing, especially at nodes measurerHave in the environment of certain scale, fluctuation can be stronger, and the communication cost of bringing can become greatly, thus impactThe load balancing rate of convergence of system, and the GSLB of system. Again, these schemes all do not haveHave and consider the overload control problem of structured P 2 P system under the full overload situations of node, when system data processingWhen request amount exceedes the maximal workload of system, can cause the entirety collapse of system, affect the stability of systemAnd the consequence of bringing on a disaster property. Therefore the dynamic management approach that, how to improve load also becomes study hotspot.
For data processing request hot issue and system load balancing in effective solution structured P 2 P system are askedTopic, current prior art scheme mainly contains several as follows:
" implementation of load balancing of distributed hashtable network and device " (Chinese patent application publication number:CN102457429A) provide a kind of dynamic load equilibrium technology shifting based on dummy node. The method headFirst, according to node planning and the default load imbalance degree threshold value of described DHT network, determine the first virtual markThe quantity Z (Z is natural number) knowing, and the whole load space of this DHT network is divided into Z part, each partLoad space identifies with mutually different described the first virtual identifying respectively; Again according to node described in eachAbility, Z the first virtual identifying is distributed to each node, be responsible for distributing to its first by each nodeThe load space that virtual identifying is corresponding. The present invention can realize the same of DHT network data memory load equilibriumTime, reduce the workload that virtual identifying management and virtual identifying move. The method can effectively reduce virtual markThe workload of knowledge management and virtual identifying migration, still, does not provide to solve and asks focus change to bringFluctuation problem is shifted in system load, and system robustness problem under the full overload situations of system, pointUnder cloth formula cluster environment, application has certain limitation.
It is a kind of that " SiteServer LBS and method " (Chinese patent application publication number: CN102436401A) providesThe load-balancing technique of multistage load strategy. The SiteServer LBS of its proposition comprises: first order load balancingDevice, multiple second level load equalizer and multiple server. Wherein, first order load equalizer is used for connecingReceive multiple tasks, and use the first strategy that described multiple tasks are dispensed to multiple second level load equalizer;Multiple second level load equalizer is used for using the second strategy that the plurality of task is dispensed to multiple servers. ItsIn, introduce stratification concept and adopt multistage load-balancing method, can reduce becoming property of load balancing centerThe possibility of energy bottleneck is supported more complicated load balancing simultaneously, has promoted the spirit of load balancing processActivity, accuracy, robustness and security. But, when it does not consider processing node dynamic change, bringLoad strategy change overhead issues, do not provide the system how solving under the full overload situations of system strong yetThe technical scheme of strong property problem.
Summary of the invention
In view of this, the object of this invention is to provide a kind of improved distributed dynamic based on DHT principle bearsCarry management system and method, the present invention realizes distributed dynamic load management under distributed network environmentSystem, can effectively solve data processing request focus and change the system load transfer fluctuation problem of bringingWhile dynamically change with node, ensure the problem of system load balancing; Also support distributed type assemblies system under P2P environmentIn the situation that each processing node of uniting all transships, can effectively ensure the robustness problem of system.
In order to achieve the above object, the invention provides a kind of distributed dynamic load management system, its featureBe: described system is by having dynamic load management function and having end-to-end P2P (Peer-to-Peer)The distributed treatment cluster of complete reciprocity framework forms, and this Processing Cluster does not have central controlled Centroid,Only be provided with multiple processing nodes, wherein each processing node can independently provide the dynamic load of distributed arbitration programManagerial ability, and its function that authority is managed with dynamic load is separately identical, but the ability of the event of processingIdentical or different; Each processing node is the control core of this dynamic load management system, and is all provided with downState four functional modules: event processing module, action listener module, case distribution module, dynamic load pipeReason module; The function of modules is as follows:
Event processing module, for the treatment of the event of particular type and attribute, produces intermediate treatment result orWhole result, and by case distribution module, the new events producing is transferred to this processing node or mail to otherProcessing node continues to process, or final process result is returned to client; This event processing module is that this isIn system, preset, or develop and be deployed on processing node by third party programmer;
Action listener module, is responsible for receiving the data processing request event from client, and from selfOr the event of other processing node; And accept the dynamic load administrative decision control of dynamic load administration moduleInstruction: in the time receiving load transfer instruction, event is sent to respectively to the processing node of specifying in this instruction;In the time receiving overload control instruction, produce the response message of system overload, and send to client; OrTransfer to event processing module to re-start processing event;
Dynamic load administration module, is provided with two parts of load monitoring unit and load decision package, wherein:Load monitoring unit action listener module assist under, in Real-time Collection and distributed storage Processing Cluster everywhereThe information of survival condition, pay(useful) load rate and the comprehensive treatment capability of reason node, and real-time update and storage warpDummy node mapping table and dynamic load information table after the establishment of load decision package or maintenance; Load decision-makingThe information of survival condition, pay(useful) load rate and comprehensive treatment capability based on each processing node is responsible in unit, withAnd the dummy node mapping table of storage, formulate for the load balancing of event handling and case distribution process,The dynamic load administrative decision control instruction that load is shifted and overload is controlled;
Case distribution module, is responsible for accepting the dynamic load administrative decision control that dynamic load administration module sendsInstruction, the new events being produced by event processing module that it is received or final process result execution event are dividedSend out and process: in the time receiving load balancing instruction, the new thing being produced by event processing module that it is receivedPart or final process result send to respectively this specified processing node of this instruction action listener module orOther processing node or client.
In order to achieve the above object, the present invention also provides a kind of employing distributed dynamic load management of the present inventionThe distributed dynamic load management method of system based on P2P technology, is characterized in that: when client is to this pointWhen cloth formula Processing Cluster sends mass data processing request in real time, certain processing node is received after request, first rootJudge according to the pay(useful) load rate of this node and overloading threshold that this processing node is current and whether transship, if overload,Carrying out load transfer or overload controls; Otherwise, process this request; Complete request process after, just by generationIntermediate object program is encapsulated as event, and equal according to dummy node mapping table and this event determinant attribute value execution loadWeighing apparatus decision-making, determines that this event still mails to other processing node by self node and continues to process; And follow-up eachProcessing node, in the time processing and distribute intermediate event, is all according to pay(useful) load rate and overloading threshold or voidPlan node mapping table and event determinant attribute value are carried out load transfer, overload control or event equilibrium distributionOperation, until produce final result, and return to client.
The innovative technology feature of system of the present invention is: realize by increasing dummy node layer based on DHT principleDistributed load equalizing, and by data processing request or event are carried out to probability ground re-orientation processes nodeNumbering realize load and shift, and abandon and realize overload and control by probability overload.
In virtual node conception in system of the present invention and DHT algorithm, the Hash ring value of uniformity Hash has phaseLike part, be all to arrive the mapping of processing node and become according to processing node comprehensive treatment capability by dummy nodeBe uniformly distributed to ratio, realize thus the load management on the first level of system of the present invention; Pass through again probabilityProperty processing node numbering be redirected, can solve the caused processing node of data processing request hot issue negativeCarry unbalanced problem, thereby realize the load management in the second aspect of system of the present invention. And pass through probabilityProperty overload abandons, and can solve distributed treatment cluster overall situation load overload problem, thereby realize of the present inventionLoad management in the 3rd aspect.
Load-balancing technique in P2P architecture environment is the focus of research already, and existing a lot of technical schemes are excellentChange the load-balancing performance of P2P system. Than these schemes, the present invention has more advantage, and its reason is:
Solving in structured P 2 P system in data processing request hot issue, the scheme of currently available technologyBeing devoted to solve the communication overhead how reducing in load transfer (is Hash ring value and processing node contrast relationshipThe communication cost changing) problem, do not consider that data processing request focus changes institute and causes that system load turnsThe fluctuation problem of moving. System of the present invention, in load transfer process, is not changed dummy node and processing nodeBetween mapping relations, just probability ground is redirected processing node numbering, according to setting probabilityFrom the set of underload processing node, (this K numerical value is that of setting in system is less to select at random KNatural number) small set of individual processing node composition, then from this small set, elect pay(useful) load rateA little processing node, and numbered as re-orientation processes node serial number. Thereby system energy of the present inventionEnough effectively solve data processing request hot issue and focus in structured P 2 P system and change the system causingFluctuation problem is shifted in load.
Moreover, in the scheme of load-balancing performance of optimizing P2P system, also do not find that there is at present technologySystem robustness problem under the full overload situations of scheme taking into account system. System of the present invention is at the full overload situations of systemUnder, adopting probability ground overload drop policy, the robustness that can effectively solve system under full overload situations is askedTopic. Thereby the distributed dynamic load management method of system of the present invention is at present existing DHT implementation methodOne improve. Therefore, popularizing application prospect of the present invention is good.
Brief description of the drawings
Fig. 1 is distributed terminator management system general structure composition schematic diagram of the present invention.
Fig. 2 is the processing node functional module composition schematic diagram in distributed terminator management system of the present invention.
Fig. 3 is the distributed dynamic load management side of distributed terminator management system of the present invention based on P2P technologyMethod operating procedure flow chart.
Fig. 4 is the flow chart that load management system of the present invention is carried out distributed dynamic load management case distribution.
Fig. 5 is the establishment renewal flow chart that load management system of the present invention is carried out distributed load equalizing table
Fig. 6 is that load management system of the present invention is carried out the case distribution flow chart that distributed dynamic load is shifted.
Fig. 7 is that load management system of the present invention is carried out the case distribution flow chart that distributed dynamic overload is controlled.
Detailed description of the invention
For making the object, technical solutions and advantages of the present invention clearer, below in conjunction with accompanying drawing, the present invention is doneFurther describe in detail.
Referring to Fig. 1, introduce the present invention there is dynamic load management function and there is end-to-end P2P completely rightStructure composition Deng the distributed processing system(DPS) of framework: this system is without Centroid (being centralized control node),But be provided with multiple processing nodes, each processing node can independently provide the load management ability of distributed arbitration program,And authority is identical with the function of dynamic load management separately for it, therefore completely reciprocity between each processing node;But it is identical or different that each processing node is processed the ability of event; Each processing node is this dynamic load pipeThe control core of reason system, and be divided on the server of different server. In actual motion environment is disposed,Same physical server can arrange and move multiple processing nodes.
Referring to Fig. 2, introduce four of the inside functional module of processing node:
(1) action listener module: be responsible for receiving the data processing request event from client, and fromThe event of the processing node of self or other; And accept the dynamic load administrative decision of dynamic load administration moduleControl instruction: in the time receiving load transfer instruction, event is sent to respectively to the processing of specifying in this instructionNode; In the time receiving overload control instruction, produce the response message of system overload, and send to client;Or transfer to event processing module to re-start processing event.
(2) event processing module: for the treatment of the event of particular type and attribute, produce intermediate treatment resultOr final process result, and by case distribution module, the new events producing is transferred to this processing node or mail toOther processing nodes continue to process, or final process result is returned to client; This event processing module isIn this system, preset, or develop and be deployed on processing node by third party programmer.
(3) dynamic load administration module: be responsible for creating with maintenance event and monitor module and case distribution module instituteNeed load information and load decision information is provided, be provided with two portions of load monitoring unit and load decision packagePart, wherein: load monitoring unit action listener module assist under, Real-time Collection and distributed storage processingThe information of the survival condition of each processing node, pay(useful) load rate and comprehensive treatment capability in cluster, and in real time moreNew dummy node mapping table and dynamic load information with storing after load decision package creates or safeguardsTable; Load decision package is responsible for survival condition, pay(useful) load rate and the integrated treatment energy based on each processing nodeThe information of power, with and the dummy node mapping table of storage, formulate for event handling and case distribution processIn the dynamic load administrative decision control instruction that load balancing, load shift and overload is controlled.
The dynamic load administrative decision function declaration of load decision package is wherein as follows:
Survival condition and the General Office of the each processing node of distributed treatment cluster providing according to load monitoring unitReason ability information, in the time of system initialization or processing node generation dynamic change, real-time servicing dummy node reflectsFiring table and dynamic load information table, adjust the load of each processing node in distributed treatment cluster, i.e. virtual jointPoint number, so that respectively process node load balancing;
When causing processing node pay(useful) load because of the hot issue of processing and distributing data processing request or eventWhen rate is unbalanced, take load transition strategy, so that respectively process node load balancing;
In the time that each processing node pay(useful) load rate is all transshipped, take to transship control strategy, so that distributed treatmentCluster keeping system stability and robustness.
(4) case distribution module: be responsible for accepting the dynamic load administrative decision that dynamic load administration module sendsControl instruction, the new events being produced by event processing module that it is received or final process result are carried out thingPart distribution processor: when receiving load balancing instruction, what it was received is produced by event processing moduleNew events or final process result send to respectively the action listener mould of this specified processing node of this instructionPiece or other processing node or client.
Case distribution module, for distributed dynamic load management method can be provided, has defined following several heavyWant concept:
Processing node pay(useful) load rate: be the pay(useful) load rate of the last load information collection period of processing nodeWith the value after the real-time load factor weighting in this cycle. The actual negative carrying capacity that this real-time load factor is processing node withThe ratio of its comprehensive treatment capability, wherein, the actual negative carrying capacity of processing node comprises disappearing of CPU and internal memoryConsumption, or the length of pending event queue; The comprehensive treatment capability of processing node comprise cpu function,The maximum permissible value of the spatial content of internal memory or pending event queue length; Utilize the effective of processing nodeLoad factor can effectively be processed the isomerism problem in distributed system, and by processing node according to pay(useful) load rateBe divided into following three ranks: underload processing node, normal load processing node and overload processing node.
Dummy node: be the concept that provides distributed load equalizing method to introduce. Dummy node is also untrueExist, just a logical symbol mutually shining upon with processing node. Each dummy node can only be mapped toA processing node, but each processing node can be mapped to multiple dummy nodes. The quantity of dummy node isIn the time that system starts, configure, its quantity is not less than 2 times of processing node number; And after system operation,The quantity of dummy node remains unchanged, in the time that the processing node in system increases or shifts out, and processing node and voidIntend internodal mapping relations and carry out corresponding change according to distributed dynamic load management.
Dummy node mapping table: be a kind of significant data that system of the present invention realizes distributed dynamic load managementStructure, for storing the mapping relations of each dummy node symbol and its processing node numbering. Each processing jointPoint has dummy node mapping table separately, and in distributed load equalizing method of the present invention, system is by looking intoLook for dummy node mapping table can search out from dummy node the processing node (physical node) of its correspondence mappings,Realize distributed load equalizing.
Dynamic load information table: be another significant data that system of the present invention realizes distributed dynamic load managementStructure, for storing each processing node management and dummy node symbol and the number thereof controlled, this is dynamically negativeCarrying information in information table and the information of dummy node mapping table is one to one.
The application scenarios of system of the present invention includes following four kinds:
Distribution Events scene: client sends mass data processing request in real time to distributed treatment cluster,Data processing request is received and dispatched when normal, and this cluster returns to result to client in real time, and ensures this clusterIn load balancing and the stability of each processing node; And in dynamic load management process, each processing nodeEqual, in this cluster without centralized load management node, each processing node can complete independentlyThe dynamic load management of distributed arbitration program;
The scene of processing node initialization and dynamic change: client sends sea in real time to distributed treatment clusterWhen amount data processing request, start in real time system or increase new processing node or shift out processing node to this cluster;And within short time as far as possible, again adjust in proportion its load according to the comprehensive treatment capability of each processing node,Make the each processing node in this cluster again reach load balancing;
Scene is shifted in processing node load: client sends mass data processing in real time to distributed treatment clusterIn request process, factor data is processed request and is had hot issue, cause the request that receives processing node effectivelyLoad factor overload, and also have processing node in the set of underload processing node time, this cluster is by probability groundRedirected data processing request mails to certain or some processing node in the set of underload processing node, so thatDynamically adjust in real time the load equilibrium of this cluster;
Processing node overload is controlled scene: client sends mass data processing in real time to distributed treatment clusterRequest, it is too huge that factor data is processed request quantity, causes the each processing node pay(useful) load rate in this clusterWhile all overload, each processing node in this cluster by probability abandon data processing request, and to clientThe response message of end answering system overload, so that the stability of this cluster of real-time ensuring and reliability.
Introduce the distributed dynamic load of distributed dynamic load management system of the present invention based on P2P technology belowManagement method: in the time that client sends mass data processing request in real time to this distributed treatment cluster, certainProcessing node receives to be needed after data processing request to be processed or event, first according to the pay(useful) load of this nodeWhether rate and overloading threshold judge that this processing node is current and transship, if overload carries out that load is shifted or overloadControl, ensure the stability of a system and load equilibrium; Otherwise, process this request. Complete request or eventAfter processing, just the intermediate object program of generation is encapsulated as to event, and closes according to dummy node mapping table and this eventKey property value is carried out load-balancing decision, determines that this event still mails to other processing node by self node and continuesContinuous processing; And follow-up each processing node is in the time processing and distribute intermediate event, is all according to pay(useful) load rateCarry out with overloading threshold or dummy node mapping table and event determinant attribute value that load is shifted, overload is controlledOr the operation of the balanced distribution of event, until produce final result, and return to client.
Referring to Fig. 3, introduce the concrete operation step of the inventive method:
Step 1, the dynamic load administration module of processing node creates or upgrades dummy node mapping table and dynamically bearCarry information table; Meanwhile, dynamic load administration module, under action listener module is assisted, is adopted according to setting cycleThe existing state of each processing node and real-time load information in concentrated-distributed cluster. This step content of operation is as follows:
(11) system is initially set up, and while creating dummy node mapping table and dynamic load information table, employing is pressedThe method that the WRR being directly proportional according to comprehensive treatment capability or Weighted random distribute initializes dummy node mappingTable; Or adopt Random assignment and ensure that equally distributed method initializes dummy node mapping table; Now,The dummy node number of distributing for the different each processing node of processing event capability should be differentiated.
(12) in the time increasing in this system or shift out processing node, equal for making system again reach as early as possible loadWeighing apparatus, dynamic load administration module will carry out real-time update to dummy node mapping table and dynamic load information table.
This step (12) comprises following content of operation:
(12a) the dynamic load administration module of each processing node detects that the processing node in this system has changeTime, this situation is informed to all processing nodes in this cluster.
(12b) for reduce system topological maintenance workload as far as possible, the dynamic load administration module of each processing nodeAccording to the initialization of processing node, the modification information that increases or shift out, adopt uniformity hash method in real time moreNew dynamic load information table and dummy node mapping table.
(13) system starts after operation, and the load monitoring unit in dynamic load administration module is at action listenerUnder module is assisted, gather the existing state of each processing node in distributed type assemblies and negative in real time according to setting cycleInformation carrying breath, and calculate its pay(useful) load rate according to the real-time load information of processing node, and by processing node respectivelyBe included in the set of underload processing node, the set of normal load processing node or the set of overload processing node.
This step (13) comprises following content of operation:
(13a) load monitoring unit is under action listener module is assisted, and the cycle gathers the real-time of each processing nodeLoad information; And in order to solve the isomerism of processing node, by all normalizings of real-time load information of processing nodeTurn to real-time load factor, calculate according to the real-time load information of each processing node and comprehensive treatment capability thereofThe real-time load factor in this this cycle of processing node.
(13b) load monitoring unit is according to the real-time load factor in the pay(useful) load rate in a upper cycle and this cycle,Adopt weighting scheme to calculate the pay(useful) load rate in this this cycle of processing node, and this processing node obtaining is hadEffect load factor value storage is in the load monitoring unit of dynamic load administration module.
(13c) load monitoring unit is first by the pay(useful) load rate score of this processing node and the load of defaultOverloading threshold compares judgement, if pay(useful) load rate score is greater than load overloading threshold, by this processing jointPoint is included in the set of overload processing node, finishes this step operation; Otherwise pay(useful) load rate score is littleIn load overloading threshold, carry out subsequent step (13d).
(13d) load monitoring unit is born the light of the pay(useful) load rate score of this processing node and default againCarry threshold value and compare judgement, if pay(useful) load rate is less than underload threshold value, this processing node is included into gentlyIn the set of load processing node; Otherwise pay(useful) load rate score is not more than load overloading threshold, just shouldProcessing node is included in the set of normal load processing node.
Step 2, needs event to be processed once the action listener module of processing node has listened to, by dynamicallyIn load management module, load decision package is according to dummy node mapping table, and load monitoring unit providesThe pay(useful) load rate information of each processing node is carried out dynamic load administrative decision, and formulates dynamic load management certainlyPlan control instruction, action listener module is carried out load transfer, overload control according to this control instruction, or by thingPart sends to event processing module to process. The content of operation of this step is as follows:
(21) once action listener module has listened to needs after event to be processed, dynamic load administration moduleIn load decision package and load monitoring unit carry out alternately, knowing the pay(useful) load rate of each processing nodeNumerical value, and judge whether the pay(useful) load rate of this processing node transships: if it does not transship, carry out follow-up stepSuddenly (22); If its overload, redirect execution step (23).
(22) load decision package without carrying out, shift or overload control strategy by load, by action listener moduleDirectly give event processing module by this event and process, finish this step operation.
(23) load decision package judges whether other processing node all transships, and if so, takes probabilityOverload drop policy abandon this event: make to prepare the processing node pay(useful) load rate that mails to higher,Its probability of carrying out the operation of overload drop policy is larger, controls real-time ensuring distributed treatment to realize overloadThe working stability of cluster and reliable; If not, also there is the processing node that effective load factor is lighter, takeProbability is redirected the processing node of this event, shifts to realize load; And in the time of transferring load, do not changeThe mapping relations of dummy node and processing node, just probability reselects the processing node numbering of this event,And this event is just processed and directly mail to new processing node without event processing module, processing node is compiledNumber be redirected according to the probability being directly proportional to this processing node pay(useful) load rate, and make this processing nodePay(useful) load rate is higher, and the probability that this processing node is carried out event re-orientation processes nodal operation is larger.
This step (23) comprises following content of operation:
(23a) load decision package judges whether other processing node all transships, and if so, carries out follow-up stepSuddenly (23b); If not, also there is the processing node that effective load factor is lighter, redirect execution step (23c);
(23b) load decision package produces the random decimal in (0, a 1) scope, and by this random decimal withThis processing node pay(useful) load rate score compares judgement: whether this random decimal is less than this processing node hasEffect load factor numerical value; If so, load decision package sends overload to action listener module and abandons instruction, thingPart is monitored module and is directly abandoned this event, and the response message of answering system overload, then at case distribution mouldPiece sends to client by this response message under assisting; If not, carry out subsequent step (23e);
(23c) load decision package produces the random decimal in (0, a 1) scope, and by this random decimal withThis processing node pay(useful) load rate score compares judgement: whether this random decimal is less than this processing node hasEffect load factor numerical value; If so, carry out subsequent step (23d); If not, redirect performs step (23e);
(23d) load decision package takes out one or more processing nodes in load node set on the lenient side at random,And one or more processing node numberings of therefrom finding out pay(useful) load rate score minimum reset as this eventTo destination, then, this processing node numbering and load transfer instruction are returned to action listener module;Action listener module, under the assistance of case distribution module, directly mails to this event this processing node numbering instituteThe processing node of representative is processed, to dynamically adjust in real time the load equilibrium of distributed treatment cluster;
(23e) action listener module is directly given event processing module by this event and is processed.
Step 3, event processing module completes after the processing of this event, produces intermediate event or finally processesAs a result, send to case distribution module; By load decision package in dynamic load administration module according to virtual jointPoint mapping table, intermediate event determinant attribute value are carried out dynamic load administrative decision, and formulate dynamic load managementDecision Control instruction, case distribution module is carried out case distribution operation according to this instruction, by intermediate event orWhole result is distributed to corresponding processing node or client, and realizes each processing node load balancing
The content of operation of this step is as described below:
(31) event processing module is finished dealing with after this event, and the intermediate event producing is belonged to according to its keyProperty value carry out uniformity Hash evaluate operation, and mould gets the dummy node number of this system, will to obtain preparationThe dummy node numbering of this intermediate event distribution, to mail to self or other processing node by this intermediate eventContinue to process.
(32) case distribution module is according to the dummy node numbering of the preparation distribution obtaining and dynamic load managementModule is carried out alternately, obtains this intermediate event want according to the dummy node mapping table in dynamic load administration moduleThe processing node numbering mailing to.
(33) the processing node number information that case distribution module will mail to according to this intermediate event, by itContinue to process toward respective handling node, until the event of generation final process result directly returns to client.
The present invention has carried out repeatedly implementing test, and system embodiment execution point of the present invention is described respectively belowCloth formula dynamic load management method in Distribution Events scene, processing node initialization, dynamic change scene, locateReason node load shifts scene and processing node overload is controlled the implementing procedure under scene.
Referring to Fig. 4, introduce the distributed dynamic load management case distribution flow process of Distribution Events scene, it is rightWhat answer is the conventional scene that native system carries out mass data processing, and data processing request does not exist focus and numberAmount does not exceed in the scene of maximal workload of system. Its operating procedure is as described below:
(1) the action listener module of processing node listens to new need event to be processed. It should be noted that:Each processing node in this distributed processing system(DPS) is all listening to the event that needs processing at any time.
(2) action listener module is obtained decision instruction request to load decision-making module.
(3) load decision-making module and load monitoring module are mutual, the pay(useful) load of obtaining the each processing node of systemRate information.
(4) load decision-making module, according to the each processing node pay(useful) load rate information obtaining, is learnt distributed placeThe reason each processing node pay(useful) load of cluster rate is normal, without carrying out load transfer or overload control strategy.
(5) action listener module sends to event processing module by this event and processes;
(6) event processing module completes after this event handling, and the event newly producing is waited for and mail to other processingNode or self processing node are proceeded to process.
(7) case distribution module is carried out Hash operation according to new events property value, and mould is got the virtual place of systemReason node number (constant of configuration when its numerical value is startup), obtains dummy node numbering.
(8) case distribution module is according to the dummy node numbering obtaining, mutual with load decision-making module, inquiryDummy node mapping table in dynamic load administration module, the processing node numbering that the event of obtaining will be distributed.
(9) new events is mail to other processing nodes or self processing node continuation processing by case distribution module,Repeat the process starting from step (1), until the event of generation final process result returns to client.
The operating process that it should be noted that above-mentioned steps (2)～(8) is invention distributed dynamic load pipeThe core process of reason method, can see that each processing node carries out respectively load management strategy, negative without centerCarrying management processing node, is a kind of typical distributed dynamic load management method based on p 2 p technology.
Referring to Fig. 5, when introducing system of the present invention and carrying out mass data processing, initialize corresponding to processing nodeOr when processing node dynamic change, the load number of the each processing node of adjustment System, makes distributed treatment collectionGroup reaches load balancing process, the i.e. establishment and the flow chart that upgrades operation of distributed load equalizing table again.
(1) in distributed processing system(DPS), the load monitoring module of processing node detects that processing node changes.
(2) the load decision-making module of all processing nodes in load monitoring module distribution of notifications formula cluster: have placeReason node changes.
(3) respectively process node load decision-making module according to processing node modification information (comprising: initialization information,Increase node and shift out nodal information) Regeneration dynamics load information table and dummy node mapping table. According to dividingIn cloth formula treatment system, the comprehensive treatment capability of each processing node distributes the load of each processing node pro rataNumber. The dynamic load information table of processing node of the present invention has been used for reference consistent with dummy node mapping table update methodProperty Hash in the time that node adds or shift out system, reduce as far as possible system topological safeguard thought.
Referring to Fig. 6, introduce with processing node load and shift the corresponding distributed dynamic load transfer thing of sceneThe distribution flow figure of part. Its correspondence be native system while carrying out mass data processing, factor data is processed request heatPoint problem causes that after the processing node load overload that will mail to, the load balancing module of processing node is taked loadShift, processing node is numbered the content of operation of redirected failover events distribution flow:
(1) action listener module listens to the event or the data processing request that need processing.
(2) action listener module and load decision-making module are mutual, send decision instruction request to it.
(3) load decision-making module and load monitoring module are mutual, obtain the effectively negative of the each processing node of this systemCarry rate information.
(4) load decision-making module is according to each processing node pay(useful) load rate information of obtaining, learns to mail toProcessing node pay(useful) load rate overload and the set of underload processing node also have processing node, load decision modelPiece carries out probability load transfer operation; (K establishes in system in load node set, to take out at random on the lenient side KA fixed less natural number) individual processing node, and find out pay(useful) load rate from this K processing nodeMinimum processing node, and this processing node numbering is returned to case distribution module, carry out processing nodeNumbering is redirected.
(5) case distribution module mails to this processing node by new events and numbers the processing node place referring toReason, to dynamically adjust in real time the load equilibrium of distributed treatment cluster.
Send in real time in mass data processing request process the present invention system to distributed treatment cluster in clientProcessing node factor data in system is processed request and is had hot issue, thereby causes mailing to the negative of processing nodeCarry unbalanced, this system by probability carry out and mail to processing node and number redirected operation; In real time dynamicallyThe load equilibrium of distributed treatment cluster is adjusted on ground.
Referring to Fig. 7, introduce the distributed dynamic overload control of the present invention corresponding to processing node overload control sceneCase distribution flow chart processed. Its correspondence be native system while carrying out mass data processing, distributed treatment clusterAfter the load of each processing node is all transshipped, the load balancing module of processing node assists overload control module to carry outThe content of operation of the process of overload drop policy:
(1) action listener module listens to needs event to be processed or data processing request.
(2) action listener module and load decision-making module are mutual, send decision instruction request to it.
(3) load decision-making module and load monitoring module are mutual, obtain the effectively negative of the each processing node of this systemCarry rate information.
(4) load decision-making module is according to each processing node pay(useful) load rate information of obtaining, learns to mail toProcessing node pay(useful) load rate overload and underload node set are empty, and distributed treatment cluster is respectively processedNode all transships, and load decision-making module starts overload controlling mechanism and carries out overload control, carries out probabilityEvent abandons.
(5) action listener module receives event and abandons information, event is abandoned, and reply system to clientSystem overload response message.
Send in real time in mass data processing request process factor data place to distributed treatment cluster in clientReason request quantity is too huge, thereby causes the each processing node in distributed treatment cluster all to transship, and distributesProbability is abandoned data processing request by the each processing node of formula Processing Cluster system; Real-time ensuring distributed treatmentThe stability of cluster and reliability.
Here introduce when load is shifted and overload controls, processing node reorientation method, probabilistic event abandonThe concrete operations content of method and load factor information collecting method. It should be noted that: following these are square in detailMethod just for example, its objective is that in statement load factor information collecting method, emphasis is multi-load Factors Weighting meterCalculate the thinking of pay(useful) load rate, that processing node reorientation method stresses is larger in processing node pay(useful) load rate,Its probability of carrying out load transfer is also larger, and is redirected the random thinking of selecting, and probabilistic event is lostAbandon in method, emphasis be larger in processing node pay(useful) load rate, its carry out overload control probability alsoLarger thinking.
The content of operation of load factor information collecting method is as follows:
(a) be effective execution that proof load is shifted and overload is controlled, each processing node pay(useful) load rate informationCollection be basis and prerequisite. Each processing node, in setting-up time cycle T, gathers single treatment nodeLoad factor loadCur (i, j): loadCur (i, j)=α × cpu (i, j)+β × mem (i, j)+γ × queLen (i, j) in real time, in formula,LoadCur (i, j) is the processing node that is numbered i in the load factor of j time cycle, and cpu (i, j) is numbered iThe utilization rate of processing node cpu in the time of j time cycle; Mem (i, j) is that the processing node that is numbered i existsThe utilization rate of internal memory when j time cycle; QueLen (i, j) is numbered the processing node of i j timeData processing request queue length and the maximum ratio that allows queue length when cycle; α, β, γ are respectively each negativeCarry the weighting proportion of rate factor, and alpha+beta+γ=1. It should be noted that, during above-mentioned load factor gathers, considerLoad factor is an example, and other load factor is also adapted to load factor information collecting method of the present invention.
(b) the real-time load factor loadCur of processing node (i, j) obtaining according to step (a), obtains processing nodeCurrent pay(useful) load rate loadAfter (i, j): loadAfter (i, j)=α × loadCur (i, j)+(1-α) × loadAfter (i, j-1);In formula, loadAfter (i, j) is the processing node that the is numbered i pay(useful) load rate j time cycle; α isWeight ratio, α is larger, represent in pay(useful) load rate load factor in real time to occupy proportion larger; And 0 < α < 1.
(c) the processing node pay(useful) load rate loadAfter (i, j) obtaining according to step (b), can be according to systemThe load overload threshhold loadMax and the underload threshold values loadMin that set, be included into underload processing by processing nodeIn node set, the set of normal load processing node and the set of overload processing node.
Namely, if loadAfter (i, j) < is loadMin, processing node is included into the set of underload processing node;If loadAfter (i, j) > is loadMax, learns processing node overload, and processing node is included into overload and processes jointPoint set; Otherwise, be included into the set of normal load processing node.
Processing node numbering reorientation method of the present invention is to be based upon on the basis of load factor information collecting method. Can obtain the pay(useful) load rate of the each processing node of distributed treatment cluster according to load factor information collecting methodLoadAfter (i, j). Its concrete operations content is:
(a) be proof load equilibrium, solve factor data processing and ask the processing node that hot issue is brought to be bornCarry unbalanced problem, the dynamic load administration module of processing node is numbered redirected behaviour at execution processing nodeDo in process, first calculate and carry out processing node at j moment processing node i and number redirected probability feasibilityProbability (i, j): probability (i, j)=random (), in formula, random () can produce one of (0,1) intervalIndividual decimal.
(b) the pay(useful) load rate with processing node by the probability feasibility probability (i, j) of processing node iLoadAfter (i, j) compares, if probability (i, j)≤loadAfter (i, j), order execution step (c),Otherwise, with regard to this end process node serial number reorientation method.
(c) according to the pay(useful) load rate loadAfter (i, j) of each processing node, from the set of underload processing nodeSelect at random the individual processing node of K (K is a less natural number of setting in system), then from selectedIn the K a going out processing node, find out a processing node of pay(useful) load rate minimum, and its numbering is doneGiving case distribution module for the processing node numbering that will mail to processes.
Probabilistic event discarding method of the present invention is to be based upon under the prerequisite of load factor information collecting method. According toLoad factor information collecting method can obtain the pay(useful) load rate loadAfter (i, j) of each processing node in this system. ItsContent of operation is:
(a) be proof load equilibrium, solve factor data and process the too huge distributed treatment that causes of request quantityThe each processing node of cluster all transships problem, carries out probabilistic event at the dynamic load administration module of processing nodeAbandon in process, first calculate and carry out processing node at j moment processing node i and number redirected probability feasibilityProbability (i, j): probability (i, j)=random (); In formula, random () can produce one of (0,1) intervalIndividual decimal.
(b) the pay(useful) load rate with processing node by the probability feasibility probability (i, j) of processing node iLoadAfter (i, j) compares, if probability (i, j)≤loadAfter (i, j), notification event distribution mouldPiece abandons event; Otherwise, finish probabilistic event discarding method at this point.
The test of the embodiment of the present invention is successfully, has realized goal of the invention.
Claims (13)
1. a distributed dynamic load management system, is characterized in that: described system is by having dynamic loadManagement function and the distributed treatment cluster with the complete reciprocity framework of end-to-end P2P form, this processingCluster does not have central controlled Centroid, is only provided with multiple processing nodes, and wherein each processing node canThe dynamic load managerial ability of distributed arbitration program is independently provided, and its merit of authority and dynamic load management separatelyCan be identical, but the ability of the event of processing is identical or different; Each processing node is this dynamic load pipeThe control core of reason system, and be all provided with following four functional modules: event processing module, action listener mouldPiece, case distribution module, dynamic load administration module; The function of modules is as follows:
Event processing module, for the treatment of the event of particular type and attribute, produces intermediate treatment result orWhole result, and by case distribution module, the new events producing is transferred to this processing node or mail to otherProcessing node continues to process, or final process result is returned to client; This event processing module is that this isIn system, preset, or develop and be deployed on processing node by third party programmer;
Action listener module, is responsible for receiving the data processing request event from client, and from selfOr the event of other processing node; And accept the dynamic load administrative decision control of dynamic load administration moduleInstruction: in the time receiving load transfer instruction, event is sent to respectively to the processing node of specifying in this instruction;In the time receiving overload control instruction, produce the response message of system overload, and send to client; OrTransfer to event processing module to re-start processing event;
Dynamic load administration module, is provided with two parts of load monitoring unit and load decision package, wherein:Load monitoring unit action listener module assist under, in Real-time Collection and distributed storage Processing Cluster everywhereThe information of survival condition, pay(useful) load rate and the comprehensive treatment capability of reason node, and real-time update and storage warpDummy node mapping table and dynamic load information table after the establishment of load decision package or maintenance; Load decision-makingThe information of survival condition, pay(useful) load rate and comprehensive treatment capability based on each processing node is responsible in unit, withAnd the dummy node mapping table of storage, formulate for the load balancing of event handling and case distribution process,The dynamic load administrative decision control instruction that load is shifted and overload is controlled;
Case distribution module, is responsible for accepting the dynamic load administrative decision control that dynamic load administration module sendsInstruction, the new events being produced by event processing module that it is received or final process result execution event are dividedSend out and process: in the time receiving load balancing instruction, the new thing being produced by event processing module that it is receivedPart or final process result send to respectively this specified processing node of this instruction action listener module orOther processing node or client.
2. distributed dynamic load management system according to claim 1, is characterized in that: described everyIndividual processing node is distributed on different servers; In actual motion environment, same physical server energyEnough arrange and move multiple processing nodes.
3. distributed dynamic load management system according to claim 1, is characterized in that: described movingThe dynamic load management function of the load decision package in state load management module is as follows:
Survival condition and the General Office of the each processing node of distributed treatment cluster providing according to load monitoring unitReason ability information, in the time of system initialization or processing node generation dynamic change, real-time servicing dummy node reflectsFiring table and dynamic load information table, adjust the load of each processing node in distributed treatment cluster, i.e. virtual jointPoint number, so that respectively process node load balancing;
When causing processing node pay(useful) load because of the hot issue of processing and distributing data processing request or eventWhen rate is unbalanced, take load transition strategy, so that respectively process node load balancing;
In the time that each processing node pay(useful) load rate is all transshipped, take to transship control strategy, so that distributed treatmentCluster keeping system stability and robustness.
4. distributed dynamic load management system according to claim 3, is characterized in that: described placeReason node pay(useful) load rate is pay(useful) load rate and this week of the last load information collection period of this processing nodeValue after the real-time load factor weighting of phase, the actual negative carrying capacity that described real-time load factor is processing node and its are combinedClose the ratio of disposal ability, wherein, the actual negative carrying capacity of processing node comprises the consumption of CPU and internal memory,Or the length of pending event queue; The comprehensive treatment capability of processing node comprises the sky of cpu function, internal memoryBetween the maximum permissible value of capacity or pending event queue length; Utilize the pay(useful) load rate of processing node,Can effectively process the isomerism problem in distributed system, and processing node is divided into according to pay(useful) load rateFollowing three ranks: underload processing node, normal load processing node and overload processing node;
Described dummy node is non-necessary being and the logical symbol that mutually shines upon with processing node, Mei GexuIntend node and can only be mapped to a processing node, but each processing node is mapped to multiple dummy nodes; EmptyThe quantity of intending node configures in the time that system starts, and its quantity is not less than 2 times of processing node number; AndAfter system operation, the quantity of dummy node remains unchanged, when the processing node in system increases or shifts outTime, the mapping relations between processing node and dummy node are carried out corresponding changing according to distributed dynamic load managementBecome;
Described dummy node mapping table is a kind of data structure that this system realizes distributed dynamic load management:For storing the mapping relations of each dummy node symbol and its processing node numbering, each processing node hasDummy node mapping table separately, system can search out from dummy node by searching dummy node mapping tableThe processing node of its correspondence mappings, realizes distributed load equalizing;
Described dynamic load information table is the another kind of data knot that this system realizes distributed dynamic load managementStructure: for storing each processing node management and dummy node symbol and the number thereof controlled, this dynamic loadThe information of the information in information table and dummy node mapping table is one to one.
5. distributed dynamic load management system according to claim 1, is characterized in that: this systemApplication scenarios as follows:
Distribution Events scene: client sends mass data processing request in real time to distributed treatment cluster,Data processing request is received and dispatched when normal, and this cluster returns to result to client in real time, and ensures this clusterIn load balancing and the stability of each processing node; And in dynamic load management process, each processing nodeEqual, in this cluster without centralized load management node, each processing node can complete independentlyThe dynamic load management of distributed arbitration program;
The scene of processing node initialization and dynamic change: client sends sea in real time to distributed treatment clusterWhen amount data processing request, start in real time system or increase new processing node or shift out processing node to this cluster;And at short notice, again adjust in proportion its load according to the comprehensive treatment capability of each processing node, makeEach processing node in this cluster reaches load balancing again;
Scene is shifted in processing node load: client sends mass data processing in real time to distributed treatment clusterIn request process, factor data is processed request and is had hot issue, cause the request that receives processing node effectivelyLoad factor overload, and also have processing node in the set of underload processing node time, this cluster is by probability groundRedirected data processing request mails to certain or some processing node in the set of underload processing node, so thatDynamically adjust in real time the load equilibrium of this cluster;
Processing node overload is controlled scene: client sends mass data processing in real time to distributed treatment clusterRequest, it is huge that factor data is processed request quantity, causes all mistakes of each processing node pay(useful) load rate in this clusterWhile carrying, each processing node in this cluster by probability abandon data processing request, and return to clientThe response message of complex system overload, so that the stability of this cluster of real-time ensuring and reliability.
One kind adopt distributed dynamic load management system claimed in claim 1 based on P2P technology pointCloth formula dynamic load management method, is characterized in that: when client sends in real time to this distributed treatment clusterWhen mass data processing request, certain processing node is received after request, first according to the pay(useful) load rate of this nodeJudge with overloading threshold that this processing node is current and whether transship, if overload carries out that load is shifted or overload controlSystem; Otherwise, process this request; The request that completes is just encapsulated as event by the intermediate object program of generation after processing,And carry out load-balancing decision according to dummy node mapping table and this event determinant attribute value, determine this event bySelf node still mails to other processing node to be continued to process; And follow-up each processing node is being processed and distributionWhen intermediate event, be all according to pay(useful) load rate and overloading threshold or dummy node mapping table and event passKey property value is carried out the operation of load transfer, overload control or the balanced distribution of event, until produce finalResult, and return to client.
7. distributed dynamic load management method according to claim 6, is characterized in that: described sideMethod comprises following operating procedure:
Step 1, the dynamic load administration module of processing node creates or upgrades dummy node mapping table and dynamically bearCarry information table; Meanwhile, dynamic load administration module, under action listener module is assisted, is adopted according to setting cycleThe existing state of each processing node and real-time load information in concentrated-distributed cluster;
Step 2, needs event to be processed once the action listener module of processing node has listened to, by dynamicallyIn load management module, load decision package is according to dummy node mapping table, and load monitoring unit providesThe pay(useful) load rate information of each processing node is carried out dynamic load administrative decision, and formulates dynamic load management certainlyPlan control instruction, action listener module is carried out load transfer, overload control according to this control instruction, or by thingPart sends to event processing module to process;
Step 3, event processing module completes after the processing of this event, produces intermediate event or finally processesAs a result, send to case distribution module; By load decision package in dynamic load administration module according to virtual jointPoint mapping table, intermediate event determinant attribute value are carried out dynamic load administrative decision, and formulate dynamic load managementDecision Control instruction, case distribution module is carried out case distribution operation according to this instruction, by intermediate event orWhole result is distributed to corresponding processing node or client, and realizes each processing node load balancing.
8. distributed dynamic load management method according to claim 7, is characterized in that: described inStep 1 comprises following operating procedure:
(11) system is initially set up, and while creating dummy node mapping table and dynamic load information table, employing is pressedThe method that the WRR being directly proportional according to comprehensive treatment capability or Weighted random distribute initializes dummy node mappingTable; Or adopt Random assignment and ensure that equally distributed method initializes dummy node mapping table; Now,The dummy node number of distributing for the different each processing node of processing event capability should be differentiated;
(12) in the time increasing in this system or shift out processing node, equal for making system again reach as early as possible loadWeighing apparatus, dynamic load administration module will carry out real-time update to dummy node mapping table and dynamic load information table;
(13) system starts after operation, and the load monitoring unit in dynamic load administration module is at action listenerUnder module is assisted, gather the existing state of each processing node in distributed type assemblies and negative in real time according to setting cycleInformation carrying breath, and calculate its pay(useful) load rate according to the real-time load information of processing node, and by processing node respectivelyBe included in the set of underload processing node, the set of normal load processing node or the set of overload processing node.
9. distributed dynamic load management method according to claim 8, is characterized in that: described stepSuddenly (12) comprise following operating procedure:
(12a) the dynamic load administration module of each processing node detects that the processing node in this system has changeTime, this situation is informed to all processing nodes in this cluster;
(12b) for reduce system topological maintenance workload as far as possible, the dynamic load administration module of each processing nodeAccording to the initialization of processing node, the modification information that increases or shift out, adopt uniformity hash method in real time moreNew dynamic load information table and dummy node mapping table.
10. distributed dynamic load management method according to claim 8, is characterized in that: described inStep (13) comprises following operating procedure:
(13a) load monitoring unit is under action listener module is assisted, and the cycle gathers the real-time of each processing nodeLoad information; And in order to solve the isomerism of processing node, by all normalizings of real-time load information of processing nodeTurn to real-time load factor, calculate according to the real-time load information of each processing node and comprehensive treatment capability thereofThe real-time load factor in this this cycle of processing node;
(13b) load monitoring unit is according to the real-time load factor in the pay(useful) load rate in a upper cycle and this cycle,Adopt weighting scheme to calculate the pay(useful) load rate in this this cycle of processing node, and this processing node obtaining is hadEffect load factor value storage is in the load monitoring unit of dynamic load administration module;
(13c) load monitoring unit is first by the pay(useful) load rate score of this processing node and the load of defaultOverloading threshold compares judgement, if pay(useful) load rate score is greater than load overloading threshold, by this processing jointPoint is included in the set of overload processing node, finishes this step operation; Otherwise pay(useful) load rate score is littleIn load overloading threshold, carry out subsequent step (13d);
(13d) load monitoring unit is born the light of the pay(useful) load rate score of this processing node and default againCarry threshold value and compare judgement, if pay(useful) load rate is less than underload threshold value, this processing node is included into gentlyIn the set of load processing node; Otherwise pay(useful) load rate score is not more than load overloading threshold, just shouldProcessing node is included in the set of normal load processing node.
11. distributed dynamic load management methods according to claim 7, is characterized in that: described inStep 2 comprises following operating procedure:
(21) once action listener module has listened to needs after event to be processed, dynamic load administration moduleIn load decision package and load monitoring unit carry out alternately, knowing the pay(useful) load rate of each processing nodeNumerical value, and judge whether the pay(useful) load rate of this processing node transships: if it does not transship, carry out follow-up stepSuddenly (22); If its overload, redirect execution step (23);
(22) load decision package without carrying out, shift or overload control strategy by load, by action listener moduleDirectly give event processing module by this event and process, finish this step operation;
(23) load decision package judges whether other processing node all transships, and if so, takes probabilityOverload drop policy abandon this event: make to prepare the processing node pay(useful) load rate that mails to higher,Its probability of carrying out the operation of overload drop policy is larger, controls real-time ensuring distributed treatment to realize overloadThe working stability of cluster and reliable; If not, also there is the processing node that effective load factor is lighter, takeProbability is redirected the processing node of this event, shifts to realize load; And in the time of transferring load, do not changeThe mapping relations of dummy node and processing node, just probability reselects the processing node numbering of this event,And this event is just processed and directly mail to new processing node without event processing module, processing node is compiledNumber be redirected according to the probability being directly proportional to this processing node pay(useful) load rate, and make this processing nodePay(useful) load rate is higher, and the probability that this processing node is carried out event re-orientation processes nodal operation is larger.
12. distributed dynamic load management methods according to claim 11, is characterized in that: described inStep (23) comprises following operating procedure:
(23a) load decision package judges whether other processing node all transships, and if so, carries out follow-up stepSuddenly (23b); If not, also there is the processing node that effective load factor is lighter, redirect execution step (23c);
(23b) load decision package produces the random decimal in (0, a 1) scope, and by this random decimal withThis processing node pay(useful) load rate score compares judgement: whether this random decimal is less than this processing node hasEffect load factor numerical value; If so, load decision package sends overload to action listener module and abandons instruction, thingPart is monitored module and is directly abandoned this event, and the response message of answering system overload, then at case distribution mouldPiece sends to client by this response message under assisting; If not, carry out subsequent step (23e);
(23c) load decision package produces the random decimal in (0, a 1) scope, and by this random decimal withThis processing node pay(useful) load rate score compares judgement: whether this random decimal is less than this processing node hasEffect load factor numerical value; If so, carry out subsequent step (23d); If not, redirect performs step (23e);
(23d) load decision package takes out one or more processing nodes in load node set on the lenient side at random,And one or more processing node numberings of therefrom finding out pay(useful) load rate score minimum reset as this eventTo destination, then, this processing node numbering and load transfer instruction are returned to action listener module;Action listener module, under the assistance of case distribution module, directly mails to this event this processing node numbering instituteThe processing node of representative is processed, to dynamically adjust in real time the load equilibrium of distributed treatment cluster;
(23e) action listener module is directly given event processing module by this event and is processed.
13. distributed dynamic load management methods according to claim 7, is characterized in that: described inStep 3 comprises following operating procedure:
(31) event processing module is finished dealing with after this event, and the intermediate event producing is belonged to according to its keyProperty value carry out uniformity Hash evaluate operation, and mould gets the dummy node number of this system, will to obtain preparationThe dummy node numbering of this intermediate event distribution, to mail to self or other processing node by this intermediate eventContinue to process;
(32) case distribution module is according to the dummy node numbering of the preparation distribution obtaining and dynamic load managementModule is carried out alternately, obtains this intermediate event want according to the dummy node mapping table in dynamic load administration moduleThe processing node numbering mailing to;
(33) the processing node number information that case distribution module will mail to according to this intermediate event, by itContinue to process toward respective handling node, until the event of generation final process result directly returns to client.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310065185.XA CN103188345B (en) | 2013-03-01 | 2013-03-01 | Distributed dynamic load management system and method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310065185.XA CN103188345B (en) | 2013-03-01 | 2013-03-01 | Distributed dynamic load management system and method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN103188345A CN103188345A (en) | 2013-07-03 |
| CN103188345B true CN103188345B (en) | 2016-05-18 |
Family
ID=48679307
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201310065185.XA Expired - Fee Related CN103188345B (en) | 2013-03-01 | 2013-03-01 | Distributed dynamic load management system and method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103188345B (en) |
Families Citing this family (36)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9917891B2 (en) * | 2013-09-27 | 2018-03-13 | Intel Corporation | Distributed in-order load spreading resilient to topology changes |
| CN103716381B (en) * | 2013-12-12 | 2017-04-12 | 华为技术有限公司 | Control method and management node of distributed system |
| CN104142862B (en) * | 2013-12-16 | 2015-09-16 | 腾讯科技（深圳）有限公司 | The overload protection method of server and device |
| CN103746838B (en) * | 2013-12-26 | 2016-12-07 | 华北计算技术研究所 | A kind of method for scheduling task of non-stop layer node computer network |
| CN103945005B (en) * | 2014-05-06 | 2017-06-13 | 山西睿信智达传媒科技股份有限公司 | Dynamic load leveling framework based on many evaluation indexes |
| KR102232900B1 (en) * | 2014-10-02 | 2021-03-26 | 에스케이플래닛 주식회사 | System for cloud streaming service, method of cloud streaming service using optumal gpu and apparatus for the same |
| CN104486445B (en) * | 2014-12-30 | 2017-03-22 | 北京天云融创软件技术有限公司 | Distributed extendable resource monitoring system based on cloud platform |
| CN104572307B (en) * | 2015-01-30 | 2019-03-05 | 无锡华云数据技术服务有限公司 | The method that a kind of pair of virtual resource carries out flexible scheduling |
| CN106209665A (en) * | 2015-04-30 | 2016-12-07 | 中兴通讯股份有限公司 | A kind of realize the method for load sharing, interface message processor (IMP), service processor and system |
| CN104951427B (en) * | 2015-06-30 | 2019-01-11 | 深圳清华大学研究院 | Distributed computing framework centered on matrix |
| CN106559448A (en) * | 2015-09-28 | 2017-04-05 | 北京国双科技有限公司 | Server load balancing method and apparatus |
| CN106302628B (en) * | 2015-12-29 | 2019-12-27 | 北京典赞科技有限公司 | Unified management scheduling method for computing resources in ARM architecture network cluster |
| CN107562531B (en) * | 2016-06-30 | 2020-10-09 | 华为技术有限公司 | Data equalization method and device |
| CN107733961B (en) * | 2016-10-19 | 2021-06-22 | 北京奥斯达兴业科技有限公司 | Resource allocation method and device |
| CN106610870B (en) * | 2016-12-28 | 2019-10-15 | 北京奇艺世纪科技有限公司 | A kind of processing number of nodes method of adjustment and device |
| CN107040476B (en) * | 2017-03-10 | 2020-05-05 | 浙江工业大学 | Dynamic step-by-step back pressure method for real-time stream calculation |
| CN109218341B (en) * | 2017-06-29 | 2022-02-25 | 北京京东尚科信息技术有限公司 | Load balancing method and device for monitoring server and server |
| CN107341055A (en) * | 2017-07-03 | 2017-11-10 | 杭州知物数据科技有限公司 | The real-time dynamic management approach and system of distributed memory file system |
| CN109408219B (en) * | 2017-08-16 | 2021-04-02 | 中国电信股份有限公司 | Distributed data receiving method, system and device for distributed data receiving |
| CN109510855B (en) * | 2017-09-15 | 2020-07-28 | 腾讯科技（深圳）有限公司 | Event distribution system, method and device |
| CN108282516B (en) * | 2017-12-21 | 2021-09-10 | 深圳创新科技术有限公司 | Distributed storage cluster load balancing method and device based on iSCSI |
| CN108829511A (en) * | 2018-05-07 | 2018-11-16 | 中山大学 | Load balancing adjusting method based on state machine replica management model |
| CN110908590A (en) * | 2018-09-17 | 2020-03-24 | 中国电力科学研究院有限公司 | Distributed storage method and system for transformer substation data |
| CN110049084B (en) * | 2018-12-10 | 2021-08-24 | 创新先进技术有限公司 | Current limiting method, device and equipment of distributed system |
| US10901805B2 (en) | 2018-12-18 | 2021-01-26 | At&T Intellectual Property I, L.P. | Distributed load balancing for processing of high-volume data streams |
| CN111522649B (en) * | 2019-02-01 | 2022-04-12 | 北京京东尚科信息技术有限公司 | Distributed task allocation method, device and system |
| CN109951394B (en) * | 2019-03-11 | 2021-04-27 | 中国银联股份有限公司 | Method and device for determining number of virtual nodes of transaction system |
| CN109903178B (en) * | 2019-04-04 | 2021-08-20 | 腾讯科技（深圳）有限公司 | Method, device and system for determining common social contact object and computing equipment |
| CN110120973A (en) * | 2019-04-28 | 2019-08-13 | 华为技术有限公司 | A kind of request control method, relevant device and computer storage medium |
| CN110213351A (en) * | 2019-05-17 | 2019-09-06 | 北京航空航天大学 | A kind of dynamic self-adapting I/O load equalization methods towards wide area high-performance computing environment |
| CN110855738B (en) * | 2019-09-25 | 2022-06-07 | 安徽四创电子股份有限公司 | Communication processing system for multi-source equipment |
| CN111338793A (en) * | 2020-02-14 | 2020-06-26 | 拉扎斯网络科技（上海）有限公司 | Cluster task distribution method, device, equipment and storage medium |
| CN111339196B (en) * | 2020-02-24 | 2022-02-25 | 深圳点链科技有限公司 | Data processing method and system based on block chain and computer readable storage medium |
| US11418417B2 (en) | 2020-12-21 | 2022-08-16 | Red Hat, Inc. | Managing stateful workloads executing on temporarily available resources of a cloud computing system |
| CN115587222B (en) * | 2022-12-12 | 2023-03-17 | 阿里巴巴（中国）有限公司 | Distributed graph calculation method, system and equipment |
| CN115766737B (en) * | 2023-01-09 | 2023-05-16 | 北京微吼时代科技有限公司 | Load balancing method and device and electronic equipment |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102457428A (en) * | 2010-10-27 | 2012-05-16 | 中兴通讯股份有限公司 | Load balancing realization method and device for DHT (distributed Hash table) network |
| CN102457429A (en) * | 2010-10-27 | 2012-05-16 | 中兴通讯股份有限公司 | Method and device for realizing load balance of DHT (Distributed Hash Table) network |
| CN102882973A (en) * | 2012-10-11 | 2013-01-16 | 北京邮电大学 | Distributed load balancing system and distributed load balancing method based on peer to peer (P2P) technology |
-
2013
- 2013-03-01 CN CN201310065185.XA patent/CN103188345B/en not_active Expired - Fee Related
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102457428A (en) * | 2010-10-27 | 2012-05-16 | 中兴通讯股份有限公司 | Load balancing realization method and device for DHT (distributed Hash table) network |
| CN102457429A (en) * | 2010-10-27 | 2012-05-16 | 中兴通讯股份有限公司 | Method and device for realizing load balance of DHT (Distributed Hash Table) network |
| CN102882973A (en) * | 2012-10-11 | 2013-01-16 | 北京邮电大学 | Distributed load balancing system and distributed load balancing method based on peer to peer (P2P) technology |
Also Published As
| Publication number | Publication date |
|---|---|
| CN103188345A (en) | 2013-07-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103188345B (en) | Distributed dynamic load management system and method | |
| Verma et al. | A survey on network methodologies for real-time analytics of massive IoT data and open research issues | |
| CN109617992B (en) | Block chain-based dynamic election method for edge computing nodes | |
| Ningning et al. | Fog computing dynamic load balancing mechanism based on graph repartitioning | |
| CN104461740B (en) | A kind of cross-domain PC cluster resource polymerization and the method for distribution | |
| CN102882973B (en) | Distributed load balancing system and distributed load balancing method based on peer to peer (P2P) technology | |
| US9031692B2 (en) | Cloud robot system and method of integrating the same | |
| CN106254240B (en) | A kind of data processing method and routing layer equipment and system | |
| CN106936892A (en) | A kind of self-organizing cloud multi-to-multi computation migration method and system | |
| CN102244685A (en) | Distributed type dynamic cache expanding method and system supporting load balancing | |
| CN103152393A (en) | Charging method and charging system for cloud computing | |
| CN103368864A (en) | Intelligent load balancing method based on c/s (Client/Server) architecture | |
| CN102164184A (en) | Computer entity access and management method for cloud computing network and cloud computing network | |
| CN104754053B (en) | A kind of distributed software defines network and the wherein method of dynamic control controller | |
| CN106130960B (en) | Judgement system, load dispatching method and the device of steal-number behavior | |
| JP6891961B2 (en) | Network control systems, methods and programs | |
| CN105094982A (en) | Multi-satellite remote sensing data processing system | |
| CN110109756A (en) | A kind of network target range construction method, system and storage medium | |
| CN102110014A (en) | Method for balancing loads of virtual machine (VM) | |
| US20240028415A1 (en) | Instance deployment method and apparatus, cloud system, computing device, and storage medium | |
| Teli et al. | Big data migration between data centers in online cloud environment | |
| CN101778002A (en) | Large-scale cluster system and building method thereof | |
| Ashraf et al. | TOPSIS-based service arbitration for autonomic internet of things | |
| CN109960586A (en) | A kind of appreciable four-layer load-equalizing device of server state and equalization methods | |
| EP3560148B1 (en) | Database functions-defined network switch |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20160518 Termination date: 20170301 |