CN106101276A - A kind of cluster load balancing method and device - Google Patents
A kind of cluster load balancing method and device Download PDFInfo
- Publication number
- CN106101276A CN106101276A CN201610654053.4A CN201610654053A CN106101276A CN 106101276 A CN106101276 A CN 106101276A CN 201610654053 A CN201610654053 A CN 201610654053A CN 106101276 A CN106101276 A CN 106101276A
- Authority
- CN
- China
- Prior art keywords
- load value
- node
- current
- load
- value
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1001—Protocols in which an application is distributed across nodes in the network for accessing one among a plurality of replicated servers
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/104—Peer-to-peer [P2P] networks
- H04L67/1044—Group management mechanisms
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Physics & Mathematics (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Theoretical Computer Science (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
- Supply And Distribution Of Alternating Current (AREA)
Abstract
The invention provides a kind of cluster load balancing method and device, wherein, the method includes: receive the task that user sends；The current load value of each node in cluster in acquisition current period, and each node is in the steady load value in previous cycle, according to the steady load value in current load value and previous cycle, determines the current steady load value of each node；According to the load changing value that current steady load value and above-mentioned task are corresponding, determine the prediction load value of each node；Distribute to above-mentioned task predict the node that load value is minimum with the ratio of maximum acceptable load value.The cluster load balancing method of present invention offer and device, it is adaptable to video field of storage, can be prevented effectively from when carrying out task distribution and the effect that flocks together occur, and, it is adaptable to the cluster that passing node equipments is differently configured.
Description
Technical field
The present invention relates to video storage and load-balancing technique field, in particular to a kind of cluster load balance side
Method and device.
Background technology
Load balancing is the one of clustering functionality, i.e. load pressure is reasonably allocated to certain of cluster according to certain algorithm
On machine, balancing the pressure of every station server, make server resource be fully used, its effectiveness determines the entirety of cluster
Performance and resource utilization.
Existing load-balancing algorithm mostly applies to configure identical computer, the fluctuation of load unconspicuous cluster system
System.When load equalizer receive ask endlessly time, load equalizer, according to its load strategy, forwards a request to phase
The server answered, the load strategy that load equalizer is used can be in random algorithm, round robin algorithm, average load algorithm
A kind of.
But, in video field of storage, load observation value is mainly I/O (input/output, the input/defeated of disk
Go out), and the I/O of disk fluctuation is very big, and the load to node needs certain time could reflect the change to load value, because of
This, when the instantaneous high-volume of video arrives, node load value now can not upgrade in time, causes the storage point of all of video
The node that load value that dispensing is thought at the moment is minimum, i.e. causes the effect that flocks together, in video field of storage, the service of each node
Device configuration may and differ, and therefore, load-balancing algorithm of the prior art is in video field of storage inapplicable.
Summary of the invention
In view of this, the purpose of the embodiment of the present invention is to provide a kind of cluster load balancing method and device, to solve
Existing load-balancing method applies the problem being easily caused the effect that flocks together in video field of storage.
First aspect, embodiments provides a kind of cluster load balancing method, and wherein, described method includes:
Receive the task that user sends；
Obtain in current period the current load value of each node in cluster, and steady in the previous cycle of each described node
Fixed load value, according to the steady load value in described current load value and previous cycle, determines the current steady of each described node
Load value；
According to the load changing value that described current steady load value and described task are corresponding, determine the pre-of each described node
Survey load value；
Described task is distributed to the described node that described prediction load value is minimum with the ratio of maximum acceptable load value.
In conjunction with first aspect, embodiments provide the first possible implementation of above-mentioned first aspect, its
In, the minimum described node of the described ratio that described task is distributed to described prediction load value and maximum acceptable load value it
After, also include:
The prediction load value of each described node after the distribution of acquisition task, after distribution of computation tasks, each described node is pre-
Survey the ratio of load value and described maximum acceptable load value；
According to the maximum acceptable load value of node and the prediction load of each described node after task distribution each described
Value, calculates the described ratio remaining load value less than or equal to the described node presetting load threshold；
The task of described node minimum for described remaining load value is transferred to the described knot that described remaining load value is maximum
Point.
In conjunction with first aspect, embodiments provide the implementation that the second of above-mentioned first aspect is possible, its
In, the described steady load value according to described current load value and previous cycle, determine that the current steady of each described node is born
Load value, including:
Calculate the current load value of each described node and described node steady load value first poor in the previous cycle
Value；
By the current steady load value of each described node and described node in the difference of the steady load value in previous cycle
It is designated as the second difference, according to the proportionate relationship between described first difference and described second difference, determines each described node
Current steady load value.
In conjunction with first aspect, embodiments provide the third possible implementation of above-mentioned first aspect, its
In, the described steady load value according to described current load value and previous cycle, determine that the current steady of each described node is born
Load value, including:
According to the current load value of node each described, each described node, in the steady load value in previous cycle, passes through
Formula (1) calculates the current steady load value of each described node；
Nm=(nm-Nm-1)/d+Nm-1 (1)
Wherein, in formula (1), NmFor the current steady load value of node arbitrary in described cluster, Nm-1For described arbitrary
Node is at the steady load value in previous cycle, nmFor the current load value of described arbitrary node, d is pre-set ratio, and m is the cycle.
In conjunction with first aspect, embodiments provide the 4th kind of possible implementation of above-mentioned first aspect, its
In, the described load changing value corresponding according to described current steady load value and described task, determine the pre-of each described node
Survey load value, including:
According to the load changing value that the current steady load value of node each described and described task are corresponding, pass through formula
(2) the prediction load value of each described node is calculated；
Fm=Dm+Nm (2)
Wherein, in formula (2), FmFor the prediction load value of node arbitrary in described cluster, DmCorresponding for described task
Load changing value, NmFor the current steady load value of described arbitrary node, m is the cycle.
Second aspect, embodiments provides a kind of cluster load balance device, and described device includes:
Receiver module, for receiving the task that user sends；
First determines module, for obtaining in current period the current load value of each node in cluster, and described in each
Node, in the steady load value in previous cycle, according to the steady load value in described current load value and previous cycle, determines each
The current steady load value of described node；
Second determines module, for the load changing value corresponding according to described current steady load value and described task, really
The prediction load value of each described node fixed；
Task allocating module, for distributing to the ratio of described prediction load value and maximum acceptable load value by described task
The described node that value is minimum.
In conjunction with second aspect, embodiments provide the first possible implementation of above-mentioned second aspect, its
In, described device also includes:
First computing module, the prediction load value of each described node, distribution of computation tasks after obtaining task distribution
The prediction load value of each described node rear and the ratio of described maximum acceptable load value；
Second computing module, for according to each institute after the maximum acceptable load value of node each described and task distribution
State the prediction load value of node, calculate the described ratio remaining load value less than or equal to the described node presetting load threshold；
Task shift module, the task for the described node that described remaining load value is minimum is transferred to described residual negative
The described node that load value is maximum.
In conjunction with second aspect, embodiments provide the implementation that the second of above-mentioned second aspect is possible, its
In, described first determines that module includes:
First computing unit, for calculating steady in the previous cycle of the current load value of each described node and described node
First difference of fixed load value；
Determine unit, for by the current steady load value of each described node and described node stablizing in the previous cycle
The difference of load value is designated as the second difference, according to the proportionate relationship between described first difference and described second difference, determines each
The current steady load value of individual described node.
In conjunction with second aspect, embodiments provide the third possible implementation of above-mentioned second aspect, its
In, described first determines that module also includes:
Second computing unit, for according to the current load value of node each described, each described node is in the previous cycle
Steady load value, calculated the current steady load value of each described node by formula (1)；
Nm=(nm-Nm-1)/d+Nm-1 (1)
Wherein, in formula (1), NmFor the current steady load value of node arbitrary in described cluster, Nm-1For described arbitrary
Node is at the steady load value in previous cycle, nmFor the current load value of described arbitrary node, d is pre-set ratio, and m is the cycle.
In conjunction with second aspect, embodiments provide the 4th kind of possible implementation of above-mentioned second aspect, its
In, described second determines that module includes:
3rd computing unit, for the load corresponding according to the current steady load value of node each described and described task
Changing value, calculates the prediction load value of each described node by formula (2)；
Fm=Dm+Nm (2)
Wherein, in formula (2), FmFor the prediction load value of node arbitrary in described cluster, DmCorresponding for described task
Load changing value, NmFor the current steady load value of described arbitrary node, m is the cycle.
The embodiment of the present invention provide cluster load balancing method and device, it is adaptable to video field of storage, carry out appoint
Can be prevented effectively from during business distribution and the effect that flocks together occurs, and, it is adaptable to the cluster that passing node equipments is differently configured.
For making the above-mentioned purpose of the present invention, feature and advantage to become apparent, preferred embodiment cited below particularly, and coordinate
Appended accompanying drawing, is described in detail below.
Accompanying drawing explanation
In order to be illustrated more clearly that the technical scheme of the embodiment of the present invention, below by embodiment required use attached
Figure is briefly described, it will be appreciated that the following drawings illustrate only certain embodiments of the present invention, and it is right to be therefore not construed as
The restriction of scope, for those of ordinary skill in the art, on the premise of not paying creative work, it is also possible to according to this
A little accompanying drawings obtain other relevant accompanying drawings.
Fig. 1 shows the flow chart of a kind of cluster load balancing method that the embodiment of the present invention 1 provided；
Fig. 2 shows the structural representation of a kind of cluster load balance device that the embodiment of the present invention 2 provided.
Detailed description of the invention
For making the purpose of the embodiment of the present invention, technical scheme and advantage clearer, below in conjunction with the embodiment of the present invention
Middle accompanying drawing, is clearly and completely described the technical scheme in the embodiment of the present invention, it is clear that described embodiment is only
It is a part of embodiment of the present invention rather than whole embodiments.Generally real with the present invention illustrated described in accompanying drawing herein
The assembly executing example can be arranged with various different configurations and design.Therefore, below to the present invention's provided in the accompanying drawings
The detailed description of embodiment is not intended to limit the scope of claimed invention, but is merely representative of the selected reality of the present invention
Execute example.Based on embodiments of the invention, the institute that those skilled in the art are obtained on the premise of not making creative work
There are other embodiments, broadly fall into the scope of protection of the invention.
In view of in correlation technique, load equalizer is mostly to use random algorithm, round robin algorithm or average load algorithm
Corresponding server or equipment is requested assignment to by receive, but, in video field of storage, load observation value is mainly magnetic
The I/O of dish, and the I/O of disk fluctuation is very big, and the load to node needs certain time could reflect the change to load value
Changing, therefore, when the instantaneous high-volume of video arrives, node load value now can not upgrade in time, causes all of video
Store the node that the load value distributing to think at the moment is minimum, i.e. cause the effect that flocks together, at video field of storage, each node
Server configures may and differ, therefore, load-balancing algorithm of the prior art is in video field of storage inapplicable.
Based on this, embodiments provide a kind of cluster load balancing method and device, be described below by embodiment.
Embodiment 1
Embodiments providing a kind of cluster load balancing method, the method is applicable to video field of storage, and
The effect that flocks together can be prevented effectively from.
As it is shown in figure 1, the cluster load balancing method that the embodiment of the present invention provides, including step S110-S140.
S110, receives the task that user sends.
Above-mentioned task can be video, when above-mentioned task is video camera when performing the video that video recording plan gathers, and shooting
Machine needs to store the transmission of video of collection to storage device, and this storage device can be disk, hard disk etc., should
Before video stores, first this video task being sent to load equalizer, load equalizer receives camera transmissions
Video task, and this video task is saved in set of tasks.Certainly, above-mentioned task can also is that other in addition to video
Requests etc., the embodiment of the present invention does not limit the particular type of above-mentioned task.
S120, the current load value of each node in cluster in acquisition current period, and each node is in the previous cycle
Steady load value, according to the steady load value in current load value and previous cycle, determines the current steady load value of each node.
Above-mentioned cluster refers to multiple stage machine and jointly cooperates, and the multicomputer system externally provided uniform services is above-mentioned many
Platform machine can be multiple server, it is also possible to be multiple computer, multiple disk, hard disk etc., each in above-mentioned cluster
Equipment is exactly a node.
During each node in the cluster is operated, load equalizer can real-time or each node of taken at regular intervals
Current load value, this current load value refers to the instant load value of each node, the current load value of each node above-mentioned
Can be the I/O of this node, owing to load equalizer is the current load value of each node in real-time or taken at regular intervals cluster,
Therefore load equalizer often can be gathered in a cluster current load value of each node as a cycle.
Each node above-mentioned in the steady load value in previous cycle, is then to rely on steady in the first two cycle of each node
Fixed load value, therefore, it can set the incipient stability load value of each node, can be by the incipient stability load value of each node
It is set to 0, thus, it is possible to according to incipient stability load value and the present load of each node in the cluster of collection in the period 1
Value, calculates the steady load value of period 1 each node interior, by that analogy, can obtain the stable negative in above-mentioned previous cycle
Load value.
The current load value of each node in load equalizer acquires current period, and obtain each node front
After the steady load value in one cycle, according to the current load value of each node in above-mentioned current period with each node previous
The steady load value in cycle, determines the current steady load value of each node, and this process specifically includes: calculate working as of each node
Front load value and node are in the first difference of the steady load value in previous cycle；By current steady load value and the knot of each node
Point is designated as the second difference in the difference of the steady load value in previous cycle, closes according to the ratio between the first difference and the second difference
System, determines the current steady load value of each node.
Wherein, direct proportionality between above-mentioned first difference and the second difference, this preset ratio coefficient can be 16 or
Person 32, it is, of course, also possible to be other numerical value, the value of above-mentioned preset ratio coefficient depends on the load value of each node in cluster
Reach the degree of fluctuation of the load value of stable time and each node, in concrete application scenarios, can be according to actual need
Aforementioned proportion coefficient to be chosen, the embodiment of the present invention does not limit the concrete value of aforementioned proportion coefficient.
Wherein, the current steady load value of each node in above-mentioned cluster, it is also possible to determine by the following method, specifically wrap
Include: according to the current load value of each node, each node, in the steady load value in previous cycle, calculates each by formula (1)
The current steady load value of individual node；
Nm=(nm-Nm-1)/d+Nm-1 (1)
Wherein, in formula (1), NmFor the current steady load value of node arbitrary in cluster, Nm-1For any of the above-described node
At the steady load value in previous cycle, nmFor the current load value of any of the above-described node, d is pre-set ratio, and m is the cycle.
Above-mentioned d is pre-set ratio, and the value of this pre-set ratio is generally 16 or 32, it is, of course, also possible to be other numerical value, on
The value stating pre-set ratio d depends on that in cluster, the load value of each node reaches the load of stable time and each node
The degree of fluctuation of value, in concrete application scenarios, can choose above-mentioned pre-set ratio according to actual needs, and the embodiment of the present invention is also
Do not limit the concrete value of above-mentioned pre-set ratio.
In embodiments of the present invention, when being calculated the current steady load value of current period each node interior by formula (1),
Depend in cluster each node in the steady load value in previous cycle, it is therefore desirable to obtain in cluster each node in the last week
The steady load value of phase, in like manner, in cluster, each node is in the acquisition of the steady load value in previous cycle, need to rely on cluster
In each node in the steady load value in the first two cycle, by that analogy, in order to calculate the current of each node in the current cycle
Steady load value, needs to set the incipient stability load value of each node in cluster, by N0It is designated as in cluster at the beginning of arbitrary node
Beginning steady load value, and by N0Value be set to 0, when collecting in the period 1 after the current load value of each node, will
In period 1, in cluster, the current load value of arbitrary node is designated as n1, appoint in the period 1 can being calculated according to formula (1)
The current steady load value N of one node1, according to the current load value of each node collected in the period 1, and each knot
The initial load value of point, can calculate each node current steady within the period 1 in cluster by formula (1) and load
Value, gathers in second round the current load value of each node in cluster, afterwards according to each node in cluster in second round
Current load value and the steady load value of each node in the period 1, can be calculated in second round by formula (1)
The current steady load value of each node in cluster, by that analogy, according to each node in cluster at the stable negative in a upper cycle
Load value and the current load value of each node current period node, calculate the steady load value of current period each node interior,
The method using above-mentioned integration, it is not necessary to record the current load value of each cycle collection, depend only on a cycle
Steady load value and the current load value of current period interior knot, it is possible to calculate working as of each node in current period cluster
Front steady load value, computational methods are simpler, and, the steady load value of the cluster that employing integration method calculates is more steady
Fixed.
S130, according to the load changing value that current steady load value and above-mentioned task are corresponding, determines the prediction of each node
Load value.
Wherein, above-mentioned task corresponding load changing value refer to the task correspondence task amount that each node receives, than
As said, in cluster, certain node receives the video store tasks of a 1MB, and load changing value corresponding to the most above-mentioned task is exactly
1MB, certainly, above-mentioned is merely illustrative the load changing value that task is corresponding, does not limit the concrete numerical value of load changing value.
In S120, although calculated the steady load value of each node in current period by integration method,
But when new task arrives node, node can not update current load value at once, the current period interior knot therefore calculated
Steady load value may not be the real load value of current node, when the instantaneous task of high-volume arrives, load equalizer
According to the load value of the current steady of each node in current period, task may be assigned on same node, thus lead
Cause flocks together effect, therefore, in order to solve the effect that flocks together, needs by negative corresponding with receiving of task of current steady load value
Carry changing value, determine the prediction load value of each node, specifically include:
Current steady load value according to each node and load changing value corresponding to above-mentioned task, counted by formula (2)
Calculate the prediction load value of each node；
Fm=Dm+Nm (2)
Wherein, in formula (2), FmFor the prediction load value of each node, DmFor the load changing value that task is corresponding, Nm
For the current steady load value of each node, m is the cycle.
Wherein, above-mentioned m represents the m-th cycle.
In embodiments of the present invention, during by task load to node, the load change of node can be estimated, knot will be loaded to
The task amount that the task of point is corresponding changes as the node load estimated, and the load change that the most above-mentioned task is corresponding, according to S120
In current steady load value in the current period that gets and load changing value corresponding to above-mentioned task, counted by formula (2)
Calculate the prediction load value of each node.
When any one node in task load to cluster, the prediction load value that node is corresponding is current steady load value
The load changing value sum corresponding with anticipated task, so prediction load value FmCloser to actual value, at this moment, corresponding the bearing of task
Carry change DmPredicting function can weaken, DmBe changed to Dm=Dm-1× (b-1)/b, wherein, in the formula, DmFor m-th week
The load changing value that task in phase is corresponding, Dm-1Being load changing value corresponding to the task in the m-1 cycle, b is present count
Value.
After above-mentioned prediction load value makes node distribution task, the change of the load value of each node can be embodied at once
On, from without because the load value change of each node slowly all distributes the major part of instantaneous submission even all tasks
On same node, thus solving the effect that flocks together, when task high-volume, instantaneous arrival, it is stable that system can reach approximation
State.
S140, distributes to above-mentioned task predict the node that load value is minimum with the ratio of maximum acceptable load value.
The prediction load value calculated by S130, can solve the effect that flocks together, but joining due to each passing node equipments
Putting may be different, iff the prediction load value using each node as load metric, assigns the task to current predictive
The node that load value is minimum, is inaccurate, and for the cluster that passing node equipments is different, the maximum acceptable load value of each node is not
With, it is thus impossible to carry out task distribution only by prediction load value.
In embodiments of the present invention, the prediction load value of each node and the maximum acceptable of each node can be calculated
The ratio of load value, using this ratio as the load metric of each node, when load equalizer receives the task that user submits to
Time, assign the task to the node that above-mentioned ratio is minimum.
Wherein, above-mentioned maximum acceptable load value is the maximum load value that each passing node equipments can be born, and works as node
Task amount when exceeding this maximum acceptable load value, node is by cisco unity malfunction.
In addition to this it is possible to load value can be received according to the prediction load value of each node and the maximum of each node,
By equation below, calculate the remaining load value of each node；
Cm=M-Fm
Wherein, in above-mentioned formula, M is the maximum acceptable load value of arbitrary node in cluster, FmFor arbitrary node
Prediction load value, CmFor arbitrary node remaining load value, m is the cycle.
After calculating the remaining load value of each node by the way, using node maximum for remaining load value as working as
The node that front load is minimum, i.e. assigns the task to the node that remaining load value is maximum.
After carrying out above-mentioned task distribution, possible cluster is not yet in equilibrium state, accordingly, it would be desirable to equal to cluster
Weighing apparatus degree judges, and according to judged result, the node in cluster is carried out task transfer.
The above-mentioned remaining load value calculating each node, according to the variance of following formula computing cluster, according to cluster
Variance judges the balanced intensity of cluster, and the variance of cluster is the least, it is believed that the balanced intensity of cluster is the highest, and i.e. the stability of cluster is more
High；
S=(Cm1-T)2+(Cm2-T)2+...+(Cmk-T)2
Wherein, in above-mentioned formula, T is the average residual load value of cluster, Cm1Residue in cluster first node
Load value, Cm2For the remaining load value of in cluster second node, CmkFor the remaining load value of k-th node in cluster, S is
The variance of cluster, m is the cycle, and k is the number of node in cluster.
In embodiments of the present invention, variance yields can be preset, by variance S of the above-mentioned cluster calculated and default variance yields
Comparing, when above-mentioned variance S is more than or equal to when presetting variance yields, then judge that cluster current equalization degree is relatively low, it is right to need
Node in cluster carries out task transfer, specifically includes:
The prediction load value of each node, the prediction load value of each node after distribution of computation tasks after the distribution of acquisition task
Ratio with maximum acceptable load value；Each node after maximum acceptable load value according to each node and task distribution
Prediction load value, calculates the ratio remaining load value less than or equal to the node presetting load threshold；Remaining load value is minimum
The task of node be transferred to the node that remaining load value is maximum.
In embodiments of the present invention, when the balanced intensity of cluster is relatively low, need the task of node in cluster is shifted
Time, it is necessary first to obtain the prediction load value of each node after task is distributed, owing to load equalizer can real-time or taken at regular intervals
The current load value of each node in cluster, when i.e. carrying out task distribution, load equalizer also can gather each node in cluster
Current load value, the current load value of each node after distributing according to task, and each node is at the stable negative in previous cycle
Load value, calculates the current steady load value of each node of current period, afterwards, according to the current steady of each node of current period
Load value and load changing value corresponding to task, the prediction load value of each node after distribution of computation tasks；
Then the maximum acceptable load value predicting load value and each node of each node after distribution of computation tasks
Ratio, compares this ratio with default load threshold, obtains aforementioned proportion less than or equal to the node presetting load threshold,
These nodes are carried out task transfer, specifically includes: calculate remaining of the aforementioned proportion node less than or equal to default load threshold
Remaining load value, is ranked up above-mentioned node according to remaining load value, chooses remaining load value minimum from the node after sequence
The node maximum with remaining load value, is transferred to, by the task of node minimum for remaining load value, the knot that remaining load value is maximum
Point.
Above-mentioned default load threshold is the numerical value pre-set, and this value can be 80%, it is also possible to for other numerical value, above-mentioned
Default load threshold can be configured according to reality application, and it is above-mentioned default load threshold that the embodiment of the present invention does not limit
Concrete numerical value.When the node calculated default load value and each node maximum acceptable load value ratio less than or etc.
When above-mentioned default load threshold, the most do not shift the task of this node.
After being provided with load threshold, can cross and reduce unnecessary task transfer, still cannot it addition, lost efficacy at node function
When detecting, now the remaining load value of failing node may be maximum in cluster, and without arranging load threshold, load is all
The task of other node in cluster may be transferred to this failing node by weighing apparatus, thus causes black-hole effect, and black-hole effect refers to
Be exactly but that node function when losing efficacy cannot detect its disabler, now the current load value of node is zero, and load is all
Weighing apparatus thinks that the load of this node is minimum all the time, and each load all can select this node.After being provided with load threshold, if collection
In Qun, Non-precondition load value and the ratio of the maximum acceptable load value of each node are more than when presetting load threshold, and load is all
The task of other nodes will not be transferred to failing node by weighing apparatus, thus solves black-hole effect.
When the task transfer carried out between node, such as, the node that remaining load value is the highest is A, and remaining load value is
Low node is B, in order to improve the balanced intensity of cluster, the task of node B can be transferred to node A, but if after Zhuan Yi,
Node A may become the node that remaining load value is minimum, and node B becomes the node that remaining load value is the highest, need the most again by
Task transfers to node B from node A, and this transfer may infinitely go on, and this phenomenon is referred to as jolting.
In order to eliminate above-mentioned phenomenon of jolting, if maximum residual load value and most descend the difference between residue load value not surpass
Cross the load capacity that the maximum task received may be brought, then terminate the transfer of this subtask.
The cluster load balancing method that the embodiment of the present invention provides, it is adaptable to video field of storage, is carrying out task distribution
Time can be prevented effectively from the effect that flocks together occurs, and, it is adaptable to the cluster that passing node equipments is differently configured.
Embodiment 2
A kind of cluster load balance device that the embodiment of the present invention provides, the load balancing apparatus that the embodiment of the present invention provides
Being used for performing the cluster load balancing method that embodiment 1 provides, this cluster load balance device can be that a kind of cluster load is equal
Weighing apparatus.
As in figure 2 it is shown, the load balancing apparatus that the embodiment of the present invention provides, including receiver module 210, first determine module
220, second determines module 230 and task allocating module 240；
Above-mentioned receiver module 210, for receiving the task that user sends；
Above-mentioned first determines module 220, for obtaining in current period the current load value of each node in cluster, and respectively
Individual node, in the steady load value in previous cycle, according to the steady load value in current load value and previous cycle, determines that each is tied
The current steady load value of point；
Above-mentioned second determines module 230, for the load changing value corresponding according to current steady load value and above-mentioned task,
Determine the prediction load value of each node；
Above-mentioned task allocating module 240, for distributing to prediction load value and maximum acceptable load value by above-mentioned task
The minimum node of ratio.
Above-mentioned cluster refers to multiple stage machine and jointly cooperates, and the multicomputer system externally provided uniform services is above-mentioned many
Platform machine can be multiple server, it is also possible to be multiple computer, multiple disk, hard disk etc., each in above-mentioned cluster
Equipment is exactly a node.
In embodiments of the present invention, when receiving of task being distributed to prediction load value and maximum acceptable load value
After the node that ratio is minimum, in order to improve the balanced intensity of cluster, in addition it is also necessary to the task of node in cluster is shifted, is
Realized by the first computing module, the second computing module and task shift module, specifically include:
Above-mentioned first computing module, the prediction load value of each node, distribution of computation tasks after obtaining task distribution
The prediction load value of each node rear and the ratio of maximum acceptable load value；Above-mentioned second computing module, for according to each
After the maximum acceptable load value of node and task distribution, the prediction load value of each node, calculates ratio less than or equal to presetting
The remaining load value of the node of load threshold；Above-mentioned task shift module, for the task of the node that remaining load value is minimum
It is transferred to the node that remaining load value is maximum.
Wherein, above-mentioned first determines the module 220 steady load value according to current load value and previous cycle, determines each
The current steady load value of node, by the first computing unit and determine that unit realizes, specifically includes:
Above-mentioned first computing unit, is used for the stable negative calculating the current load value of each node and node in the previous cycle
First difference of load value；Above-mentioned determine unit, for by the current steady load value of each node and node in the previous cycle
The difference of steady load value is designated as the second difference, according to the proportionate relationship between the first difference and the second difference, determines that each is tied
The current steady load value of point.
Wherein, above-mentioned first determines the module 220 steady load value according to current load value and previous cycle, determines each
The current steady load value of node, it is also possible to realized by the second computing unit, specifically include:
Above-mentioned second determines unit, and for the current load value according to each node, each node is steady the previous cycle
Fixed load value, calculates the current steady load value of each node by formula (1)；
Nm=(nm-Nm-1)/d+Nm-1 (1)
Wherein, in formula (1), NmFor the current steady load value of each node, Nm-1For each node in the previous cycle
Steady load value, nmFor the current load value of each node, d is pre-set ratio, and m is the cycle.
Above-mentioned d is pre-set ratio, and the value of this pre-set ratio is generally 16 or 32, it is, of course, also possible to be other numerical value, on
The value stating pre-set ratio d depends on that in cluster, the load value of each node reaches the load of stable time and each node
The degree of fluctuation of value, in concrete application scenarios, can choose above-mentioned pre-set ratio according to actual needs, and the embodiment of the present invention is also
Do not limit the concrete value of above-mentioned pre-set ratio.
Wherein, above-mentioned second determine module 230 according to load changing value corresponding to current steady load value and above-mentioned task,
Determine the prediction load value of each node, realized by the 3rd computing unit, specifically include:
Above-mentioned Unit the 3rd, for the load change that the current steady load value according to each node and above-mentioned task are corresponding
Value, calculates the prediction load value of each node by formula (2)；
Fm=Dm+Nm (2)
Wherein, in formula (2), FmFor the prediction load value of each node, DmFor the load change that above-mentioned task is corresponding
Value, NmCurrent steady load value for each node.
The cluster load balance device that the embodiment of the present invention provides, including receiver module, first determines module, second determines
Module and task allocating module, it is adaptable to video field of storage, can be prevented effectively from when carrying out task distribution and the effect that flocks together occur
Should, and, it is adaptable to the cluster that passing node equipments is differently configured.
The cluster load balance device that the embodiment of the present invention is provided can be the specific hardware on equipment or be installed on
Software on equipment or firmware etc..The device that the embodiment of the present invention is provided, it realizes principle and the technique effect of generation and front
State embodiment of the method identical, for briefly describing, the not mentioned part of device embodiment part, refer to phase in preceding method embodiment
Answer content.Those skilled in the art is it can be understood that arrive, for convenience and simplicity of description, system described above,
The specific works process of device and unit, is all referred to the corresponding process in said method embodiment, does not repeats them here.
In embodiment provided by the present invention, it should be understood that disclosed apparatus and method, can be by other side
Formula realizes.Device embodiment described above is only that schematically such as, the division of described unit, the most only one are patrolled
Volume function divides, and actual can have other dividing mode when realizing, the most such as, multiple unit or assembly can in conjunction with or can
To be integrated into another system, or some features can be ignored, or does not performs.Another point, shown or discussed each other
Coupling direct-coupling or communication connection can be the INDIRECT COUPLING by some communication interfaces, device or unit or communication link
Connect, can be electrical, machinery or other form.
The described unit illustrated as separating component can be or may not be physically separate, shows as unit
The parts shown can be or may not be physical location, i.e. may be located at a place, or can also be distributed to multiple
On NE.Some or all of unit therein can be selected according to the actual needs to realize the mesh of the present embodiment scheme
's.
It addition, each functional unit in the embodiment that the present invention provides can be integrated in a processing unit, it is possible to
Being that unit is individually physically present, it is also possible to two or more unit are integrated in a unit.
If described function is using the form realization of SFU software functional unit and as independent production marketing or use, permissible
It is stored in a computer read/write memory medium.Based on such understanding, technical scheme is the most in other words
The part contributing prior art or the part of this technical scheme can embody with the form of software product, this meter
Calculation machine software product is stored in a storage medium, including some instructions with so that a computer equipment (can be individual
People's computer, server, or the network equipment etc.) perform all or part of step of method described in each embodiment of the present invention.
And aforesaid storage medium includes: USB flash disk, portable hard drive, read only memory (ROM, Read-Only Memory), random access memory are deposited
The various media that can store program code such as reservoir (RAM, Random Access Memory), magnetic disc or CD.
It should also be noted that similar label and letter represent similar terms, therefore, the most a certain Xiang Yi in following accompanying drawing
Individual accompanying drawing is defined, then need not it be defined further and explains in accompanying drawing subsequently, additionally, term " the
One ", " second ", " the 3rd " etc. are only used for distinguishing and describe, and it is not intended that instruction or hint relative importance.
It is last it is noted that the detailed description of the invention of embodiment described above, the only present invention, in order to the present invention to be described
Technical scheme, be not intended to limit, protection scope of the present invention is not limited thereto, although with reference to previous embodiment to this
Bright it is described in detail, it will be understood by those within the art that: any those familiar with the art
In the technical scope that the invention discloses, the technical scheme described in previous embodiment still can be modified or can be light by it
It is readily conceivable that change, or wherein portion of techniques feature is carried out equivalent；And these are revised, change or replace, do not make
The essence of appropriate technical solution departs from the spirit and scope of embodiment of the present invention technical scheme.All should contain the protection in the present invention
Within the scope of.Therefore, protection scope of the present invention should be as the criterion with described scope of the claims.
Claims (10)
1. a cluster load balancing method, it is characterised in that described method includes:
Receive the task that user sends；
The current load value of each node in cluster in acquisition current period, and each described node is at the stable negative in previous cycle
Load value, according to the steady load value in described current load value and previous cycle, determines the current steady load of each described node
Value；
According to the load changing value that described current steady load value and described task are corresponding, determine that the prediction of each described node is born
Load value；
Described task is distributed to the described node that described prediction load value is minimum with the ratio of maximum acceptable load value.
Method the most according to claim 1, it is characterised in that described described task is distributed to described prediction load value with
After the described node that the ratio of maximum acceptable load value is minimum, also include:
The prediction load value of each described node after the distribution of acquisition task, after distribution of computation tasks, the prediction of each described node is born
Load value and the ratio of described maximum acceptable load value；
According to the prediction load value of each described node, meter after the maximum acceptable load value of node each described and task distribution
Calculate the described ratio remaining load value less than or equal to the described node presetting load threshold；
The task of described node minimum for described remaining load value is transferred to the described node that described remaining load value is maximum.
Method the most according to claim 1, it is characterised in that described steady according to described current load value and previous cycle
Fixed load value, determines the current steady load value of each described node, including:
Calculate the current load value of each described node and described node the first difference in the steady load value in previous cycle；
Current steady load value and the described node of each described node are designated as in the difference of the steady load value in previous cycle
Second difference, according to the proportionate relationship between described first difference and described second difference, determines the current of each described node
Steady load value.
Method the most according to claim 1, it is characterised in that described steady according to described current load value and previous cycle
Fixed load value, determines the current steady load value of each described node, including:
According to the current load value of node each described, each described node, in the steady load value in previous cycle, passes through formula
(1) the current steady load value of each described node is calculated；
Nm=(nm-Nm-1)/d+Nm-1 (1)
Wherein, in formula (1), NmFor the current steady load value of node arbitrary in described cluster, Nm-1For described arbitrary node
At the steady load value in previous cycle, nmFor the current load value of described arbitrary node, d is pre-set ratio, and m is the cycle.
Method the most according to claim 1, it is characterised in that described according to described current steady load value and described task
Corresponding load changing value, determines the prediction load value of each described node, including:
According to the load changing value that the current steady load value of node each described and described task are corresponding, counted by formula (2)
Calculate the prediction load value of each described node；
Fm=Dm+Nm (2)
Wherein, in formula (2), FmFor the prediction load value of node arbitrary in described cluster, DmFor the load that described task is corresponding
Changing value, NmFor the current steady load value of described arbitrary node, m is the cycle.
6. a cluster load balance device, it is characterised in that described device includes:
Receiver module, for receiving the task that user sends；
First determines module, for obtaining in current period the current load value of each node in cluster, and each described node
In the steady load value in previous cycle, according to the steady load value in described current load value and previous cycle, determine described in each
The current steady load value of node；
Second determines module, for the load changing value corresponding according to described current steady load value and described task, determines each
The prediction load value of individual described node；
Task allocating module, for distributing to the ratio of described prediction load value and maximum acceptable load value by described task
Little described node.
Device the most according to claim 6, it is characterised in that described device also includes:
First computing module, the prediction load value of each described node after obtaining task distribution, each after distribution of computation tasks
The prediction load value of individual described node and the ratio of described maximum acceptable load value；
Second computing module, for according to each described knot after the maximum acceptable load value of node each described and task distribution
The prediction load value of point, calculates the described ratio remaining load value less than or equal to the described node presetting load threshold；
Task shift module, the task for the described node that described remaining load value is minimum is transferred to described remaining load value
Maximum described node.
Device the most according to claim 6, it is characterised in that described first determines that module includes:
First computing unit, is used for the stable negative calculating the current load value of each described node and described node in the previous cycle
First difference of load value；
Determine unit, for by the current steady load value of each described node and described node in the steady load in previous cycle
The difference of value is designated as the second difference, according to the proportionate relationship between described first difference and described second difference, determines each institute
State the current steady load value of node.
Device the most according to claim 6, it is characterised in that described first determines that module also includes:
Second computing unit, for according to the current load value of node each described, each described node is steady the previous cycle
Fixed load value, calculates the current steady load value of each described node by formula (1)；
Nm=(nm-Nm-1)/d+Nm-1 (1)
Wherein, in formula (1), NmFor the current steady load value of node arbitrary in described cluster, Nm-1For described arbitrary node
At the steady load value in previous cycle, nmFor the current load value of described arbitrary node, d is pre-set ratio, and m is the cycle.
Device the most according to claim 6, it is characterised in that described second determines that module includes:
3rd computing unit, for the load change corresponding according to the current steady load value of node each described and described task
Value, calculates the prediction load value of each described node by formula (2)；
Fm=Dm+Nm (2)
Wherein, in formula (2), FmFor the prediction load value of node arbitrary in described cluster, DmFor the load that described task is corresponding
Changing value, NmFor the current steady load value of described arbitrary node, m is the cycle.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610654053.4A CN106101276B (en) | 2016-08-10 | 2016-08-10 | A kind of cluster load balancing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610654053.4A CN106101276B (en) | 2016-08-10 | 2016-08-10 | A kind of cluster load balancing method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106101276A true CN106101276A (en) | 2016-11-09 |
| CN106101276B CN106101276B (en) | 2019-07-09 |
Family
ID=57455778
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610654053.4A Active CN106101276B (en) | 2016-08-10 | 2016-08-10 | A kind of cluster load balancing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106101276B (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108696594A (en) * | 2018-05-27 | 2018-10-23 | 佛山市虚拟现实大数据产业研究院有限公司 | A kind of the big data traffic load equalization methods and device of market surpervision block chain |
| CN108712422A (en) * | 2018-05-18 | 2018-10-26 | 网宿科技股份有限公司 | A kind of method and apparatus creating transcoding task |
| CN110602545A (en) * | 2019-09-26 | 2019-12-20 | 杭州米络星科技(集团)有限公司 | Distributed recording execution method for network live broadcast |
| CN111683132A (en) * | 2020-06-04 | 2020-09-18 | 重庆英楼维兴信息科技有限公司 | Business distribution method based on micro-service architecture and related device |
| CN112650582A (en) * | 2020-12-21 | 2021-04-13 | 贝壳技术有限公司 | Distributed task processing method and system and processor |
| CN113342665A (en) * | 2021-06-17 | 2021-09-03 | 北京百度网讯科技有限公司 | Task allocation method and device, electronic equipment and computer readable medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102232282A (en) * | 2010-10-29 | 2011-11-02 | 华为技术有限公司 | Method and apparatus for realizing load balance of resources in data center |
| CN103036979A (en) * | 2012-12-12 | 2013-04-10 | 广州尚融网络科技有限公司 | Server loading balancing method and loading balancer |
| WO2015175671A1 (en) * | 2014-05-13 | 2015-11-19 | Nutanix, Inc. | Mechanism for providing load balancing to an external node utilizing a clustered environment for storage management |
| CN105516369A (en) * | 2016-02-04 | 2016-04-20 | 城云科技（杭州）有限公司 | Video cloud platform load balancing method and video cloud platform load balancing dispatcher |
-
2016
- 2016-08-10 CN CN201610654053.4A patent/CN106101276B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102232282A (en) * | 2010-10-29 | 2011-11-02 | 华为技术有限公司 | Method and apparatus for realizing load balance of resources in data center |
| CN103036979A (en) * | 2012-12-12 | 2013-04-10 | 广州尚融网络科技有限公司 | Server loading balancing method and loading balancer |
| WO2015175671A1 (en) * | 2014-05-13 | 2015-11-19 | Nutanix, Inc. | Mechanism for providing load balancing to an external node utilizing a clustered environment for storage management |
| CN105516369A (en) * | 2016-02-04 | 2016-04-20 | 城云科技（杭州）有限公司 | Video cloud platform load balancing method and video cloud platform load balancing dispatcher |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108712422A (en) * | 2018-05-18 | 2018-10-26 | 网宿科技股份有限公司 | A kind of method and apparatus creating transcoding task |
| CN108696594A (en) * | 2018-05-27 | 2018-10-23 | 佛山市虚拟现实大数据产业研究院有限公司 | A kind of the big data traffic load equalization methods and device of market surpervision block chain |
| CN110602545A (en) * | 2019-09-26 | 2019-12-20 | 杭州米络星科技(集团)有限公司 | Distributed recording execution method for network live broadcast |
| CN111683132A (en) * | 2020-06-04 | 2020-09-18 | 重庆英楼维兴信息科技有限公司 | Business distribution method based on micro-service architecture and related device |
| CN111683132B (en) * | 2020-06-04 | 2022-07-19 | 重庆金窝窝网络科技有限公司 | Business distribution method based on micro-service architecture and related device |
| CN112650582A (en) * | 2020-12-21 | 2021-04-13 | 贝壳技术有限公司 | Distributed task processing method and system and processor |
| CN113342665A (en) * | 2021-06-17 | 2021-09-03 | 北京百度网讯科技有限公司 | Task allocation method and device, electronic equipment and computer readable medium |
| CN113342665B (en) * | 2021-06-17 | 2023-10-20 | 北京百度网讯科技有限公司 | Task allocation method and device, electronic equipment and computer readable medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106101276B (en) | 2019-07-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106101276A (en) | A kind of cluster load balancing method and device | |
| JP5654022B2 (en) | Dynamic load balancing and scaling of allocated cloud resources within the corporate network | |
| CN106233276B (en) | The coordination admission control of network-accessible block storage device | |
| CN103401947A (en) | Method and device for allocating tasks to multiple servers | |
| CN104038540B (en) | Method and system for automatically selecting application proxy server | |
| CN109218355A (en) | Load equalizing engine, client, distributed computing system and load-balancing method | |
| US20130097321A1 (en) | Method and system for work load balancing | |
| US10027760B2 (en) | Methods, systems, and computer readable media for short and long term policy and charging rules function (PCRF) load balancing | |
| CN105656973A (en) | Distributed method and system for scheduling tasks in node group | |
| US20160019084A1 (en) | Method and system for inter-cloud virtual machines assignment | |
| CN108965014A (en) | The service chaining backup method and system of QoS perception | |
| CN102970379A (en) | Method for realizing load balance among multiple servers | |
| CN112261120B (en) | Cloud-side cooperative task unloading method and device for power distribution internet of things | |
| CN106020977B (en) | Distributed task dispatching method and device for monitoring system | |
| CN113348651A (en) | Dynamic inter-cloud placement of sliced virtual network functions | |
| CN108063784A (en) | The methods, devices and systems of application cluster resource allocation under a kind of cloud environment | |
| CN103441918A (en) | Self-organizing cluster server system and self-organizing method thereof | |
| CN106059940B (en) | A kind of flow control methods and device | |
| Gkatzikis et al. | Mobiles on cloud nine: Efficient task migration policies for cloud computing systems | |
| CN107220108A (en) | A kind of method and system for realizing cloud data center load balancing | |
| CN116909735A (en) | Calculation power scheduling method and device, server and storage medium | |
| CN105915630A (en) | Cross-network data transmission system and load balancing scheduling method | |
| KR101448413B1 (en) | Method and apparatus for scheduling communication traffic in atca-based equipment | |
| CN105335376B (en) | A kind of method for stream processing, apparatus and system | |
| CN108200185A (en) | A kind of method and device for realizing load balancing |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| PP01 | Preservation of patent right | ||
| PP01 | Preservation of patent right |
Effective date of registration: 20220726 Granted publication date: 20190709 |